This code will enable you to use an LLM on your local system. You must atleast have 6GB of V-RAM for a smooth performance. Users with a lesser RAM of GPU or without one might get a laggy performace.
https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/
LM-Studio quantized models: https://lmstudio.ai/models/gemma-4
Steps to run code:
- Install all the libraries
- Run the file
gemma-main.py - Use the file
test_reequest.pyto send requests and intitiate chat