This is now the age of Generative AI. And the age of using AI any-where and every-where. But what if you want to use AI in your VSCode environment, but without an internet connection? As well as Unlimited API Calls? Well, this article will show you how.

Step 0: Pre-requisites

Before we start, make sure you have the following:

  • VSCode installed on your computer

  • Moderately powerful PC / Laptop

Step 1: Install Ollama

Install Ollama installed on your computer. You can download it from here.

Once Ollama is installed, you can check whether it’s installed or not running the following command in your terminal:

1ollama --version

You should see the version of Ollama installed on your computer. For example:

1ollama version is 0.5.4

Step 2: Download few models to Ollama

To install the models with Ollama, Start a command prompt and run:

1ollama pull gemma2:2b
2ollama pull qwen2.5-coder:1.5b

Depending on what you want to do, you can search and install other models as well to your computer. You can search for available models from Ollama Models Search page

Step 3: Install Continue Plugin

Inside VS Code, install the Continue plugin.

Step 4: Configure Continue

The behavior of Continue engine is controlled from ~/.continue/config.json file.

  • Click the Continue logo on the Activity Bar.

    Continue logo

  • Click on Settings button on the Continue.

    Settings

  • This should open the config.json file which you can edit/extend.

    Extend the config file to select your model for tab completion.

  • Update the .config.json with the models that you want to use.

    A sample config file could look like this.

     1{
 2    "models": [
 3        {
 4          "title": "Gemma 2 2b IT",
 5          "model": "gemma2:2b",
 6          "provider": "ollama"
 7        }
 8    ],
 9    "tabAutocompleteModel": {
10        "title": "Qwen2.5-Coder 1.5B",
11        "model": "qwen2.5-coder:1.5b",
12        "provider": "ollama"
13    },
14    ...
15}

    Note: ... is used in above example to shorten this blog post. It’s not part of the actual file.

Step 5: See it in action

  • Create a new file or open an existing one that you want to work on.

  • The Continue plugin will automatically trigger the LLM, providing seamless integration with local computing power.

  • In VS Code, open OUTPUT –> Continue - LLM Prompt/Completion.

    You will see what input is being passed to the prompt and the output it is generating.

  • Disconnect from the internet, and see that the action still continues.

Summary

Using Offline AI within VS Code opens up a whole new dimension of possibilities. It allows you to harness the power of AI locally without requiring an internet connection for tasks such as text generation and analysis. This is especially valuable when:

  • You’re working offline for privacy or reliability reasons.

  • You want to try unlimited access to AI without being bound by free usage restrictions.

Note: Depending of RAM and GPU you have access to, there may be some sluggish-ness in response of the tool and you may have to choose different models (with lower computational resources) or configurations. A lower resource hungry model would by definition can not outperform the other larger models.

Happy coding! ЁЯЪА

Next Steps

Experiment with different models, configurations, and prompts to see what works best for your specific use case. Happy experimenting! ЁЯЪА

Disclaimer

AI helped me write this page…