> ## Documentation Index
> Fetch the complete documentation index at: https://docs.okchat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Using with Ollama Models

> This documentation guides you through setting up OKCHAT.AI with Ollama models using Ngrok for accessibility from anywhere. The integration allows you to leverage local LLM models through Ollama while providing a chat interface through OKCHAT.AI.

## Overview

This documentation guides you through setting up OKCHAT.AI with Ollama models using Ngrok for accessibility from anywhere. The integration allows you to leverage local LLM models through Ollama while providing a chat interface through OKCHAT.AI.

## Prerequisites

* [Ollama](https://ollama.ai/) installed on your machine
* [Ngrok](https://ngrok.com/) account and client installed
* Basic knowledge of terminal/command line

## Step 1: Install and Configure Ollama

1. Install Ollama following the instructions at [ollama.ai](https://ollama.ai/)
2. Pull your desired model:
   ```bash theme={null}
   ollama pull deepseek-r1:7b
   ```
3. Verify the model is downloaded:
   ```bash theme={null}
   ollama list
   ```

## Step 2: Configure Ollama to Accept External Connections

By default, Ollama only listens to localhost connections. To make it accessible externally:

```bash theme={null}
OLLAMA_HOST=0.0.0.0:11434 ollama serve
```

This command makes Ollama listen on all network interfaces instead of just localhost.

## Step 3: Set Up Ngrok

1. Sign up for an Ngrok account at [ngrok.com](https://ngrok.com/)

2. Install Ngrok:

   ```bash theme={null}
   # macOS with Homebrew
   brew install ngrok

   # Linux with snap
   snap install ngrok

   # Or download from ngrok.com/download
   ```

3. Authenticate Ngrok with your auth token:
   ```bash theme={null}
   ngrok config add-authtoken YOUR_AUTH_TOKEN
   ```

4. Create a tunnel to your Ollama instance:
   ```bash theme={null}
   ngrok http 11434
   ```

5. Note the forwarding URL generated by Ngrok (e.g., `https://61ff-128-199-189-88.ngrok-free.app`)

## Step 4: Configure OKCHAT.AI

1. Open the OKCHAT.AI demo interface
2. Configure the following settings in the interface:
   * **Model Type**: Select "Local Model"
   * **Ollama API URL**: Enter your Ngrok forwarding URL (e.g., `https://61ff-128-199-189-88.ngrok-free.app`)
   * **Model Name**: Enter the name of your model (e.g., `deepseek-r1:7b`)
   * **Temperature**: Set to desired value (e.g., `0.7` for balanced creativity)
   * **Max Tokens**: Set the maximum response length (e.g., `2500`)

## Step 5: Test Your Setup

1. Type a message in the chat interface
2. Send the message to confirm the connection is working
3. You should receive a response from your local Ollama model through the Ngrok tunnel

## Troubleshooting

### 403 Forbidden Errors

If you receive 403 errors in your Ngrok console:

1. Ensure Ollama is running with `OLLAMA_HOST=0.0.0.0:11434`
2. Try adding authentication to your Ngrok tunnel:
   ```bash theme={null}
   ngrok http 11434 --basic-auth="username:password"
   ```
3. Update the Ollama API URL in OKCHAT.AI to include these credentials

### No Response

If OKCHAT.AI doesn't respond:

1. Check if your model is downloaded and running in Ollama
2. Verify the Ngrok tunnel is active
3. Confirm the correct URL is entered in the OKCHAT.AI interface
4. Ensure the model name exactly matches what's in your Ollama instance

### Connection Limitations

Note that the free Ngrok plan has limitations on connections and bandwidth. For production use, consider upgrading to a paid plan.

## Important Notes

* The OKCHAT.AI interface currently notes that "local LLM doesn't support tool calling functionality"
* Make sure your Ollama instance is running before starting the Ngrok tunnel
* For security reasons, consider using authentication with your Ngrok tunnel in production environments
* The model must be downloaded in your Ollama instance before it can be used

By following this guide, you'll have successfully set up OKCHAT.AI to work with your local Ollama models, accessible from anywhere via Ngrok.