​
Overview

This documentation guides you through setting up OKCHAT.AI with Ollama models using Ngrok for accessibility from anywhere. The integration allows you to leverage local LLM models through Ollama while providing a chat interface through OKCHAT.AI.

​
Prerequisites

  • Ollama installed on your machine
  • Ngrok account and client installed
  • Basic knowledge of terminal/command line

​
Step 1: Install and Configure Ollama

  1. Install Ollama following the instructions at ollama.ai
  2. Pull your desired model: 
    ollama pull deepseek-r1:7b
  3. Verify the model is downloaded: 
    ollama list

​
Step 2: Configure Ollama to Accept External Connections

By default, Ollama only listens to localhost connections. To make it accessible externally:

OLLAMA_HOST=0.0.0.0:11434 ollama serve

This command makes Ollama listen on all network interfaces instead of just localhost.

​
Step 3: Set Up Ngrok

  1. Sign up for an Ngrok account at ngrok.com

  2. Install Ngrok:

    # macOS with Homebrew
brew install ngrok

# Linux with snap
snap install ngrok

# Or download from ngrok.com/download

  3. Authenticate Ngrok with your auth token:

    ngrok config add-authtoken YOUR_AUTH_TOKEN

  4. Create a tunnel to your Ollama instance:

    ngrok http 11434

  5. Note the forwarding URL generated by Ngrok (e.g., https://61ff-128-199-189-88.ngrok-free.app)

​
Step 4: Configure OKCHAT.AI

  1. Open the OKCHAT.AI demo interface
  2. Configure the following settings in the interface:
    • Model Type: Select “Local Model”
    • Ollama API URL: Enter your Ngrok forwarding URL (e.g., https://61ff-128-199-189-88.ngrok-free.app)
    • Model Name: Enter the name of your model (e.g., deepseek-r1:7b)
    • Temperature: Set to desired value (e.g., 0.7 for balanced creativity)
    • Max Tokens: Set the maximum response length (e.g., 2500)

​
Step 5: Test Your Setup

  1. Type a message in the chat interface
  2. Send the message to confirm the connection is working
  3. You should receive a response from your local Ollama model through the Ngrok tunnel

​
Troubleshooting

​
403 Forbidden Errors

If you receive 403 errors in your Ngrok console:

  1. Ensure Ollama is running with OLLAMA_HOST=0.0.0.0:11434
  2. Try adding authentication to your Ngrok tunnel: 
    ngrok http 11434 --basic-auth="username:password"
  3. Update the Ollama API URL in OKCHAT.AI to include these credentials

​
No Response

If OKCHAT.AI doesn’t respond:

  1. Check if your model is downloaded and running in Ollama
  2. Verify the Ngrok tunnel is active
  3. Confirm the correct URL is entered in the OKCHAT.AI interface
  4. Ensure the model name exactly matches what’s in your Ollama instance

​
Connection Limitations

Note that the free Ngrok plan has limitations on connections and bandwidth. For production use, consider upgrading to a paid plan.

​
Important Notes

  • The OKCHAT.AI interface currently notes that “local LLM doesn’t support tool calling functionality”
  • Make sure your Ollama instance is running before starting the Ngrok tunnel
  • For security reasons, consider using authentication with your Ngrok tunnel in production environments
  • The model must be downloaded in your Ollama instance before it can be used

By following this guide, you’ll have successfully set up OKCHAT.AI to work with your local Ollama models, accessible from anywhere via Ngrok.