Steps to integrate Ollama with Wyn as an LLM

Background: 

This guide explains how to install Ollama on Windows, download and run a local LLM model, and configure Wyn to connect to Ollama for:

• LLM (chat/generation)
• Text Embedding (text-to-vector conversion scenarios)

Prerequisites 

• Windows machine with permission to install applications
• Wyn Admin access
• Ollama installed and running locally
• Internet connection to download models (if not already downloaded)

1. Download and install Ollama (Windows)

• Download Ollama from the official website: Download Ollama on Windows
  • Download as Windows installer:
    • https://ollama.com/download/OllamaSetup.exe
  • Download via CMD/PowerShell:
    • irm https://ollama.com/install.ps1 | iex

2. Download a local LLM model in Ollama

• Open the installed Ollama application.
• Choose any available LLM (Cloud-based / local models).
• In this example, we use the local model: gpt-oss:20b
• Let the model download in the background.
• Note: Downloading time is affected by model size, machine capabilities, and network speed.

3. Run the installed LLM (enable it) and verify models

• Open CMD as Administrator.
• Run the installed LLM (example):
  • ollama run gpt-oss:20b
• Verify installed models:
  • ollama list

4. Verify Ollama is reachable from a web browser

• Open a browser and verify using these links:
  • http://localhost:11434
  • http://localhost:11434/api/chat?model=gpt-oss:20b

5. Create an Ollama API key (Optional)

• On the Ollama website, go to: Ollama keys · Settings
• Create an API key.
• This key can be used later in Wyn’s LLM setup page.
• Note: API key is not mandatory in localhost environments.

6. Configure Wyn LLM Settings (connect Wyn to Ollama)

• In Wyn, go to:
  • Admin Portal → Configuration → AI settings → LLM Settings
• Fill in the settings:
  • Model name:
    • Your own model name (example: gpt-oss:20b)
  • API Key:
    • Localhost: not mandatory
    • Cloud/non-localhost: required (use the key generated on the Ollama website)
  • Endpoint:
    • http://localhost:11434/v1
    • Important: add “/v1” after the base URL to make it work
• Click Test:
  • You should see: LLM connect success
• Click Save Model

7. Update Ollama context size (recommended)

• In Ollama application settings:
  • Change the context size to 32k
• Reason:
  • So the model can get a larger/complete message context.

8. Text Embedding Model Setting (required)

• Text Embedding requires another model eligible for text data conversion.
• In this example, we use:
  • qwen3-embedding:8b

9. Pull/download the embedding model via CMD

• From CMD run:ollama pull qwen3-embedding:8b

10. Confirm the embedding model is installed

• Run:
  • ollama list
• Confirm qwen3-embedding:8b has been added.

11. Configure Wyn Text Embedding Model Settings

• In Wyn, go to:
  • Admin Portal → Configuration → AI settings → Text Embedding Model Settings
• Fill in:
  • Model name:
    • Your choice (example: qwen3-embedding:8b)
  • API Key:
    • Any key as long as you are on localhost
  • Endpoint:
    • http://localhost:11434/v1

12. Test and save the embedding model connection

• Click Test:
  • You should receive a successful connection message
• Save changes.

Troubleshooting (common checks) 

• Wyn “Test” fails (LLM or Embedding)
  • Confirm Ollama is running:
    • http://localhost:11434
  • Confirm Wyn endpoint includes /v1:
    • http://localhost:11434/v1
  • Confirm the model name matches exactly what Ollama shows:
    • ollama list
• Model not found
  • Confirm installed models:
    • ollama list
  • If missing, pull the model:
    • ollama pull <model-name>

Notes

• Large models may require significant RAM/VRAM depending on model size.
• For non-localhost deployments, use proper network security controls and API keys where applicable.