Documentation
Models
Model Management

Model Management

This guide provides comprehensive instructions on adding, customizing, and deleting models within Jan.

Local Model

Jan offers flexible options for managing local models through its Cortex (opens in a new tab) engine. Currently, Jan only supports GGUF format models.

Local models run directly on your computer, which means they use your computer's memory (RAM) and processing power. Please choose models carefully based on your hardware specifications for Mac, Windows, & Linux.

Add Models

1. Download from Jan Hub (Recommended)

The easiest way to get started is using Jan's built-in model hub:

  1. Go to the Hub
  2. Browse available models and click on any model to see details about it
  3. Choose a model that fits your needs & hardware specifications
  4. Click Download on your chosen model

Jan will indicate if a model might be Slow on your device or requires Not enough RAM based on your system specifications.


Download Model


2. Import from Hugging Face (opens in a new tab)

You can import GGUF models directly from Hugging Face (opens in a new tab):

Option A: Import in Jan
  1. Visit Hugging Face Models (opens in a new tab).
  2. Find a GGUF model you want to use
  3. Copy the model ID (e.g., TheBloke/Mistral-7B-v0.1-GGUF) or its URL
  4. In Jan, paste the model ID/URL to Search bar in Hub or in Settings > My Models
  5. Select your preferred quantized version to download

Download Model


Option B: Use Deep Link

You can use Jan's deep link feature to quickly import models:

  1. Visit Hugging Face Models (opens in a new tab).
  2. Find the GGUF model you want to use
  3. Copy the model ID, for example: TheBloke/Mistral-7B-v0.1-GGUF
  4. Create a deep link URL in this format:

jan://models/huggingface/<model_id>

  1. Enter the URL in your browser & Enter, for example:

jan://models/huggingface/TheBloke/Mistral-7B-v0.1-GGUF

  1. A prompt will appear: This site is trying to open Jan, click Open to open Jan app.
  2. Select your preferred quantized version to download
⚠️

Deep linking won't work for models requiring API tokens or usage agreements. You'll need to download these models manually through the Hugging Face website.


Downlosd Model


3. Import Local Files

If you already have GGUF model files on your computer:

  1. In Jan, go to Hub or Settings > My Models
  2. Click Import Model
  3. Select your GGUF file
  4. Choose how you want to import:
  • Link Files: Creates symbolic links to your model files (saves space)
  • Duplicate: Makes a copy of model files in Jan's directory
  1. Click Import to complete
⚠️

You need to own your model configurations, use at your own risk. Misconfigurations may result in lower quality or unexpected outputs.


Download Model


4. Manual Setup

For advanced users who add a specific model that is not available within Jan Hub:

  1. Navigate to ~/jan/data/models/
  2. Create a new Folder for your model
  3. Add a model.json file with your configuration:

"id": "<unique_identifier_of_the_model>",
"object": "<type_of_object, e.g., model, tool>",
"name": "<name_of_the_model>",
"version": "<version_number>",
"description": "<brief_description_of_the_model>",
"format": "<format_of_the_model_api_or_other>",
"settings": "<additional_settings_as_needed>",
"parameters": {
"max_tokens": "<maximum_number_of_tokens_the_model_can_generate>",
"temperature": "<temperature_setting_for_randomness_in_generation>"
},
"metadata": {
"author": "<name_of_the_creator_or_organization>",
"tags": ["<list_of_relevant_tags_describing_the_model>"]
},
"engine": "<engine_or_platform_the_model_runs_on>",
"source": "<url_or_source_of_the_model_information>"

Key fields to configure:

  1. Settings is where you can set your engine configurations.
  2. Parameters are the adjustable settings that affect how your model operates or processes the data. The fields in parameters are typically general and can be the same across models. Here is an example of model parameters:

"parameters":{
"temperature": 0.7,
"top_p": 0.95,
"stream": true,
"max_tokens": 4096,
"frequency_penalty": 0,
"presence_penalty": 0

Delete Models

  1. Go to Settings > My Models
  2. Find the model you want to remove
  3. Select the three dots next to it and select Delete Model

Delete Model


Cloud model

When using cloud models, be aware of any associated costs and rate limits from the providers.

Jan supports connecting to various AI cloud providers that are OpenAI API-compatible, including: OpenAI (GPT-4, o1,...), Anthropic (Claude), Groq, Mistral, and more.

  1. Open Settings
  2. Under Model Provider section in left sidebar (OpenAI, Anthropic, etc.), choose a provider
  3. Enter your API key
  4. The activated cloud models will be available in your model selector in Threads

Download Model


Cloud models cannot be deleted, but you can hide them by disabling their respective provider engines in Settings > Engines.


Download Model


ModelsModel Parameters