The AI Assistant Settings tab configures how RTILA X’s AI features work — which AI model to use, how it processes your requests, and how it manages memory. These settings affect the AI Assistant chat, AI Vision Analysis, and AI Self-Correction.
To open AI Settings:
- Click your profile name in the bottom-left corner of the sidebar.
- Click Preferences.
- Click the AI Assistant tab.
| Setting |
Location |
Type |
Default |
Description |
| AI Provider |
AI Assistant → AI Provider |
Dropdown |
OpenRouter |
Which AI service to use: OpenRouter or Local LLM |
| API Key |
AI Assistant → API Key |
Text |
Empty |
Your OpenRouter API key (OpenRouter only) |
| Model |
AI Assistant → Model |
Dropdown/Text |
gemini-3.5-flash-lite |
The AI model to use |
| Verify |
AI Assistant → Verify |
Button |
— |
Validates a custom model name against OpenRouter |
| LLM Mode |
AI Assistant → Mode |
Toggle |
GGUF Model |
Local LLM mode: GGUF Model (HuggingFace) or Custom API Endpoint |
| Model Identifier |
AI Assistant → Model Identifier |
Text |
Empty |
HuggingFace GGUF model identifier (GGUF mode only) |
| API Base URL |
AI Assistant → API Base URL |
Text |
Empty |
OpenAI-compatible API endpoint (Custom API mode only) |
| Context Size |
AI Assistant → Server Configuration → Context Size |
Dropdown |
32K |
How much context the AI can process per conversation |
| GPU Layers |
AI Assistant → Server Configuration → GPU Layers |
Number |
99 |
Number of model layers to offload to GPU |
| CPU Threads |
AI Assistant → Server Configuration → CPU Threads |
Number |
4 |
Number of CPU threads for inference |
| Batch Size |
AI Assistant → Server Configuration → Batch Size |
Number |
512 |
Tokens processed per batch |
| Additional Flags |
AI Assistant → Server Configuration → Additional Flags |
Text |
-fa on |
Advanced command-line flags for llama.cpp |
| KV Cache Type K |
AI Assistant → KV Cache → KV Cache Type K |
Dropdown |
F16 |
Quantization for the Key cache |
| KV Cache Type V |
AI Assistant → KV Cache → KV Cache Type V |
Dropdown |
F16 |
Quantization for the Value cache |
| Flash Attention |
AI Assistant → KV Cache → Flash Attention |
Toggle |
Off |
Enables flash attention kernels |
| Parallel Slots |
AI Assistant → KV Cache → Parallel Slots |
Number |
1 |
Concurrent conversation slots |
| No MMap |
AI Assistant → KV Cache → No MMap |
Toggle |
Off |
Disables memory-mapped file loading |
| MLock |
AI Assistant → KV Cache → MLock |
Toggle |
Off |
Locks model weights in RAM |
| Provider |
Description |
Best For |
| OpenRouter |
Access hundreds of cloud AI models through a single API key |
Best quality, fastest responses, no local hardware needed |
| Local LLM |
Run any GGUF model locally via llama.cpp, or connect to any OpenAI-compatible API |
Maximum privacy, offline use, no ongoing costs |
| Setting |
Description |
| API Key |
Your OpenRouter API key. Get one at openrouter.ai/keys |
| Model |
Select from the dropdown or enter a custom model name |
| Verify |
Click to validate a custom model name against OpenRouter’s model list |
Available models in the dropdown:
| Model |
Description |
minimax/minimax-m3 |
MiniMax’s latest model |
deepseek/deepseek-v4-flash |
DeepSeek’s fast variant |
z-ai/glm-5.3-flash |
Zhipu AI’s flash model |
google/gemini-3.5-flash-lite |
Google’s lightweight model (default) |
google/gemini-3.1-flash-lite |
Previous Gemini flash |
openai/gpt-5.6-luna |
OpenAI’s latest |
xiaomi/mimo-v2.5 |
Xiaomi’s model |
google/gemini-2.5-flash-lite |
Older Gemini flash |
google/gemini-2.5-flash |
Gemini flash (full) |
Tip: You can enter any OpenRouter model identifier in the model field, even if it’s not in the dropdown. Use the Verify button to check if a custom model name is valid.
The Local LLM provider has two modes:
| Setting |
Description |
| Model Identifier |
HuggingFace model identifier in the format user/repo/file.gguf |
Example identifiers:
bartowski/Qwen2.5-7B-Instruct-GGUF/Qwen2.5-7B-Instruct-Q4_K_M.ggufbartowski/Meta-Llama-3.1-8B-Instruct-GGUF/Meta-Llama-3.1-8B-Instruct-Q4_K_M.ggufbartowski/Mistral-7B-Instruct-v0.3-GGUF/Mistral-7B-Instruct-v0.3-Q4_K_M.gguf
Tip: Browse all available GGUF models at huggingface.co/models?library=gguf.
| Setting |
Description |
| API Base URL |
The base URL of your OpenAI-compatible API server |
| API Key |
Optional API key (most local servers don’t require one) |
| Model Name |
Optional model name (auto-detected if empty) |
Supported servers:
| Server |
Default Base URL |
| Ollama |
http://localhost:11434/v1 |
| LM Studio |
http://localhost:1234/v1 |
| vLLM |
http://localhost:8000/v1 |
| text-generation-webui |
http://localhost:5000/v1 |
| llama-server (bundled) |
http://localhost:8080/v1 |
These settings control how the bundled llama.cpp server runs when using GGUF models.
| Option |
Value |
Best For |
| 4K |
4,096 tokens |
Very low-RAM machines (4 GB) |
| 8K |
8,192 tokens |
Basic tasks on limited hardware |
| 16K |
16,384 tokens |
Standard use on 8 GB RAM |
| 32K (Default) |
32,768 tokens |
Recommended for most users |
| 64K |
65,536 tokens |
Complex multi-page tasks |
| 128K |
131,072 tokens |
Very complex tasks, 16+ GB RAM |
| 256K |
262,144 tokens |
Large context needs, 32+ GB RAM |
| 512K |
524,288 tokens |
Extreme context, 64+ GB RAM |
| 1M |
1,048,576 tokens |
Maximum context, 128+ GB RAM |
Changed in v8.3.2: The default context size increased from 8K to 32K. This is required to fit the full system prompt and complete command reference that RTILA X injects into every conversation.
| Setting |
Default |
Description |
| GPU Layers |
99 |
Number of model layers to offload to your GPU. Set to 99 to offload all layers. Lower this if you experience out-of-memory errors. Set to 0 for CPU-only inference. |
| Setting |
Default |
Description |
| CPU Threads |
4 |
Number of CPU threads used for inference. Higher values can speed up generation but may slow down other applications. |
| Setting |
Default |
Description |
| Batch Size |
512 |
Number of tokens processed per batch during prompt processing. Higher values speed up initial prompt processing but use more memory. |
| Setting |
Default |
Description |
| Additional Flags |
-fa on |
Advanced command-line flags passed directly to the llama.cpp server. Use this for expert-level tuning. Separate multiple flags with spaces. |
The KV (Key-Value) cache stores the attention state of previously processed tokens. By tuning the cache, you can significantly reduce memory usage and improve response speed.
| Option |
Memory Usage |
Quality Loss |
Best For |
| F32 |
Highest (4x F16) |
None |
Maximum precision (debugging only) |
| F16 (Default) |
High (baseline) |
None |
Best quality for most users |
| Q8_0 |
~50% of F16 |
Negligible |
Recommended for most users — halves memory with virtually no quality impact |
| Q4_0 |
~25% of F16 |
Slight |
Very low-RAM machines |
Important: Quantized KV cache types (Q8_0, Q4_0) require Flash Attention to be enabled on GPU backends. If you set KV Cache Type to Q8_0 or Q4_0, also enable Flash Attention.
| Setting |
Default |
Description |
| Flash Attention |
Off |
Enables flash attention kernels for reduced memory usage during generation. Required for quantized KV cache on GPU (NVIDIA Ampere+, Apple M1+, AMD RDNA2+). |
| Setting |
Default |
Description |
| Parallel Slots |
1 |
Number of concurrent conversation slots sharing the same loaded model. Each slot maintains its own KV cache segment. More slots use more memory. |
| Setting |
Default |
Description |
| No MMap |
Off |
Disables memory-mapped file loading. Uses more RAM but avoids disk I/O stalls. Useful on systems with slow disks. |
| Setting |
Default |
Description |
| MLock |
Off |
Locks model weights in RAM to prevent the operating system from swapping them to disk. Useful on systems with limited RAM. |
These tips are displayed in the AI Assistant settings panel:
- KV Cache: The system prompt and reference document are loaded once per conversation slot and cached by llama-server. Subsequent messages reuse this cache, dramatically reducing response time.
- Q8_0 KV cache halves memory usage with negligible quality impact. Recommended for systems with less than 16 GB RAM.
- Flash Attention reduces peak memory during generation. Enable it if your GPU supports it. It is required when using quantized KV cache types.
- Parallel slots allow multiple chat tabs to share the same loaded model without reloading.
- A 32K context window is the recommended default. Increase only if your hardware has sufficient RAM (rule of thumb: ~1 GB RAM per 8K context for 7B–9B models at Q4 quantization).
⚠️ The following settings have been removed in v8.3.2 and no longer exist:
| Removed Setting |
Previous Location |
Reason for Removal |
| RAG Top K |
Preferences → AI Assistant |
RAG system removed. Full-context injection replaced it. |
| RAG Similarity Threshold |
Preferences → AI Assistant |
RAG system removed. Full-context injection replaced it. |
| RAG Server Status |
Preferences → AI Assistant |
RAG system removed. No vector database needed. |
| Custom Provider |
Preferences → AI Assistant → AI Provider |
Merged into Local LLM → Custom API Endpoint mode. |
| Custom Local Model |
Preferences → AI Assistant |
Replaced by GGUF model identifier input. |
If you were using these settings:
- RAG settings: No action needed. The full-context approach works automatically.
- Custom Provider: Switch to Local LLM → Custom API Endpoint and enter your API base URL there.
- Custom Local Model: Enter the HuggingFace GGUF identifier in the Model Identifier field under Local LLM → GGUF Mode.
See Release Notes for the complete migration guide.