Zum Hauptinhalt springen

LLM Models & Usage

azuma nori is completely agnostic to the underlying LLM provider, giving you the flexibility to choose between raw performance, data privacy boundaries, and API cost efficiency.

Connection Types​

The engine supports two primary paradigms for model connectivity:

  1. Direct Cloud APIs: Connect directly to Anthropic (Claude), OpenAI (GPT), or Google (Gemini).
    • Native Gemini SDK: Direct Cloud API calls for Gemini connect natively using Google's SDK, supporting Gemini 3 thinking models with native token tracking and extended request timeouts for large codebases.
  2. Local Executable Wrappers: You can configure nori to use specialized command-line agent wrappers (like the Cursor CLI, Gemini CLI, Claude CLI, or OpenAI Codex CLI). These run as local processes on your workstation.
  3. On-Premise / Private Cloud: By overriding the API Endpoint in your configuration, you can point nori to a completely private, firewalled model instance (such as Azure OpenAI deployed within your corporate VNet or a localized open-source model). In this mode your code is evaluated entirely within your own network and is never sent to an external LLM.

Cost Optimization & Per-Phase Models​

Evaluating an entire codebase and indexing documentation for compliance is resource-intensive. Nori includes several advanced cost-control and performance mechanisms:

  • Per-Phase Model Assignment: You can select different AI models for different pipeline phases. For example, assign a cheaper, faster model for mechanical tasks (like Wiki Generation or Document Indexing) and reserve your strongest frontier model for reasoning-heavy tasks (Control Analysis).
  • Per-Run Override: Choose a different AI model just for the current run from the launch options without modifying default project settings.
  • Concise Wikis: Enable the Concise option to produce shorter, fact-dense wiki pages, capping text size and saving downstream evaluation tokens.
  • MaxBudgetUsd: Set a hard spending limit (USD) per run. The engine halts if projected token costs exceed this budget.
  • Context Compression: Enabled by default, this feature strips unnecessary whitespace, filler words, and formatting from the Wiki context, saving up to 60% on context window tokens.
  • Usage & Token Tracking: Nori tracks exact token counts (including cached read/creation tokens) across supported providers and calculates the precise USD cost of every run.

Choosing a Provider​

When selecting your provider in the setup phase, consider:

  • Anthropic Claude (Opus / Sonnet class): A strong default for its large context window and reasoning quality when reading dense source code and compliance documents.
  • OpenAI GPT (GPT-5 class or newer): Reliable structured JSON output and strict adherence to the agentic JSON schema nori expects.
  • Google Gemini (3 thinking models): Large context windows with native token tracking; well suited to large codebases over the Direct Cloud API.

nori is agnostic to the exact model version — always check your provider for the current model IDs.