optimclaw/docs/LAZY_TOOLS.md

OptimClaw Lazy Tool Loading

Overview

Lazy tool loading is an optimization that dramatically reduces the initial system prompt size sent to the LLM. Instead of injecting full JSON schemas for every available tool into each request, OptimClaw loads only a small core set of tools eagerly and defers the rest until the LLM requests them.

Impact: System prompt size drops from approximately 13,000 tokens to approximately 4,000 tokens -- a 70% reduction. This saves cost on every LLM call and leaves more of the context window available for conversation history and tool outputs.

How to Enable

Set the environment variable:

export OPTIMCLAW_LAZY_TOOLS=1

Or add it to ~/.optimclaw/.env:

OPTIMCLAW_LAZY_TOOLS=1

To disable (default behavior -- all tools loaded eagerly):

export OPTIMCLAW_LAZY_TOOLS=0
# or simply unset it
unset OPTIMCLAW_LAZY_TOOLS

Core Tools (Always Loaded)

When lazy loading is enabled, the following 12 core tools are always included in the system prompt. These are the tools the LLM needs most frequently and cover the essential interaction patterns:

#	Tool	Purpose
1	echo	Return text to the user
2	time	Get current date and time
3	json	Parse and query JSON data
4	http	Make HTTP requests to allowed endpoints
5	web_fetch	Fetch and extract content from web pages
6	file_read	Read files from the workspace
7	file_write	Write files to the workspace
8	shell	Execute shell commands in the sandbox
9	memory_search	Search persistent memory (hybrid FTS + vector)
10	memory_write	Write to persistent memory
11	message	Send messages to channels
12	tool_info	Discover and load additional tools on demand

Tool Discovery with tool_info

The tool_info tool is the mechanism by which the LLM discovers and loads deferred tools. When the LLM determines it needs a tool that is not in its current context, it calls tool_info to retrieve the full schema.

How It Works

1. The system prompt includes a brief note listing the names of all available (but not yet loaded) tools.
2. When the LLM needs one of these tools, it calls tool_info with the tool name or a search query.
3. tool_info returns the full JSON schema (parameters, description, examples) for the matched tools.
4. The LLM can then call the newly loaded tool in subsequent turns.

tool_info Parameters

{
  "name": "tool_info",
  "parameters": {
    "query": {
      "type": "string",
      "description": "Exact tool name or keyword search query"
    },
    "max_results": {
      "type": "number",
      "description": "Maximum tools to return (default: 5)"
    }
  }
}

Example Flow

System prompt includes:

Additional tools available (use tool_info to load): job_create, job_status, job_cancel, routine_create, routine_list, skill_search, skill_install, extension_install, secrets_set, secrets_get, ...

LLM decides it needs to create a background job:

LLM -> tool_info(query="job_create")

tool_info returns:
{
  "tools": [{
    "name": "job_create",
    "description": "Create a new background job with the given prompt and priority",
    "parameters": {
      "prompt": { "type": "string", "required": true },
      "priority": { "type": "number", "default": 5 },
      "timeout_secs": { "type": "number", "default": 300 }
    }
  }]
}

LLM -> job_create(prompt="Summarize today's news", priority=3)

When to Use Lazy Loading

Scenario	Recommendation
Production deployment with many tools/MCP servers	Enable -- significant token savings
Development and debugging	Disable -- easier to see all available tools
Cost-sensitive usage with expensive models	Enable -- reduces per-request cost
Clusters with heterogeneous tool sets	Enable -- each node may have different tools
Simple setups with few tools (<15 total)	Either -- minimal difference

Performance Characteristics

• First request: Faster, because the system prompt is smaller and the LLM processes fewer tokens.
• Tool discovery round-trip: When the LLM calls tool_info, it adds one extra turn before the actual tool call. In practice this is rare because the 12 core tools handle the majority of interactions.
• Subsequent requests in the same session: Tool schemas loaded via tool_info remain in the conversation context for the duration of the session, so discovery cost is paid at most once per tool per session.

Interaction with Other Features

• MCP tools: MCP-connected tool schemas are also deferred when lazy loading is enabled. They appear in the "additional tools available" list and are loaded via tool_info.
• WASM tools: Same behavior as MCP tools -- deferred and discoverable.
• Skills: Skill tools (skill_list, skill_search, skill_install, skill_remove) are deferred. The skill system itself is unaffected.
• Mesh cluster: Lazy loading is a per-node setting. Different nodes in a cluster can have different settings.