Apideck CLI – An AI-agent interface with much lower context consumption than MCP
TL;DR Highlight
MCP tool definitions alone can consume 55,000+ tokens of context bloat, and Apideck proposes a CLI-based agent interface that uses only ~80 tokens as an alternative.
Who Should Read
Backend and fullstack developers building AI agents or LLM-based automation systems who are experiencing or worried about context window exhaustion when integrating MCP servers.
Core Mechanics
- A standard MCP server with many tools can consume 55,000+ tokens just for tool definitions — a significant chunk of most LLMs' context windows before any actual work begins.
- Apideck's CLI-based approach encodes tool availability as a compact command-line interface description (~80 tokens) rather than full JSON schemas, letting the agent 'discover' what it needs on demand.
- This lazy-loading approach means the agent fetches full tool details only when it decides to use a specific tool, keeping baseline context consumption near zero.
- The tradeoff: the agent needs an extra round-trip to look up tool details before calling them, adding latency. But for long sessions with many available tools, the context savings far outweigh the latency cost.
- The post argues that MCP's current design — front-loading all tool definitions — is fundamentally mismatched with context window economics and needs rethinking for large-scale tool ecosystems.
Evidence
- Commenters verified the 55K token figure by measuring real MCP servers — one person checked an enterprise CRM MCP and found it exceeded 80K tokens for tool definitions alone.
- Several developers noted they'd hit this problem in practice and resorted to workarounds like splitting tools across multiple MCP servers or selectively disabling tools.
- The Apideck team shared benchmark data showing response quality was comparable between the full-schema and CLI approaches for common API tasks, with the CLI approach using ~680x fewer tokens for tool definitions.
- Some skeptics argued that the CLI approach sacrifices type safety and discoverability — the LLM has less precise information about parameter formats, potentially increasing errors.
How to Apply
- Audit your current MCP setup: run a token counter on all tool definitions. If you exceed 10K tokens just for definitions, you have a context bloat problem worth solving.
- Group related tools and load only the relevant group for each task context. For example, 'database tools' vs 'API tools' vs 'file tools' as separate MCP servers.
- Consider implementing a tool registry pattern: expose a 'list_tools' meta-tool that returns brief descriptions, then 'get_tool_schema' for details only when needed.
- For the highest-frequency tools (the 20% you use 80% of the time), keep full schemas in context. For the long tail, use lazy-loading.
Terminology
Related Papers
Show HN: ctx – Search the coding agent history already on your machine
Claude Code, Cursor, Codex 등 코딩 에이전트가 이전 세션의 논의·결정·실패 시도를 잊지 않도록 SQLite로 인덱싱해 재사용할 수 있게 해주는 오픈소스 CLI 도구다.
Micro-Agent: Beat Frontier Models with Collaboration Inside Model API
vLLM 팀이 단일 모델 API 호출 뒤에서 여러 모델이 협업하는 'Micro-Agent' 개념을 공개했습니다. 별도의 에이전트 코드 없이 라우터 레이어에서 모델 조합을 실행해 GPT-4급 결과를 더 저렴하게 낼 수 있다는 아이디어입니다.
Ornith-1.0: self-improving open-source models for agentic coding
Gemma 4와 Qwen 3.5를 기반으로 파인튜닝한 코딩 특화 오픈소스 모델로, RL(강화학습)을 통해 스캐폴드(에이전트 실행 구조)까지 함께 최적화하는 방식을 주장하지만, 커뮤니티에서는 벤치마크 과최적화에 불과하다는 의심을 받고 있다.
Entity Binding Failures in Tool-Augmented Agents
AI 에이전트가 올바른 도구를 선택해도 잘못된 대상에 실행하는 'Entity Binding 실패' 문제를 정의하고, 이를 막는 실행 정책을 평가한 논문.
Herdr: Agent multiplexer that lives in your terminal
여러 AI 코딩 에이전트(Claude, Codex 등)를 하나의 터미널에서 동시에 실행·관리할 수 있는 Rust 기반 오픈소스 툴로, tmux처럼 세션이 유지되고 SSH로 원격 접속도 가능해 멀티 에이전트 워크플로우를 크게 단순화해준다.
Ornith-1.0: Self-scaffolding LLMs for agentic coding
모델이 문제 풀이 전략(scaffold)을 직접 생성하고 개선하는 자기강화 학습 프레임워크를 적용한 오픈소스 코딩 특화 LLM으로, 9B 소형 모델부터 397B 대형 모델까지 라인업을 갖추고 SWE-Bench 등 주요 벤치마크에서 Claude Opus 4.7을 능가하는 성능을 보여줬다.