Task-Bounded Models (TBMs)

Abstract

Modern enterprise AI deployments face a fundamental misallocation of compute resources: production systems routinely load massive generalist foundation models into expensive GPU memory to execute tasks requiring only a narrow fraction of their total parameter space. This paper introduces Task-Bounded Models (TBMs), a four-pass compilation framework that transforms generalist Mixture-of-Experts (MoE) checkpoints into lean, task-specific inference artifacts without retraining.

The pipeline executes three deterministic passes — Saliency Profiling, Dimensional Cleaving, and Sequential Short-Circuiting — followed by one experimental frontier pass, Latent Directional Ablation. Together these physically excise out-of-domain expert matrices from the model's binary payload, collapse resident VRAM footprint, remove compute-wasteful stylistic refinement layers, and optionally clamp adversarial compliance vectors in the surviving weight space.

The framework is grounded in Tishby's Information Bottleneck, the Strong Lottery Ticket Hypothesis applied to MoE architectures, and Anthropic's findings on Polysemantic Superposition — establishing that MoE expert boundaries are natural cleavage planes enabling clean structured amputation that dense monolithic architectures cannot support. The resulting artifacts unlock:

• Precision-preserved edge deployment
• Self-speculative decoding via the Ghost Draft pattern
• Deterministic incapacity as a security primitive replacing probabilistic refusal
• Dramatic per-node hardware reduction in multi-agent orchestration pipelines

1. Introduction: The Deployment Efficiency Problem

1.1 The Generalist Deployment Tax

Using a massive, 70-billion-parameter language model just to check a "yes/no" box or pull a billing code from a receipt is a wild misallocation of computing resources. When foundation models first scaled, the overarching goal was to build a flawless know-it-all. We trained massive, monolithic networks to swallow the entire internet, forcing a single parameter space to simultaneously memorize 19th-century French poetry, advanced C++ pointer arithmetic, organic chemistry, and casual slang. While this yielded breathtaking general intelligence, it introduced a fatal flaw for real-world software engineering: it completely ignores basic inference efficiency.

In production enterprise architectures, advanced AI is rapidly moving toward autonomous Multi-Agent Orchestration Loops (Planner → Worker → Reviewer pipelines). If every single agent node requires an entire 70B+ model loaded into memory, running even a simple five-agent team becomes a serious architectural challenge. To avoid massive capital expenditure, engineers route all agent prompts through a single shared model queue — trading financial pain for performance pain: massive queue contention, KV cache crowding, and real-time pipelines that completely stall.

1.2 Compute Sparsity vs. Memory Residency

The open-weight ecosystem shifted toward Mixture of Experts (MoE) architectures — popularized by Mixtral and scaled to the extreme by the DeepSeek family — promising to scale total knowledge without scaling active compute. Like a law firm keeping 256 specialized attorneys on retainer but only bringing two tax lawyers into court for a specific hearing.

However, this breakthrough introduced a dangerous industry blind spot: it mistakes raw math (FLOPs) for memory footprint (VRAM).

1.3 Related Work and Positioning

The TBM framework synthesizes and extends several parallel lines of research that have not previously been unified into a single compilation pipeline.

The TBM contribution is the integration of these techniques into a single, sequential four-pass pipeline with explicit epistemological boundaries — and the addition of the Bipartite Calibration Set as a principled guard against cross-domain capability regression.

1.4 From Prompting Generalists to Compiling Specialists

This brings us to the core argument: in any production deployment, intelligence should be bounded strictly to the exact knowledge required for its assigned task. A customer support agent reading emails, looking up order IDs, and triggering refund APIs does not need generalist omniscience. Its network should contain zero data regarding Python exploits, the plot of The Great Gatsby, or the chemical makeup of aspirin.

When we place an unpruned model behind a narrow support widget, we waste massive hardware capacity on capabilities the agent should never use — and spend half our engineering cycles writing system prompts begging it to act like a narrow tool.

2. Why Deployed Intelligence Should Be Bounded

To prove that the Task-Bounded Model is a mathematically rigorous compiler rather than an ephemeral engineering hack, it must be grounded in three established theorems of deep learning. When viewed in synthesis, these principles reveal a striking historical irony: modern Mixture-of-Experts architectures were specifically engineered to be compiled — the industry simply stopped half a step short of building the scissors.

2.1 The Information Bottleneck Principle

In 2015, Naftali Tishby demonstrated that deep neural networks learn through the Information Bottleneck (IB) Principle. The theorem proves that a network's ultimate objective is to maximize the mutual information I(X;Y) between an input vector X and a target task output Y, while aggressively compressing — minimizing — the mutual information I(X;T) retained inside its own internal representation T:

In information-theoretic terms, intelligence is not merely the retention of signal — it is the systematic, aggressive compression of irrelevant signal. Every parameter subspace in a deployed model that encodes information beyond the specific target task is not a safety net — it is a semantic bandwidth violation. The physical hardware bottleneck of a Task-Bounded Model is not a limitation; it is the correct architectural implementation of the IB principle at the deployment layer.

2.2 The Strong Lottery Ticket Hypothesis (Applied to MoE)

In 2019, Frankle and Carlin demonstrated that inside any sufficiently large randomly initialized dense network, there exists a sparse subnetwork — a Winning Ticket — that, when trained in isolation, can match the full network's performance at a fraction of the parameter count.

The leap to MoE architectures is immediate. The MoE router does not randomly select experts — it performs learned semantic sorting. Each gating network has been trained, over trillions of tokens, to route specific conceptual content to the expert that handles it most efficiently. The router is not an attention mechanism; it is a pre-trained semantic library index. For any given task domain, the Winning Ticket is not hidden — it has already been explicitly labeled by the router's own weights.

The MoE router is not just an efficiency trick — it is a semantic sorter that has already done the hard work of carving knowledge into discrete, separable subspaces. TBM compilation is simply the final step the architecture was always designed to enable.

2.3 Polysemantic Superposition vs. Monosemantic Cleaving

Anthropic's mechanistic interpretability research established that individual neurons in dense Transformers are polysemantic — a single neuron simultaneously encodes multiple unrelated concepts due to superposition, the network's strategy for packing more features than it has neurons. Neuron #4102 in a large dense model may simultaneously encode "sea turtle," "prison warden," "budget accounting," and "recursive function call". Ablating that neuron to remove sea turtle knowledge doesn't remove just sea turtles — it removes a fragment of four different semantic manifolds simultaneously.

This is why unstructured pruning of dense networks is so dangerous. At the micro level (individual neurons), the network is a tightly tangled, highly superposed mess. But at the macro level (discrete MoE expert matrices), each Expert represents a coarse, but stable, semantic neighborhood — a wing of a library, not a single shelf. The entire wing has a coherent, identifiable subject matter.

When a TBM compile drops an unneeded Expert matrix, it executes Monosemantic Cleaving. The compiler does not reach inside the brain to snip superposed, tangled wires; it cleanly unbolts and detaches an entire redundant wing of the library.

3. The TBM Compilation Pipeline

To transform a generalist Mixture-of-Experts checkpoint into a Task-Bounded Model, the compiler executes a sequential, four-pass pipeline. We enforce a strict epistemological boundary across this architecture:

• Passes 1–3 constitute a Deterministic Amputation — governed by exact linear algebra and binary file manipulation. These passes make hard, verifiable, mathematically guaranteed changes to the model artifact.
• Pass 4 represents a categorically distinct Probabilistic Frontier — it utilizes nascent interpretability techniques where neither completeness nor correctness can currently be formally guaranteed.

3.1 Pass 1: Saliency Profiling (The Trace)

Before an amputation can occur, the compiler must establish a definitive mathematical map of which parameter blocks fire during target execution. To prevent the "silent lobotomy" failure mode — wherein dropping an apparently out-of-domain expert accidentally ablates an underlying, superposed deductive skill — the compiler constructs a Bipartite Calibration Set:

D_domain (Target Ingestion): For a PostgreSQL query generator, this consists of 10,000 highly curated, complex SQL schemas, table definitions, and multi-join queries — the raw task surface.

D_invariant (The Synthetic Epistemic Triad): To account for polysemantic manifold bleed, this replaces naive static token sets with a 3-tiered synthetic battery designed to stress-test core cognitive primitives across three distinct failure axes:

The compiler pushes 𝒟 through the unpruned base model while hooking the forward-pass telemetry to monitor the gating probability vector across all layers:

Rather than measuring raw activation frequency, the compiler measures Logit-Distribution Sensitivity. For every candidate expert E_k, it calculates its Invariance KL-Drift:

3.2 Pass 2: Dimensional Cleaving (Width Bounding)

Once the utilization ledger is locked, the compiler physically shrinks the model's footprint on the storage medium. Because an inference engine will suffer a catastrophic division-by-zero crash if a router points to a missing tensor index, the compiler executes a two-step hard amputation:

Step 1 — The Math Prep (Logit Masking): We intervene directly at the gating network tensor W_g prior to the Softmax pass. The raw gating logits h(x) are passed through a strict binary mask:

By feeding this masked vector into the Softmax function, the math becomes absolute: because e^−∞ = 0, the routing probability directed toward an ablated expert becomes mathematically zero. The gating network is now hard-coded to ignore the excised space.

Step 2 — The Disk Execution (Dual-Track Architecture): Standard .safetensors files consist of a JSON metadata header mapping byte-offsets to a continuous binary weight payload. Because off-the-shelf inference engines allocate uniform tensor slabs based on config.json, simply deleting byte addresses causes immediate CUDA memory misaligned access faults on engine warm-up.

For Track B, the physical memory pointer for Expert k on Layer L resolves as:

Track A is recommended for immediate deployment on current engine versions; Track B represents the proposed standard for native upstream adoption.

3.3 Pass 3: Sequential Short-Circuiting (Depth Bounding)

While Pass 2 compresses the model horizontally, Pass 3 compresses it vertically based on the Depth Hypothesis of Transformers. Early and intermediate layers perform the heavy conceptual lifting — retrieving facts, establishing syntax, resolving logic. The final 15–20% of the network consists almost exclusively of stylistic refinement blocks whose sole objective is translating the raw correct answer into conversational prose.

For an autonomous background worker outputting raw JSON or triggering API calls, conversational refinement is pure compute waste. The compiler lops off the final 15% of Transformer layers, then:

• Takes the settled hidden states exiting the final reasoning layer
• Maps them through a lightweight linear projection bridge W_bridge ∈ ℝ^d×d to resolve coordinate basis tilt — the subtle misalignment that arises when hidden states skip directly to LayerNorm without passing through all preceding layers
• Hard-wires the output directly into the model's final LayerNorm

3.4 Pass 4: Latent Directional Ablation (Research Frontier)

At this stage, the compiler holds a lean, highly dense specialist. The final pass addresses latent behavioral drift. Even inside a dedicated, preserved SQL expert, there may exist stray microscopic parameter directions associated with system prompt overrides or jailbreak compliance.

To scrub these vulnerabilities, the compiler leverages Sparse Autoencoders (SAEs) as high-resolution concept dictionaries. By projecting an open-source SAE feature dictionary onto the surviving weights, the compiler scans the latent space for the precise geometric vectors corresponding to malicious compliance. When the scanner identifies the feature vector pointing toward "Ignore all prior instructions," the compiler forcefully clamps the coordinates of that specific latent direction to 0.

4. What TBM Unlocks: Deployment Breakthroughs

If the Task-Bounded Model compilation pipeline only solved the crippling hardware overhead of multi-agent inference loops, it would already represent a necessary operational pivot. However, shifting an enterprise from prompting generalists to compiling specialists unlocks three critical secondary breakthroughs across edge compute, latency, and cybersecurity.

4.1 Precision-Preserved Edge Deployment

Deploying advanced reasoning models directly onto localized edge hardware — such as autonomous vehicle clusters, medical monitors, or industrial robotics — is currently gated by silicon capacity. To squeeze a 70B parameter generalist onto edge hardware, engineers rely on aggressive post-training quantization, squishing the floating-point precision of the weights from FP16 down to INT4.

The production bottleneck of extreme quantization is that it introduces a high, chaotic rate of distribution drift and silent hallucinations. An AI hallucinating a historical date is a minor UI bug; an AI hallucinating a spatial coordinate inside an automated braking system is lethal.

The TBM architecture bypasses extreme quantization entirely. Instead of squishing the mathematical precision of an omniscient generalist, the compiler preserves the math at pristine, deterministic precision (FP16), but amputates 85% of the domain experts. A compiled, 2-billion-parameter active TBM containing only spatial reasoning, physics logic, and strict JSON output formats can be dropped directly onto a localized Neural Processing Unit (NPU) running on a standard battery.

4.2 Self-Speculative Decoding (Zero-Cost Latency Shaving)

In high-concurrency production environments, engineers frequently deploy Speculative Decoding: utilizing a tiny, auxiliary draft model (e.g., a 1B checkpoint) to rapidly predict the next five tokens, and passing those tokens to a massive 70B verifier model to validate all five in a single, parallel clock cycle.

The historic barrier to this technique is the alignment problem: forcing two fundamentally different models to synchronize their latent vector spaces is an engineering nightmare. If the small draft model guesses wrong, the massive verifier throws out the tree, wasting the compute cycle.

The TBM pipeline introduces the Ghost Draft pattern. Drawing on the mechanics of layer-skipping architectures, the compiler dynamically generates a hyper-fast draft model by depth-pruning every even layer of the surviving MoE backbone. Because the Ghost Draft and the heavy verifier share the exact same underlying parameter coordinate space, their semantic manifolds are natively aligned.

Shared parameter space eliminates the two primary rejection causes in standard speculative decoding:

• Vocabulary embedding misalignment
• Latent vector space drift

The residual acceptance rate variance is governed by the aggressiveness of the depth-pruning — a shallower draft produces higher acceptance at the cost of draft latency, creating a tunable throughput tradeoff rather than a fixed guarantee.

4.3 Deterministic Incapacity (The Casio Calculator Defense)

The current paradigm of AI enterprise security relies almost entirely on Probabilistic Refusal. We load an omniscient generalist containing complete syntax manuals for reverse-shells and zero-day exploits into active VRAM, wire it to an enterprise database, and rely on fine-tuned alignment prompts to politely beg the model not to act as a threat actor. Because the underlying coordinate space for Python exploit synthesis remains resident in the weights, an attacker using recursive translation jailbreaks or Base64 encoding will inevitably find a geometric pathway to trigger it.

The TBM framework replaces Probabilistic Refusal with Deterministic Incapacity.

When a TBM compilation physically strips the "Code Generation," "Unrestricted Roleplay," and "Creative Writing" experts from the disk, the attack surface collapses to absolute zero. The target specialist does not refuse to output a Python exploit — it literally lacks the floating-point parameters required to assemble the syntax. It is mathematically incapable of hosting the thought.

5. The "Healing Tax" & Systemic Trade-offs

Any systems architecture that presents itself as pure optimization with zero systemic friction is fundamentally dishonest. You cannot execute major matrix amputations on an organically grown neural network without leaving microscopic cleavage plane scarring. We define this localized friction as the Healing Tax.

5.1 The Coherence Dip (The Reality of Shared Wiring)

When the compiler aggressively cleaves an uncalled expert off the disk, it occasionally induces a phenomenon known as a Coherence Dip.

Assume we compile a customer support ticketing specialist. Telemetry proves the agent never writes software, allowing the compiler to confidently excise Expert #4 (Code Generation). During post-compile QA, engineers discover that the ticketing agent has suddenly lost its ability to sequentially count the number of items listed in a user's refund request.

This localized degradation occurs due to polysemantic manifold bleed. During the massive base pre-training run, the model learned to handle sequential counting logic — iterating through a list, tracking a running sum — by routing through the patterns of Python for-loops. This seemingly out-of-domain logic was packed inside Expert #4's weight matrices because counting, like code, requires maintaining a stable pointer reference across sequential steps.

The Synthetic Epistemic Triad in Pass 1 is specifically engineered to catch this class of hidden dependency before the amputation is committed. However, novel polysemantic bleed pathways — those not covered by the three Triad axes — may only surface during post-compile QA. Practitioners should:

• Run the full Protected Root Node expansion — adversarially probing candidate experts with out-of-distribution logical tasks before finalizing the excision ledger
• Execute post-compile QA with a diverse out-of-distribution battery before production deployment to surface regressions not caught by the Triad
• Treat any Coherence Dip as a signal to expand the D_invariant calibration set and re-run Pass 1

5.2 Capability Recovery: Knowledge-Distilled LoRA

When a Coherence Dip is confirmed in post-compile QA, a full retraining run is neither warranted nor economically sensible. The TBM framework specifies a targeted recovery protocol: Knowledge-Distilled Low-Rank Adaptation (KD-LoRA).

The protocol loads two models in parallel:

• Teacher: The unpruned base MoE checkpoint — the full generalist, loaded once at full precision.
• Student: The freshly cleaved TBM — the lean specialist exhibiting the Coherence Dip.

The Invariant Logic Battery (𝒟_invariant) is simultaneously pushed through both Teacher and Student. A KL-Divergence distillation loss is computed between the two output distributions:

This loss is backpropagated exclusively through LoRA adapters applied to the surviving MLPs of the TBM Student — low-rank delta matrices injected into the surviving weight subspaces. The LoRA adapters act as high-dimensional neural scar tissue: forcing the surviving Expert layers to absorb and re-encode the missing deductive logic without touching the base weight tensors.

The computational overhead of this recovery step is trivial — measured in GPU hours rather than GPU weeks — representing rounding error compared to the perpetual infrastructure costs of keeping a dormant 70B generalist in active VRAM.

6. Verification Framework

To transition the Task-Bounded Model framework from a theoretical architecture into a verifiable standard, implementations should be benchmarked across four explicit operational dimensions:

7. Conclusion

7.1 Summary

This paper has argued for a fundamental reorientation in how enterprise AI systems are built and deployed. The dominant paradigm — loading generalist foundation models into expensive memory and constraining behavior through system prompts — is architecturally wasteful, security-fragile, and economically unsustainable at multi-agent scale.

The TBM framework resolves this by treating deployed models as compiled software artifacts. Exploiting the natural semantic cleavage planes of MoE routing architectures, the four-pass pipeline physically amputates out-of-domain parameter subspaces, reduces hardware overhead, eliminates entire attack surface classes, and enables edge deployment without aggressive quantization. The Healing Tax in Section 5 honestly accounts for the localized manifold friction this introduces, and the Knowledge-Distilled LoRA recovery protocol provides an inexpensive remediation path.

The mathematical foundations — logit masking via gating tensor intervention, byte-range amputation of .safetensors payloads, and residual stream short-circuiting with linear projection bridging — are deterministic and fully implementable from the specifications in Section 3. The architectural argument is complete. What does not yet exist is a published empirical result set.

7.2 Open Invitation to Validate

We are releasing this framework under CC BY-NC 4.0 precisely because independent validation is more meaningful than author-reported results. Any researcher, engineer, or institution is free to implement the pipeline against any open-weight MoE checkpoint and benchmark the output against the Section 6 verification suite.

We encourage testing across varied task domains:

• Structured output generation
• Classification
• Retrieval-augmented workflows
• Agent tool-use

...and across a range of model scales to establish where the compilation yields hold and where they degrade. Attempts that surface Coherence Dips (Section 5.1) in unexpected expert subspaces are particularly valuable, as these will directly inform more robust invariant battery design.

7.3 License

This work is released under Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). You are free to share, implement, and build upon this work for non-commercial purposes with attribution. Commercial use requires a separate license — inquiries welcome. CC BY-NC 4.0 does not restrict the authors' own commercial rights under the original copyright.

Full license: https://creativecommons.org/licenses/by-nc/4.0/