AI ENGINEER HELP Runs in your browser

THE ENGINEERING LAB / CONTEXT

Less context.
Still the whole answer?

A model can’t use evidence it never sees. Explore how query-aware compression chooses what stays—and find out what gets lost.

Start with the four techniques
CONTEXT, WITH INTENT
SOURCE
query→
KEPT
Same words.
Less noise.
Select evidence. Preserve its meaning.

01 / THE TECHNIQUES

Four ways to cut context, step by step.

One handbook. One question.
Two facts the answer depends on.

The question

“Can I get a refund for an annual subscription after exporting client archives?”

The budget

Keep 40% of a 462-token customer handbook.

To answer correctly, keep

the 30-day refund rule and the exception that cancels it after an export.

02 / BEYOND LEXICAL

What models change.

The same split → score → pack idea,
with a model doing the scoring.

The four techniques above score text with word counts you can check by hand. These methods replace that scoring with a model. They fix some of the failures you just saw and bring new ones. Adapters for each are included in the code, but none run on this page: local models are disabled, and remote ones need keys and your consent. Nothing below is a measured result.

Semantic ranking

BGE embeddings

Scores each sentence by how close its meaning is to the question, not its exact words.

question + each sentencebge-small-en-v1.5 embeds bothcosine similaritysame budget packer
Would help here
The synonym miss. “Money back on a yearly plan” and “full refund for annual subscriptions” share no words but are close in meaning.
Watch out
Close in meaning isn’t the same as needed. An exception that contradicts the question can score lower than a sentence that restates it.

Adapter included. Not run on this page: local models are off.

Model card ↗

Learned word deletion

LLMLingua-2

A small classifier predicts, for every token, the probability that it should be kept. It was trained on compressions written by a large model.

textBERT-size token classifierkeep probability per wordkeep the most likely words
Would help here
Word deletion’s broken grammar: a trained model learns that “not” and “except” carry meaning.
Watch out
It never sees the question. At tight budgets it keeps what’s generally informative, which may be the wrong section.

Adapter included. Not run on this page: local models are off.

Paper ↗

Perplexity pruning

LLMLingua

A small language model reads the text and drops the tokens it finds most predictable. Predictable tokens add little information.

textsmall causal LM (Phi-2 here)surprise per tokendrop the least surprising
Would help here
Trimming filler and boilerplate inside long passages, not just whole sentences.
Watch out
Question-blind. A rare-but-irrelevant detail looks surprising, so it may survive while the answer is pruned.

Adapter included. Not run on this page: local models are off.

Paper ↗

Question-aware perplexity

LongLLMLingua

Asks how much each document makes the question more likely, ranks and reorders documents, then prunes tokens inside them.

question + documentssmall causal LMrank and reorder documentsprune tokens
Would help here
Many retrieved passages where the answer is buried in the middle, the case it was designed for.
Watch out
Several model passes per request add latency. Reordering changes what the answerer reads first.

Adapter included. Not run on this page: local models are off.

Paper ↗

Model as relevance judge

Laya remote

Asks a remote model a typed yes/no question for each sentence (“is this relevant?”) and uses the probability as the score.

question + each sentenceremote model judgementprobability of “relevant”same budget packer
Would help here
Questions that need understanding rather than matching: “does this sentence change the answer?”
Watch out
One judgement per sentence costs time and money, and the probabilities need checking on real data.

Adapter included. Needs a configured remote server and your consent per call.

Project ↗

Hosted service

Compresr Latte

Send text and the question to an API and get compressed text back. Paragraph or token-level modes, fixed or automatic ratio.

text + questionhosted APIcompressed text
Would help here
Production use without running models yourself.
Watch out
Your text leaves the device and each call is billed. The engine is closed, and the output has no offsets, so the lab can’t show exactly what was cut.

Adapter included. Needs an API key and your consent per paid call.

Docs ↗

03 / YOUR TURN

Put any context on a budget.

Three scenarios, every setting, your own text.
Runs in your browser · model runs are explicit.

Full context

Everything available to the answerer

Compressed context

Selected source spans · expand a cut to inspect it
Kept source textRemoved from contextToken encoding: cl100k_base

THE EVIDENCE CHECK

Did the important facts survive?

Checks use annotated source spans, not an LLM judge. Retained facts do not guarantee a correct answer.

Under the hood Inspect every relevance score ↗

Lexical overlap with an inverse-document-frequency weight, divided by √unique terms. Heading words count too. Scores are rankings, not probabilities. Evidence labels never influence selection.

Try your own text Choose where it runs ↗

Custom text has no annotated evidence check. Offline methods stay on this device; hosted Compresr receives text only when you explicitly run it. Nothing is persisted automatically.

04 / THE TRADE-OFF

Fewer tokens is a metric.
Enough evidence is the goal.

Try a smaller budget. Turn off the read’s focus. Switch to truncation. A compressor can produce a very small context while losing exactly the sentence that matters.

Retrieval can fail first

A compressor cannot recover a fact your retriever never returned. The RAG example reports those misses separately.

Context has dependencies

A correction, a negation or a table heading may change another sentence’s meaning. Selection by relevance alone cannot guarantee safety.

Savings have overhead

A live compressor adds inference and network time. Measure total latency, output quality and cost—including prompt caching—before claiming a win.

05 / ADVANCED · TWO RUNS, ONE QUESTION

Does the shorter context still work?

Separate answers. A blind judge.
Or two agents choosing their own reads.

Everything so far checks whether facts survive. That’s necessary, not sufficient: a model can still misread a compressed context, or answer correctly without it. This section runs the answering step itself, with and without compression.

Use the question and compressor selected in the lab. Start with a scripted workflow to learn the mechanics; live generation requires separate, explicit consent.

No workflow has run yet. Scripted mode is clearly labeled and never presented as model-generated answers or an accuracy benchmark.