Antigravity Agent Architecture and Evaluation Loop Design

1. Introduction

Recently, natural language AI models from major vendors have focused heavily on agent-based coding and reasoning capabilities. Because of this focus, model outputs frequently exhibit a distinct, mechanical AI style or translationese phrasing.

In practice, several repetitive patterns stand out:

• Overusing double or single quotation marks to forcefully emphasize words and phrases
• Spurring unnecessary em-dashes (—) throughout English drafts
• Utilizing unnatural demonstrative pronouns that disrupt regional prose flow
• Mechanically repeating contrastive structures like not just A, but B or going beyond A in sentence openings
• Adopting strange, unrealistic metaphors such as rhythm, swaying, or smelling a scent
• Restricting vocabulary by flattening nuanced terms like adaptable or prepared into a generic powerful label
• Obsessively prepending assertions like key or essential before presenting claims
• Writing out cardinal numbers as words instead of digits (e.g., spelling out three options instead of using 3 options, or writing four items instead of 4 items)

As these styles appear repeatedly across agent systems, developers and readers alike are experiencing AI-style fatigue. To explore harness engineering, I investigated how to quantitatively measure these stylistic flaws and define guardrail metrics. The core agent architecture logic and evaluation codebase have been structured and documented. You can explore the repository structure and configurations at the following link:

KennethanCeyer/natural-writing-harness

Preview unavailable.

https://github.com/KennethanCeyer/natural-writing-harness

Let us look at why giving full authority to a single AI model leads to failures, and explore the structure of our 3 separate stages. Traditionally, when instructing an AI to edit text, the model tries to handle evaluation, rewriting, and file-saving all in one go. In this process, links are frequently dropped and files are corrupted. To prevent these failures, each stage must be physically decoupled.

• Evaluator: Scans the document to detect repeated sentence openings or shifts in tone, saving concrete evidence of issues.
• Revision Harness: Edits only the targeted problem passages and validates that numbers and links are preserved in the output.
• Antigravity Agent: Restricts tool permissions so that only a validated candidate is written to disk after explicit user approval.

By separating these responsibilities, we prevent the AI from corrupting files or executing unauthorized actions on your workspace. In this project, we adopted a high-performance inference model for passage rewriting and contextual checks, while separating and controlling file-write authority via the agent runtime policy. You can read more about the model details at the following link:

Gemini 3.5 Flash — Google DeepMind

Preview unavailable.

https://deepmind.google/models/gemini/flash/

Through this article, you can master and apply the following capabilities directly to your projects:

• AI Style Diagnostics: Identify and quantitatively measure repetitive openings, overused punctuation, and translationese patterns to diagnose drafts.
• Contextual Refinement Harness: Decouple document rewriting into local target regions to preserve hyperlinks and facts while refining prose.
• Governed Save Boundaries: Implement a deny-by-default sandbox that restricts file-saving authority using hash verification to secure your workspace.

How is this 3-stage layout concretely implemented at the local source code level? Let us now dissect the working principles of the architecture by examining the repository layout and module specifications.

2. Harness and agent architecture

Let's look at what these terms actually mean and why they are necessary.

By examining the codebase directory structure of the repository, we can gain a concrete understanding of how the validation system works. Below is the layout and role specification of the key modules located under src/natural_writing_harness/:

src/natural_writing_harness/
├── style_diagnostics.py       # Style check rules (detecting repeated openings, etc.)
├── semantic_diagnostics.py    # Semantic flow check (assessing robotic AI tone intensity)
├── refinement_service.py      # Refinement service (direct cleanups or routing to LLM)
├── quality_gate.py            # Original fact gatekeeper (protecting numbers and links)
├── rewrite_policy.py          # Approval decision maker (verifying warning reductions)
├── agent_runtime.py           # Agent security setup (handling user approval popups)
└── agent_state.py             # Safe file saver (matching filename and hash ID)

As shown in the folder layout, each file takes charge of a specific step in the check, rewrite, and save sequence. Let us first review the concrete duties of these files in the table below:

As summarized in the table, each module handles its assigned duties independently. Let us divide the entire system where these files cooperate into 2 architectural areas to examine how they function:

Verification harness design

The validation harness is a guardrail system that monitors the LLM's inputs and outputs. If you instruct an LLM to rewrite a document directly, it often omits or distorts original details. To prevent this, the refinement service (refinement_service.py) orchestrates the editing workflow. It guides the model to rewrite only targeted sections, and filters out candidates that fail to preserve original context, numbers, or links.

Specifically, style_diagnostics.py quantifies syntactic anomalies like repeated openings, while semantic_diagnostics.py leverages the Gemini API to analyze claim strength shifts. Once the revision is ready, quality_gate.py validates that links and numbers remain intact, and rewrite_policy.py compares style failure counts to decide whether to accept the new draft.

Agent architecture design

Writing prompt constraints is not enough to protect local source files. Giving write permissions directly to an LLM increases the risk of file damage due to hallucinations or errors. To block this risk, we decoupled write authority from the model's action space. The agent stashes approved revisions in a buffer with a unique SHA-256 Hash ID, and authorizes a save operation only when the user's manual approval matches both the approved filename and hash ID.

To run these separated stages together in local development, we needed a unified command-line tool. The nwh CLI, described in the next section, is the primary interface for this architecture.

3. nwh CLI

During development, we designed a standalone command-line tool named nwh (Natural Writing Harness). Our goal was to isolate document verification and local metrics from agent runtimes and IDE extensions, making it easy to run benchmarks and integrate validation into automated CI/CD pipelines.

The primary commands provided by the nwh CLI and their functionalities are summarized below:

Running these CLI commands triggers a sequence of comparisons and diagnostics. Let us trace how the workflow coordinates the 3 candidate texts and evaluation algorithms.

4. Revision and validation workflow

4.1 Pinned comparison conditions

To measure performance, we compare 3 candidates generated from the same prompt: a human-written reference, a baseline candidate created without harness constraints, and a revised candidate shaped by the harness. The human reference serves as a baseline comparison point for how a developer approached the same task, and is excluded from the rewrite prompts.

4.2 Balances between rules and context checks

The evaluation loop handles style defects based on their nature. Code-based rules track missing numbers or dropped links, while a Gemini model evaluates characteristics like claim strength or flow. Crucially, the semantic evaluator answers specific, narrow questions and returns ABSTAIN when context is insufficient, preventing personal stylistic preferences from hardening into automated rules.

While working on this code, I found that controlling natural language based on traditional string-matching or simple statistical metrics alone is harder than it looks. Quantifying subtle shifts in tone and phrasing requires design patterns that go far beyond simple rule-matching.

4.3 Post-revision verification

Because shortening sentences or formatting lists can delete details, the harness runs the same diagnostic checks on the generated candidate. If the edit introduces new style warnings or fails to reduce the trigger signal by at least 25%, the candidate is rejected.

Let's look at the code. The following Python function implements the validation rules that compare pre- and post-revision signals:

rewrite_policy.py
def require_contextual_improvement(baseline_signals, revised_signals):
    actionable = {"medium", "high"}
    baseline_levels = {
        signal.code: max(signal.normalized_excess, 1.0)
        for signal in baseline_signals
        if signal.severity.value in actionable
    }
    revised_levels = {
        signal.code: max(signal.normalized_excess, 1.0)
        for signal in revised_signals
        if signal.severity.value in actionable
    }
    new_codes = set(revised_levels) - set(baseline_levels)
    if new_codes:
        raise RewriteRejected(
            "The revision introduced actionable style failures"
        )
    improved = any(
        code not in revised_levels
        or revised_levels[code] <= baseline_levels[code] * 0.75
        for code in baseline_levels
    )
    if baseline_levels and not improved:
        raise RewriteRejected(
            "The revision did not materially reduce its trigger"
        )

The function extracts medium- and high-severity signals before and after revision. Any new signal code raises a RewriteRejected exception. Furthermore, at least one original warning must disappear or decrease by 25%, which prevents low-value edits from overwriting local files.

Once the validation rules are established, the engine executes the actual text rewriting logic. In the next section, we examine the service implementation that orchestrates contextual rewrites and static cleanups.

5. Local revision harness

The harness accepts the draft alongside target audience profiles, writing objectives, brand tone guidelines, and source material. This ensures that the engine evaluates stylistic anomalies against the specific genre and context of the document before suggesting changes.

Let's look at the code to see how the harness routes edits through two paths: static cleanup and contextual rewriting. Simple issues like extra whitespaces or comma adjustments are resolved instantly using fixed, rule-based functions. The system calls the LLM only when contextual rewriting is necessary, and all resulting drafts must pass a quality gate before proceeding to the save boundary.

refinement_service.py
def prepare_revision(draft, *, context, profile, services, source_context=""):
    before_report = evaluate_text(draft, context, profile)
    before_signals = diagnose_response(draft, language=context.language)
    example = local_example(context, source_context)
    semantic_before, _ = services.semantic_reviewer.inspect(example, draft)
    decision = select_revision_operation(draft, before_signals, semantic_before)

    if decision.operation is RevisionOperation.CONTEXTUAL_REWRITE:
        rewrite = services.contextual_rewriter.rewrite(
            draft, before_report, context, profile, before_signals, semantic_before
        )
    elif decision.operation is RevisionOperation.EXACT_BOUNDARY_CLEANUP:
        rewrite = apply_exact_boundary_cleanup(draft)
    else:
        rewrite = RewriteResult(revised_text=draft, applied_changes=[])

    validation = validate_rewrite(
        draft,
        rewrite.revised_text,
        profile,
        authoritative_source=source_context or None,
    )
    if not validation.safe:
        raise RewriteRejected("Quality gate rejected revision")

The system runs the model only for CONTEXTUAL_REWRITE tasks, relies on static regex replacements for EXACT_BOUNDARY_CLEANUP, and bypasses changes entirely when the draft satisfies all constraints. All edits are validated in validate_rewrite to confirm that facts, numbers, and links are kept intact.

Even when the harness prepares a safe revision, writing it to disk still requires separate permission controls. Let us examine how the Antigravity framework sandbox controls the agent's tool permissions during the save stage.

6. Governing file writes

Let's look at the Antigravity integration that controls agent tool permissions.

We integrated an agent execution framework^SDK that manages tool permissions and execution policies. The architectural details and specifications can be reviewed at the following link:

Google Antigravity

Google Antigravity - Build the new way

https://antigravity.google/

We restricted the agent's capabilities to 3 tools: prepare_candidate, save_candidate, and finish.

The reason we chose and tested with Antigravity is clear. LLM models themselves have no native awareness of workspace security or file system boundaries. If a model encounters a prompt injection attack or suffers a hallucination, it might attempt to execute arbitrary commands or corrupt files. Antigravity provides a sandbox that separates tool definitions from execution policies, allowing us to enforce a strict deny-by-default runtime. This ensures that the workspace remains secure even if the model behaves unexpectedly, making it the ideal security guardrail for our agent.

Let's look at the agent runtime configuration.

agent_runtime.py
config = LocalAgentConfig(
    model=model,
    api_key=api_key,
    response_schema=AgentFinal,
    tools=[prepare_candidate, save_candidate],
    capabilities=types.CapabilitiesConfig(
        enable_subagents=False,
        enabled_tools=[types.BuiltinTools.FINISH],
    ),
    policies=[
        policy.deny_all(),
        policy.allow("prepare_candidate"),
        policy.ask_user(
            "save_candidate",
            handler=approval_handler(
                auto_approve=auto_approve, slot=slot, audit_path=audit_path
            ),
        ),
        policy.allow(types.BuiltinTools.FINISH.value),
    ],
    hooks=[restrict_save, audit_tool, audit_turn, sanitize_tool_error],
)

The policy.deny_all() hook strips all tool access. The policy then registers prepare_candidate for sandbox use, while save_candidate is gated by the ask_user hook. The finish tool is the only built-in capability allowed, which prevents the agent from executing arbitrary system actions.

To ensure that the approved candidate matches the file written to disk, CandidateSlot generates a SHA-256 hash. The save function verifies this hash and checks the target directory boundaries to block path-traversal write attempts. Let's look at the code.

agent_state.py
def save(self, candidate_id: str, filename: str) -> dict[str, str]:
    if candidate_id != self.candidate_id:
        raise ValueError("The requested candidate is not the validated candidate.")
    if safe_filename(filename) != self._output_path.name:
        raise ValueError("The filename differs from the approved filename.")

    target = (self._output_path.parent / filename).resolve()
    if target.parent != self._output_path.parent:
        raise ValueError("Resolved output path escapes the approved directory.")

    target.write_text(
        self.candidate.rewrite.revised_text.strip() + "\n", encoding="utf-8"
    )
    write_audit(
        self._audit_path,
        "candidate_saved",
        candidate_sha256=self.candidate_id,
        filename=filename,
    )

By verifying the hash, target path, and filename, the save function blocks unauthorized writes and keeps the file system safe.

With the entire loop of evaluation, refinement, validation, and saving governed, the next question is narrower: what did the harness actually change, and does the current run justify a quality claim?

7. Benchmark results

The frozen run contains 12 English prompts from 1 pinned No Robots revision. 2 incomplete prompt groups were excluded before analysis, leaving 10 matched triads: the dataset reference, the direct model response, and the harness result for the same prompt. The reference is a comparison arm, not a numerical definition of naturalness. No blinded human-preference ratings were collected for this run, so I do not report a headline naturalness score or a radar chart that would make uncalibrated features look like validated qualities.

7.1 Did the harness revise the text?

This comes before quality interpretation. Normalized token edit distance and character-trigram Jensen–Shannon distance both equal 0 when the baseline and harness texts are identical. Only 2 of the 10 matched prompts changed; the other 8 are explicitly marked identical rather than being left as unexplained empty bars.

This is an abstention result as much as an editing result. It shows that the pipeline did not rewrite most candidates, but it also means that an aggregate comparison cannot establish a broad harness effect: 80% of the harness arm simply repeats the baseline.

7.2 What changed in the observed distributions?

The next chart reports descriptive means, not scores. The matched dataset references average 89.6 words, the baseline 178.2, and the harness 154.1. Mean lexical diversity is 0.775, 0.757, and 0.768 respectively. Sentence-length coefficient of variation is 0.400, 0.444, and 0.377. These quantities describe length, vocabulary reuse, and sentence-length variation; none has a universal “more natural” direction.

The paired forest plot visualizes the shifts for each prompt between the baseline and revised outputs. The chart elements can be interpreted as follows:

• Dot: Represents the amount of change in descriptive features (e.g., word count or sentence variation) from the baseline to the revised text.
• Horizontal Bar: Represents the statistical uncertainty range based on the sample variance. If the bar crosses the center line (0), the difference before and after the revision is not statistically significant.
• Sign (+/-): The direction of the change does not indicate quality. For instance, a decrease (-) in word count could mean eliminating redundant fluff (positive) or losing crucial details (negative).

7.3 What can a style representation tell us?

The earlier version of this article plotted individual words from a term-document matrix in 2 dimensions and inferred “human vocabulary richness” from wider scatter. That inference was not warranted. Point spread in an LSA[1] or t-SNE[2]-style projection depends on token frequency, projection choices, and local geometry; it is not a calibrated naturalness measure.

The replacement uses the frozen StyleDistance/mstyledistance encoder to produce 1 768-dimensional style representation per complete response. The left panel subtracts each prompt mean before applying principal component analysis^PCA, so topic shared within a prompt is partly controlled. It remains an exploratory 2D view: the first 2 components retain only 38.8% and 35.2% of the residual variance, and visual proximity can be distorted by projection.

The right panel therefore performs the consequential comparison in the original 768-dimensional space. For each prompt, it measures cosine distance from the baseline and harness responses to their matched dataset reference. The segment joins 2 measurements from the same prompt; it is not a generation trajectory. 1 harness result moved closer to the reference, 1 moved farther away, and 8 were unchanged.

The representation is useful for finding prompts that merit sentence-level review. It does not identify which wording is wrong by itself, and similarity to 1 reference is not equivalent to human preference. The present evidence verifies the run, exposes the limited revision coverage, and supplies diagnostic hypotheses. It does not demonstrate a systematic naturalness improvement. That claim requires more genres and languages, calibrated annotations, and blinded pairwise judgments.

8. Conclusion

To design a reliable writing agent, we needed to separate evaluation, revision, validation, and write authorization into distinct layers. Each layer's responsibilities must be clear to protect the workspace from corrupting files, and to allow failures to be fed back as feedback for the next iteration.

My takeaway is that fuzzy evaluation criteria make a refinement loop difficult to calibrate and easy to overstate. Deterministic checks remain useful for preservation and obvious regressions, while representation models and bounded semantic inspection can identify passages that need review. Neither should be collapsed into an invented all-purpose score.

The architecture is therefore the result I can defend today. The quality claim remains open until the same protocol is run across more genres and languages and judged through blinded human preference.

• Separation of Concerns: Kept evaluation, refinement, validation, and authorized saving as distinct stages with auditable outputs.
• Matched Evidence: Compared 10 prompt-matched triads and reported that only two harness outputs changed, without converting descriptive features into a quality score.
• Release Criterion: Treats frozen representations as diagnostics and reserves the improvement claim for multilingual, blinded human preference.