SOFTWARE DESIGN · JEV
EVALUATION · TYPESAFE JEV · THE PLAIN ENGLISH TEST

Jev names the missing data structure 98% of the time, and its 0–4 match score separates clean code from smelly by 1.29 points

Say in one sentence how the code works; every noun in that sentence should be a type. Where a concept is carried as loose parts instead, there is a data structure waiting to break out. Jev finds which shape it takes — and it does not need the sentence: withholding it costs nothing (98% blind vs 98% with the sentence given).

jev-1.13.0 · 160 REQUESTS · 40 SNIPPETS · 2 REFERENCE ARMS · 1 CHOICE + 4 NOUL + 1 SCORE PER REQUEST · 2026-09-17
5-WAY · NO REFERENCE
97.5%
SENTENCE 98 · BLIND 98
5-WAY · WITH REFERENCE
100.0%
SENTENCE 100 · BLIND 100
HARD TIER · NEAR MISSES
90.0%
WITH REFERENCE 100 · N 20
MATCH SCORE SEPARATION
1.29OF 4
CLEAN 3.30 · SMELLY 2.00
REQUESTS
160
40 CASES · 2 CONDITIONS · 2 ARMS
COST
0.0141USD
310K TOK · P50 281 MS
WHAT WAS TESTEDTHE TEST · THE CORPUS · THE AXES

The test. State in plain English how the thing works. Then read that sentence beside the code. Every concept the sentence names should exist as a type; every type should be a concept in the sentence. Where the sentence says "an address" and the code carries five strings that always move together, the address is a data structure waiting to break out. The failure is not ugliness — it is that the reader has to rebuild the concept from parts, at every call site, forever.

The corpus. 40 TypeScript snippets, each with a context line, the authored sentence, and the code. 22 are clean and 18 carry exactly one smell of a known shape. Tiers: 17 obvious, 13 subtle, and 10 hard — near-misses written in both directions, clean code that looks smelly and smelly code that looks clean. The hard tier exists because the first 30 cases scored 100% in both arms and therefore measured nothing.

The axes. sentence supplies the plain-English sentence in the state; blind withholds it, so the model must write the sentence itself before it can apply the test. none defines the shapes only in the question criteria; article adds a 439-word condensed statement of the test as a plain_english_reference state field.

ClassShapeWhat it meansN per armMatch · noneMatch · ref
no_structurecleanEvery concept the sentence names is already a type.223.303.42
tandem_paramstandem parametersTwo or more values always travel together, in the same order, from the same source.162.431.14
branch_tablebranch tableA conditional whose branches assign the same fields to different constants — the rows of a table.142.230.65
parallel_collectionsparallel collectionsTwo collections share an index or key and must be changed in lockstep.141.870.60
stringly_encodedstringly encodedA record packed into a string, filename or bitfield and re-parsed by every reader.141.420.44

The last two columns are the mean 0–4 match score per truth class. It falls monotonically from clean to the worst shape in both arms, which is what a usable severity signal looks like.

ONE REQUEST, VERBATIMdisplay_type.blind · 275 MS · 1625 TOK IN

This is a quiz question, asked the bench's way. Question 3 of the Mirdin quiz shows a DisplayType carried as a string plus a pair of numbers, and asks what to do about it. Here the same code arrives with no options and no sentence — the blind condition — and the model is asked only which shape of missing structure it sees.

Jev answers branch_table at 1.00 and scores the type/sentence match 2 of 4. On the quiz's own four-option form, in the tab to the right, it gets this same question wrong in all 40 runs — it picks the enum-of-constants answer over the algebraic one.

REQUEST · POST api.typesafe.ai/v1/systemoneAUTHORIZATION: BEARER …
{
  "state": {
    "context": "Product page on an ecommerce website",
    "code": "public setProductDisplayType(displayType: string): void {\n    if (displayType === \"compact\") {\n        this.maxDescriptionLength = 140;\n        this.numSimilarProductsToShow = 2;\n        this.numReviewsToShow = 1;\n    } else if (displayType === \"detailed\") {\n        this.maxDescriptionLength = 500;\n        this.numSimilarProductsToShow = 5;\n        this.numReviewsToShow = 7;\n    }\n}",
    "notes": "`code` is representative, not complete. Judge the shape of the data, not missing error handling, naming style or tests."
  },
  "model": "jev-latest",
  "questions": {
    "verdict": {
      "type": "choice",
      "instructions": "Is there a data structure waiting to break out of this code? Say in one sentence how `code` works — `plain_english_sentence` gives that sentence when it is present — then check whether every concept the sentence names is a type in the code. Where a concept the sentence needs is carried as loose parts, that concept is a data structure waiting to break out. Pick the shape it takes, or `no_structure` if the code already carries every concept its sentence names. Judge the shape of the data only; ignore naming, error handling, performance and missing tests.",
      "criteria": {
        "no_structure": {
          "what": "The types in the code already match the sentence: every concept the sentence names exists as a type, and no group of values is being carried as loose parts.",
          "not_for": "Code that is merely short or tidy; check each value that travels with another."
        },
        "tandem_params": {
          "what": "Two or more values always travel together as separate parameters or separate fields, in the same order, from the same source.",
          "not_for": "Parameters that genuinely vary independently between call sites.",
          "examples": [
            "A constructor taking x, y, width, height instead of a point and a size",
            "Fields repeated with a prefix: shipCity, shipPostcode / billCity, billPostcode"
          ]
        },
        "branch_table": {
          "what": "A conditional whose branches each assign the same set of fields to different constants. The branches are rows of a table and the tuple of constants is the missing type.",
          "not_for": "A conditional whose branches do genuinely different work rather than assigning the same fields.",
          "examples": [
            "if plan === 'free' set three limits, else if 'team' set the same three limits",
            "A switch on severity setting colour, icon and repage delay"
          ]
        },
        "parallel_collections": {
          "what": "Two or more collections share an index or a key and must be inserted, removed and reordered in lockstep. The row is the missing type.",
          "not_for": "A single collection of records, or two collections that are genuinely unrelated.",
          "examples": [
            "names[], emails[] and scores[] spliced together at the same index",
            "Map<UserId, string> beside Map<UserId, Role> written and deleted together"
          ]
        },
        "stringly_encoded": {
          "what": "A record is packed into a string, a filename, a delimited column or a bitfield and taken apart again by every reader with its own parsing.",
          "not_for": "A string that really is one opaque value, such as free text or an id.",
          "examples": [
            "status = 'paid|2026-01-03|card' read with split('|') in three functions",
            "A version string re-split by every function that compares versions"
          ]
        }
      }
    },
    "tandem_params": {
      "type": "noul",
      "instructions": "About the data in `code`: Do two or more values always appear together, in the same order, from the same source?",
      "criteria": {
        "true": "Yes, clearly: values always travel together as separate parameters or fields.",
        "false": "No: values always travel together as separate parameters or fields does not describe this code."
      }
    },
    "branch_table": {
      "type": "noul",
      "instructions": "About the data in `code`: Does a conditional assign the same set of fields to different constants in each branch?",
      "criteria": {
        "true": "Yes, clearly: a conditional assigns a tuple of constants per branch.",
        "false": "No: a conditional assigns a tuple of constants per branch does not describe this code."
      }
    },
    "parallel_collections": {
      "type": "noul",
      "instructions": "About the data in `code`: Do two or more collections share an index or key and have to be changed together?",
      "criteria": {
        "true": "Yes, clearly: collections are indexed or keyed in lockstep.",
        "false": "No: collections are indexed or keyed in lockstep does not describe this code."
      }
    },
    "stringly_encoded": {
      "type": "noul",
      "instructions": "About the data in `code`: Is a multi-field record encoded into one string, filename or number and re-parsed by its readers?",
      "criteria": {
        "true": "Yes, clearly: a record is packed into a string, filename or bitfield.",
        "false": "No: a record is packed into a string, filename or bitfield does not describe this code."
      }
    },
    "plain_english_match": {
      "type": "score",
      "instructions": "How well do the types in `code` match a one-sentence plain-English description of how it works? Compare the concepts the sentence needs with the types the code actually has.",
      "criteria": [
        "The reader must rebuild every concept from loose parts: values travel in groups, records live inside strings, collections share an index.",
        "One concept the sentence names exists as a type; the rest are carried as loose parts.",
        "The main concept exists as a type, but several of its parts are still passed or stored separately.",
        "Nearly every concept the sentence names is a type; one small group of values is still loose.",
        "Every concept the sentence names is a type in the code, and every type is a concept in the sentence."
      ]
    }
  }
}
RESPONSE · HTTP 200jev-1.13.0
{
  "model": "jev-1.13.0",
  "answers": {
    "verdict": {
      "type": "choice",
      "choice": "branch_table",
      "confidence": 0.99,
      "probabilities": {
        "branch_table": 1.0,
        "stringly_encoded": 0.0,
        "tandem_params": 0.0,
        "parallel_collections": 0.0,
        "no_structure": 0.0
      }
    },
    "tandem_params": {
      "type": "noul",
      "noul": 0.6
    },
    "branch_table": {
      "type": "noul",
      "noul": 0.89
    },
    "parallel_collections": {
      "type": "noul",
      "noul": 0.17
    },
    "stringly_encoded": {
      "type": "noul",
      "noul": 0.09
    },
    "plain_english_match": {
      "type": "score",
      "score": 1.9,
      "confidence": 0.46,
      "legend": {
        "0": "The reader must rebuild every concept from loose parts: values travel in groups, records live inside strings, collections share an index.",
        "1": "One concept the sentence names exists as a type; the rest are carried as loose parts.",
        "2": "The main concept exists as a type, but several of its parts are still passed or stored separately.",
        "3": "Nearly every concept the sentence names is a type; one small group of values is still loose.",
        "4": "Every concept the sentence names is a type in the code, and every type is a concept in the sentence."
      },
      "probabilities": {
        "0": 0.07,
        "1": 0.24,
        "2": 0.54,
        "3": 0.04,
        "4": 0.11
      }
    }
  },
  "usage": {
    "input_tokens": 1625,
    "output_tokens": 156
  }
}

The score primitive takes its criteria as an ordered array of level descriptions — position in the array is the level. It is not a min/max range, and sending one returns HTTP 422.

WITH VS WITHOUT THE TEST IN THE STATE80 PAIRED REQUESTS PER ARM · Δ = REFERENCE − NONE
MetricNoneReferenceΔ
5-way · all98100+3
5-way · sentence given98100+3
5-way · sentence withheld98100+3
5-way · excluding the two quiz cases97100+3
smell found · recall98100+2
clean kept · specificity95100+5
hard tier90100+10
match · clean mean3.303.42+0.12
match · smelly mean2.000.72−1.28
match · separation1.292.69+1.40
match · rank AUC0.9441.000+0.056
ECE0.0940.079−0.015
input tokens per request16022278+676
latency p50 ms290277−13

Paired by case: both right 78, right only without the reference 0, right only with it 2, both wrong 0.

The classification gain is two cases out of 80 — small, and both in the hard tier. The real change is the match score: smelly code drops from 2.00 to 0.72 while clean code stays put, so the rank AUC goes to 1.00. Reading the rule makes the model harsher about loose parts without making it suspicious of clean code. That is the opposite of what the same technique did on the MIRO tab, where the reference only moved the bias.

STATE.PLAIN_ENGLISH_REFERENCE · SENT VERBATIM IN THE REFERENCE ARM439 WORDS
THE PLAIN ENGLISH TEST, AND DATA STRUCTURES WAITING TO BREAK OUT.

Say in one sentence how the code works. Then read the sentence and the code side by side.
Every noun in the sentence should be a type in the code, and every type in the code should
be a noun in the sentence. Where the sentence names a concept the code does not have, that
concept is a data structure waiting to break out. This is the Embedded Design Principle:
design decisions should be visible in the code, not reconstructed by the reader.

Asked what data defines a rectangle, almost nobody says four integers. They say a corner and
a size. A constructor taking four numbers therefore fails the test twice: it does not match
the sentence, and it allows 24 argument orderings of which 23 are wrong.

FOUR SHAPES THE MISSING STRUCTURE TAKES

1. VALUES THAT TRAVEL TOGETHER. The same two, three or five parameters appear in signature
   after signature, always in the same order, always from the same source. Fields with a
   shared prefix (shipLine1, shipCity, shipCountry / billLine1, billCity, billCountry) are
   the same thing with the type spelled into the names. The fix is one type holding them.

2. A CONDITIONAL THAT ASSIGNS A TUPLE OF CONSTANTS. Each branch writes the same set of
   fields with different constants. The branches are rows of a table and the tuple is the
   type. Replacing the strings with an enum, or the if-chain with a switch, renames the
   conditional; passing a record containing the tuple removes it. A default branch that
   assigns real-looking values is the sign that the missing type would have made a gap
   explicit.

3. COLLECTIONS INDEXED IN LOCKSTEP. Two arrays with a shared index, or two maps with a
   shared key, written together, deleted together and read together. Any operation that
   reorders or removes must be performed once per collection, and the day one is forgotten
   the data is silently wrong. The row is the type.

4. A RECORD PACKED INTO A STRING, A FILENAME OR A BITFIELD. "paid|2026-01-03|card",
   "user:42:posts:page3", IMG_20260102_083000_lisbon.jpg, flags |= EXPORT. Every reader
   splits it again with its own parsing, every writer rebuilds it by hand, and the format is
   defined by whichever function last touched it. Parse once at the edge into a real type.

WHAT IS NOT A MISSING STRUCTURE
   Code whose types already match its sentence, even when it is short, mutable, or could be
   faster. A discriminated union with one branch per case in the sentence is the goal, not a
   smell. Judge the shape of the data, not the naming, the error handling or the tests.
THE MATCH SCORE0–4 · HOW WELL THE TYPES MATCH THE SENTENCE
NO REFERENCE · AUC 0.94
WITH REFERENCE · AUC 1.00

Distribution of the 0–4 score, clean cases against smelly ones. With the reference in the state the two stop overlapping entirely: every clean snippet outranks every smelly one.

A single ordered number is more useful in review than a class name — it sorts a diff. But it is scored on the same 40 snippets it was tuned against, and the levels were written by the same hand that wrote the cases.

WHERE VERDICTS LANDROWS = TRUTH · COLS = JEV · SHADE = SHARE OF ROW
NO REFERENCE
WITH REFERENCE

1 clean snippet called smelly and 1 smelly snippet called clean, out of 80 — and both are hard-tier near-misses. With the reference, neither happens.

PER CLASS · SENTENCE GIVENNO REFERENCE · N = 40
TruthNPrec %Recall %F1State
no_structure11921000.96ok
tandem_params8100880.93ok
branch_table71001001.00ok
parallel_collections71001001.00ok
stringly_encoded71001001.00ok
PER CLASS · SENTENCE WITHHELDNO REFERENCE · N = 40
TruthNPrec %Recall %F1State
no_structure11100910.95ok
tandem_params8891000.94ok
branch_table71001001.00ok
parallel_collections71001001.00ok
stringly_encoded71001001.00ok
NOUL DETECTORSNO REFERENCE · ONE PER SHAPE
DetectorAUCBest cutPrec %Recall %State
tandem_params0.790.755750warn
branch_table1.000.83100100ok
parallel_collections1.000.96100100ok
stringly_encoded1.000.7888100ok

The standalone yes/no probabilities are much weaker than the choice question that carries all five options at once. Asking "is this tandem parameters?" in isolation loses the comparison that makes the answer obvious.

CALIBRATIONNO REFERENCE · ECE 0.094
TierNNo referenceWith reference
obvious34
100%
100%
subtle26
100%
100%
hard20
90%
100%

At 97.5% accuracy the calibration curve has almost nothing to separate — nearly every bin is right. ECE here measures under-confidence, not error.

EVERY SNIPPET4 REQUESTS EACH · 2 CONDITIONS × 2 ARMS
CaseThe sentenceTruthTierNo referenceWith referenceMatchMatch · ref
h_dto_tandemAn event happens over an interval of time in a named timezone.tandem_paramshard
50%
100%
2.851.90
h_independent_paramsA template is rendered for a locale, in a timezone, with a theme.no_structurehard
50%
100%
3.252.83
version_stringA version is a major, a minor, a patch and an optional pre-release tag.stringly_encodedobvious
100%
100%
0.690.13
pipe_statusA payment has a state, a date and a method.stringly_encodedobvious
100%
100%
0.790.12
three_arraysEach student has a name, an email and a score.parallel_collectionsobvious
100%
100%
0.990.11
csv_permissionsA member holds a set of permissions.stringly_encodedsubtle
100%
100%
1.090.45
socket_arraysEach connected player has a socket, a last-seen time and a score.parallel_collectionssubtle
100%
100%
1.290.21
h_typed_wrapper_splitA cache entry is identified by a user, a resource and a page.stringly_encodedhard
100%
100%
1.310.47
table_columnsA column has a name, a type and a width.parallel_collectionsobvious
100%
100%
1.410.34
cache_key_stringA cache entry is identified by a user, a resource and a page.stringly_encodedobvious
100%
100%
1.420.33
smtp_five_argsSending mail needs a server to connect to and an account to connect as.tandem_paramsobvious
100%
100%
1.560.27
device_gridEach device class gets a grid width, a column count and a thumbnail size.branch_tablesubtle
100%
100%
1.580.35
graph_weightsAn edge goes from one node to another and has a cost.parallel_collectionssubtle
100%
100%
1.730.24
display_typeA display type sets how much of a product to show. · from the quizbranch_tableobvious
100%
100%
1.790.57
draw_text_themeLabels are drawn at a position in the chart's label style.tandem_paramssubtle
100%
100%
1.830.61
schedule_threeA report runs at a time of day in a given timezone.tandem_paramsobvious
100%
100%
1.870.43
filename_metadataAn imported photo knows when and where it was taken.stringly_encodedsubtle
100%
100%
1.870.18
severity_styleEach severity has a colour, an icon and how long to wait before paging again.branch_tableobvious
100%
100%
2.060.49
country_taxEach country has a VAT rate, a currency and a legal footer.branch_tablesubtle
100%
100%
2.110.32
role_booleansWhat a member may do depends on their role.branch_tablesubtle
100%
100%
2.250.72
two_mapsEach connected user has a display name and a role.parallel_collectionsobvious
100%
100%
2.340.62
h_builder_tandemAn axis is drawn over a range of values with a tick interval.tandem_paramshard
100%
100%
2.561.82
h_record_mapsA plan comes with a seat limit, a storage allowance and a support channel.parallel_collectionshard
100%
100%
2.650.77
address_fieldsAn order ships to an address.tandem_paramssubtle
100%
100%
2.671.18
chart_seriesA series is a list of labelled points.parallel_collectionssubtle
100%
100%
2.671.92
bitfield_flagsA user has some set of features switched on.stringly_encodedsubtle
100%
100%
2.751.39
plan_limitsA plan comes with a seat limit, a storage allowance and a support channel.branch_tableobvious
100%
100%
2.850.56
h_wrapped_flagsA user has a set of features switched on.no_structurehard
100%
100%
2.903.25
rect_four_intsA rectangle is a corner and a size. · from the quiztandem_paramsobvious
100%
100%
2.950.98
h_two_level_branchHow a notification is delivered depends on its channel and its urgency.branch_tablehard
100%
100%
2.951.54
latlng_fourA leg of a journey goes from one place to another.tandem_paramsobvious
100%
100%
3.171.94
ok_retry_policyA job waits for the delay its retry policy gives, and stops when the policy says stop.no_structureobvious
100%
100%
3.213.24
ok_date_rangeA booking covers a range of dates and two bookings clash when their ranges overlap.no_structureobvious
100%
100%
3.213.58
h_opaque_tokenA request carries a session token, which the auth service either accepts or rejects.no_structurehard
100%
100%
3.233.35
h_real_dispatchEvaluating an expression means doing the thing that kind of expression means.no_structurehard
100%
100%
3.273.68
ok_parse_tokenA token is a word, a number or a symbol, and each carries where it was found.no_structureobvious
100%
100%
3.293.66
ok_parsed_signupA signup request is parsed into a valid signup, or rejected with the fields that failed.no_structuresubtle
100%
100%
3.313.59
ok_money_plusAdding two amounts of the same currency gives an amount of that currency.no_structureobvious
100%
100%
3.483.75
h_unrelated_mapsThe gateway remembers which users are rate limited and which have beta access.no_structurehard
100%
100%
3.483.00
ok_sum_linesThe total of an invoice is the sum of its line amounts.no_structuresubtle
100%
100%
3.633.67
EVERY SAMPLECLICK A ROW FOR THE CODE, THE AUTHORED REASON AND THE FULL DISTRIBUTION
IDTruthVerdictConfMatchRefCondTier
CAVEATSREAD BEFORE QUOTING A NUMBER
  • 97.5% and 100.0% are near the ceiling, and a ceiling measures the corpus, not the model. The first 30 cases scored 100% in both arms; the hard tier was added precisely because of that, and it is the only tier where anything is still being measured.
  • Every snippet carries exactly one smell of a known shape, and the five shapes are named in the question. Real code has several at once, or one that fits no name on the list.
  • The cases were written by Claude with the shape chosen first. The authored sentence is the one that makes the smell visible; a different true sentence about the same code might not.
  • The match score's five levels and the 40 snippets come from the same author, so the AUC of 1.00 in the reference arm is an upper bound on a friendly distribution.
  • One prompt design, one run per arm. Repeats are not needed: on the quiz tab the same request returned the same choice in 10 of 10 runs, 20 times out of 20.