Say in one sentence how the code works; every noun in that sentence should be a type. Where a concept is carried as loose parts instead, there is a data structure waiting to break out. Jev finds which shape it takes — and it does not need the sentence: withholding it costs nothing (98% blind vs 98% with the sentence given).
The test. State in plain English how the thing works. Then read that sentence beside the code. Every concept the sentence names should exist as a type; every type should be a concept in the sentence. Where the sentence says "an address" and the code carries five strings that always move together, the address is a data structure waiting to break out. The failure is not ugliness — it is that the reader has to rebuild the concept from parts, at every call site, forever.
The corpus. 40 TypeScript snippets, each with a context line, the authored sentence, and the code. 22 are clean and 18 carry exactly one smell of a known shape. Tiers: 17 obvious, 13 subtle, and 10 hard — near-misses written in both directions, clean code that looks smelly and smelly code that looks clean. The hard tier exists because the first 30 cases scored 100% in both arms and therefore measured nothing.
The axes. sentence supplies the plain-English sentence in the state; blind withholds it, so the model must write the sentence itself before it can apply the test. none defines the shapes only in the question criteria; article adds a 439-word condensed statement of the test as a plain_english_reference state field.
| Class | Shape | What it means | N per arm | Match · none | Match · ref |
|---|---|---|---|---|---|
| no_structure | clean | Every concept the sentence names is already a type. | 22 | 3.30 | 3.42 |
| tandem_params | tandem parameters | Two or more values always travel together, in the same order, from the same source. | 16 | 2.43 | 1.14 |
| branch_table | branch table | A conditional whose branches assign the same fields to different constants — the rows of a table. | 14 | 2.23 | 0.65 |
| parallel_collections | parallel collections | Two collections share an index or key and must be changed in lockstep. | 14 | 1.87 | 0.60 |
| stringly_encoded | stringly encoded | A record packed into a string, filename or bitfield and re-parsed by every reader. | 14 | 1.42 | 0.44 |
The last two columns are the mean 0–4 match score per truth class. It falls monotonically from clean to the worst shape in both arms, which is what a usable severity signal looks like.
This is a quiz question, asked the bench's way. Question 3 of the Mirdin quiz shows a DisplayType carried as a string plus a pair of numbers, and asks what to do about it. Here the same code arrives with no options and no sentence — the blind condition — and the model is asked only which shape of missing structure it sees.
Jev answers branch_table at 1.00 and scores the type/sentence match 2 of 4. On the quiz's own four-option form, in the tab to the right, it gets this same question wrong in all 40 runs — it picks the enum-of-constants answer over the algebraic one.
{
"state": {
"context": "Product page on an ecommerce website",
"code": "public setProductDisplayType(displayType: string): void {\n if (displayType === \"compact\") {\n this.maxDescriptionLength = 140;\n this.numSimilarProductsToShow = 2;\n this.numReviewsToShow = 1;\n } else if (displayType === \"detailed\") {\n this.maxDescriptionLength = 500;\n this.numSimilarProductsToShow = 5;\n this.numReviewsToShow = 7;\n }\n}",
"notes": "`code` is representative, not complete. Judge the shape of the data, not missing error handling, naming style or tests."
},
"model": "jev-latest",
"questions": {
"verdict": {
"type": "choice",
"instructions": "Is there a data structure waiting to break out of this code? Say in one sentence how `code` works — `plain_english_sentence` gives that sentence when it is present — then check whether every concept the sentence names is a type in the code. Where a concept the sentence needs is carried as loose parts, that concept is a data structure waiting to break out. Pick the shape it takes, or `no_structure` if the code already carries every concept its sentence names. Judge the shape of the data only; ignore naming, error handling, performance and missing tests.",
"criteria": {
"no_structure": {
"what": "The types in the code already match the sentence: every concept the sentence names exists as a type, and no group of values is being carried as loose parts.",
"not_for": "Code that is merely short or tidy; check each value that travels with another."
},
"tandem_params": {
"what": "Two or more values always travel together as separate parameters or separate fields, in the same order, from the same source.",
"not_for": "Parameters that genuinely vary independently between call sites.",
"examples": [
"A constructor taking x, y, width, height instead of a point and a size",
"Fields repeated with a prefix: shipCity, shipPostcode / billCity, billPostcode"
]
},
"branch_table": {
"what": "A conditional whose branches each assign the same set of fields to different constants. The branches are rows of a table and the tuple of constants is the missing type.",
"not_for": "A conditional whose branches do genuinely different work rather than assigning the same fields.",
"examples": [
"if plan === 'free' set three limits, else if 'team' set the same three limits",
"A switch on severity setting colour, icon and repage delay"
]
},
"parallel_collections": {
"what": "Two or more collections share an index or a key and must be inserted, removed and reordered in lockstep. The row is the missing type.",
"not_for": "A single collection of records, or two collections that are genuinely unrelated.",
"examples": [
"names[], emails[] and scores[] spliced together at the same index",
"Map<UserId, string> beside Map<UserId, Role> written and deleted together"
]
},
"stringly_encoded": {
"what": "A record is packed into a string, a filename, a delimited column or a bitfield and taken apart again by every reader with its own parsing.",
"not_for": "A string that really is one opaque value, such as free text or an id.",
"examples": [
"status = 'paid|2026-01-03|card' read with split('|') in three functions",
"A version string re-split by every function that compares versions"
]
}
}
},
"tandem_params": {
"type": "noul",
"instructions": "About the data in `code`: Do two or more values always appear together, in the same order, from the same source?",
"criteria": {
"true": "Yes, clearly: values always travel together as separate parameters or fields.",
"false": "No: values always travel together as separate parameters or fields does not describe this code."
}
},
"branch_table": {
"type": "noul",
"instructions": "About the data in `code`: Does a conditional assign the same set of fields to different constants in each branch?",
"criteria": {
"true": "Yes, clearly: a conditional assigns a tuple of constants per branch.",
"false": "No: a conditional assigns a tuple of constants per branch does not describe this code."
}
},
"parallel_collections": {
"type": "noul",
"instructions": "About the data in `code`: Do two or more collections share an index or key and have to be changed together?",
"criteria": {
"true": "Yes, clearly: collections are indexed or keyed in lockstep.",
"false": "No: collections are indexed or keyed in lockstep does not describe this code."
}
},
"stringly_encoded": {
"type": "noul",
"instructions": "About the data in `code`: Is a multi-field record encoded into one string, filename or number and re-parsed by its readers?",
"criteria": {
"true": "Yes, clearly: a record is packed into a string, filename or bitfield.",
"false": "No: a record is packed into a string, filename or bitfield does not describe this code."
}
},
"plain_english_match": {
"type": "score",
"instructions": "How well do the types in `code` match a one-sentence plain-English description of how it works? Compare the concepts the sentence needs with the types the code actually has.",
"criteria": [
"The reader must rebuild every concept from loose parts: values travel in groups, records live inside strings, collections share an index.",
"One concept the sentence names exists as a type; the rest are carried as loose parts.",
"The main concept exists as a type, but several of its parts are still passed or stored separately.",
"Nearly every concept the sentence names is a type; one small group of values is still loose.",
"Every concept the sentence names is a type in the code, and every type is a concept in the sentence."
]
}
}
}{
"model": "jev-1.13.0",
"answers": {
"verdict": {
"type": "choice",
"choice": "branch_table",
"confidence": 0.99,
"probabilities": {
"branch_table": 1.0,
"stringly_encoded": 0.0,
"tandem_params": 0.0,
"parallel_collections": 0.0,
"no_structure": 0.0
}
},
"tandem_params": {
"type": "noul",
"noul": 0.6
},
"branch_table": {
"type": "noul",
"noul": 0.89
},
"parallel_collections": {
"type": "noul",
"noul": 0.17
},
"stringly_encoded": {
"type": "noul",
"noul": 0.09
},
"plain_english_match": {
"type": "score",
"score": 1.9,
"confidence": 0.46,
"legend": {
"0": "The reader must rebuild every concept from loose parts: values travel in groups, records live inside strings, collections share an index.",
"1": "One concept the sentence names exists as a type; the rest are carried as loose parts.",
"2": "The main concept exists as a type, but several of its parts are still passed or stored separately.",
"3": "Nearly every concept the sentence names is a type; one small group of values is still loose.",
"4": "Every concept the sentence names is a type in the code, and every type is a concept in the sentence."
},
"probabilities": {
"0": 0.07,
"1": 0.24,
"2": 0.54,
"3": 0.04,
"4": 0.11
}
}
},
"usage": {
"input_tokens": 1625,
"output_tokens": 156
}
}The score primitive takes its criteria as an ordered array of level descriptions — position in the array is the level. It is not a min/max range, and sending one returns HTTP 422.
| Metric | None | Reference | Δ |
|---|---|---|---|
| 5-way · all | 98 | 100 | +3 |
| 5-way · sentence given | 98 | 100 | +3 |
| 5-way · sentence withheld | 98 | 100 | +3 |
| 5-way · excluding the two quiz cases | 97 | 100 | +3 |
| smell found · recall | 98 | 100 | +2 |
| clean kept · specificity | 95 | 100 | +5 |
| hard tier | 90 | 100 | +10 |
| match · clean mean | 3.30 | 3.42 | +0.12 |
| match · smelly mean | 2.00 | 0.72 | −1.28 |
| match · separation | 1.29 | 2.69 | +1.40 |
| match · rank AUC | 0.944 | 1.000 | +0.056 |
| ECE | 0.094 | 0.079 | −0.015 |
| input tokens per request | 1602 | 2278 | +676 |
| latency p50 ms | 290 | 277 | −13 |
Paired by case: both right 78, right only without the reference 0, right only with it 2, both wrong 0.
The classification gain is two cases out of 80 — small, and both in the hard tier. The real change is the match score: smelly code drops from 2.00 to 0.72 while clean code stays put, so the rank AUC goes to 1.00. Reading the rule makes the model harsher about loose parts without making it suspicious of clean code. That is the opposite of what the same technique did on the MIRO tab, where the reference only moved the bias.
THE PLAIN ENGLISH TEST, AND DATA STRUCTURES WAITING TO BREAK OUT. Say in one sentence how the code works. Then read the sentence and the code side by side. Every noun in the sentence should be a type in the code, and every type in the code should be a noun in the sentence. Where the sentence names a concept the code does not have, that concept is a data structure waiting to break out. This is the Embedded Design Principle: design decisions should be visible in the code, not reconstructed by the reader. Asked what data defines a rectangle, almost nobody says four integers. They say a corner and a size. A constructor taking four numbers therefore fails the test twice: it does not match the sentence, and it allows 24 argument orderings of which 23 are wrong. FOUR SHAPES THE MISSING STRUCTURE TAKES 1. VALUES THAT TRAVEL TOGETHER. The same two, three or five parameters appear in signature after signature, always in the same order, always from the same source. Fields with a shared prefix (shipLine1, shipCity, shipCountry / billLine1, billCity, billCountry) are the same thing with the type spelled into the names. The fix is one type holding them. 2. A CONDITIONAL THAT ASSIGNS A TUPLE OF CONSTANTS. Each branch writes the same set of fields with different constants. The branches are rows of a table and the tuple is the type. Replacing the strings with an enum, or the if-chain with a switch, renames the conditional; passing a record containing the tuple removes it. A default branch that assigns real-looking values is the sign that the missing type would have made a gap explicit. 3. COLLECTIONS INDEXED IN LOCKSTEP. Two arrays with a shared index, or two maps with a shared key, written together, deleted together and read together. Any operation that reorders or removes must be performed once per collection, and the day one is forgotten the data is silently wrong. The row is the type. 4. A RECORD PACKED INTO A STRING, A FILENAME OR A BITFIELD. "paid|2026-01-03|card", "user:42:posts:page3", IMG_20260102_083000_lisbon.jpg, flags |= EXPORT. Every reader splits it again with its own parsing, every writer rebuilds it by hand, and the format is defined by whichever function last touched it. Parse once at the edge into a real type. WHAT IS NOT A MISSING STRUCTURE Code whose types already match its sentence, even when it is short, mutable, or could be faster. A discriminated union with one branch per case in the sentence is the goal, not a smell. Judge the shape of the data, not the naming, the error handling or the tests.
Distribution of the 0–4 score, clean cases against smelly ones. With the reference in the state the two stop overlapping entirely: every clean snippet outranks every smelly one.
A single ordered number is more useful in review than a class name — it sorts a diff. But it is scored on the same 40 snippets it was tuned against, and the levels were written by the same hand that wrote the cases.
1 clean snippet called smelly and 1 smelly snippet called clean, out of 80 — and both are hard-tier near-misses. With the reference, neither happens.
| Truth | N | Prec % | Recall % | F1 | State |
|---|---|---|---|---|---|
| no_structure | 11 | 92 | 100 | 0.96 | ok |
| tandem_params | 8 | 100 | 88 | 0.93 | ok |
| branch_table | 7 | 100 | 100 | 1.00 | ok |
| parallel_collections | 7 | 100 | 100 | 1.00 | ok |
| stringly_encoded | 7 | 100 | 100 | 1.00 | ok |
| Truth | N | Prec % | Recall % | F1 | State |
|---|---|---|---|---|---|
| no_structure | 11 | 100 | 91 | 0.95 | ok |
| tandem_params | 8 | 89 | 100 | 0.94 | ok |
| branch_table | 7 | 100 | 100 | 1.00 | ok |
| parallel_collections | 7 | 100 | 100 | 1.00 | ok |
| stringly_encoded | 7 | 100 | 100 | 1.00 | ok |
| Detector | AUC | Best cut | Prec % | Recall % | State |
|---|---|---|---|---|---|
| tandem_params | 0.79 | 0.75 | 57 | 50 | warn |
| branch_table | 1.00 | 0.83 | 100 | 100 | ok |
| parallel_collections | 1.00 | 0.96 | 100 | 100 | ok |
| stringly_encoded | 1.00 | 0.78 | 88 | 100 | ok |
The standalone yes/no probabilities are much weaker than the choice question that carries all five options at once. Asking "is this tandem parameters?" in isolation loses the comparison that makes the answer obvious.
| Tier | N | No reference | With reference |
|---|---|---|---|
| obvious | 34 | 100% | 100% |
| subtle | 26 | 100% | 100% |
| hard | 20 | 90% | 100% |
At 97.5% accuracy the calibration curve has almost nothing to separate — nearly every bin is right. ECE here measures under-confidence, not error.
| Case | The sentence | Truth | Tier | No reference | With reference | Match | Match · ref |
|---|---|---|---|---|---|---|---|
| h_dto_tandem | An event happens over an interval of time in a named timezone. | tandem_params | hard | 50% | 100% | 2.85 | 1.90 |
| h_independent_params | A template is rendered for a locale, in a timezone, with a theme. | no_structure | hard | 50% | 100% | 3.25 | 2.83 |
| version_string | A version is a major, a minor, a patch and an optional pre-release tag. | stringly_encoded | obvious | 100% | 100% | 0.69 | 0.13 |
| pipe_status | A payment has a state, a date and a method. | stringly_encoded | obvious | 100% | 100% | 0.79 | 0.12 |
| three_arrays | Each student has a name, an email and a score. | parallel_collections | obvious | 100% | 100% | 0.99 | 0.11 |
| csv_permissions | A member holds a set of permissions. | stringly_encoded | subtle | 100% | 100% | 1.09 | 0.45 |
| socket_arrays | Each connected player has a socket, a last-seen time and a score. | parallel_collections | subtle | 100% | 100% | 1.29 | 0.21 |
| h_typed_wrapper_split | A cache entry is identified by a user, a resource and a page. | stringly_encoded | hard | 100% | 100% | 1.31 | 0.47 |
| table_columns | A column has a name, a type and a width. | parallel_collections | obvious | 100% | 100% | 1.41 | 0.34 |
| cache_key_string | A cache entry is identified by a user, a resource and a page. | stringly_encoded | obvious | 100% | 100% | 1.42 | 0.33 |
| smtp_five_args | Sending mail needs a server to connect to and an account to connect as. | tandem_params | obvious | 100% | 100% | 1.56 | 0.27 |
| device_grid | Each device class gets a grid width, a column count and a thumbnail size. | branch_table | subtle | 100% | 100% | 1.58 | 0.35 |
| graph_weights | An edge goes from one node to another and has a cost. | parallel_collections | subtle | 100% | 100% | 1.73 | 0.24 |
| display_type | A display type sets how much of a product to show. · from the quiz | branch_table | obvious | 100% | 100% | 1.79 | 0.57 |
| draw_text_theme | Labels are drawn at a position in the chart's label style. | tandem_params | subtle | 100% | 100% | 1.83 | 0.61 |
| schedule_three | A report runs at a time of day in a given timezone. | tandem_params | obvious | 100% | 100% | 1.87 | 0.43 |
| filename_metadata | An imported photo knows when and where it was taken. | stringly_encoded | subtle | 100% | 100% | 1.87 | 0.18 |
| severity_style | Each severity has a colour, an icon and how long to wait before paging again. | branch_table | obvious | 100% | 100% | 2.06 | 0.49 |
| country_tax | Each country has a VAT rate, a currency and a legal footer. | branch_table | subtle | 100% | 100% | 2.11 | 0.32 |
| role_booleans | What a member may do depends on their role. | branch_table | subtle | 100% | 100% | 2.25 | 0.72 |
| two_maps | Each connected user has a display name and a role. | parallel_collections | obvious | 100% | 100% | 2.34 | 0.62 |
| h_builder_tandem | An axis is drawn over a range of values with a tick interval. | tandem_params | hard | 100% | 100% | 2.56 | 1.82 |
| h_record_maps | A plan comes with a seat limit, a storage allowance and a support channel. | parallel_collections | hard | 100% | 100% | 2.65 | 0.77 |
| address_fields | An order ships to an address. | tandem_params | subtle | 100% | 100% | 2.67 | 1.18 |
| chart_series | A series is a list of labelled points. | parallel_collections | subtle | 100% | 100% | 2.67 | 1.92 |
| bitfield_flags | A user has some set of features switched on. | stringly_encoded | subtle | 100% | 100% | 2.75 | 1.39 |
| plan_limits | A plan comes with a seat limit, a storage allowance and a support channel. | branch_table | obvious | 100% | 100% | 2.85 | 0.56 |
| h_wrapped_flags | A user has a set of features switched on. | no_structure | hard | 100% | 100% | 2.90 | 3.25 |
| rect_four_ints | A rectangle is a corner and a size. · from the quiz | tandem_params | obvious | 100% | 100% | 2.95 | 0.98 |
| h_two_level_branch | How a notification is delivered depends on its channel and its urgency. | branch_table | hard | 100% | 100% | 2.95 | 1.54 |
| latlng_four | A leg of a journey goes from one place to another. | tandem_params | obvious | 100% | 100% | 3.17 | 1.94 |
| ok_retry_policy | A job waits for the delay its retry policy gives, and stops when the policy says stop. | no_structure | obvious | 100% | 100% | 3.21 | 3.24 |
| ok_date_range | A booking covers a range of dates and two bookings clash when their ranges overlap. | no_structure | obvious | 100% | 100% | 3.21 | 3.58 |
| h_opaque_token | A request carries a session token, which the auth service either accepts or rejects. | no_structure | hard | 100% | 100% | 3.23 | 3.35 |
| h_real_dispatch | Evaluating an expression means doing the thing that kind of expression means. | no_structure | hard | 100% | 100% | 3.27 | 3.68 |
| ok_parse_token | A token is a word, a number or a symbol, and each carries where it was found. | no_structure | obvious | 100% | 100% | 3.29 | 3.66 |
| ok_parsed_signup | A signup request is parsed into a valid signup, or rejected with the fields that failed. | no_structure | subtle | 100% | 100% | 3.31 | 3.59 |
| ok_money_plus | Adding two amounts of the same currency gives an amount of that currency. | no_structure | obvious | 100% | 100% | 3.48 | 3.75 |
| h_unrelated_maps | The gateway remembers which users are rate limited and which have beta access. | no_structure | hard | 100% | 100% | 3.48 | 3.00 |
| ok_sum_lines | The total of an invoice is the sum of its line amounts. | no_structure | subtle | 100% | 100% | 3.63 | 3.67 |
| ID | Truth | Verdict | Conf | Match | Ref | Cond | Tier |
|---|