A Blank PDF Form Is My New Favorite AI Benchmark

It was a busy week for model releases, but two experiments told me more than the usual benchmark charts did.

The most interesting model tests are becoming tests of persistence, spatial reasoning, and judgment, not just whether a model can produce the right answer once.

GPT-5.6 impressed me by running for more than 40 hours and producing roughly 200,000 lines of C++ while implementing a complex technical standard. Grok 4.5 impressed me in a completely different way: it turned a flat PDF into a fillable form with surprising precision.

Those tasks look unrelated. I think they are measuring the same underlying shift.

Models are getting better at doing the whole job.

Forty hours is a different kind of intelligence

I decided to put GPT-5.6 through its paces with a task that was too large to fake with a clever demo. I asked it to implement a complex technical standard in C++.

It ran for more than 40 hours. By the end, it had generated about 200,000 lines of code.

More importantly, it did an excellent and thorough job.

I am still reviewing the result, and line count is obviously not a quality metric by itself. Two hundred thousand lines of bad code would just be a larger problem. What impressed me was the model's ability to keep moving through a long specification, maintain context across a sprawling implementation, and continue doing useful work without begging out early.

My experience with GPT-5.5 was that it could often do the work, but it sometimes stopped before I thought the task was finished. A few additional prompts would usually get it there. GPT-5.6 seems more willing to carry the task all the way to the finish line.

This changes what intelligence feels like in practice. A slightly smarter answer is useful. A system that can sustain good judgment for 40 hours can change the scale of work I am willing to hand it.

Token efficiency should be measured against finished work

I have seen people say that GPT-5.6 burns through tokens faster than GPT-5.5. That has not been my experience, at least not in the way that matters to me.

GPT-5.6 may use more tokens after a single prompt because it does more. That is different from requiring more tokens to complete the task.

If GPT-5.5 stops early and needs two or three follow-up prompts, the apparently cheaper response may not be cheaper by the time the work is actually done. It also costs more of my attention. I have to notice that the model stopped, figure out what remains, and restart it.

So I have started thinking about completion efficiency instead of response efficiency.

Response efficiency asks how many tokens the model used before it handed control back. Completion efficiency asks how much money, time, prompting, and supervision it took to reach an acceptable result.

By that measure, GPT-5.6 has felt a little more efficient than GPT-5.5 while also being smarter.

I mostly agree with the early consensus I have seen that its pure intelligence may sit a slight notch below Fable 5. Even so, GPT-5.6 is an excellent general model that seems capable of doing almost anything I would reasonably ask of it, especially in the Sol variant. OpenAI has been doing a good job of releasing models that are not just impressive, but useful and efficient enough to put to work.

Blank space is harder than it looks

My other test is much smaller, but it may be more revealing.

I give a model a flat PDF form and ask it to turn the document into a fillable PDF.

The form might have labels, boxes, lines, checkmarks, tables, and large areas of blank space. A person immediately understands that the empty rectangle beside "Last name" is where an interactive text field belongs. To a vision model, that rectangle is mostly an absence of pixels.

This makes the task an interesting combination of perception and inference. The model has to read the document, understand the purpose of each element, infer where user input belongs, choose the right field type, and place the field precisely enough that the result feels professionally made.

It is not enough to recognize what is visible. The model has to understand what the blank space means.

GPT-5.6 Sol does an excellent job at this. The problem is cost. Sol is somewhat expensive if the goal is to use it as a commodity form-processing engine.

Terra and Luna do an acceptable job, but they are not perfect. In my tests, most fields land near the right place, but many are shifted slightly. The output is close enough to be encouraging and not quite good enough to use without human cleanup.

That distinction is important. A field can be semantically correct and still feel broken if its boundary is a few pixels off.

Grok 4.5 showed its work

Grok 4.5 was the surprise.

I do not think it is yet in the same class as GPT-5.6 or Fable 5 overall. It feels more competitive with the earlier GPT-5 and Opus models, which still makes it a very good model.

On the PDF task, though, it did an excellent job.

It was also fun to watch. Grok did not simply look at the full page once and guess. It rendered images of the form, zoomed into sections, created slices, and drew guide lines to help its vision system place fields accurately.

That process may be as interesting as the final result.

The model recognized that its first view of the problem was not precise enough, so it created better intermediate representations. It effectively built its own measuring tools.

This is a useful mental model for agentic systems. The best model is not always the one with the strongest raw perception. It may be the one that notices uncertainty, changes the problem into a form it can reason about, and verifies the answer before returning it.

A useful benchmark may be hiding in old forms

I am now thinking about turning this test into a proper model benchmark.

The basic structure is straightforward. Start with a collection of PDF forms that already contain interactive fields. Save the original field geometry and metadata as the ground truth. Remove the fields to create flat versions, ask each model to reconstruct them, and then compare the generated forms with the originals.

The score could measure several things: whether the model found every field, whether it invented unnecessary fields, how closely each rectangle overlaps the original, whether it selected the correct field type, and whether the tab order makes sense.

Geometry could be scored using intersection over union, the same general idea used in object-detection benchmarks. Field-type accuracy and precision-recall would capture different failure modes. Cost, runtime, and the amount of human cleanup required would make the result more useful for real deployments.

There are complications. Existing PDFs are not automatically perfect ground truth. Some have sloppy field placement or odd authoring decisions. A serious benchmark would need a reviewed dataset, multiple form styles, scanned pages, tables, checkboxes, multiline text areas, and documents with ambiguous layouts.

Still, I think the task has unusual promise. It is objective enough to score automatically, visually subtle enough to challenge current models, and practical enough that better performance would matter outside a leaderboard.

It also tests more than vision. It tests document understanding, spatial precision, tool use, persistence, and the ability to infer human intent from empty space.

The finish line is becoming the benchmark

This week left me impressed with both GPT-5.6 and Grok 4.5 for different reasons.

GPT-5.6 showed me what sustained execution looks like at a scale I would not have trusted to an earlier model. Grok 4.5 showed me how much good tool use and self-generated visual aids can compensate for a hard perception problem.

Neither experience is summarized very well by a single intelligence score.

What I increasingly want to know is simpler: can the model recognize the real shape of the task, keep working, check itself, and deliver something I can actually use?

A 40-hour C++ implementation and a blank PDF form are very different tests. Both are really asking whether the model can find the finish line.

If I turn the PDF idea into a benchmark, I would love to hear from people who work on document AI, PDF tooling, or model evaluation. There is probably a good research project hiding in all that blank space.