Same census.
Different answers.
Mary, Cherry, Nancy or Chancy? Four ChatGPT responses gave four readings of one handwritten name. Here is what changed, what we can conclude, and how to check an AI transcription before it enters your family tree.
ChatGPT read the same census four ways
Compare the changing names, ages and ditto marks, then check the original record. Four responses to one page illustrate a verification problem; they do not establish a general AI error rate.
Click to load the YouTube player. Playback does not start automatically. Captions are available.
One line. Four possible readings.
At this point, I was collecting potential ancestors faster than I could verify them. Funny on a screen. Less funny if I spend the weekend researching the wrong person.
What was actually tested?
I used ChatGPT free, without logging in. I supplied the same full-page image and prompt twice, starting a fresh conversation for the repeat. Then I ran a second pair with the full page plus a closer crop and one additional sentence explaining the crop.
The prompt asked for lines 12–15, family number 1319; preservation of spelling, fractions and ditto marks; explicit uncertainty; and a separation between recorded information and interpretation. It instructed the tool not to use web searches or family-tree information.
The differences went beyond names.
| Field | Full page 1 | Full page 2 | Page + crop 1 | Page + crop 2 |
|---|---|---|---|---|
| Line 13 age | 23 | 20? | 20 | Ditto, interpreted as 22 |
| Line 14 age | 2 | 2 | 9 | 9 |
| Line 15 age | [2?] | 2? | 5/3 [?] | [5?] |
| Occupations after “Farmer” | Blank | Ditto | Blank | [illegible] |
Calling an age a ditto mark changes the claim about what is physically on the page. Calling a field illegible is also different from calling it blank. One response labeled occupations illegible while describing a field as apparently blank.
Both crop runs agreed on age 9 for line 14. Agreement is not independent verification. We still need to examine the original strokes and the correct row.
Useful safeguards. Unsettled readings.
What the responses did well
All four included uncertainty warnings and declined to establish specific family relationships from the grouping alone.
What this example cannot show
It does not establish a general error rate, explain why the responses changed, or prove that cropping usually helps or hurts accuracy.
In these two crop runs, adding a close-up did not eliminate substantive disagreement. A caution beneath a table does not verify the entries inside it.
Try the same careful prompt.
Adapt the line numbers and family number for your document. Keep the original page available, even when you add a crop.
Download the exact original prompt (.txt)Read and select the original prompt
Transcribe only lines 12 through 15 of the attached census page, the group with family number 1319. Use the full page to compare handwriting and read the column headings. Do not search the web or use family-tree information. Return a table with: line number; name as written; age as written; sex; occupation; birthplace as written; and uncertain characters or marks. Preserve spelling, initials, abbreviations, fractions, and ditto marks. Show unreadable text as [illegible] and uncertain readings in brackets with a question mark. Do not silently expand a repeated surname or place name; explain any expansion separately. Then separate (1) what the record explicitly states, (2) your interpretations, and (3) anything the image cannot establish. If the form has no column for a fact, do not present that fact as recorded. Explain your reasons for uncertain readings without assigning numerical confidence percentages.
Added for the crop condition: “I’ve included a closer crop of the target entries; use the full-page image for headings and surrounding context.”
Before you copy it into your tree
- Save the evidence. Keep the original image, its citation, the exact prompt, the date and the response.
- Check every field. Compare names, ages and marks with the source. Distinguish blank cells, ditto marks and unreadable writing.
- Keep the full page. Use headings and surrounding handwriting. Check that a crop has not clipped a letter or fraction.
- Preserve uncertainty. Leave unresolved readings marked. Do not settle a question by majority vote across AI responses.
- Separate reading from relationship. A transcription is not proof of parentage. Seek additional evidence for a family connection.
A polished guess is still a guess. It just has better formatting.
Keep the question ready to continue.
The Brick Wall Case File gives you a place to organize your question, source notes, unresolved readings and next steps. It includes editable Word worksheets and printable PDFs. The prompt and checklist on this page are free.
Preview the Case File →Source and method notes
1850 U.S. census, Schedule 1, Guernsey County, Ohio, printed page 229, family 1319, lines 12–15. Digital image preserved from an Ancestry member gallery.
- Inspect the full page used in the test
- Ancestry gallery source ↗ (sign-in or access restrictions may apply)
- Census Bureau: 1850 enumeration instructions ↗
The instructions explain recording ages under one year as fractions in twelfths. This helps interpret the form; it does not settle the handwriting in this example. The township expansion and archival roll reference remain unverified and are omitted. This is an image-level citation, not a completed archival citation. No definitive transcription is asserted.