Summarising Long Documents with AI, and What It Misses

A summary is a map, not the territory

Turning a report or set of meeting minutes dozens of pages long into a few lines is genuinely convenient. It is especially good for deciding whether a document deserves close reading, or for getting its overall structure in your head first. But a summary cannot replace the original. The AI decides what to keep and what to cut, and nothing guarantees its criteria match your purpose. Use the summary as a map into the source, and make important judgements from the source itself. When you read the summary first and then the original, take care not to be led by it. If you only confirm what the summary told you, whatever it left out will never come into view.

What summaries tend to miss

Summaries follow the main line of a text, so small sentences that sit off that line are the first to be cut. The trouble is that those sentences are often the most important ones. Even if these items do not appear in the summary, look for them in the original yourself.

• Conditions and exceptions after "however", "provided that" or "except"
• Specific figures such as deadlines, amounts and percentages
• Minority opinions or objections that reach a different conclusion
• Footnotes, appendices and the small print under tables
• Who agreed to do what, and by when

Set a purpose before asking

"Summarise this" returns the most average summary possible. Telling it why you are reading changes what it keeps. The same minutes call for different summaries depending on whether you want the decisions or your own action items. Requests such as "list only the tasks and deadlines for our team", "separate the arguments for and against" or "keep every sentence containing a number exactly as written" produce far more useful results.

Summarise long documents in parts

There is a limit to how much text an AI can handle at once. Paste in a very long document whole and only part of it may be reflected, or the summary may treat the opening in detail and the ending in passing. It is safer to split a long document by chapter or section, summarise each, then combine those summaries into a final overview. Include the section headings when splitting so locations stay clear. Estimating the reading time beforehand also helps you judge how finely to split.

Ask for a source location for each point

The most effective way to verify a summary is to ask it to mark where in the original each point came from. With a chapter number, page or the first words of a paragraph, you can check against the source immediately. Be suspicious of any point it cannot locate, or one whose tone differs from the source when you look it up. It also helps to ask for key sentences quoted verbatim rather than paraphrased; searching the original for the quote is a quick check.

When content not in the source creeps in

While organising a document, AI sometimes adds things that are common knowledge or typical of similar documents. The source says "under review" while the summary says "planned for introduction", or a reason appears that the source never gave. Because it sounds plausible, it is all the harder to notice. Before passing a summary on to others or copying it into a report, confirm that the conclusions and figures in particular really are stated that way in the original. Adding a condition such as "add nothing that is not in the source, and mark anything you are unsure of" to your request can reduce this to some extent.

Cautions by document type

Never agree to or sign a contract or terms of service based on a summary. Unfavourable clauses such as cancellation conditions, penalties and automatic renewal are usually short and tucked away. For important contracts, seek professional advice. For research papers, summaries sometimes state conclusions more strongly than the study does, so read the limitations section yourself. For minutes, ask it to separate decisions from discussion. For work documents, check company policy on whether internal material may go into an outside service at all.

Check before trusting a summary

Before making a decision based on a summary or passing it on, confirm the items below.

• You told it your reading purpose before getting the summary
• You checked exceptions and figures directly in the original
• You found the source location for each main point
• You confirmed that documents with personal or confidential data may be entered