The Dilemma

The deep research tools I tried never quite gave me what I wanted. They produced polished summaries, but when I asked “how do you know this?” the answer was nowhere in the report, so I set out to build a system whose work I could poke at, trace, and argue with instead of a summary I had to take on faith.

That system is a deep research skill I built on Hermes, the AI agent I run in an isolated VM on dedicated hardware as my personal assistant. The first versions worked, but they still did not meet the standard I care about, so I kept iterating over the last few months, and only recently has the output reached the point where I genuinely like the reports it hands me (I have even shared some with friends to get some extra opinions). This post explain how it works, one mechanism at a time, and it ends with a picture of the full report so you can use that as a starting point if you want to do something similar yourself.

What was the goal?

I wanted to have a report where facts carry their sources, inferences are labeled as such (with the supporting facts referenced), and a score tells me how solid the whole set of facts and inferences is.

The features I needed

• It builds an HTML artifact from markdown and JSON sources, so the result presents beautifully when I need to share it with other people;

• It marks which parts of the report are facts, cited with their sources, and which are inference, meaning what the model worked out from those facts, then shows the ratio between them in a nice HTML widget;

• It produces a “Confidence Score” that grades the evidence behind the report, also shown as a nice HTML widget;

• I can use any combination of models for the research and for the contester, which refines the research results one step further.

What is Hermes Agent and what does it have to do with all that?

Hermes is the AI agent I run on a side machine. I give it a job in plain language and it uses a skill to read and execute the same instructions every time.

Hermes runs the deep research following a pipeline and ends with one report in 3 artifacts: a markdown document, a machine readable JSON file, and a web page meant for human consumption. The skill contains the instructions, but it also has some scripts to add some determinism. E.g., a script does some automated checks to confirm the 3 artifacts are in sync before the report is handed to me.

What does this look like?

Here is a detailed view of each element of the final HTML artifact, with some pictures to give a broader idea of what can be achieved.

We start with the deep research name, and below it we show metadata: the number of sources, the original query that generated that and some other useful data about the research:

Below that, we show a TOC and the Methodology section, which explains how Hermes executed the research, with the details relevant to its context:

Then, we present the “Executive Summary.” That is a general overview of the report, that can be read quickly to have a broad idea of the results. This is the first place we use the “FACT” and “INFERENCE” pills, that help separate the facts from the model inference based on the data:

At the right side of the report, there are also some additional widgets that give some quantitative data, which are:

• FACT MIX: The distribution of facts VS inference

• CONFIDENCE: the model uses this to evaluate its confidence in the report results using a classification system based on “tiers”. Together with the Confidence Score x, it also shows in the widget “Why not x-1” and “Why not x+1”, which help us put into context how the model has come to the final number.

Here is an image of both widgets:

The Findings section holds the core of the report. It organizes the findings into numbered statements, which are also indicated as corroborated by other sources or not. Inside each statement, it shows the facts and inferences that are part of it.

After that, you have the Recommendation section, which also depends on what the subject of the research was. It will be dominated by inference, but it will show links to the sources that support it:

If there were any contradictions, debates or counter-arguments to the facts Hermes gathered initially, you will find them under the Contradictions & Debates section:

The Confidence Score depends on corroboration of the facts and central claims, how much verification the agent was able to make, the knowledge gaps found and other factors. And talking about the knowledge gaps, they also have their own section in the report if any were found:

And the last section of the report is the References section, which has a table containing all the URLs, their publishing dates, their contribution to the research facts/recommendations and whether they were verified:

In the next sections I explain better and in more detail each element of the HTML artifact.

How a research run works

Planning. Hermes breaks the question into three to seven smaller ones: what the established facts are, where sources disagree, what the timeline looks like, what evidence actually exists. Then it writes a search plan of five to ten queries, phrased differently on purpose (academic, news, official announcements) so it does not draw only from one kind of source.

First round: wide, and read in full. Several searches run at once and collect 15 to 50 candidate pages, from which Hermes picks the 8 to 15 most promising and reads each one in full. For every page it records the publication date and the last update date.

Second round: close the gaps. The results from the first round decide the second: 2 to 5 targeted searches that fill the holes, chase a claim back to whoever made it first, and look specifically for evidence contradicting what was found. Hermes keeps and reports contradictory evidence.

Scoring every source. Each source (page, pdf, etc.) gets a relevance score from 0 to 100, which is a grade of how useful it was for this report. Five factors decide it, in order: how directly it addresses the claim, the authority of the publisher, how recent it is, how far independent sources agree with it, and how deeply it covers the topic. The ranges:

• 90 to 100: answers the core question directly, usually from a primary source;
• 70 to 89: strongly relevant and credible;
• 50 to 69: moderate context;
• 25 to 49: marginal or dated, listed anyway for transparency;
• 0 to 24: weak, kept with a caveat instead of hidden.

Writing up. Hermes groups the findings into what answers the question, what supports it, where sources disagree, background, and what stays unknown. Every report then has the same parts: an executive summary, tagged key findings (fact/inference pills at the beginning of each finding ), a contradictions section, knowledge gaps, and a reference list showing each source’s score and verification status.

The Independent Fact Check

The feature that changed how much I trust the output comes from the independent fact check. Every central claim goes to a contester (reviewer) that begins from zero: a fresh AI instance with no memory of the searches and no sight of the pages this research collected, given one instruction: do not accept this claim and try to prove it wrong. Those rules make that review independent.

Source independence. A source counts as independent only if its origin does not trace back to a source already supporting the same claim. E.g. 5 sites retelling one press release count as one source instead of five.

Model independence. The reviewer runs on a different AI model than the one that did the research. Hermes picks the reviewing model from its configuration and mentions it into the methodology of every report. For example, GLM 5.3 Flash makes the research, then DeepSeek 4.1 Flash acts as the reviewer. If the two procedures (research and review) ever share a model, the report says so and flags the extra risk of correlated mistakes.

Each research claim comes back with one of the verdicts below:

• Corroborated: at least two independent sources agree.
• Single-source: exactly one independent source exists.
• Contradicted: independent sources disagree, so the report shows both positions.
• Not found: a targeted search located no independent source at all.

Where the reviewer and the research models disagree, both readings land in the contradictions section.

The Fact and Inference Tags

Hermes tags every claim in the report before delivery, and there are only 2 tags:

• FACT: Hermes read it directly in a source, a quotation, an official document, a pdf file, or a number it measured live while doing the research.

• INFERENCE: Hermes concluded it from a reading, an implication or a synthesis of several facts. When it cannot decide which tag a statement belongs to, it tags it INFERENCE.

Three rules about the tagging:

1. A tag sits in front of its claim, in its own paragraph, so one paragraph never hides two differently tagged claims (those are the “pills” I referred as previously, because they render in the html report as a “pill”).

2. The whole report’s split is counted (facts VS inference), and that count shows as a widget on the right of the web page next to the confidence score and inside the JSON file.

3. A script does an automated check and compares the tags in the text, the counts in the data file, and the widget, and if any of them disagree, the report fails and is not delivered.

For you as a reader, the split is simple: a FACT is something the AI saw, and an INFERENCE is something the AI concluded by itself. A section heavy in inferences is exactly where you must direct your attention to and do some check by yourself as a human (by sampling or really reading each reference that is linked to that inference).

The Confidence Score

Every report states a Confidence Score on a 1 to 10 scale (e.g. “Score: 7 out of 10, yellow tier”). The three ranges:

• 1 to 5, red: evidence gaps or a contested claim the report leans on. Read the report as a hypothesis, or as a recommendation to proceed with caution.

• 6 to 7, yellow: the direction is corroborated by independent sources, but details may still be worked on.

• 8 to 10, green: corroborated across independent sources and verified live, where the open gaps are non-structural, meaning they do not change the conclusions.

The score is calculated as:

1. How broadly the central claims are corroborated;
2. The share of primary versus secondary sources;
3. How much Hermes verified live, meaning Hermes checked the current source during the research instead of an article about it;
4. The reviewer’s verdicts;
5. How many claims were contradicted;
6. And how many knowledge gaps remain open.

The score has a legend of the ranges in plain language and two boundary paragraphs:

• why the number is not one point lower: what would justify a downgrade and why that condition does not apply;
• why it is not one point higher: what the next level demands and what specifically blocks it.

That is why I consider the boundaries (score-1, score+1) as the audit trail for the number itself: if you think the score is wrong or insufficient, the two paragraphs below it tell you exactly what evidence would improve the report.

Result: the AI “thought process” for the Deep Research

The 3 mechanisms together (the Independent Fact Check, the Fact and Inference Tags, and the Confidence Score ) are the closest thing I have to a readable “thought process” of an AI: The fact and inference tags show what it saw versus what it concluded, the verification verdicts show which conclusions a second model checked and which rest on one source, and the score shows how solid the whole base is.

So, I can read the report results with much more context. I can focus on where the report itself admits weakness, the Single-source and Contradicted findings and the paragraphs heavy with inferences, while treating the green, corroborated parts as already tested. I can even ask Hermes to follow-up on what would make the report Confident Score reach “score + 1” (using its own judgement as the input), and keep iterating on it until it reaches a limit it cannot be improved anymore or that I would have diminishing returns from doing that.

How can I verify the report manually

That is what helped me build confidence in it during the many iterations I did to reach this point: I open any finished report and confirm it has all five parts: the methodology that says how the research was done, the tagged key findings, the confidence score with both boundary paragraphs, the contradictions section, and the reference list with scores and verification badges. Then I take one SINGLE-SOURCE finding and do a manual search for one independent second source on that claim myself, and take one INFERENCE paragraph and ask myself whether the facts related to it actually support the conclusion I just read.

Final considerations

• The score grades evidence, not conclusions. 7 out of 10 does not mean the conclusions are 70% likely. It describes the state of the evidence (facts) and nothing more.

• Single-source findings. A SINGLE-SOURCE finding can still be correct, and I can still do one manual search and/or read it before I can trust it.

• Independent review reduces shared blind spots; it does not remove them. 2 different models can make the same mistake. What is left unknown is listed under knowledge gaps.

• If the independent check cannot run, the report says so. The methodology records the fallback rather than pretending a review happened.

• Dates expire. A score reflects the evidence on the day the research ran; anything that moved since then could change how applicable the research is.

I must also mention that this is the current state of this solution, and I may still iterate on it or improve it as I keep using it more and more and sharing the reports it produces with other people around me. I will keep this post updated as this workflow evolves.

Complete Report Picture

Here is a complete picture of the report: