Thread and Tree of Thoughts

The last two strategies go beyond a single line of reasoning. Thread of Thoughts works through a long context in parts. Tree of Thoughts explores several possible next steps and keeps the most promising. The examples are in the reasoning-agents project.

Thread of Thoughts

When a question must be answered from a long or noisy context, such as a set of retrieved passages that mixes relevant facts with distractors, presenting everything at once can cause the model to miss or confuse details. Thread of Thoughts (Zhou et al., 2023) walks the model through the context one part at a time. For each part, the model analyses it with respect to the question and rewrites a running set of notes, pointing out where the part contradicts or refines what came before. A final call answers the question from the notes.

ThreadOfThoughts takes the parts as a list:

!solve(string question, list parts, string answer)

In this example, the third part is a distractor and the second refines the first:

import astra.langchain4j.reasoning.*;

agent Passages extends ThreadOfThoughts {
    module System S;

    rule +!main(list args) {
        try {
            model.loadApiKeyFromEnv("OPENAI_API_KEY");
            model.initialize("gpt-4o-mini",
                "astra.langchain4j.configuration.OpenAiConfiguration", []);

            list parts = [
                "The library opens at 9:00 on weekdays. The cafe next door opens at 8:00.",
                "From June to August, the library opens one hour later than usual.",
                "The library's car park is closed for repairs throughout July."
            ];
            !solve("When does the library open on a weekday in July?", parts, string answer);
            console.println("Answer: " + answer);
        } recover {
            console.println("The question could not be answered.");
        }
        S.exit();
    }
}

Run it with mvn -Dastra.main=Passages. The agent prints the notes after each part, then the answer, 10:00.

The strategy makes one call per part plus one for the answer. How the context is divided is up to the agent: the parts could be documents, search results, sections of a file, or sentences built from beliefs.

Tree of Thoughts

Some problems need search: an early step that looks reasonable can lead to a dead end. Tree of Thoughts (Yao et al., 2023) treats reasoning as a tree. At each level, the model proposes several possible next steps for each path kept so far. A separate call scores each new path from 1 to 10, and only the best paths are kept for the next level. When the tree reaches its full depth, the best path is turned into the answer.

TreeOfThoughts performs a breadth-first search with a beam, configured by beliefs:

Belief

Meaning

Default

branches(int)

Next steps proposed for each path

3

beam(int)

Paths kept at each level

2

depth(int)

Steps in a complete path

3

This example makes the search smaller than the defaults by replacing two of the beliefs:

import astra.langchain4j.reasoning.*;

agent Tree extends TreeOfThoughts {
    module System S;

    rule +!main(list args) {
        try {
            model.loadApiKeyFromEnv("OPENAI_API_KEY");
            model.initialize("gpt-4o-mini",
                "astra.langchain4j.configuration.OpenAiConfiguration", []);

            // A smaller search than the defaults: 2 proposals per path, 2 levels
            -branches(3); +branches(2);
            -depth(3); +depth(2);

            !solve("Using each of the numbers 2, 3 and 4 exactly once, "
                + "with + and *, make 20.", string answer);
            console.println("Answer: " + answer);
        } recover {
            console.println("The question could not be answered.");
        }
        S.exit();
    }
}

Run it with mvn -Dastra.main=Tree. The agent prints each level, with the score of each proposed step, then the best path and the answer:

[main] ===== level 1 =====
[main] [8] Add 2 and 3 to get 5.
[main] [4] Multiply 3 and 4 to get 12.
[main] ===== level 2 =====
...
[main] ===== best path =====
...
[main] Answer: (2 + 3) * 4 = 20

The steps, scores and wording vary between runs.

Where next?

The reasoning agents work within one ASTRA program. Agent-to-Agent communication lets them offer their skills to, and use the skills of, agents elsewhere.