ReAct 论文

1
2
3
4
5
6
7
8
9
10

英文名称: REACT: SYNERGIZING REASONING AND ACTING IN LANGUAGE MODELS
中文名称: ReAct：在语言模型中协同推理与行动
链接: https://arxiv.org/abs/2210.03629
简版文档(推荐)：https://react-lm.github.io/
代码: langchain中包含React实现
作者: Shunyu Yao∗*, Jeffrey Zhao , Dian Yu , Nan Du , Izhak Shafran , Karthik Narasimhan, Yuan Cao
机构: 谷歌研究院，普林斯顿大学
日期: 2023-03-10
引用次数: 1300

读后感

正文 9 页，全文 33 页，后面都是具体例子的展示。

之前的方法通常是先思考，将大问题拆解成小步骤（step-by-step）后再执行。然后，当其中一步出错时，可能会导致接连的错误。ReAct 方法在每步执行后，将观察到的结果和最初问题一起交给大模型，以修正下一步动作，从而获得更好的效果。也就是说，在每个时点上都要思考当前的最佳策略。

结合思维和行动如下图所示：

摘要

  • 目标：通过将推理和行动结合，提升语言模型解决复杂任务的能力。
  • 方法：ReAct 模型交替生成推理痕迹和动作，与环境交互以逐步解决问题。
  • 结论：ReAct 模型在多任务中表现优于仅推理或仅行动的模型，具有可解释性，更强的灵活性和泛化能力。

核心技术：结合推理和实践

ReAct 的核心思想是扩展智能体的动作空间，使其不仅能够执行实际操作，还可以进行语言思考。虽然这种思考不会影响外部环境，但通过推理生成有用的信息，帮助智能体更好地理解上下文并做出更好的决策。

具体实现主要依赖于结构设计和提示设计。

ReAct 提示由几个示例任务解决过程组成，这些过程包括人工编写的推理步骤和操作，以及环境对这些操作的反馈。

1
2
3
4
5
6
Thought 1
Act 1
Obs 1
Thought 2
Act 2
Obs 3

效果

示例

程序的目标是获取 Evan 的工资。过程分为两步：首先，通过人名获取员工 ID；然后，通过 ID 获取工资。（从中可以看到，函数调用的顺序和调用过程会自动实现）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
from langchain_openai import ChatOpenAI
from langchain.tools import tool
from langchain import hub
from langchain.agents import create_react_agent, AgentExecutor
import os

os.environ['HTTP_PROXY'] = 'http://192.168.10.166:12346'
os.environ['HTTPS_PROXY'] = 'http://192.168.10.166:12346'
os.environ["OPENAI_API_KEY"]="sk-xxx"

@tool
def get_employee_id(name):
  """
  To get employee id, it takes employee name as arguments
  name(str): Name of the employee
  """
  fake_employees = {
    "Alice": "E001",
    "Bob": "E002",
    "Charlie": "E003",
    "Diana": "E004",
    "Evan": "E005",
    "Fiona": "E006",
    "George": "E007",
    "Hannah": "E008",
    "Ian": "E009",
    "Jasmine": "E010"}
  
  return fake_employees.get(name,"Employee not found")

@tool
def get_employee_salary(employee_id):
  """
  To get the salary of an employee, it takes employee_id as input and return salary
  """
  employee_salaries = {
    "E001": 56000,
    "E002": 47000,
    "E003": 52000,
    "E004": 61000,
    "E005": 45000,
    "E006": 58000,
    "E007": 49000,
    "E008": 53000,
    "E009": 50000,
    "E010": 55000
    }
  return employee_salaries.get(employee_id,"Employee not found")

prompt = hub.pull("hwchase17/react")
model = ChatOpenAI(model='gpt-4o')

tools = [get_employee_salary, get_employee_id]
agent = create_react_agent(model, tools, prompt)
agent_executor = AgentExecutor(agent=agent,tools=tools,verbose=True)
agent_executor.invoke({"input":"What is the Salary of Evan?"})

查看提示

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# print(prompt.template)

Answer the following questions as best you can. You have access to the following tools:

{tools}

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
… (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: {input}
Thought:{agent_scratchpad}

例子详解（每一轮的具体信息）

https://medium.com/@prabhakaran_arivalagan/behind-the-scene-react-agents-in-langchain-4f7f48c2476d