mirror of
https://github.com/Shubhamsaboo/awesome-llm-apps.git
synced 2026-04-28 22:28:59 -05:00
Merge pull request #208 from libw0430/main
Added EvoAgentX to multi agent app
This commit is contained in:
Shubham Saboo
GitHub
@@ -75,6 +75,7 @@ We're launching a Global AI Agent Hackathon in collaboration with AI Agent ecosy
|
||||
* [🗞️ AI Journalist Agent](advanced_ai_agents/single_agent_apps/ai_journalist_agent/)
|
||||
* [🧠 AI Mental Wellbeing Agent](advanced_ai_agents/multi_agent_apps/ai_mental_wellbeing_agent/)
|
||||
* [📑 AI Meeting Agent](advanced_ai_agents/single_agent_apps/ai_meeting_agent/)
|
||||
* [🧬 AI Self-Evolving Agent](advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/)
|
||||
|
||||
### 🎮 Autonomous Game Playing Agents
|
||||
|
||||
|
||||
@@ -0,0 +1,344 @@
|
||||
<!-- Add logo here -->
|
||||
<div align="center">
|
||||
<a href="https://github.com/EvoAgentX/EvoAgentX">
|
||||
<img src="./assets/EAXLoGo.svg" alt="EvoAgentX" width="50%">
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<h2 align="center">
|
||||
Building a Self-Evolving Ecosystem of AI Agents
|
||||
</h2>
|
||||
|
||||
<div align="center">
|
||||
|
||||
[](https://evoagentx.org/)
|
||||
[](https://EvoAgentX.github.io/EvoAgentX/)
|
||||
[](https://discord.gg/SUEkfTYn)
|
||||
[](https://x.com/EvoAgentX)
|
||||
[](./assets/wechat_info.md)
|
||||
[](https://star-history.com/#EvoAgentX/EvoAgentX)
|
||||
[](https://github.com/EvoAgentX/EvoAgentX/fork)
|
||||
[](https://github.com/EvoAgentX/EvoAgentX/blob/main/LICENSE)
|
||||
<!-- [](https://EvoAgentX.github.io/EvoAgentX/) -->
|
||||
<!-- [](https://huggingface.co/EvoAgentX) -->
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
|
||||
<h3 align="center">
|
||||
|
||||
<a href="./README.md" style="text-decoration: underline;">English</a> | <a href="./README-zh.md">简体中文</a>
|
||||
|
||||
</h3>
|
||||
|
||||
</div>
|
||||
|
||||
<h4 align="center">
|
||||
<i>An automated framework for evaluating and evolving agentic workflows.</i>
|
||||
</h4>
|
||||
|
||||
<p align="center">
|
||||
<img src="./assets/framework_en.jpg">
|
||||
</p>
|
||||
|
||||
|
||||
## 🔥 Latest News
|
||||
- **[May 2025]** 🎉 **EvoAgentX** has been officially released!
|
||||
|
||||
## ⚡ Get Started
|
||||
- [🔥 Latest News](#-latest-news)
|
||||
- [⚡ Get Started](#-get-started)
|
||||
- [Installation](#installation)
|
||||
- [LLM Configuration](#llm-configuration)
|
||||
- [API Key Configuration](#api-key-configuration)
|
||||
- [Configure and Use the LLM](#configure-and-use-the-llm)
|
||||
- [Automatic WorkFlow Generation](#automatic-workflow-generation)
|
||||
- [Demo Video](#demo-video)
|
||||
- [✨ Final Results](#-final-results)
|
||||
- [Evolution Algorithms](#evolution-algorithms)
|
||||
- [📊 Results](#-results)
|
||||
- [Applications](#applications)
|
||||
- [Tutorial and Use Cases](#tutorial-and-use-cases)
|
||||
- [🎯 Roadmap](#-roadmap)
|
||||
- [🙋 Support](#-support)
|
||||
- [Join the Community](#join-the-community)
|
||||
- [Contact Information](#contact-information)
|
||||
- [🙌 Contributing to EvoAgentX](#-contributing-to-evoagentx)
|
||||
- [📚 Acknowledgements](#-acknowledgements)
|
||||
- [📄 License](#-license)
|
||||
|
||||
## Installation
|
||||
|
||||
We recommend installing EvoAgentX using `pip`:
|
||||
|
||||
```bash
|
||||
pip install git+https://github.com/EvoAgentX/EvoAgentX.git
|
||||
```
|
||||
|
||||
For local development or detailed setup (e.g., using conda), refer to the [Installation Guide for EvoAgentX](./docs/installation.md).
|
||||
|
||||
<details>
|
||||
<summary>Example (optional, for local development):</summary>
|
||||
|
||||
```bash
|
||||
git clone https://github.com/EvoAgentX/EvoAgentX.git
|
||||
cd EvoAgentX
|
||||
# Create a new conda environment
|
||||
conda create -n evoagentx python=3.10
|
||||
|
||||
# Activate the environment
|
||||
conda activate evoagentx
|
||||
|
||||
# Install the package
|
||||
pip install -r requirements.txt
|
||||
# OR install in development mode
|
||||
pip install -e .
|
||||
```
|
||||
</details>
|
||||
|
||||
## LLM Configuration
|
||||
|
||||
### API Key Configuration
|
||||
|
||||
To use LLMs with EvoAgentX (e.g., OpenAI), you must set up your API key.
|
||||
|
||||
<details>
|
||||
<summary>Option 1: Set API Key via Environment Variable</summary>
|
||||
|
||||
- Linux/macOS:
|
||||
```bash
|
||||
export OPENAI_API_KEY=<your-openai-api-key>
|
||||
```
|
||||
|
||||
- Windows Command Prompt:
|
||||
```cmd
|
||||
set OPENAI_API_KEY=<your-openai-api-key>
|
||||
```
|
||||
|
||||
- Windows PowerShell:
|
||||
```powershell
|
||||
$env:OPENAI_API_KEY="<your-openai-api-key>" # " is required
|
||||
```
|
||||
|
||||
Once set, you can access the key in your Python code with:
|
||||
```python
|
||||
import os
|
||||
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Option 2: Use .env File</summary>
|
||||
|
||||
- Create a .env file in your project root and add the following:
|
||||
```bash
|
||||
OPENAI_API_KEY=<your-openai-api-key>
|
||||
```
|
||||
|
||||
Then load it in Python:
|
||||
```python
|
||||
from dotenv import load_dotenv
|
||||
import os
|
||||
|
||||
load_dotenv() # Loads environment variables from .env file
|
||||
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
|
||||
```
|
||||
</details>
|
||||
<!-- > 🔐 Tip: Don't forget to add `.env` to your `.gitignore` to avoid committing secrets. -->
|
||||
|
||||
### Configure and Use the LLM
|
||||
Once the API key is set, initialise the LLM with:
|
||||
|
||||
```python
|
||||
from evoagentx.models import OpenAILLMConfig, OpenAILLM
|
||||
|
||||
# Load the API key from environment
|
||||
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
|
||||
|
||||
# Define LLM configuration
|
||||
openai_config = OpenAILLMConfig(
|
||||
model="gpt-4o-mini", # Specify the model name
|
||||
openai_key=OPENAI_API_KEY, # Pass the key directly
|
||||
stream=True, # Enable streaming response
|
||||
output_response=True # Print response to stdout
|
||||
)
|
||||
|
||||
# Initialize the language model
|
||||
llm = OpenAILLM(config=openai_config)
|
||||
|
||||
# Generate a response from the LLM
|
||||
response = llm.generate(prompt="What is Agentic Workflow?")
|
||||
```
|
||||
> 📖 More details on supported models and config options: [LLM module guide](./docs/modules/llm.md).
|
||||
|
||||
|
||||
## Automatic WorkFlow Generation
|
||||
Once your API key and language model are configured, you can automatically generate and execute multi-agent workflows in EvoAgentX.
|
||||
|
||||
🧩 Core Steps:
|
||||
1. Define a natural language goal
|
||||
2. Generate the workflow with `WorkFlowGenerator`
|
||||
3. Instantiate agents using `AgentManager`
|
||||
4. Execute the workflow via `WorkFlow`
|
||||
|
||||
💡 Minimal Example:
|
||||
```python
|
||||
from evoagentx.workflow import WorkFlowGenerator, WorkFlowGraph, WorkFlow
|
||||
from evoagentx.agents import AgentManager
|
||||
|
||||
goal = "Generate html code for the Tetris game"
|
||||
workflow_graph = WorkFlowGenerator(llm=llm).generate_workflow(goal)
|
||||
|
||||
agent_manager = AgentManager()
|
||||
agent_manager.add_agents_from_workflow(workflow_graph, llm_config=openai_config)
|
||||
|
||||
workflow = WorkFlow(graph=workflow_graph, agent_manager=agent_manager, llm=llm)
|
||||
output = workflow.execute()
|
||||
print(output)
|
||||
```
|
||||
|
||||
You can also:
|
||||
- 📊 Visualise the workflow: `workflow_graph.display()`
|
||||
- 💾 Save/load workflows: `save_module()` / `from_file()`
|
||||
|
||||
> 📂 For a complete working example, check out the [`workflow_demo.py`](https://github.com/EvoAgentX/EvoAgentX/blob/main/examples/workflow_demo.py)
|
||||
|
||||
|
||||
## Demo Video
|
||||
|
||||
|
||||
[](https://www.youtube.com/watch?v=Wu0ZydYDqgg)
|
||||
|
||||
<div align="center">
|
||||
<video src="https://github.com/user-attachments/assets/8f65d1af-9398-40c3-a625-4f493e13e5a5.mp4" autoplay loop muted playsinline width="600">
|
||||
Your browser does not support the video tag.
|
||||
</video>
|
||||
</div>
|
||||
|
||||
In this demo, we showcase the workflow generation and execution capabilities of EvoAgentX through two examples:
|
||||
|
||||
- Application 1: Intelligent Job Recommendation from Resume
|
||||
- Application 2: Visual Analysis of A-Share Stocks
|
||||
|
||||
|
||||
### ✨ Final Results
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td align="center">
|
||||
<img src="./assets/demo_result_1.png" width="400"><br>
|
||||
<strong>Application 1:</strong><br>Job Recommendation
|
||||
</td>
|
||||
<td align="center">
|
||||
<img src="./assets/demo_result_2.jpeg" width="400"><br>
|
||||
<strong>Application 2:</strong><br>Stock Visual Analysis
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## Evolution Algorithms
|
||||
|
||||
We have integrated some existing agent/workflow evolution algorithms into EvoAgentX, including [TextGrad](https://www.nature.com/articles/s41586-025-08661-4), [MIPRO](https://arxiv.org/abs/2406.11695) and [AFlow](https://arxiv.org/abs/2410.10762).
|
||||
|
||||
To evaluate the performance, we use them to optimize the same agent system on three different tasks: multi-hop QA (HotPotQA), code generation (MBPP) and reasoning (MATH). We randomly sample 50 examples for validation and other 100 examples for testing.
|
||||
|
||||
> Tip: We have integrated these benchmark and evaluation code in EvoAgentX. Please refer to the [benchmark and evaluation tutorial](https://github.com/EvoAgentX/EvoAgentX/blob/main/docs/tutorial/benchmark_and_evaluation.md) for more details.
|
||||
|
||||
### 📊 Results
|
||||
|
||||
| Method | HotPotQA<br>(F1%) | MBPP<br>(Pass@1 %) | MATH<br>(Solve Rate %) |
|
||||
|----------|--------------------|---------------------|--------------------------|
|
||||
| Original | 63.58 | 69.00 | 66.00 |
|
||||
| TextGrad | 71.02 | 71.00 | 76.00 |
|
||||
| AFlow | 65.09 | 79.00 | 71.00 |
|
||||
| MIPRO | 69.16 | 68.00 | 72.30
|
||||
|
||||
Please refer to the `examples/optimization` folder for more details.
|
||||
|
||||
## Applications
|
||||
|
||||
We use our framework to optimize existing multi-agent systems on the [GAIA](https://huggingface.co/spaces/gaia-benchmark/leaderboard) benchmark. We select [Open Deep Research](https://github.com/huggingface/smolagents/tree/main/examples/open_deep_research) and [OWL](https://github.com/camel-ai/owl), two representative multi-agent framework from the GAIA leaderboard that is open-source and runnable.
|
||||
|
||||
We apply EvoAgentX to optimize their prompts. The performance of the optimized agents on the GAIA benchmark validation set is shown in the figure below.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td align="center" width="50%">
|
||||
<img src="./assets/open_deep_research_optimization_report.png" alt="Open Deep Research Optimization" width="100%"><br>
|
||||
<strong>Open Deep Research</strong>
|
||||
</td>
|
||||
<td align="center" width="50%">
|
||||
<img src="./assets/owl_optimization_result.png" alt="OWL Optimization" width="100%"><br>
|
||||
<strong>OWL Agent</strong>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
> Full Optimization Reports: [Open Deep Research](https://github.com/eax6/smolagents) and [OWL](https://github.com/TedSIWEILIU/owl).
|
||||
|
||||
## Tutorial and Use Cases
|
||||
|
||||
> 💡 **New to EvoAgentX?** Start with the [Quickstart Guide](./docs/quickstart.md) for a step-by-step introduction.
|
||||
|
||||
|
||||
Explore how to effectively use EvoAgentX with the following resources:
|
||||
|
||||
| Cookbook | Description |
|
||||
|:---|:---|
|
||||
| **[Build Your First Agent](./docs/tutorial/first_agent.md)** | Quickly create and manage agents with multi-action capabilities. |
|
||||
| **[Build Your First Workflow](./docs/tutorial/first_workflow.md)** | Learn to build collaborative workflows with multiple agents. |
|
||||
| **[Automatic Workflow Generation](./docs/quickstart.md#automatic-workflow-generation-and-execution)** | Automatically generate workflows from natural language goals. |
|
||||
| **[Benchmark and Evaluation Tutorial](./docs/tutorial/benchmark_and_evaluation.md)** | Evaluate agent performance using benchmark datasets. |
|
||||
| **[TextGrad Optimizer Tutorial](./docs/tutorial/textgrad_optimizer.md)** | Automatically optimise the prompts within multi-agent workflow with TextGrad. |
|
||||
| **[AFlow Optimizer Tutorial](./docs/tutorial/aflow_optimizer.md)** | Automatically optimise both the prompts and structure of multi-agent workflow with AFlow. |
|
||||
<!-- | **[SEW Optimizer Tutorial](./docs/tutorial/sew_optimizer.md)** | Create SEW (Self-Evolving Workflows) to enhance agent systems. | -->
|
||||
|
||||
🛠️ Follow the tutorials to build and optimize your EvoAgentX workflows.
|
||||
|
||||
🚀 We're actively working on expanding our library of use cases and optimization strategies. **More coming soon — stay tuned!**
|
||||
|
||||
## 🎯 Roadmap
|
||||
- [ ] **Modularize Evolution Algorithms**: Abstract optimization algorithms into plug-and-play modules that can be easily integrated into custom workflows.
|
||||
- [ ] **Develop Task Templates and Agent Modules**: Build reusable templates for typical tasks and standardized agent components to streamline application development.
|
||||
- [ ] **Integrate Self-Evolving Agent Algorithms**: Incorporate more recent and advanced agent self-evolution across multiple dimensions, including prompt tuning, workflow structures, and memory modules.
|
||||
- [ ] **Enable Visual Workflow Editing Interface**: Provide a visual interface for workflow structure display and editing to improve usability and debugging.
|
||||
|
||||
|
||||
|
||||
## 🙋 Support
|
||||
|
||||
### Join the Community
|
||||
|
||||
📢 Stay connected and be part of the **EvoAgentX** journey!
|
||||
🚩 Join our community to get the latest updates, share your ideas, and collaborate with AI enthusiasts worldwide.
|
||||
|
||||
- [Discord](https://discord.gg/SUEkfTYn) — Chat, discuss, and collaborate in real-time.
|
||||
- [X (formerly Twitter)](https://x.com/EvoAgentX) — Follow us for news, updates, and insights.
|
||||
- [WeChat](https://github.com/EvoAgentX/EvoAgentX/blob/main/assets/wechat_info.md) — Connect with our Chinese community.
|
||||
|
||||
### Contact Information
|
||||
|
||||
If you have any questions or feedback about this project, please feel free to contact us. We highly appreciate your suggestions!
|
||||
|
||||
- **Email:** evoagentx.ai@gmail.com
|
||||
|
||||
We will respond to all questions within 2-3 business days.
|
||||
|
||||
## 🙌 Contributing to EvoAgentX
|
||||
Thanks go to these awesome contributors
|
||||
|
||||
<a href="https://github.com/EvoAgentX/EvoAgentX/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=EvoAgentX/EvoAgentX" />
|
||||
</a>
|
||||
|
||||
We appreciate your interest in contributing to our open-source initiative. We provide a document of [contributing guidelines](https://github.com/EvoAgentX/EvoAgentX/blob/main/CONTRIBUTING.md) which outlines the steps for contributing to EvoAgentX. Please refer to this guide to ensure smooth collaboration and successful contributions. 🤝🚀
|
||||
|
||||
[](https://www.star-history.com/#EvoAgentX/EvoAgentX&Date)
|
||||
|
||||
|
||||
## 📚 Acknowledgements
|
||||
This project builds upon several outstanding open-source projects: [AFlow](https://github.com/FoundationAgents/MetaGPT/tree/main/metagpt/ext/aflow), [TextGrad](https://github.com/zou-group/textgrad), [DSPy](https://github.com/stanfordnlp/dspy), [LiveCodeBench](https://github.com/LiveCodeBench/LiveCodeBench), and more. We would like to thank the developers and maintainers of these frameworks for their valuable contributions to the open-source community.
|
||||
|
||||
## 📄 License
|
||||
|
||||
Source code in this repository is made available under the [MIT License](./LICENSE).
|
||||
@@ -0,0 +1,86 @@
|
||||
import os
|
||||
from dotenv import load_dotenv
|
||||
from evoagentx.models import OpenAILLMConfig, OpenAILLM, LiteLLMConfig, LiteLLM
|
||||
from evoagentx.workflow import WorkFlowGenerator, WorkFlowGraph, WorkFlow
|
||||
from evoagentx.agents import AgentManager
|
||||
from evoagentx.actions.code_extraction import CodeExtraction
|
||||
from evoagentx.actions.code_verification import CodeVerification
|
||||
from evoagentx.core.module_utils import extract_code_blocks
|
||||
|
||||
load_dotenv() # Loads environment variables from .env file
|
||||
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
|
||||
ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY")
|
||||
|
||||
def main():
|
||||
|
||||
# LLM configuration
|
||||
openai_config = OpenAILLMConfig(model="gpt-4o-mini", openai_key=OPENAI_API_KEY, stream=True, output_response=True, max_tokens=16000)
|
||||
# Initialize the language model
|
||||
llm = OpenAILLM(config=openai_config)
|
||||
|
||||
goal = "Generate html code for the Tetris game that can be played in the browser."
|
||||
target_directory = "examples/output/tetris_game"
|
||||
|
||||
wf_generator = WorkFlowGenerator(llm=llm)
|
||||
workflow_graph: WorkFlowGraph = wf_generator.generate_workflow(goal=goal)
|
||||
|
||||
# [optional] display workflow
|
||||
workflow_graph.display()
|
||||
# [optional] save workflow
|
||||
# workflow_graph.save_module(f"{target_directory}/workflow_demo_4o_mini.json")
|
||||
#[optional] load saved workflow
|
||||
# workflow_graph: WorkFlowGraph = WorkFlowGraph.from_file(f"{target_directory}/workflow_demo_4o_mini.json")
|
||||
|
||||
agent_manager = AgentManager()
|
||||
agent_manager.add_agents_from_workflow(workflow_graph, llm_config=openai_config)
|
||||
|
||||
workflow = WorkFlow(graph=workflow_graph, agent_manager=agent_manager, llm=llm)
|
||||
output = workflow.execute()
|
||||
|
||||
# verfiy the code
|
||||
verification_llm_config = LiteLLMConfig(model="anthropic/claude-3-7-sonnet-20250219", anthropic_key=ANTHROPIC_API_KEY, stream=True, output_response=True, max_tokens=20000)
|
||||
verification_llm = LiteLLM(config=verification_llm_config)
|
||||
|
||||
code_verifier = CodeVerification()
|
||||
output = code_verifier.execute(
|
||||
llm = verification_llm,
|
||||
inputs={
|
||||
"requirements": goal,
|
||||
"code": output
|
||||
}
|
||||
).verified_code
|
||||
|
||||
# extract the code
|
||||
os.makedirs(target_directory, exist_ok=True)
|
||||
code_blocks = extract_code_blocks(output)
|
||||
if len(code_blocks) == 1:
|
||||
file_path = os.path.join(target_directory, "index.html")
|
||||
with open(file_path, "w") as f:
|
||||
f.write(code_blocks[0])
|
||||
print(f"You can open this HTML file in a browser to play the Tetris game: {file_path}")
|
||||
return
|
||||
|
||||
code_extractor = CodeExtraction()
|
||||
results = code_extractor.execute(
|
||||
llm=llm,
|
||||
inputs={
|
||||
"code_string": output,
|
||||
"target_directory": target_directory,
|
||||
}
|
||||
)
|
||||
|
||||
print(f"Extracted {len(results.extracted_files)} files:")
|
||||
for filename, path in results.extracted_files.items():
|
||||
print(f" - {filename}: {path}")
|
||||
|
||||
if results.main_file:
|
||||
print(f"\nMain file: {results.main_file}")
|
||||
file_type = os.path.splitext(results.main_file)[1].lower()
|
||||
if file_type == '.html':
|
||||
print(f"You can open this HTML file in a browser to play the Tetris game")
|
||||
else:
|
||||
print(f"This is the main entry point for your application")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,52 @@
|
||||
pytest
|
||||
pytest-cov
|
||||
pytest-mock
|
||||
pytest-asyncio
|
||||
pytest-subtests
|
||||
pytest-json-report
|
||||
ruff
|
||||
|
||||
sympy
|
||||
stopit
|
||||
scipy
|
||||
setuptools
|
||||
tree_sitter
|
||||
tree_sitter_python
|
||||
antlr4-python3-runtime==4.11
|
||||
tenacity
|
||||
networkx>=3.3
|
||||
nltk>=3.9.1
|
||||
numpy>=1.26.4
|
||||
openai>=1.55.3
|
||||
litellm>=1.55.6
|
||||
# pydantic>=2.9.0
|
||||
pydantic>=2.9.0,<=2.10.6
|
||||
pydantic-settings==2.8.1
|
||||
pydantic_core>=2.23.2,<=2.27.2
|
||||
loguru>=0.7.3
|
||||
pandas>=2.2.3
|
||||
matplotlib>=3.10.0
|
||||
# --extra-index-url https://download.pytorch.org/whl/cu118
|
||||
# torch==2.2.1
|
||||
# torchvision==0.17.1
|
||||
# torchaudio==2.2.1
|
||||
transformers>=4.47.1
|
||||
datasets>=3.4.0
|
||||
faiss-cpu==1.8.0.post1
|
||||
textgrad>=0.1.8
|
||||
|
||||
# fast api dependencies
|
||||
fastapi>=0.115.11
|
||||
motor>=3.7.0
|
||||
uvicorn>=0.34.0
|
||||
sqlalchemy>=2.0.38
|
||||
python-jose>=3.3.0
|
||||
passlib>=1.7.4
|
||||
python-multipart>=0.0.6
|
||||
bcrypt>=4.0.1
|
||||
celery>=5.3.4
|
||||
redis>=5.0.0
|
||||
httpx>=0.24.1
|
||||
asgi-lifespan>=1.0.1
|
||||
python-dotenv>=1.0.0
|
||||
jwt>=1.3.1
|
||||
Reference in New Issue
Block a user