From 44730e2f42ee7e42bda3e1667dfe2ce63cf9d9e7 Mon Sep 17 00:00:00 2001 From: ShubhamSaboo Date: Sat, 11 May 2024 17:21:27 -0500 Subject: [PATCH] Added new demo --- README.md | 26 +++++++++++------ web_scrapping_ai_agent/README.md | 38 +++++++++++++++++++++++++ web_scrapping_ai_agent/ai_scrapper.py | 38 +++++++++++++++++++++++++ web_scrapping_ai_agent/requirements.txt | 3 ++ 4 files changed, 96 insertions(+), 9 deletions(-) create mode 100644 web_scrapping_ai_agent/README.md create mode 100644 web_scrapping_ai_agent/ai_scrapper.py create mode 100644 web_scrapping_ai_agent/requirements.txt diff --git a/README.md b/README.md index da0779d..8c13878 100644 --- a/README.md +++ b/README.md @@ -14,27 +14,35 @@
# 🌟 Awesome LLM Apps -A curated collection of awesome LLM apps using RAG with OpenAI, Anthropic, Gemini and opensource models. +A curated collection of awesome LLM apps built with RAG and AI agents. This repository features LLM apps that use models from OpenAI, Anthropic, Google, and even open-source models like LLaMA that you can run locally on your computer. -## 📂 Projects +## 🤔 Why Awesome LLM Apps? +- 💡 Discover practical and creative ways LLMs can be applied across different domains, from code repositories to email inboxes and more. +- 🔥 Explore apps that combines LLMs from OpenAI, Anthropic, Gemini, and open-source alternatives with RAG and AI Agents. +- 🎓 Learn from well-documented projects and contribute to the growing opensource ecosystem of LLM-powered applications. + +## 📂 Featured Projects ### 💻 Local Lllama-3 with RAG -Streamlit app that allows you to chat with any webpage using local Llama-3 and Retrieval Augmented Generation (RAG). This app runs entirely on your computer, making it 100% free and without the need for an internet connection. +Chat with any webpage using local Llama-3 and Retrieval Augmented Generation (RAG) in a Streamlit app. Enjoy 100% free and offline functionality. ### 💬 Chat with GitHub Repo -Engage in natural conversations with your GitHub repositories using GPT-4. Uncover valuable insights and documentation effortlessly, as if you were chatting with your codebase itself. +Engage in natural conversations with your GitHub repositories using GPT-4. Uncover valuable insights and documentation effortlessly. ### 📨 Chat with Gmail Interact with your Gmail inbox using natural language. Get accurate answers to your questions based on the content of your emails with Retrieval Augmented Generation (RAG). ### 📝 Chat with Substack Newsletter -Streamlit app that allows you to chat with a Substack newsletter using OpenAI's API and the Embedchain library. This app leverages GPT-4 to provide accurate answers to questions based on the content of the specified Substack newsletter. +Chat with a Substack newsletter using OpenAI's API and the Embedchain library in a Streamlit app. Leverage GPT-4 for precise answers based on newsletter content. ### 📄 Chat with PDF -Engage in intelligent conversation and question-answering based on the content of your PDF documents. Simply upload a PDF document and start asking questions. Chat with PDF will analyze the document, extract relevant information, and generate accurate responses to your queries. +Engage in intelligent conversation and question-answering based on the content of your PDF documents. Simply upload and start asking questions. ### 📽️ Chat with YouTube Videos -Dive into the world of video content with interactive conversation and question-answering based on YouTube videos. Simply provide a YouTube video URL and start engaging with the video's content through natural language. +Dive into video content with interactive conversation and question-answering based on YouTube videos. Provide a URL and engage with the video's content through natural language. + +### 💻 Web Scraping AI Agent +Intelligently scrape websites using OpenAI API and the scrapegraphai library. Specify the URL and extraction requirements, and let the AI agent handle the rest. ## 🚀 Getting Started @@ -59,5 +67,5 @@ pip install -r requirements.txt 4. Follow the project-specific instructions in each project's README.md file to set up and run the app. -### 🤝 Contributing -Contributions are welcome! If you have any ideas, improvements, or new apps to add, please submit a pull request. Make sure to follow the existing project structure and include a detailed README.md for each new app. +## 🤝 Contributing to Opensource +Contributions are welcome! If you have any ideas, improvements, or new apps to add, please create a new [GitHub Issue](https://github.com/Shubhamsaboo/awesome-llm-apps/issues) or submit a pull request. Make sure to follow the existing project structure and include a detailed README.md for each new app. diff --git a/web_scrapping_ai_agent/README.md b/web_scrapping_ai_agent/README.md new file mode 100644 index 0000000..4932fe4 --- /dev/null +++ b/web_scrapping_ai_agent/README.md @@ -0,0 +1,38 @@ +## 💻 Web Scrapping AI Agent +This Streamlit app allows you to scrape a website using OpenAI API and the scrapegraphai library. Simply provide your OpenAI API key, enter the URL of the website you want to scrape, and specify what you want the AI agent to extract from the website. + +### Features +- Scrape any website by providing the URL +- Utilize OpenAI's LLMs (GPT-3.5-turbo or GPT-4) for intelligent scraping +- Customize the scraping task by specifying what you want the AI agent to extract + +### How to get Started? + +1. Clone the GitHub repository + +```bash +git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git +``` +2. Install the required dependencies: + +```bash +pip install -r requirements.txt +``` +3. Get your OpenAI API Key + +- Sign up for an [OpenAI account](https://platform.openai.com/) (or the LLM provider of your choice) and obtain your API key. + +4. Run the Streamlit App +```bash +streamlit run ai_scrapper.py +``` + +### How it Works? + +- The app prompts you to enter your OpenAI API key, which is used to authenticate and access the OpenAI language models. +- You can select the desired language model (GPT-3.5-turbo or GPT-4) for the scraping task. +- Enter the URL of the website you want to scrape in the provided text input field. +- Specify what you want the AI agent to extract from the website by entering a user prompt. +- The app creates a SmartScraperGraph object using the provided URL, user prompt, and OpenAI configuration. +- The SmartScraperGraph object scrapes the website and extracts the requested information using the specified language model. +- The scraped results are displayed in the app for you to view \ No newline at end of file diff --git a/web_scrapping_ai_agent/ai_scrapper.py b/web_scrapping_ai_agent/ai_scrapper.py new file mode 100644 index 0000000..891e069 --- /dev/null +++ b/web_scrapping_ai_agent/ai_scrapper.py @@ -0,0 +1,38 @@ +# Import the required libraries +import streamlit as st +from scrapegraphai.graphs import SmartScraperGraph + +# Set up the Streamlit app +st.title("Web Scrapping AI Agent 🕵️‍♂️") +st.caption("This app allows you to scrape a website using OpenAI API") + +# Get OpenAI API key from user +openai_access_token = st.text_input("OpenAI API Key", type="password") + +if openai_access_token: + model = st.radio( + "Select the model", + ["gpt-3.5-turbo", "gpt-4"], + index=0, + ) + graph_config = { + "llm": { + "api_key": openai_access_token, + "model": model, + }, + } + # Get the URL of the website to scrape + url = st.text_input("Enter the URL of the website you want to scrape") + # Get the user prompt + user_prompt = st.text_input("What you want the AI agent to scrae from the website?") + + # Create a SmartScraperGraph object + smart_scraper_graph = SmartScraperGraph( + prompt=user_prompt, + source=url, + config=graph_config + ) + # Scrape the website + if st.button("Scrape"): + result = smart_scraper_graph.run() + st.write(result) \ No newline at end of file diff --git a/web_scrapping_ai_agent/requirements.txt b/web_scrapping_ai_agent/requirements.txt new file mode 100644 index 0000000..bc34f2b --- /dev/null +++ b/web_scrapping_ai_agent/requirements.txt @@ -0,0 +1,3 @@ +streamlit +scrapegraphai +playwright \ No newline at end of file