Web Content Analysis with AI

Web Content Analysis with AI

This example demonstrates how to create an AI workflow that analyzes web page content using Agentic WorkFlow’s browser extension capabilities combined with LangChain AI models.

Overview

This workflow extracts content from the current web page and uses AI to analyze, summarize, and extract insights from the content. It showcases the integration between browser context manipulation and AI processing.

Workflow Components

1. Content Extraction

• GetAllText Node: Extracts all text content from the current web page
• GetSelectedText Node: Captures user-selected text for focused analysis
• GetAllHTML Node: Retrieves HTML structure for more detailed analysis

2. AI Processing

• Basic LLM Chain: Processes the extracted content with AI models
• Text Splitter: Breaks down large content into manageable chunks
• Summarization Chain: Creates concise summaries of web content

3. Output Processing

• Structured Output Parser: Formats AI analysis results
• Content Insertion: Optionally inserts analysis results back into the page

Example Workflow

// Workflow: Web Content Analysis
// 1. Extract page content
const pageContent = await GetAllText.execute();

// 2. Process with AI
const analysis = await BasicLLMChain.execute({
  input: pageContent,
  prompt: "Analyze this web content and provide: 1) Main topics, 2) Key insights, 3) Summary"
});

// 3. Structure the output
const structuredAnalysis = await StructuredOutputParser.execute({
  input: analysis,
  schema: {
    topics: "array",
    insights: "array",
    summary: "string"
  }
});

Use Cases

Content Research

• Automatically analyze research papers or articles
• Extract key points from documentation
• Summarize lengthy blog posts or news articles

Competitive Analysis

• Analyze competitor websites for insights
• Extract feature lists and pricing information
• Monitor content changes over time

Content Quality Assessment

• Evaluate readability and structure
• Identify missing information or gaps
• Suggest content improvements

Browser Security Considerations

When implementing web content analysis workflows:

• Content Security Policy: Some sites may restrict AI API calls
• Rate Limiting: Implement delays between API calls to avoid rate limits
• Privacy: Be mindful of sensitive content when sending to AI models
• Permissions: Ensure proper browser extension permissions for content access

Advanced Patterns

Multi-Page Analysis

Combine content from multiple pages for comprehensive analysis:

// Collect content from multiple tabs
const allContent = await Promise.all([
  GetAllText.execute({ tabId: tab1 }),
  GetAllText.execute({ tabId: tab2 }),
  GetAllText.execute({ tabId: tab3 })
]);

// Analyze combined content
const comparativeAnalysis = await Agent.execute({
  input: allContent,
  tools: [VectorStoreTool, CalculatorTool],
  prompt: "Compare and contrast the content from these pages"
});

Real-Time Analysis

Set up workflows that trigger automatically when pages load:

// Auto-trigger on page load
browser.tabs.onUpdated.addListener((tabId, changeInfo) => {
  if (changeInfo.status === 'complete') {
    triggerContentAnalysis(tabId);
  }
});

Integration with Vector Stores

Store analysis results for future reference:

// Store analysis in vector database
await VectorStore.execute({
  documents: [{
    content: structuredAnalysis.summary,
    metadata: {
      url: currentPage.url,
      timestamp: Date.now(),
      topics: structuredAnalysis.topics
    }
  }]
});

This enables building a knowledge base of analyzed content that can be queried later for insights and patterns.