How Information Flows Through Your Automations

Think of your automations like a factory assembly line. Information starts at one end, gets processed and improved at each station, and comes out as a finished product at the end. Let’s learn how this “information assembly line” works!

What You’ll Learn

By the end of this guide, you’ll understand:

How information moves from one automation step to the next (like a conveyor belt)
What happens to your information as it gets processed (like how raw materials become finished products)
How to see what’s happening when your automation runs (like watching the assembly line work)
How to fix problems when information doesn’t flow correctly
How to make your automations more efficient and reliable

Before You Start

You should have:

✅ Built your first automation (First Workflow guide here)
✅ 30 minutes to learn the concepts
✅ Curiosity about how your automations actually work behind the scenes

💡 Think of information flow like: A recipe where each step takes the results from the previous step and does something new with it. Flour + eggs = dough, dough + heat = bread, bread + butter = toast!

Understanding Your Information Assembly Line

How Information Moves Between Steps

In your automations, information flows like items on a conveyor belt:

Your Selection → Step 1 → Improved Information → Step 2 → Final Result

Each step (node) in your automation:

Receives information from the previous step
Does something useful with that information (cleans it, adds to it, organizes it)
Passes the improved information to the next step

Types of Information Your Automations Handle

📝 Text Information

What it is: Words, sentences, paragraphs you select from websites
Example: “The best pizza in Chicago is at Tony’s Restaurant”

📊 Organized Information

What it is: Information arranged in a structured way (like a digital filing cabinet)

Example:

Text: "The best pizza in Chicago is at Tony's Restaurant"
Word Count: 9 words
Date Captured: March 15, 2024
Source Website: food-blog.com

📋 Lists of Information

What it is: Multiple items collected together
Example: A list of all links on a webpage, or all images, or all phone numbers

🔢 Numbers and Measurements

What it is: Counts, calculations, measurements
Example: Number of words, number of links found, reading time estimate

Real Example: Watching Information Transform

Let’s follow some text through a simple automation:

🎬 Step 1: You select text

"The quick brown fox jumps over the lazy dog."

🎬 Step 2: Text Grabber captures it

Selected Text: "The quick brown fox jumps over the lazy dog."
Where Found: In a paragraph on the webpage
Position: Characters 26-70 on the page
Surrounding Context: "This is a sample paragraph. The quick brown fox jumps over the lazy dog. It contains all letters."

🎬 Step 3: Text Processor improves it

Original Text: "The quick brown fox jumps over the lazy dog."
Word Count: 9 words
Character Count: 44 characters
Contains Numbers: No
Date Captured: March 15, 2024 at 2:30 PM
Source: sample-paragraph

🎬 Step 4: File Saver prepares final result

Filename: "text-extraction-2024-03-15.json"
File Size: 245 bytes
Ready to Download: Yes
Contains: All the information from previous steps, organized and ready to save

Visual Data Flow Examples

Example 1: Simple Text Processing

Let’s trace data through a simple text extraction workflow:

Workflow Structure:

GetSelectedText → EditFields → Filter → DownloadAsFile

Step-by-step data transformation:

GetSelectedText Output:

{
  "selectedText": "The quick brown fox jumps over the lazy dog.",
  "context": "This is a sample paragraph. The quick brown fox jumps over the lazy dog. It contains all letters of the alphabet.",
  "element": {
    "tagName": "P",
    "id": "sample-text",
    "className": "demo-paragraph"
  },
  "position": {
    "start": 26,
    "end": 70
  }
}

EditFields Processing:

{
  "selectedText": "The quick brown fox jumps over the lazy dog.",
  "wordCount": 9,
  "characterCount": 44,
  "hasNumbers": false,
  "extractedAt": "2024-01-15T10:30:45.123Z",
  "source": "demo-paragraph",
  "context": "This is a sample paragraph. The quick brown fox jumps over the lazy dog. It contains all letters of the alphabet."
}

Filter Node (word count > 5):

{
  "selectedText": "The quick brown fox jumps over the lazy dog.",
  "wordCount": 9,
  "characterCount": 44,
  "passed": true,
  "filterReason": "Word count (9) exceeds minimum (5)"
}

DownloadAsFile Output:

{
  "filename": "text-extraction-2024-01-15.json",
  "content": "{ ... all previous data ... }",
  "size": 245,
  "downloadReady": true
}

Example 2: Multi-Source Data Collection

Workflow Structure:

GetAllLinks → EditFields → Merge ← GetAllImages → EditFields
     ↓                              ↓
DownloadAsFile ← Filter ← EditFields ← Merge

Data flow explanation:

Parallel Processing:
- GetAllLinks extracts all page links
- GetAllImages extracts all page images
- Both process simultaneously

Data Merging:

// Links data:
{
  "links": [
    {"url": "https://example.com", "text": "Example Link"},
    {"url": "https://test.com", "text": "Test Link"}
  ],
  "linkCount": 2
}

// Images data:
{
  "images": [
    {"src": "image1.jpg", "alt": "Sample Image"},
    {"src": "image2.png", "alt": "Another Image"}
  ],
  "imageCount": 2
}

// Merged result:
{
  "pageAssets": {
    "links": [...],
    "images": [...],
    "totalAssets": 4,
    "extractedAt": "2024-01-15T10:30:45.123Z"
  }
}

Common Data Transformation Patterns

Pattern 1: Data Enrichment

Purpose: Add metadata and calculated fields to extracted data

Implementation:

// Input: Basic text extraction
{
  "text": "JavaScript is a programming language"
}

// EditFields configuration:
{
  "operations": [
    {
      "field": "wordCount",
      "action": "calculate",
      "expression": "{{$json.text.split(' ').length}}"
    },
    {
      "field": "language",
      "action": "detect",
      "method": "keyword-analysis"
    },
    {
      "field": "category",
      "action": "classify",
      "rules": [
        {"contains": "programming", "category": "technology"},
        {"contains": "recipe", "category": "cooking"}
      ]
    }
  ]
}

// Output: Enriched data
{
  "text": "JavaScript is a programming language",
  "wordCount": 5,
  "language": "english",
  "category": "technology",
  "enrichedAt": "2024-01-15T10:30:45.123Z"
}

Pattern 2: Data Filtering and Validation

Purpose: Remove unwanted data and validate content quality

Implementation:

// Input: Mixed quality links
{
  "links": [
    {"url": "https://example.com", "text": "Good Link"},
    {"url": "javascript:void(0)", "text": ""},
    {"url": "https://spam.com", "text": "Click Here!!!"},
    {"url": "https://quality-site.com", "text": "Quality Content"}
  ]
}

// Filter configuration:
{
  "conditions": [
    {"field": "url", "operator": "not_contains", "value": "javascript:"},
    {"field": "text", "operator": "not_empty"},
    {"field": "text", "operator": "not_contains", "value": "!!!"}
  ]
}

// Output: Filtered data
{
  "validLinks": [
    {"url": "https://example.com", "text": "Good Link"},
    {"url": "https://quality-site.com", "text": "Quality Content"}
  ],
  "filteredCount": 2,
  "removedCount": 2
}

Pattern 3: Data Aggregation

Purpose: Combine and summarize data from multiple sources

Implementation:

// Input: Multiple text extractions
[
  {"text": "First paragraph", "wordCount": 2},
  {"text": "Second paragraph with more words", "wordCount": 5},
  {"text": "Third paragraph", "wordCount": 2}
]

// Aggregation logic:
{
  "totalParagraphs": 3,
  "totalWords": 9,
  "averageWordsPerParagraph": 3,
  "longestParagraph": "Second paragraph with more words",
  "combinedText": "First paragraph. Second paragraph with more words. Third paragraph."
}

Debugging Data Flow

Using the Execution Panel

Step-by-step debugging:

Enable Debug Mode
- Click “Settings” in the workflow toolbar
- Enable “Debug Mode” and “Detailed Logging”
- Save settings and re-execute workflow
Monitor Node Execution
- Watch nodes light up during execution
- Green = successful processing
- Yellow = processing in progress
- Red = error occurred
Inspect Node Data
- Click on any executed node
- View “Input Data” and “Output Data” tabs
- Compare expected vs. actual data structures

Common Data Flow Issues

Issue 1: Undefined or Null Data

Symptoms:

// Expected data:
{"text": "Sample text", "wordCount": 2}

// Actual data:
{"text": null, "wordCount": undefined}

Causes:

Previous node failed to extract data
Incorrect field references in expressions
Missing error handling

Solutions:

// Add data validation:
{
  "field": "text",
  "action": "validate",
  "rules": [
    {"required": true},
    {"minLength": 1}
  ],
  "fallback": "No text extracted"
}

Issue 2: Data Type Mismatches

Symptoms:

// Expected: Number
{"wordCount": 5}

// Actual: String
{"wordCount": "5"}

Solutions:

// Type conversion in EditFields:
{
  "field": "wordCount",
  "action": "convert",
  "toType": "number",
  "fallback": 0
}

Issue 3: Missing Data Properties

Symptoms:

Nodes expecting certain fields that don’t exist
Workflow fails at specific transformation steps

Solutions:

// Defensive data handling:
{
  "field": "safeWordCount",
  "action": "calculate",
  "expression": "{{($json.text || '').split(' ').length}}"
}

Data Flow Visualization

Creating data flow diagrams:

[Web Page]
    ↓ (HTML content)
[GetAllText]
    ↓ (raw text array)
[EditFields: Clean & Process]
    ↓ (structured text objects)
[Filter: Quality Check]
    ↓ (validated text objects)
[EditFields: Add Metadata]
    ↓ (enriched text objects)
[DownloadAsFile]
    ↓ (JSON file)
[User's Computer]

Best Practices for Data Flow

1. Design for Data Consistency

Standardize data structures:

// Good: Consistent structure
{
  "content": "...",
  "metadata": {
    "extractedAt": "...",
    "source": "...",
    "type": "..."
  },
  "metrics": {
    "wordCount": 0,
    "characterCount": 0
  }
}

// Avoid: Inconsistent structures
{"text": "...", "words": 5}  // First node
{"content": "...", "wordCount": 5}  // Second node

2. Implement Error Handling

Graceful failure patterns:

// Add error handling to transformations:
{
  "field": "processedText",
  "action": "process",
  "onError": "continue",
  "fallback": "Processing failed",
  "logError": true
}

3. Optimize Data Size

Minimize data transfer:

// Good: Only pass necessary data
{
  "essentialData": {
    "text": "...",
    "wordCount": 5
  }
}

// Avoid: Passing unnecessary large objects
{
  "text": "...",
  "fullPageHTML": "<html>...</html>", // Usually not needed
  "allPageImages": [...] // May be very large
}

4. Use Meaningful Field Names

Clear naming conventions:

// Good: Descriptive names
{
  "extractedText": "...",
  "cleanedText": "...",
  "finalProcessedText": "..."
}

// Avoid: Ambiguous names
{
  "text1": "...",
  "text2": "...",
  "result": "..."
}

Practical Exercise: Building a Data Flow

Let’s build a workflow that demonstrates complex data flow:

Exercise: Article Analysis Workflow

Goal: Extract and analyze article content with comprehensive data flow

Workflow Design:

GetAllText → EditFields (clean) → Filter (quality) → EditFields (analyze) → DownloadAsFile
     ↓
GetAllLinks → EditFields (validate) → Merge → EditFields (combine) ↗

Step 1: Set up the workflow

Create a new workflow named “Article Analysis”
Add the nodes as shown in the diagram above
Connect them according to the flow

Step 2: Configure data processing

GetAllText configuration:

{
  "includeHidden": false,
  "preserveFormatting": true,
  "excludeNavigation": true
}

EditFields (clean) configuration:

{
  "operations": [
    {
      "field": "cleanText",
      "action": "clean",
      "removeExtraSpaces": true,
      "removeSpecialChars": false
    },
    {
      "field": "wordCount",
      "action": "calculate",
      "expression": "{{$json.cleanText.split(' ').length}}"
    }
  ]
}

Filter (quality) configuration:

{
  "conditions": [
    {"field": "wordCount", "operator": "greater_than", "value": 50},
    {"field": "cleanText", "operator": "not_empty"}
  ]
}

Step 3: Test and debug

Execute the workflow on a news article
Inspect data at each node
Verify data transformations are working correctly

Expected final output:

{
  "article": {
    "text": "...",
    "wordCount": 245,
    "readingTime": "2 minutes",
    "quality": "high"
  },
  "links": {
    "internal": 5,
    "external": 12,
    "total": 17
  },
  "analysis": {
    "extractedAt": "2024-01-15T10:30:45.123Z",
    "processingTime": "1.2 seconds",
    "dataQuality": "excellent"
  }
}

Next Steps

You now understand how data flows through browser automation workflows. You’re ready to:

Build Intermediate Workflows - Create more complex data processing workflows
Learn Error Handling - Handle data flow errors gracefully
Explore Advanced Patterns - Master advanced data transformation techniques

Additional Resources

EditFields Node Reference - Complete data transformation documentation
Filter Node Guide - Data filtering and validation
Merge Node Documentation - Combining data from multiple sources

Estimated Time: 35-40 minutes Difficulty: Beginner Prerequisites: Completed first workflow tutorial