Skip to content
Agentic WorkFlow

Web LLM

Web LLM

Section titled “Web LLM”

Use LLM locally directly on your device

What it does

Section titled “What it does”

Runs language models locally in the browser using WebGPU for hardware acceleration.

When to use it

Section titled “When to use it”

Connect this to AI agents for privacy-focused, offline AI that runs entirely in the browser without external dependencies.

Manage which models are installed, pre-download them, and free up space on the Local Models page.

Using it with tools

Section titled “Using it with tools”

Web LLM works with the Tools Agent. WebLLM models have no built-in tool API, so tool calling goes through the prompt, the same way as for Transformers Chat:

  1. The connected tools are described in the system prompt.
  2. The model calls a tool by writing the call in its reply; the common formats models use are all understood (<tool_call>{…}</tool_call>, [TOOL_CALLS] […], <function=name>{…}</function>, fenced JSON, or a reply that is only the JSON call).
  3. The agent runs the tool and sends the result back as a <tool_response> message; the model then answers or calls another tool.

How reliably a model calls tools depends on the model: instruct models trained for tool use do much better than very small ones. A call to a tool that isn’t connected, or with invalid JSON, is kept as plain text instead of being run.

Inputs and settings

Section titled “Inputs and settings”
SettingNotes
ModelThe in-browser model to run. The picker marks installed models and shows them first; others show an estimated size and can be downloaded right from the dropdown (with progress). Use Manage models to open Local Models. Defaults to your chosen default local model.
TemperatureSampling temperature for the model (higher = more varied output).
OptionsAdditional model options.

Outputs

Section titled “Outputs”

Returns a chat model dependency that can be connected to AI agent nodes.

Dependencies and credentials

Section titled “Dependencies and credentials”
  • No explicit credential or node dependency is declared in the node description.

Example workflow

Section titled “Example workflow”

Connect Web LLM to an AI agent or dependency input that accepts this dependency type, then run the agent with data from previous nodes.

Troubleshooting

Section titled “Troubleshooting”
  • Check that required settings are present before running the node.
  • If the node uses browser page data, run it on the target tab after the page has loaded.
  • If it calls an external service, verify credentials, permissions, and rate limits.
  • No dedicated source test was found next to this node; verify behavior manually when changing this page.
Section titled “Related pages”