Agentic Pilot
Agentic-Pilot is a powerful local agent that automates web tasks using natural language. It runs entirely on your machine using Ollama (qwen2.5 and qwen2.5-vl), ensuring your data and credentials never leave your local environment. Built with a LangGraph multi-step reasoning core, Playwright browser automation, and a native Tauri desktop shell.
Tech Stack & Infrastructure
The Problem
Browser automation today requires either cloud-based AI services that expose sensitive credentials and browsing data, or brittle rule-based scripts that break with every UI change.
The Solution
A fully local agentic system that combines LLM reasoning (via Ollama) with robust browser automation (Playwright), wrapped in a native desktop app (Tauri). Vision-Language Model fallback handles dynamic UIs, while ChromaDB provides long-term semantic memory.
Architecture Overview
The system consists of a LangGraph-powered reasoning core that decomposes natural language instructions into browser actions, a Playwright browser pool for headless execution with rich DOM parsing, a ChromaDB-backed memory engine for semantic and episodic recall, and a Tauri + React native desktop shell.
Engineering Decisions
Chose LangGraph over simple chain-of-thought for its native support of cycles, branching, and state persistence in agentic workflows. Tauri was selected over Electron for its smaller binary size and Rust-based security model.
Key Tradeoffs
Running vision-language models locally demands significant GPU resources. DOM-first parsing is faster but less robust against dynamic SPAs, hence the VLM fallback.
Core Challenges
Maintaining agent coherence across multi-step tasks where browser state changes unpredictably between actions, and tuning memory retrieval to surface relevant past experiences without noise.
Results & Impact
Achieved reliable end-to-end web task automation with post-task inspection, full local execution, and zero credential exposure to external services.
Future Roadmap
Add multi-tab parallel execution, support for authenticated workflows with encrypted credential vaults, and a visual task recording/replay builder.