INTELISCAPE-X: AI-Powered Web Scraping in Excel

This xlwings Lite app turns Excel into a web scraping platform:

• Extract structured data from multiple websites
• Process batches of URLs with custom column specifications
• Use AI to identify and extract data elements from web pages
• Apply custom filtering instructions to refine results
• Live status per URL while it runs; finished URLs are kept, so a run can be resumed
• Rows per URL (VALIDATION), errors and token usage (DASHBOARD)
• Results go straight into Excel tables

1. Download & open: get the workbook and open xlwings Lite (Home > Add-ins). The HOW_IT_WORKS sheet explains every step; the button is on the MASTER sheet.
2. Add a Gemini API key: free at aistudio.google.com. In xlwings Lite, Settings > Environment Variables, add GEMINI_API_KEY with scope Local, then restart xlwings Lite. Jina Reader works without a key; add JINA_API_KEY only for higher limits.
3. Define extraction columns in COLUMN_INPUTS: column names, a description of each, and optional filtering instructions
4. Add target URLs in URL_LIST
5. Click "1. Scrape URLs": status per URL shows live in URL_LIST
6. Review results: DATA, VALIDATION, ERROR_LOG and DASHBOARD

The web scraper processes URLs through a three-stage pipeline:

Stage 1: Configuration

• xlwings Lite runs Python code directly in Excel via WebAssembly
• Reads the model and settings from MASTER; API keys come from xlwings Lite environment variables
• Loads column definitions from COLUMN_INPUTS

Stage 2: Data Extraction

• Sends each URL to Jina Reader (r.jina.ai), which returns the page as Markdown
• Passes rendered content to Gemini API with column specifications
• Gemini returns JSON that follows a schema built from your column list

Stage 3: Results

• Writes rows to DATA as each URL finishes; rows from finished URLs are kept across runs
• Logs errors to ERROR_LOG sheet with timestamps
• Generates DASHBOARD with performance metrics

Process Flow