In this article I show how to build a voice-powered AI agentic app to automate business tasks and analyze live data warehouses.
Chat is just one of the many things LLMs can do…They can act and execute.
𝐕𝐓𝐄𝐗𝐄𝐑: 𝐕𝐨𝐢𝐜𝐞-𝐄𝐧𝐚𝐛𝐥𝐞𝐝 𝐋𝐋𝐌 𝐀𝐜𝐭𝐢𝐨𝐧 𝐀𝐠𝐞𝐧𝐭 𝐀𝐩𝐩 𝐟𝐨𝐫 𝐓𝐚𝐬𝐤 𝐄𝐱𝐞𝐜𝐮𝐭𝐢𝐨𝐧 Automation Query and Research. I demonstrate how you can, with natural language voice instructions:
- Update Excel Sheet- Update Google Sheet- Update records in remote Data Warehouses- Query, Analyze and Transform data in remote Data Warehouses- Generate and email print formatted report in PDF format- Generate and email deck format- Carry out custom web search- Provide user menu option to choose amongst multiple LLM Agents
And yes, it can also chat.
Detailed, step-by-step, hands-on implementation guide, source code, GitHub repo, Schemas, Blueprints shared in sections below.
The source deployed as-is on Vercel etc. will give you a fully functional voice-bot right off the bat (with restricted app features)
𝐓𝐫𝐲 𝐃𝐞𝐦𝐨 𝐀𝐩𝐩 (with restricted features)vtexer-shared.tigzig.com.Use demo OpenAI API key at: tigzig.com/ap (password: genai123).
By leveraging platforms like Flowise AI and Make.com, you can interact with and execute tasks in voice and natural language text with:- your business data warehouses,- thousands of platforms (Zoho, QuickBooks, Salesforce, AWS, Google, Microsoft, Azure, Slack HubSpot, Stripe, Woo Commerce, Shopify….)- and most importantly your own existing API’sThe implementations are relatively rapid, with part of it being no-code, and any coding being done by AI-assisted coding tools.
Andrej Karpathy on X “…most of my “programming” is now writing English…I basically can’t imagine going back to “unassisted” coding at this point…”
As part of the series, I show you how to use AI Assisted Coding Tools like Claude Dev and Cursor AI to develop LLM Apps with natural language commands. And deploy to open internet.
𝐓𝐡𝐞 𝐚𝐩𝐩 𝐮𝐬𝐞𝐬 𝐋𝐋𝐌 𝐑𝐞𝐀𝐜𝐭 𝐀𝐠𝐞𝐧𝐭𝐬, Reasoning and Action Agents. Called as such since they can execute tasks via API calls, also called Function Calls / Tool Calling. Fairly easy to setup in Flowise. Flowise has a marvelous functionality to access an agent / chat flow via an API endpoint. Allows you to fully customize the UI and response flow. Using that extensively here
- Custom Frontend: React.js- Rapid Deploy Frontend: Flowise- LLM ReAct Agents: Flowise (Flowiseai.com)- Workflow: Make.com- Automation Scripts: Google Script- AWS MySQL DB Connection: Flowise ReAct Agent & Custom built FastAPI Server
𝐀𝐋𝐋 𝐜𝐨𝐝𝐢𝐧𝐠 𝐝𝐨𝐧𝐞 𝐛𝐲 𝐆𝐏𝐓, 𝐋𝐋𝐌𝐬 & 𝐀𝐈 𝐀𝐬𝐬𝐢𝐭𝐞𝐝 𝐂𝐨𝐝𝐢𝐧𝐠 𝐓𝐨𝐨𝐥𝐬. For Google Scripts and Python I like GPT-4o. For React in general it is Claude Sonnet. For React.js apps like this one, my current favorites are Claude Dev, a VS Code extension and Cursor AI IDE, a VS Code fork. Claude Dev works on Github Codespaces also… I have a bit of preference for cloud over local. Both are able to create new files as well as modify files across whole codebase. Claude Dev even has terminal access, so does pretty much everything.
Hands on Implementation Guide
Video guide
With step by step instructions. Complete Playlist at : https://www.youtube.com/watch?v=Efn2hdLOPiU&list=PL4CpDSkww6gAuwuA-0CxJguDUnleDbn50
📌Part 1: Demo and Agent Process Flow
📌Part 2 : How to update Excel, Google Sheets and Databases with AI / LLM Voice Agents
📌Part 3: How To Automate Excel to PDF, Excel to Slides, Automatic Email with AI Agents and Google Scripts
📌Part 4: How to build AI App with natural language with Claude Dev and Cursor AI
📌Part 5: How to create AI/ LLM Agents that query databases, do web search and take action
𝐒𝐨𝐮𝐫𝐜𝐞 𝐜𝐨𝐝𝐞, JSON Schemas and blueprints onGitHub:
The repo, deployed ASIS to Vercel / Netlify etc will give you a fully functional voice-bot (with restricted app features)
ANALYZER App: Build Analytics Assistant LLM AppAWS & Azure MySQL Example . Guide on Medium. Free access:https://medium.com/@amarharolikar/build-an-analytics-assistant-app-flowise-ai-text-to-sql-fastapi-01d8378243b4?source=friends_link&sk=48b9edc66f16ea2fe40be044deb0ed8f
𝐓𝐨𝐩 𝐅𝐥𝐨𝐰𝐢𝐬𝐞 𝐀𝐈 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞:
Leon Van Zyl’s videos are a must-watch — from simple chatbots to complex agentic flows.Leon van Zyl — YouTube
𝐓𝐨𝐩 Make.com Resource:
Nick Sarev’s channel has brilliant video tutorials on Make.comNick Saraev — YouTube
AI Assisted Coding: Must Read
A very practical view about how best to use AI Assisted coding, it’s limitations and how to manage those. With lots of practical suggestions.Software Engineers and IT Leaders are Dead Wrong about AI (youtube.com)
Implementation Videos with Time-Stamped Sections
Complete Playlist at :
Part 1: Demo and Agent Process Flow
00:02 — Introduction and LLM Agent Capabilities
01:10 — LLM Agent Demo: Updating Excel and Google Sheets
03:31 — Generating Report and Deck Formats
05:12 — Data SQL Analyst and Other Functionalities
08:06 — Overview of Architecture and Flow
09:47 — Tracing the Flow of the Agent
14:01 — Details on Data SQL Analyst and General Analyst
17:50 — Build Process and Implementation Overview
18:51 — Breakdown of the Four-Part Guide
Part 2 : How to update Excel, Google Sheets and Databases with AI / LLM Voice Agents
00:00 — Overview and Introduction
03:55 — Setting up Flowise and Learning Resources
05:10 — Top Tips for Working with GPT
10:10 — Building the Chat Flow in Flowise
11:30 — Custom Tool for Function Calls in Flowise
17:15 — Connecting Flow Wise to Make.com
20:10 — Testing the Flow and Updating Google Sheets
22:54 — Connecting and updating Google Sheets and Excel Sheets
29:10 — Connecting Flow Wise to AWS RDS MySQL
35:00 — Adding Voice Functionality to the Flowise Bot
38:00 — Connecting Flow Wise to a React Frontend
41:00 — Deploying the React Frontend Using Vercel
43:00 — Conclusion and Next Steps
Part 3: How To Automate Excel to PDF, Excel to Slides, Automatic Email with AI Agents and Google Scripts
00:03 — Introduction and Recap
00:36 — Creating Automation Scripts in Google Script
07:38 — Deploying Google Script and connecting to Make.com
10:38 — Configuring Triggers and Filters in Make.com
14:26 — Demonstrating the Functioning Web App and Chatbot
17:29 — Connecting React Frontend to Flowwise
21:10 — Deploy and Go Live with Vercel
Part 4: How to Build AI Voice Action Agent App with natural language with Claude Dev and Cursor AI
00:04 — Introduction to Part Four and Custom Frontend
00:36 — Deploying the Voicebot on a Website and Repository Setup
01:36 — Overview of Voicebot Components: Voice to Text, Chat Completion, Text to Speech
04:18 — Building the Demo App with Claude Dev and Cursor AI
15:26 — Deploying the Voicebot App Live on Vercel
16:38 — Connecting GitHub Repository to Vercel for Deployment
17:12 — Final Testing of Live Voicebot App
19:03 — Demonstrating Multilingual Support in Voicebot
20:21 — Differentiating Between Transcription and Translation in API Usage
21:22 — Overview of API Endpoints for Speech and Text Processing
21:49 — Routing Text to LLM Agent for Response Generation
23:58 — Finalizing API Integrations for Voicebot Functionality
25:48 — Conclusion and Next Steps for Voicebot Enhancement
Part 5: How to create LLM Agents that query databases, do web search and take action
00:04 — Introduction and Overview
00:32 — Analyst Team and LLM Agents
04:16 — Configuring Perplexity Search Tool
15:09 — Implementing Session IDs in React
16:08 — Demonstrating Session Memory
18:07 — Integrating Data SQL and Document PDF Analysts
23:50 — Pushing to GitHub and Deploying to Vercel
25:42 — Conclusion
Hope you find the guide useful and enjoy using it as much as I enjoyed making it.