Progress so far…

Travel AI

Project Developement

0. Project setup

Started Learning about flask

• Setting up Routes, passing information from the backend to frontend
• Jinja2 template for easy UI rendering
• Learnt basic bootstrap and referred documentation to get basic UI Setup

1. User Authentication

Learnt how user authentication systems work, how to store and validate the user and logout in the end

• Implemented a dummy Login, Register → Dashboard → Logout system to learn auth in Flask using cookies

• Some demo Pictures

  

  

  

• Implemented in my App with the following improvement

  • User should be Authorized to access any of the existed routes other register and login, Other wise should be redirected back to login
  • Stored user data in database with hashed password for security
  • Each user is assigned a session token which is checked before giving access to a route in the app
  • Logout clears the session from the list

• WTForms also provides form.hidden_tag() which includes security features such as CSRF token

2. AI Generated Plans

• Worked with GeminiAPI to make the necessary API calls
• Prompt Engineering
  • Prompt is devided into 2 parts
    1. Instruction and input format prompt
    2. JSON output format prompt
  • API call returns a string which is dumped into JSON for later use

3. Location Coordinates

• For plotting the position of the map, leaflet.js requires us to pass the latitude and longitude for each place. So get the accurate data we can’t use LLMs.
• GooglerSerperAPI uses Places API (and it’s free for 2500 tokens) to get the required data as JSON which is processed as needed

4. Mapping

• Using Leaflet mapping and routing engine.
• Easy to use library for rendering map
• For the map tiles it uses open street maps (which is Free and Open source)

5. Frontend

• Use community from Figma community to make the UI and get started form their
• Responsive Breakpoints and Dropdowns implemented

6. Storing Plans

• Used FlaskSQLAlechemy which is an ORM wrapper. It represent each record as a class so developers don’t have to write SQL queries and also they can change just change the db they are using without changing a single line which deals with database operations. This can be done by just changing the databse URI.
• Added Functionality to add event to collection.
• See added plans in the collections page.

7. AI Agent

• In the initial stages of the project, used only a single prompt with no tools to get the basic project running
• During the end of the project replaced it with an AI Agent build on top of Lang chain
• AI Agent built with langchain has the tools to do the following tasks
  • get accurate travel time and pricing.
  • gets accurate information about the places to visit
  • curates the plan according to the weather conditions and other Local parameters
• AI Markdown agent convert the output into proper markdown which makes the work of the next agent easier
• AI JSON agent finally reconciles and gives the output in JSON format which is then used to render it int he UI

8. Travel Plan

• Worked on prompt engineering
• Making sure that output is as structured as possible so that bad data is not passed on further
• This also used Google Serper to get the latitude and longitude of the places (Google places can be used but this was in use before i Setup GCP Account)
• Finally the JSON from AI JSON agent and the coordinates from Google Serper are sent for rendering

9. Blind Plan

• Worked with Google Autocomplete and Google Nearby Search and Google Place Photos api get the relevant datapoints
• Rendered the Required data for the user to decide
• Gave them the option to See the location on Google Maps website for more info, Select the location and generate more suggestion nearby that location
• On reaching MAX_LIMIT set by me, they can click on end plan to generate an AI plan for them which goes through the same Rendering pipeline

Problems Faced

1. Data sharing between frontend and backend

In flask it is very easy to pass information from Backend to Frontend which is based on Jinja2 templates. Which makes rendering very easy 😀. But sending any information to JS on the frontend was causing problem since Jinja templating didn’t work

So I had 2 problems to deal with

1. Wrap the data as block
2. Store it in a buffer which both Python and JS can access
3. Access it whenever required

So here’s what I did

1. Since the Plan data was well structures as a python dictionary, I wrapped the information as JSON block
2. Setting and accessing cookies is easy in both Python and JS
   1. For the buffer I am currently using Browser Cookies (Planning to move to Local storage soon)

Idea for the Mapping part

1. Scrappy solution

KEY: Using Google Serper which has 2500 Free searches which returned the latitude, longitude, name and some other info based on the prompt. It is backed by the Google Place AI internally so the values we get are not hallucination

1. Places data: Google Serper
2. Route and Plotting: Leaflet.js

<Under Construction>

Skills used

• Flask
  • flask-sqlalchemy
  • WTForms
  • session for Auth
  • cookies for Info storage
  • custom middleware for page restrictions
• API handling in flask
• Mapping
• AI Agents
• Web Dev
• Version control
• Excallli draw

Learnings

CSRF and injection

Tool Calling for LLMs

Checkpoints

• [x] Codebase setup
• [x] User Auth
• [x] Design + Frontend
• [x] Getting AI gen plan
• [x] Getting coordinates
• [x] Mapping
• [x] Final Plan with added info
• [x] Getting top K places
• [x] Final Plan for blind travel
• [x] User Dashboard + add ons
• [ ] Hosting the site (with limited usage)