ISHAN BHARATH

SHORTCOMING: BROKEN ARCHITECTURE

First attempt at building a custom conversational agent using an LSTM‑based sequence model. Explored tokenisation, seq2seq training, and dialogue flows, but incorrect loss (MSE on token IDs) and mismatched output layers caused unstable training and unusable responses.

SHORTCOMING: DATA OVERFITTING

Rebuilt MIA on top of GPT‑2 (124M parameters) to gain stronger language understanding. Fine‑tuned on ~150 curated dialogue pairs, which gave good performance on training data but severe overfitting in real use, highlighting the need for larger datasets and better regularisation/evaluation.

SCOPE: LOCKED & REALISTIC

A focused planner that converts exam dates and topic weights into a concrete, daily revision schedule. You define subjects, topics, and exams; the system splits remaining days into time slots so each topic gets proportional attention, instead of last‑minute panic.

Frontend is plain HTML/CSS to keep things simple. Backend is Python (Flask/FastAPI) with SQLite, giving a realistic but manageable stack for learning backend design, scheduling logic, and data modelling.

CURRENT PHASE (Summer 2026)

• Design REST endpoints for subjects, exams, topics, and weights.
• Model data cleanly in SQLite with relationships between entities.
• Implement schedule generation based on remaining days and weights.
• Expose a minimal HTML/CSS UI to create and review the plan.

SEM 3 UPGRADE (Not now)

• Containerise the app with Docker.
• Deploy to AWS EC2.
• Add user auth with bcrypt‑based password hashing.
• Set up basic uptime and health monitoring.

This remains a learning project with tightly controlled scope: learn backend fundamentals, not chase features.

GOAL: 6‑STAGE ML PIPELINE

An end‑to‑end system that ingests financial news and market data, computes sentiment and technical indicators, then trains an XGBoost model with proper time‑series validation to estimate the probability a stock closes up. Outputs feed into a Streamlit dashboard and paper‑trading layer.

CONFIRMED STACK

• Data: NewsAPI, yfinance, Reddit PRAW, NSE via nsepy/jugaad‑trader, ET/Moneycontrol RSS.
• Storage: SQLite locally; PostgreSQL on AWS in production.
• Processing: pandas, regex, spaCy NER for cleaning and entity extraction.
• Sentiment: VADER locally; FinBERT on GPU (EC2) in later phases.
• Features: pandas‑ta (RSI, MACD, Bollinger Bands) plus sector‑level sentiment.
• Model: XGBoost with TimeSeriesSplit (no shuffling of time).
• Dashboard: Streamlit + Plotly with Kelly‑style position sizing.
• Orchestration: APScheduler locally; Apache Airflow on AWS.
• Paper trading: Alpaca or Zerodha Kite sandbox.
• Secrets: python‑dotenv locally; AWS Secrets Manager in production.

BUILD ORDER

• Phase 1: News fetcher → CSV → cleaning → binary labels.
• Phase 2: Core XGBoost pipeline with dummy features using TimeSeriesSplit.
• Phase 3: Add VADER, RSI/MACD/Bollinger Bands, sector sentiment features.
• Phase 4: Ship Streamlit dashboard, Airflow DAGs, EC2 + Secrets Manager, upgrade to FinBERT.

REALISTIC ACCURACY TARGETS

• VADER‑only features: ~54–56% directional accuracy.
• FinBERT + technical + price features: ~58–63% directional accuracy.
• Anything much higher on random splits usually signals data leakage.

Includes a dedicated db_manager.py for upserts and backfill, India‑specific NSE + Zerodha integration, and secret management designed for deployment from day one.