Autonomous ML Pipeline with AI Agents

LangGraph agents that run an entire ML workflow autonomously.

LangGraphAgentic AIXGBoostSHAPOllamaScikit-learn

View code ↗

Output

Specialized agents execute the pipeline autonomously via LangGraph: data exploration, feature selection, baselines, and XGBoost tuning, each logging its decisions.

Feature selection by consensus: Lasso L1 importance and Mutual Information rank features; zero-variance and weak features are dropped.

Model comparison: tuned XGBoost (0.945 accuracy, 0.989 AUC) beats Lasso and Ridge baselines, with the classification report.

SHAP summary: global feature importance and per-sample impact on model output across the top features.

SHAP waterfall: instance-level explanation showing how each feature pushes a single prediction.

Problem

End-to-end ML work (exploration, feature engineering, modeling, evaluation) is repetitive and human-driven. Can specialized agents run the whole pipeline and explain every decision they make?

Approach

A LangGraph StateGraph of specialized agents: a Data agent handles exploration, cleaning, encoding, and feature selection; a Model agent trains and tunes an XGBoost baseline; an Eval agent computes metrics and SHAP explanations. A super-agent uses LLM reasoning to decide the next step and generates a human-readable run report. Every agent decision is logged in natural language.

Result

A fully autonomous pipeline that goes from raw data to a deployable XGBoost model (exported as Pickle), with global and instance-level SHAP interpretability and an LLM-generated summary of the entire run.

Highlights

LangGraph StateGraph orchestrating Data / Model / Eval / Super agents
LLM-driven control flow: the super-agent reasons about the next action
SHAP global + instance explanations for every run
Exports deployable models (Pickle) with a natural-language run report

Next projectLLM Evaluation & Hallucination Analysis →