# Your AI Coding History Is a Dataset. Here's How I'd Actually Use It.

Published: 2026-09-17 · Tag: AI
Canonical: https://vondraysanford.com/writing/2026-09-17-your-ai-coding-history-is-a-dataset-here-s-how-i-d-actually-.html
Author: Vondray Sanford (https://vondraysanford.com)

> The ai-data-extractor project can pull your entire Claude Code, Cursor, and Cline chat history into structured data. That's interesting. What you do with it afterward is where things get genuinely useful.

There's a project trending on GitHub right now called [ai-data-extractor](https://github.com/kruzovic7/ai-data-extractor) that pulls structured data out of your AI coding assistant chat histories — Claude Code, Cursor, Windsurf, Aider, Cline, and more. It's a Python tool, open source, free. Most of the conversation around it is "oh cool, I can export my chats." That's the wrong reaction. The right reaction is: *I've been generating a labeled dataset about my own engineering behavior for months and I didn't even realize it.*

I've been running Claude Code heavily as part of my [10X Engineer Toolkit](https://vondraysanford.com) work, so when I saw this tool I immediately started thinking about what those logs actually contain. Prompts, model responses, file diffs, error messages, follow-up corrections — that's not chat history. That's a structured record of every problem you hit, how you described it, and whether the model's first answer worked or needed three rounds of fixing.

## What the Data Actually Looks Like

When you extract your Claude Code session history, you're not getting a flat transcript. You're getting a sequence of turns that maps fairly cleanly to an engineering workflow: you describe a task, the model generates code, you either accept it or push back, and eventually something compiles and passes tests. Each correction turn is a signal.

Think about the shape of the data per session: initial prompt, number of follow-up corrections, nature of the corrections (wrong type, wrong logic, missed edge case, hallucinated API), and final resolution. If you've been using one of these tools for three to six months, you probably have hundreds of sessions. That's a meaningful sample size.

I did some research on what ai-data-extractor actually outputs and the schema is pretty clean — you get turn-level JSON with timestamps, role, and content. From there you can do basic aggregation in pandas in about twenty lines:

```
import json
import pandas as pd

with open("claude_code_sessions.json") as f:
    sessions = json.load(f)

rows = []
for session in sessions:
    turns = session.get("turns", [])
    user_turns = [t for t in turns if t["role"] == "user"]
    assistant_turns = [t for t in turns if t["role"] == "assistant"]
    rows.append({
        "session_id": session["id"],
        "total_turns": len(turns),
        "user_corrections": len(user_turns) - 1,  # subtract initial prompt
        "session_length_chars": sum(len(t["content"]) for t in turns),
    })

df = pd.DataFrame(rows)
print(df.describe())

```

That's enough to start asking real questions. Which sessions needed the most corrections? What were the initial prompts in those sessions? Are there prompt patterns that correlate with one-shot success versus five-round back-and-forth?

## The Patterns You're Going to Find, and What to Do With Them

I've been thinking about this in the context of my own Claude Code usage and a few patterns I'd expect to surface in almost anyone's data.

**Underspecified prompts cluster at the top of your correction count.** The sessions where you wrote "add error handling to the save function" probably needed more rounds than the sessions where you wrote "add a try/catch around the SaveReport call that catches IOException, logs the path and exception message to the logger, and re-throws as a custom StorageException." Not a surprising finding — but seeing it in your own data, quantified, changes how you write prompts going forward. There's a real difference between knowing something abstractly and having a number attached to it.

Domain knowledge gaps show up as hallucinated APIs. If the model keeps inventing methods that don't exist in a library you use, your prompts aren't grounding the model in the right context. I ran into this with some of the MCP tooling in AgentReview — the model would generate plausible-looking tool call signatures that weren't real. The fix was including the actual interface definition in context, not rephrasing the prompt. If your extracted data shows a pattern of "that method doesn't exist" corrections, the intervention is clear.

File-level context matters more than people admit. Sessions where I pasted the relevant class or function upfront almost always resolved in fewer turns than sessions where I described the code in prose. You can test this hypothesis with your own extracted data by categorizing sessions where the first user turn includes a code block versus sessions that are pure natural language, then comparing average correction counts between the two groups.

## Turning This Into a Personal Prompt Improvement Loop

Here's the practical workflow I'd actually run with this data. Extract your sessions, run the correction-count aggregation, and pull the top 10% of sessions by correction count. Read through the initial prompts in that cohort. You're looking for what they have in common — missing context, ambiguous scope, no example of the expected output format.

Then do the same for the bottom 10%, your one-shot or two-shot wins. What did those prompts include that the others didn't? Write that down. That's your personal prompt template. Not a generic template from a blog post, but one derived from your actual behavior with the actual models you use on the actual codebases you work in.

If you want to go further, you can fine-tune a small classifier on this data to predict session difficulty from the initial prompt. Not a production use case, but a genuinely good exercise for understanding what prompt features carry signal. I'd use a simple TF-IDF vectorizer and logistic regression before reaching for anything heavier — the dataset is small and you want interpretable features, not a black box.

```
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

# Label: 1 if corrections > median, 0 otherwise
df["hard"] = (df["user_corrections"] > df["user_corrections"].median()).astype(int)

X_train, X_test, y_train, y_test = train_test_split(
    initial_prompts, df["hard"], test_size=0.2, random_state=42
)

vec = TfidfVectorizer(max_features=500)
clf = LogisticRegression()
clf.fit(vec.fit_transform(X_train), y_train)

# Top positive coefficients = features correlated with hard sessions
feature_names = vec.get_feature_names_out()
top_hard = sorted(zip(clf.coef_[0], feature_names), reverse=True)[:10]
print("Features correlated with difficult sessions:", top_hard)

```

The coefficients are the interesting part. If words like "refactor" or "somehow" show up as predictors of hard sessions, that's telling you something real about how you write prompts under uncertainty.

## My Take

The ai-data-extractor tool is useful, but it's a means to an end. The actual value is that every engineer using AI coding assistants has been running an uncontrolled experiment on themselves for months, and nobody is looking at the results. The data is sitting in local SQLite files and JSON caches and nobody's touching it.

I'm planning to run this against my own Claude Code sessions from the AgentReview and DocQuery work and see what surfaces. My guess is that my prompts in C# and .NET contexts are tighter than my prompts in newer territory — I naturally include more grounding context when I know the domain well, which means the model has less room to hallucinate. That's a hypothesis worth testing.

If you're using any of the supported tools, spend an hour on this. Extract the data, run the aggregation, read the hard sessions. You'll come out with a more honest picture of where your AI-assisted workflow actually breaks down, and that's worth more than any general-purpose prompting guide.