There's a project trending on GitHub right now called ai-data-extractor that pulls structured data out of your AI coding assistant chat histories: Claude Code, Cursor, Windsurf, Aider, Cline, and more. It's a free, open-source Python tool. Most of the reaction to it has been "oh cool, I can export my chats." Mine was closer to this: I've been generating a labeled dataset about my own engineering behavior for months and didn't even realize it.

I've been running Claude Code heavily as part of my 10X Engineer Toolkit work, so when I saw this tool I started thinking about what those logs contain. Prompts, model responses, file diffs, error messages, follow-up corrections. Put together, that's a structured record of every problem you hit, how you described it, and whether the model's first answer worked or needed three rounds of fixing.

What the Data Looks Like

When you extract your Claude Code session history, you get a sequence of turns that maps fairly cleanly to an engineering workflow: you describe a task, the model generates code, you accept it or push back, and eventually something compiles and passes tests. Every correction turn is a signal.

Think about the shape of the data per session: the initial prompt, how many follow-up corrections it took, what kind of corrections they were (wrong type, wrong logic, missed edge case, hallucinated API), and how it finally resolved. If you've been using one of these tools for three to six months, you probably have hundreds of sessions. That's a meaningful sample size.

I did some research on what ai-data-extractor outputs, and the schema is pretty clean: JSONL with one conversation per line, where each message carries a role, content, and a timestamp. From there, basic aggregation in pandas takes about twenty lines:

import json
import pandas as pd

with open("claude_code_sessions.jsonl") as f:
    sessions = [json.loads(line) for line in f]

rows = []
for session in sessions:
    turns = session.get("messages", [])
    user_turns = [t for t in turns if t["role"] == "user"]
    assistant_turns = [t for t in turns if t["role"] == "assistant"]
    rows.append({
        "session_id": session["session_id"],
        "total_turns": len(turns),
        "user_corrections": len(user_turns) - 1,  # subtract initial prompt
        "session_length_chars": sum(len(t["content"]) for t in turns),
        "initial_prompt": user_turns[0]["content"] if user_turns else "",
    })

df = pd.DataFrame(rows)
print(df.describe())

That's enough to start asking real questions. Which sessions needed the most corrections? What were the initial prompts in those sessions? Are there prompt patterns that correlate with one-shot success versus five rounds of back-and-forth?

The Patterns You'll Probably Find, and What to Do With Them

Thinking about my own Claude Code usage, there are a few patterns I'd expect to show up in almost anyone's data.

Underspecified prompts cluster at the top of your correction count. The sessions where you wrote "add error handling to the save function" probably needed more rounds than the ones where you wrote "add a try/catch around the SaveReport call that catches IOException, logs the path and exception message to the logger, and re-throws as a custom StorageException." That won't surprise anyone. Seeing it in your own data with a number attached is different, and it changes how you write prompts going forward.

Domain knowledge gaps show up as hallucinated APIs. If the model keeps inventing methods that don't exist in a library you use, your prompts aren't grounding it in the right context. I ran into this with some of the MCP tooling in AgentReview, where the model would generate plausible-looking tool call signatures that weren't real. Rephrasing the prompt didn't fix it. Putting the real interface definition in context did. If your extracted data shows a run of "that method doesn't exist" corrections, you know what to change.

File-level context matters more than people admit. My impression is that sessions where I pasted the relevant class or function up front resolved in fewer turns than sessions where I described the code in prose. That's testable with your own data: split sessions by whether the first user turn includes a code block, then compare average correction counts between the two groups.

Turning This Into a Personal Prompt Improvement Loop

Here's the workflow I'd run. Extract your sessions, run the correction-count aggregation, and pull the top 10% of sessions by correction count. Read through the initial prompts in that group and look for what they have in common: missing context, ambiguous scope, no example of the expected output format.

Then do the same for the bottom 10%, your one-shot and two-shot wins. What did those prompts include that the others didn't? Write that down. That's your personal prompt template, and unlike the generic ones in blog posts (this one included), it comes from how you work with the models you use on the codebases you work in.

If you want to go further, train a small classifier on this data to predict session difficulty from the initial prompt. Nobody's shipping that to production, but it's a good exercise for figuring out which prompt features carry signal. I'd start with a TF-IDF vectorizer and logistic regression before reaching for anything heavier. The dataset is small and you want features you can interpret.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

# Label: 1 if corrections > median, 0 otherwise
df["hard"] = (df["user_corrections"] > df["user_corrections"].median()).astype(int)
initial_prompts = df["initial_prompt"]

X_train, X_test, y_train, y_test = train_test_split(
    initial_prompts, df["hard"], test_size=0.2, random_state=42
)

vec = TfidfVectorizer(max_features=500)
clf = LogisticRegression()
clf.fit(vec.fit_transform(X_train), y_train)

# Top positive coefficients = features correlated with hard sessions
feature_names = vec.get_feature_names_out()
top_hard = sorted(zip(clf.coef_[0], feature_names), reverse=True)[:10]
print("Features correlated with difficult sessions:", top_hard)

The coefficients are the interesting part. If words like "refactor" or "somehow" predict hard sessions, that tells you something about how you write prompts when you aren't sure what you want yet.

My Take

ai-data-extractor is useful, but it's a means to an end. Every engineer using an AI coding assistant has been running an uncontrolled experiment on themselves for months, and almost nobody is looking at the results. The data is sitting in local SQLite databases and JSON logs, untouched.

I'm planning to run this against my own Claude Code sessions from the AgentReview and DocQuery work. My guess is that my prompts are tighter in C# and .NET than in newer territory, because when I know the domain I include more grounding context without thinking about it, and that leaves the model less room to hallucinate. The data will tell me whether I'm right.

If you're using any of the supported tools, spend an hour on this. Extract the data, run the aggregation, read the hard sessions. You'll come out with a more honest picture of where your AI-assisted workflow actually breaks down, and that's worth more than any general-purpose prompting guide.