The thesis behind DriftWatch is one I keep coming back to: the model is maybe 20% of an ML system. The other 80% is versioning, deployment, monitoring, and retraining. That 80% looks almost identical to the enterprise DevOps work I've been doing for years, and this project exists to prove it end-to-end, with real infra, real drift, and a real retrain loop.

DriftWatch is my third public AI engineering project after DocQuery and AgentReview. The target problem is predicting turbofan engine failure from NASA's C-MAPSS sensor dataset, tracked in Azure ML, deployed through a secretless GitHub Actions pipeline, and eventually caught mid-drift by a detector that triggers an automatic retrain. Six phases total. Phase 0 was supposed to be the boring one: just get the environment and Azure footprint stood up.

It was not the boring one.

Decisions I locked in before writing any code

A few architectural choices got made upfront, and I want to explain the reasoning because they shape everything that comes after.

Train on FD001 only. C-MAPSS ships as four subsets with different operating conditions. FD002 and FD004 stay untouched until the drift phase, where they get replayed as "production" traffic. The whole point is that the drift detector will face a real regime change, not synthetic noise I injected myself. If I peeked at those subsets now, I'd be manufacturing the outcome.

Deploy twice, on purpose. An Azure ML managed endpoint as a demonstration artifact (capture the evidence, tear it down, it bills around the clock), and Azure Container Apps at zero minimum replicas as the persistent demo. Zero minimum replicas means it costs nothing when idle and scales up on a request. That's the right default for a side project demo.

No stored cloud secrets. The CI/CD pipeline authenticates with OIDC federated credentials. Nothing to leak, nothing to rotate. I'm not trying to give a security lecture here. Once you understand federated identity, this is just the easier path. Setting it up once is far less work than managing secret rotation forever.

Infrastructure is code. The entire Azure footprint is Bicep, not portal clicks. Teardown and rebuild are one command each way. I've watched too many projects become impossible to reproduce because someone clicked something in the portal six months ago and nobody wrote it down.

What Phase 0 actually shipped

Four things, all unglamorous, all load-bearing:

First, a pinned Python 3.11 environment. Every package in requirements.txt is an exact version with a comment explaining each cap. That comment will matter when someone (probably me, six months from now) tries to understand why a dependency is frozen at a specific release.

Second, the Azure footprint as a single Bicep deployment: resource group, ML workspace, container registry, storage, key vault, and App Insights, all in eastus2. The whole thing can be torn down and rebuilt in one command each direction.

Third, a budget alert that exists before anything that can spend money. Thirty dollars a month with email notifications at 50%, 80%, and 100%. Cost guardrails are a feature of this project, not an afterthought.

Fourth, MLflow pointed at the Azure ML workspace and verified with a smoke run that shows up in Azure ML Studio. The workspace is the tracking server and model registry. No local MLflow server to babysit, no "works on my machine" tracking state.

The war stories

Setup weekends always have war stories. Here are the four that cost the most time.

A 2019 CLI from a 2026 resolver. I installed the Azure CLI with uv and got version 2.0.67, released in 2019. What happened: the resolver defaulted to Python 3.13, the latest azure-cli wouldn't resolve on it, so it backtracked seven years to a version with loose enough metadata to satisfy the constraints. Completely legal resolution, completely useless result. The fix was pinning both the version and the Python interpreter: azure-cli==2.89.1 on 3.11. The lesson isn't about uv specifically. An unpinned install doesn't mean "latest." It means "whatever resolves."

The venv with no pip. Adding the Azure ML CLI extension failed with a vague "Pip failed with status code 1." The debug log showed what was actually happening: uv builds its tool environments without pip, and the az CLI shells out to python -m pip to install extensions. One ensurepip call later and everything worked. Debug flags exist for a reason. I should've checked the full log first instead of guessing at the cause.

The Intel Homebrew on an Apple Silicon Mac. XGBoost wouldn't import: no OpenMP runtime. I went to install one and discovered my Homebrew was the Intel build, migrated from an old machine years ago, silently running every command under Rosetta 2 and reporting my Apple Silicon Mac as an x86 "westmere" CPU. It had zero packages installed and would have handed me an x86 libomp that an arm64 XGBoost could never load. I deleted it, installed native Homebrew, and imports went green immediately. I'd been running that broken Homebrew for probably two years without noticing because nothing else had needed a native shared library.

The version cap chain. Current numpy and xgboost want Python 3.12. The Azure MLflow plugin caps mlflow below a certain version. That version of mlflow caps pandas below 3.0. No single README tells you the full chain. You find it by resolving the whole set together and reading the conflict errors. That's exactly why everything in this project is pinned and commented. Future me will thank current me.

Why setup phases deserve a real post

I've seen teams treat environment and infra setup as something to blur past: "yeah we set up the project, anyway here's the model." That's not honest, and it's not useful to anyone trying to learn from the work.

I don't consider the war stories above embarrassing. They're the normal texture of getting a real system running on real hardware with real cloud dependencies. A 2019 CLI resolver, a missing pip, a Rosetta 2 Homebrew: none of these show up in tutorials, because tutorials use a clean VM with no history. Production environments and developer laptops have history.

Every problem I hit in Phase 0 is a problem someone else will hit. I'd rather document it than pretend the setup was frictionless.

What's next

Phase 1 is data: DVC-versioned ingestion, rolling and lag features, and train/test splits by engine unit rather than by row. That last one matters: rows within a single engine's run are correlated, so a row-level split will make your validation metrics look great while telling you almost nothing about generalization. I'll have the first real numbers worth reporting in that post. The boring phase is behind us.