Data Quality: Why a 96% Accurate Model Can Still Be Completely Useless

Previously in This Series

Day 3 went column by column, checking whether individual features and labels could be trusted. Today we zoom out to the dataset as a whole: are there gaps in it, weird extreme values, or a label split so lopsided that "high accuracy" stops meaning anything?

Learning Objectives

• Diagnose missing values correctly — including the cases where the fact that something is missing is itself informative.

• Detect outliers and skewed distributions, and know which kinds of models actually care about them.

• Recognize severe class imbalance and understand exactly why accuracy alone becomes a misleading metric once it's present.

• Walk away with a short, practical checklist to run on any new dataset before training anything at all.

Why This Matters

"Garbage in, garbage out" is a cliché precisely because it's true, and most garbage isn't dramatic — it's a 20% missing column, a handful of clerical zero values, or a label split that's 97/3 instead of 50/50. None of that looks alarming in a .head() call. All of it can quietly invalidate a model.

This also connects directly forward: Phase 5's production monitoring is largely an automated, always-on version of the three checks in this lesson. And you can't meaningfully interpret precision, recall, or any other evaluation metric — covered properly in Phase 3 — until you already know whether your classes are balanced. Today gives you just enough of that vocabulary to recognize the problem; Phase 3 gives you the full toolkit to act on it.

Mental Model

Treat a new dataset the way a doctor treats a new patient's chart, before trusting anything written in it:

• Missing values are gaps in the history. Sometimes harmless. Sometimes the patient skipped a test for a reason — and the gap itself is a symptom.

• Outliers are one extreme vital sign reading. Sometimes a measurement error. Sometimes the single most important number on the page.

• Class imbalance is studying a population that doesn't match the question you're asking. If 97 of your 100 patients are healthy, "high accuracy" on this group tells you almost nothing about the 3 who are actually sick — and they were the entire reason you ran the study.

History in 60 Seconds

• 1976 — Donald Rubin formalizes MCAR, MAR, and MNAR, giving statisticians a vocabulary for exactly the "is this gap itself informative?" question Titanic's age column raises later in this lesson.

• 1977 — John Tukey's Exploratory Data Analysis introduces the box plot and the IQR-based outlier detection method used in today's hands-on example.

• 2002 — Chawla, Bowyer, Hall, and Kegelmeyer publish SMOTE, the first widely-adopted technique for handling severe class imbalance by synthesizing new minority-class examples instead of simply duplicating or discarding data.

• 2009 — Fei-Fei Li's ImageNet project demonstrates, at enormous scale, that data curation effort can matter as much as algorithm design — a lesson the deep learning models trained on it would go on to prove three years later.

Key Terminology Table

Core Concepts

Missing Values: Not All Gaps Mean the Same Thing

The first question for any missing value isn't "how do I fill it in?" — it's "why is it missing?" Rubin's taxonomy gives you three possible answers:

• MCAR (Missing Completely At Random) — the gap has nothing to do with anything, observed or not. A sensor occasionally drops a reading for no systematic reason.

• MAR (Missing At Random) — the gap relates to other observed variables, but not to the missing value itself. Older survey respondents might skip an income question more often, regardless of their actual income.

• MNAR (Missing Not At Random) — the gap relates to the unobserved value itself. This is the dangerous one, because the missingness can be secretly informative — though, as you'll see in today's hands-on example, telling MNAR apart from MAR using only what you can observe is genuinely harder than it sounds.

Once you know which kind you're facing, your options are: drop the column entirely (when too much is missing to recover anything useful), drop the rows (only safe when missingness is MCAR and rare), impute a reasonable estimate like the median, or — often the most honest option when you suspect MNAR — add a separate "was this missing?" indicator feature, so the model can use the missingness pattern itself rather than having it erased by imputation. One more honest caveat worth knowing upfront: MAR and MNAR are, in general, impossible to fully distinguish from observed data alone — you can build evidence for one over the other, but you can rarely prove it outright.

Outliers and Skew: When Extreme Values Matter (and When They Don't)

An outlier is a value unusually far from the rest of its distribution — detectable with simple rules like the IQR method (anything more than 1.5×IQR beyond the 25th or 75th percentile) or a z-score threshold. But "detect" and "remove" are different decisions, and the two get conflated constantly.

Some outliers are measurement errors and should go. Others are the single most important rows in your entire dataset — a fraud case, a rare disease, a catastrophic equipment failure. Deleting those because they're "statistically unusual" deletes the exact thing you were trying to predict.

What outliers and skew genuinely affect is algorithm-dependent, more than most tutorials let on. A model that learns its own coefficient for a feature (like logistic regression) can absorb an oddly-scaled, skewed column without it touching the model's accuracy — it just learns a correspondingly tiny coefficient. A model that measures raw distance between points (like KNN) has no such protection: one feature with a huge numeric range can silently dominate every distance calculation, regardless of how predictive it actually is. You'll see both sides of this directly in today's code.

Class Imbalance: When Accuracy Stops Meaning Anything

A dataset is imbalanced when one class vastly outnumbers another. The math here is blunt: if 97% of your examples belong to one class, a model that learns absolutely nothing and just predicts that class every time scores 97% accuracy. That number was never about the model's intelligence — it's just the majority class's share of the data, restated.

This is why, on imbalanced problems, you need precision (of the cases you flagged as positive, how many actually were) and recall (of the cases that were actually positive, how many you caught) alongside accuracy, not instead of it. We'll only use them descriptively today — the full toolkit for choosing and balancing these metrics belongs to Phase 3's Model Evaluation. For now, the goal is simpler: recognize when accuracy alone is lying to you.

Visual Explanations

A quick reference for all three problems at once:

And a single checklist worth running on every new dataset, before any modeling starts:

flowchart TD
    D[New dataset] --> M{Any columns<br/>missing values?}
    M -->|Yes| M2[Check % missing and correlation with the label,<br/>then re-check after controlling for other variables]
    M -->|No| O{Any extreme or<br/>skewed numeric columns?}
    M2 --> O
    O -->|Yes| O2[Check which algorithms you'll use —<br/>scale-sensitive ones will need a transform]
    O -->|No| C{Is the label severely<br/>imbalanced?}
    O2 --> C
    C -->|Yes| C2[Stop trusting accuracy alone —<br/>check recall and precision per class]
    C -->|No| Go[Proceed to modeling]
    C2 --> Go

Hands-On Example

Three real checks, on two real datasets:

1. Missing values on the Titanic dataset. We'll measure exactly how much is missing in each column, then test whether missing age values correlate with survival — and then take the one extra step most tutorials skip: checking whether that correlation survives once we control for passenger class, which is exactly the MAR-vs-MNAR question from Core Concepts, answered with real numbers instead of a guess.

2. Outliers and skew on the same dataset's fare column. We'll flag outliers with the IQR method, then compare a log-transformed version of fare against the raw version, across two different algorithm types, to see exactly where the transform matters and where it doesn't.

3. Severe class imbalance, built from the real Breast Cancer dataset. We'll deliberately keep only 12 malignant cases against the full set of benign ones, to create a realistic, severe imbalance, and watch what happens to a model that doesn't account for it.

Environment Setup

python -m venv venv

source venv/bin/activate
# Windows
venv\Scripts\activate

pip install numpy pandas seaborn scikit-learn

Complete Working Code

Every number in the breakdown below came from actually running this script.

"""
day_04_data_quality.py
Three real data quality problems, demonstrated honestly: missing values,
outliers/skew, and class imbalance.
"""

import numpy as np
import pandas as pd
import seaborn as sns
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, recall_score, precision_score

# ============================================================
# PART 1: MISSING VALUES
# ============================================================
titanic = sns.load_dataset("titanic")

missing_pct = (titanic.isnull().sum() / len(titanic) * 100).sort_values(ascending=False)
print("=== MISSING VALUES BY COLUMN ===")
print(missing_pct[missing_pct > 0].round(1))

# Is missingness itself informative? Compare survival rate by whether age was recorded
titanic["age_missing"] = titanic["age"].isnull().astype(int)
survival_by_missingness = titanic.groupby("age_missing")["survived"].mean()
print("\n=== SURVIVAL RATE: AGE RECORDED vs AGE MISSING (unconditional) ===")
print(f"Age recorded:  {survival_by_missingness[0]:.3f}")
print(f"Age missing:   {survival_by_missingness[1]:.3f}")

# Before concluding MNAR, check whether missingness just tracks an OBSERVED variable instead
print("\n=== AGE-MISSINGNESS RATE BY PASSENGER CLASS ===")
print(titanic.groupby("pclass")["age_missing"].mean().round(3))

print("\n=== SURVIVAL RATE, RECORDED vs MISSING, WITHIN EACH CLASS ===")
for pclass in [1, 2, 3]:
    sub = titanic[titanic["pclass"] == pclass]
    rates = sub.groupby("age_missing")["survived"].mean()
    counts = sub.groupby("age_missing")["survived"].count()
    print(f"Class {pclass}: recorded={rates.get(0, float('nan')):.3f} (n={counts.get(0, 0)}) "
          f"| missing={rates.get(1, float('nan')):.3f} (n={counts.get(1, 0)})")

# ============================================================
# PART 2: OUTLIERS AND SKEW (the 'fare' column)
# ============================================================
df = titanic.copy()
df["age"] = df["age"].fillna(df["age"].median())
df["embarked"] = df["embarked"].fillna(df["embarked"].mode()[0])
df["sex"] = df["sex"].map({"male": 0, "female": 1})
df["embarked_enc"] = df["embarked"].map({"S": 0, "C": 1, "Q": 2})
df["log_fare"] = np.log1p(df["fare"])  # log1p handles the zero fares safely

q1, q3 = df["fare"].quantile(0.25), df["fare"].quantile(0.75)
iqr = q3 - q1
lower, upper = q1 - 1.5 * iqr, q3 + 1.5 * iqr
n_outliers = ((df["fare"] < lower) | (df["fare"] > upper)).sum()
print(f"\n=== FARE OUTLIERS (IQR METHOD) ===")
print(f"Flagged as outliers: {n_outliers} / {len(df)} rows")
print(f"Max fare: {df['fare'].max():.2f}, fares of exactly 0: {(df['fare']==0).sum()}")

base_features = ["pclass", "sex", "age", "sibsp", "parch", "embarked_enc"]
y = df["survived"]

print("\n=== DOES THE FARE TRANSFORM MATTER? ===")
for model_name, model in [("LogisticRegression", LogisticRegression(max_iter=500)),
                           ("KNN (k=5)", KNeighborsClassifier(n_neighbors=5))]:
    for fare_col in ["fare", "log_fare"]:
        X = df[base_features + [fare_col]]
        X_train, X_test, y_train, y_test = train_test_split(
            X, y, test_size=0.3, random_state=42, stratify=y
        )
        model.fit(X_train, y_train)
        acc = accuracy_score(y_test, model.predict(X_test))
        print(f"{model_name:20s} | {fare_col:10s} | accuracy = {acc:.3f}")

# ============================================================
# PART 3: CLASS IMBALANCE
# ============================================================
data = load_breast_cancer()
X_full = pd.DataFrame(data.data, columns=data.feature_names)
y_bc = pd.Series(data.target)  # 0 = malignant, 1 = benign

# Use weakly-predictive features only, to keep this a genuinely hard problem
weak_features = ["mean smoothness", "mean symmetry", "mean fractal dimension", "texture error"]
X_weak = X_full[weak_features]

# Deliberately undersample the malignant class to create severe, realistic imbalance
rng = np.random.RandomState(42)
minority_idx = y_bc[y_bc == 0].index
majority_idx = y_bc[y_bc == 1].index
keep_minority = rng.choice(minority_idx, size=12, replace=False)
imbalanced_idx = np.concatenate([keep_minority, majority_idx])
X_imb = X_weak.loc[imbalanced_idx].reset_index(drop=True)
y_imb = y_bc.loc[imbalanced_idx].reset_index(drop=True)

print(f"\n=== CLASS IMBALANCE: {y_imb.value_counts(normalize=True)[0]*100:.1f}% malignant, "
      f"{y_imb.value_counts(normalize=True)[1]*100:.1f}% benign (n={len(y_imb)}) ===")

X_train_b, X_test_b, y_train_b, y_test_b = train_test_split(
    X_imb, y_imb, test_size=0.3, random_state=42, stratify=y_imb
)

naive = LogisticRegression(max_iter=2000)
naive.fit(X_train_b, y_train_b)
naive_preds = naive.predict(X_test_b)
print("\nNaive LogisticRegression:")
print(f"  Accuracy: {accuracy_score(y_test_b, naive_preds):.3f}")
print(f"  Recall on malignant (the class that matters): "
      f"{recall_score(y_test_b, naive_preds, pos_label=0, zero_division=0):.3f}")

balanced = LogisticRegression(max_iter=2000, class_weight="balanced")
balanced.fit(X_train_b, y_train_b)
balanced_preds = balanced.predict(X_test_b)
print("\nWith class_weight='balanced':")
print(f"  Accuracy: {accuracy_score(y_test_b, balanced_preds):.3f}")
print(f"  Recall on malignant: {recall_score(y_test_b, balanced_preds, pos_label=0, zero_division=0):.3f}")
print(f"  Precision on malignant: "
      f"{precision_score(y_test_b, balanced_preds, pos_label=0, zero_division=0):.3f}")

Running this prints:

=== MISSING VALUES BY COLUMN ===
deck           77.2
age            19.9
embarked        0.2
embark_town     0.2

=== SURVIVAL RATE: AGE RECORDED vs AGE MISSING (unconditional) ===
Age recorded:  0.406
Age missing:   0.294

=== AGE-MISSINGNESS RATE BY PASSENGER CLASS ===
pclass
1    0.139
2    0.060
3    0.277
Name: age_missing, dtype: float64

=== SURVIVAL RATE, RECORDED vs MISSING, WITHIN EACH CLASS ===
Class 1: recorded=0.656 (n=186) | missing=0.467 (n=30)
Class 2: recorded=0.480 (n=173) | missing=0.364 (n=11)
Class 3: recorded=0.239 (n=355) | missing=0.250 (n=136)

=== FARE OUTLIERS (IQR METHOD) ===
Flagged as outliers: 116 / 891 rows
Max fare: 512.33, fares of exactly 0: 15

=== DOES THE FARE TRANSFORM MATTER? ===
LogisticRegression   | fare       | accuracy = 0.799
LogisticRegression   | log_fare   | accuracy = 0.799
KNN (k=5)            | fare       | accuracy = 0.694
KNN (k=5)            | log_fare   | accuracy = 0.724

=== CLASS IMBALANCE: 3.3% malignant, 96.7% benign (n=369) ===

Naive LogisticRegression:
  Accuracy: 0.964
  Recall on malignant (the class that matters): 0.000

With class_weight='balanced':
  Accuracy: 0.640
  Recall on malignant: 0.250
  Precision on malignant: 0.026

Code Breakdown

Missing values. deck is missing 77.2% of the time — too much to recover, and the right call is usually to drop it entirely rather than impute three-quarters of a column. age is missing 19.9% of the time, which is recoverable with imputation.

The unconditional comparison looks like a clean MNAR story: passengers with a recorded age survived 40.6% of the time, versus only 29.4% for those with a missing age — an 11-point gap. It's tempting to stop right there and call it MNAR, the way the first draft of this very article did. But look at what happens next: age-missingness correlates strongly with passenger class — 13.9% missing in 1st class, 6.0% in 2nd, 27.7% in 3rd — and class is fully observed for every passenger. Once you stratify by class, most of that 11-point gap evaporates. In 3rd class specifically, where 136 of the 177 total missing-age passengers are, survival is 23.9% with age recorded versus 25.0% with age missing — essentially no gap at all. Small, less reliable gaps remain in 1st and 2nd class (n=30 and n=11 missing respectively), so a residual MNAR-flavored effect isn't fully ruled out — but the dominant story is that missing-age passengers are disproportionately 3rd class, and 3rd class survives less regardless of whether age was recorded. That's the signature of MAR — missingness tied to an observed variable — not MNAR. Either way, the practical conclusion holds: median-imputing age and stopping there, with no stratified check at all, would have missed this story completely, in either direction.

Outliers and skew. The IQR method flags 116 of 891 fares (13%) as outliers, with a maximum of 512.33 against a median far lower, and 15 passengers who paid exactly 0 — almost certainly crew or special-case tickets. Now look at what the log transform actually changes: for logistic regression, accuracy is identical either way (0.799), but the learned coefficient on fare jumps from 0.00258 (raw) to 0.403 (log) — same information, wildly different scale. For KNN, which measures raw distance between points, the same transform improves accuracy from 0.694 to 0.724 — a real, meaningful gain, because the untransformed fare column's huge range was quietly dominating every distance calculation before the transform fixed that.

Class imbalance. With only 12 malignant cases against 357 benign ones, a naive logistic regression hits 96.4% accuracy — which is, not coincidentally, almost exactly the majority-class share of this data. Its recall on the malignant class is exactly 0.000: it never once correctly identifies the case that actually matters. Adding class_weight="balanced" changes the picture, but not into a clean win — accuracy drops to 0.640, recall improves to 0.250, and precision falls to 0.026. The model now finds 1 of 4 real malignant cases instead of 0, at the cost of a flood of false alarms. That's not a bug in the demonstration — that's the actual, honest tradeoff severe imbalance forces on you, and no single setting makes it disappear.

Common Mistakes

• Concluding MNAR from a single unconditional comparison. Checking whether missingness correlates with the label is a good first step, but it isn't the last one — as today's age/survived check showed, that raw gap can collapse once you control for an observed variable like passenger class. Stratify before you conclude.

• Mean-imputing a skewed column. The mean is pulled toward extreme values; the median usually isn't. For a column like fare, median imputation is the safer default.

• Deleting outliers reflexively. Some outliers are the entire reason you're building the model — check whether an "extreme" row is an error or the actual phenomenon you care about before removing it.

• Trusting accuracy on an imbalanced dataset without question. As shown above, 96.4% accuracy and 0% real-world usefulness can coexist in the same model.

• Treating class_weight or resampling as a fix-all. Both shift a tradeoff, they don't eliminate one — always check precision alongside recall after rebalancing, the way today's demo did.

• Assuming a transform that helps one algorithm will help another. The fare transform did nothing for logistic regression's accuracy and meaningfully helped KNN's — test on the actual algorithm you're using, not on general advice.

Best Practices

• Profile every new dataset's missingness, distributions, and class balance before writing a single line of model code — the checklist diagram above takes a few minutes and can save days.

• Default to median over mean imputation whenever a column shows meaningful skew or outliers.

• If you're unsure whether scale or skew matters for your problem, test more than one algorithm type — the gap between logistic regression and KNN in today's demo is exactly the kind of thing general advice won't tell you in advance.

• On imbalanced problems, never report accuracy alone. Report it alongside recall and precision for the class that actually matters, and treat the majority-class baseline as the number your model has to beat to be worth deploying at all.

Production Perspective

Missing-value patterns and class balance can both drift after launch, even when they looked fine in training. A sign-up form field that becomes optional six months after launch can cause that column's missingness rate to jump from 2% to 20% with no code change anywhere near the model — a real, common production surprise.

Outlier handling needs similar care over time: a detector tuned to "normal" behavior at one point can keep flagging genuinely new, valid behavior as out-of-range indefinitely if the underlying data shifts — a preview of the drift detection topic waiting in Phase 5.

Real-world imbalance is often more severe than anything in an offline sample — fraud rates well under 1% are common, far more extreme than today's already-severe 3.3%. Past a certain point, teams often stop treating this as a classification problem at all and reframe it as anomaly detection instead.

The cost side is where this gets concrete. Every rebalancing decision — a resampling ratio, a class_weight, a decision threshold — is really a stand-in for a business cost tradeoff. A missed fraud case and a false fraud alert that annoys a real customer do not cost the same amount, and that tradeoff has to be set deliberately by the people who own the business outcome, not left to whatever a library's default happens to be.

Real-World Applications

• Fraud detection — typically far more imbalanced than today's demo, often well under 1% positive cases.

• Rare disease screening and medical diagnostics — where the minority class is, by definition, the entire point of the model.

• Manufacturing defect detection — defects are rare by design, which is good for the factory and hard for the classifier.

• Customer churn prediction — usually a milder, but still real, imbalance problem.

• Any survey- or signup-derived dataset — where certain fields are systematically skipped by specific user segments, echoing today's age missingness finding.

Interview Questions

1. What's the difference between MCAR, MAR, and MNAR, and why does it matter for how you handle missing values? MCAR means missingness is unrelated to anything; MAR means it relates to other observed variables; MNAR means it relates to the missing value itself. It matters because the three can look identical in a simple unconditional comparison — today's age/survived check looked like MNAR at first glance, and turned out to be mostly a MAR-style confound with passenger class once stratified. Skipping that stratification step, and concluding MNAR from the raw comparison alone, is the actual mistake worth avoiding.

2. Why might mean imputation be a worse choice than median imputation for some columns? The mean is sensitive to extreme values and skew, so imputing it can introduce a value that doesn't represent the typical case. The median holds steady against outliers and skew in a way the mean doesn't, making it the safer default for columns like fare.

3. When should you not remove an outlier, even though it's statistically extreme? When the outlier represents the actual phenomenon you're trying to detect — fraud, rare disease, equipment failure — rather than a data entry or measurement error. Removing it would remove the signal the model exists to find.

4. Why does a model with 96% accuracy sometimes provide zero real value? If the positive class makes up only 4% of the data, a model that predicts the majority class every single time reaches 96% accuracy while never correctly identifying a single real positive case — which is exactly what today's naive demo did.

5. What's the difference between fixing class imbalance with class_weight versus resampling techniques like SMOTE?class_weight changes how much the loss function penalizes mistakes on each class during training, without changing the actual data. Resampling techniques like SMOTE change the training data itself, by oversampling the minority class (often by synthesizing new examples) or undersampling the majority class.

6. If a feature's missingness rate jumps from 2% to 20% in production, what would you investigate? Whether an upstream change — a form field becoming optional, an API change, a new data source feeding the pipeline — altered how or whether that field gets populated, and whether the new missingness pattern is MCAR or carries the same kind of signal a sudden shift like this often does.

7. Why might log-transforming a skewed feature help one algorithm and do nothing for another? Algorithms that learn their own per-feature weighting, like logistic regression, can absorb scale and skew into the learned coefficient without it affecting predictions. Algorithms that depend on raw distances between points, like KNN, have no such protection — an unscaled, skewed feature can dominate every distance calculation regardless of its actual predictive value.

8. Besides accuracy, what would you report when evaluating a model on a severely imbalanced dataset? Precision and recall for the minority class specifically, and the majority-class baseline accuracy as context — a model's accuracy is meaningless on its own without knowing what a trivial baseline would already score.

Self-Assessment

• Can you explain why the raw age/survived comparison wasn't enough by itself, and what additional check turned out to matter?

• Can you describe, in your own words, a different situation where a confounding variable could make MAR look like MNAR (or the reverse)?

• Could you write a five-minute data quality checklist you'd actually run on a new dataset before modeling?

• Can you explain why today's class-weighted model's accuracy went down while its real-world usefulness arguably went up?

• Can you name a dataset you've worked with where class imbalance might have been quietly inflating an accuracy number you trusted?

Portfolio Challenges

Beginner Challenge

Run the same missingness-vs-survival check from today on a different Titanic column with missing data (embarked), including the same class-stratified follow-up check. Report whether the raw gap (if any) survives stratification, or turns out to be a confound the way age's did.

Intermediate Challenge

Repeat the class imbalance demo with different minority-class sizes (try 5, 12, 25, and 50 retained malignant cases) and report how the naive model's recall changes as the imbalance becomes less severe. Identify roughly where recall stops being zero.

Advanced Challenge

Install imbalanced-learn (pip install imbalanced-learn) and apply real SMOTE oversampling to the same imbalanced breast cancer scenario from today. Compare its precision and recall on the malignant class against both the naive model and the class_weight="balanced" model from this article.

Advanced Insights

Today only introduced enough precision and recall vocabulary to recognize when accuracy is lying to you — Phase 3's Model Evaluation will give you the full toolkit (precision-recall curves, F1, ROC-AUC, and how to actually choose a decision threshold) for properly judging models on imbalanced data.

These same problems persist in deep learning, just with a different surface. Missing values become padding tokens and masking strategies in sequence models. Severe class imbalance shows up as long-tail label distributions in large-scale classification, where a handful of common classes dominate thousands of rare ones. The underlying principles from today — check what's missing and why, check what's extreme and why, never trust accuracy alone — carry forward essentially unchanged, even as the specific techniques evolve.

Key Takeaways

• Missing values aren't all the same, and you can't tell which kind you're facing from a single unconditional comparison. Titanic's age column looked like a clean MNAR case at first — an 11-point survival gap — and turned out to be mostly a MAR-style confound with passenger class once stratified.

• Outlier and skew handling can matter a lot for some algorithms (KNN, +3 points of accuracy here) and barely at all for the headline accuracy of others (logistic regression), though it can still affect coefficient interpretability either way.

• Severe class imbalance can make accuracy alone meaningless — today's naive model hit 96.4% accuracy while catching zero real cases of the class that mattered.

• Fixing imbalance is a tradeoff, not a free win. Rebalancing improved recall in today's demo but cratered precision — there's no single "correct" setting without knowing the real-world cost of each kind of mistake.

What's Next

Day 5 moves from diagnosing data problems to the math underneath every model that will fix them: Statistics Essentials — the handful of statistical ideas (distributions, variance, correlation, hypothesis testing) that show up constantly in practice, taught with exactly as much rigor as you need and not more.

References

Beginner

• Google: Machine Learning Crash Course — includes a dedicated module on data quality and common data problems.

• scikit-learn: User Guide — official documentation covering the imputation, model, and metric tools used in this article's code.

Intermediate

• Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani, An Introduction to Statistical Learning — free at statlearning.com, with solid grounding in the distributional thinking behind outlier and skew detection.

Professional

• Aurélien Géron, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow (O'Reilly) — covers practical data cleaning and imbalance-handling pipelines in depth. Available through O'Reilly and major booksellers.

Advanced

• Donald B. Rubin, "Inference and Missing Data," Biometrika, 1976 — the paper that formalized MCAR, MAR, and MNAR. Official record: doi.org/10.1093/biomet/63.3.581.

• Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer, "SMOTE: Synthetic Minority Over-sampling Technique," Journal of Artificial Intelligence Research, 2002 — the foundational paper behind the SMOTE technique referenced in today's Advanced Challenge. Freely available, open access: jair.org.