Training Data Quality and Bias

A model reflects its examples

A model has no knowledge except what is encoded by training on examples. If the examples are narrow, mislabelled, outdated or unbalanced, the model will reflect that, however sophisticated the algorithm. The old saying “garbage in, garbage out” is a useful summary, though in practice the problems are often subtler than obviously bad data. Data can look perfectly tidy and still teach the wrong lesson.

This matters whether you gather your own examples for a small project or use a pretrained model built from data you cannot see. The general process of model training is covered elsewhere; here the focus is on what goes into it.

Quality problems: the everyday ones

Even without any question of fairness, ordinary data defects cause trouble.

Wrong labels. If a photo labelled “cracked” is actually fine, the model learns from a false example. A modest amount of random label noise can often be tolerated, but systematic mistakes, such as one person consistently labelling borderline cases differently from another, are more damaging.
Inconsistent labelling rules. If it is unclear what counts as “loud” or “ripe” or “defective”, different labellers will disagree and the model will learn the muddle. Writing down a short labelling guide helps.
Duplicates and near-duplicates. If copies of the same example appear in both the training and test sets, test results look better than they truly are.
Missing or corrupted values. Sensor dropouts, clipped audio and stuck readings can be mistaken for real patterns.
Too little data. With few examples, a model may memorise them rather than learn a general pattern.
Out-of-date data. If the world has changed, old examples may no longer represent it.

Hidden shortcuts

A model will use whatever signal in the data best predicts the label, even if that signal is not the one you intended. Classic cautionary examples are well known in general terms: a detector that learns to recognise a background or a watermark that happens to accompany a class, rather than the object itself.

For a maker, this might look like:

Photos of “good” parts all taken on a white desk and “faulty” parts on a workbench, so the model learns the surface instead of the fault.
Audio of a machine running normally recorded in the morning and faults recorded in the afternoon, so the model learns the time-of-day room noise.
All the positive examples captured with one camera, and negatives with another.

The check is to ask, for each class, whether anything other than the target differs systematically between classes. Collect examples so that background, lighting, device and time are mixed across classes.

Where bias comes from

In everyday use, bias means a systematic tilt in results. It can enter at many points.

Sampling bias: the examples are not representative of the situations where the model will be used. A voice command model trained mainly on a few accents may work poorly for others. A plant-health model trained on photos from one season may fail in another.
Selection by convenience: data that is easy to collect, such as what is online, nearby or already digitised, differs from data that is harder to reach, and the easy data may over-represent some groups, places and conditions.
Measurement bias: the sensor or process captures some cases better than others. A camera may render some skin tones, materials or lighting conditions less clearly than others, so the data itself carries uneven detail.
Label bias: human labellers bring assumptions. If historical decisions were themselves skewed, a model trained to imitate them can reproduce the skew.
Class imbalance: when one outcome is far more common, a model can score well by favouring it. If only a few per cent of examples show a fault, a model that never reports faults will look accurate and be useless.
Feedback loops: if a model’s outputs influence what data is collected next, an initial tilt can grow over time.
Proxy variables: even if you leave out a sensitive attribute, other fields may stand in for it.

Bias is not only about people

Discussions of bias often centre on people, and that is where the consequences can be most serious, such as models used to judge or rank individuals. But the same mechanism affects hobby projects: a camera trap that detects some animals better than others, or an anomaly detector that treats a summer baseline as the only normal. The shared principle is that performance is uneven across conditions you did not think to include.

If a project decides things about people, such as access, scoring or monitoring, take particular care, consider whether a model should be involved at all, and be aware that laws on automated decisions and personal data may apply.

Practical checks for your own dataset

You do not need special software to catch many problems.

Look at the data. Browse a few dozen random examples per class. Surprises show up quickly.
Count. How many examples per class, per condition, per device, per day? Look for lopsided counts.
Review labels. Have a second person label a sample and compare. Disagreement points to unclear rules.
Split carefully. Keep related examples, such as frames from the same video or readings from the same session, together in one split so the test set is truly unseen.
Test on slices. Report results separately for different lighting, devices, speakers or locations, not just one overall number. A good average can hide a poor slice.
Test in the field. Gather a small set of examples from the real deployment, kept apart from training, and see whether results hold.
Document the dataset. Write a short note on where the data came from, when, how it was labelled, and known gaps. It helps you and anyone who reuses it.

Collecting data responsibly

Think about consent and privacy before you gather examples. Recording people’s faces or voices, even at home, raises questions covered in camera privacy. Scraping material from the internet can conflict with copyright or site terms, and the legal position varies, so check rules where you live. If you share the dataset, consider whether it contains personal information and what its licence should be; the same thinking as for software licenses applies.

Improving a problem dataset

When you find a weakness, typical remedies are to collect more examples of underrepresented cases, rebalance by gathering rather than only duplicating, fix label rules and relabel, remove duplicates, and augment data with sensible variations such as small changes in brightness or crop. Augmentation can help robustness but cannot create cases that never appeared, so it is not a substitute for real coverage.

Be wary of promises that any technique “removes” bias. Reducing known problems is realistic; guaranteeing none remain is not.

Why this belongs in your planning

Data work is usually the largest part of a machine learning project, and the part most likely to determine whether it works in practice. It also affects the claims you can make: a model proved on a narrow dataset should be described as such. Include data collection and upkeep when you weigh costs in an AI feature evaluation, and remember that fluent outputs from a language model built on enormous, uneven text collections can still be wrong, a point developed under model hallucinations.