What is data science?
Data are facts we record: numbers, words, pictures, clicks. Data science is the skill of turning data into useful answers using statistics (maths of data), computing and knowledge of the topic.
Data has value when it helps someone decide better: what to stock, when to water crops, which patient to see first.
A short history of managing data
People first kept data on clay tablets and paper ledgers. Then came punched cards (around 1890), computer databases (1960s–70s), spreadsheets (1980s), the internet and big data (2000s), and now machine learning, where computers learn patterns from huge datasets.
The data science project cycle
- Ask: write a clear question. "Does study time affect marks?"
- Collect: survey, sensors, records, open datasets.
- Clean: remove impossible values, fix typing errors, handle missing data, remove duplicates.
- Explore: draw charts, find averages and patterns.
- Model: build a rule that explains or predicts.
- Report: share findings with charts and simple words, and say what the limits are.
The cycle repeats: the answer often leads to a better question.
Models and what they output
A model is a simplified rule learned from data.
- Prediction (regression) gives a number: marks, price, temperature. Example: marks ≈ 35 + 7 × hours.
- Classification gives a group: spam or not spam, pass or at risk, healthy or ill.
- Clustering finds groups nobody named before, like types of customers.
To read a model's output, ask: what does each number mean, and in what units? A slope of 7 means each extra hour adds about 7 marks.
Comparing and evaluating models
Always test a model on new data it did not learn from.
- Accuracy (for classification) = correct answers ÷ total × 100%.
- Average error (for prediction) = average of how far each guess is from the truth.
The better model has higher accuracy or lower error on new data. A model that is perfect on old data but poor on new data has overfitted: it memorised instead of learning.
Uses, ethics and persistence
Uses: health (spotting disease), sport (player tactics), farming (rain and pest alerts), transport (traffic), business (stock and prices), science (climate studies).
Ethics: protect privacy, ask permission, and check the data is fair. A model trained on biased data gives biased answers.
Persistence: real data is messy. Models often fail at first. Good data scientists try again, test ideas one at a time and write down what they learn.
Try it at home
Ask 10 friends how many hours they slept last night and how alert they feel (score 1–10). Write the data in a table. Remove anything impossible (like 30 hours). Plot the points on squared paper and draw a line through the middle. Use your line to predict the score for 7 hours of sleep. Then ask two more friends and check how close you were.
Key formulas and definitions
- Project cycle: Ask → Collect → Clean → Explore → Model → Report
- Line model: y = a + b × x (b = slope = change in y for each 1 unit of x)
- Accuracy = correct ÷ total × 100%
- Average (mean absolute) error = sum of |actual − predicted| ÷ number of cases
Worked examples
1. Use the model marks ≈ 35 + 7 × hours to predict marks for 4 hours of study.
35 + 7 × 4 = 35 + 28 = 63 marks.
2. A spam filter checks 200 emails and gets 184 right. What is its accuracy?
184 ÷ 200 × 100% = 92%.
3. A model predicts 60, 72 and 80 marks. The real marks were 64, 70 and 85. Find the average error.
Errors: |64 − 60| = 4, |70 − 72| = 2, |85 − 80| = 5. Average = (4 + 2 + 5) ÷ 3 = 11 ÷ 3 ≈ 3.7 marks.
4. Model A has 90% accuracy on training data and 70% on new data. Model B has 82% and 80%. Which should we use?
Model B. It works almost as well on new data. Model A has overfitted: it memorised the old data.
Common mistakes
- Skipping cleaning. One typing error like 140 marks out of 100 can pull the whole model the wrong way.
- Testing a model on the same data it learned from. Always test on new data.
- Thinking a model is always right. It gives a best guess with some error.
- Confusing prediction with classification. One gives a number, the other gives a group.