Do You Really Need AI for That?

The calculator that thinks

A while ago I saw someone put AI into a calculator. Obviously a joke I hope.

Think about what that means. You type 7 × 8. Somewhere, a data centre wakes up, a model reads your question, weighs a few billion parameters and replies with the word it thinks is most likely to come next. It will probably say 56. Probably.

Your calculator already knew. It has known since 1972, it runs on a solar cell the size of a stamp and it has never once been unsure.

AI is everywhere right now. Everyone and their mum uses it, and every product seems to need a sparkly button that says “Ask AI”. Some of that is genuinely useful. A lot of it is a very expensive way to do something a twenty-line function already does better.

Somewhere along the way we lost track of the most important skill in computer science: knowing which tool fits which problem. So in this piece I want to take one real problem, predicting how students will do in school, and work out honestly where AI helps, where it doesn't and where the answer is a bit of both.

But first, a quick reminder of what we're actually choosing between.

Three ways to solve a problem

People usually frame this as “AI or not AI”. That framing is the reason the calculator happened. There are really three levels, and most of the interesting decisions sit in the middle one.

Level 1: plain algorithms

An algorithm is a recipe a human writes. Given the same input, it gives the same output, every single time. “If attendance is below 80%, flag the student.” You can read it, test it and explain it to a parent in one sentence. When it's wrong, you know exactly which line to fix.

The catch is that you have to know the rule in advance. Algorithms are only as smart as the person who wrote them.

Level 2: machine learning

Machine learning flips the recipe around. Instead of writing the rule, you show the computer thousands of examples and it works out the rule itself. What comes out is a probability: “this student has a 72% chance of failing”.

This is still small and cheap. A model like this trains on a laptop in under a second. It's also mostly inspectable: you can ask which factors it leaned on.

Level 3: generative AI

This is what most people mean by “AI” today: large language models like ChatGPT or Claude. They are trained on enormous amounts of text and are extremely flexible. They can write, summarise, explain and chat.

They are also enormous, expensive to run and not built for counting things in a spreadsheet. Asking one to predict a grade from a table of numbers is like hiring a novelist to do your taxes. They might get it right, but they're not the right hire.

The skill is picking the lowest level that actually solves the problem. Let's try it on something real.

Meet the students

Our data comes from two secondary schools in Portugal, collected in the 2005–06 school year, published by Paulo Cortez and Alice Silva in 2008, and now one of the most used teaching datasets on the internet. There are 395 students in the maths class. For each one we know about 30 things, and three grades on a scale of 0 to 20: G1 (first term), G2 (second term) and G3 (the final grade). Below 10 is a fail, and 130 of the 395 students (about a third) failed. 38 of those have a final grade of 0, which mostly means they didn't sit the exam at all.

The question a school actually cares about: which students are going to fail, and can we spot them early enough to help?

Here are some of the things we know about each student:

Play with it for a minute. Some factors obviously matter. Some look like they should and don't. Keep your guesses in mind, because we're about to test them.

Round one: the rule wins

Say it's late in the year. The second-term grades are in. Can we predict who fails the final?

Try this rule: if a student's G2 is below 10, they'll probably fail.

That's it. One comparison. No training, no model, no GPU.

Now let's throw a proper machine learning model at the same problem, with every factor we're allowed to use, including G1 and G2. I tried three common ones: a logistic regression, which gives each factor a weight and adds them up; a decision tree, which asks a short series of yes-or-no questions; and a random forest, which builds 300 slightly different trees and lets them vote.

One comparison beat all three models on accuracy and on students caught. The forest raises one fewer false alarm, and that's its only win. The slider shows the trade-off inside the rule itself: at G2 < 8 it never raises a false alarm but only catches half of the struggling students, and at 10 it catches 122 of 130.

The fancy models don't just fail to beat one line of code. They lose to it. The reason is simple: the best predictor of your final grade is your grade a few months earlier. The models know that too, but they also try to use 27 other columns, and those mostly add noise.

This is the AI calculator problem in disguise. When one obvious signal does almost all the work, a model just adds cost, complexity and a harder explanation for the teacher. When the answer is already sitting in one column, use the algorithm.

But there's a problem with this rule. By the time G2 is in, the year is almost over. We caught the students who were struggling, but too late to help most of them.

Round two: nobody wins outright

Let's make it harder, and more useful. It's the first week of term. There are no grades yet. All we have is what we know about the students themselves. Who needs help?

Your turn to write the rule.

You'll probably find a decent first rule quickly. Past failures is a strong one. Then it gets fiddly. Adding a second condition helps a bit. A third one starts catching students in one group while missing them in another. Study time matters, unless family support is high. Absences matter, but only past a certain point. Going out a lot matters more for students who already failed once.

These are interactions, and humans are bad at writing them by hand. You end up with a twelve-line if statement that nobody trusts, including you.

So surely this is where machine learning shines. A model looks at every combination at once and finds the patterns you'd never think to write. Let's see.

I expected the model to win here. On accuracy, it didn't. The one-line rule beat every model, and the two simplest models were less accurate than just predicting that everyone passes.

But accuracy is the wrong scoreboard. Two thirds of students pass, so a system that ignores every struggling student still scores 67%. What a school cares about is how many struggling students it catches. On that, the logistic regression catches 72 of 130, against 52 for the rule. That's 20 more students who get help in the first week.

The price is false alarms: 76 instead of 31. And the most sophisticated model of all, the random forest, caught the fewest students (37), because it played it safe and mostly predicted a pass.

The tree's first question is past failures, the same thing the simple rule looks at. After that it checks extra school support, absences above 17 and study time. One branch flags students who get extra school support and study more than two hours a week, probably because that support goes to students who are already struggling.

So the honest result of round two isn't “the model wins”. Without grades, nobody predicts well, and the rule and the model are making different trade-offs. The rule is cautious and precise. The model casts a wider net. Which is better depends on a question the data can't answer: how many false alarms is one caught student worth?

When the signals are weak, a model's real value isn't accuracy. It's a risk score you can tune, and someone still has to decide where to set the dial.

Round three: both

So the model tells us Ana has a 0.71 chance of failing. Now what?

A probability isn't a decision. Someone has to turn it into an action, and that someone is usually a busy teacher with thirty other students. If we hand them a list of 395 probabilities, they'll ignore it. If we hand them a black box that says “call Ana's parents”, they shouldn't trust it.

This is where the algorithm comes back. The model and the rule each do the part they're good at:

1. The model scores. At the start of term, every student gets a risk score from 0 to 1.
2. The rule decides. Risk above 0.7 means a tutor check-in. Risk between 0.4 and 0.7 means the teacher keeps an eye out.
3. The rule updates. Once first-term grades (G1) arrive, anyone below 10 gets a check-in, and so does anyone the model rated above 0.7 who scraped a 10 or 11.

Here's how that played out on the real data:

At the start of term, 0.7 is a strict line: only 43 struggling students get a check-in, although 42 more land in the “keep an eye out” band. That line is the dial from round two, now in the school's hands. Set it at 0.5 and you get exactly the logistic regression from round two: 72 caught, 76 false alarms. Raise it and you get fewer, more certain check-ins. Once G1 arrives, the grade does the heavy lifting and the system catches 108 of 130.

But look at the last row. Once G1 exists, the plain rule “G1 below 10” catches 104 students on its own. Keeping the model in adds just 4 more, at the cost of 8 extra false alarms, and overall accuracy drops slightly. So the model earns its place at the start of term, when there are no grades. Once real grades arrive, its job is mostly done and the algorithm takes over again. That's what “both” really means: each tool does the part of the year it's best at, then steps aside.

Why is this better than either alone?

• The model catches patterns early that no human would write down.
• The rule makes the final call explainable. A teacher can say exactly why Ana was flagged.
• The rule is where the school sets its own priorities. Want to catch more at-risk students and accept more false alarms? Move the threshold. No retraining needed.
• Each tool is used only when it's the best tool. The model handles the early, messy part of the year. The algorithm handles the late, obvious part.

This is how most good real-world systems are built. Fraud detection, spam filters and medical triage all look like this underneath: a model that scores, wrapped in rules that decide. The answer becomes “both” when you need the model's pattern-finding and a human's accountability at the same time.

The uncomfortable bit

Remember the factors list? Some of those columns are about the student. Some are about their parents.

The model doesn't care. If parents' education or a mother's job improves accuracy by a percentage point, the model will happily use it. Mathematically, that's fine. Ethically, it means a fifteen-year-old gets flagged partly because of who their parents are, which is precisely the thing schools are supposed to help students rise above.

This is another reason the hybrid matters. The rule layer is where you write down what the system is not allowed to do. “Never use parental education or job when deciding who to contact” is one line, and a model can't argue with it. You can also drop those columns before training, and in this project I did.

I dropped six columns before training: both parents' education, both parents' jobs, sex and home address type. Then I added them back to see what they were worth.

They were worth nothing. Accuracy actually fell slightly, from 66.1% to 64.8%. The model caught 2 more struggling students and raised 7 more false alarms. But it changed the flag for 57 of the 395 students.

That's the uncomfortable part. Family background barely changes how well the model does, but it changes a lot about who it points at. Roughly one student in seven would be treated differently because of their parents, for no gain at all.

The broader point is that “should we use AI?” was never only a technical question. Choosing the tool also means choosing who gets to understand and challenge the decision.

Under the bonnet

For anyone who wants to see how this was built. The whole analysis is a short Python notebook, and the interactive parts run in the browser with no server.

1. Prepare the data. Load the maths class CSV, turn yes/no columns into 1/0, one-hot encode the categorical ones and create the target: fail = G3 < 10. Drop the columns excluded for fairness reasons.

import pandas as pd

df = pd.read_csv("student-mat.csv", sep=";")
df["fail"] = (df["G3"] < 10).astype(int)

excluded = ["Medu", "Fedu", "Mjob", "Fjob", "sex", "address"]
X = pd.get_dummies(df.drop(columns=["G3", "fail", *excluded]),
                   drop_first=True)
y = df["fail"]

2. Two scenarios. Late year, where G1 and G2 are allowed as features. Early year, where they are dropped. Mixing these up is the most common mistake people make with this dataset, because G2 makes everything look easy.

3. Baseline rules. The G2 rule for late year, and the best hand-written rule for early year. Always build the dumb baseline first. If you can't say what the model is beating, you don't know if it's any good.

4. Models. Logistic regression (readable weights) and a shallow decision tree (readable structure). Evaluated with stratified 5-fold cross-validation, because 395 students is small and one lucky split can lie to you.

from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score, StratifiedKFold

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
for name, model in [
    ("logistic", LogisticRegression(max_iter=1000)),
    ("tree", DecisionTreeClassifier(max_depth=3)),
]:
    scores = cross_val_score(model, X, y, cv=cv, scoring="recall")
    print(name, scores.mean())

5. Pick the right metric. Accuracy is misleading here, since most students pass. What a school actually cares about is recall (how many struggling students did we catch?) balanced against precision (how many false alarms did the teacher have to deal with?).

6. The hybrid. The model outputs probabilities with predict_proba. The policy layer is a plain function written in a few lines, so anyone can read it and change it.

7. The browser bit. The trained tree and the logistic regression weights are exported to JSON, and the interactive sections recompute predictions in JavaScript as the reader moves sliders. No model is called at runtime. Which, given the topic, felt only fair.

So, do you need AI for that?

Looking back over the three rounds:

None of this is anti-AI. I study it and I like it. But reaching for the biggest tool first isn't ambition, it's skipping the thinking. The job is to understand the problem well enough that the right tool becomes obvious.

So next time you see an “Ask AI” button on something that used to be a calculator, ask the question we asked here. What is the simplest thing that actually works?