Your mannequin has 95% accuracy — so it should be good, proper? Not essentially. Right here’s how one can inform in case your mannequin is definitely serving to or simply faking it.
While you prepare your first machine studying mannequin, likelihood is you’ll take a look at one metric: accuracy.
And whereas accuracy is an effective start line, it’s additionally tremendous straightforward to misread.
Think about this:
- You construct a mannequin to foretell if a transaction is fraud.
- Just one% of transactions are literally fraudulent.
- Your mannequin predicts “not fraud” each time.
- Increase: 99% accuracy — and utterly ineffective.
Let’s dig into the instruments and strategies that inform you in case your mannequin is nice, unhealthy, or simply memorizing patterns.
First rule of ML: by no means consider your mannequin on the identical information you educated it with.
That’s like working towards solutions earlier than a take a look at — after which grading your reminiscence, not your understanding.
Use train_test_split() to carry out a portion of your information:
from sklearn.model_selection import…
