An unsupervised anomaly-detection model trained on the ATO Individual Sample File 2022–23. It segments taxpayers into peer groups with K-Means, scores each return against its own group with an Isolation Forest, and explains every score with SHAP.
Undram JargalsaikhanMaster of Data Science capstone · September 2026
Figure 1. Eight key variables among the taxpayers who report them, log scale $1–$3m. Amounts span seven orders of magnitude, so the model uses sign(x)·ln(1+|x|). Population aggregates only.
Feature engineering — the twelve ratios
43 amount features survive screening; twelve scale-free ratios are added so that $10,000 means something different on a $30,000 salary than on a $300,000 one.
Income & deductions
Deductions÷ total income
Work expenses÷ salary
Gifts÷ total income
Taxable-income identity÷ expected taxable
Activity-specific
Rental deductions÷ gross rent
Business expenses÷ business income
Investment deductions÷ investment income
Personal super÷ total income
Filing controls
Tax-affairs cost÷ total income
PAYG instalments÷ total income
Items reportedcount of non-zero fields
Lodged via agent0 / 1
Peer groups — K-Means
Grouped on seven income-source shares plus scale. Deductions are deliberately kept out, so high claims cannot become "normal" by construction.
k = 8 is the best-separated choice
Silhouette by number of groups, 20,000-record sample.
Group size
Share of the 321,890 returns.
Figure 2. Silhouette 0.658, Davies–Bouldin 0.72; the smallest group still holds 4,223 returns, enough for its own forest. Clustering raw dollar amounts instead gives silhouette 0.05–0.13.
The groups separate in two principal components
20,000 taxpayers coloured by group; PC1 carries 24% of variance, PC2 14%.
Figure 3. Each group occupies its own region of the nine-feature composition space, projected to two dimensions. Labels mark group centroids; the wage group dominates, the rest fan out by income source.
What each group earns from
Mean share of total income by source. Darker = larger share.
Figure 4. Each group is 75–94% one income type on average. Occupation was never given to the algorithm, yet it aligns: 23% of the wage group are professionals, while 60–96% of the non-wage groups state no occupation.
Input: 55 features · 8 groupsOutput: anomaly score and 2% flag per return
One 200-tree forest per group scores each return by how few random splits separate it from its peers — a landlord is compared with landlords, not with everyone.
Where the review cut-off falls
Anomaly scores in the wage group (233,317 returns). Red marks the top 2%.
Flags look the way a reviewer expects
How much more often each pattern appears among flags (lift).
Figure 5. Left: scores cluster near 0.35 with a long right tail; the 2% budget is a review capacity — a policy choice — not an estimate of how many returns are wrong. Right: 87% of flags trigger at least one reviewer-style check, against 19% of unflagged returns; three or more items rare in the group are 23× more common among flags.
Rare kinds of return are no longer over-flagged
Share of each group flagged at the same 2% budget.
One forest for everyoneOne forest per group
The two designs agree on 41% of the list
Returns flagged at the 2% budget.
Figure 6. Left: a single population-wide forest reviews 7.7% of investment-income taxpayers and 6.9% of landlords but 0.05% of the no-income group — flagged for being rare, not irregular. Per-group forests hold every group at 2%. Right: only 3,752 taxpayers appear on both lists (Jaccard 0.41), although scores correlate at ρ = 0.79.
The list grows with the budget; agreement barely does
Returns flagged by the per-group design, and share also on the population-wide list.
Ratios move flags from the richest to the rest
Ablation: where flags land when only the 12 ratios are used.
Flags in top income decileFlags below median income
Figure 7. Left: from 1% to 10%, the flagged set grows from 3,224 to 32,192 while agreement rises only from 39% to 52%. Right: with ratios only, flags on the top 10% of incomes fall from 49% to 19% and flags below median income rise from 14% to 46%. Segmentation alone does not remove income confounding — flagged returns have 2.5× the median income under either design — the feature space does.
0.98
score correlation across five random seeds
72%
of flags survive all five seeds
2.5×
median income of flagged returns vs population, either design
Stage 2 result. Segmentation buys equity and interpretability, not sensitivity. The feature space decides what gets flagged; the peer group decides who.
Input: 55 features + Isolation Forest flag as labelOutput: precision, recall, F1, PR-AUC
Random forest and histogram gradient boosting are trained to reproduce the Stage 2 flag on a hold-out set of 96,567 returns (1,933 flagged, ≈ 2% positives). Accuracy is 0.98 for both — and for predicting "normal" for everyone — so it is not used.
Both models recover most of the list at a matched 2% budget
Hold-out set, top 2% of predicted probability flagged.
Hist. gradient boostingRandom forest
Random forest at the 0.5 threshold
Confusion matrix, hold-out set of 96,567 returns.
Predicted flagged
Predicted normal
Actually flagged
1,225
true positives
708
false negatives
Actually normal
373
false positives
94,261
true negatives
Figure 8. At the matched budget, gradient boosting reaches precision 0.79 / recall 0.79 (PR-AUC 0.88) and random forest 0.71 / 0.71 (PR-AUC 0.80). At the default 0.5 threshold the trade-off differs: RF 0.77 / 0.63, HGB 0.50 / 0.96.
Model
Threshold
Precision
Recall
F1
PR-AUC
Random forest
0.50
0.767
0.634
0.694
0.795
Random forest
top 2%
0.711
0.710
0.710
0.795
Hist. gradient boosting
0.50
0.497
0.961
0.655
0.881
Hist. gradient boosting
top 2%
0.791
0.790
0.791
0.881
0.700 ± 0.010
five-fold CV F1, random forest (HGB 0.647 ± 0.003)
12×
faster training for gradient boosting (8.2 s vs 95.5 s)
79%
of the Stage 2 list reproduced by gradient boosting at the same budget
Stage 3 result. The unsupervised flag is a consistent, learnable rule — not noise. These numbers measure agreement with Stage 2, not fraud-detection performance.
Input: fitted forests and a surrogate for K-MeansOutput: per-taxpayer explanations
Why a taxpayer is in a group
Mean |SHAP|, surrogate model at 99.7% fidelity.
Why a taxpayer is flagged
Mean |SHAP| across the eight forests, top 10 of 55 features.
Figure 9. Left: group membership is about what kind of income is reported, not how much — scale and super balance contribute almost nothing. Right: forests respond to the presence of items most peers never report — rent, capital gains, rental interest, personal super, gifts. High values push a return toward "anomalous"; absence is neutral.
99.7%
fidelity of the surrogate random forest that explains group membership
1.00
correlation between summed SHAP values and the forest's own score, every group
3
named items are enough to explain each of the ten highest scores
Why was this return flagged?
The ten highest-scoring returns in the file. Click a record to open its explanation: how it compares with its group and which items drove the score. Values are rounded multiples of population statistics; no amount can be reconstructed.
Record
Segment
Score
Top item
× average
× top 1%
Figure 10. "× average" divides the reported value by the population mean of that item; "× top 1%" by the 99th percentile. Every one of the ten shares the same structure: several items most peers never report, each at many times the average. None of this establishes that any claim is incorrect.
Enter a return, or start from a preset, and the fitted TaxLens model runs here in your browser: it places the return in a peer group, scores it against that group, and explains the score with SHAP — the same pipeline as Stages 1–4. Nothing you type leaves this page.
Start from:
Pick a preset or fill in the return, then select Analyse return.
A flag is a statistical peer-group signal only — never evidence of non-compliance or fraud. Amounts are for the 2022–23 income year; fields left blank count as $0. Investment income up to $10,000 is treated as interest and half of anything above that as capital gains (with the 50% discount); employer super is assumed at 10.5% of salary; the totals are derived automatically.
A flag is
A statistical anomaly relative to a peer group
At most, a potential compliance-risk indicator
A ranked, explained starting point for human review
A flag is not
Evidence of fraud or non-compliance
A finding — false positives dominate by design
Stable across segmentations: boundary taxpayers can change status
What the evidence does not cover
No audit labelsOnly statistical anomaly is measured; sensitivity to real non-compliance is unknown.
One year, 2% sampleUnusual in one year may be a normal transition. The upper tail is compressed.
Entity-spanning behaviourArrangements across companies, trusts, years or families are invisible to 55 features.
SHAP explains the modelNot the data-generating process; attributions depend on the feature set.