Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher

Venue: AAAI 2023 · arXiv: 2205.08096 Verification: verified_metadata_only → now partial_read; dblp for venue, arXiv API for the ID, PDF fetched.

Read status: pp. 3–4 read from page images — §Proposed Method (Eqs. 1–3), §Zero Retrain Forgetting Metric (Eqs. 4–5), §Experiments setup, §Forget Acc. vs Information Exposure Trade-off. §§1–2, results tables and appendix not read.

1. The method

Three models. The student S is initialised with the parameters of the fully trained model. The competent/smart teacher T_s is that fully trained model. The incompetent/dumb teacher T_d is a randomly initialised network.

The student is pulled toward T_d on forget data and toward T_s on retain data.

Equation 1 — KL from the incompetent teacher to the student:

Equation 2 — KL from the competent teacher:

where i indexes the data class.

Equation 3 — the unlearning objective:

where l_u is the unlearning label. §Preliminaries defines it exactly:

"Each data point is assigned an unlearning label, l_u, which is 1 if the sample belongs to \mathcal{D}_f and 0 if it belongs to \mathcal{D}_r. The subset used for unlearning is \{(x_i, l_{u_i})\}_{i=1}^{p}, p is total number of samples, and l_{u_i} is unlearning label corresponding to each sample x_i."

So Eq. 3 is a per-example hard switch between the two teachers, not a weighted blend: forget examples are trained purely toward the incompetent teacher, retain examples purely toward the competent one. There is no interpolation and no tunable mixing weight — a notable contrast with SCRUB, whose Eq. 3 carries \alpha and \gamma balancing its retain-side terms against the forget term.

(Provenance note: the definition of l_u is in Preliminaries, not in the Proposed Method section that states Eq. 3, which only says "l_u is the unlearning label". An earlier draft of this file inferred the 1/0 assignment from the structure of Eq. 3; it is now quoted from the paper.)

Data used: "all the samples from \mathcal{D}_f and a small subset of samples of \mathcal{D}_r."

Stated intuition: "we selectively transfer bad knowledge about forget data \mathcal{D}_f from teacher T_d … and the accurate knowledge corresponding to \mathcal{D}_r is fed from T_s … The student learns to mimic T_d for \mathcal{D}_f, thus removing information exclusively pertaining to those samples while retaining all the generic information which can be obtained by other samples of same class."

They also note a deliberate design choice: passing random knowledge rather than driving accuracy to zero "serves as a protection against the risk of information exposure about the samples to forget."

Contrast with SCRUB, which is the point of the comparison

SCRUB's §4 says the difference is "fundamental": Bad-T moves the student toward a different, 'bad' teacher on forget data, whereas SCRUB moves it away from the single teacher. Structurally: Bad-T minimises a KL to T_d; SCRUB maximises a KL to T_s. Both descend, but Bad-T has a target and SCRUB has an anti-target.

⚠️ Two deviations in SCRUB's reproduction of Bad-T — we follow SCRUB

The temperature difference is easy to miss and materially changes the softened distributions both KL terms are computed over. Both deviations go in docs/specs/bad_teacher.md as AMBIGUITY/CHOICE/RATIONALE comments and in docs/REPRODUCTION.md.

2. ZRF — the Zero Retrain Forgetting metric

Motivated by a complaint that also motivates this project: most unlearning metrics "require a retrained model i.e., training a model from scratch without using the forget set. These metrics can only be interpreted with reference to such a retrained model. Such dependency on the retrained model for unlearning evaluation would lead to higher time and computational costs."

Equation 4 — Jensen–Shannon divergence between the unlearned model M and the incompetent teacher:

Equation 5 — the metric:

over the n_f samples of \mathcal{D}_f. Range [0,1]: close to 1 if the model's behaviour on forget samples is completely random, close to 0 if it shows a specific pattern.

What the ideal value is — and this is the subtle part. Neither extreme is good:

"The ZRF will be 0 when the model almost always classifies a Boeing aircraft as an animal or some other totally different class. The ZRF will be 1 if the model always classifies all classes with same random probability … Both of these (\sim 0 or \sim 1) are not the desirable outcomes."

Ideally ZRF equals what a model trained without the forget set would give; lacking that, they propose the ZRF on a test set as a proxy, "a set that the model has never learned anything specifically."

This is structurally the same trick as SCRUB's rewinding: use an identically-distributed held-out set to stand in for the unavailable retrained reference. Two independent papers reaching for the same substitute is worth a sentence in the review.

3. The Streisand effect — Bad-T names the over-forgetting hazard

Under "Forget Acc. Vs Information Exposure Trade-off":

"An unlearning method should bring the forget set performance closer to the gold (retrained) model instead of simply reducing it. If the performance of the unlearned model deviates a lot from the gold model, it could lead to Streisand effect … unexpected behaviour of the model on forget samples which may leak information about that data. The leak could be in the form of being consistently & maximally incorrect about only the forget samples, signalling that a deliberate effort was made to forget a selected set of samples."

This is exactly the requirement in the project brief — "the target is to MATCH the oracle, not to minimise forget accuracy. Below-oracle forget accuracy is over-forgetting and is itself a privacy leak" — and it now has a citable name and source. It is also the same hazard SCRUB's rewinding exists to fix. Stage 6's signed oracle gap implements it.

They also note the forget-set accuracy "may drop slightly, may not drop at all, or even increase in some cases" when the retain set contains points similar to forget points — worth remembering before treating any accuracy drop as evidence of forgetting.

4. Experimental setup

Datasets: CIFAR-10, CIFAR-100, epileptic seizure recognition (Andrzejak et al. 2001), activity recognition (Anguita et al. 2013). Models: ResNet-18, ResNet-34, MobileNetv2, Vision Transformer, AllCNN for image classification; a 3-layer DNN for seizure; an LSTM for activity recognition. Hardware: NVIDIA Tesla V100 (32 GB), Intel Xeon. PyTorch 1.5.0. KL temperature: 1 for all experiments. Baseline: Amnesiac (Amnesiac Machine Learning), chosen because "unlike most other methods, this method achieves unlearning in an already trained model without putting any constraints on the training procedure." Two unlearning types: sample unlearning and class unlearning.

Evaluation measures: accuracy on \mathcal{D}_f and \mathcal{D}_r; membership inference attack; activation distance (mean L_2 between the unlearned and retrained models' predicted probabilities on the forget set); JS-divergence between unlearned and retrained predictions; ZRF.

An appendix result worth flagging for Stage 7: a partially trained (PT) model — one trained for 1–2 epochs on 50% of the retain data — can substitute for the randomly initialised incompetent teacher. That is a second configuration axis our implementation should expose rather than hardcode.

5. What the paper does not test (from sections read)

• ZRF's behaviour is characterised qualitatively; no calibration of what "close to the test-set proxy" means numerically in the pages read.
• Results tables unread, so no claimed numbers are recorded here.
• Goel et al. predict Bad-T will underperform on the Interclass Confusion test because it makes limited use of retain-set information; SCRUB independently reports Bad-T "forgets poorly … and degrades model quality". Neither critique is addressed in this paper, which predates both.

6. Links

• Baseline Bad-T in Towards Unbounded Machine Unlearning's comparison table, with the temperature and retain-fraction deviations above.
• Its Streisand-effect argument is the same hazard SCRUB's rewinding addresses.
• Criticised in advance by Towards Adversarial Evaluations for Inexact Machine Unlearning §3.1 for methods not using retain-set information.
• Compares against Amnesiac Machine Learning.
• Its ZRF motivation — avoiding dependence on a retrained reference — converges with Machine Unlearning via Information Theoretic Regularization's auditability critique.

@inproceedings{chundawat2023badteacher,
  title         = {Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher},
  author        = {Vikram S. Chundawat and Ayush K. Tarun and Murari Mandal and Mohan S. Kankanhalli},
  year          = {2023},
  booktitle     = {AAAI 2023},
  eprint        = {2205.08096},
  archiveprefix = {arXiv},
  url           = {https://arxiv.org/abs/2205.08096}
}