Towards Adversarial Evaluations for Inexact Machine Unlearning

Venue: arXiv:2201.06640, Tech Report. No peer-reviewed venue found (no journal-ref; dblp indexes no conference or journal entry). Title history: v1 (17 Jan 2022) and v2 (23 Feb 2022) were "Evaluating Inexact Unlearning Requires Revisiting Forgetting" — the title Towards Unbounded Machine Unlearning cites. v3 (22 Feb 2023) carries the current title. Code: https://github.com/shash42/Evaluating-Inexact-Unlearning Verification: verified_fetched — arXiv abs (version history) and the PDF.

Read status: pp. 4–7 read from page images — §2.2 Shortcomings, Theorem 1, §2.3 the Interclass Confusion test, §3 Unlearning Baselines including the CF-k/EU-k definitions, §4.1 setup. §1, §4.2–§5 and appendices not read.

Enters this project as the origin of two Stage 7 methods. Per the panel constraint, the peer-reviewed anchor is SCRUB's NeurIPS 2023 comparison table; Goel et al. is credited as origin. See docs/REPRODUCTION.md.

1. CF-k and EU-k — the specification

From §3.2 verbatim in substance. Both "require only access to S \setminus S_f" — i.e. retain set only, no forget-set access.

EU-k (Exact-unlearning the last k layers).

"We retrain the last k layers of M from scratch using the same training procedure T on retain set S \setminus S_f while freezing prior layers."

CF-k (Catastrophically forgetting the last k layers).

"We finetune the last k layers of M on the retain set S \setminus S_f using the same training procedure T while freezing prior layers, hoping to catastrophically forget S_f. As we avoid re-initializing the last k layers unlike EU-k, we need far fewer epochs, making CF-k more efficient than EU-k."

So the pair differs in exactly one thing: EU-k re-initialises the unfrozen layers, CF-k does not. Everything else — frozen prefix, retain-only data, training procedure — is shared.

Motivation for the layerwise framing (§3.1, "Targeting Areas to Unlearn"): early layers capture generic low-level representations while later layers hold dataset-specific information, and early layers are also the most computationally expensive — so restricting unlearning to the last k layers is both plausible and cheap.

⚠️ SCRUB deviates from this definition, and we follow SCRUB

The re-initialisation difference is substantive: Goel's EU-k restarts those layers from a random init, SCRUB's restarts them from the CIFAR-100 pretrained weights. These are different algorithms and will give different numbers.

Decision for Stage 4/7: implement SCRUB's variant, because Tier A is a reproduction of SCRUB's table. Document the divergence from Goel et al. in the spec and in docs/REPRODUCTION.md, and expose the choice as a config flag rather than hardcoding it, so the Goel variant remains reachable next semester.

2. Theorem 1 — why weight proximity is not sufficient

This is the result Towards Unbounded Machine Unlearning §4 cites when it argues against indistinguishability-in-weight-space.

Theorem 1. There exists a distribution \mathcal{D} such that for any \epsilon, \alpha \ge 0, there exist two \ell-layered fully connected linear NNs parameterised by \mathcal{W}_1, \mathcal{W}_2 which are simultaneously:

• Close in Weights: \|\mathcal{W}_1 - \mathcal{W}_2\|_F \le \epsilon
• Close in Test Error: R(f_{\mathcal{W}_1}) \le R(f_{\mathcal{W}_2}) + \alpha
• Far in Robustness: R_{\mathrm{adv}}(f_{\mathcal{W}_1}) \ge R_{\mathrm{adv}}(f_{\mathcal{W}_2}) + 1 - 2\alpha
• Far in Fairness: \Gamma(f_{\mathcal{W}_1}) = \Gamma(f_{\mathcal{W}_2}) - 1

Their gloss: two models arbitrarily close in weights and test error can be arbitrarily far in adversarial robustness and fairness — indeed "models get farther in adversarial robustness as they get closer in test error." Hence unlearning evaluations "must either measure indistinguishability in terms of more adversarial quantities like robustness and fairness or use strategic non-I.I.D deletion sets."

The separate "not necessary" argument (§2.2). Comparing M_u against a single retrained M^T_r "can exclude a large set of correct solutions which have no information from S_f but behave differently from M^T_r. Consider a randomly initialized network. It clearly has no information from S_f, and indeed satisfies model indistinguishability if we consider T' to be the random initialization process. However, it will be arbitrarily far from M^T_r and will be unnecessarily penalized."

Bears on our Stage 6 harness, alongside Machine Unlearning via Information Theoretic Regularization's auditability critique. Two independent lines now converge on the same point: matching a particular retrained model is neither necessary nor sufficient for unlearning. Our oracle-gap metric is still the right primary measure for a reproduction — it is what SCRUB reports — but the review should state that this is a convention the field has settled on, not a definitional truth, and both critiques should be cited when it is.

3. The Interclass Confusion (IC) test — origin of SCRUB's RC application

SCRUB's "Resolving Confusion" scenario is this test. Its §5.3 says "following Goel et al. [2022]", and its RC metrics are "inspired from Goel et al."

Procedure (§2.3), for a deletion set of n samples:

1. Take \tfrac{n}{2} samples each from two classes in the train data to form S' \subset S (targeted sampling).
2. Swap labels between the two classes of samples in S' (adversarial manipulation) to get the confused set S_f. The dataset for training the original model M is (S \setminus S') \cup S_f.
3. Select S_f as the data to be deleted (strategic deletion set).
4. Evaluate memorisation and property generalisation by measuring error on training and testing sets S' and S'_u corresponding to the two classes.

Ablation — I.I.D. Confusion: select n samples uniformly at random from S and mislabel each to a uniform random different class. Replaces the targeted manipulation with I.I.D. label noise; note the removal is then not I.I.D.

Error vs. Targeted Error — the metric distinction that becomes SCRUB's IC-Err/Fgt-Err:

Their illustration: on a 10-class dataset a random model has 90% error but 10% targeted error. And the measurement split — "Error/Targeted Error when computed on the set S' measures memorization, and when computed on the unseen (test) set samples S_t from the same distribution as set S' measures property generalization."

Mapping to SCRUB's RC metrics (from SCRUB §5.3): SCRUB's IC-Err "counts mistakes where an example from either of the two confused classes was incorrectly predicted to belong to any other class" ≈ Goel's Error; SCRUB's Fgt-Err "counts only misclassification between the confused classes" ≈ Goel's Targeted Error.

This mapping needs confirming against SCRUB's appendix §10 (Formal Description of Metrics), which was not read. It matters for Phase 1's stretch goal (RC, Tables 13–14).

4. Desiderata (§3.1) — including a direct criticism of Bad-T

1. Unlearning procedures need the ability to learn. With complete interclass confusion, an ERM model reaches ~100% train accuracy but ~50% test accuracy; after deleting the mislabelled samples, a model retrained on S \setminus S_f achieves "much larger than 50%". So "solely erasing information from the model is not enough, and the ability to learn may be necessary for ideal unlearning procedures. Consequently, we expect methods which do not use information about the retain set [Chundawat et al. 2022] will have limitations when handling arbitrary deletions and will not perform well on the IC test." → A named prediction about Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher, and consistent with SCRUB's own finding that Bad-T "forgets poorly compared to SCRUB in all applications, and degrades model quality."
2. Scalability to large deletion sets. Popular methods "either explicitly assume tiny deletion sets or scale poorly beyond them in practice."
3. Targeting areas to unlearn. The layerwise motivation for CF-k/EU-k above.

5. Experimental setup (§4.1)

Small-CIFAR-5 (the Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks 5-class subset of CIFAR-10), ResNet-20 (note: not ResNet-18 as in SCRUB), class removal as S_f. Baselines: Fisher (Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks), NTK-Fisher (Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations), Amnesiac (Amnesiac Machine Learning), LCODEC (Mehta et al. 2022).

Two observations worth recording:

• "We could not run Fisher, NTK-Fisher for our larger datasets like CIFAR due to large memory requirements." An independent corroboration of SCRUB's scalability critique of the Golatkar methods — and a warning for our own Fisher implementation, now on 8 GB VRAM.
• "Amnesiac relies on deletion set samples belonging to only a few batches. However, this assumption does not scale to large deletion sets and we find all batches are affected in our experiment." Independent support for this project's decision to drop Amnesiac's batch-subtraction variant.

6. What the paper does not test (from sections read)

• Only ResNet-20 on Small-CIFAR-5 in the section read; k is a free parameter but no sweep over it appears in the pages read.
• §4.2–§4.3 (comparing forgetting tests, making models amenable to unlearning) and §5 Conclusion/Limitations unread.

7. Links

• Both baselines appear in Towards Unbounded Machine Unlearning's comparison table, with the deviations tabulated in §1 above.
• Theorem 1 is the "weight proximity insufficient" result SCRUB cites alongside Unrolling SGD: Understanding Factors Influencing Machine Unlearning.
• The IC test is the origin of SCRUB's RC application and metrics.
• Predicts the failure mode of Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher on retain-set-free methods.
• Its evaluation critique converges with Machine Unlearning via Information Theoretic Regularization and On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning.

@misc{goel2022adversarial,
  title         = {Towards Adversarial Evaluations for Inexact Machine Unlearning},
  author        = {Shashwat Goel and Ameya Prabhu and Amartya Sanyal and Ser-Nam Lim and Philip Torr and Ponnurangam Kumaraguru},
  year          = {2022},
  howpublished  = {arXiv preprint (listed as Tech Report; no journal-ref, no dblp venue entry found)},
  eprint        = {2201.06640},
  archiveprefix = {arXiv},
  url           = {https://arxiv.org/abs/2201.06640},
  note          = {Not peer-reviewed. Also circulated as: Evaluating Inexact Unlearning Requires Revisiting Forgetting (v1/v2 title, and the one SCRUB cites)}
}