Amnesiac Machine Learning

Venue: AAAI 2021 · arXiv: 2010.10981 · Affiliation: University of Waterloo. Verification: dblp for venue; arXiv API for the ID; PDF fetched.

Read status: pp. 3–4 read from page images — §Proposed Methods in full (Naive Retraining, Unlearning, Amnesiac Unlearning), §Threat Model, §Empirical Evaluation setup. §1–2, results and §Discussion/Conclusion not read.

Read specifically to resolve an outstanding project decision: is the relabeling variant genuinely separable from batch subtraction, and is it logging-free? Answered in §1 below.

1. ✅ RESOLVED — the Amnesiac decision

The paper proposes three methods, not two, and the relabeling variant is a distinct, separately-named method in its own subsection.

"Unlearning" (the relabeling variant):

"we relabel the sensitive data with randomly selected incorrect labels and then retrain the network for some iterations on the modified dataset. Unlearning for an entire class is achieved by replacing the label for each example in that class with a randomly selected incorrect label, whereas unlearning for a select set of examples is achieved by removing those examples and inserting a small number of copies of each of them with randomly selected incorrect labels. This relabeling is computationally inexpensive and our evaluation shows that this method is effective with only a very small number of training iterations on the modified dataset."

"Amnesiac Unlearning" (batch subtraction) — the one we dropped:

And the decisive passage, from the authors themselves:

"One potential downside to this method is the large storage space required to keep a set of parameter update values from each batch. While this cost can be quite large, especially for state-of-the-art large models … Model owners concerned about this storage cost may be better off using a method such as unlearning that doesn't require this storage overhead."

Verdict on the three test conditions:

So Amnesiac's relabeling variant enters Priority 2 next semester as a normal post-hoc method. The batch-subtraction variant remains permanently dropped. Originals stay uninstrumented, and Stage 5 is unaffected. This closes the decision left open in papers.yaml and in the project brief.

One caveat to carry into its spec: "the data holder must maintain a copy of the sensitive data during the unlearning process, which may potentially have legal significance." That is a requirement at unlearning time only, and SCRUB, NegGrad+ and Bad-T all share it — but it is worth stating, since it is exactly the property When to Forget? Complexity Trade-offs in Machine Unlearning's retain-only algorithm class excludes.

2. Confirms why batch subtraction was rightly dropped

Our earlier reasoning was that the tractable form of batch subtraction requires the forget set to be known at training time. The paper says exactly this:

"A comprehensive approach would be to keep a record of each batch update. However, if the data holder is only concerned about possible potential removal of a subset of the data, they need only keep the parameter updates from batches containing that data."

So either you store every batch update — infeasible at ResNet scale — or you must know in advance which data might be deleted, which contradicts splits being derived independently from (dataset, seed, split_spec).

The authors also flag the non-commutativity that makes the method approximate:

"a model trained on three batches b_0, b_1, and b_2 that then has the parameter updates from b_1 undone will not in fact be the same as a model (with the same initial parameters) trained directly on b_0 and b_2, because the parameter updates from learning on b_2 will be different."

Their mitigation is to keep |SB| small: "When the number of batches in SB is low, the difference between \theta_M and \theta_{M'} is comparatively small… When the number of batches in SB is higher, the model needs a small amount of fine-tuning after the amnesiac unlearning step to regain performance."

This is precisely the failure Towards Adversarial Evaluations for Inexact Machine Unlearning reports empirically — "Amnesiac relies on deletion set samples belonging to only a few batches. However, this assumption does not scale to large deletion sets and we find all batches are affected in our experiment."

3. A lineage worth recording for taxonomy.md

Graves' "Unlearning" — relabel the forget set randomly, then finetune — is the method the NeurIPS competition report calls Random Label, and SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation builds directly on it. From Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition §5:

"Random Label that assigns a random label to each example in the forget set and finetunes on that re-labelled forget set; we added a 'repair phase' that finetunes on the retain set … SalUn based on Random-Label, that enhances that unlearning algorithm by only allowing it to update the 'salient weights'."

So: Graves 2021 (relabeling) → Random Label → SalUn 2024 (+ saliency mask). That is a clean three-step lineage for one branch of the taxonomy, and it means implementing Graves' relabeling variant next semester also lays the groundwork for SalUn.

4. The third method, and a definitional wrinkle

"Naive Retraining" is not retraining from scratch. It is: train on \mathcal{D}, then continue training on \mathcal{D}\setminus S, relying on catastrophic interference.

"Over time a network originally trained on a dataset D and then trained on the same dataset without some subset of that data D \setminus S has a high probability of losing information about S. Although this has a high probability of causing the neural network to forget the desired information, it gives no information or assurances about how long it could take."

⚠️ A terminology trap for the review. "Naive retraining" here means what every other paper in our reading calls Finetune (or CF-k without freezing). It is not the retrain-from-scratch oracle. Their reported conclusion — "naive retraining takes far too long to be considered as a practical solution and even a large amount of naive retraining does not prevent data leaks" — is therefore a statement about finetuning, not about the oracle, and must not be quoted as evidence against retraining from scratch.

5. Setup

Datasets: MNIST (10-class, 60k train / 10k test, 1×28×28 grayscale) and CIFAR-100 (100-class, 600 images per class, 3×32×32). Model: ResNet-18. Environment: Python 3.7, PyTorch, Amazon SageMaker ml.g4dn.xlarge — 4 vCPUs, 1 GPU, 16 GB.

Threat model: white-box access to the currently published version of the model, but not to any previously published versions — explicitly weaker than When Machine Unlearning Jeopardizes Privacy, which assumes access to both. Two attacks: a modified model inversion attack after Fredrikson et al. 2015, and membership inference.

Their model-inversion modifications are worth noting as reimplementation gotchas: they apply the PROCESS image-processing step only every n \in [500,1000] gradient steps rather than every step; they add "a small amount of noise to each feature" at initialisation so each inversion differs; and they continue the attack for a fixed number of iterations rather than stopping when the loss change falls below a threshold. "These modifications allowed us to generate inversions even on complex convolutional architectures."

6. What the paper does not do (from sections read)

• Results, ablations and Discussion unread; no reported number is recorded in this project.
• The relabeling variant's hyperparameters — how many random-label copies to insert, how many finetuning iterations — are described qualitatively ("a small number of copies", "a very small number of training iterations") but no values appear in the sections read. These must come from the paper's evaluation section or its code before a Priority 2 spec.
• No comparison against a retrain-from-scratch oracle in the sections read; the baseline is their own "naive retraining", i.e. finetuning.

7. Links

• Baseline Amnesiac in Towards Unbounded Machine Unlearning's comparison table, and the sole baseline in Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher, which chose it because it "achieves unlearning in an already trained model without putting any constraints on the training procedure".
• Batch-subtraction scaling criticised empirically by Towards Adversarial Evaluations for Inexact Machine Unlearning.
• Its relabeling variant is the ancestor of Random Label and hence of SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation.
• Weaker threat model than When Machine Unlearning Jeopardizes Privacy, which assumes access to both model versions.

@inproceedings{graves2021amnesiac,
  title         = {Amnesiac Machine Learning},
  author        = {Laura Graves and Vineel Nagisetty and Vijay Ganesh},
  year          = {2021},
  booktitle     = {AAAI 2021},
  eprint        = {2010.10981},
  archiveprefix = {arXiv},
  url           = {https://arxiv.org/abs/2010.10981}
}