SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation

Venue: ICLR 2024 · arXiv: 2310.12508 Code: https://github.com/OPTML-Group/Unlearn-Saliency Verification: dblp for venue; arXiv API for the ID; PDF fetched.

Read status: §1 Introduction, §2 Related Work, §3 Preliminaries and Problem Statement, §4 Challenges in full; §5 SalUn in full (Eqs. 3–7, the soft-thresholding variant); §6.1 experimental setup; §6.2 opening discussion; Appendix B (soft-thresholding, Eq. 8), Appendix C.1 (sparsity ablation), Appendix D (unlearning configurations), and the discussions of Tables A1 and A3. Remaining appendices and the numeric bodies of the result tables not read.

Pagination, against the pinned copy. external/MANIFEST.yaml pins this PDF by SHA-256 (f46704f2…, 16,291,581 bytes). In that copy pdftotext page N is printed page N — there is no offset. An earlier reading of this file recorded a +2 offset; that observation was made against a different retrieval of the arXiv record and does not hold for the pinned artefact. Page references below are printed pages in the pinned copy.

Project status: Priority 2, gated to a later phase. Scope: the image-classification half only. Equation 7 covers diffusion-model unlearning, which is out of scope for this thesis.

1. The problem the paper sets itself

SalUn is not framed as another unlearning heuristic. It is framed as a response to two named defects in the existing ones, and the framing is what makes the paper worth reading closely even though only half of it is in scope.

The organising question, boxed in §1 (p. 2):

"Is there a principled approach for effective MU in both classification and generation tasks?"

The three contributions are enumerated in the same paragraph: identifying two limitations of current MU techniques, introducing weight saliency and SalUn, and validating against 7 classification baselines and 2 concept-erasing baselines.

§3 fixes the vocabulary this project also uses. \mathcal{D} = \{\mathbf{z}_i\}_{i=1}^{N} is the training set, \mathcal{D}_f \subseteq \mathcal{D} the forgetting set, \mathcal{D}_r = \mathcal{D} \setminus \mathcal{D}_f the remaining set, and \theta_o the original model. On the oracle they are unambiguous:

"we regard Retrain as the MU's gold standard, which involves training model parameters (\theta) from scratch over \mathcal{D}_r."

§3 also splits image-classification MU into class-wise forgetting and random data forgetting, and adopts wholesale the "full-stack" evaluation of Model Sparsity Can Simplify Machine Unlearning: UA (unlearning accuracy, 1 - accuracy of \theta_u on \mathcal{D}_f), MIA on \mathcal{D}_f, RA (remaining accuracy), TA (testing accuracy) and RTE (run-time efficiency). That is five metrics adopted from a paper this corpus also holds, and it is the same metric set our harness plans.

2. §4 — the instability finding, which is the paper's real contribution here

§4 re-examines five MU methods — FT, RL, GA, IU and \ell_1-sparse — and names two limitations: "the lack of unlearning stability and generality".

The instability limitation is the one that matters for this project's evaluation design:

"There has been limited evaluation exploring how the unlearning performance is affected by varying quantities of forgetting data."

Their Figure 2a plots the average gap to Retrain across UA, RA, TA and MIA at 10% and 50% forgetting, and the conclusion is stated flatly:

"the unlearning effectiveness of MU methods (①-⑤) observed at a 10% forgetting data quantity does not necessarily hold when the forgetting data quantity is increased to 50%."

A second form of instability is hyperparameter sensitivity, shown for influence unlearning:

"we observe a notably high variance in the unlearning performance of IU compared to Retrain. By contrast, the integration with our proposal (SalUn) reduces this instability."

Consequence for this project's harness. A method ranked at one forgetting ratio may not hold that rank at another, and the claim is made by a paper that measured it across ratios with 10 seeds. Reporting a single forget-set size is therefore a design decision with a known failure mode attached, not a neutral simplification. This is independent support for varying |\mathcal{D}_f| in Stage 6 rather than fixing it once.

The generality limitation is the diffusion-model half and is out of scope, recorded only so the argument is not mistaken for a classification result: existing methods "either over-forget, resulting in poor generation quality for image classes in \mathcal{D}_r …, or under-forget, leading to unsuccessful unlearning".

3. The method

The rationale, from §5 (p. 5). Weight saliency is motivated by modularity — "contemporary ML models exhibit modularity characteristics to some extent" — and positioned explicitly against weight sparsity, which is Model Sparsity Can Simplify Machine Unlearning's mechanism. Two limitations of sparsity are given:

"(1) Determining the appropriate sparse pattern for an ML model (e.g., a DM) can be a challenging task in itself; (2) Even when sparsity is achievable, some applications may not favor delivering a sparse model after MU due to the observed performance decline, as exemplified by the \ell_1-sparse MU method in Sec. 4."

That is a second, mechanistic criticism of \ell_1-sparse, distinct from the single-metric criticism in §6 recorded below. §2 places the idea in a wider lineage: weight saliency is contrasted with input saliency and data attribution, related to weight pruning, and connected to the "editable model region" notion from NLP model editing.

Equation 3 — the gradient-based weight saliency map:

\mathbb{1}(\mathbf{g}\ge\gamma) is an element-wise indicator, |\cdot| element-wise absolute value, and \gamma>0 a hard threshold.

The median setting, and its limits. The authors state: "we have observed that setting \gamma to the median of the gradient vector … is a sufficiently effective choice", and Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition reports "the sparsity of weight saliency is set at 50%". But Appendix C.1 qualifies this and must be read with it: "the choice of a 50% sparsity ratio is not universal, and when dealing with a larger amount of forgetting data, a higher sparsity ratio is preferred." A higher sparsity under-forgets, a lower one over-forgets, and §6.1 reports searching sparsity over [0.1, 0.9] rather than fixing it at the median. The 50% figure is a default, not a determined value, and a spec that hardcodes it inherits a tuning decision the paper did not make for us.

Equation 4 — the masked update:

"during weight updating in MU, the attention can be directed towards the salient weights." Non-salient weights are frozen at their original values.

Equation 5 — the forgetting loss used to build the map:

They note "the forgetting loss in GA (gradient ascent) (Thudi et al., 2022a) presents an effective and simple solution" — i.e. the saliency map is built from the plain cross-entropy gradient on the forget set. (Note the citation is to Thudi 2022a, the Unrolling SGD paper, consistent with the citation-key mapping recorded in thudi2022unrolling.md.)

Equation 6 — SalUn for image classification, the in-scope objective:

with y' a random label different from y, \theta_u from Eq. 4, and \alpha>0 balancing forgetting against retain-set generalisation.

The lineage is confirmed by SalUn's own construction

Equation 6 is exactly Random Label on the forget set plus a retain-set cross-entropy regulariser, restricted to salient weights. And Random Label is Amnesiac Machine Learning's relabeling variant. So the chain recorded earlier — Graves 2021 (relabeling) → Random Label → SalUn 2024 (+ saliency mask) — is not an inference from the competition report's description; it is visible in SalUn's objective.

The authors confirm the choice was empirical, not incidental: Appendix C's discussion of Table A1 says "the weight saliency-augmented RL … surpasses all other baselines. This justifies why we selected RL as the foundational building block in SalUn."

Practical consequence: implementing Graves' relabeling variant gives Eq. 6 minus the mask, so SalUn becomes a small increment rather than a fresh build.

Plug-and-play, and the ablation that supports it. "One advantage of SalUn is its plug-and-play capability, allowing it to be applied on top of existing unlearning methods." Table A1 integrates \mathbf{m}_S into FT, RL, GA and IU; the accompanying text reports that "the integration with \mathbf{m}_S improves the performance of existing baselines, bringing them closer to the Retrain benchmark". The mask is therefore a modifier, and our method interface should expose it as such rather than hardcoding it into one method.

Soft-thresholding variant (SalUn-soft) replaces the hard threshold with an \ell_1 penalty (their Eq. 8) solved by proximal gradient, and "requires tuning an additional hyperparameter". It "does not outperform the hard-thresholding version. Thus, we will focus on the hard-thresholding implementation by default." Appendix B adds that "employing a linearly decaying scheduler … yields better results than other schemes".

Equation 7 — SalUn for image generation. Out of scope; recorded only so the spec does not accidentally implement it.

4. Two methodological points worth taking

(a) A third independent statement of the match-the-oracle principle. "a better approximate unlearning method should exhibit a smaller performance gap compared to Retrain." They operationalise it as an avg. gap metric: the mean of performance gaps against Retrain across UA, MIA, RA and TA. That is the same signed-gap-to-oracle logic our Stage 6 harness implements, now stated by Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher (Streisand effect), Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition (utility ratios) and here.

(b) An explicit warning against single-metric evaluation, with a worked example:

"it is important to avoid evaluating the performance of MU using a single metric, as it can lead to a misleading sense of effectiveness. For instance, \ell_1-sparse may appear to be the strongest baseline in the 10% random data forgetting scenario when considering UA alone. However, this apparent strength comes at the cost of sacrificing RA and TA."

The \ell_1-sparse baseline is Model Sparsity Can Simplify Machine Unlearning — so this is a direct, citable critique of another paper in our database, and a concrete argument for the multi-metric reporting our harness already plans.

5. Sequential unlearning — SalUn does report it

Appendix C reports an iterative unlearning experiment: 10% of the data forgotten per iteration over five iterations, 50% in total, on ResNet-18/CIFAR-10, compared against Retrain and FT.

"even as data points are progressively forgotten, SalUn demonstrates a consistently minimal performance gap with Retrain".

This corrects a claim made elsewhere in this project. gaps.md Gap 2 and LITERATURE_REVIEW.md §8 both stated that sequential behaviour appears in none of the vision-method notes. That was true of the notes as written; it is not true of the literature. SalUn Table A3 is a sequential-request experiment in a vision method, and the gap must be restated as what it actually is: sequential evaluation is rare and unstandardised, not absent. The correction is recorded in VERIFICATION_REPORT.md.

6. Setup

Primary: CIFAR-10 with ResNet-18, 10% and 50% random data forgetting. Additional: CIFAR-100, SVHN, Tiny ImageNet; VGG-16 and Swin-T architectures; class-wise forgetting (their Appendix C.2). Seeds: 10 independent trials, reported as mean ± standard deviation. Five metrics: UA, MIA, RA, TA, RTE (minutes). Nine baselines: FT, RL, GA, IU, \ell_1-sparse, BS, BE, ESD, FMN.

The oracle configuration, from Appendix D. "For the Retrain method, training is conducted over 182 epochs using the SGD optimizer with a cosine-scheduled learning rate initialized at 0.1." SalUn itself is "trained for 10 epochs, searching for learning rates in the range [5\times10^{-4}, 5\times10^{-2}] and sparsity ratios in the range [0.1, 0.9]".

Note the seed count: 10 trials. More than SCRUB's 3 and more than our planned 5. Relevant to the Stage 5 seed decision — the better-resourced 2024 papers are running 10.

7. What the paper does not do, and what it does not state

• \alpha in Equation 6 is never given a numeric value. This is now a confirmed absence rather than an unread section: Appendix D specifies learning rates, epoch counts and sparsity ranges for every method including SalUn, and specifies \beta for the generation objective (Eq. 7), but the classification objective's retain-regularisation weight \alpha appears nowhere in the configurations. A reproduction must take \alpha from the released code or treat it as a swept hyperparameter, and must say which.
• The numeric bodies of Tables 1 and A1–A11 are unread; every claim above is the authors' qualitative statement, recorded as theirs.
• Random data forgetting is the primary setting; class-wise forgetting is in Appendix C.2.
• Sequential unlearning is reported (Table A3) but only for SalUn, Retrain and FT, at one request size, on one dataset — so it establishes that the setting is studiable, not that it is standardised.

8. Links

• Builds on Random Label, hence on Amnesiac Machine Learning's relabeling variant; the lineage is visible in Eq. 6 and confirmed by their Table A1 ablation.
• Uses the gradient-ascent forgetting loss it attributes to Unrolling SGD: Understanding Factors Influencing Machine Unlearning (2022a).
• Criticises Model Sparsity Can Simplify Machine Unlearning twice over: mechanistically in §5 (sparse patterns are hard to find and may not be wanted) and empirically in §6 (single-metric evaluation flatters it).
• Adopts Model Sparsity Can Simplify Machine Unlearning's five-metric "full-stack" evaluation wholesale.
• Its avg.-gap-to-Retrain metric agrees with Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher and Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition on matching rather than maximising.
• Described compactly by Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition §5, which is what first flagged the Random-Label connection.
• Cites Membership Inference Attacks From First Principles for the membership-inference framing of MU evaluation.
• Its §4 instability finding bears on Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy's argument that aggregate forgetting numbers hide per-example variation: both say a single reported figure is doing less work than it appears to.

@inproceedings{fan2024salun,
  title         = {{SalUn:} Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation},
  author        = {Chongyu Fan and Jiancheng Liu and Yihua Zhang and Eric Wong and Dennis Wei and Sijia Liu},
  year          = {2024},
  booktitle     = {ICLR 2024},
  eprint        = {2310.12508},
  archiveprefix = {arXiv},
  url           = {https://arxiv.org/abs/2310.12508}
}