Assessing perceptual and recommendation mutation of adversarially-poisoned visual recommenders

Visually-aware recommendation leverages visual signals of product images extracted through Deep Neural Networks to improve the recommendation performance. However, human-imperceptible adversarial noise can alter recommendation outcomes, e.g., pushing/nuking specific product categories. In this work, we provide 24 combinations of attack/defense strategies, and visual-based recommenders to 1) access performance alteration on recommendation and 2) empirically verify the effect on final users through offline-visual metrics. The results suggest defense is not protecting recommender models as expected, and shed light on the importance of human evaluation to identify visual attacks on recommendations. Source code, data, and experimental parameters are available at https://github.com/sisinflab/ Perceptual-Rec-Mutation-of-Adv-VRs.