AWEncoder: Adversarial Watermarking Pre-Trained Encoders in Contrastive Learning
Source abstract
Abstract
As a self-supervised learning paradigm, contrastive learning has been widely used to pre-train a powerful encoder as an effective feature extractor for various downstream tasks. This process requires numerous unlabeled training data and computational resources, which makes the pre-trained encoder become the valuable intellectual property of the owner. However, the lack of a priori knowledge of downstream tasks makes it non-trivial to protect the intellectual property of the pre-trained encoder by applying conventional watermarking methods. To deal with this problem, in this paper, we introduce AWEncoder, an adversarial method for watermarking the pre-trained encoder in contrastive learning. First, as an adversarial perturbation, the watermark is generated by enforcing the training samples to be marked to deviate respective location and surround a randomly selected key image in the embedding space. Then, the watermark is embedded into the pre-trained encoder by further optimizing a joint loss function. As a result, the watermarked encoder not only performs very well for downstream tasks, but also enables us to verify its ownership by analyzing the discrepancy of output provided using the encoder as the backbone under both white-box and black-box conditions. Extensive experiments demonstrate that the proposed work enjoys quite good effectiveness and robustness on different contrastive learning algorithms and downstream tasks, which has verified the superiority and applicability of the proposed work.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Adversarial Robustness in Machine Learning · Digital Media Forensic Detection · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China
Wenzhou Institute, Shanghai University, Wenzhou 325088, China
Funding
National Natural Science Foundation of China · Scientific and Technological Innovation Plan of Shanghai STC · Wuxi Municipal Health Commission Translational Medicine Research Project
References (34)
Show source-provided references
1. Self-supervised visual feature learning with deep neural networks: A survey10.1109/TPAMI.2020.2992393 ↗
2. Bootstrap your own latent: A new approach to self-supervised learning
3. Chen, X., Fan, H., Girshick, R., and He, K. (2020). Improved baselines with momentum contrastive learning. arXiv.
4. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020, January 13–18). A simple framework for contrastive learning of visual representations. Proceedings of the 37th International Conference on Machine Learning, Virtual.
5. He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 13–19). Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00975 ↗
6. Orekondy, T., Schiele, B., and Fritz, M. (2019, January 15–20). Knockoff Nets: Stealing functionality of black-box models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00509 ↗
7. Chandrasekaran, V., Chaudhuri, K., Giacomelli, I., Jha, S., and Yan, S. (2020, January 12–14). Exploring connections between active learning and model extraction. Proceedings of the 29th USENIX Conference on Security Symposium, Boston, MA, USA.
8. Rethinking deep neural network ownership verification: Embedding passports to defeat ambiguity attacks
9. Zhang, J., Chen, D., Liao, J., Fang, H., Zhang, W., Zhou, W., Cui, H., and Yu, N. (2020, January 7–12). Model watermarking for image processing networks. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
10. Chen, J., Wang, J., Peng, T., Sun, Y., Cheng, P., Ji, S., Ma, X., Li, B., and Song, D. (2021). Copy, Right? A testing framework for copyright protection of deep learning models. arXiv.10.1109/SP46214.2022.9833747 ↗
11. Watermarking neural networks with watermarked images10.1109/TCSVT.2020.3030671 ↗
12. Uchida, Y., Nagai, Y., Sakazawa, S., and Satoh, S. (2017, January 6–9). Embedding watermarks into deep neural networks. Proceedings of the ACM International Conference on Multimedia Retrieval, Bucharest, Romania.10.1145/3078971.3078974 ↗
13. Namba, R., and Sakuma, J. (2019, January 9–12). Robust watermarking of neural network with exponential weighting. Proceedings of the ACM Asia Conference on Computer and Communications Security, Auckland, New Zealand.10.1145/3321705.3329808 ↗
14. Wang, J., Wu, H., Zhang, X., and Yao, Y. (, January 26–30). Watermarking in deep neural networks via error back-propagation. Proceedings of the IS&T Electronic Imaging, Media Watermarking, Security and Forensics, Burlingame, CA, USA.
15. Rouhani, B.D., Chen, H., and Koushanfar, F. (2019, January 13–17). Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks. Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems, Providence, RI, USA.
16. Zhao, X., Yao, Y., Wu, H., and Zhang, X. (2021, January 7–10). Structural watermarking to deep neural networks via network channel pruning. Proceedings of the IEEE Workshop on Information Forensics and Security, Montpellier, France.10.1109/WIFS53200.2021.9648376 ↗
17. Adi, Y., Baum, C., Cisse, M., Pinkas, B., and Keshet, J. (2018, January 15–17). Turning your weakness into a strength: Watermarking deep neural networks by backdooring. Proceedings of the 27th USENIX Security Symposium, Baltimore, MD, USA.
18. Badnets: Evaluating backdooring attacks on deep neural networks10.1109/ACCESS.2019.2909068 ↗
19. Jia, J., Liu, Y., and Gong, N.Z. (2021). Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning. arXiv.10.1109/SP46214.2022.9833644 ↗
20. Adversarial frontier stitching for remote neural network watermarking10.1007/s00521-019-04434-z ↗
21. Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., and Frossard, P. (2017, January 21–26). Universal adversarial perturbations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.17 ↗
22. Liu, K., Dolan-Gavitt, B., and Garg, S. (2018, January 10–12). Fine-pruning: Defending against backdooring attacks on deep neural networks. Proceedings of the International Symposium on Research in Attacks, Intrusions, and Defenses, Crete, Greece.10.1007/978-3-030-00470-5_13 ↗
23. Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H.P. (2016). Pruning filters for efficient convnets. arXiv.
24. Jia, H., Choquette-Choo, C.A., Chandrasekaran, V., and Papernot, N. (2021, January 11–13). Entangled watermarks as a defense against model extraction. Proceedings of the 30th USENIX Security Symposium, Virtual.
25. Zhang, J., Gu, Z., Jang, J., Wu, H., Stoecklin, M.P., Huang, H., and Molloy, I. (2018, January 10–14). Protecting intellectual property of deep neural networks with watermarking. Proceedings of the ACM Asia Conference on Computer and Communications Security, Melbourne, VI, Australia.10.1145/3196494.3196550 ↗
26. Wu, Y., Qiu, H., Zhang, T., Li, J., and Qiu, M. (2022). Watermarking pre-trained encoders in contrastive learning. arXiv.10.1109/ICDIS55630.2022.00042 ↗
27. Cong, T., He, X., and Zhang, Y. (2022). SSLGuard: A watermarking scheme for self-supervised learning pre-trained encoders. arXiv.10.1145/3548606.3559355 ↗
28. Chen, X., Liu, C., Li, B., Lu, K., and Song, D. (2017). Targeted backdoor attacks on deep learning systems using data poisoning. arXiv.
29. Zhang, H., Yu, Y., Jiao, J., Xing, E., Ghaoui, L.E., and Jordan, M. (2019, January 9–15). Theoretically principled trade-off between robustness and accuracy. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.
30. Krizhevsky, A., and Hinton, G. (2009). Learning Multiple Layers of Features from Tiny Images, University of Toronto. Personal communication.
31. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗
32. Coates, A., Ng, A., and Lee, H. (2011, January 11–13). An analysis of single-layer networks in unsupervised feature learning. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA.
33. Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (August, January 31). The German traffic sign recognition benchmark: A multi-class classification competition. Proceedings of the International Joint Conference Neural Networks, San Jose, CA, USA.
34. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.