Application of Compressed Sensing in Federated Learning

Authors

  • Lei Fang
  • Yucheng Ge

DOI:

https://doi.org/10.61173/5ybc1664

Keywords:

Compressive Sensing, Federated Learning, Data privacy

Abstract

Compressed sensing, as an important signal processing technology, uses the sparsity or structure of the signal to reconstruct the original signal with samples much lower than the traditional sampling rate. Federated learning (FL), as a distributed machine learning method that allows model training without sharing sensitive data, provides an effective way to share knowledge across devices and organizations. Combining compressed sensing with federated learning has potential synergistic advantages, which can not only achieve efficient information extraction and transmission, but also protect personal privacy data. This paper aims to explore how compressed sensing technology can be applied to federated learning to accelerate the model training process, improve model accuracy, and ensure data privacy. By deeply studying this field, this paper reveals the potential applications, challenges and future development directions of compressed sensing in the federated learning environment, and promotes further innovation and application in the field of federated learning. Through comprehensive experiments and analysis, this paper provides insights into the integration of compressed sensing within federated learning frameworks, highlighting its role in optimizing communication efficiency and resource utilization.

References

[1] Reddi S, Charles Z, Zaheer M, et al. Adaptive federated optimization. arXiv preprint arXiv:2003.00295, 2020.

[2] Karimireddy S P, Kale S, Mohri M, et al. Scaffold: Stochastic controlled averaging for federated learning//International conference on machine learning. PMLR, 2020: 5132-5143.

[3] Condat L, Malinovsky G, Richtárik P. Distributed proximal splitting algorithms with rates and acceleration. Frontiers in Signal Processing, 2022, 1: 776825.

[4] Candès E J, Wakin M B. An introduction to compressive sampling. IEEE signal processing magazine, 2008, 25(2): 21-30.

[5] Baraniuk R G, Cevher V, Duarte M F, et al. Model-based compressive sensing. IEEE Transactions on information theory, 2010, 56(4): 1982-2001.

[6] Candès E J, Romberg J, Tao T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on information theory, 2006, 52(2): 489-509..

[7] Donoho D L. Compressed sensing. IEEE Transactions on information theory, 2006, 52(4): 1289-1306.

[8] Miao Y, Zheng W, Li X, et al. Secure model-contrastive federated learning with improved compressive sensing. IEEE Transactions on Information Forensics and Security, 2023, 18: 3430-3444.

[9] Islam M M, Alawad M. Node Selection in Federated Learning Using Sparsity Regularization and Compressive Sensing//2023 International Conference on Machine Learning and Applications (ICMLA). IEEE, 2023: 1319-1323.

[10] Candes E J, Tao T. Near-optimal signal recovery from random projections: Universal encoding strategies. IEEE transactions on information theory, 2006, 52(12): 5406-5425.

[11] Kim S J, Koh K, Lustig M, et al. An efficient method for compressed sensing//2007 IEEE International Conference on Image Processing. IEEE, 2007, 3: III-117-III-120.

[12] Khaled A, Sebbouh O, Loizou N, et al. Unified analysis of stochastic gradient methods for composite convex and smooth optimization. Journal of Optimization Theory and Applications, 2023, 199(2): 499-540.

[13] Jin C, Chen X, Gu Y, et al. FedDyn: A dynamic and efficient federated distillation approach on Recommender System//2022 IEEE 28th international conference on parallel and distributed systems (ICPADS). IEEE, 2023: 786-793.

Downloads

Published

2024-12-31