Range Penalization: Theoretical Insights with Applications in Federated Learning
Document Type
Lecture
Publication Date
11-14-2024
Abstract
This talk introduces range regularization to enhance statistical accuracy and transmission efficiency, essential for reducing communication and computational demands in federated learning without compromising performance. Our approach identifies features with shared weights across different clients and adaptively clusters the weights of personalized features at extreme values, a process we refer to as polar clustering. Theoretical analysis of the associated estimators poses significant challenges due to the seminorm nature and non-decomposability of the regularizer. We develop new proof techniques for the nonasymptotic analysis of statistical accuracy and faithful pattern recovery. Moreover, a fast optimization algorithm that leverages varying degrees of local strong convexity is proposed to reduce iteration complexity. Experiments support the efficacy and efficiency of our approach.
Relational Format
presentation
Recommended Citation
She, Yiyuan, "Range Penalization: Theoretical Insights with Applications in Federated Learning" (2024). Probability & Statistics Seminar. 5.
https://egrove.olemiss.edu/math_statistics/5