- This event has passed.
MS. Defense: Toan Ngoc
April 18, 2023 @ 12:30 pm - 1:30 pm EDT
Title: Layerwise Batch Clipping for Differential Private Stochastic Gradient Descent
Master’s Candidate: Toan Nguyen Ngoc
Major Advisor: Dr. Marten Van Dijk
Associate Advisors: Dr. Caiwen Ding, Dr. Kaleel Madmood
Date/Time: Tuesday, April 18th, 2023, 12:30 pm
Location: HBL 1102
Meeting link:
https://uconn-cmr.webex.com/uconn-cmr/j.php?MTID=m1aab26e36150c96ece6383fbd51daad7
Meeting number:
2622 679 8135
Password:
ttVADCQi822
Host key:
543001
Join by video system
Dial 26226798135@uconn-cmr.webex.com
You can also dial 173.243.2.68 and enter your meeting number.
Join by phone
+1-415-655-0002 US Toll
Access code: 2622 679 8135
Abstract
Private training for deep learning models has become more and more popular because these models training dataset can be exposed by many kinds of attacks such as Membership Inference Attack, Deep leakage from gradients attack,… A powerful private training is DPSGD where the gradient of each round is clipped and the Gaussian noise is added to each entry of every gradient. However, this method has many drawbacks: it forces us to use a subsampling method with minibatch SGD optimizer, heavy computation on per sample gradients for clipped gradient computation and narrow choice of parameter for good testing accuracy.
On the other hand, a generalized DPSGD framework is proposed to enable running any optimization algorithm with DP guarantee. However, in this framework, the clipping value $C$ is fixed and the noise is added to any entry of each layer in the Deep Learning models. Therefore, we propose a new method to customize the clipping value and the Gaussian noise for each layer while maintaining the DP guarantee $G_{\sqrt{gE\sqrt{L}}/\sigma}$ with group size $g$, the number of epochs $E$ and the number of layer $L$. Moreover, our algorithm also shows testing accuracy improvement compared to Opacus’s algorithm
