The Double Descent Behavior In Two Layer Neural Network For Binary Classification
Document Type
Lecture
Publication Date
4-20-2023
Abstract
Recent studies observed a surprising concept about test error called the double descent phenomenon where the increasing model complexity decreases the test error first and then the error increases and decreases again. To observe this, we worked on a two-layer neural network model with a ReLU activation function designed for binary classification under supervised learning. Our aim was to observe and find the mathematical concept behind the double descent behavior of the test error in the model for varying over-parameterization and under-parameterization ratios. We have been able to derive a closed-form solution for the test error of the model and a theorem to find the parameters with optimal empirical loss when model complexity increases. We proved the existence of the double descent phenomenon in our model for square loss function using the theorems derived.
Relational Format
presentation
Recommended Citation
Abeykoon, Chathurika, "The Double Descent Behavior In Two Layer Neural Network For Binary Classification" (2023). Probability & Statistics Seminar. 17.
https://egrove.olemiss.edu/math_statistics/17
