The study shows that deep neural networks can be improved by considering the entire margin distribution, not just the minimum margin. By focusing on the ratio of margin standard deviation to expected margin, the complexity of the networks can be better controlled. The researchers used a convex margin distribution loss function to optimize this ratio and found that it correlates with the generalization gap. Experiments and visualizations confirmed the effectiveness of this approach in enhancing the generalization performance of deep neural networks.