Robustness Analysis of Neural Networks via Semidefinite Programming

Deep neural networks have dramatically impacted machine learning problems in numerous fields. Despite these major advances, neural networks are not robust and hence not suitable for safety-critical applications. In this lecture, we will present a novel framework for analyzing the robustness of deep neural networks against norm-bounded nonlinearities. In particular, we develop a semidefinite programming (SDP) framework for safety verification and robustness analysis of neural networks with general activation functions. Our main idea is to abstract various properties of activation functions (e.g., monotonicity, bounded slope, bounded values, and repetition across layers) with the formalism of quadratic constraints. We then analyze the safety properties of the abstracted network via the S-procedure and semidefinite programming. Compared to other approaches proposed in the literature, our method is less conservative, especially for deep networks, with an order of magnitude reduction in computational complexity. Furthermore, our approach is applicable to any activation functions. Such bounds are very important in analyzing the safety of control systems regulated by neural networks.