Study on linear optimization of activation function of homomorphic encryption neural network

Study on linear optimization of activation function of homomorphic encryption neural network

연구 분야: Cryptography

학회: CSSE '21: Proceedings of the 4th International Conference on Computer Science and Software Engineering

Since the 20th century, the research on cipher neural network is more and more in-depth. Encryption scheme based on neural network is from layered homomorphic encryption scheme to fully homomorphic encryption scheme, the accuracy of neural network and the resource consumption in calculation are always the core of the research. This paper mainly studies the optimization of the excitation layer in the neural network. Since the convolution layer and pooling layer in the hidden layer are linear operations, they can support the secret state operation, and the excitation layer introduces nonlinear factors into the neural network in the form of nonlinear functions, so the secret state calculation cannot be directly applied to the layer. In order to solve this problem, this paper on the basis of CryptoNets model, only in the case of low degree to consider about the commonly used three kinds of activation function : Sigmoid, Tanh and ReLU function for the linear approximation, and proposed a new approximate method : An approximate function is constructed by using the structural characteristics of the derivative of the function, compared with the commonly used approximate method, a new method of approximate function replacement after activation function, the neural network prediction accuracy and operation time in both performance is more outstanding.