2026-09-20T02:41:57Zhttps://api.repository.cam.ac.uk/server/oai/request

oai:www.repository.cam.ac.uk:1810/3348992023-12-22T12:56:38Zcom_1810_195217com_1810_256065col_1810_219484

Design of Deep Neural Networks Formulated as Optimisation Problems 10.17863/CAM.82337 Zhang, Sushen Lapkin, Alexei 0000000176210889 Vassiliadis, Vassilios The design of deep neural networks (DNNs) involves the explicit definition of network architecture as well as the training of the network weights. Each process can be formulated into an optimisation algorithm and can be investigated with regard to optimisation performance. The training of the network weights is defined as a minimisation of the objective function with regard to network parameters. The architecture search is an optimisation of the objective function with regard to the presence/absence of layers or neurons. I draw similarity between the two scenarios, and propose frameworks that define either the training or the architectural optimisation of DNNs, or a combination of both. The contribution of the thesis is six-fold, in which I propose: 1) a quasi-Newton training algorithm based on Truncated Newton and Gradient Flow methods, 2) a lifting scheme to allow network sparsification, 3) a lifting framework to automatically evolve neural architectures, 4) a multi-scale hierarchical search framework involving sensitivity analysis suitable for the training of neural networks, 5) a heuristic search algorithm for architectural optimisation of a dynamic model, and 6) a dynamic cascade learning model solved in the context of de novo drug design. In each contribution, I define the optimisation problem and solve the optimisation problem under different frameworks. The ultimate aim of this research is to facilitate the democratisation of AI, enabling people with less domain expertise to participate in the design of a deep neural network under a guided framework. University of Cambridge 2021-10-01 Thesis Doctoral Doctor of Philosophy (PhD) eng Cambridge Overseas Trust and China Scholarship Council https://www.repository.cam.ac.uk/handle/1810/334899 https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/a6a39126-2730-4f8b-945b-1b72ac43c0f0/download 9dd173424c1790227ea7cd7172786477 https://www.rioxx.net/licenses/all-rights-reserved/ Neural Network Optimisation Architecture Search