^{1}

^{2}

^{3}

The authors have declared that no competing interests exist.

During development, biological neural networks produce more synapses and neurons than needed. Many of these synapses and neurons are later removed in a process known as neural pruning. Why networks should initially be over-populated, and the processes that determine which synapses and neurons are ultimately pruned, remains unclear. We study the mechanisms and significance of neural pruning in model neural networks. In a deep Boltzmann machine model of sensory encoding, we find that (1) synaptic pruning is necessary to learn efficient network architectures that retain computationally-relevant connections, (2) pruning by synaptic weight alone does not optimize network size and (3) pruning based on a locally-available measure of importance based on Fisher information allows the network to identify structurally important vs. unimportant connections and neurons. This locally-available measure of importance has a biological interpretation in terms of the correlations between presynaptic and postsynaptic neurons, and implies an efficient activity-driven pruning rule. Overall, we show how local activity-dependent synaptic pruning can solve the global problem of optimizing a network architecture. We relate these findings to biology as follows: (I) Synaptic over-production is necessary for activity-dependent connectivity optimization. (II) In networks that have more neurons than needed, cells compete for activity, and only the most important and selective neurons are retained. (III) Cells may also be pruned due to a loss of synapses on their axons. This occurs when the information they convey is not relevant to the target population.

Biological neural networks need to be efficient and compact, as synapses and neurons require space to store and energy to operate and maintain. This favors an optimized network topology that minimizes redundant neurons and connections. Large numbers of extra neurons and synapses are produced during development, and later removed as the brain matures. A key question to understand this process is how neurons determine which synapses are important. We used statistical models of neural networks to simulate developmental pruning. We show that neurons in such networks can use locally available information to measure the importance of their synapses in a biologically plausible way. We demonstrate that this pruning rule, which is motivated by information theoretic considerations, retains network topologies that can efficiently encode sensory inputs. In contrast, pruning at random, or based on synaptic weights alone, was less able to identify redundant neurons.

The number of neurons and synapses initially formed during brain development far exceeds those in the mature brain [

What characterizes the cells and synapses that survive as opposed to the ones that die? A key factor for survival is neuronal activity. Initially, spontaneous activity is thought to drive survival. The refinement of cortical circuits then relies increasingly on sensory-driven and thus experience-dependent neuronal activity [

Here, we explore several local and biologically plausible rules for iteratively pruning unimportant synapses and units from artificial neural network models. Many models of neural pruning simply remove small synaptic weights [

Information-theoretic approaches to network reduction provide a principled starting point. For example, the Optimal Brain Surgeon algorithm [

Estimates of parameter importance based on Fisher Information (FI) have recently been used to overcome catastrophic forgetting in artificial neural networks [

Similar to neuronal sensory systems, RBMs extract and encode the latent statistical causes of their inputs. They consist of two layers of stochastic binary units, resembling the spiking of neurons. The visible units correspond to sensory input, while the hidden units encode a latent representation of this data. By adjusting their hidden representations to maximize the likelihood of the data [

We organize the results as follows: we first introduce our estimates of synaptic importance based on locally-available activity statistics. We then discuss the overall network curvature and demonstrate that important synapses center on overall highly informative neurons. Based on these observations, we introduce local pruning rules to iteratively remove synapses and neurons from RBMs and DBMs that were trained on image patches from two different data sets. We evaluate the fit of the pruned models across different pruning criteria by assessing their generative and encoding performance. Finally, we discuss the biological plausibility of our activity-dependent pruning rules by comparing them to alternative rules used in our own and related work and provide implications of our findings.

The goal of this work is to derive and use a local pruning rule to reduce network size and identify a relevant network topology in restricted and deep Boltzmann machines. By ‘relevant network topology’ we mean a topology optimized for computational needs that includes only neurons and synapses that are relevant for the task at hand. In our experiments this task is the encoding of visual stimuli in hidden representations of RBMs and DBMs. RBMs are energy-based models whose energy function is given by:
_{i} stands for visible unit _{j} stands for hidden unit _{ij} for the bidirectional weight connecting them. Lower energy corresponds to higher probability of the respective model state. We work with Bernoulli RBMs, and all neurons are binary: _{i} and _{j} are either firing (1) or silent (0). The biases ^{v}, ^{h}}.

The Hessian of the objective function of a model gives information about the importance of parameters, and can be used for pruning [_{v,h} is the probability of jointly observing visible pattern _{i} and _{j} is slightly perturbed.

A crucial advantage of the FIM is that its values are closely connected to the distribution of activity and can be estimated by sampling from the distribution _{v,h}. In the case of the RBM, the Fisher information for two weights _{ij} and _{kl} takes the form:
_{ij} = _{kl}). It captures the effect on the error when we modify a single weight. For the FI of the weights,

In a first line of experiments, we inspected the curvature of the energy landscape of overcomplete models that had more latent units than needed to encode the visible stimuli. We started by fitting overcomplete RBMs to circular, binarized patches of images of natural scenes (see

(A) Preparation of dataset and general model architecture: circles of different radii were sliced out of CIFAR-10 images and binarized. The pixels corresponded to the visible units of a standard RBM with one hidden layer. (B) Fisher information for an exemplary over-parameterized RBM initialized with _{v} = 13 visible units and _{h} = 70 hidden units. The FIM is sparse, indicating many irrelevant parameters. (C) Importance of each parameter, as summarized by the value for each parameter in the vector of steepest curvature in the FIM (i.e. the leading eigenvector). The rectangular plot shows the normalized importance for all weights _{ij} connecting visible units (_{i}; vertical axis) and hidden units (_{j}; horizontal axis). The importance for biases for the hidden (^{h}) and visible (^{v}) units is shown above (horizontal) and to the right (vertical), respectively. (D) Normalized parameter importance directly estimated from the diagonal entries of the FIM. (FIM = Fisher Information Matrix).

The sparseness of Fisher information is also evident from a visualization of the first eigenvector and the diagonal of the FIM. This is plotted separately for the weights, hidden biases and visible biases in _{0}) is larger than the second largest eigenvalue by an order of magnitude (first eigenvalue 23.97, second eigenvalue 1.60). This low-rank structure implies that the FIM can be approximated by a rank-1 outer product of its leading eigenvector,

Strikingly, important synaptic weights were not uniformly distributed throughout the network. Instead, clusters of important weights (and biases) were associated with specific units (see

Overall, these empirical investigations reveal that there are only few important units, and that important weights align with them. This confirms that parameter importance as revealed by the FIM contains meaningful information for pruning, and predicts that synaptic pruning will eventually lead to the removal of whole neurons that lack relevant afferent or efferent connections.

We now present our results of applying the local estimate of Fisher information as a criterion to prune RBMs. We compare this estimate to removing poorly specified, unimportant weights according to different pruning criteria and random pruning. We also test a pruning criterion based on the full FIM to confirm that the diagonal approximation closely tracks importance. For this, we assess importance as the magnitude assigned to each weight in the leading eigenvector of the FIM.

_{i} _{j}〉. We refer to this as the “variance” estimate of the FIM diagonal. However, neurons need not track this coincident firing explicitly. Activity-driven correlations influence synaptic weights and vice versa. As a result, the statistical quantities relevant to pruning are tied to the synaptic weights themselves. This opens up a broader range of possible biological implementations of the rules we explore here. It also highlights that apparent correlations between synaptic weights and pruning

For our synaptic pruning experiments, we ranked weights according to their importance assessed by one of the aforementioned local estimates, or by the first eigenvector of the FIM. The unpruned full model initially had _{v} = 13 visible units, _{h} = 70 hidden units, and _{w} = _{v} × _{h} = 910 weights. We then iteratively deleted half of the weights with lowest estimated importance, while monitoring model fit.

We assessed the fit of a given model based on its generative performance, i.e. how well the generated patterns in the visible layer match the patterns available in the training data. Following the approach by Reichert et al. [

Unconnected hidden units were removed from the model. Absolute weight magnitude, random weight pruning, and removal of the

Half of the weights were removed in each of 3 pruning iterations; all plots reflect 10 independent simulations. (A) Mean-Squared-Error (MSE; vertical axis) between the log-probability of a pattern in the training data (horizontal axis) and the log-probability of the corresponding pattern generated by the trained model. Black points reflect the average MSE over 10 realizations; Blue lines indicate the 25^{th}-75^{th} percentile range. (B) Log-probability matching after three iterations of synaptic pruning (according to different criteria) and retraining. Accuracy remains good for pruning rules based on weight magnitude and Fisher information. (C) Kullback-Leibler divergence between the distribution over training instances vs. the distribution over generated samples, after three iterations of pruning before and after-retraining. Retraining generally restores generative performance.

Excessive pruning degraded generative performance. Regardless of the pruning criterion, the distributions of generated patterns vs. the training data generally diverged immediately after pruning (

For “random weight” and “Anti-FI” pruning, the distributions of generated patterns diverged from the training data and could only be partially rescued by retraining (see

We conjectured that the FI-based rules allowed local synaptic dynamics to identify and remove unimportant neurons. To confirm this, we ran a simulation which removed a random subset of neurons. This showed a large divergence between the distribution of training patterns and generated patterns immediately after pruning, but performance could be restored by re-training (

While the generative performance after pruning and retraining was comparable for models pruned according to weight magnitude and Fisher Information, the resulting model architectures differed markedly in their number of hidden units. _{h} after each of the three pruning iterations, averaged across ten simulations. Although the number of remaining weights _{w} was matched across pruning criteria, different numbers of units were disconnected through repeated weight pruning. Fisher information typically concentrated on few hidden units (see

Each table entry contains the mean “_{h} after each pruning iteration, and their standard deviation (_{v} = 13, _{h} = 70, and _{w} = _{v} × _{h} = 910 weights. During each of the three pruning iterations half of the weights were removed according to different criteria. _{w} describes the remaining weights at the given phase of pruning.

Pruning criterion | _{w} = 455 |
_{w} = 227 |
_{w} = 113 |
---|---|---|---|

Variance FI | 47.6 (5.75) | 10 (1.26) | |

Heuristic FI | 58.0 (4.12) | 42.3 (4.8) | 14.3 (7.06) |

First eigenvector | 69.8 (0.4) | 42.5 (4.8) | 9.5 (0.67) |

Weight magnitude | 69.9 (0.3) | 58.1 (2.7) | 32.2 (3.06) |

Anti-FI | 55 (3.35) | 44 (2.69) | 33.6 (2.24) |

Random Weight | 70 (0.0) | 69 (0.75) | 58.0 (2.79) |

Random Unit | 35 (0.0) | 18 (0.0) | 9 (0.0) |

In sum, for all pruning strategies the network could recover to some extent from the loss of weights and units through retraining [

Above, we demonstrated an iterative approach to pruning weights from RBMs while monitoring their fit from the frequency of its generated patterns.

In small RBMs that encoded simple patterns, retraining compensated for the loss of synapses regardless of the pruning criterion. However, we found that FI pruning, as opposed to pruning small weights, reduced the cost of the network in terms of the number of remaining neurons. This supports the view that the function of pruning is to find an appropriate architecture, rather than specific parameter configurations.

To investigate the effect of the pruning method on learned representations, we increased the complexity of the model architecture by adding another hidden layer, resulting in a deep Boltzmann machine. We trained this multi-layer model on a labeled dataset to quantify the fit of the model and the quality of the latent representation during iterative pruning. Specifically, we used the MNIST handwritten digits dataset [

Each image of the dataset was binarized and scaled to 20×20 pixels. To simplify computation and add biological realism, we restricted the receptive fields of each unit in the first hidden layer ^{1} to a small region of the visible inputs. To encode digits, the network was therefore forced to combine these lower-level features in the second hidden layer ^{2} (see

(A) Data representation and deep Boltzmann machine architecture. MNIST digits were cropped and binarized. Each unit from hidden layer ^{1} had an individual receptive field covering neighboring input pixels; ^{2} was fully connected. Only a few connections are shown due to clarity. The classifier was trained on latent encodings from ^{2}. (B) Classification error of a logistic regression classifier trained on ^{2} encodings as a function of remaining weights _{w}. The dotted line stands for the baseline error of a classifier trained on the raw digits. All data points are averages from 10 independent simulations, and error bars denote one standard deviation. (C) Number of latent units in ^{1} and ^{2} as a function of remaining weights over the course of pruning. (D) Final visible layer connectivity after pruning according to different criteria. The probability of a unit being disconnected is shown in gray-scale, with black denoting units that were disconnected in all simulations.

On each iteration, we removed the least important 10% of weights as assessed by weight-specific FI or absolute weight magnitude in ^{1}, and the least important 25% of weights in ^{2}. For the FI-based rules, more than 25% of FI estimates of weights in ^{2} were zero in the first pruning iteration. In this scenario, we instead pruned all synapses with zero importance, which led to a comparably faster rate of model reduction. Units of the intermediate layer ^{1} that became disconnected from either side (^{2}), were completely removed. Thus, the total number of deleted weights may be larger than specified by the importance measure as a secondary effect. The removal of such “dead-end” units is also the reason for the high variability of the number of remaining weights for the Anti FI pruned models (see

Similarly to the results for the single-layer RBM, pruning a modest number of parameters had little effect on the latent encoding, as measured by the classification error (

In contrast, random removal of weights or complete units led to rapid degradation, and pruning important weights (Anti-FI pruning) was particularly harmful: classification performance deteriorated after the first pruning event, and eventually remained below chance even with retraining. This suggests that Anti-FI pruning causes a loss of topologically relevant neurons. The visualization of the pixels corresponding to the remaining units in the final visible layers

The encoding also remained useful for classification. When weights were pruned according to our heuristic estimate of Fisher information, the performance of the classifier only fell below baseline after the number of weights was reduced by more than one order of magnitude. Models pruned according to the variance FI estimate dipped in their performance after the third iteration of pruning. Arguably, the optimal model size was already reached after the first pruning iteration. We hypothesize that an increase of the average activation of hidden units may indicate the arrival at the optimal model size and thus be a signal to stop pruning (see Fig A in ^{1} is less sparse from the beginning, as the receptive fields pre-structure the network and the number of hidden units in this layer equals the number of visible units (average activity in ^{1} in the first pruning iteration ranged between 0.37–0.43 as opposed to 0.0006–0.018 in ^{2}). Initial activity in ^{2} is sparse and as a consequence FI pruning quickly identifies all units with low (covariant) activity: their weights are pruned and the unimportant units are removed from the model. This also allows a faster reduction of model size: to reach a comparable total number of weights, only six pruning iterations were necessary for our FI estimates, but ten iterations for the random weight and weight magnitude pruning.

FI pruning topologically optimizes the model: more weights (and units) are removed in ^{2} (see ^{1} the pruning rate agrees with the pre-specified weight reduction per iteration (10%). Such a topological optimization within multiple layers cannot be achieved by random unit removal. In this control case, where we pruned a comparable number of weights by randomly deleting hidden units, too many units were removed from ^{1} for the encoding to remain useful for classification.

Random weight pruning and pruning by absolute weight magnitude failed to reduce the model size in terms of neuron number: Neither pruning criterion led to any of the hidden units becoming disconnected. In the visible layer, a maximum of one pixel became disconnected in each case (see

Taken together, these results show that the pruning criterion matters in deeper networks. The network performance can recover through retraining if activity-dependent pruning is used (FI and weight-magnitude pruning), but is permanently damaged by random or Anti-FI pruning. Furthermore, FI-pruning produces the most efficient network topology and selectively retains the most important neurons, unlike simpler strategies such as pruning by weight magnitude.

Previous studies explored deep Boltzmann machines as a generative model of hallucinations, for example those seen in Charles Bonnet Syndrome [

Inspired by these results, we asked whether pruned networks retain a meaningful description of the latent structure in their inputs. We assessed this by measuring the generative performance of the networks over the course of pruning. To quantify generative performance, we asked a classifier trained on the raw digits to categorize samples and predict probabilities of them belonging to each digit class. These were summarized as digit-wise quality scores and diversity scores for the generated patterns (Materials and Methods:

We used the classification confidence as a proxy for digit quality (

(A) Maximum class probability assigned to generated samples from pruned networks averaged over 10 runs. This summarizes the confidence of a classifier that the generated digits belonged to a specific class. Error bars denote standard deviations. The black and gray dashed line show the confidence of the classifier for the MNIST test digits and randomly generated patterns, respectively. (B) Entropy over the distribution of generated digits. An entropy value of ≈ 2.30 nats corresponds to even coverage of all digits, which is achieved by the test digits (dotted line). All data points are averages from 10 independent simulations, and error bars denote one standard deviation. (C) Examples of generated patterns after pruning completed.

Finally, we compared the diversity of the generated digits (

Pruning according to our FI estimates decreased digit diversity more so than the other criteria. Note, however, that this assessment of generative performance is not meaningful when the network generated degraded digits that were hard to classify, as was the case for Anti-FI or random pruning (compare examples in

These results suggest that pruning impairs the generative capabilities of the networks, either degrading the generated representations (

Overall, FI-based pruning preserved the generative quality to a similar degree. In contrast, the Anti-FI pruning algorithm disconnected important units, leading to both sensory “blindness” and an inability of the model to meaningfully report the visual correlates of latent activity. Weight magnitude alone is an insufficient indication of parameter importance, and while networks pruned in this way still can be used to classify the digits well, they lack a structural optimization. In contrast, FI pruning resulted in a more efficient encoding in terms of the number of remaining units.

In this work, we used stochastic latent-variable models of sensory encoding to derive new theoretical results on neural pruning, and explored these results through numerical simulations. By examining the energy-based interpretation of such models, we showed that the importance of synapses and neurons can be computed from the statistics of local activity. To our knowledge, our work is the first empirical study of network reduction guided by an activity-dependent estimate of Fisher information in this context. Our pruning rule operates at single synapses, and provides a principled route to eliminating redundant units. Searching for biological analogues of these quantities and processes may provide insight into naturally occurring pruning in the developing brain.

In biology, over-production of neurons and synapses likely confers advantages. These extra neurons could accelerate initial learning [

The importance of a precise pruning rule in identifying efficient network topologies is highlighted by the comparison of the different pruning criteria. Weight-based pruning yields networks with fewer synapses that still perform well, but unlike during FI-based pruning, few neurons are disconnected. Anti-FI pruning, in contrast, removes the most important synapses and vital units first, exactly those components that are left intact by FI-based learning rules. In our simulations, the irrelevant units are in the visual periphery as they carry no information. However, parameter importance may be less obvious in deeper layers of the network, and more generally for other stimuli and tasks.

State-of-the-art models in machine learning are often over-parameterized, and optimizing such models via pruning is a subject of active research. The vast reduction of the number of neurons in the hidden layers in our FI-pruned models show how pruning could play a similar role in architecture optimization in the brain.

Yet some studies call into question the usefulness of pruning. It can be difficult to continue training networks with pruned weights without incurring performance losses [

Pruning studies in artificial neural networks often estimate parameter importance with respect to the curvature of a cost function on network performance or discriminative power in supervised learning problems [

The physiological correlates of our FI-based pruning rule are unknown; to address this question, data from long-term imaging experiments of synapses could be fit to different potential pruning rules [

We derived the FI-based pruning rules for Boltzmann machines, a class of energy-based stochastic neural network models that, owing to their symmetric connectivity, have a well defined equilibrium distribution and allow computing the FIM exactly. While the symmetric connectivity and learning rule (the wake-sleep algorithm, which requires symmetric connections) in these networks are not biologically plausible, they implement minimization of variational free energy as an unsupervised learning objective. This is a plausible first principle for learning in neural circuits, in particular in sensory systems where limited bandwidth requires efficient coding [

In biology, pruning and learning occur simultaneously. To emulate this, we retrained networks for ten epochs after each batch of pruning. This batched form of sensory-driven plasticity allowed the networks to recover from a performance drop observed immediately after pruning. In biology, numerous types of homeostatic plasticity could compensate for the loss of cells and synapses without error-driven retraining [

Model networks can provide insight into neurological disorders. For example, Reichert et al. [

Statistical theories view hallucinations as errors in internal predictive models [

This suggests a speculative connection to schizophrenia, for which hallucinations and altered cognition are core symptoms. Schizophrenia also involves pathological neural pruning [

Apart from neurological disorders, understanding pruning is important for understanding learning and cognitive flexibility. In all of our experiments, the sensory encoding task was fixed. Overzealous optimization to a fixed distribution could impair flexibility, a phenomenon that might relate to developmental critical periods and neurological disorders [

Overall, we have shown that local activity-dependent synaptic pruning can solve the global problem of optimizing a network architecture. In contrast to pruning rules based on synaptic weights, our information-based procedure readily identified redundant neurons and led to more efficient and compact networks. The pruning strategy we outline uses quantities locally available to each synapse, and is biologically plausible. The artificial neural networks explored here are abstract. If analogous processes operate in biology, then a similar pruning procedure could optimize metabolic cost by eliminating redundant neurons. An important future direction is to analyze these pruning rules in conjunction with synapse formation, which has been hypothesized to provide an additional mechanism for functional optimization of network topologies [

We used two different datasets of visual image patches to train and evaluate our models. For Figs

For Figs

RBMs are generative stochastic encoder models consisting of _{i} has undirected weighted connections (synapses) to each hidden neuron _{j} and vice versa, but neurons within a layer are not connected to each other (see

The energy function given in

The training objective is to adjust the parameters

The positive-negative or wake-sleep algorithm [_{j}|_{i}|

A deep Boltzmann machine consists of a visible layer and multiple hidden layers. Analogous to

Our parameter set

All models were implemented in TensorFlow [^{th} Gibbs step. The number of samples per layer was the same as the number of training instances.

Single-layer RBMs were fit to CIFAR-10 patches using one-step contrastive divergence [_{i})/(1 − _{i}))], where _{i}) corresponds to the fraction of training instances where pixel

The comparably large number of 400 pixels of each cropped MNIST image required their segmentation in receptive fields (see ^{1} had the same number of units as the visible layer ^{1} was connected to a rectangle spanning a maximum number of 5 × 5 neighboring units from ^{1} from originally 400 × 400 = 160, 000 to 8, 836. The fully connected hidden layer ^{2} with 676 units then combined the receptive field encodings of parts of the image into a latent representation of the full image.

The deep Boltzmann machine was built from two individual RBMs that were pre-trained for 20 epochs with one-step contrastive divergence. After fitting the first RBM with receptive fields, its hidden units were sampled with

When stacking the two RBMs, the hidden layer of the first RBM and the visible layer of the second RBM were merged by averaging their biases. The resulting deep Boltzmann machine with two hidden layers was trained jointly for 20 epochs following a mean-field variational inference approach using persistent contrastive divergence with 100 persistent Markov chains [

We compared six different weight pruning criteria throughout our simulations, three of which targeted the removal of weights that carried low FI. For small RBMs, computing the full FIM and its eigenvectors was feasible, using code from

To simulate synaptic pruning, weight importance was estimated according to the selected pruning criterion. According to this criterion, a threshold was set at a pre-specified percentile. All weights whose estimated importance did not meet this threshold were removed from the network by fixing their values to zero. For RBMs trained on CIFAR-10 patches, the threshold corresponded to the 50^{th} percentile of weight importance, meaning that half of the weights were removed. For DBMs trained on MNIST, the threshold corresponded to the 10^{th} percentile of weight importance for ^{1} and to the 25^{th} percentile for ^{2}. If all incoming weights to a hidden unit were pruned, the unit was removed from the network. In DBMs, a hidden unit was also deleted if all its outgoing weights were pruned. After pruning, the model was re-initialized with unaltered values of the remaining parameters for the next pruning iteration. The RBMs fit to CIFAR-10 patches were retrained for 2 epochs after each of 3 pruning iterations. For DBMs, the retraining period was shortened from 20 to 10 epochs and mini-batches of size 10 were used to accelerate retraining after each pruning iteration. DBMs were pruned for a maximum of 10 iterations. In Figs

Each experiment started by fitting a sufficiently large model to the data. While the visible layer size was determined by the number of pixels in each training image, the (final) hidden layer was set to be larger. The resulting over-parameterized model was iteratively pruned, while its fit was monitored.

For small RBMs that were trained on CIFAR-10 patches, we evaluated a model’s generative performance by comparing the probabilities of generated visible layer samples to the probabilities of patterns occurring in the training data. Furthermore, we computed the Kullback-Leibler divergence between the data distribution and the distribution of generated samples _{KL}(data || model).

For DBMs that were trained on MNIST digits, we made use of the labeled data to evaluate both the encoding and generative performance.

First, we considered the encodings of the data in the final hidden layer of the network. While the visible layer was clamped to one training instance at a time, the hidden unit activations were sampled. We expect these latent representations to comprise a more useful training set for the classifier than the raw images. The resulting set of 60,000 final hidden layer encodings was used to train a multinomial logistic regression classifier, which had to distinguish between the 10 digit categories. We refer to the classification accuracy of this classifier built on top of the final hidden layer as the encoding quality of the model.

Second, we evaluated the patterns generated by the network. Since Boltzmann machines try to approximate the data distribution with their model distribution, these generated patterns should ideally resemble digits. Thus, we trained a multinomial logistic regression classifier on the 60,000 raw MNIST images. After training, this classifier received patterns generated by the network. For each of the ten digit classes, it returned a probability of the current sample belonging into it. The argmax over this probability distribution was used to assign the class. The average of the winning class probabilities was used as a confidence score. It served as a measure of digit quality. Furthermore, the entropy of the distribution of assigned classes served as a measure of digit diversity. A maximum entropy of approximately 2.30 nats corresponds to completely balanced digits.

Moreover, the generative performance was compared to that of a classifier trained on the raw digits. The quality of generated digits was compared to the quality of the 10,000 held-out MNIST test digits and to the quality of random patterns, using the same classifier.

Parameter importance is reflected in the curvature of the energy landscape of an RBM when slightly changing two parameters. Computing this for each parameter pair leads to the FIM (see _{ij} stands for the Fisher information of the considered couple (_{i}, _{j}). The entries of the FIM thus have the form [

The diagonal of the FIM corresponds to evaluating changes in the energy landscape of the model when perturbing just one parameter. The importance of a weight thus simplifies to the average coincident firing of pre- and postsynaptic neurons. The importance of a bias value is estimated by the variance of a neuron’s firing.

In this section, we derive a mean-field approximation estimate of 〈_{i}_{j}〉 in terms of a small deviation from the case where presynpatic and postsynaptic activity are statistically independent, 〈_{i}_{j}〉 ≈ 〈_{i}〉〈_{j}〉 that accounts for correlated activity introduced by the synapse between neurons

This can be computed if there is a local mechanism for tracking and storing the correlation of pre- and postsynaptic firing rates ⟨_{i}_{j}〉. This expectation is closely related to more readily available local statistic, like the mean rates and weight magnitude. Since _{i} and _{j} are binary in {0, 1}, the expectation ⟨_{i}_{j}〉 amounts to estimating the probability that _{i} and _{j} are simultaneously 1, i.e. _{i} = 1, _{j} = 1). One can express this in terms of a mean rate and the conditional activation of presynaptic neuron given a postsynaptic spike, using the chain rule of conditional probability:
_{j} using mean-field. The activation of a visible unit is given by _{j}, one computes:
_{j} = 1,
_{i}〉. Given this mean rate, we can estimate the activation as ^{−1}(〈_{i}〉). This estimate can replace the

This has an intuitive interpretation: for fixed 〈_{i}〉 and 〈_{j}〉, the coincidence in presynaptic and postsynaptic firing is a sigmoidal function of the weights. This implies the magnitude of synaptic weights, when combined with information about average firing rates, provides a useful proxy for computing pre-post correlations, and therefore estimating a synapse’s importance in the network.

(PDF)

(PDF)

Dear Dr. Hennig,

Thank you very much for submitting your manuscript "The Information Theory of Developmental Pruning: Optimizing Global Network Architecture Using Local Synaptic Rules" for consideration at PLOS Computational Biology.

As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. In light of the reviews (below this email), we would like to invite the resubmission of a significantly-revised version that takes into account the reviewers' comments. With regards to the question of relevance for biology, we encourage you to highlight the important algorithmic-level links between the type of systems you are studying here and real neural circuits.

We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent to reviewers for further evaluation.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.

Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Blake A Richards

Associate Editor

PLOS Computational Biology

Lyle Graham

Deputy Editor

PLOS Computational Biology

***********************

Reviewer's Responses to Questions

Reviewer #1: The paper investigates the pruning of connections with low Fisher Information in restricted and deep Boltzmann machines. This approach is compared with pruning based on a locally computable proxy of Fisher information, the maximum Fisher information, and the synaptic weight, as well as random pruning.

As the most interesting property, the authors demonstrate that the low FI approach typically removes all connections to a hidden neuron, such that the synapse pruning can also implement neuron pruning. At the same time, the pruned networks retain high encoding/classification and generative performance. The methods the authors use to demonstrate their finding are sound and appropriate. However, I am missing error estimates or a validation by multiple trials.

Moreover, although the topic of pruning is a very interesting and relevant topic both in biological and artificial neural networks, I think the findings and writing of the paper are more on the artificial intelligence than on the biological networks side (usage of RBM, training on MNIST/CIFAR, no comparison to experimental data), which should be changed to be suitable for publication in this journal.

Major points:

Errors:

It seems as if all the results stem from single network instances and there are no error estimates reported. As the pruning and RBM training heavily depend on the initial network, I think it would be good to compare results over multiple initialization.

Link to biology:

The bidirectional weights and their training within RBMs are not exactly biological. Thus, it is unclear whether the advantage FI provides over weight-dependent pruning will still hold in biologically more realistic settings. Moreover, if the biological realistic implementation of an information based learning rule is in the focus, it should be highlighted and discussed better in the results instead of being hidden in the methods/appendix.

Neuron removal:

Throughout the paper, the authors often stress the fact that FI-based approaches prune connections such that whole neurons can be removed. I feel like there is a proper control missing for this to demonstrate the superiority of the FI approach. I would propose to introduce a pruning strategy which removes whole neurons (either randomly or by minimum sum of incoming weights). In that way, there would be a control case with similar neuron number to compare with.

Minor points:

all Figures: Colors of Random and weight-dependent pruning were not so well discernible for me.

p.2 l.81: it should be b^h in the parameter-set instead of h^h. Also, I would move the sentence about the Bernoulli RBM after the description of the activities, which are the binary quantities

p.3 l.87 I would move the discussion of diagonality after the example FI (Eqs.2/3), because i think these examples would make it easier to understand that non-local information is needed for the full FI

p.3 l.95 I am not completely sure what is meant by redundant here. Is each parameter redundant for the network or are they redundant (covariant or so) w.r.t. each other? Pleas clarify.

Fig 1C/D: the labels h and v were a bit confusing. Shouldn't it be b^v and b^h?

Fig 2: It is hard to see a strong effect in the pattern distributions. Is there a way to make this clearer? Possibly show ratios?

Also the demonstrated changes in distributions will strongly depend on the size of the model and initial overparametrization

p.7 l 210 The explanation on generative performance is a bit distracting here. I would suggest moving it to the respective section.

p.7 l 223 The reason for calculating the percentile is unclear. Are these the pruned synapses? Please clarify! Also, making more clear that these are the 10% of the weights with the lowest FI, would make sense.

p.7 l.227 The reading flow here was a bit unsteady. Possibly move the statement of the before-pruning accuracy to line 210, where they are introduced.

p.7 l. 236 Isn't this a consequence of your FI-estimator being zero for many more units than the weight such that you remove more connections. I think the plot 3b is a fair way to compare the pruning strategies. If you want to compare, let the weight-pruning run longer.

Fig. 3C: Plotting over n_wh does not add much here. I would plot over the iteration number for comparability.

Fig. 4D: Why invert the x-axis?

p.9 l.311 *Anti-FI

p.9 l.300 Typo "versus"

l. 353 There is a lot of experimental and theoretical work on connectivity overshoot, e.g. by Arjen van Oojen and Jaap van Pelt (based on activity dependent rewiring). Also spine turnover (especially removal) has been demonstrated to be much higher during maturation. Would this be the direction this is aiming at? What would be the functional use of a "critical phase"?

Reviewer #2: A salient feature of neural development is pruning: the number of neurons and connections first grows, but at some point both decrease. It is not known why this happens, or which connections and neurons are pruned. The authors address the latter question, and propose that weights that have the smallest effects on activity, as measured by the Fisher information, are preferentially pruned; and if all connections from or to a neuron are pruned, the neuron is pruned as well.

I'm not an expert, but this is, I believe, a novel and interesting hypothesis, and will make a nice contribution to the field. My comments are almost exclusively about presentation -- there were a lot of places I simply got lost. That didn't really detract from the big picture, but it would be nice if things were clarified.

1. Sloppiness is mentioned in the abstract, and not again until methods. It would make sense to me to either drop it in the abstract or mention it in the main text.

2. I personally would drop initials, and spell out RBM, DBM, FIM and FI. Or at least spell out the first three. I'm not sure why anybody uses initials; in my view authors should be allowed to use at most one. There's actually a reason for that: people rarely read papers beginning to end, and it's very annoying to have to hunt through a paper to look for the meaning of initials.

3. Given the definition of the energy, I believe Eq. 2 is wrong: the derivative should be with respect to the partition function, not the energy. If so, that should be corrected.

4. I don't think P_v,h was ever defined.

5. l 95-6: "When F_ij tends towards zero, the two parameters phi_i and phi_j are redundant." I can't for the life of me even guess what that means. It should be explained. Or dropped; I don't think it was ever used.

6. l 99-100: "The resulting entries of the FIM depend on coincident firing of pre- and postsynaptic neurons and are arguably locally available." How can something that depends on two presynaptic and two postsynaptic neurons be locally available?

7. I could not figure out what's in fig. 1c and d. This should be explained much more clearly.

8. l 115-7: "The correspondence between parameter importance estimated from the first eigenvector and from the diagonal supports our use of Optimal Brain Damage for larger models, when computing the full FIM was no longer feasible." Couldn't make sense of this. It doesn't help that Optimal Brain Damage was (I believe) never explained.

9. l 117-22:. "Strikingly, the important weights typically aligned with few hidden units and their biases. This structure of the FIM suggests a separation into important hidden units and unimportant ones. It follows that FI motivated pruning likely leads to entire units becoming disconnected, which would allow their removal from the network. This would correspond to neuron apoptosis after excessive synaptic pruning." The second two sentences make sense. But the first two don't, and so it's not clear how the second follow from the first.

10. l 131-3: "For a pruning criterion based on the full FIM, we used the weight-specific entries of its first eigenvector as a direct indicator of weight importance." Couldn't make sense of this.

11. After Eq. 4, I think it's important to write down the expression for Fisher information (Eq. 19) that was actually used, written in a human-readable form. As far as I can tell, Eq. 19 can be written

<v_i h_j=""> = <v_i><h_j>/[<v_i> + (1-<v_i>)exp(-w_ij (1-<h_j>))].

Algebra mistakes are possible, but I believe the correct expression looks something like this. And it's kind of easy to make sense of: besides the dependence on v_i and h_j, it's an approximately sigmoidal function of the weights. So it would be nice to include it.

12. l 160: "Generally, excessive pruning was detrimental to generative performance" It seems that it's only for the orange line (Heuristic FI?) that pruning was detrimental to generative performance.

13. Fig. 2 caption: "For all pruning strategies except Anti-FI pruning the model retains the ability to match the distribution of the training data (dashed lines) after retraining, indicating good generative performance." As far as I can tell, Anti-FI matched the true distribution in all but one panel.

14. l 186-8: "In sum, for all pruning strategies (except Anti-FI pruning, which removed a large fraction of important weights), the network could recover from the loss of weights and units through retraining". It looks to me like Anti-FI recovers as well.

15. mnist is 28x28. why scale to 20x20? The reason for this should probably be explained.

16. Fig. 3B: why did random and [w] stop at 10^5?

17. fig. 4: probability may not be the best measure -- it's possible in principle that probability deteriorates but performance stays high. I think it would be good to report both performance and probability.

And again, why stop at 10^5 for random and [w], and even higher for Anti-FI?

18. l 350-3: "A recent study of the effects of perturbing the input during different time points of training in neural networks suggests that a critical learning period may be visible in a plateau of the FIM trace [45]." I couldn't make sense of this.</h_j></v_i></v_i></h_j></v_i></v_i>

**********

Large-scale datasets should be made available via a public repository as described in the

Reviewer #1: Yes

**********

PLOS authors have the option to publish the peer review history of their article (

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Reviewer #1: No

Reviewer #2: No

**********

The

Reviewer #2: Yes

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool,

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example in PLOS Biology see here:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at

Submitted filename:

Dear Dr. Hennig,

Thank you very much for submitting your manuscript "The Information Theory of Developmental Pruning: Optimizing Global Network Architecture Using Local Synaptic Rules" for consideration at PLOS Computational Biology. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. The reviewers appreciated the attention to an important topic. Based on the reviews, we are likely to accept this manuscript for publication, providing that you modify the manuscript according to the review recommendations.

Please be sure to address the remaining smaller comments from the reviewers. If you feel that the additional analyses proposed by reviewer 1 are outside the scope of this particular paper (which the reviewer recognised they may be), please just state this clearly in your response to reviewers and consider adding commentary in the discussion in this direction.

Please prepare and submit your revised manuscript within 30 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to all review comments, and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Thank you again for your submission to our journal. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Blake A Richards

Associate Editor

PLOS Computational Biology

Lyle Graham

Deputy Editor

PLOS Computational Biology

***********************

A link appears below if there are any accompanying review attachments. If you believe any reviews to be missing, please contact

[LINK]

Please be sure to address the remaining smaller comments from the reviewers. If you feel that the additional analyses proposed by reviewer 1 are outside the scope of this particular paper (which the reviewer recognised they may be), please just clarify and consider adding commentary in the discussion in this direction.

Reviewer's Responses to Questions

Reviewer #1: The authors put great effort into improving the paper and addressed most of my concerns. I only have a few remaining issues (and suggestions) before I recommend publication:

Concerning 1.4: Random unit pruning:

(a) "... in the one-layer RBM all visible units are connected to all hidden units.When a hidden unit is removed here, the activity and Fisher information can completely re-arrange with re-training". As the unit pruning seems to be the main advantage of the FI-approach, I think this control case should also be included for the single layer RBM. The above intuition can the be discussed and demonstrated in Fig 2C: One would see a large divergence before but not after retraining.

(b) From what I see in Figure 3, the random unit pruning is not really fair with respect to the units in hidden layer 1, as it removes an order of mangitude more neurons in that layer. To really demonstrate that FI-pruning leads to a better network structure more quickly, I propose to adapt this and remove less neurons in h1 to arrive at a comparable structure

l.134/l.333f Could you provide more motivation why it is easier to track firing rates and weights instead of correlation and weights. Biologically, Ca or CaMKII are thought to be local proxies of correlated activity, but I am not aware of molecular signals tracking especially the presynaptic rate.

l.165 I guess here you need to discuss the results a bit deeper, as otherwise panel 2C would have been sufficient to make the point. Specifically, I noticed that the generative performance only seems to be poor for seldom patterns whereas the performance for abundant patterns seem to match (although with larger variation in the Anti-FI case). Is this really so bad for neural system? From an information theoretic viewpoint, they are surely the most informative patterns. However, as these unmatched patterns are rare, the error introduced by them may be negligible.

l.372 The statement seems a bit bold. Maybe use "activity-dependent pruning that aims to identify uninformative neurons"

Suggestions to improve readability:

- In my opinion, it would make sense to move the introduction of the RBMs (l.23-33) to the end of the introduction (after l.55)

- l.70 Maybe one could also mention the relation between energy and pattern probability in equation 1.

- l.101 I would mention how the models were fitted here (wake sleep algorithm).

- l.101 It is not immediately clear what is meant by "parameter-wise" (first mention). I would stick to the terms full and diagonal or at least specify what is meant in this sentence. Moreover, I think it is may be less confusing to discuss the results in the order they are presented in the figure and move the Also, an activity dependent form is only available from Equation 3 or 4, right?

- l.141 It is not immediately clear why the FI introduced before is "variance" based. Maybe the term could be introduced together with the method and the motivation of "variance" could be explained.

- l.150 I think it should be shortly motivated what the generative performance means/relates to in the neuronal/biological system, to give a better intuition what the FI-approach actually preserves.

Finally, I would have another suggestion:

Another advantage of the FI-dependent pruning over other methods may be the fact that it could be used to determine when pruning should be stopped. At the moment this is not the case as the lowest-FI quantile of synapses is always removed. If, instead, only synapses below an FI-threshold would be removed, pruning would naturally stop if all synapses have high FI. Such a convergence would remove the necessity to select a suitable number of pruning iterations for the model and prevent the performance loss of the FI-based models after massive pruning in Fig 3. Assuming that pruning stops after all synapses have high FI, one would get one "optimal" pruned model (instead of one per pruning iteration). Determining these optimal models for different input statistic would also allow predictions on the number of surviving synapses and neurons as well as weight distributions (for example comparing the networks after training with a 5-class MNIST subset and the full dataset). Varying the input statistics and getting different resulting models would greatly underline the point that FI-pruning actually selects input-related "optimal" model architectures and not just "smaller" models whose size is determined by the number of iterations. Moreover, such an analysis would provide more insight into the relation between the encoding of the Boltzmann machine and optimal pruned models, which, I guess, was a goal of this line of research.

The differences in the resulting optimal networks could, in turn, be compared with existing data on network complexity/ neuron and spine densities in animals reared in different environments (e.g. dark rearing, rearing with differently oriented bars, normal cages, enriched environments). This would make a nice connection to biology and provide actually testable pre/postdictions (Concerning the experiments you proposed: at least the experimentalists I know say that it is not feasible to track pre- and postsynaptic activity and the weight of an identified synapse over time at the moment).

I am aware that this additional analysis may be work-intensive and beyond the scope of this paper. However, I think it may greatly improve the manuscript or at least provide an interesting direction for future research.

Reviewer #2: The paper is _much_ improved, and I'm happy with it. Only two comments:

1. in Eq. 10, I believe the weights should have superscripts.

2. I would suggest moving A1 and A2 to Methods. I suspected the more mathematically inclined will be interested. I certainly was, since I got it wrong the first time around. ;) This is, though, completely up to the authors.

**********

The

Reviewer #1: Yes

Reviewer #2: Yes

**********

PLOS authors have the option to publish the peer review history of their article (

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Reviewer #1: No

Reviewer #2: No

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool,

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example in PLOS Biology see here:

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at

References:

Review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.

Submitted filename:

Dear Dr. Hennig,

We are pleased to inform you that your manuscript 'The Information Theory of Developmental Pruning: Optimizing Global Network Architecture Using Local Synaptic Rules' has been provisionally accepted for publication in PLOS Computational Biology.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Computational Biology.

Best regards,

Blake A Richards

Associate Editor

PLOS Computational Biology

Lyle Graham

Deputy Editor

PLOS Computational Biology

***********************************************************

PCOMPBIOL-D-20-02149R2

The Information Theory of Developmental Pruning: Optimizing Global Network Architectures Using Local Synaptic Rules

Dear Dr Hennig,

I am pleased to inform you that your manuscript has been formally accepted for publication in PLOS Computational Biology. Your manuscript is now with our production department and you will be notified of the publication date in due course.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript.

Soon after your final files are uploaded, unless you have opted out, the early version of your manuscript will be published online. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting PLOS Computational Biology and open-access publishing. We are looking forward to publishing your work!

With kind regards,

Andrea Szabo

PLOS Computational Biology | Carlyle House, Carlyle Road, Cambridge CB4 3DN | United Kingdom