Abstract
Neural network pruning serves as a critical technique for enhancing the efficiency of deep learning models. Unlike unstructured pruning, which only sets specific parameters to zero, structured pruning eliminates entire channels, thus yielding direct computational and storage benefits. However, the diverse patterns for coupling parameters, such as residual connections and group convolutions, the diverse deep learning frameworks, and the various time stages at which pruning can be performed make existing pruning methods less adaptable to different architectures, frameworks, and pruning criteria. To address this, we introduce Structurally Prune Anything (SPA), a versatile structured pruning framework that can prune neural networks with any architecture, from any framework, and at any stage of training. SPA leverages a standardized computational graph and ONNX representation to prune diverse neural network architectures without the need for manual intervention. SPA employs a group-level importance estimation method, which groups dependent computational operators, estimates their importance, and prunes unimportant coupled channels. This enables the transfer of various existing pruning criteria into a structured group style. As a result, SPA supports pruning at any time, either before training, after training with fine-tuning, or after training without fine-tuning. In the context of the latter, we introduce Optimal Brain SPA (OBSPA), an algorithm that achieves state-of-the-art pruning results needing neither fine-tuning nor calibration data. In extensive experiments, SPA shows competitive to state-of-the-art pruning performance across various architectures, from popular frameworks, at different pruning times.
Method
Structured pruning removes entire channels rather than setting individual weights to zero. This makes the parameter arrays smaller and reduces arithmetic and storage requirements. Because channels are connected across layers, their removal must be coordinated to preserve the network’s structure.
1 Standardized computational graph
SPA converts the source model into ONNX, a common representation of machine-learning models. Standardized operators describe the model’s computation, shapes, and parameters, allowing the pruning logic to operate independently of the original framework’s layer implementations.
2 Channel dependency analysis
Mask propagation follows candidate channel removals through the graph using operator-specific rules. It identifies channels that must change together across additions, concatenations, grouped convolutions, and other operations. For example, channels on two branches joined by an addition must remain aligned.
3 Group-level importance estimation
Coupled channels are organized into groups and scored using information across their linked parameters. This separates the structural question of what must be removed together from the choice of importance criterion, allowing existing pruning criteria to be used in a structured group setting.
4 Coordinated pruning and export
SPA removes lower-scoring candidates together and updates the parameter arrays and shapes in the ONNX model. The smaller model can then be converted to a suitable framework for further training or deployment. Compatibility depends on model conversion and support for the operators involved.
Pruning Settings
By separating dependency analysis from importance estimation, SPA supports pruning at three stages:
- Before training: prune the structure, then train the smaller network.
- After training, with fine-tuning: prune a trained model, then use additional training to recover accuracy.
- After training, without fine-tuning: produce a smaller model without a subsequent training stage.
Optimal Brain SPA (OBSPA)
For pruning without fine-tuning, OBSPA selects whole coupled channels for removal and adjusts the remaining weights to help preserve each layer’s outputs. It can operate without the original training data: the strict data-free setting uses synthetic inputs sampled from a uniform distribution for calibration.
Experimental Results
The paper evaluates 11 architectures and demonstrates pruning models originating in PyTorch, TensorFlow, MXNet, and JAX. The experiments include convolutional networks, a vision transformer, and DistilBERT for sentiment classification.
| Model | Original accuracy | Pruned accuracy | FLOPs reduction factor |
|---|---|---|---|
| ResNet-50 | 93.26% | 93.42% | 2.13× |
| ViT-base | 95.35% | 96.10% | 2.05× |
| DistilBERT | 91.06% | 88.88% | 2.04× |
A 2× FLOPs reduction factor means roughly half as many arithmetic operations, not necessarily half the latency on a particular device. The results illustrate the accuracy–computation trade-off across different models and tasks.
Takeaways
- Prune connected structures, not isolated channels. Explicit dependency analysis keeps coupled parts of the network consistent when channels are removed.
- Separate structure from scoring. A shared graph representation lets different importance criteria work across architectures, source frameworks, and pruning stages.
- Match pruning to the available resources. SPA supports workflows with or without further training; OBSPA addresses the setting without fine-tuning or original training data.
BibTeX
@misc{wang2024structurallypruneanythingarchitecture,
title={Structurally Prune Anything: Any Architecture, Any Framework, Any Time},
author={Xun Wang and John Rachwan and Stephan Günnemann and Bertrand Charpentier},
year={2024},
eprint={2403.18955},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2403.18955},
}