전체 글72 [Paper Review] Quantized Feature Distillation for Network Quantization 논문 원본 : https://arxiv.org/abs/2307.10638 Quantized Feature Distillation for Network QuantizationNeural network quantization aims to accelerate and trim full-precision neural network models by using low bit approximations. Methods adopting the quantization aware training (QAT) paradigm have recently seen a rapid growth, but are often conceptually comparxiv.org Abstract & Introduction신경망 양자화(Neura.. 2024. 12. 29. [Paper Review] Sparse Model Soups : A Recipe For Improved Pruning Via Model Averaging 논문 원본 : https://arxiv.org/abs/2306.16788 Sparse Model Soups: A Recipe for Improved Pruning via Model AveragingNeural networks can be significantly compressed by pruning, yielding sparse models with reduced storage and computational demands while preserving predictive performance. Model soups (Wortsman et al., 2022) enhance generalization and out-of-distribution (Oarxiv.org Abstract신경망은 pruning을 .. 2024. 12. 22. [Paper review] Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time 논문 원본 : https://arxiv.org/abs/2203.05482 Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timeThe conventional recipe for maximizing model accuracy is to (1) train multiple models with various hyperparameters and (2) pick the individual model which performs best on a held-out validation set, discarding the remainder. In this paper, we re.. 2024. 12. 22. [Paper Review] Self-Supervised Quantization-Aware Knowledge Distillation 논문 원본 : https://arxiv.org/abs/2403.11106 Self-Supervised Quantization-Aware Knowledge DistillationQuantization-aware training (QAT) and Knowledge Distillation (KD) are combined to achieve competitive performance in creating low-bit deep learning models. However, existing works applying KD to QAT require tedious hyper-parameter tuning to balance the weiarxiv.org AbstractQuantization-aware trainin.. 2024. 12. 1. [Paper Review] Learned Step Size Quantization 논문 원본 : https://arxiv.org/abs/1902.08153 Learned Step Size QuantizationDeep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaining high accuracy as precision decreases. Here, we present a method for trainarxiv.org Abstract딥러닝 네트워크는 inference 시 계산 비용을 줄이고 효율성을 높이기 위해 저정밀도(ex... 2024. 11. 30. [Paper Review] OverComing Oscillations in Quantization-Aware Training 논문 원본 : https://arxiv.org/abs/2203.11086 Overcoming Oscillations in Quantization-Aware TrainingWhen training neural networks with simulated quantization, we observe that quantized weights can, rather unexpectedly, oscillate between two grid-points. The importance of this effect and its impact on quantization-aware training (QAT) are not well-understarxiv.org Abstract딥러닝 모델을 simulated quantizati.. 2024. 11. 28. 이전 1 2 3 4 5 6 ··· 12 다음
