求助，300i duo卡使用msmodelslim中的w8a8量化，用mindie部署报错_MindIE_昇腾论坛

# 求助，300i duo卡使用msmodelslim中的w8a8量化，用mindie部署报错_MindIE_昇腾论坛

## 概述

本文档基于昇腾社区论坛帖子生成的技术教程。

**原始链接**: https://www.hiascend.com/forum/thread-0250190022756769207-1-1.html

**生成时间**: 2025-08-27 10:33:56

---

## 问题描述

w8a8量化成功了 2025-08-08 15:54:46,726 - msmodelslim-logger - INFO - layer: 'model.layers.39.mlp.up_proj.quant_input', range: tensor(17.3750, dtype=torch.float16), Automatically set the drop rate: 1.0 100%|| 48/48 [02:39<00:00, 3.33s/it] 2025-08-08 15:57:21,113 - msmodelslim-logger - INFO - Calibration end! 2025-08-08 15:57:21,153 - msmodelslim-logger - INFO - calibration success 2025-08-08 15:57:21,154 - msmodelslim-logger - INFO - ==========QUANTIZATION: PERSIST========== 2025-08-08 15:57:21,154 - msmodelslim-logger - INFO - write directory exists, write file to directory '/models9' 2025-08-08 15:57:21,154 - msmodelslim-logger - WARNING - ascendV1 is new version, numpy and safe_tensor will be ignored, only ascendV1 will be saved. 2025-08-08 15:57:21,154 - msmodelslim-logger - WARNING - invalid `safetensors_name`, defaulting to `quant_model_weight_w8a8_mix.safetensors` 2025-08-08 15:57:21,154 - msmodelslim-logger - INFO - write directory exists, write file to directory '/models9' 2025-08-0...

## 完整内容

w8a8量化成功了 2025-08-08 15:54:46,726 - msmodelslim-logger - INFO - layer: 'model.layers.39.mlp.up_proj.quant_input', range: tensor(17.3750, dtype=torch.float16), Automatically set the drop rate: 1.0 100%|| 48/48 [02:39<00:00, 3.33s/it] 2025-08-08 15:57:21,113 - msmodelslim-logger - INFO - Calibration end! 2025-08-08 15:57:21,153 - msmodelslim-logger - INFO - calibration success 2025-08-08 15:57:21,154 - msmodelslim-logger - INFO - ==========QUANTIZATION: PERSIST========== 2025-08-08 15:57:21,154 - msmodelslim-logger - INFO - write directory exists, write file to directory '/models9' 2025-08-08 15:57:21,154 - msmodelslim-logger - WARNING - ascendV1 is new version, numpy and safe_tensor will be ignored, only ascendV1 will be saved. 2025-08-08 15:57:21,154 - msmodelslim-logger - WARNING - invalid `safetensors_name`, defaulting to `quant_model_weight_w8a8_mix.safetensors` 2025-08-08 15:57:21,154 - msmodelslim-logger - INFO - write directory exists, write file to directory '/models9' 2025-08-08 15:57:21,155 - msmodelslim-logger - INFO - write directory exists, write file to directory '/models9' Collect quant param: 100%|| 1308/1308 [00:33<00:00, 38.77it/s] 2025-08-08 15:57:56,290 - msmodelslim-logger - INFO - Save safetensors files to /models9 successfully 2025-08-08 15:57:56,290 - msmodelslim-logger - INFO - Path of quant_model_description_json is '/models9/quant_model_description.json' 2025-08-08 15:57:56,311 - msmodelslim-logger - INFO - Save quant_model_description_json success! 2025-08-08 15:57:56,311 - msmodelslim-logger - INFO - Safetensors weight saved successfully 2025-08-08 15:57:56,311 - msmodelslim-logger - INFO - Save successfully! 2025-08-08 15:57:56,323 - msmodelslim-logger - INFO - ==========QUANTIZATION: END========== 2025-08-08 15:57:56,328 - msmodelslim-logger - INFO - ===========SUCCESS=========== 但是在我用mindie拉起服务化部署的时候报错如下 [2025-08-08 16:01:05,534] [26322] [281463904006496] [llm] [INFO] [cache_pool.py-178] : Allocating npu_cache, acl format is NZ, sepd mode is off. [2025-08-08 16:01:05,538] [26322] [281463904006496] [llm] [INFO] [batch.py-50] : The default sampling params are: (1., 0., 0., 0.6, 20., 0.95, 1., 0.) [2025-08-08 16:01:05,574] [26322] [281463904006496] [llm] [INFO] [batch.py-132] : The initial capacity of state cache is `2048`. [2025-08-08 16:01:05.008+0800] [26322] [281472663146848] [batchscheduler] [ERROR] [model.py:61] : [Model] >>> Exception:'FlashQwenLayer' object has no attribute 'attn' Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 59, in initialize return self.python_model.initialize(config)  File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 133, in initialize self.generator = Generator(  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 299, in __init__ self.cache_manager = self.warm_up(  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 456, in warm_up npu_mem = self.__warmup_standard(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times)  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 626, in __warmup_standard npu_mem = self.__warmup_prefill(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times) 这是什么原因啊难道说300i duo这张卡不支持w8a8量化么

---

## 技术要点总结

基于以上内容，主要技术要点包括：

1. **问题类型**: 错误处理
2. **涉及技术**: 昇腾, NPU, MindIE
3. **解决方案**: 请参考完整内容中的解决方案

## 相关资源

- 昇腾社区: https://www.hiascend.com/
- 昇腾论坛: https://www.hiascend.com/forum/

---

*本文档由AI自动生成，仅供参考。如有疑问，请参考原始帖子。*