启动dsv3报错_MindIE_昇腾论坛

# 启动dsv3报错_MindIE_昇腾论坛

## 概述

本文档基于昇腾社区论坛帖子生成的技术教程。

**原始链接**: https://www.hiascend.com/forum/thread-0279186637038199059-1-1.html

**生成时间**: 2025-08-27 10:34:00

---

## 问题描述

2025-06-30 11:34:56.384906 25954 LLM log default format: [yyyy-mm-dd hh:mm:ss.uuuuuu][processid] [threadid] [llm] [loglevel][file:line] [error code] msg [2025-06-30 11:34:56.384965] [25840] [281472055767392] [llm] [INFO] [post_processing_manager.cpp:146] Get post processing manager [2025-06-30 11:34:56.385030] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281470688538976 [2025-06-30 11:34:56.385089] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468831068512 [2025-06-30 11:34:56.385142] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468822614368 [2025-06-30 11:34:56.385191] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468814160224 [2025-06-30 11:34:56.385325] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468735582560 [2025-06-30 11:34:56.385376] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create p...

## 完整内容

2025-06-30 11:34:56.384906 25954 LLM log default format: [yyyy-mm-dd hh:mm:ss.uuuuuu][processid] [threadid] [llm] [loglevel][file:line] [error code] msg [2025-06-30 11:34:56.384965] [25840] [281472055767392] [llm] [INFO] [post_processing_manager.cpp:146] Get post processing manager [2025-06-30 11:34:56.385030] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281470688538976 [2025-06-30 11:34:56.385089] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468831068512 [2025-06-30 11:34:56.385142] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468822614368 [2025-06-30 11:34:56.385191] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468814160224 [2025-06-30 11:34:56.385325] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468735582560 [2025-06-30 11:34:56.385376] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468727128416 [2025-06-30 11:34:56.385428] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468718674272 [2025-06-30 11:34:56.385475] [25840] [281472055767392] [llm] [INFO] [thread_pool.cpp:51] Create pthread id: 281468710220128 [2025-06-30 11:34:59.737941] [25840] [281472055767392] [llm] [ERROR] [acl_nn_operation.cpp:143] gmmNode call SetAclNNWorkspaceExecutor fail, error:161002 [2025-06-30 11:34:59.738065] [25840] [281472055767392] [llm] [ERROR] [acl_nn_operation.cpp:116] gmmNode call CreateAclNNOpCache fail, error:12 [2025-06-30 11:34:59.738083] [25840] [281472055767392] [llm] [ERROR] [acl_nn_operation.cpp:59] gmmNode call UpdateAclNNOpCache, error:12 [2025-06-30 11:34:59.738146] [error] [25954] [graph_runner.cpp:762] integrated_gmmRunner_5_4_9_0_2:0 node[0] setup fail, error code:12 [2025-06-30 11:34:59.738177] [error] [25954] [graph_runner.cpp:270] integrated_gmmRunner_5_4_9_0_2:0 node[0] SetupNodeRunners fail, error code:12 [2025-06-30 11:34:59.738187] [error] [25954] [graph_runner.cpp:762] MoeMlpRunner_5_4_9_0:0 node[2] setup fail, error code:12 [2025-06-30 11:34:59.738195] [error] [25954] [graph_runner.cpp:270] MoeMlpRunner_5_4_9_0:0 node[2] SetupNodeRunners fail, error code:12 [2025-06-30 11:34:59.738202] [error] [25954] [graph_runner.cpp:762] DynamicEpMoERunner_5_4_9:0 node[0] setup fail, error code:12 [2025-06-30 11:34:59.738209] [error] [25954] [graph_runner.cpp:270] DynamicEpMoERunner_5_4_9:0 node[0] SetupNodeRunners fail, error code:12 [2025-06-30 11:34:59.738216] [error] [25954] [graph_runner.cpp:762] SparseMoeRunner_5_4:0 node[9] setup fail, error code:12 [2025-06-30 11:34:59.738223] [error] [25954] [graph_runner.cpp:270] SparseMoeRunner_5_4:0 node[9] SetupNodeRunners fail, error code:12 [2025-06-30 11:34:59.738230] [error] [25954] [graph_runner.cpp:762] Prefill_layerRunner_5:0 node[4] setup fail, error code:12 [2025-06-30 11:34:59.738237] [error] [25954] [graph_runner.cpp:270] Prefill_layerRunner_5:0 node[4] SetupNodeRunners fail, error code:12 [2025-06-30 11:34:59.738245] [error] [25954] [operation_base.cpp:462] Prefill_layer_5 runner setup fail [2025-06-30 11:34:59.738252] [error] [25954] [operation_base.cpp:518] Prefill_layer_5 setup fail, error code: 12 [2025-06-30 11:34:59.030+0800] [25840] [281473121775968] [mindie-server] [ERROR] [model.py:40] : [Model] >>> Exception:Setup fail, enable log: export ASDOPS_LOG_LEVEL=ERROR, export ASDOPS_LOG_TO_STDOUT=1 to find the first error. For more details, see the MindIE official document. Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 38, in initialize return self.python_model.initialize(config)  File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 93, in initialize self.generator = Generator(  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 125, in __init__ self.warm_up(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times, inference_mode) File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 316, in warm_up raise e File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 309, in warm_up self._generate_inputs_warm_up_backend(input_metadata, inference_mode, dummy=True) File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 392, in _generate_inputs_warm_up_backend self.generator_backend._warm_up(model_inputs, inference_mode=inference_mode) File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 204, in _warm_up super()._warm_up(model_inputs) File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_backend.py", line 176, in _warm_up _ = self.forward(model_inputs, **kwargs)  File "/usr/local/lib/python3.11/site-packages/mindie_llm/utils/decorators/time_decorator.py", line 73, in wrapper return func(*args, **kwargs)  File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 170, in forward logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)  File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 109, in forward logits = self.forward_tensor(  File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 136, in forward_tensor logits = self.model_runner.forward(  File "/usr/local/Ascend/atb-models/atb_llm/runner/model_runner.py", line 220, in forward return self.model.forward(**kwargs)  File "/usr/local/Ascend/atb-models/atb_llm/models/base/flash_causal_lm.py", line 466, in forward logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)  File "/usr/local/Ascend/atb-models/atb_llm/models/base/flash_causal_lm.py", line 418, in execute_ascend_operator acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)  RuntimeError: Setup fail, enable log: export ASDOPS_LOG_LEVEL=ERROR, export ASDOPS_LOG_TO_STDOUT=1 to find the first error. For more details, see the MindIE official document. [2025-06-30 11:34:59.030+0800] [25840] [281473121775968] [mindie-server] [ERROR] [model.py:43] : [MIE04E13030A] [Model] >>> return initialize error result: {'status': 'error', 'npuBlockNum': '0', 'cpuBlockNum': '0'} [2025-06-30 11:34:59.747+08:00] [25822] [25823] [mindie-server] [WARN] [llm_daemon.cpp:66] : [MIE04W01011A] [daemon] Received exit signal[17] [2025-06-30 11:34:59.747+08:00] [25822] [25823] [mindie-server] [INFO] [llm_daemon.cpp:71] : [daemon] Daemon wait pid with 25840, status 9 [2025-06-30 11:34:59.747+08:00] [25822] [25823] [mindie-server] [ERROR] [llm_daemon.cpp:76] : [MIE04E010109] [daemon] Daemon wait pid with 25840 exit, Please check the service log or python log. [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! /usr/lib64/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 30 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d ' Daemon is killing... [2025-06-30 11:35:04.747+08:00] [25822] [blma4-0-0] [system] [server] [stop endpoint] [success] [2025-06-30 11:35:04.747+08:00] [25822] [blma4-0-0] [system] [server] [stop mindie server] [success] Killed gmmNode call SetAclNNWorkspaceExecutor fail, 通过报错信息得知是在group matmul 算子报错麻烦你开一下日志看看大概率是服务化的一些参数设置的不对

---

## 技术要点总结

基于以上内容，主要技术要点包括：

1. **问题类型**: 错误处理
2. **涉及技术**: 昇腾, NPU, AI, MindIE
3. **解决方案**: 请参考完整内容中的解决方案

## 相关资源

- 昇腾社区: https://www.hiascend.com/
- 昇腾论坛: https://www.hiascend.com/forum/

---

*本文档由AI自动生成，仅供参考。如有疑问，请参考原始帖子。*