Qwen3-Reranker-4B重排序模型能够正常启动，访问重排序模型接口时报错，NPU out of memory，请问该如何解决_MindIE_昇腾论坛

# Qwen3-Reranker-4B重排序模型能够正常启动，访问重排序模型接口时报错，NPU out of memory，请问该如何解决_MindIE_昇腾论坛

## 概述

本文档基于昇腾社区论坛帖子生成的技术教程。

**原始链接**: https://www.hiascend.com/forum/thread-02102190968459319313-1-1.html

**生成时间**: 2025-08-27 10:33:56

---

## 问题描述

环境说明 昇腾910B3显卡合计8张单卡显存64G 启动命令 docker run -itd --privileged=true -e TEI_NPU_DEVICE=2 -p 8888:8888 -e max_split_size_mb=512 -e MAX_CLIENT_BATCH_SIZE=128 -e POOLING=mean -e IS_RERANK=1 -e DEFAULT_PROMPT="<Query>: <s1>\\n<Document>: <s2>" -e ENABLE_BOOST=True -e ASCEND_VISIBLE_DEVICES=2 --name=Qwen3-Reranker-4B --net=host -v /home/data:/home/HwHiAiUser/model swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.1.RC1-800I-A2-aarch64 Qwen3-Reranker-4B 0.0.0.0 8888 尝试过调整这2个参数的大小往低调整也是报错例如max_split_size_mb=512 MAX_CLIENT_BATCH_SIZE=100 报错信息 2025-08-19T01:56:36.682075Z INFO batching_task:next_batch: text_embeddings_core::queue: core/src/queue.rs:188: inference batch size is:33 2025-08-19T01:56:37.872152Z ERROR python-backend: text_embeddings_backend_python::logging: backends/python/src/logging.rs:40: Method Predict encountered an error. Traceback (most recent call last): File "/usr/local/bin/python-text-embeddings-server", line 8, in <module> sys.exit(app()) File "/usr/local/lib/python3.11/dis...

## 完整内容

环境说明 昇腾910B3显卡合计8张单卡显存64G 启动命令 docker run -itd --privileged=true -e TEI_NPU_DEVICE=2 -p 8888:8888 -e max_split_size_mb=512 -e MAX_CLIENT_BATCH_SIZE=128 -e POOLING=mean -e IS_RERANK=1 -e DEFAULT_PROMPT="<Query>: <s1>\\n<Document>: <s2>" -e ENABLE_BOOST=True -e ASCEND_VISIBLE_DEVICES=2 --name=Qwen3-Reranker-4B --net=host -v /home/data:/home/HwHiAiUser/model swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.1.RC1-800I-A2-aarch64 Qwen3-Reranker-4B 0.0.0.0 8888 尝试过调整这2个参数的大小往低调整也是报错例如max_split_size_mb=512 MAX_CLIENT_BATCH_SIZE=100 报错信息 2025-08-19T01:56:36.682075Z INFO batching_task:next_batch: text_embeddings_core::queue: core/src/queue.rs:188: inference batch size is:33 2025-08-19T01:56:37.872152Z ERROR python-backend: text_embeddings_backend_python::logging: backends/python/src/logging.rs:40: Method Predict encountered an error. Traceback (most recent call last): File "/usr/local/bin/python-text-embeddings-server", line 8, in <module> sys.exit(app()) File "/usr/local/lib/python3.11/dist-packages/typer/main.py", line 311, in __call__ return get_command(self)(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/typer/core.py", line 716, in main return _main( File "/usr/local/lib/python3.11/dist-packages/typer/core.py", line 216, in _main rv = self.invoke(ctx) File "/usr/local/lib/python3.11/dist-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.11/dist-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/typer/main.py", line 683, in wrapper return callback(**use_params) # type: ignore File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/cli.py", line 51, in serve server.serve(model_path, dtype, uds_path, pool) File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/server.py", line 118, in serve asyncio.run(serve_inner(model_path, dtype)) File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) File "/usr/lib/python3.11/asyncio/base_events.py", line 641, in run_until_complete self.run_forever() File "/usr/lib/python3.11/asyncio/base_events.py", line 608, in run_forever self._run_once() File "/usr/lib/python3.11/asyncio/base_events.py", line 1936, in _run_once handle._run() File "/usr/lib/python3.11/asyncio/events.py", line 84, in _run self._context.run(self._callback, *self._args) File "/usr/local/lib/python3.11/dist-packages/grpc_interceptor/server.py", line 159, in invoke_intercept_method return await self.intercept( > File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/utils/interceptor.py", line 21, in intercept return await response File "/usr/local/lib/python3.11/dist-packages/opentelemetry/instrumentation/grpc/_aio_server.py", line 120, in _unary_interceptor raise error File "/usr/local/lib/python3.11/dist-packages/opentelemetry/instrumentation/grpc/_aio_server.py", line 111, in _unary_interceptor return await behavior(request_or_iterator, context) File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/server.py", line 62, in Predict scores = self.model.predict(batch) File "/usr/lib/python3.11/contextlib.py", line 81, in inner return func(*args, **kwds) File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/models/qwen3_rerank_model.py", line 88, in predict outputs = self.model(**inputs) File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/transformers/utils/generic.py", line 965, in wrapper output = func(self, *args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/transformers/utils/deprecation.py", line 172, in wrapped_func return func(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/transformers/models/qwen3/modeling_qwen3.py", line 866, in forward logits = self.lm_head(hidden_states[:, slice_indices, :]) File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872448Z ERROR backend_task:predict:predict: backend_grpc_client: backends/grpc-client/src/lib.rs:25: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872664Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872693Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872717Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872732Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872745Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872759Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872773Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872788Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872802Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872816Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872829Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872843Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872856Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872871Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872887Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872901Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872914Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872928Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872962Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872977Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.872990Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873003Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873017Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873030Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873042Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873056Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873072Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873085Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873098Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873111Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873124Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873138Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 2025-08-19T01:56:37.873151Z ERROR openai_rerank:predict{truncate=false truncation_direction=Right raw_scores=false}: text_embeddings_core::infer: core/src/infer.rs:427: Server error: NPU out of memory. Tried to allocate 2.06 GiB (NPU 2; 60.97 GiB total capacity; 9.59 GiB already allocated; 9.59 GiB current active; 266.62 MiB free; 11.14 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. 请问解决了吗遇到了同样的情况 没有论坛上没有人回复

---

## 技术要点总结

基于以上内容，主要技术要点包括：

1. **问题类型**: 错误处理
2. **涉及技术**: PyTorch, NPU, Docker, MindIE, 昇腾, AI
3. **解决方案**: 请参考完整内容中的解决方案

## 相关资源

- 昇腾社区: https://www.hiascend.com/
- 昇腾论坛: https://www.hiascend.com/forum/

---

*本文档由AI自动生成，仅供参考。如有疑问，请参考原始帖子。*