Failed to import from vllm._C with ImportError问题解决

1.问题描述

笔者在使用FastChat框架中的vllm_work部署DeepSeek-Coder-V2时遇到了如下报错：

$ python3 /ssdwork/FastChat/fastchat/serve/vllm_worker.py --model-path /ssdwork/DeepSeek-Coder-V2-Instruct/ --num-gpus 8
WARNING 08-13 02:36:34 _custom_ops.py:14] Failed to import from vllm._C with ImportError('/ssdwork/.local/lib/python3.10/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit11parseSchemaERKSs')
INFO 08-13 02:36:39 config.py:715] Defaulting to use mp for distributed inference
WARNING 08-13 02:36:39 arg_utils.py:762] Chunked prefill is enabled by default for models with max_model_len > 32K. Currently, chunked prefill might not work with some features or models. If you encounter any issues, please disable chunked prefill by setting --enable-chunked-prefill=False.
INFO 08-13 02:36:39 config.py:806] Chunked prefill is enabled with max_num_batched_tokens=512.
INFO 08-13 02:36:39 llm_engine.py:176] Initializing an LLM engine (v0.5.3.post1) with config: model='/ssdwork/DeepSeek-Coder-V2-Instruct/', speculative_config=None, tokenizer='/ssdwork/DeepSeek-Coder-V2-Instruct/', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=163840, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=8, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None), seed=0, served_model_name=/ssdwork/DeepSeek-Coder-V2-Instruct/, use_v2_block_manager=False, enable_prefix_caching=False)
INFO 08-13 02:36:39 custom_cache_manager.py:17] Setting Triton cache manager to: vllm.triton_utils.custom_cache_manager:CustomCacheManager
WARNING 08-13 02:36:41 _custom_ops.py:14] Failed to import from vllm._C with ImportError('/ssdwork/.local/lib/python3.10/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit11parseSchemaERKSs')
WARNING 08-13 02:36:41 _custom_ops.py:14] Failed to import from vllm._C with ImportError('/ssdwork/.local/lib/python3.10/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit11parseSchemaERKSs')
WARNING 08-13 02:36:41 _custom_ops.py:14] Failed to import from vllm._C with ImportError('/ssdwork/.local/lib/python3.10/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit11parseSchemaERKSs')
WARNING 08-13 02:36:41 _custom_ops.py:14] Failed to import from vllm._C with ImportError('/ssdwork/.local/lib/python3.10/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit11parseSchemaERKSs')
WARNING 08-13 02:36:41 _custom_ops.py:14] Failed to import from vllm._C with ImportError('/ssdwork/.local/lib/python3.10/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit11parseSchemaERKSs')
WARNING 08-13 02:36:41 _custom_ops.py:14] Failed to import from vllm._C with ImportError('/ssdwork/.local/lib/python3.10/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit11parseSchemaERKSs')
WARNING 08-13 02:36:41 _custom_ops.py:14] Failed to import from vllm._C with ImportError('/ssdwork/.local/lib/python3.10/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit11parseSchemaERKSs')

尽管是warning信息，但会导致模型加载180GB显存左右时突然卡住变成僵尸进程。

2.问题解决

通过上网查询资料，发现大概率是pip包版本之间的冲突，使用如下命令可以进行检测是否有包版本与vllm相关包形成了冲突：

pip check

结果就是vllm、vllm-flash-attn、vllm-nccl-cu12这些包对应的torch版本为2.3.1，而我的torch是最新版本2.4.0，卸载掉重新安装好对应正确版本的包即可。