Cython编译报错“numpy/arrayobject.h: No such file or directory”解决方案

DechinPhy

发布于 2024-06-27 09:16:30

460

发布于 2024-06-27 09:16:30

文章被收录于专栏：Dechin的专栏Dechin的专栏

问题背景

Cython是用来加速Python程序性能的一个工具，其基本使用逻辑就是将类Python代码(*.pyx扩展格式)编译成

*.c,*.so

动态链接库文件，然后就可以在正常的Python脚本文件中调用动态链接库的内部函数。编译过程中因为会去索引一些头文件，如果找不到路径就有可能报错。

测试案例

我们可以用Cython做一个简单的基于numpy array输入的求和函数：

# test_sum.pyx
import numpy as np
cimport numpy as np

cpdef double my_sum(double[:] arr):
    cdef double s = 0.0
    cdef int i
    for i in range(arr.shape[0]):
        s += arr[i]
    return s

def main():
    a = np.array([1.0, 2.0, 3.0])
    print(my_sum(a))

然后编译：

$ cythonize -i test_sum.pyx
running build_ext
building 'test_sum' extension
creating /home/mindsponge/tests/tmpwo3gq_ad/home
creating /home/mindsponge/tests/tmpwo3gq_ad/home/mindsponge
creating /home/mindsponge/tests/tmpwo3gq_ad/home/mindsponge/tests
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/python-3.7.5/include/python3.7m -c /home/mindsponge/tests/test_sum.c -o /home/mindsponge/tests/tmpwo3gq_ad/home/mindsponge/tests/test_sum.o
/home/mindsponge/tests/test_sum.c:1240:10: fatal error: numpy/arrayobject.h: No such file or directory
 #include "numpy/arrayobject.h"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1

这个报错是找不到numpy下的一个头文件，那么解决问题的思路就很直接，找到相关头文件的路径，添加到gcc编译的环境变量中即可。先看一下numpy的安装路径：

$ python3 -m pip show numpy
Name: numpy
Version: 1.21.6
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: 
License: BSD
Location: /usr/local/python-3.7.5/lib/python3.7/site-packages
Requires: 
Required-by: bitshuffle, fabio, h5py, matplotlib, mindinsight, mindspore, mindspore-serving, pandas, pyopencl, scikit-learn, scipy, silx, Xponge

然后在这个Location中找到对应的头文件：

$ find /usr/local/python-3.7.5/lib/python3.7/site-packages -name 'arrayobject.h'
/usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h

找到以后先查看一下环境变量中是否已有值，然后再将上面这个路径添加到环境变量中：

$ export | grep C_INCLUDE_PATH
$ export C_INCLUDE_PATH=/usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/

再次执行编译：

# cythonize -i test_sum.pyx                                                                                          
running build_ext
building 'test_sum' extension
creating /home/mindsponge/tests/tmpw_icoc3b/home
creating /home/mindsponge/tests/tmpw_icoc3b/home/mindsponge
creating /home/mindsponge/tests/tmpw_icoc3b/home/mindsponge/tests
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/python-3.7.5/include/python3.7m -c /home/mindsponge/tests/test_sum.c -o /home/mindsponge/tests/tmpw_icoc3b/home/mindsponge/tests/test_sum.o
In file included from /usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1969:0,
                 from /usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from /home/mindsponge/tests/test_sum.c:1240:
/usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it with " \
  ^~~~~~~
gcc -pthread -shared /home/mindsponge/tests/tmpw_icoc3b/home/mindsponge/tests/test_sum.o -L/usr/local/python-3.7.5/lib -lpython3.7m -o /home/mindsponge/tests/test_sum.cpython-37m-x86_64-linux-gnu.so

编译顺利通过，并且会在当前路径下生成一个*.c文件和一个*.so动态链接库文件。然后就可以在python中直接引用动态链接库的内部函数：

In [1]: import numpy as np

In [2]: a = np.array([1.0, 2.0, 3.0])

In [3]: from test_erf import my_sum

In [4]: my_sum(a)
Out[4]: 6.0

总结概要

本文介绍了一个在使用Cython进行Python高性能编程时有可能遇到的一个问题，就是找不到的对应的C语言的头文件，例如numpy中的一些头文件。解决思路就是先在本地找到相应的头文件路径，然后将其添加到编译器的环境变量中即可。

版权声明

本文首发链接为：https://www.cnblogs.com/dechinphy/p/win11-vmd.html

作者ID：DechinPhy

更多原著文章：https://www.cnblogs.com/dechinphy/

请博主喝咖啡：https://www.cnblogs.com/dechinphy/gallery/image/379634.html

本文参与腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。

原始发表：2024-06-26，如有侵权请联系 cloudcommunity@tencent.com 删除

numpy