pip install xmlsec crashes on Debian

[yaroslav-ilin]

I’m on Debian 7.7

Linux *.com 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2+deb7u2 x86_64 GNU/Linux

user@host:~$ dpkg --get-selections | grep libxml2
libxml2:amd64                   install
libxml2-dev:amd64               install
user@host:~$ dpkg --get-selections | grep libxmlsec1
libxmlsec1                  install
libxmlsec1-dev                  install
libxmlsec1-gcrypt               install
libxmlsec1-gnutls               install
libxmlsec1-nss                  install
libxmlsec1-openssl              install
user@host:~$ dpkg --get-selections | grep python-dev
python-dev                  install

Running pip install xmlsec fails to build with following message:

creating build/temp.linux-x86_64-2.7/src/xmlsec

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_OPENSSL_098=1 -DXMLSEC_NO_GOST=1 -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_SIZE_T= -DXMLSEC_CRYPTO=openssl -I/usr/local/lib/python2.7/dist-packages/lxml -I/usr/local/lib/python2.7/dist-packages/lxml/includes -Isrc -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/usr/include/python2.7 -c src/xmlsec/constants.c -o build/temp.linux-x86_64-2.7/src/xmlsec/constants.o -DXMLSEC_CRYPTO_OPENSSL=1

gcc: error: src/xmlsec/constants.c: No such file or directory

gcc: fatal error: no input files

compilation terminated.

error: command 'gcc' failed with exit status 4

What am I doing wrong?

[mehcode]

I'm guessing that you don't have the neccessary requirements.

apt-get install libxml2-dev libxmlsec1-dev

I'm on ubuntu at the moment so I'll give it a try in a minute (from a fresh virtualenv).

---

Oh.. obviously you do have them.. weird. We'll I'm trying it now.

---

Are you using python 2 or 3 ?

---

It seems you're on 2.7.x; I'll install that and try.

[yaroslav-ilin]

Yeah, I’m on 2.7.3

$ python --version
Python 2.7.3

[mehcode]

It installs for me fine in both 3.4.2 and 2.7.9. I can check 2.7.3 but I don't think the patch version number is the problem.

Can you install pip install lxml just fine?

Do you have the pkg-config utility available? This library makes use of https://github.com/matze/pkgconfig

[mehcode]

Downloading/unpacking xmlsec
  Downloading xmlsec-0.2.0.tar.gz
  Running setup.py (path:/home/ryan/.pyenv/versions/scratch-2.7.3/build/xmlsec/setup.py) egg_info for package xmlsec
    /home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/setuptools-8.2-py2.7.egg/pkg_resources.py:2443: PEP440Warning: 'nose (dev)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
    no previously-included directories found matching 'doc/.build'

    Installed /tmp/easy_install-crJbGG/pkgconfig-1.1.0/.eggs/nose-1.3.4-py2.7.egg
    zip_safe flag not set; analyzing archive contents...

    Installed /home/ryan/.pyenv/versions/scratch-2.7.3/build/xmlsec/.eggs/pkgconfig-1.1.0-py2.7.egg
    Searching for setuptools-cython
    Reading https://pypi.python.org/simple/setuptools_cython/
    Best match: setuptools-cython 0.2.1
    Downloading https://pypi.python.org/packages/source/s/setuptools_cython/setuptools_cython-0.2.1.tar.gz#md5=c29fc5bf2894e69cbd98e3800cef4707
    Processing setuptools_cython-0.2.1.tar.gz
    Writing /tmp/easy_install-Bt6L_7/setuptools_cython-0.2.1/setup.cfg
    Running setuptools_cython-0.2.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Bt6L_7/setuptools_cython-0.2.1/egg-dist-tmp-AQeBUd
    zip_safe flag not set; analyzing archive contents...

    Installed /home/ryan/.pyenv/versions/scratch-2.7.3/build/xmlsec/.eggs/setuptools_cython-0.2.1-py2.7.egg

Requirement already satisfied (use --upgrade to upgrade): lxml>=3.0 in /home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages (from xmlsec)
Installing collected packages: xmlsec
  Running setup.py install for xmlsec
    cythoning src/xmlsec/constants.pyx to src/xmlsec/constants.c
    building 'xmlsec.constants' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_NO_GOST=1 -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_SIZE_T= -DXMLSEC_CRYPTO=openssl -DXMLSEC_OPENSSL_100=1 -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml/includes -Isrc -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/home/ryan/.pyenv/versions/2.7.3/include/python2.7 -c src/xmlsec/constants.c -o build/temp.linux-x86_64-2.7/src/xmlsec/constants.o -DXMLSEC_CRYPTO_OPENSSL=1
    gcc -pthread -shared -L/home/ryan/.pyenv/versions/2.7.3/lib build/temp.linux-x86_64-2.7/src/xmlsec/constants.o -lxmlsec1 -lssl -lxslt -lxml2 -lcrypto -lxmlsec1-openssl -o build/lib.linux-x86_64-2.7/xmlsec/constants.so
    cythoning src/xmlsec/utils.pyx to src/xmlsec/utils.c
    building 'xmlsec.utils' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_NO_GOST=1 -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_SIZE_T= -DXMLSEC_CRYPTO=openssl -DXMLSEC_OPENSSL_100=1 -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml/includes -Isrc -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/home/ryan/.pyenv/versions/2.7.3/include/python2.7 -c src/xmlsec/utils.c -o build/temp.linux-x86_64-2.7/src/xmlsec/utils.o -DXMLSEC_CRYPTO_OPENSSL=1
    gcc -pthread -shared -L/home/ryan/.pyenv/versions/2.7.3/lib build/temp.linux-x86_64-2.7/src/xmlsec/utils.o -lxmlsec1 -lssl -lxslt -lxml2 -lcrypto -lxmlsec1-openssl -o build/lib.linux-x86_64-2.7/xmlsec/utils.so
    cythoning src/xmlsec/tree.pyx to src/xmlsec/tree.c
    building 'xmlsec.tree' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_NO_GOST=1 -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_SIZE_T= -DXMLSEC_CRYPTO=openssl -DXMLSEC_OPENSSL_100=1 -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml/includes -Isrc -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/home/ryan/.pyenv/versions/2.7.3/include/python2.7 -c src/xmlsec/tree.c -o build/temp.linux-x86_64-2.7/src/xmlsec/tree.o -DXMLSEC_CRYPTO_OPENSSL=1
    gcc -pthread -shared -L/home/ryan/.pyenv/versions/2.7.3/lib build/temp.linux-x86_64-2.7/src/xmlsec/tree.o -lxmlsec1 -lssl -lxslt -lxml2 -lcrypto -lxmlsec1-openssl -o build/lib.linux-x86_64-2.7/xmlsec/tree.so
    cythoning src/xmlsec/key.pyx to src/xmlsec/key.c
    building 'xmlsec.key' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_NO_GOST=1 -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_SIZE_T= -DXMLSEC_CRYPTO=openssl -DXMLSEC_OPENSSL_100=1 -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml/includes -Isrc -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/home/ryan/.pyenv/versions/2.7.3/include/python2.7 -c src/xmlsec/key.c -o build/temp.linux-x86_64-2.7/src/xmlsec/key.o -DXMLSEC_CRYPTO_OPENSSL=1
    gcc -pthread -shared -L/home/ryan/.pyenv/versions/2.7.3/lib build/temp.linux-x86_64-2.7/src/xmlsec/key.o -lxmlsec1 -lssl -lxslt -lxml2 -lcrypto -lxmlsec1-openssl -o build/lib.linux-x86_64-2.7/xmlsec/key.so
    cythoning src/xmlsec/ds.pyx to src/xmlsec/ds.c
    building 'xmlsec.ds' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_NO_GOST=1 -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_SIZE_T= -DXMLSEC_CRYPTO=openssl -DXMLSEC_OPENSSL_100=1 -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml/includes -Isrc -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/home/ryan/.pyenv/versions/2.7.3/include/python2.7 -c src/xmlsec/ds.c -o build/temp.linux-x86_64-2.7/src/xmlsec/ds.o -DXMLSEC_CRYPTO_OPENSSL=1
    gcc -pthread -shared -L/home/ryan/.pyenv/versions/2.7.3/lib build/temp.linux-x86_64-2.7/src/xmlsec/ds.o -lxmlsec1 -lssl -lxslt -lxml2 -lcrypto -lxmlsec1-openssl -o build/lib.linux-x86_64-2.7/xmlsec/ds.so
    cythoning src/xmlsec/template.pyx to src/xmlsec/template.c
    building 'xmlsec.template' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_NO_GOST=1 -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_SIZE_T= -DXMLSEC_CRYPTO=openssl -DXMLSEC_OPENSSL_100=1 -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml -I/home/ryan/.pyenv/versions/scratch-2.7.3/lib/python2.7/site-packages/lxml/includes -Isrc -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/home/ryan/.pyenv/versions/2.7.3/include/python2.7 -c src/xmlsec/template.c -o build/temp.linux-x86_64-2.7/src/xmlsec/template.o -DXMLSEC_CRYPTO_OPENSSL=1
    gcc -pthread -shared -L/home/ryan/.pyenv/versions/2.7.3/lib build/temp.linux-x86_64-2.7/src/xmlsec/template.o -lxmlsec1 -lssl -lxslt -lxml2 -lcrypto -lxmlsec1-openssl -o build/lib.linux-x86_64-2.7/xmlsec/template.so

Successfully installed xmlsec
Cleaning up...

[mehcode]

Can you paste the complete log for pip install xmlsec? I may be able to glean something.

[yaroslav-ilin]

On a fresh VPS installed all apt-get dependencies just fine, Python version is same, pkg-config is available, had no troubles with pip install lxml.

And here is a full log of pip install xmlsec:

Downloading/unpacking xmlsec
  Downloading xmlsec-0.2.0.tar.gz
  Running setup.py egg_info for package xmlsec
    no previously-included directories found matching 'doc/.build'

    Installed /tmp/easy_install-qFMhy3/pkgconfig-1.1.0/nose-1.3.4-py2.7.egg
    zip_safe flag not set; analyzing archive contents...

    Installed /root/build/xmlsec/pkgconfig-1.1.0-py2.7.egg
    Searching for setuptools-cython
    Reading http://pypi.python.org/simple/setuptools_cython/
    Best match: setuptools-cython 0.2.1
    Downloading https://pypi.python.org/packages/source/s/setuptools_cython/setuptools_cython-0.2.1.tar.gz#md5=c29fc5bf2894e69cbd98e3800cef4707
    Processing setuptools_cython-0.2.1.tar.gz
    Running setuptools_cython-0.2.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-N38FqT/setuptools_cython-0.2.1/egg-dist-tmp-I2UqBd
    zip_safe flag not set; analyzing archive contents...

    Installed /root/build/xmlsec/setuptools_cython-0.2.1-py2.7.egg
    Searching for Cython
    Reading http://pypi.python.org/simple/Cython/
    Best match: Cython 0.21.1
    Downloading https://pypi.python.org/packages/source/C/Cython/Cython-0.21.1.zip#md5=3e8ed7f5d49cb931105d3c99263be060
    Processing Cython-0.21.1.zip
    Running Cython-0.21.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-DLRENX/Cython-0.21.1/egg-dist-tmp-lfA5T6
    warning: no files found matching '.gitrev'
    warning: no files found matching '*.pyx' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.h' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Utility'
    Compiling module Cython.Plex.Actions ...

    Installed /root/build/xmlsec/Cython-0.21.1-py2.7-linux-x86_64.egg

Requirement already satisfied (use --upgrade to upgrade): lxml>=3.0 in /usr/local/lib/python2.7/dist-packages (from xmlsec)
Installing collected packages: xmlsec
  Running setup.py install for xmlsec
    building 'xmlsec.constants' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_OPENSSL_098=1 -DXMLSEC_NO_GOST=1 -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_SIZE_T= -DXMLSEC_CRYPTO=openssl -I/usr/local/lib/python2.7/dist-packages/lxml -I/usr/local/lib/python2.7/dist-packages/lxml/includes -Isrc -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/usr/include/python2.7 -c src/xmlsec/constants.c -o build/temp.linux-x86_64-2.7/src/xmlsec/constants.o -DXMLSEC_CRYPTO_OPENSSL=1
    gcc: error: src/xmlsec/constants.c: No such file or directory
    gcc: fatal error: no input files
    compilation terminated.
    error: command 'gcc' failed with exit status 4
    Complete output from command /usr/bin/python -c "import setuptools;__file__='/root/build/xmlsec/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-n6X5kG-record/install-record.txt:
    running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/xmlsec

copying src/xmlsec/__init__.py -> build/lib.linux-x86_64-2.7/xmlsec

copying src/xmlsec/meta.py -> build/lib.linux-x86_64-2.7/xmlsec

running build_ext

building 'xmlsec.constants' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/src

creating build/temp.linux-x86_64-2.7/src/xmlsec

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_OPENSSL_098=1 -DXMLSEC_NO_GOST=1 -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_SIZE_T= -DXMLSEC_CRYPTO=openssl -I/usr/local/lib/python2.7/dist-packages/lxml -I/usr/local/lib/python2.7/dist-packages/lxml/includes -Isrc -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/usr/include/python2.7 -c src/xmlsec/constants.c -o build/temp.linux-x86_64-2.7/src/xmlsec/constants.o -DXMLSEC_CRYPTO_OPENSSL=1

gcc: error: src/xmlsec/constants.c: No such file or directory

gcc: fatal error: no input files

compilation terminated.

error: command 'gcc' failed with exit status 4

----------------------------------------
Command /usr/bin/python -c "import setuptools;__file__='/root/build/xmlsec/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-n6X5kG-record/install-record.txt failed with error code 1 in /root/build/xmlsec
Storing complete log in /root/.pip/pip.log

[russian-developer]

nilfalse, +1

[russian-developer]

solved.
you install setuptools from apt-get / yum / etc ? You use very very very old version.
update your setuptool (e.g pip install setuptool --upgrade )

[mehcode]

@unk2k That makes sense. Thank you for commenting.

@nilfalse ^

[yaroslav-ilin]

Thank you guys for reply, that helped.

Unfortunately I already solved my task another way (not using python-xmlsec).

[pitbulk]

I also experience this issue on Ubuntu 13.10, using a fresh virtualenv and python 2.7

Installed /tmp/easy_install-gPYx1N/xmlsec-0.2.0/setuptools_cython-0.2.1-py2.7.egg
Searching for Cython
Reading http://pypi.python.org/simple/Cython/
Best match: cython 0.22
Downloading https://pypi.python.org/packages/source/C/Cython/cython-0.22.tar.gz#md5=1ae25add4ef7b63ee9b4af697300d6b6
Processing cython-0.22.tar.gz
Running Cython-0.22/setup.py -q bdist_egg --dist-dir /tmp/easy_install-gPYx1N/xmlsec-0.2.0/temp/easy_install-sCF8HY/Cython-0.22/egg-dist-tmp-dGqj4r
Unable to find pgen, not compiling formal grammar.
Compiling module Cython.Plex.Scanners ...
Compiling module Cython.Plex.Actions ...
Compiling module Cython.Compiler.Lexicon ...
Compiling module Cython.Compiler.Scanning ...
Compiling module Cython.Compiler.Parsing ...
Compiling module Cython.Compiler.Visitor ...
Compiling module Cython.Compiler.FlowControl ...
Compiling module Cython.Compiler.Code ...
Compiling module Cython.Tempita._tempita ...
warning: no files found matching '*.pyx' under directory 'Cython/Debugger/Tests'
warning: no files found matching '*.pxd' under directory 'Cython/Debugger/Tests'
warning: no files found matching '*.h' under directory 'Cython/Debugger/Tests'
warning: no files found matching '*.pxd' under directory 'Cython/Utility'

Installed /tmp/easy_install-gPYx1N/xmlsec-0.2.0/Cython-0.22-py2.7-linux-x86_64.egg
x86_64-linux-gnu-gcc: error: src/xmlsec/constants.c: No existe el archivo o el directorio
x86_64-linux-gnu-gcc: fatal error: no input files
compilation terminated.
error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 4

[mehcode]

@pitbulk
Can you try to upgrade setuptools in your fresh virtualenv?

pip install setuptools --upgrade