[C++] Compilation error when linking arrow-flight-perf-server

[asfimport]

I wanted to play around with Flight benchmark after seeing the discussion regarding Flight's throughput in arrow dev mailing list today.

I met the following error when trying to build the benchmark from latest source code:

[ 95%] Linking CXX executable ../../../debug/arrow-flight-perf-server
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::filesystem::detail::canonical(boost::filesystem::path const&, boost::filesystem::path const&, boost::system::error_code*)'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::system::system_category()'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::filesystem::path::parent_path() const'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `deflate'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `deflateEnd'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::system::generic_category()'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::filesystem::detail::current_path(boost::system::error_code*)'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `inflateInit2_'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `inflate'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `deflateInit2_'
../../../debug/libarrow_flight.so.18.0.0: undefined reference to `inflateEnd'
../../../debug/libarrow_flight_testing.so.18.0.0: undefined reference to `boost::filesystem::path::operator/=(boost::filesystem::path const&)'
collect2: error: ld returned 1 exit status
src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/build.make:154: recipe for target 'debug/arrow-flight-perf-server' failed
make[2]: *** [debug/arrow-flight-perf-server] Error 1
CMakeFiles/Makefile2:2609: recipe for target 'src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all' failed
make[1]: *** [src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

I was using cmake .. -DCMAKE_BUILD_TYPE=Debug -DARROW_DEPENDENCY_SOURCE=AUTO -DARROW_FLIGHT=ON -DARROW_BUILD_BENCHMARKS=ON -DARROW_CXXFLAGS="-lboost_filesystem -lboost_system" to configure the build.
I noticed that there was a ARROW_BOOST_BUILD_VERSION: 1.71.0 in the output, but the Boost library that I installed from the package manger was of this version: 1.65.1.0ubuntu1. Could this be the cause of the problem?

PS:
I was able to build the benchmark before. It was on AWS with the OS being ubuntu-bionic-18.04-amd64-server-20191002, which should be very similar to the one I'm using on my laptop.

Environment: Linux HP 5.3.0-46-generic #38~18.04.1-Ubuntu SMP Tue Mar 31 04:17:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Reporter: Chengxin Ma / @MaChengxin
Assignee: Antoine Pitrou / @pitrou

PRs and other links:

• GitHub Pull Request #7031

_{Note: This issue was originally created as ARROW-8587. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
Weirdly, I get the same error now on Ubuntu 18.04. I used to be able to build it, so something broke along the way.

[asfimport]

Chengxin Ma / @MaChengxin:
Additional information: I still saw this error after rolling back the code base by: git checkout apache-arrow-0.17.0

[asfimport]

Antoine Pitrou / @pitrou:
I've bisected and the culprit is ARROW-7869.

[asfimport]

Antoine Pitrou / @pitrou:
By the way, you should build the benchmarks in release mode, not debug.

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 7031
#7031

[asfimport]

Chengxin Ma / @MaChengxin:
Thanks for the quick fix.

Unfortunately I still see the following error messages:

[ 96%] Linking CXX executable ../../../release/arrow-flight-perf-server
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflateInit2_'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflate'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflateInit2_'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflate'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflateEnd'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflateEnd'
collect2: error: ld returned 1 exit status
src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/build.make:156: recipe for target 'release/arrow-flight-perf-server' failed
make[2]: *** [release/arrow-flight-perf-server] Error 1
CMakeFiles/Makefile2:2648: recipe for target 'src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all' failed
make[1]: *** [src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

This seems to be a problem related to zlib. On my computer it is the latest version: zlib1g-dev is already the newest version (1:1.2.11.dfsg-0ubuntu2).

I guess this issue is still related to ThirdpartyToolchain.cmake?

[asfimport]

Antoine Pitrou / @pitrou:
I don't see that error myself. Can you try to pass -DARROW_WITH_ZSTD=on ?

[asfimport]

Chengxin Ma / @MaChengxin:
Adding -DARROW_WITH_ZLIB=ON solved this problem.

(I was expecting that the build system could find zlib on my system automatically so I didn't set this flag.)

[asfimport]

Wes McKinney / @wesm:
Is the zlib dependency coming from gRPC? It shouldn't be necessary to add ARROW_WITH_ZLIB=ON here (so this should be enabled automatically if it's needed)

[asfimport]

Antoine Pitrou / @pitrou:
I suppose so. I had no idea that gRPC required zlib (probably for optional compression, though we don't use it).

[asfimport]

Chengxin Ma / @MaChengxin:
I've built the Flight benchmark on different platforms before and never needed to use ARROW_WITH_ZLIB=ON. I have no idea why it became necessary this time.
I just reproduced the compilation error in a Singularity container. The container was built from this definition file.

Singularity> cd temp/arrow/cpp/
Singularity> mkdir release
Singularity> cd release/
Singularity> cmake .. -DCMAKE_BUILD_TYPE=Release -DARROW_DEPENDENCY_SOURCE=AUTO -DARROW_FLIGHT=ON -DARROW_BUILD_BENCHMARKS=ON
...
Singularity> make
...
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflateInit2_'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflate'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflateInit2_'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflate'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `deflateEnd'
../../../release/libarrow_flight.so.18.0.0: undefined reference to `inflateEnd'
collect2: error: ld returned 1 exit status
src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/build.make:156: recipe for target 'release/arrow-flight-perf-server' failed
make[2]: *** [release/arrow-flight-perf-server] Error 1
CMakeFiles/Makefile2:2648: recipe for target 'src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all' failed
make[1]: *** [src/arrow/flight/CMakeFiles/arrow-flight-perf-server.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

I cloned the latest apache/arrow repo (to temp/arrow) to ensure this problem is not caused by my accidental mess-up to my local arrow repo.
The latest commit of the temporary clone is:

Singularity> git log
commit 045fe1382349485e6eb2ce9466757efedf207c94 (HEAD -> master, origin/master, origin/HEAD)

[asfimport]

Antoine Pitrou / @pitrou:
@MaChengxin Do you still encounter this issue on git master?

[asfimport]

Chengxin Ma / @MaChengxin:
@pitrou The problem is gone.

I checked with HEAD pointing to 3ccecfb36.
 ```Java

Singularity> git rev-parse --short HEAD
3ccecfb