CMake support

[dzenanz]

CMake support will be added in the future.

This was written more than 2 years ago. The future is here, where is CMake support? 😄

Seriously, is there some plan or a timeline?

[jbms]

No one specifically asked for it previously so it wasn't a priority.

But it sounds like there is some interest now so we will try to prioritize.

Probably we will need to create a system to auto-convert the Bazel build rules into CMake rules, which should be fairly straightforward. The hardest part will be dealing with getting all of the dependencies available through cmake as well.

[laramiel]

Can you tell me more about why cmake support is useful to you? It's a pretty large step to take given the current dependency set.

[dzenanz]

I want to add Zarr support to https://github.com/InsightSoftwareConsortium/ITK. ITK's build system is CMake-based, so a library needs to be CMake-ified in order for us to use it. I already got started, relying on xtensor-zarr which has CMake-based build system. But it has out-of-date documentation, and many other outstanding issues.

[blasscoc]

Thanks for contributing the tensorstore!

My use cases for cmake are one, I want to be able to just compile tensorstore and use it as a library with other projects that are already using cmake. And a far second, I haven't been able to get the intellisense and debuggers working with Bazel in VSCode. I haven't used Bazel before so it could be these things are easy to do.

You right though, supporting a whole new build system does seem like a pretty big lift. Is there anyway I could help out if you decided to do that?

[2bndy5]

Maybe this repo could serve as a starting point.

[laramiel]

Sure, I'm aware of @haberman's repo there. I have actually started doing something based on very similar concepts, so a preliminary, incomplete CMake framework should be visible whenever we do the next export. The main issues that I've run into so far are:

• tensorstore has a lot of dependencies that are not CMake visible.
• CMake is really not a good language.

So if you would like to help this happen, look at the dependencies under third_party and see which ones don't have CMake support; that's what has to happen next.

[2bndy5]

CMake is really not a good language

This seems biased, so I'll disregard. I do sympathize with your frustration in porting to a different build system though. That said, I know CMake (syntax seems similar to Lua which is not ideal for OOP) far better than I understand how to use Bazel (which is not at all). I think an argument could be made about python's execution speed (used in Bazel) vs CMake's native execution speed, but this is getting off-topic...

tensorstore has a lot of dependencies that are not CMake visible

My experience with CMake is that their find_library() is quite versatile. It can be made to look in certain locations (for instance if the lib was installed in non-default locations - other than /usr/local/lib).

Some scenarios may be satisfied by CMake-provided "Find Modules" shipped with CMake (be mindful of the CMake version specified). Typically, if acquiring a dependency is not satisfied by these builtin modules, then it is pretty common for software devs to write their own "Find Module" (see the tutorial here). I understand how this convention is undesirable here since you prefer to only support the Bazel system, but if the find module(s) is written well, then it could be a set-and-forget tactic.

---

I don't really have a use case for this lib; I landed here out of curiosity via reference from @jbms. I just thought I could pass along what I've learned from working with CMake/pybind11 in the past. Please don't take offense if I don't volunteer contributions. I'm staying subscribed to this issue so my comments won't be "drive-by" observations.

[laramiel]

Yes, my opinion that cmake is not a good language is totally subjective. But that's my experience so far. And a lot of other folks share it: https://news.ycombinator.com/item?id=25701041 But it has become a key integration point, however I view it.

I've been trying to use the newer functionality of FetchContent and friends, which also maintains a somewhat hermetic build when coupled with our existing bazel dependency downloading, but the sticking point right now is dependencies that are not exposed via cmake. Thanks for the recommendations, I may need to use them at some point.

[2bndy5]

Sorry, I didn't realize that Bazel was downloading deps. That seems like an alternative to using git submodules, but it is better for pip installing a C-ext from sdist. Thanks for that FetchContent lead.

[laramiel]

I've added preliminary CMake generation; it's not ready yet, but anyone is free to play with it and maybe update it:

  cd /path/to/tensorstore
  python3 CMake/bazel_to_cmake.py
  mkdir build/
  cd build
  cmake ..

NOTE: This will not yet build tensorstore.

[dzenanz]

I am starting a vacation + conference trip tomorrow, so I will not be able to devote significant effort to this until July.

How important is requirement of C++17 standard? In ITK, we are just about to release a version after switching from C++11 to C++14.

Trying cmake configure yields:

Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044.
The CXX compiler identification is MSVC 19.29.30142.1
Detecting CXX compiler ABI info
Detecting CXX compiler ABI info - done
Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
Detecting CXX compile features
Detecting CXX compile features - done
Looking for C++ include pthread.h
Looking for C++ include pthread.h - not found
Found Threads: TRUE  
Could NOT find GTest (missing: GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY) 
The C compiler identification is MSVC 19.29.30142.1
Detecting C compiler ABI info
Detecting C compiler ABI info - done
Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
Detecting C compile features
Detecting C compile features - done
Found Python: C:/Program Files/Python39/python.exe (found version "3.9.6") found components: Interpreter 
pybind11 v2.8.1 
Found PythonInterp: C:/Program Files/Python39/python.exe (found version "3.9.6") 
Found PythonLibs: C:/Program Files/Python39/libs/python39.lib
Performing Test HAS_MSVC_GL_LTCG
Performing Test HAS_MSVC_GL_LTCG - Success
Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.35.2.windows.1") 
git version: v0.0.0 normalized to 0.0.0
Version: 1.6.0
Performing Test HAVE_CXX_FLAG_EHS_
Performing Test HAVE_CXX_FLAG_EHS_ - Success
Performing Test HAVE_CXX_FLAG_EHA_
Performing Test HAVE_CXX_FLAG_EHA_ - Success
Performing Test HAVE_STD_REGEX
Performing Test HAVE_STD_REGEX
Performing Test HAVE_STD_REGEX -- success
Performing Test HAVE_GNU_POSIX_REGEX
Performing Test HAVE_GNU_POSIX_REGEX
Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
Performing Test HAVE_POSIX_REGEX
Performing Test HAVE_POSIX_REGEX
Performing Test HAVE_POSIX_REGEX -- failed to compile
CMake Warning at C:/Misc/tensorstore-vs19/_deps/com_google_benchmark-src/CMakeLists.txt:287 (message):
  Using std::regex with exceptions disabled is not fully supported


Performing Test HAVE_STEADY_CLOCK
Performing Test HAVE_STEADY_CLOCK
Performing Test HAVE_STEADY_CLOCK -- success
The ASM_NASM compiler identification is unknown
Didn't find assembler
CMake Error at C:/Misc/tensorstore-vs19/_deps/com_google_boringssl-src/CMakeLists.txt:115 (enable_language):
  No CMAKE_ASM_NASM_COMPILER could be found.



Configuring incomplete, errors occurred!
See also "C:/Misc/tensorstore-vs19/CMakeFiles/CMakeOutput.log".
See also "C:/Misc/tensorstore-vs19/CMakeFiles/CMakeError.log".

Notably:

The ASM_NASM compiler identification is unknown
Didn't find assembler
CMake Error at C:/Misc/tensorstore-vs19/_deps/com_google_boringssl-src/CMakeLists.txt:115 (enable_language):
  No CMAKE_ASM_NASM_COMPILER could be found.

This means I can't build it on my main machine yet.

[jbms]

C++17 is definitely required.

I believe the cmake support is still a work in progress and not yet expected to actually work.

[dzenanz]

C++17 is definitely required.

One more reason for us to move to C++17 sooner, rather than later. As we plan for Zarr/NGFF IO implementation to live in a repository separate from the main one, we can implicitly require C++17 only if Zarr/NGFF is enabled.

[dzenanz]

I remember having trouble building BoringSSL a few years ago as part of GRPC. I eventually got it, but it wasn't quite so boring 😄

[laramiel]

Echoing jbms@, this does not yet build tensorstore, particularly since some of the dependencies don't yet work with CMake.

Also, the CMake build will require NASM, among other things, and it will not install NASM.