Adding CMake support

[HerrNamenlos123]

Please consult https://forum.openframeworks.cc/t/all-about-cmake/41777 for discussions

[ofTheo]

Maybe we can say instead "CMake support is a new addition to openFrameworks. There is an entire page dedicated to CMake support.

[HerrNamenlos123]

I pushed a change, better?

[ofTheo]

Great thanks!! 👍

[HerrNamenlos123]

Oh wow, apparently the macos unit tests fail. I am however pretty certain it has nothing to do with my changes, as only the networkTcp test failed on macos only and I did not do a single change in this file. I assume this is a spontaneous error.

[ofTheo]

oh yeah - those tests always fail occasionally ( the ports don't get released quick enough )

[danoli3]

This is looking great! Lets look at merging CMake support after the Android PR merge as that is also adding some CMake ability for Android, lets jut make sure its all compadible

[Nejrup]

Has anyone got this working on MacOS yet? CMake configuration passes for me, but can't build any of the examples..

[HerrNamenlos123]

@Nejrup what are the build errors?

[Nejrup]

@Nejrup what are the build errors?

first it complains about missing glut, if i workaround that it cant find utf8.h
Suspect it might be some linking error with the downloaded lib, but can't figure out what's going on

[HerrNamenlos123]

According to #7562 (comment), I will now migrate this branch to using find_package instead of linking to libs directly. This means for now, apothecary will not be used and CMake find modules must be provided by installing system libs.

Later, when apothecary still must be used, custom FindModules can be written that again redirect to local libs, but this is a necessary step so that the CMake dependency system is standard and future-proof, not home baked.

[danoli3]

Yeah find package not a good idea btw. It comes a nightmare as it links against binaries and architectures built for wrong targets and SDK's and even C++ levels leading to sometimes linking errors and or worse cash runtime crashes.

How's this going btw, we should merge this shiz and fix it up

[HerrNamenlos123]

Well I did not have a lot of time recently to put into OF, but I agree that we should finally decide on a solution. As I daid somewhere else implementation is not an issue, but we need to decide what to do.

I think using find_package is the right thing to do because you can simply write a few FindModules along with OF, which means it will find its own dependencies like before. But importantly, that allows experienced users to replace libraries against their own without changing OF, if they want to, and it makes the system more flexible. Also, it allows out-of-the-box Conan support.

But we finally need to decide how to handle dependencies. Do we REALLY want to keep the apothecary and just rewrite and rename it? Or do we want to decide on a package manager like Conan or vcpkg? Or do we want to ship archives including all binaries for all platforms and let CMake choose, given that the filesize is not too overwhelming?

Also, do we want to ship a dozen of archives, forcing an unknowing user to choose? Or do we simply ship a CMake distribution without binaries that will automatically download the correct apothecary binaries for the platform, when running CMake configure?

Until all of this is answered, there is no point in continuing CMake development...

[kritzikratzi]

i have the impression there are two questions here, and they seems to be mixed up a bit:

1. do we want cmake as internal build tool for OF, then it should for instance replace the apothecary but not the project generator.
2. do we want OF to be used as a cmake dependency. then i wouldn't touch the apothecary, and try to coexist with the project generator.

i think both are extremely tricky to do, and i also believe one is useful without the other and that such a transition can be taken one step at a time.

i think (1) is easier, and (2) is more useful. no idea where that leaves us :D

finally, a tiny bit of feedback on your code, which i have to admit i only skimmed in a hurry. please allow me to be direct where i disagree. first: rest assured that i'm amazed by all the work you did. second: i also know nothing, and this is just my opinion :)

i think you are going about it from the point of view of someone experienced in cpm+cmake, but i would go about it more from the point of view of people who already know OF. i'm not fond of cpm, because it's a universe in itself and so is OF. i see some interesting things, i'm not sure i see the synergy.

the biggest impact this has is that i would really opt against the philosophy of having a mandatory cmakelists.txt in each and every addon. for three reasons, actually:

a. for OF addons the directory structure IS the build system, so there is generally not a need for build instructions. you got the folders include,src,data,lib for the basics. and you can do libs/* which can be seen as a recursion on the whole thing. then there is addons.mk which is a list of dependencies (either named, or as local path), and addon_config.mk which can include/exclude paths dependening on the target platform and not much more. it's simple, but gets you very far.
b. there are a million tiny useful of plugins, it would be a shame to lose access to them or have to write a custom external cmake script to be able to use them
c. i believe we can do without this

i sortof have something like this in mind when i think about how OF can be used inside cmake:

set(OF_PLATFORM "" CACHE FILEPATH "Target platform passed to OF")
set(OF_CORE_PATH "dependencies/openframeworks")

# cmake_project.txt is a library to create of projects in cmake. 
# it immediately checks is OF_CORE_PATH is explicity set.
# if not it tries to guess.
include("dependencies/openframeworks/scripts/cmake_project.txt")

# of_add_target does little more than adding an executable and 
# setting up include paths. much of the work (like signing) has to be
# done at the end when it's know which plugins are in use
# PLATFORM would be optional, and fall back to macos on macos, windows on windows, etc. 
# the option needs to be passed manually for android, ios, etc.
of_add_target(my_cool_app PLATFORM ${OF_PLATFORM})

# of_add_addon will set up some include paths, copy files phases, link static and dynamic libs. 
# some things, like resolving dependencies, can not really be done here. 
of_add_addon(my_cool_app LOCAL dependencies/incredibly_strange_addon_location/MyOfxAddon)
# or... it is "somewhere", maybe a core addon
of_add_addon(my_cool_app NAME MyOfxAddon)
# or maybe even fancy
of_add_addon(my_cool_app GIT https://github.com/monkey/MyOfxAddon GIT_TAG V1.3-final)

# of_collect_target() performs two tasks: 
# 1. it is a simple dependency manager that walks the addons and their addons.make files, 
#    and complains if any dependency cannot be added. 
#    it also reports a list of all the dependencies that are actively in use
#    maybe parts of CPM can help here, not sure. 
# 2. it does the painful tasks nobody wants to talk about, like using install_name_tool, 
#    setting up the code signing, etc.
of_collect_target(my_cool_app)

this still let's you do things like use cpm and have it add include paths and libraries and whatnot to your target. but it should also manage to take of the really painful things like copying around files, linking to the correct binaries, doing the dance with the compiler flags, etc.

i really wouldn't use FindPackage() for things like glm, at least not just yet. in my experience OF is extremely solid once it's set up, and one of those reasons is that it brings it's own dependencies.

either way, you spent a monumental amount of time on this, and i will absolutely look more into what you did exactly.

[kritzikratzi]

ps. i've had to make like ten changes to my comment, and i think i can leave it as it is now. sorry about that.

[HerrNamenlos123]

@kritzikratzi thank you for your feedback. I haven't touched OF in quite some time, so I'm not sure where I left off last time.
To your point with CPM: I am actually not sure if it is actually still used. You are right that CPM is in itself large and we don't actually need it. I used it in the past because it feels nice to write a single line of cmake and have a dependency. However, in the past years I switched to using plain FetchContent, because with about 6 to 8 lines of FetchContent you can achieve the same thing.

So yes, we can get rid of CPM and I think it would be great to have a of_add_package command that wraps around FetchContent.

To your point with the addons: I see why you want to keep old addons compatible, I think I am not aware how many addons exist and what they have in common. I love the idea of being able to write of_add_addon(my_cool_app GIT https://github.com/monkey/MyOfxAddon GIT_TAG V1.3-final), and that's all that's needed to use it. I think it could work if the directory structure is really well defined and closely followed, although I have the dull feeling that this is not exactly true in the OF world... I am not sure if the advantages outweigh the disadvantages...

Anyways I agree that this change must be adopted step by step. This is why I haven't done anything in OF itself recently, but only the apothecary. Or not the apothecary because I am trying to replace it: https://github.com/openframeworks/packages

It was a lot of work to get everything to work, but currently all supported packages are automatically built from source in CI using a python build script, in debug and release mode and for all compilers and architectures. You only need to commit the build recipe and CI builds everything and automatically uploads the artifacts to the releases: https://github.com/openframeworks/packages/releases

As you can see, there are binaries for all packages, for all versions and compilers and architectures. You can download an archive and inspect it: Every archive contains headers and binaries for debug and release, and cmake modules to make them findable. I want to provide all packages like this, which is really really hard because most OF dependencies are archaic and tremendously hard to build. Packages that instantly build using cmake or another tool are trivial.

This is the first step. Then, when all packages are served this way, I want to adapt the OF CMake script to use FetchContent to download only the packages it actually needs on-demand and link them by using find_package and setting up the CMake module path so it only looks in the folders it downloaded. (That means it is in itself completely solid because it only searches the files it downloaded, which it will hopefully always find, but it allows you to instead use system packages, if you want). In the same way you can pre-download the same packages and distribute an archive if you want to build OF without internet connection. That is the second step.

Only then is the moment we can think about how to include addons, i think. IMO the decision to rebuild apothecary is inevitable, and the decision to use CMake to build OF in the future is also already done i think. If CMake is only used internally to build binaries, or by every OF user is still open for discussion.

So, i like your idea to include local addons but when we get there, we still need to discuss if it is worth it to not use a CMakeLists in the addon, because addons have dependencies. It might be simpler to write a 2-line CMakeLists rather than inventing a system for specifying dependencies without CMake.

So if someone wants to speed up development (because I currently don't have any time to spare for OF), can help me by looking at https://github.com/openframeworks/packages, finding OF dependencies that are easy to build, and writing a simple Python file to build it (just copy the packages and workflows folder for mbedtls for example). Only when all packages are available, does it make sense to continue CMake development.

[HerrNamenlos123]

It might be doable by not having a CMakeLists in the addon if the addon does not have dependencies (or only trivial ones) and to add a CMakeLists if the addon relies on other packages. I am trying to get rid of all Makefiles because they are very system dependent and do not help if you want to streamline development. The point of CMake over Make is that it works on all platforms including cross-compiling. Makefiles do not.

As such, keeping Makefiles to specify dependencies for backwards compatibility defies the purpose of using a cross-platform build system, and just the idea of using a textfile to list dependencies instead of a CMakeLists is also a case of working around the technology instead of effectively using it. Thus, it might be the better option to have the CMakeLists only optional if the addon has no dependencies. But that is to be discussed once the packages system is up and OF itself can build with clean dependencies and without CPM.