[libclc] Ensure directory exists for libclc files

[frasercrmck]

Some generators like Unix Makefiles wouldn't automatically ensure that this directory exists, which can break the build.

[hdelan]

LGTM. How was this working before?

[jchlanda]

LGTM. How was this working before?

I guess that could only be triggered if you manually remove the dir, if you go from the configure/compile scripts it would always be there.

[frasercrmck]

LGTM. How was this working before?

I'm glad you asked. I don't think this fix is the best. Let me retry it and that should explain it better.

[frasercrmck]

LGTM. How was this working before?

Aye okay so I found that despite my fix, there were "earlier" steps that also needed the same directory, but were expressed using a CMake variable (LIBCLC_LIBRARY_OUTPUT_INTDIR) so I missed them. Those targets also weren't getting their directories created. So it just so happened that when I built the remangled-* targets, it would try and fail to create the libclc dummy file before it had actually created the prepare-* targets, which also need the same directory. If I built the prepare-* targets they'd fail with the same issue.

I tried file(MAKE_DIRECTORY ${LIBCLC_LIBRARY_OUTPUT_INTDIR}) in the top-level CMakeLists.txt, but that only does it at configure time, and if that directory is deleted and another CMake file is touched then it doesn't reconfigure the top-level file and so the directory isn't recreated.

We could also use a custom command or target in the top-level CMakeLists but I found it wasn't running reliably. Perhaps the targets that need that directory would need that target/directory as a dependency.

In the end I decided to go with explicit COMMAND cmake -E make_directory ${LIBCLC_LIBRARY_OUTPUT_INTDIR} in every custom command that needs the directory. This is done elsewhere in LLVM and I think it's the most explicit (dependencies are harder to track, imo).

Basically I think this all worked before "the changes" was because we had set_output_directory(builtins.link... LIBRARY_DIR ${LIBCLC_LIBRARY_OUTPUT_INTDIR}) quite early on and that probably ensured that the directory was present in time for everything afterwards?

Please take another look!

[hdelan]

What is the dependency DAG like? I think it looks like

(Using -> for depends on)

dummy_target -> libclc_dummy_in.cc
remangled_target -> remangled.bc -> linked.bc -> opt.bc -> libspirv.bc

I might have got the order wrong but I think there are just two dependency chains. Correct me if I am wrong.

So using this DAG we should only need to create the dir if we are making the original libspirv.bc or the dummy.bc? The other targets in the second chain will automatically make libspirv.bc which will make the dir

[frasercrmck]

What is the dependency DAG like? I think it looks like

(Using -> for depends on)

dummy_target -> libclc_dummy_in.cc
remangled_target -> remangled.bc -> linked.bc -> opt.bc -> libspirv.bc

I might have got the order wrong but I think there are just two dependency chains. Correct me if I am wrong.

So using this DAG we should only need to create the dir if we are making the original libspirv.bc or the dummy.bc? The other targets in the second chain will automatically make libspirv.bc which will make the dir

I think that's accurate, yep.

There's also the external-calls-${obj_suffix} target which looks broken to me - I'll need to fix that, but I'm not sure anyone's running it.

There's also the aliasing targets, which depend on prepare-${target} so presumably do inherit the directory. Despite that, I've added an explicit make_directory to those as they're in a separate macro, and I think this way it's still nice and self-contained - we don't implicitly get our directory from another target from another macro/function.

[frasercrmck]

@intel/llvm-gatekeepers This looks good to merge, thank you.