New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build: enable Clang-cl Windows builds #35433
base: main
Are you sure you want to change the base?
Conversation
| # else | ||
| *pCPU = IMAGE_FILE_MACHINE_UNKNOWN; | ||
| *pCPU = IMAGE_FILE_MACHINE_AMD64; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is for #34201
| #include <smmintrin.h> | ||
| #include <tmmintrin.h> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I opened https://chromium-review.googlesource.com/c/chromium/src/+/2281008 for this
node.gypi
Outdated
| # 'msvs_precompiled_header': 'tools/msvs/pch/node_pch.h', | ||
| # 'msvs_precompiled_source': 'tools/msvs/pch/node_pch.cc', | ||
| # 'sources': [ | ||
| # '<(_msvs_precompiled_header)', | ||
| # '<(_msvs_precompiled_source)', | ||
| # ], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get an error about non-portable includes if precompiled headers are enabled.
| @@ -2799,7 +2799,7 @@ def _GetMSBuildLocalProperties(msbuild_toolset): | |||
| if msbuild_toolset: | |||
| properties = [ | |||
| ['PropertyGroup', {'Label': 'Locals'}, | |||
| ['PlatformToolset', msbuild_toolset], | |||
| ['PlatformToolset', 'ClangCL'], | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This obviously has to come from configuration in the end
|
With all the above hacks, I am blocked on this error: Related warning in the build output: node/deps/v8/src/objects/objects.h Lines 815 to 833 in 4a6005c
node/deps/v8/src/objects/string.h Lines 821 to 836 in 4a6005c
|
|
/cc @nodejs/v8 |
|
Another tricky thing is that we can't even test this on github action due to space limit https://github.com/nodejs/node/pull/35433/checks?check_run_id=1189469828 |
|
Regarding the disk space issue: the D:\ drive on Windows VMs apparently has 14GB available - this is also described in the official docs. A workaround is to move things to the C:\ drive which has 80GB+ as described in actions/runner-images#1341 (comment) |
|
@targos Can you rebase this, so we can working on new V8 ? |
I investigate a little bit, looks like not an easy thing to do: actions/checkout#197. Also, below won't work either on windows CI on github action. Not sure why they default to drive D with such a limited disk space. |
|
Rebased on #35700 |
|
I don't know if CI has clang-cl. Let's try: https://ci.nodejs.org/job/node-test-commit-windows-fanned/38888/ Edit: well, it doesn't 😄 |
|
@nodejs/build-infra Would it be easy to install ClangCL build tools on the Windows machines? |
Not sure this will help: https://github.com/appveyor/build-images/blob/27bde614bc60d7ef7a8bc46182f4d7582fa11b56/scripts/Windows/install_vs2019.ps1#L192. |
|
So I tried to build it on my local machine. There's one warning that is very noisy (I think it's printed for every V8 source file: Then it fails with 3 errors (related warnings included): |
|
Looks like all related to inline function on windows platform. |
This comment has been minimized.
This comment has been minimized.
|
Now stucks with Maybe related: https://developercommunity.visualstudio.com/content/problem/1110835/clangcl-ltcg-is-passed-to-llvm-lib.html This occurred when I update my visual studio to latest. I also get some ideas to patch V8 to make the windows build works. |
|
Just for the record, Visual Studio 16.8 was released yesterday and has the following mentioned in its release notes:
Thought I'd mention it here as it might help for this PR. I have a Surface Pro X based on the arm64 architecture, let me know if you'd like me to test something 👍 |
|
Just tried to build using @gengjiawen Does it make any difference if you update to Visual Studio 16.8, which was released yesterday? |
I forget to remove '/P'. I should try my build now.
Have you tried the official arm64 build for windows. Will it run ? (I have no device to test it) |
Yes (see nodejs/build#2450 (comment)), and it's blazing fast 🚀 I use it on a daily basis and to work on things like an arm64 build for GitHub Desktop: desktop/desktop#9691 |
|
This is more likely a bug from what I test (refactor the inline implementation). Not sure this need to reported to clang team or msvc team. |
|
This warning appears thousands of times: |
|
The build ended up failing with "No space left on device" 😞 |
|
Hm, that's regrettable. Maybe it's possible to build on C:\ instead of D:? GitHub's machines have two disks/mounts. The second one has about 10-15 GB free space, the first one is about twice the size when you remove a couple of big packages; .NET SDK + Android SDK + GHC is around 30 GB. A full list is here: https://github.com/actions/runner-images/blob/main/images/win/Windows2019-Readme.md |
|
I was able to try it on my Windows PC. It errors here: BTW at this point the |
|
I think I found a way to fix precompiled headers. Same errors. |
|
Next step will be to port |
|
Rebased in case someone wants to help with this. |
|
The warnings regarding the c++17 flag have to do with the fact that the current configuration sets the C++ standard in this manner... So this is set unconditionally, irrespective of whether we are building C or C++ code. I think that a proper approach would set the language standards for each programming language, like so: "ClCompile": {
'LanguageStandard': 'stdcpp17',
'LanguageStandard_C': 'stdc17'
} |
|
@targos I will have a PR soon that should help. |
|
See targos#9 I think that this is enough to build all dependencies. It should be good enough. I have stopped short of fixing the v8 assembly error. When v8 builds itself under Windows, it calls |
1. To avoid many warnings, this PR declares the C and C++ standards separately. 2. This PR extends gyp so that we can build with AVX-512. Nevertheless, getting runtime dispatching with ClangCl through Visual Studio is challenging, so we disable it. It only affects base64 and one component of zip, so the effect on runtime performance should be negligible. Note that other dependencies such as simdutf do not need to this build support for runtime dispatching (so you still get AVX2, AVX-512 support in these dependencies).
|
The challenge regarding assembly when building with clangcl under Windows is related to a documented issue with gn. The assembly files in gn are assembled by MASM on Windows builds, but this causes difficulties with LLVM. However, clang (and clangcl) is happy to build inline assembly, so you don't need to mess around with assembly files at all. So the v8 workaround is this Python function... import argparse
import sys
def asm_to_inl_asm(in_filename, out_filename):
with open(in_filename, 'r') as infile, open(out_filename, 'wb') as outfile:
outfile.write(b'__asm__(\n')
for line in infile:
# Escape " in .S file before outputing it to inline asm file.
line = line.replace('"', '\\"')
outfile.write(b' "%s\\n"\n' % line.rstrip().encode('utf8'))
outfile.write(b');\n')
return 0... which just embeds the assembly code (.s) into a C++ (.cc) file. You'd only need to do this conditionally, when under Windows assuming that Windows builds move to clangcl. However, it should work everywhere.... Concretely, you want a file like... ... to be transformed into fake C++ (C++ embedding inline assembly)... and then you want to compile and assemble that and not the assembly file. It is simple enough and probably a matter of adding a few well placed lines of code. In the v8 gn file... we have In node, the snapshooting is somewhat different. The easiest route would be to find someone who is already familiar with how tools/snapshot works under Node. @joyeecheung @addaleax : any idea? |
|
At a glance, the target |

This is very hacky at the moment. I'm trying to find out what needs to be done to enable it and where.
I'm opening a PR to discuss the changes and seek some help, because it doesn't work yet!
To try it:
.\vcbuild.bat