Skip to content

/ codeql-cli-binaries

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

On MacOS, libtrace.dylib can't load into platform binaries for build instrumentation during DB creation #26

Open
xpcmdshell opened this issue Apr 19, 2020 · 6 comments
Open

On MacOS, libtrace.dylib can't load into platform binaries for build instrumentation during DB creation #26

xpcmdshell opened this issue Apr 19, 2020 · 6 comments
Assignees
@matt-gretton-dann
Labels
CLI

Comments

@xpcmdshell
Copy link

@xpcmdshell xpcmdshell commented Apr 19, 2020

Description

During database creation for compiled languages, it looks like CodeQL tries to add some instrumentation to your compiler to get more info. This fails for "platform binaries" on OSX, which are protected from DYLD_INSERT_LIBRARIES as well astask_for_pid and friends. For example, below I'm attempting to create a database for a particular revision of WebKit:

╭─user@box ~/project/webkit_stuff/WebKit.git
╰─$ uname -a
Darwin box.local 19.4.0 Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64 x86_64
╭─user@box ~/project/webkit_stuff/WebKit.git
╰─$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.15.4
BuildVersion:	19E287
╭─user@box ~/project/webkit_stuff/WebKit.git
╰─$ codeql database create ~/project/codeql-home/webkit_regex_codeql --language=cpp --command="./Tools/Scripts/build-webkit --jsc-only --debug"
Initializing database at /Users/user/project/codeql-home/webkit_regex_codeql.
Running command [./Tools/Scripts/build-webkit, --jsc-only, --debug] in /Users/user/project/webkit_stuff/WebKit.git.
[2020-04-19 09:55:49] [build-err] error: /Users/user/Workspace/Xcode_10.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: changing install names or rpaths can't be redone for: /Users/user/project/codeql-home/webkit_regex_codeql/working/copy-root/000001F5/Users/user/Workspace/Xcode_10.1.app/Contents/Developer/usr/bin/xcodebuild.semmle.00001694 (for architecture x86_64) because larger updated load commands do not fit (the program must be relinked, and you may need to use -headerpad or -headerpad_max_install_names)
[2020-04-19 09:55:49] [build-err] dyld: warning: could not load inserted library '/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib' into hardened process because no suitable image found.  Did find:
[2020-04-19 09:55:49] [build-err] 	/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib: code signature in (/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib) not valid for use in process using Library Validation: mapping process is a platform binary, but mapped file is not
[2020-04-19 09:55:49] [build-err] 	/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib: stat() failed with errno=1
[2020-04-19 09:55:57] [build] +  cmake -DPORT="JSCOnly" -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_BUILD_TYPE=Debug -DSHOW_BINDINGS_GENERATION_PROGRESS=1 -DDEVELOPER_MODE=ON -DENABLE_EXPERIMENTAL_FEATURES=ON "/Users/user/project/webkit_stuff/WebKit.git"
[2020-04-19 09:55:57] [build] -- The C compiler identification is AppleClang 10.0.0.10001145
[2020-04-19 09:55:57] [build] -- The CXX compiler identification is AppleClang 10.0.0.10001145
[2020-04-19 09:55:57] [build] -- Check for working C compiler: /Users/user/Workspace/Xcode_10.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
[2020-04-19 09:55:58] [build] -- Check for working C compiler: /Users/user/Workspace/Xcode_10.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - works

[ SNIP - Compilation continues ]

[2020-04-19 10:07:24] [build] [100%] Linking CXX executable ../../../bin/jsc
[2020-04-19 10:07:24] [build] [100%] Built target jsc
[2020-04-19 10:09:21] [build] [100%] Linking CXX executable ../../../bin/testb3
[2020-04-19 10:09:22] [build] [100%] Built target testb3
[2020-04-19 10:09:22] [build] ====================================================================
[2020-04-19 10:09:22] [build]  WebKit is now built (13m:33s).
[2020-04-19 10:09:22] [build] ====================================================================
Finalizing database at /Users/user/project/codeql-home/webkit_regex_codeql.
Successfully created database at /Users/user/project/codeql-home/webkit_regex_codeql.

You can see above the lines about libtrace.dylib injection failing, specifically:

[2020-04-19 09:55:49] [build-err] dyld: warning: could not load inserted library '/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib' into hardened process because no suitable image found.  Did find:
[2020-04-19 09:55:49] [build-err] 	/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib: code signature in (/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib) not valid for use in process using Library Validation: mapping process is a platform binary, but mapped file is not
[2020-04-19 09:55:49] [build-err] 	/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib: stat() failed with errno=1

Disabling SIP doesn't appear to work around this, the outcome is the same. This is one example that under the hood probably uses things like xcodebuild and clang.

Database creation "finishes", but it's unclear how much information is actually missing. I actually don't know what all CodeQL tries to gather from the compilation process. How messed up will my database be if instrumentation fails during the build step?
@matt-gretton-dann matt-gretton-dann self-assigned this Apr 20, 2020
@matt-gretton-dann
Copy link

@matt-gretton-dann matt-gretton-dann commented Apr 20, 2020

Thank you for your issue report.

Looking at the report above I suggest the following steps:

  • Please ensure you are using the latest CodeQL CLI binaries.
  • Use an Xcode app with a shorter path. (This needs to be its realpath and not a symlink.

I have successfully done a CodeQL managed build using Xcode installed in /Applications/Xcode.app using latest WebKit.git master and CodeQL CLI Binaries version 2.1.0 with the command line you provided:

codeql database create /tmp/db --language=cpp --command="./Tools/Scripts/build-webkit --jsc-only --debug"

/tmp/db ends up being about 2.8GB in size.

If you have any further questions please feel free to ask.
@xpcmdshell
Copy link
Author

@xpcmdshell xpcmdshell commented Apr 20, 2020

Hi, thanks for checking out my issue. I'm still seeing the same errors about libptrace.dylib not being injected into platform binaries. Using Xcode from /Applications doesn't seem to help. What version of MacOS are you using to test this, out of curiosity?

My database after generation for latest commit of webkit is also about 2.8GB, so maybe there's nothing significant missing 🙂
@matt-gretton-dann
Copy link

@matt-gretton-dann matt-gretton-dann commented Apr 21, 2020

sw_vers reports:

ProductName:    Mac OS X
ProductVersion: 10.15.4
BuildVersion:   19E287

xcodebuild -version reports:

Xcode 11.1
Build version 11A1027

Another thing to try (along with using Xcode in /Applications) is to put the database in /tmp when creating it (the directory can be relocated afterwards).

If that still fails I would appreciate a copy of /log/build-tracer.log and /log/database-create-*.log. These are plain text files, so feel free to examine what they contain before you send them to ensure they have no secrets (via DropBox or equivalent is fine). These may help to diagnose what is happening.
@xpcmdshell
Copy link
Author

@xpcmdshell xpcmdshell commented Apr 21, 2020

Same thing with creating the database in /tmp/ 🤔

Here's a copy of my logs, let me know if there's anything else I can provide to be helpful:
https://drive.google.com/open?id=1Gm-lQIV0sNOCt863esjjm6Ayv9rxx2_5
@matt-gretton-dann
Copy link

@matt-gretton-dann matt-gretton-dann commented Apr 22, 2020

Thank you for these. I have taken local copies - so don't feel the need to preserve them in your Google Drive.

The good news is that from the database create log I think the vast majority of the build is being processed correctly - so the database should be usable.

Unfortunately I don't have a solution to the problem you've been having yet. I need to do some further investigations locally, and this will take some time.
@xpcmdshell
Copy link
Author

@xpcmdshell xpcmdshell commented Apr 22, 2020

Sounds good, thanks for taking a look into this
@adityasharad adityasharad added the CLI label Jul 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matt-gretton-dann matt-gretton-dann

Labels
CLI
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
@adityasharad @xpcmdshell @matt-gretton-dann
You can’t perform that action at this time.