Skip to content

Add instructions on fixing fuzz build - #953

Merged
Martin1887 merged 1 commit into
pulldown-cmark:masterfrom
kdarkhan:fuzzer-docs
Sep 9, 2024
Merged

Martin1887 merged 1 commit into
pulldown-cmark:masterfrom
kdarkhan:fuzzer-docs

Conversation

@kdarkhan

@kdarkhan kdarkhan commented Sep 8, 2024 •

Copy link
Copy Markdown
Contributor

When I was testing #948,
I built it directly using cargo build from within fuzz crate which worked as expected.
However, when using fuzzers with cargo fuzz, it overrides a lot of compilation parameters and that fails the build.

: rust-lld: error: undefined symbol: __sancov_gen_.1094
          >>> referenced by parse.3be71763e9de75d0-cgu.0
          >>>               /home/user/projects/pulldown-cmark/target/x86_64-unknown-linux-gnu/release/deps/parse-4bac226fcf249aac.parse.3be71763e9de75d0-cgu.0.rcgu.o:(asan.module_dtor.1168)

I spent some time investigating the issue and the easiest fix seems to be overriding lto config in release profile which could be done using environment variables:

CARGO_PROFILE_RELEASE_LTO=thin cargo fuzz build

This PR adds documentation on the issue and how to work around it.

@Martin1887
Martin1887 merged commit 64a38ec into pulldown-cmark:master Sep 9, 2024
@Martin1887

Copy link
Copy Markdown
Collaborator

Thanks!

@ollpu

ollpu commented Sep 9, 2024 •

Copy link
Copy Markdown
Collaborator

I wonder if this is cause to revert #948 - cargo fuzz just doesn't work if you've compiled something else before?

Are there other benefits to a shared workspace considering everything has to be rebuilt under different flags anyway?

@Martin1887

Copy link
Copy Markdown
Collaborator

Good point... especially when fuzzers are a different monster.

@kdarkhan

kdarkhan commented Sep 9, 2024 •

Copy link
Copy Markdown
Contributor Author

I wonder if this is cause to revert #948 - cargo fuzz just doesn't work if you've compiled something else before?

rustc is smart enough to recompile everything if you override lto config. Failure to build fuzz does not depend on previous builds done, it forces a new build. Reverting the whole PR is not worth it I think. The PR introduced 2 changes:

  1. Switches mozjs to upstream instead of the fork maintained by @notriddle
  2. Makes fuzz part of the workspace.

I just verified that before my PR fuzz crate was not buildable with default llvm deps in Debian but was buildable in Archlinux. Now, with the PR merged it builds correctly in both Debian/Archlinux. We can revert the workspace sharing changes but mozjs switch from 1 should still stay.

Are there other benefits to a shared workspace considering everything has to be rebuilt under different flags anyway?

For regular devs who don't use fuzz, nothing should change. Because fuzz is not listed in default-members, all commands not explicitly stating the --package will ignore fuzz.
For people who want to work on fuzzers, sharing the workspace has better editor support. I did face an issue with rust-analyzer with default settings, it would complain that fuzz files are not part of the workspace and diagnostics stops working. The fix was opening the editor from fuzz dir which adds some friction. Another benefit is sharing the build cache. I am planning to introduce build manifest caching to github action + building fuzz dir during PR build. Building fuzz from scratch is slow but if previous manifest are cached it should be much quicker. This will allow catching issues like #947 before merge. It is technically possible to do the same with separate fuzz workspace but will require setting up different caches for root and fuzz workspaces.

@ollpu

ollpu commented Sep 9, 2024

Copy link
Copy Markdown
Collaborator

mozjs switch from 1 should still stay.

Yes! I didn't think of that part.

Editor support is a good point. I had similar issues while working on dos-fuzzer.

I suppose then the current status is fine. Hopefully the issue can eventually be fixed in cargo-fuzz (or wherever in the stack).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants