xmldom: integrate new project #11036

karfau · 2023-09-28T19:34:29Z

No description provided.

google-cla · 2023-09-28T19:34:33Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

github-actions · 2023-09-28T19:36:02Z

karfau is integrating a new project:
- Main repo: https://github.com/xmldom/xmldom.git
- Criticality score: 0.65152

karfau · 2023-09-28T21:06:32Z

I'm quite new to the fuzzing topic, testing it locally right now, and maybe that's good enough, since for js there is only one kind of fuzzer possible. ONce I decided I will close or push to the PR.

DavidKorczynski · 2023-09-28T22:23:15Z

Could you list some of the more prominent users of xmldom and what makes it a security critical project?

karfau · 2023-09-29T05:48:14Z

Thx for asking, as far as I'm aware it's the most mature library for parsing xml in the NodeJs ecosystem.

https://github.com/aws/jsii -> https://deps.dev/npm/jsii-pacmak and https://deps.dev/npm/jsii-rosetta which are used as part of building the aws-cdk for differnt languages

Several libraries related to SAML/soap:

The most prominent library for dealing with plist files depends on it

https://deps.dev/npm/plist -> https://deps.dev/npm/plist/3.1.0/dependents

And some "apps"

some "dev tools"

Is that enough information?

karfau · 2023-09-29T09:58:16Z

And just to make sure I'm on the right track:
I created a fuzz target that already found some stuff, I fixed that locally and now it's running for quite a while without finding anything new.

That setup would be something I could configure to run as part of this project and it would run twice a day for a limited time and create a github issue in case it finds an issue, which is more likely because it uses a different seed each time but preserves the corpus of previous runs, right?

jonathanmetzman · 2023-09-29T19:28:27Z

And just to make sure I'm on the right track: I created a fuzz target that already found some stuff, I fixed that locally and now it's running for quite a while without finding anything new.

That setup would be something I could configure to run as part of this project and it would run twice a day for a limited time

It will run continuously on our infra. If you think that's wasteful you can manage a ClusterFuzzLite setup on github actions

and create a github issue in case it finds an issue, which is more likely because it uses a different seed each time but preserves the corpus of previous runs, right?

Yes. Note that github issues are really pointers to the issues we file in our bug tracker that has access control so not everyone can see the bug (this makes most sense for e.g. use after free bugs in C++)

jonathanmetzman · 2023-09-29T19:28:45Z

We accept this project. Let me know if you want me to merge this PR.

karfau · 2023-10-01T20:55:09Z

That is great news.

For now I put priority on fixing the issues found by the tool when running it locally.

I want to avoid being "flooded" by issues in the start, without being able to resolve them in a timely manner.

Afterwards I will find a way to provide the basic corpus for getting started that I'm currently using. (Is it common for people to add that to the docker image?)

So I marked the PR as a draft for now.

google/oss-fuzz#11036

tested most steps locally as far as I could, seems to work

karfau · 2023-10-09T05:47:20Z

I think the current configuration should already work as expected.
It is currently cloning a specific branch, since I'm struggling with the regression tests for the targets in the project repository.
I think I will split the regression test part into a separate PR/branch in order to unblock this PR and drop the branch from the clone, but I would already love to get some feedback on what I have done so far.
Thx

after moving them im the source repo

in order to integrate them as part of google/oss-fuzz#11036

since the PR as been merged in the source repo

karfau · 2023-10-11T05:47:09Z

The fuzz target regression test integration in the source repo is finished, so the Dockerfile no longer clones a branch.

@jonathanmetzman From my perspective this PR is ready, unless I receive some feedback.
Thx

karfau · 2023-10-11T11:01:55Z

The "libfuzzer, coverage" job looks like what I expected/anticipated, but there are multiple other jobs that are marked as required, which are failing.

I have no clue what I need to do regarding those. I had the impression that I followed all the steps from the docs/instructions

DavidKorczynski

Could you try and add the following to project.yaml:

fuzzing_engines:
- libfuzzer
sanitizers:
- none

I think missing this is what's causing the build issues.

karfau · 2023-10-11T15:36:24Z

Makes sense, thx for the hint, I think for some reason the init command didn't apply the javascript defaults, I guess I had a typo.

Done

DavidKorczynski · 2023-10-11T15:47:14Z

I think for some reason the init command didn't apply the javascript defaults

Did you use python3 infra/helper.py generate ... to create the initial set up? I can see it doesn't provide these default values -- however, it definitely should set the settings accurately since it's always required.

karfau · 2023-10-11T15:52:53Z

I'm pretty sure I did.
I was also surprised afterwards since the Dockerfile was not using the javascript base image after the command. Needed to figure out the details using the docs.

karfau · 2023-10-11T16:34:09Z

Is there some way to see those things running?
I would like to better understand what is happening with this config.

Update: Found it at https://oss-fuzz.com/

karfau · 2023-10-12T09:23:16Z

It looks like the build failed because it was not able to find the corpus directory that I added to the docker image.
I will need help to understand how to fix this.

Whe I tested the doker image locally I did run the targets inside the container, but it looks like this is not what is happening over there.

The xmltest directory which can not be found is what I extract the corpus to.

What is the proper way to provide a corpus in that setup?

https://oss-fuzz-build-logs.storage.googleapis.com/index.html#xmldom

Step #4 - "build-check-libfuzzer-none-x86_64": INFO: performing bad build checks for /tmp/not-out/tmp3icbd7qq/dom-parser.xml.target
Step #4 - "build-check-libfuzzer-none-x86_64": INFO: performing bad build checks for /tmp/not-out/tmp3icbd7qq/dom-parser.html.target
Step #4 - "build-check-libfuzzer-none-x86_64": Retrying failed fuzz targets sequentially 2
Step #4 - "build-check-libfuzzer-none-x86_64": INFO: performing bad build checks for /tmp/not-out/tmp3icbd7qq/dom-parser.xml.target
Step #4 - "build-check-libfuzzer-none-x86_64": INFO: performing bad build checks for /tmp/not-out/tmp3icbd7qq/dom-parser.html.target
Step #4 - "build-check-libfuzzer-none-x86_64": Broken fuzz targets 2
Step #4 - "build-check-libfuzzer-none-x86_64": ('/tmp/not-out/tmp3icbd7qq/dom-parser.xml.target', CompletedProcess(args=['bad_build_check', '/tmp/not-out/tmp3icbd7qq/dom-parser.xml.target'], returncode=1, stdout=b'BAD BUILD: /tmp/not-out/tmp3icbd7qq/dom-parser.xml.target seems to have either startup crash or exit:\n/tmp/not-out/tmp3icbd7qq/dom-parser.xml.target -rss_limit_mb=2560 -timeout=25 -seed=1337 -runs=4 < /dev/null\nINFO: New number of coverage counters 1024\nINFO: New number of coverage counters 2048\nINFO: using inputs from: xmltest\nERROR: The required directory "xmltest" does not exist\n', stderr=b''))
Step #4 - "build-check-libfuzzer-none-x86_64": BAD BUILD: /tmp/not-out/tmp3icbd7qq/dom-parser.xml.target seems to have either startup crash or exit:
Step #4 - "build-check-libfuzzer-none-x86_64": /tmp/not-out/tmp3icbd7qq/dom-parser.xml.target -rss_limit_mb=2560 -timeout=25 -seed=1337 -runs=4 < /dev/null
Step #4 - "build-check-libfuzzer-none-x86_64": INFO: New number of coverage counters 1024
Step #4 - "build-check-libfuzzer-none-x86_64": INFO: New number of coverage counters 2048
Step #4 - "build-check-libfuzzer-none-x86_64": INFO: using inputs from: xmltest
Step #4 - "build-check-libfuzzer-none-x86_64": ERROR: The required directory "xmltest" does not exist
Step #4 - "build-check-libfuzzer-none-x86_64": 
Step #4 - "build-check-libfuzzer-none-x86_64": ('/tmp/not-out/tmp3icbd7qq/dom-parser.html.target', CompletedProcess(args=['bad_build_check', '/tmp/not-out/tmp3icbd7qq/dom-parser.html.target'], returncode=1, stdout=b'BAD BUILD: /tmp/not-out/tmp3icbd7qq/dom-parser.html.target seems to have either startup crash or exit:\n/tmp/not-out/tmp3icbd7qq/dom-parser.html.target -rss_limit_mb=2560 -timeout=25 -seed=1337 -runs=4 < /dev/null\nINFO: New number of coverage counters 1024\nINFO: New number of coverage counters 2048\nINFO: using inputs from: xmltest\nERROR: The required directory "xmltest" does not exist\n', stderr=b''))
Step #4 - "build-check-libfuzzer-none-x86_64": BAD BUILD: /tmp/not-out/tmp3icbd7qq/dom-parser.html.target seems to have either startup crash or exit:
Step #4 - "build-check-libfuzzer-none-x86_64": /tmp/not-out/tmp3icbd7qq/dom-parser.html.target -rss_limit_mb=2560 -timeout=25 -seed=1337 -runs=4 < /dev/null
Step #4 - "build-check-libfuzzer-none-x86_64": INFO: New number of coverage counters 1024
Step #4 - "build-check-libfuzzer-none-x86_64": INFO: New number of coverage counters 2048
Step #4 - "build-check-libfuzzer-none-x86_64": INFO: using inputs from: xmltest
Step #4 - "build-check-libfuzzer-none-x86_64": ERROR: The required directory "xmltest" does not exist
Step #4 - "build-check-libfuzzer-none-x86_64": 
Step #4 - "build-check-libfuzzer-none-x86_64": ERROR: 100.0% of fuzz targets seem to be broken. See the list above for a detailed information.
Step #4 - "build-check-libfuzzer-none-x86_64": ********************************************************************************
Step #4 - "build-check-libfuzzer-none-x86_64": Build checks failed.
Step #4 - "build-check-libfuzzer-none-x86_64": To reproduce, run:
Step #4 - "build-check-libfuzzer-none-x86_64": python infra/helper.py build_image xmldom
Step #4 - "build-check-libfuzzer-none-x86_64": python infra/helper.py build_fuzzers --sanitizer none --engine libfuzzer --architecture x86_64 xmldom
Step #4 - "build-check-libfuzzer-none-x86_64": python infra/helper.py check_build --sanitizer none --engine libfuzzer --architecture x86_64 xmldom
Step #4 - "build-check-libfuzzer-none-x86_64": ********************************************************************************
Finished Step #4 - "build-check-libfuzzer-none-x86_64"
ERROR
ERROR: build step 4 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1

@karfau

Fixes: #11036 (comment) CC @karfau To test this from outside the containers: ``` python3 infra/helper.py build_fuzzers xmldom python3 infra/helper.py run_fuzzer xmldom dom-parser.xml.target python3 infra/helper.py run_fuzzer xmldom dom-parser.html.target ``` Signed-off-by: David Korczynski <[email protected]>

xmldom: accept new project

887538f

karfau marked this pull request as draft October 1, 2023 20:51

karfau added a commit to xmldom/xmldom that referenced this pull request Oct 9, 2023

test: add fuzzer.js and first fuzz target

016abe5

google/oss-fuzz#11036

karfau mentioned this pull request Oct 9, 2023

test: add fuzz target and regression tests xmldom/xmldom#556

Merged

feat: configure xmldom project

d093993

tested most steps locally as far as I could, seems to work

karfau marked this pull request as ready for review October 9, 2023 05:46

Merge branch 'master' into patch-1

2434b30

karfau added 2 commits October 9, 2023 17:11

fix: update path to fuzz targets

ecfcbd2

after moving them im the source repo

fix: update path to fuzz targets

a529367

after moving them im the source repo

karfau added a commit to xmldom/xmldom that referenced this pull request Oct 11, 2023

test: add fuzz target and regression tests (#556)

a5bbc19

in order to integrate them as part of google/oss-fuzz#11036

karfau added 2 commits October 11, 2023 07:44

chore: clone default branch

733d110

since the PR as been merged in the source repo

Merge remote-tracking branch 'upstream/master' into patch-1

10848cc

karfau changed the title ~~xmldom: accept new project~~ xmldom: integrate new project Oct 11, 2023

DavidKorczynski requested changes Oct 11, 2023

View reviewed changes

fix: add missing project config

d345da9

DavidKorczynski approved these changes Oct 11, 2023

View reviewed changes

DavidKorczynski merged commit 4cebc73 into google:master Oct 11, 2023
15 checks passed

karfau deleted the patch-1 branch October 11, 2023 15:50

DavidKorczynski mentioned this pull request Oct 11, 2023

Adjust template generation to language-specific attributes #11103

Open

DavidKorczynski mentioned this pull request Oct 12, 2023

xmldom: fix build by copying corpus to OUT #11108

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

xmldom: integrate new project #11036

xmldom: integrate new project #11036

karfau commented Sep 28, 2023

google-cla bot commented Sep 28, 2023

github-actions bot commented Sep 28, 2023

karfau commented Sep 28, 2023

DavidKorczynski commented Sep 28, 2023

karfau commented Sep 29, 2023

karfau commented Sep 29, 2023 •

edited

Loading

jonathanmetzman commented Sep 29, 2023

jonathanmetzman commented Sep 29, 2023

karfau commented Oct 1, 2023

karfau commented Oct 9, 2023

karfau commented Oct 11, 2023 •

edited

Loading

karfau commented Oct 11, 2023

DavidKorczynski left a comment •

edited

Loading

karfau commented Oct 11, 2023

DavidKorczynski commented Oct 11, 2023 •

edited

Loading

karfau commented Oct 11, 2023 •

edited

Loading

karfau commented Oct 11, 2023 •

edited

Loading

karfau commented Oct 12, 2023

xmldom: integrate new project #11036

xmldom: integrate new project #11036

Conversation

karfau commented Sep 28, 2023

google-cla bot commented Sep 28, 2023

github-actions bot commented Sep 28, 2023

karfau commented Sep 28, 2023

DavidKorczynski commented Sep 28, 2023

karfau commented Sep 29, 2023

karfau commented Sep 29, 2023 • edited Loading

jonathanmetzman commented Sep 29, 2023

jonathanmetzman commented Sep 29, 2023

karfau commented Oct 1, 2023

karfau commented Oct 9, 2023

karfau commented Oct 11, 2023 • edited Loading

karfau commented Oct 11, 2023

DavidKorczynski left a comment • edited Loading

Choose a reason for hiding this comment

karfau commented Oct 11, 2023

DavidKorczynski commented Oct 11, 2023 • edited Loading

karfau commented Oct 11, 2023 • edited Loading

karfau commented Oct 11, 2023 • edited Loading

karfau commented Oct 12, 2023

karfau commented Sep 29, 2023 •

edited

Loading

karfau commented Oct 11, 2023 •

edited

Loading

DavidKorczynski left a comment •

edited

Loading

DavidKorczynski commented Oct 11, 2023 •

edited

Loading

karfau commented Oct 11, 2023 •

edited

Loading

karfau commented Oct 11, 2023 •

edited

Loading