Skip to content

ci(docs-infra): fix test_docs_examples_ivy job - #30593

Closed
gkalpak wants to merge 1 commit into
angular:masterfrom
gkalpak:ci-aio-docs-examples-ivy
Closed

gkalpak wants to merge 1 commit into
angular:masterfrom
gkalpak:ci-aio-docs-examples-ivy

Conversation

@gkalpak

@gkalpak gkalpak commented May 21, 2019 •

Copy link
Copy Markdown
Member

Context:
As part of the test_docs_examples_ivy job, we run 5 concurrent builds on each VM (each for a different example/project). Additionally, all example projects share the same node_modules/ (via a symlink to aio/tools/examples/shared/node_modules/), so all concurrent builds operate on the same files.

Previously, we pre-ran ngcc with --properties module to process the fesm5 bundles. Since we have switched to es2015 in 661a57d, we now need the fesm2015 bundles. As a result, the initial ngcc run is redundant and ngcc runs again during each build (to process the fesm2015 bundles). Since there are 5 concurrent builds, we often end up with multiple ngcc instances processing the same package and trying to write to the same directories at the same time, causing a file already exists error.

This commit fixes it by pre-processing the fesm2015 bundles, so there is no need to re-run ngcc during each concurrent build.

Fixes #30577.

@gkalpak gkalpak added action: merge The PR is ready for merge by the caretaker area: build & ci Related the build and CI infrastructure of the project comp: docs-infra target: patch This PR is targeted for the next patch release labels May 21, 2019
@gkalpak
gkalpak requested a review from a team May 21, 2019 15:14
@ngbot ngbot Bot added this to the needsTriage milestone May 21, 2019
@gkalpak
gkalpak force-pushed the ci-aio-docs-examples-ivy branch from 8f49a84 to 2662dae Compare May 21, 2019 15:17
@mary-poppins

Copy link
Copy Markdown

You can preview 8f49a84 at https://pr30593-8f49a84.ngbuilds.io/.

@mary-poppins

Copy link
Copy Markdown

You can preview 2662dae at https://pr30593-2662dae.ngbuilds.io/.

Context:
As part of the `test_docs_examples_ivy` job, we run 5 concurrent builds
on each VM (each for a different example/project). Additionally, all
example projects share the same `node_modules/` (via a symlink to
`aio/tools/examples/shared/node_modules/`), so all concurrent builds
operate on the same files.

Previously, we pre-ran ngcc with `--properties module` to process the
fesm5 bundles. Since we have switched to es2015 in 661a57d, we now
need the esm2015 bundles. As a result, the initial ngcc run is
redundant and ngcc runs again during each build (to process the fesm2015
bundles). Since there are 5 concurrent builds, we often end up with
multiple ngcc instances processing the same package and trying to write
to the same directories at the same time, causing a
`file already exists` error

This commit fixes it by pre-processing the esm2015 bundles, so there is
no need to re-run ngcc during each concurrent build.

Fixes angular#30577
@gkalpak
gkalpak force-pushed the ci-aio-docs-examples-ivy branch from 2662dae to c9d1895 Compare May 21, 2019 18:56
@gkalpak

gkalpak commented May 21, 2019

Copy link
Copy Markdown
Member Author

FYI: Changed --properties fesm2015 to --properties es2015 to avoid an extra no-op ngcc pass (it shouldn't make any difference in behavior).

@mary-poppins

Copy link
Copy Markdown

You can preview c9d1895 at https://pr30593-c9d1895.ngbuilds.io/.

@benlesh

benlesh commented May 21, 2019

Copy link
Copy Markdown
Contributor

This is blocking another release blocker here: #30547

@jasonaden jasonaden closed this in ebfbc04 May 21, 2019
jasonaden pushed a commit that referenced this pull request May 21, 2019
Context:
As part of the `test_docs_examples_ivy` job, we run 5 concurrent builds
on each VM (each for a different example/project). Additionally, all
example projects share the same `node_modules/` (via a symlink to
`aio/tools/examples/shared/node_modules/`), so all concurrent builds
operate on the same files.

Previously, we pre-ran ngcc with `--properties module` to process the
fesm5 bundles. Since we have switched to es2015 in 661a57d, we now
need the esm2015 bundles. As a result, the initial ngcc run is
redundant and ngcc runs again during each build (to process the fesm2015
bundles). Since there are 5 concurrent builds, we often end up with
multiple ngcc instances processing the same package and trying to write
to the same directories at the same time, causing a
`file already exists` error

This commit fixes it by pre-processing the esm2015 bundles, so there is
no need to re-run ngcc during each concurrent build.

Fixes #30577

PR Close #30593
@kara

kara commented May 21, 2019

Copy link
Copy Markdown
Contributor

@benlesh Your CI error looks different than what this PR is fixing

@angular-automatic-lock-bot

Copy link
Copy Markdown

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot Bot locked and limited conversation to collaborators Sep 15, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

action: merge The PR is ready for merge by the caretaker area: build & ci Related the build and CI infrastructure of the project cla: yes target: patch This PR is targeted for the next patch release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NGCC flake in AIO docs examples tests

6 participants