Skip to content

[Test][Zeta] Drive FileCollectReader idle close and rediscovery with a manual clock - #12472

Open
goutamadwant wants to merge 1 commit into
apache:devfrom
goutamadwant:test/file-collect-reader-flake
Open

goutamadwant wants to merge 1 commit into
apache:devfrom
goutamadwant:test/file-collect-reader-flake

Conversation

@goutamadwant

Copy link
Copy Markdown
Collaborator

Purpose of this pull request

FileCollectReaderBehaviorTest.rediscoversFileAfterInactiveCursorClosed fails intermittently on Windows unit-test jobs with:

FileCollectReaderBehaviorTest.rediscoversFileAfterInactiveCursorClosed:101 » ConditionTimeout
Caused by: org.opentest4j.AssertionFailedError: expected: <1> but was: <0>

Why it fails. The test infers two internal transitions from wall-clock arithmetic with 20–30 ms of slack: the inactive-cursor close (close_inactive_ms = 500) and the glob rediscovery (glob_scan_interval_ms = 20). Inside one poll(), FileCollectReader runs discovery before it closes idle cursors, so rediscovery needs a later poll at least one scan interval after the closing poll.

A single stall of a few tens of milliseconds near the idle deadline pushes the closing poll late. The second Awaitility loop then exits before the next scan is due, and "second\n" is appended while the file is still untracked. With read_from_beginning = false and no stored position, the rediscovered file is opened at its end, so the new line is skipped. The final untilAsserted then times out with expected: <1> but was: <0>. A longer timeout cannot help, because the line is already behind the cursor.

The change.

  • FileCollectReader gets a package-private constructor that takes a millisecond clock (LongSupplier). It uses that clock for the glob scan interval, idle-cursor bookkeeping and line timestamps, and passes it to FileTailCursor.
  • FileTailCursor gets a matching constructor overload. It has to be public, because FileTailCursor lives in the cursor subpackage.
  • The existing constructors delegate with System::currentTimeMillis, so runtime behaviour does not change. The updatedAt value written to the position store still uses wall-clock time.
  • The test steps a manual clock past close_inactive_ms and then one glob_scan_interval_ms, so each transition happens in a known poll. All the original assertions on the emitted event are kept: exactly one event, which contains second and not first.
  • The test also asserts that the event's line metadata is 1. Line numbering only restarts if the cursor was really closed and the file rediscovered, so the test no longer passes when the idle close does nothing.

This overlaps #12381, which raises the same test's Awaitility budgets from 3 s to 10 s. In the failing runs, the time is lost before the append, not in the final wait, so the larger budget does not cover this case. If #12381 merges first, I will rebase.

Does this PR introduce any user-facing change?

No. The new constructors are for tests, and the default clock is System::currentTimeMillis, as before.

How was this patch tested?

Forced timing window on unchanged dev (deb16a3c3). A local-only hook, not in this PR, inserted a single 45 ms Thread.sleep in the first Awaitility loop, 35 ms before the idle deadline. Each run executed the single method repeatedly in one JVM through the JUnit Platform launcher:

Code JDK 8u172 JDK 11.0.19
Unchanged dev, with hook 15/20 failed 7/20 failed
This patch, stalls injected at the same points 0/20 failed 0/20 failed

On dev, every failure had the CI message (expected: <1> but was: <0> within 3 seconds). Without the hook, dev passed 50/50 per JDK on macOS under CPU contention. That fits a failure that needs a stall of about 20 ms or more at a specific moment, which Windows runners hit more often.

For the patched run, the injected stalls were 45 ms before the close step, 700 ms after the close, and 700 ms after rediscovery. The 700 ms stalls are longer than close_inactive_ms, which would re-close the cursor under a wall clock.

Stress after the patch. 200/200 passed on JDK 8 and 200/200 on JDK 11, with 20 busy-loop processes competing for CPU.

Mutation checks. The patched test fails as expected in both cases:

  • globResolver.forget(...) removed from closeInactiveCursors, so the file is never rediscovered.
  • closeInactiveCursors(now) removed from poll(), so the cursor is never closed.

Module builds. ./mvnw -B -o -pl seatunnel-edge-agent/seatunnel-edge-agent-connector,seatunnel-edge-agent/seatunnel-edge-agent-starter clean verify -DskipUT=false -DskipIT=true passed on JDK 8 and JDK 11, including spotless: connector 38 tests and starter 103 tests, with 0 failures.

Not run locally: Windows, and seatunnel-edge-agent-e2e (EdgeAgentFileToEngineIT). The E2E suite needs the Docker engine setup; it runs in this PR's edge-agent-it job.

Check list

…a manual clock

FileCollectReaderBehaviorTest.rediscoversFileAfterInactiveCursorClosed
inferred the inactive close and the glob rediscovery from wall-clock
arithmetic with 20-30 ms of slack. A single scheduler stall near the idle
deadline moves the closing poll late enough that rediscovery happens
only after "second" is appended. With read_from_beginning=false the
rediscovered file is opened at its end, the line is skipped, and the
final assertion times out with "expected: <1> but was: <0>" (seen on
Windows JDK 8 and JDK 11 unit-test jobs).

FileCollectReader and FileTailCursor now take an optional millisecond
clock (LongSupplier). The existing constructors keep using
System::currentTimeMillis, so runtime behaviour is unchanged. The test
steps the clock past close_inactive_ms and then one glob scan interval,
so each transition happens in a known poll. All original assertions on
the emitted event are kept, and the test now also checks that line
numbering restarted, which only happens if the cursor was really closed
and the file rediscovered.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant