Skip to content

Stop-Job intermittently failing in UNIX CI builds #7249

Description

@rjmholt

Stop-Job Feature test sometimes fails on UNIX platforms in CI

It "Stop-Job will stop a job" {
Stop-Job -Id $j.Id
$out = Receive-Job $j -ErrorVariable err
$out | Should -BeNullOrEmpty
$err | Should -BeNullOrEmpty
$j.State | Should -BeExactly "Stopped"
}

With the error

 Context jobs which take time
++[localhost] The background process closed or ended abnormally.
+ CategoryInfo : OpenError: (localhost:String) [], PSRemotingTransportException
+ FullyQualifiedErrorId : 2101,PSSessionStateBroken
 [-] Stop-Job will stop a job 647ms
 Expected $null or empty, but got {[localhost] The background process closed or ended abnormally.}.
 65: $err | Should -BeNullOrEmpty
 at Invoke-Assertion, /home/travis/build/PowerShell/PowerShell/src/powershell-unix/bin/Release/netcoreapp2.1/linux-x64/publish/Modules/Pester/4.2.0/Functions/Assertions/Should.ps1: line 206
 at <ScriptBlock>, /home/travis/build/PowerShell/PowerShell/test/powershell/Modules/Microsoft.PowerShell.Core/Job.Tests.ps1: line 65

Steps to reproduce

Haven't been able to work out how to reproduce this error, it has only been seen in CI on UNIX so far.

Expected behavior

Stop-Job should cause the job state to be set to Stopped and the test should pass.

Actual behavior

The job state comes back as Failed and the test fails. Given the error message and where it comes from in the code:

protected void OnExited(object sender, EventArgs e)
{
TransportMethodEnum transportMethod = TransportMethodEnum.Unknown;
lock (syncObject)
{
// There is no need to return when IsClosed==true here as in a legitimate case process exits
// after Close is called..In that legitimate case, Exit handler is removed before
// calling Exit..So, this Exit must have been called abnormally.
if (isClosed)
{
transportMethod = TransportMethodEnum.CloseShellOperationEx;
}
// dont let the writer write new data as the process is exited.
// Not assigning null to stdInWriter to fix the race condition between OnExited() and CloseAsync() methods.
//
stdInWriter.StopWriting();
}
PSRemotingTransportException psrte = new PSRemotingTransportException(PSRemotingErrorId.IPCServerProcessExited,
RemotingErrorIdStrings.IPCServerProcessExited);
RaiseErrorHandler(new TransportErrorOccuredEventArgs(psrte, transportMethod));
}

It looks like the job process itself is exiting prematurely for some reason.

Environment data

Occurring in master branch of PowerShell on Linux and macOS.

Proposed Solution

  • Add information to the error message about the process' exit code and stderr
  • Add syslog (on Linux) and os_log (on macOS) output to build artefacts in Travis
  • Add core dump files to build artefacts on Travis

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Issue-BugIssue has been identified as a bug in the productResolution-No ActivityIssue has had no activity for 6 months or moreWG-RemotingPSRP issues with any transport layer

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions