Skip to content

Unit tests need to make sure they reset the state of singletons #103

Closed

Description

@josemduarte

I've come across this very subtle issue recently. I have found that running all tests in one bundle with "mvn install" can have a very different behaviour than running tests individually. Maven by default uses one single JVM to run the tests. Static variables like those used in singletons (e.g. AtomCache) are not necessarily reinitialise at the beginning of each test. Thus the tests have a different outcome when running in a bundle or running separately.

One workaround is always using this setting in the pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.16</version>
        <configuration>
            <reuseForks>false</reuseForks>
            <forkCount>1</forkCount>
        </configuration>
</plugin>

which makes sure that each test is run in a new process (see http://stackoverflow.com/questions/3365628/junit-tests-pass-in-eclipse-but-fail-in-maven-surefire)

For instance if one uses that, then 2 tests fail: TestBond and SiftsChainToUniprotMappingTest (easily reproducible by just running the tests individually).

For TestBond the issue seems to be a PDB parsing bug in entry 145D. Apparently when running tests in one bundle the flag setCreateAtomBonds was not set as expected and then the bug was not seen.

For SiftsChainToUniprotMappingTest the issue seems to be that the sifts file was not downloaded if run individually but it was if run in a bundle. Spencer proposed a fix for that, see this commit d6b64f7

Activity

  1. josemduarte commented on Mar 18, 2014

    @josemduarte
    ContributorAuthor

    Another weird behaviour was with TestAltLocs: in test3PIU(), the structure would sometimes be parsed from PDB file and sometimes from mmCIF (depending if run in bundle or individually). There seems to be a bug in mmCIF altLoc parsing (see #76 ) and thus sometimes it would fail and sometimes not. The patch in d6b64f7 demonstrates that.

  2. andreasprlic commented on Mar 18, 2014

    @andreasprlic
    Member

    thanks, well spotted!

  3. andreasprlic commented on Mar 18, 2014

    @andreasprlic
    Member

    I checked out the testsissue branch and fixed the SiftsChainToUniprotMappingTest. mvn clean install now works fine for me, however when I commit this to my fork on github andreasprlic@3b5f130 , travis still has some issues...

  4. sbliven commented on Mar 19, 2014

    @sbliven
    Member

    Jose is a power tester! Maybe sorting out these problems with static variables and cached files will make the travis builds more stable, as I feel like it often complains about things that work fine on my desktop.

  5. josemduarte commented on Mar 19, 2014

    @josemduarte
    ContributorAuthor

    So one important detail (it took me most of the day to figure it out!): the pom.xml patch above only works on surefire >2.14. So in andreasprlic@3b5f130 it is having no effect unless one changes the version to 2.14 or newer. Once you put that in, then both mvn install and mvn -Dtest=TestBond test fail for TestBond.test145D()

  6. heuermh commented on Mar 19, 2014

    @heuermh
    Member

    @josemduarte we should update to surefire 2.17 in any case, or the build won't work on jdk 8, which was released yesterday. I can do that in a separate pull request, if that would be better.

  7. heuermh commented on Mar 19, 2014

    @heuermh
    Member

    I created pull request #104 for surefire 2.17 and

    biojava/biojava-legacy#11

    for the same for biojava-legacy.

  8. andreasprlic commented on Mar 19, 2014

    @andreasprlic
    Member

    I can reproduce the problem now, after upgrading to surefire 2.17.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBugs and bugfixes

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions