Skip to content

Feature downloaddemo rework and tests - #306

Merged
tiborauer merged 13 commits into
automaticanalysis:masterfrom
AljenU:feature-downloaddemo-rework-and-tests
Dec 8, 2021
Merged

Feature downloaddemo rework and tests#306
tiborauer merged 13 commits into
automaticanalysis:masterfrom
AljenU:feature-downloaddemo-rework-and-tests

Conversation

@AljenU

@AljenU AljenU commented Nov 26, 2021

Copy link
Copy Markdown
Collaborator
  • Rework downloaddemo to a simpler interface, be windows compatbile, and make it possible to automatically test the download of the demo datasets.
  • Add test for all downloaddemo datasets
  • Rework tutorial_1 to use downloaddemo and be automatically testable
  • Add test for tutorial_1
  • Add test that only installs minimal tooling (only aa and spm) and no data, and run it on examples that should work with such a minimal install (and have been adapted so they can run automatically, currently only tutorial_1)

Fix issue #275
Fix issue #305

Only predefined datasets can be downloaded by aa_downloaddemo: this makes it possible to test
that the datasets are still available through Continuous Integration tests.

No rawdatadir manipulations inside aa_downloaddemo: When using aa_downloaddemo, the input must
specify exactly where the data should go. Since this function is for use in demo scripts, can
require that aap.directory_conventions.rawdatadir is set to expectation of this function. And
makes this function single-purpose.

No aas_shell calls in aa_downloaddemo: only use Matlab provided functions, no need to use system
functions. This also makes aa_downloaddemo windows compatible.

Add aa_downloaddemo_datasets to hold the list of predefined datasets. This makes it easy to loop
over all predefined datasets in a test.

Adapt all examples that use aa_downloaddemo to use the new calling convention.
Although this tutorial is to help users set up a parameter file, adapt it so it can also run without user interaction. In order to be able to have an automated test for tutorial_1
Tests for downloaddemo and tutorial_1, also made part of aa_test list of tests
Introduce tag-based filtering in aa_test, to have more fine-grained filtering. And add tags to the use-case tests.

Add a dedicated actions yml file for the non-use-case tests.
One of the demo datasets is an unzipped tar, which can give an error with untar if an incorrect file extension is specified.
Single-step install and configure, make parameter file name an env var
@tiborauer
tiborauer self-requested a review November 26, 2021 23:26

@tiborauer tiborauer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the CI has overwritten one artifact with the other: https://github.com/automaticanalysis/automaticanalysis/actions/runs/1507207336

@AljenU

AljenU commented Dec 1, 2021

Copy link
Copy Markdown
Collaborator Author

It seems the CI has overwritten one artifact with the other: https://github.com/automaticanalysis/automaticanalysis/actions/runs/1507207336

Ah, it took me while, but i see what you mean now: test_log.txt is created three times, but since it is uploaded with the same name, only one is available at the end. Will look into it.

So they do not overwrite each other. Also add aa_test.log artifact to use_case tests
@tiborauer

Copy link
Copy Markdown
Member

Thank you, @AljenU. Would it be possible to have the log for the downloaddemo_test more informative? Right now it says
testExamples/downloaddemo_test(dataset=value1) and testExamples/downloaddemo_test(dataset=value2). Could it tell the dataset ID, instead?

@AljenU

AljenU commented Dec 8, 2021 via email

Copy link
Copy Markdown
Collaborator Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants