Refactor example scripts for portability, implement basic automated testing - #161
Merged
Conversation
Conflicts: aa_recipes_and_parametersets/aap_parameters_defaults_CBSU.xml
…input in place of ROIfile setting for greater flexibility, update MTI example
…ive SPM paths, tidy up dirs
…r for more portable scripts
1. aarecipe now creates a new seed aap_parameters_user.xml file if no defaults file is specified. This put user-specific configuration outside of version control (where it should be). 2. The example scripts all call aarecipe without a defaults file, so that the above functionality makes it possible to run tests regardless of site (as long as the data is publicly available). 3. aa_test/aatest - new test function for running tests on the above examples in an automated manner. 4. Re-organised the examples directory for clarity and to make it possible to run particular test flavours (e.g., all CBU-specific tests are under the cbu folder). Added light-weight documentation for this. 5. Big tidy up of legacy examples. 6. Relocated the task lists to the examples directory, and renamed to always match the user script it goes with. Renamed aa_recipes_and_parametersets aa_parametersets to reflect this change (this directory only contains defaults now).
store aap_parameters_user in ~/.aa/ and add this directory to path
PR161 + review
Member
|
Why did you merged it if #163 also contained it? |
Member
|
I see now!:) |
Collaborator
Author
|
Github magic! |
jones-michael-s
pushed a commit
to jones-michael-s/automaticanalysis
that referenced
this pull request
Aug 6, 2021
Version upgrade
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a big one! This pull request tries to achieve two major objectives, which I will cover first. Then I'll mention a few minor changes that have also crept into this PR.
I highly recommend that you check out this pull request locally and try running the tests. I have reorganised the repo a bit so it might also be more apparent how things look now by just checking it out for yourself.
Major changes
Easier to get started with aa - the examples are shorter, have documentation, and should just run (famous last words) on a new machine, thanks to many under the hood changes to how AA defines file paths.
Rudimentary automated testing that runs through the examples above.
To achieve this I added many under-the-hood features:
shell expansion in aarecipe - you can reference any shell$variable or $ (expression) in any XML field, and the result will be evaluated at run-time.
aarecipe looks for aap_parameters_user.xml if no defaults.xml file is specified. If this file does not exist, aarecipe can generate it by asking the user to select one of the existing defaults as the parent.
default aap.acq_details.root - use the above functionality to place data in sensible default locations ($HOME/aa in defaults.xml, /imaging/$USER/aa in defaults_CBSU.xml).
We now setenv an SPMDIR shell variable to make it easy to reference SPM-relative paths in defaults and tasklists, using the above expansion functionality.
BIDS import improvements: select custom events.tsv column for first-level model regressor definition, select sub-set of subjects.
handy function for automatically downloading the aa demo dataset from Rhodri's S3 storage, if it's needed.
Minor changes
all roi_extract modules now expect input from the rois stream instead of the ROIFile option. This makes AA more modular and flexible. You will need to modify user scripts by adding aas_addinitialstream.
quieter, less repetitive logging, especially when using aas_shell
new modules to reorient nifti header dimensions to the format expected by fslview and similarly stupid viewers.
Update CBSU default with new version of various toolboxes. Lock versions to specific versions rather than referencing 'latest' for greater reproducibility. Set more sensible defaults (e.g., numdummies should arguably default to 0).
removed old code: aa_config, aas_configforSPM12, many old examples.
tidy-up - change all file endings and line breaks to unix.
tidy-up - co-locate example user scripts and tasklists in the example folder. I think this makes more sense since these files are used together only.
update readme to be somewhat more current.