Improvements and debugs by tiborauer · Pull Request #173 · automaticanalysis/automaticanalysis

tiborauer · 2018-10-17T16:14:02Z

Improvements

CoSMoMVPA:
- allow specifying mask
- classification
- settings for tasks
- balance partitions for classification
generic modules
- coreg_extended_2
- norm_write(_dartel)
- dartel_denorm
seperate streams for mask_fromsegment
provenance - align with NIDM-W
diagnostics visual
- more intuitive tsdiffana plots
- consistent colours for segmentations (histogram and overlay)
checkreg can now process any number of overlays (i.e. ROIs)

Debugs:

connection
- with generic modules
- with optional stream
epifromnifti with dummies
*fromnifti modules uncompress data into aa rather than into the raw directory
aamod_freesurfer_register on study level
connecting to pipeline + initialstream
firstlevel_threshold report when not all are preceded by firstlevel_contrasts

PR

PR159

tiborauer · 2019-01-31T17:31:08Z

I have resolved the conflict. What "BIDS dataset problems" do you mean? Is it about the demo we are using (ds114) or the one you mention in #186 ?

jpeelle · 2019-02-01T12:52:04Z

When running the ds114 demo script my computer crashes (= turns off, which I suspect might be a segfault). I have not seen this behavior at all doing any other activity so I don't think it's something like a bad hard drive. However, I haven't been back at my desk long enough to investigate further. It could be totally unrelated (but suspicious that it happened 5 times in a row when running that script, but no others). (I realize segfaults while running these analyses should be rate/nonexistent but that's all I can think of.) That was also on Matlab 2016b, although that's never been an issue in the past.

jpeelle · 2019-02-01T12:53:24Z

Given how unusual the problem is, and that this is my first time running through the aa ds114 example script, .I suspect that whatever problem is occurring is not because of the PR. Thus I'm happy to merge it. But since I haven't officially tested it as I should I wanted to document that before moving ahead (I'm assuming no one else's computer is shutting down when running example scripts).

PR180

tiborauer · 2019-02-01T16:35:21Z

The ds114 has DWI data, and the pipeline contains aamod_diffusion_dtinlfit, which is memory intensive and may crash with segmentation fault, so I guess it is that particular module which causes the issue. Can you confirm this?
Have you tried aamod_diffusion_dtinlfit before or any other dataset? What happens if you replace aamod_diffusion_dtinlfit with aamod_diffusion_dtifit?

tiborauer · 2019-02-07T16:22:19Z

Hi, @jpeelle, Any news on this #173?

jpeelle · 2019-02-07T16:25:59Z

No, unfortunately I have not had time to try this! Very sorry. I'm happy to merge the PR based on the non-DWI data passing easily.

…

On Thu, Feb 7, 2019 at 10:22 AM Tibor Auer ***@***.***> wrote: Hi, @jpeelle <https://github.com/jpeelle>, Any news on this #173 <#173>? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#173 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGtPZprlj9TF002Jvmxbh23HuQViP76ks5vLFK8gaJpZM4XkYC1> .

tiborauer · 2019-02-07T16:44:27Z

It is quite a bit PR, and I also do not want to plant seed of some potential issues. But it is your call, and we can also debug things later. Merging it may open up space for wider testing. :)

Also, if you merge it here, you may also need to update your fork(s) from where you submit PR(s).

jpeelle · 2019-02-07T21:23:49Z

@jooh what do you think?

jooh · 2019-02-08T08:07:00Z

Sorry, I've been busy on the python side of things. I can run the tests at cbu if you haven't already.

One small point - it might be worth doing a new example script for your MVPA modules so those get some test coverage.

jooh · 2019-02-12T09:23:43Z

Actually, @tiborauer, to speed up testing of your various PRs, would it make sense to consolidate them all into a single PR that I can test once?

tiborauer · 2019-02-12T14:00:19Z

Can do. Shall I merge the into this one?

jooh · 2019-02-12T14:31:12Z

Sure!

…

On Tue, 12 Feb 2019 at 14:00, Tibor Auer ***@***.***> wrote: Can do. Shall I merge the into this one? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#173 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABfNs7c5s3AN6xYhbBj-_fLFfsLPUS8wks5vMsjzgaJpZM4XkYC1> .

-- Johan Carlin, PhD Investigator Scientist MRC Cognition and Brain Sciences Unit University of Cambridge

DEBUG: standalone according to the recent updates

Devel

tiborauer · 2019-02-12T14:35:17Z

Done!

jooh · 2019-02-12T16:04:39Z

Ok, tests running... I will run tests both under CSH and Bash this time, so we will probably need to fix #191 to get this one through. But we shall see!

jooh · 2019-02-13T10:21:59Z

Yes, indeed. We crash on aa_demo on CSH shells. So I suggest we get this fixed ASAP by including the fix in this PR.

***LINUX ERROR FROM SHELL ulimit: Command not found.
        
***WHILE RUNNING COMMAND
setenv TERM dumb;ulimit -m
        
        
**** automatic analysis failed - see reason and line numbers below
***WITH ENVIRONMENT VARIABLES
REMOTEHOST=login27.mrc-cbu.cam.ac.uk
NNTPSERVER=nntp-serv.cam.ac.uk
MANPATH=/cluster-software/gold/2.2.0.5/man:/cluster-software/torque-2.3/man:/usr/share/man
HOSTNAME=login12
FS_OVERRIDE=0
HOST=login12
TERM=screen-256color
SHELL=/bin/tcsh
SSH_CLIENT=172.31.120.37 44716 22
PERL5LIB=/imaging/local/software/freesurfer/latest/x86_64/mni/lib/perl5/5.8.5:/imaging/local/software/freesurfer/latest/x86_64/mni/lib/perl5/5.8.5:/imaging/local/lib/perl5
OLDPWD=/hpc-software/matlab/r2018a
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
OS=Linux
SSH_TTY=/dev/pts/2
LOCAL_DIR=/imaging/local/software/freesurfer/latest/x86_64/local
sourcefile=/imaging/local/software/freesurfer/latest/x86_64/FreeSurferEnv.csh
GROUP=ftp
USER=train01linux
LD_LIBRARY_PATH=/imaging/local/software/mrtrix/v0.2.10/lib:/imaging/local/software/mrtrix/v0.2.10/lib:/imaging/local/linux/lib
LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lz=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.bz=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.rar=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
AUTOMOUNT_MAP=
FSFAST_HOME=/imaging/local/software/freesurfer/latest/x86_64/fsfast
MNI_PERL5LIB=/imaging/local/software/freesurfer/latest/x86_64/mni/lib/perl5/5.8.5
FMRI_ANALYSIS_DIR=/imaging/local/software/freesurfer/latest/x86_64/fsfast
HOSTTYPE=x86_64-linux
FSF_OUTPUT_FORMAT=nii
GOLD_HOME=/cluster-software/gold/2.2.0.5
MINC_BIN_DIR=/imaging/local/software/freesurfer/latest/x86_64/mni/bin
OSG_LD_LIBRARY_PATH=/hpc-software/matlab/r2018a/sys/openscenegraph/lib/glnxa64
SUBJECTS_DIR=/imaging/train01linux/subjects
TOOLBOX=/hpc-software/matlab/r2018a/toolbox
MAIL=/var/spool/mail/train01linux
PATH=/imaging/local/software/mrtrix/v0.2.10/bin:/imaging/local/software/freesurfer/latest/x86_64/bin:/imaging/local/software/freesurfer/latest/x86_64/fsfast/bin:/imaging/local/software/freesurfer/latest/x86_64/tktools:/imaging/local/software/fsl/v5.0.6/x86_64/fsl/bin:/imaging/local/software/freesurfer/latest/x86_64/mni/bin:/imaging/local/software/fsl/v5.0.6/x86_64/fsl/bin:/hpc-software/bin:/imaging/local/software:/opt/TurboVNC/bin:/opt/TurboVNC/bin:/imaging/local/software/anaconda/latest/x86_64/bin:/opt/gold/bin:/opt/gold/sbin:/imaging/local/software/mrtrix/v0.2.10/bin:/imaging/local/software/freesurfer/latest/x86_64/bin:/imaging/local/software/freesurfer/latest/x86_64/fsfast/bin:/imaging/local/software/freesurfer/latest/x86_64/tktools:/imaging/local/software/fsl/v5.0.6/x86_64/fsl/bin:/imaging/local/software/freesurfer/latest/x86_64/mni/bin:/imaging/local/software/fsl/v5.0.6/x86_64/fsl/bin:/hpc-software/bin:/imaging/local/software:/cluster-software/gold/2.2.0.5/sbin:/cluster-software/maui/bin:/cluster-software/torque-2.3/sbin:/cluster-software/torque-2.3/bin:/opt/TurboVNC/bin:/opt/TurboVNC/bin:/usr/lib64/qt-3.3/bin:/imaging/local/software/anaconda/latest/x86_64/bin:/opt/gold/bin:/opt/gold/sbin:/usr/local/bin:/bin:/usr/bin:/opt/bin:/usr/local/wrapper/bin:/usr/local/bin:/usr/bin/X11:/usr/local/cbu/scripts/:/imaging/local/linux/bin:/imaging/local/linux/bin/elekta:/imaging/local/linux/bin/tractography/bin:/imaging/local/scripts:/home/train01linux/bin:/imaging/local/software/dcmtk/v3.6.0/dcmtk/bin:/usr/local/wrapper/bin:/usr/local/bin:/usr/bin/X11:/usr/local/cbu/scripts/:/imaging/local/linux/bin:/imaging/local/linux/bin/elekta:/imaging/local/linux/bin/tractography/bin:/imaging/local/scripts:/home/train01linux/bin
PWD=/imaging/train01linux/aa/aa_demo/auditory/aamod_convert_structural_00001/S1/structurals
FUNCTIONALS_DIR=/imaging/train01linux/sessions
_LMFILES_=/cluster-software/modulefiles/torque/2.3.0:/cluster-software/modulefiles/maui:/cluster-software/modulefiles/gold/2.2.0.5
EDITOR=vi
MOABHOMEDIR=/cluster-software/maui-home
MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/cluster-software/modulefiles
MAUIHOMEDIR=/cluster-software/maui-home
LOADEDMODULES=torque/2.3.0:maui:gold/2.2.0.5
KDEDIRS=/usr
XFILESEARCHPATH=/hpc-software/matlab/r2018a/sys/java/jre/glnxa64/jre/lib/locale/for help, see the <a href="https://github.com/rhodricusack/automaticanalysis/wiki">aa wiki</a>
FAIL - /home/train01linux/matlab/automaticanalysis/examples/aa_user_demo.m

jooh · 2019-02-14T11:30:45Z

On the Bash side, cbu/aa_user_diffusion appears to be broken.

running test aa_user_diffusion.m...

Please wait a moment, adding <a href = "matlab: cd /home/jc01/matlab/aa5_devel"></a> to the path
Welcome to aa version 5.4.0 (222744c4128647cad4c2c4c2d5330a8fbb3d1e74) Feb 2019
 If you publish work that has used aa, please cite our manuscript:
 <a href = "http://dx.doi.org/10.3389/fninf.2014.00090">Cusack R, Vicente-Grabovetsky A, Mitchell DJ, Wild CJ, Auer T, Linke AC, Peelle JE (2015)</a>
 <a href = "http://dx.doi.org/10.3389/fninf.2014.00090">Automatic analysis (aa): Efficient neuroimaging workflows and parallel processing using Matlab and XML</a>
 <a href = "http://dx.doi.org/10.3389/fninf.2014.00090">Frontiers in Neuroinformatics 8:90</a>

Please visit <a href = "http://automaticanalysis.org">The aa website</a> for more information!

Here you can find example <a href = "matlab: cd /home/jc01/matlab/aa5_devel/aa_parametersets">tasklists</a> and <a href = "matlab: cd /home/jc01/matlab/aa5_devel/examples">scripts</a>.
Ready.


**** automatic analysis failed - see reason and line numbers below
ERROR: Stream normalised_grey of module aamod_diffusion_dartel_denormDKI not found!
for help, see the <a href="https://github.com/rhodricusack/automaticanalysis/wiki">aa wiki</a>

This appears to be because the stream names have changed, which the example script does not reflect.

tiborauer · 2019-02-20T16:43:14Z

I have looked at the script and, indeed, its streamname has no prefix according to my commit.
I guess we have not started to consistently remove prefixes...

The quick fix is aap = aas_renamestream(aap, 'aamod_diffusion_dartel_denormDKI_00001', 'grey', 'normalised_white', 'input');. However, the output will be also named "normalised_white" (no change in streamname within the module).

Pr173 fixes

jooh · 2019-03-07T14:54:37Z

And that's it, all the tests pass. Merging. Thanks Tibor!

Fmri connectivity

tiborauer added 30 commits August 13, 2018 10:16

NEW FEATURE: generic denorm for EPI

b21adbf

UPDATE: get rid of streamname prefixes - dartel_denorm

963aa0f

UPDATE: generic (multi-domain) dartel_denorm

0b780e3

UPDATE: dartel_denormepi on subject domain

7f2c093

DEBUG: dartel_denorm

fd6e4a7

UPDATE: seperate streams - mask_fromsegment

92bbffe

DEBUG: seperate streams - mask_fromsegment

89bd0e6

UPDATE: CoSMoMVPS - generic mask

36cfde5

DEBUG: renamestream - multiple targets for the same output

2b62f0d

DEBUG: renamestream - multiple targets for the same output

5c5080a

UPDATE: aamod_maths for study/subject level

d5e9c60

UPDATE: aamod_maths for study/subject level

b266caf

DEBUG: CoSMoMVPA with single mask

96bd94d

Merge pull request #60 from rhodricusack/master

b571b75

PR

Merge pull request #61 from tiborauer/master

752d4e3

PR

NEW: CoSMoMVPA classification

c234da8

DEBUG: convertseries crashing upon low memory

ccecef0

UPDATE: CoSMoMVPA settings for RSA and Classification

b3953d1

UPDATE: generic norm_write(_dartel)

d9fb590

DEBUG: CoSMoMVPA - multiple runs with single mask

45f546c

DEBUG: CoSMoMVPA with no RSA task

a3d4845

DEBUG: CoSMoMVPA with Classification task

86ec63f

DEBUG: generic norm_write(_dartel)

bdd0822

DEBUG: generic coreg_extended_2 and norm_write(_dartel)

d8919ae

DEBUG: connect with general purpose modules

882d14a

DEBUG: connect with general purpose modules

03d0139

DEBUG: connect with general purpose modules (documentation)

b80ae54

UPDATE: nonDCS Pool and Task attributes

a99e3f2

Merge pull request #62 from rhodricusack/master

0191ba3

PR159

Merge pull request #63 from tiborauer/master

3a04c0a

PR159

Merge branch 'master' into master

c4471fc

Merge pull request #81 from tiborauer/master

6d8801b

PR180

DEBUG: standalone according to the recent updates

7b63736

tiborauer added 2 commits February 12, 2019 14:33

Merge pull request #82 from tiborauer/standalone

be30e1f

DEBUG: standalone according to the recent updates

Merge pull request #83 from tiborauer/devel

222744c

Devel

tiborauer mentioned this pull request Feb 15, 2019

DEBUG: Fixed potentially empty string in dummylist #185

Closed

jooh and others added 4 commits February 21, 2019 12:00

handle missing ulimit on e.g. csh shells. resolves #191

0bf9a9f

update stream renaming in example

439a08b

use new stream names

a1e131a

Merge pull request #84 from jooh/pr173

80beecd

Pr173 fixes

jooh merged commit 1a68e1d into automaticanalysis:master Mar 7, 2019

tiborauer added a commit that referenced this pull request Oct 29, 2021

Merge pull request #173 from tiborauer/fmri_connectivity

33c0401

Fmri connectivity

Uh oh!

Conversation

tiborauer commented Oct 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tiborauer commented Jan 31, 2019

Uh oh!

jpeelle commented Feb 1, 2019

Uh oh!

jpeelle commented Feb 1, 2019

Uh oh!

tiborauer commented Feb 1, 2019

Uh oh!

tiborauer commented Feb 7, 2019

Uh oh!

jpeelle commented Feb 7, 2019 via email

Uh oh!

tiborauer commented Feb 7, 2019

Uh oh!

jpeelle commented Feb 7, 2019

Uh oh!

jooh commented Feb 8, 2019

Uh oh!

jooh commented Feb 12, 2019

Uh oh!

tiborauer commented Feb 12, 2019

Uh oh!

jooh commented Feb 12, 2019 via email

Uh oh!

tiborauer commented Feb 12, 2019

Uh oh!

jooh commented Feb 12, 2019

Uh oh!

jooh commented Feb 13, 2019

Uh oh!

jooh commented Feb 14, 2019

Uh oh!

tiborauer commented Feb 20, 2019

Uh oh!

jooh commented Mar 7, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tiborauer commented Oct 17, 2018 •

edited

Loading