Due to the way it evolved, PYME has a lot of interdependencies between components, which makes it a bit hard to maintain and to approach as a new contributor. This is probably best illustrated in the following images:
where hierachy is indicated by black lines, module imports by coloured lines.
Or somewhat more-succinctly, when reduced to imports between top-level modules:
What I'd like to aim for is something along the following lines, which has a clearly delineated hierarchy:
Key concepts:
- lower levels of stack should not depend on/import from higher levels (i.e. algorithms etc.. should not call GUI code)
- where practical, the interface between UI programs and algorithms should be via the recipe modules layer
- there should be minimal algorithm logic in either the recipe or UI layers
- strict separation between
PYMEAcquire and PYME core in that PYME core should not import anything from PYME Acquire.
- ditto for PYME.util, which should not import anything from PYME core (PYME.util contains stuff like profiling routines which are not really PYME specific).
- foundation, algorithm, and recipe layers to aim for 100% test coverage
Because PYME.util stuff should be much more broadly applicable than just PYME, it should probably be factored out into separate module(s) and repositories. I'm still undecided on PYMEAcquire - there are arguments each way:
PYMEAcquire as a separate project?:
For:
- Analysis and visualisation functions are useful without acquisition
- Dependency footprint is very different from PYME core (and currently not accurately captured in PYME dependencies due to the difficulty in getting a resolvable environment). IE we don't currently distribute a functional PYMEAcquire in the current automated installers etc ...
- Much of the acquisition code is not testable on development machines / via CI etc (i.e. hard to get any automated test coverage etc ....)
- Would decrease overall repo size, making CI operations etc faster
Against:
- PYMEAcquire not really standalone, will likely still require extensive linkages to ui and IO components
- Potential overhead in maintaining 2 packages
- Implicit data model dependencies on acquisition
- History / potential for disruption
A reasonable compromise in the interim would be to keep PYMEAcquire within the main package, but to try and achieve and maintain import separation (i.e. move things like spoolers into IO) so that it could easily be spun out if needed.
Due to the way it evolved, PYME has a lot of interdependencies between components, which makes it a bit hard to maintain and to approach as a new contributor. This is probably best illustrated in the following images:
where hierachy is indicated by black lines, module imports by coloured lines.
Or somewhat more-succinctly, when reduced to imports between top-level modules:
What I'd like to aim for is something along the following lines, which has a clearly delineated hierarchy:
Key concepts:
PYMEAcquireandPYME corein that PYME core should not import anything from PYME Acquire.Because PYME.util stuff should be much more broadly applicable than just PYME, it should probably be factored out into separate module(s) and repositories. I'm still undecided on PYMEAcquire - there are arguments each way:
PYMEAcquire as a separate project?:
For:
Against:
A reasonable compromise in the interim would be to keep PYMEAcquire within the main package, but to try and achieve and maintain import separation (i.e. move things like spoolers into IO) so that it could easily be spun out if needed.