Now the import problems is solved by removing the old code that was there to limit GPU mem usage, which is maybe not something we need in general.

But with the recent version of tensorflow I cannot load the model from the JSON file, see CI. This problem looks very much like the one in keras-team/keras#20081

Solutions seems to be to load model with older keras version 2 and then save as .keras file that should work with keras 3.

too bad, that there is no backwards compatibility. The code was written when keras was its own package, then it moved to tensorflow and now it will be independent again.
The issue with GPU memory usage was that tensorflow blocks all available memory even if only a fraction is needed. By limiting the memory you can avoid that smaller processes on a GPU fail, e.g. if the graphical output of a laptop user is also on the dedicated GPU

Here I used a newer version of tensorflow and the GPU settings as follows:
TF_CONFIG = tf.compat.v1.ConfigProto(gpu_options=tf.compat.v1.GPUOptions(per_process_gpu_memory_fraction=0.9), allow_soft_placement=True)

Thanks for chiming in @jpolz!

Should I add the TF_CONFIG... stuff you linked above or should I just keep it as it is now, without the GPU memory config?

I would add the TF_CONFIG. It is the safe way I'd say. I used it with tensorflow version 2.7.0

Regarding the problem with loading the model from JSON, I now did this in one of the WIP notebooks:

from tensorflow.keras.models import model_from_json
modeljson_fn = '../../pycomlink/processing/wet_dry/cnn_model_files/model_2020.002.180m.json'
json_file = open(modeljson_fn, "r")
loaded_model_json = json_file.read()
json_file.close()
model = model_from_json(loaded_model_json)
model.save('../../pycomlink/processing/wet_dry/cnn_model_files/model_2020.002.180m.keras')

let's see if this works...

Unfortunately there seems to be a major compatibility problem between keras 2 and keras3 when trying to load models from 2 in 3. In tensorflow they switched to keras 3 with version 2.16.

So now, there is this error in the CI:

TypeError: Could not deserialize class 'Functional' because its parent module keras.src.engine.functional cannot be imported. 

Based on this comment I conclude that there is no easy way to fix this... Two comments below someone writes

Solution was rebuilding the model layer by layer in Keras 3 and then loading weights file from Keras 2 model. Now able to run the model and obtain predictions.

So that might be the only solution... 😬

Closed, because currently the only fix for the problem with tensorflow was #151.

Later, when there is more capacity, we might move the kera2 models to keras3 as discussed in #150 (comment). Then, we could add the tensorflow code back here, or we host everything that uses tensorflow (and/or pytorch) for CML processing in a new package.