Implement lazy Session objects

[alixdamman]

@gdementen almost all our users define their arrays in some params.py module as "zeros" arrays and fill them with proper data later (by reading data from CSV, Excel or HDF files). They have a lot of arrays but they often use only some of them when they run a part of their model.

We need to find a way to spare memory and to use memory only when it's needed.
What's your opinion on that?

[gdementen]

I have some working proof-of-concept code for lazy Session objects in my experiments folder. It assumes you declare your sessions and arrays like this:

class Demo(StaticSession):
    hh = ArrayDef()
    pop = ArrayDef("a=a0..a3")

instead of creating arrays filled with zeros. This also gets you completion for session arrays. Arrays are then loaded when first used. FWIW, in the example above "hh" is lazy but can have any shape/axes while "pop" is checked that it contains an a axis with those labels. I don't remember how much work is still needed for this to be actually usable, but I think it was significant. Anyway, I think 0.30 is way too early for that feature (at least if I have to do it).

I think we could also support some form of LazyArray outside of a session but that probably needs even more work (if at all possible) and I am unsure it is worth it. I would have to think about that. It's hard to say before hand.

[alixdamman]

I don't think having LazyArray outside of a session is needed. IMO, implementing StaticSession would be enough. Users should NOT declare variables as global.

I wonder if after implementing StaticSession we should not make global_arrays and arrays private functions and use them internally only when calling view() or edit() without argument?

[gdementen]

I wonder if after implementing StaticSession we should not make global_arrays and arrays private functions and use them internally only when calling view() or edit() without argument?

I don't think so. There are cases where global_arrays()/arrays() will still be more practical than creating a Session explicitly. It's ugly but practical... Time will tell...