Nasal re-loadable modules

[jsb]

Hi,
I just want to announce that there is support for re-loadable modules now (02/2020) on next. See http://wiki.flightgear.org/Modules.nas
If you have larger Nasal projects this might be interesting. Please let me know if the documentation is unclear or incomplete.

Have fun

[Necolatis]

That is great news. Thanks to the dev(s) of this.

[Necolatis]

This deserves a mention in the newsletter..

Edit: done

[Necolatis]

So the module name is that also the namespace?
If yes, then I have a question: Can 1 module have more than 1 Nasal file attached?

[jsb]

Yes, the namespace name defaults to the module name but it can be changed if you have the need.
However, be very careful if you use an existing namespace as it will be killed if you use the unload() or reload().
Generally speaking, you should NOT use any namespace of existing "load-once" modules, e.g. names of the first level subdirectorys in FGDATA/Nasal

Regarding multiple files the recommendation is to have one main file that uses io.include or io.load_nasal as needed (for an example see https://sourceforge.net/p/flightgear/fgdata/ci/next/tree/Nasal/modules/canvas_efis/main.nas)
Reading that file again I wonder why I defined the namespace as a string in the file instead of getting it from

Code: Select all

module.getNamespaceName()

- I might update this later.

Please also note that there are currently two usecases for modules.nas that are different in details
1) developing aircraft subsystems in Nasal
2) Nasal frameworks that require reload support

[jsb]

I hope the http://wiki.flightgear.org/Modules.nas#HowTo_.2F_Examples helps?

[Hooray]

Florent also mentioned your talks about using this for the updated addon framework - for the namespace issue, you can check if it's "new" (nil) or previously defined, and log a warning to the console, especially if a reloadable module is loaded into an existing namespace, using the directory() API you could also get a list of files in $FG_ROOT/Nasal and use those to create a vector with a blacklist of module names, so that people don't run into that problem.

Apart from that, wrapping _setlistener/_settimer and maketimer respectively per module via overloading, is rather powerful, because you can provide statistics per module for the number of timers/listeners registered, as well as actual invocations, by using a counter in the outer closure - that way, developers can much more easily see what's going on, e.g. also using debug.benchmark()

EDIT: Referring to your wiki update: http://wiki.flightgear.org/index.php?ti ... did=121650

Code: Select all

# First living example is the canvas_efis framework which can be included into an aircraft with a single line of code:    
var efis_module = modules.load("canvas_efis");


That's actually very compact and elegant, and I believe it'd be worthwhile to explore if the MapStructure/ND stuff we once added, could be also loaded that way given your current framework.
This should free up resources for people who don't need the MapStructure/NavDisplay modules loaded, and it would also unify the code, because we implemented module reloading manually, so that MapStructure layers could be easily reloaded on demand, i.e. at runtime - for faster prototyping/testing.

I am not sure if you are familiar with MapStructure, but it's not exactly "typical" Nasal code, it's using a few sophisticated concepts, but other than that, it'd be good for FlightGear if unneeded modules could also be loaded on demand, especially for people on hardware lacking resources.

MapStructure uses the MVC pattern to use a set of 3 different files per "map layer": one to control the layer, one to control/draw and load drawable symbols.

And since you've also been working on EFIS-related functionality, the structural changes needed by Modules.nas to support MapStructure/ND stuff, could also benefit your EFIS work, e.g. to support loading EFIS related functionality on demand ?

PS: You might want to version your framework, so that you can safely make changes that may break backward compatibility without having to worry about it, i.e. people using the new/updated version of your framework, could simply tell the framework explicitly, while people who haven't yet updated their code, can continue to use it "as is".

[jsb]

Apart from that, wrapping _setlistener/_settimer and maketimer respectively per module via overloading, is rather powerful, because you can provide statistics per module for the number of timers/listeners registered, as well as actual invocations, by using a counter in the outer closure - that way, developers can much more easily see what's going on, e.g. also using debug.benchmark()

That actually happens, some stats are written to the property tree under /nasal/modules/... so they can be easiely monitored at runtime in the property browser.
It is also possible to print a log line every time a tracked listener is hit by calling setDebug (see http://wiki.flightgear.org/Modules.nas#modules.setDebug.28.29) in case you like it really noisy

[Hooray]

yes, I actually looked at your code when I wrote that - the point was that, having a helper function/class to track these things could be enormously useful for people wanting to look behind the scenes, i.e. to gather number of listeners/timers added "per module", as well as actual invocation frequency of callbacks.

Richard has posted here about related work to track Nasal GC overhead - think of it as the equivalent of the existing "performance monitor": http://wiki.flightgear.org/Howto:Use_the_system_monitor
Just with a focus on Nasal modules.

In other words, as long as all the per-module data is kept/updated in a hash, arbitrary getters could be used, including tooltips or a dedicated Canvas dialog that could allow people to see what their modules are doing under the hood.

I once wrote a simple Canvas based tool to easily test reset/reinit using the same method: http://wiki.flightgear.org/Howto:Reset/ ... leshooting


A while before Florent reworked the addon system, and prior to your current work, I tinkered with a similar scheme also based on overloading the corresponding Nasal APIs:

http://wiki.flightgear.org/Howto:Troubl ... _Callbacks


Integrating these ideas would means that any Nasal code using your "modules.nas" framework, could easily provide the equivalent of a "system monitor" for each module (probably including addons)

Hooking this up to existing MFD/Canvas code, could also tell people what is going on in terms of concrete property I/O (number of getprop/setprop calls, frequency of updates etc)

[jsb]

I already had a version that creates a debug subtree in the property tree where every tracked listener creates somthing like /debug/listener_hits/<prop_path_as_string> (INT)
but having some hundred nodes beeing monitored got somehow confusing and I missed a sort button in the property browser because what interested me at that time was the question "Which listeners (props) fire most often?"
I did not follow up the idea because it was not my main focus at that time - so I cleared it away before submitting modules.nas.

[Hooray]

you've got a very valid point about performance, but it might still make for an interesting option - all that is needed is really just said hash to gather said meta information, so that another script can pick up that info - with an option to enable/disable tracking, i.e. for troubleshooting purposes.

The property browser itself is mostly PUI/XML + Nasal, so it can be easily extended - I once added a simple canvas-widget that is dynamically hidden/shown when browsing nodes under /canvas, so that you end up with a "preview" function:

http://wiki.flightgear.org/Canvas_Troubleshooting


However, as you alluded to, the current property-list widget is hard-coded in C++, so cannot be easily modified to implement sorting functionality.

It is however pretty straightforward to implement our own custom property-list widget in canvas space, simply by using the existing ScrollArea, a vertical layout and by adding one button per node - and then calling that function recursively whenever a new sub-node is entered:

http://wiki.flightgear.org/Howto:Proces ... perty-list


(This was never meant to look pretty, ist was just supposed to "work" well enough - I think it's roughly 50 lines of code to approximate the property-list picker, but it would be really easy to take that code, adapt it as needed to implement sorting-functionality or whatever else people may find useful)

Again, this is pure Nasal, no C++ changes needed whatsoever

[jsb]

I added a section about the propery tree interface to the wiki page about modules.nas

[Hooray]

Regarding the future of your framework, you may find it interesting to look up the original talks between James and Philosopher about "modular Nasal space bootstrapping": Modular Nasal bootstrapping (again)

You will see that this is fully in line with your current approach.
Besides, Philosopher wrote some of the most sophisticated Nasal tutorials we have in the wiki, including the "Nasal internals" document in $FG_ROOT/Docs: https://sourceforge.net/p/flightgear/fg ... format=raw

The whole point of the original idea was to decouple loading of Nasal modules, to better formalize dependencies between modules, but also between Nasal modules and C++ subsystems - e.g. think about tanker.nas requiring AI traffic, think about the pilot-list requiring multiplayer - think about canvas APIs for the canvas system etc.

The original thought was to use a subsystem-level "signal" (boolean property) to signal to other code that a system is available or not.

That way, it would become possible to dynamically load/unload Nasal modules as required, e.g. for regression testing purposes, but also to lower overall resource utilization by keeping stuff unloaded that simply isn't required.

Subsequently, Torsten and Florent came up with the dedicated Nasal based add-on system, where Florent also considered adding some kind of dependency tracking/resolution mechanism, so that modules could depend on other modules - e.g. think your "draw API" being required by other add-ons