Board index FlightGear Development Nasal

re: porting aircraft.nas /cppunit testing (pointers)

Nasal is the scripting language of FlightGear.

re: porting aircraft.nas /cppunit testing (pointers)

Postby Hooray » Sun Sep 15, 2019 2:30 pm

Hi, just briefly, since I am basically still AFK when it comes FG matters, i.e. haven't run/built FG in months (read: years...):

Referring to your talks on the devel list (still lurking/skimming occasionally):


As a matter of fact, Torsten himself originally considered providing exactly such a bindings layer/wrapper: viewtopic.php?f=66&t=15189&p=149376&#p149376

Regarding the needed changes for the testing infrastructure/machinery, quoted below:

https://sourceforge.net/p/flightgear/ma ... /36763063/
legoboyvdlp wrote:By the looks of it: we want to load up the Nasal subsystem in
FGTestAPI/Nasal.cxx (new file)
(e.g.)
globals->add_new_subsystem<FGNasalSys>(SGSubsystemMgr::INIT);

as well as all subsystems that nasal depends on. For instance, I think the
event_mgr, properties, and logger at least would need initialised but I can
figure that out by trial and error.



These have also been previously worked on, for pointers see:
http://wiki.flightgear.org/Initializing_Nasal_early


I would suggest to take a look at Andy's original github repo, and look at the standalone nasal binary that you can find there.
You will also find a number of Nasal specific regression tests.

A couple of years back, Philosopher worked out a simgear-based adaption of the standalone Nasal binary that would also include our property tree, timers (events) etc - Icecode GL ended up using a subset of this for his work on the fgradar infrastructure, which also still can be found on line:

You can still look at the original commits/changes we made when we reworked fgfs to support partial/incremental re-initialization: https://gitorious.org/fg/philosophers-f ... s/fgcanvas

To look at some of the original work done by Philosopher, see: https://gitorious.org/fg/philosophers-f ... nded-nasal

This includes a basic cmake-integration for Andy's original unit tests, which should come in handy for the kind of project that bugman is working on: https://gitorious.org/fg/philosophers-f ... 0e72fea88c

Hopefully, these pointers will save you some time and headache when working on things related to this - obviously, it's up to you to use these or not ;-)
Still, while none of these are intended to apply cleanly given how much time has passed, all the infrastructure changes to cmakelists.txt involving simgear/nasal should apply "as is", because they merely introduce new, and optional, stuff.

As you can see in the original comments, James Turner was originally supportive of these changes and encouraged work on integrating nasal into the testing suite.

PS: Sorry, if I am being late to the party ... again, only lurking - good luck anyways ;-)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: re: porting aircraft.nas /cppunit testing (pointers)

Postby Hooray » Sun Sep 15, 2019 2:41 pm

Okay, sorry for responding to myself (once again) - but just following up on bugman's posting https://sourceforge.net/p/flightgear/ma ... /36762109/

the "std" stuff you ran into is a canvas-related addition that is flightgear specific, so the std namespace/module isn't available in simgear, I think it was added by TheTom at the time. You should find a corresponding "std" folder somewhere in $FG_ROOT/Nasal

Regarding the comments about running a subset of the fgfs binary with required subsystems/dependencies only, this too, has been previously done successfully - even without coming up with a new binary, just by disabling certain blocks during startup, for dynamical dependency resolution and the pitfalls involved (background), search the forum/wiki for "nasal + postinit", this will provide all the background info that you need to pull that off, many comments can still be found in the original FGPython discussion:

viewtopic.php?f=6&t=28378&p=330191&hilit=postinit#p330191
viewtopic.php?f=6&t=28378&p=277303&hilit=postinit#p277303
viewtopic.php?f=6&t=28378&p=277163&hilit=postinit#p277163
viewtopic.php?f=30&t=28084&p=265842&hilit=postinit#p265842
viewtopic.php?f=30&t=28084&p=265822&hilit=postinit#p265822
viewtopic.php?f=71&t=23499&p=214849&hilit=postinit#p214849
viewtopic.php?f=71&t=23499&p=214631&hilit=postinit#p214631

You will find that this work was originally done when we worked out how to initialize Nasal earlier, to make fgfs start up with a subset of subsystems, e.g. to run in a so called "fgcanvas" mode, with only nasal + canvas running, e.g. for multi-instance setups: http://wiki.flightgear.org/FGCanvas


This work also involved bootstrapping scripting space in a modular fashion: viewtopic.php?f=30&t=21083&p=214423&hilit=postinit#p214423


Again, please freel to disgregard/ignore these pointers, i.e. use these as you deem fit, but if you are seriously considering to work on anything related to this, these pointers could probably save you a ton of time in the process, even if just to show you that you are pursuing a dead end, or doing redundant work - because others have succeeded doing these things already 7 years ago ;-)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: re: porting aircraft.nas /cppunit testing (pointers)

Postby legoboyvdlp » Sun Sep 15, 2019 3:13 pm

What?

Who is this directed to, Lars or me or..?

Feel free to respond directly to the mailing list to avoid confusion ;)

Anyway, thanks as always for your comments.

re-implementing stuff like aircraft.nas is redundant and isn't needed at all


I'm afraid I don't see the link with property rules and specific classes in Nasal scripting, but yes - that's quite right; they run at an average of 15 microseconds which is hardly worth the effort for marginal if any benefit :)

Do you mean things like the lowpass filter? So instead of calling a Nasal class, it calls an equivalent function in the property rules framework?

Just so you're clear on what ended up happening:
- implementing classes in aircraft.nas as C++ is not happening.
- Others such as tanker.nas may end up moving to C++ space if the AI stuff can be exposed to scripting (per Thorsten's comments)
- Extending the unit testing framework to support nasal is the first thing on the list.
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: re: porting aircraft.nas /cppunit testing (pointers)

Postby Hooray » Sun Sep 15, 2019 3:51 pm

Again, this wasn't intended to discourage any of your work, just to provide a few more pointers, if you are seriously pursuing any of this.

Do you mean things like the lowpass filter? So instead of calling a Nasal class, it calls an equivalent function in the property rules framework?

yeah, I think you guys talked about moving certain filters out of scripting space into C++ by re-implemting said code there, my point was that this wouldn't be needed, because Torsten completed that work years ago, all that is needed is a "bridge" to expose this existing C++ functionality to scripting space.

Originally, this would be done via so called fgcommands (bindings), later on people would use dedicated Nasal extension functions, and then dedicated SGSubsystems using listeners to provide an API on top of the property tree, analogous to how the Canvas system works, but the mechanism could also use cppbind to expose existing C++ classes to Nasal (and vice versa!), equally Richard's Emesary work could come in handy here. If in doubt, it's best to look at these different options and see what makes most sense to you given your own background/goals and roadmap/priorities.

Personally, I'd probably use cppbind, because it can be easily used to expose any existing C++ class to Nasal, without much work required. There is a simple tutorial to be found on the wiki (link above).

What this means is that you'd register a new Nasal function that is called to create a property rule dynamically, which will then instantiate the system on demand.
But again, that really only makes sense if you have identified any low-hanging fruits.

However, conceptually, the same approach will work for the AI system - re-implementing tanker.nas in C++ space is a long-standing idea, and we've grown a huge wiki article over time gathering tons of useful information over the years: http://wiki.flightgear.org/Scripted_AI_Objects

For some related background information, you might want to look at Durk's own comments about the state of the AI system in FG: http://wiki.flightgear.org/FGTraffic


Depending on who's currently around, and willing/able to help, your best bet is probably cppbind and/or Emesary.

I once talked with Richard about providing fgcommand/cppbind-bindings for Emesary so that this could be accessed from Nasal, including SGIOChannel bindings so that the SGSocket stuff could be trivially abstracted away from folks wanting to play with custom protocols/interfacing mechanisms, which could come in pretty handy when it comes to creating a generic tanker.nas re-implementation.

Again, best to make up your own mind about your goals/priorities here - as can be seen by the FGTraffic article/comments, single-handedly re-architecting fgfs from scratch isn't gonna happen easily, there are numerous projects exemplifying that, including the PUI/Qt transition, HLA/RTI or the transition from Nasal to Python.

Like you said, starting with the unit testing portion of the effort certainly sounds like a good idea, you will find the previously mentioned articles/efforts and patches (git commits) to come in pretty handy, i.e.:

- initializing nasal early
- Andy's original regression tests
- fgradar for a standalone property/events-enabled nasal-bin executable
- listener-based postinit to incrementally init scripting space

So feel free to ask for clarification if you cannot seem to make heads and tails of the pointers/patches I mentioned, even though I am sure bugman is meanwhile fairly familiar with most of those, because we repeatedly discussed those over the years in the context of various FGPython/cppunit discussions ;-)

Finally, for anything involving C++ work that you'd like to see incorporated back into fgfs, I would definitely suggest to stay in touch with several fg core committers - many of the patches mentioned above were originally created at the encouragements of some core devs at the time, who either disappeared or lost interest in some of their own ideas over time, so once you are considering to spend any considerable amount of time coding, it's best to check back with the devel list to make sure that you have not just one mentor, or your work may face a similar destiny - then again, good idea tend to have a certain shelf time, i.e. even things like the Canvas system took years to materialize ;-)



PS: Like I said, I am only lurking occasionally, so I am definitely not subscribed to the devel list, or going to participate there ;-)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: re: porting aircraft.nas /cppunit testing (pointers)

Postby legoboyvdlp » Sun Sep 15, 2019 5:28 pm

Thanks :)
Ok - that makes sense. The only problem I'm not sure if there is existing lowpass filter in property rules; if not that would involve adding it, then linking the nasal function to the respective property rule via cppbond? Is that correct?
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: re: porting aircraft.nas /cppunit testing (pointers)

Postby Hooray » Sun Sep 15, 2019 5:44 pm

The property rule/autopilot system is incredibly flexible, at some point even the JSBSim folks were discussing on their mailing list that the FG version would make sense to "back-port" into JSBSim, because its capabilities are way beyond what JSBSim could do at the time.

Right now, it works through the property tree, so there is an XML file that is loaded into the property tree, and a corresponding C++ class (object) watching that property location for "events", this is how the underlying C++ data structures and algorithms are set up (again, last I checked, but I doubt that there have been too many changes, because it used to be highly functional and effective "as is").

What would need to be done is to make the instantiation of the object dynamical, i.e. look at fg_init.cxx to see how the autopilot/property-rule subsystems are set up, the initialization used there (one linking to sim rate, the other one to fdm rate) would be moved into a Nasal accessible API, either as an fgcommand, an extension function or via cppbind - alternately, via a SGPropertyChangeListener-based wrapper.

How to proceed from here really depends on your background, i.e. what makes most sense to you in C++ terms - cppbind can look daunting, Emesary may seem "roundabout" if you have never done any message-based programming, fgcommands look simple and the extension function based approach is a really good way to understand how Nasal and C++ can be hooked up.

So conceptually, it would probably make sense to go through several iterations, e.g. tinkering with an fgcommand that you make available, and then adapting your code to use an extension function:

http://wiki.flightgear.org/Howto:Add_ne ... FlightGear
http://wiki.flightgear.org/Howto:Extend_Nasal

Even if all you end up coding is a simple "hello world" function that returns a number, you will learn how to interface between Nasal and C++ space.
With a working C++ build environment that has sg/fg/fgdata clones checked out, you should be up and running within just a few hours of tinkering, even without much coding experience other than Nasal/JavaScript.

With that background, it will be much easier to look at cppbind, and possibly Emesary - but for the latter, I would definitely check back with Stuart or Richard ideally first ...

Regarding the "init-nasal-earlier" and unit-testing work that bugman mentioned, there is also the "headless" effort that aims to provide a headless fgfs binary, with just certain console-only subsystems available: http://wiki.flightgear.org/FlightGear_Headless

As you can undoubtedly tell, many of these ideas have been turned into code over the years, they just didn't get committed at the time for a variety of reasons, but that doesn't necessarily mean that you cannot use such work, discussions or patches as reference when working on your own ideas.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: re: porting aircraft.nas /cppunit testing (pointers)

Postby bugman » Mon Sep 16, 2019 8:39 am

Hooray wrote in Sun Sep 15, 2019 2:41 pm:Okay, sorry for responding to myself (once again) - but just following up on bugman's posting https://sourceforge.net/p/flightgear/ma ... /36762109/

the "std" stuff you ran into is a canvas-related addition that is flightgear specific, so the std namespace/module isn't available in simgear, I think it was added by TheTom at the time. You should find a corresponding "std" folder somewhere in $FG_ROOT/Nasal


That's where the file is located: $FG_ROOT/Nasal/std/Vector.nas.test. I'm not sure why I cannot load std.Vector in standalone Nasal.

Regards,
Edward
bugman
Moderator
 
Posts: 1808
Joined: Thu Mar 19, 2015 10:01 am
Version: next

Re: re: porting aircraft.nas /cppunit testing (pointers)

Postby Hooray » Mon Sep 16, 2019 5:35 pm

I guess, that depends on how exactly you are trying to load such files ?

The issue here being that Nasal, originally at least, didn't have any importing directive.

It was only later on, around 2002/2003, that Andy started working on support for Nasal sub-contexts, so that Nasal code could load other Nasal code.

I believe at the time, the person developing AlgoScore required such a mechanism.

This is why we have meanwhile several somewhat confusing (competing/conflicting) mechanisms to load Nasal code into FlightGear:

- the old hard-coded way: via FGNasalSys, parsing *.nas in $FG_ROOT/Nasal
- ThorstenB's Nasal sub-modules, processing all directories under $FG_ROOT/Nasal to load all files in each directory into a directory-based Nasal namespace (aka Vector into std)
- and io.load_nasal() using an actual readfile() call to load/parse and compile a Nasal blob directly
- TorstenD's addon mechanism, under the hood using the latter

What you can find in Andy's github repo is unlikely to work "as is" for the reasons mentioned above, i.e. you will definitely need to use io.load_nasal() or manually load/parse and compile the code directly via Nasal's helpers, which is what can be found in Andy's repo in the form of "import" (IIRC)

Speaking in general, Nasal initialization has become rather messy with tons of places dumping/loading and executing Nasal blobs - which is why making things work in a standalone setup requires some familiarity with fgfs/nasal internals, I ran into that when I prototyped the pui2canvas parser, which also needed a way to deal with Nasal blobs embedded in XML files, but also with procedurally created dialogs.

It would be best to show us how you are trying to load such modules, so that we can compare things with the way fgfs handles everything (another reason to really only use a single binary and incrementally fix up that over time).

PS: Just briefly skimming over Vector.nas , I don't believe/remember that some of the APIs used there are actually available in standalone Nasal, e.g. debug.nas resides in $FG_ROOT/Nasal and isn't available in Andy's sources, it's FG specific
Either way, the first step really is loading Vector.nas into the std namespace: https://sourceforge.net/p/flightgear/fg ... Vector.nas

If that didn't take place first, the test will inevitably fail, and it should tell you why by looking at the error log, which can be retrieved by passing an empty vector to the call() directive, which will the be populated with any errors that were triggered, the common thing to do is to run a size(vector) check to see if the error vector is populated or not (= no errors).

Feel free to ask for details, but also be sure to provide some actionable information, ideally taking into account that I won't be running fgfs and also won't be checking out any git clones, but I should still be able to get you going with that pretty easily :-)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU


Return to Nasal

Who is online

Users browsing this forum: No registered users and 5 guests