Board index FlightGear Development Nasal

Nasal must go

Nasal is the scripting language of FlightGear.

Re: Nasal must go

Postby Thorsten » Sat Oct 08, 2016 6:55 am

I guess implementing something because you like it is somewhat different from asking everyone to change the design of FG rather massively or else... The latter requires quite a different amount of justification than the former.
Thorsten
 
Posts: 11193
Joined: Mon Nov 02, 2009 8:33 am

Re: Nasal must go

Postby Hooray » Sat Oct 08, 2016 8:21 am

While the "yak-shaving" aspect is definitely true when it comes to Nasal, that is actually a major advantage - especially in the light of the contraints that the scripting interpreter has to work in, i.e. FlightGear's legacy architecture.
I am the first person to admit that numpy etc would be cool to have for FlightGear scripting, but definitely not running within the FlightGear main loop.
Thus, when it comes to the plethora of 3rd party packages available, we may want those, but realistically cannot afford having/supporting them until quite a few construction sites are addressed - such as asynchronous subsystems, proper event management and package management (dependencies in particular).

For instance, right now you can safely assume that _any_ FlightGear script will work on _any_ OS/platform supported by FlightGear, without requiring workarounds or platform/OS specific code.

Compared to Lua, Python, Ruby, Perl etc - the major benefit that is undisputed is excellent support, tooling (debugging, profiling), documentation and maintenance/future development handled by a sizable community, rather than adding to our own todo list.

Likewise, Lua has meanwhile become a real thing for embedded scripting, so that there are alternate GC schemes, which would definitely be good to have for FG scripting, too.

But at the end of the day, the main bottleneck is not the language/interpreter supported by FlightGear, but the way it is integrated - and the legacy SGSubsystem/Mgr approach is simply not suitable for large-scale scripting, certainly not the kind of scripting that would definitely be boosted by supporting a more mainstream language like Lua or Python.

Thus, there are things that need to be addressed first, which is why it makes sense to review Richard's postings in this thread, and explore using Torsten's mongoose/Phi approach, i.e. remote properties and async fgcommands to use those as the foundation for a scripting subsystem that runs outside the fgfs main loop.

Apart from that, Thorsten is absolutely right - Nasal may be a niche language given its use/application, but apart from that, its syntax is fairly standard (close enough to C/C++ to look familiar to experienced programmers), and it is largely based on concepts/ideas borrowed from ECMAScript/JavaScript.

Thus, somebody suggesting on the one hand to support JavaScript instead of/next to Nasal, while at the same time complaining about Nasal's syntax, simply doesn't know what they're talking about at all.

All the FG related functionality ("APIs") would likely continue to look "as is", e.g. property access, fgcommands, timers, listeners - just with very minor changes to support the async/distributed use-case.

For the time being, FGPythonSys remains the strongest option to explore supporting other scripting languages - but that should definitely not be using the SGSubsystem/Mgr and FGNasalSys approach, but instead be built on something along the lines of Richard's IPC comments regarding mongoose/Phi, otherwise all the people wanting python would immediately learn that all the Nasal folks were right about the bottleneck being elsewhere, and Nasal not being the culprit - just imagine for a second, FGPythonSys would be available right now, and provide what FGNasalSys provides, using the same implementation strategy: We'd see all sorts of 3rd party modules using all sorts of platform/OS specific modules, quite possibly with certain aircraft, instruments and scenery requiring different versions of certain modules to be installed.

Nasal shortcomings in this area are a rather strong benefit when it comes to sandboxing the system - which is not to say that Nasal is the greatest piece of technology in FlightGear, or even that it was the best scripting option at the time - but it happens to be well integrated and just works; like Thorsten said, had we had Python, perl, Lua or Ruby support, we'd be using that instead.

But let's keep in mind that despite the current situation with Nasal, many core developers are already concerned that scripting is being used too widely - regardless of Nasal being de-facto under-documented, and hardly maintained - this situation would only get worse once/if a real mainstream language is supported.

In summary, it's not like Nasal's syntax is esoteric in any way - it is far more standard than Fortran, Forth, LISP or Scheme - despite those being definitely "more mainstream" than Nasal - and it being close enough to C, C++ and GLSL, also means that people doing FlightGear scripting get a feeling for the syntax that will come in handy later, when looking at other, non-nasal, FlightGear source code.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11376
Joined: Tue Mar 25, 2008 8:40 am

Re: Nasal must go

Postby Hooray » Sat Oct 08, 2016 12:09 pm

Since you don't seem to have read/understood the responses given here (or simply ignored them), I am responding again specifically to this:
Beaver wrote:I have done some experimentation around using Python's asyncio with the websocket API (which is faster than the telnet btw), and if I were to write an extension I would probably use this approach. It means I can use the whole scientific python stuff (for example GIS, but also machine learning), and have all my processing in a separate process. It's conceivable to implement AI planes, virtual instructors, missions and much more with this. Even for aircraft systems I see some advantages. I've been perpetually frustrated with setting up my input devices for flight gear, and thinking about writing scripts to circumvent the clumsy configuration system in FG. Using different configurations for different aircraft is also a pain point, but that's another story.
[...]
So, the only paradigm which allows me to do what I want with FG, is remote controlling FG through a Python framework. Unfortunately, no such framework exists. And instead of making it easier for other people to use my extensions, it would make it harder. In my opinion already ridiculously hard to use FlightGear even as a pure user.


Like I said previously, I really suggest to check your facts - obviously, a number of folks in this thread have pointed out how the Phi/mongoose setup works, and you specifically mentioned websockets and that Python, unlike Nasal, comes with "batteries included" (aka no "yak shaving").

We also have examples on interfacing to various FlightGear I/O and IPC mechanisms (think httpd, telnet/props), including some using Python: https://sourceforge.net/p/flightgear/fl ... ts/python/

It also seems that both, bugman and Curt have more up-to-date examples using Python to access the FlightGear property tree.
Torsten obviously provided the RPC mechanisms to get/set properties and call fgcommands for mongoose.

With all that being said, there really isn't much holding you back from using a corresponding Python module to do just that, i.e. write a Python module to remote-control FlightGear using Torsten's groundwork - and in fact, Torsten encouraged such work on various ocassions.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11376
Joined: Tue Mar 25, 2008 8:40 am

Re: Nasal must go

Postby www2 » Sat Oct 08, 2016 1:47 pm

@Thorsten you mis my point what i say about python vs nasal.
First i say about the better choose for some use case.
I have no problem with nasal use for the front frond end functions as displays, switches, etc.
For systems like real time data where GPS/glonas/galieo sat's in the sky, weather (no offends for nasal) using GFS and real time lightning (using websocket where flightgear is on the clients site of the connection) is a data beside the current Meter reports, and more complex glass cockpits e.g garmin G1000 where the back end modules are writing in python and frond end can be nasal.

edit:
For some cases i wand use a on demand socket interface than from fgfs's startup for client site connections.
www2
 
Posts: 260
Joined: Thu Apr 16, 2009 1:58 pm

Re: Nasal must go

Postby Hooray » Sat Oct 08, 2016 2:04 pm

For systems like real time data where GPS/glonas/galieo sat's in the sky, weather (no offends for nasal) using GFS and real time lightning (using websocket where flightgear is on the clients site of the connection) is a data beside the current Meter reports, and more complex glass cockpits e.g garmin G1000 where the back end modules are writing in python and frond end can be nasal.

FlightGear, and Nasal specifically, does have socket functionality, and it would be straightforward to extend this if necessary, while being much less work than exposing the current degree of FG/Nasal functionality to Python, which is how this works (all imagery is fetched "live" from online web services):

http://wiki.flightgear.org/Canvas_Snipp ... e_tile_map
Image

I am not particularly fond of Nasal, but I am very much fond of managing resources properly - and given the use-case you detailed above, it would take a fraction of the time to turn the Canvas subsystem into an asynchronous component with each Canvas having its own private property tree that merely receives/dispatches events, possibly even with its own FGNasalSys instance to ensure that there is no unnecessary serialization overhead - at that point, you could update Canvas textures ("displays") asynchronously and let OSG's CompositeViewer handle the nitty gritty details of getting each sub-camera drawn/updated without running in the main loop.

As a matter of fact, Nasal's http bindings (those used to fetch the tiled map above), are using TheTom's cppbind framework, so would be easy to extend to also support other use-cases.

Besides, keep in mind that Nasal can even use multiple threads (and in fact, http downloads do already take place asynchronously) - which is to say that it would take only very minor additions/enhancements to make sure that your use-case does not clutter the main loop, whereas equipping Python/Lua/Perl etc with the corresponding FG functionality (as per FGNasalSys) would represent literally years of effort.

Equally, socket I/O could be exposed "on demand" - it would literally require 50 lines of C++ code to make this available to Nasal via cppbind, so it's not like it's rocket science - numpy etc are a completely different beast, and a valid example of something that should definitely run outside the main fgfs process
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11376
Joined: Tue Mar 25, 2008 8:40 am

Re: Nasal must go

Postby Thorsten » Sat Oct 08, 2016 2:08 pm

First i say about the better choose for some use case.


I know that's what you say, but that doesn't make it so - you're expected to bring evidence for your points.

Why is Python the language (aka without libs) superior to Nasal the language - besides the fact that you're more familiar with the first?
Thorsten
 
Posts: 11193
Joined: Mon Nov 02, 2009 8:33 am

Re: Nasal must go

Postby Hooray » Sat Oct 08, 2016 2:15 pm

Thorsten wrote in Sat Oct 08, 2016 2:08 pm:Why is Python the language (aka without libs) superior to Nasal the language - besides the fact that you're more familiar with the first?


I think, the common line of reasoning is basically this:
<devils-advocate mode="on">
  • It has much better, and much more, end-user documentation (in dozens of languages, possibly even Elvish :D ), which we don't have to write/maintain ourselves
  • it has much better tooling to help with troubleshooting, debugging and profiling code (think IDEs, integrated editors for writing/debuggnig code)
  • it also is being actively developed/maintained by an enormous community of contributors, which is to say we don't have to bother updating it, or fixing bugs in the runtime
  • Python is, and will continue to remain, relevant in the software engingeering industry even 30+ years from now (aka, it is more future-safe, and its success/availability is not tied to a single entity/use-case or application)
  • alternate/different GC implementations
  • JIT compilation/optimization opportunities
  • excellent support for threading and running code asynchronously (well, the GIL is there)
  • much better accessibility, and much lower barrier to entry: if I need help with Nasal, it's kinda problematic to expect the other party to have the runtime environment installed - whereas Python is a rather common thing these days, and it is kinda hard to imagine systems without a suitable Python implementation
  • FlightGear adopting industry solutions, and standards, is likely to become more relevant to professional players/companies/contributors (think being multi-platform, using OpenGL instead of DirectX, striving for OSG/HLA or CIGI adoption)
  • FlightGear supporting Python for embedded scripting/extension purposes is more likely to be appealing to educational setups already working with Python (think universities, college, CS/SE courses or aerospace programs)
  • With Python, when compared to Nasal, there is an "infinite" supply of people familiar with Python who use it professionally at their day jobs, many of whom are quite willing to use a language that is used in the real world, whereas learning a "niche" language is unlikely to be too appealing to people who need to make a living coding, simply because putting "Nasal experience" on your CV is unlikely to impress anyone (unless you happen to be an ENT doctor...)
</devils-advocate>
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11376
Joined: Tue Mar 25, 2008 8:40 am

Re: Nasal must go

Postby Thorsten » Sat Oct 08, 2016 2:44 pm

I've heard all that, but that's largely 'Python the user community' or 'Python the environment' rather than 'Python the language'.

As I said, I don't buy into the 'we'll attract plenty of new developers if only...' line of reasoning any more. I grant you that the documentation and tooling is (much) better - but that's hardly the 'killer app enabler' justifying the pain of major architecture changes, new security risks, ...

I get OSG because it enabled a lot of things. I don't see what this enables we don't have right now. Especially if I get to hear 'I want to write a flight instructor module, but I can't do it in Nasal because I really need numerical libraries to do that.' That just doesn't make any sense to me.
Thorsten
 
Posts: 11193
Joined: Mon Nov 02, 2009 8:33 am

Re: Nasal must go

Postby Hooray » Sat Oct 08, 2016 2:55 pm

Ultimately, the language is hardly likely to matter at all given the cirumstances - i.e. as long as the host application (fgfs) isn't updated accordingly, there will always be the equivalent of getprop(), setprop(), setlistener() etc - just using a different language/syntax, with all the pros & cons of using the SGSubsystem approach.

Obviously, people already familiar with coding, are not bothered by having to use a different syntax - but many peope don't have this sort of background, or they simply are not interested in learning a niche language that will never be relevant.

FlightGear is using C and C++ as its implementation language, and some of the early developers actually considered using Java or even Ada instead - arguably, FlightGear's success depended on C/C++ being used at the time (Java still being too slow back then).

So there is something to be said about technology-enablers - like you said a few days ago, had FG had a Python/Ruby or Lua integration, we would simply be using that instead of the Nasal engine. Like you also said, the real question is not whether there are better alternatives, but if there is anything to be gained from pursuing any of those - i.e. in terms of pain/gain to make this work, especially in the light of tons of legacy functionality and features that cannot simply be ripped out of the fgfs core, just because everybody suddenly decides that we'll go down the Python route.

Technically, what would make sense is loosening the degree of Nasal integration, to make it optional (for a variety of reasons, such as troubleshooting/debugging and profiling) - and to rework the base package, and fgaddon to only use certain parts of Nasal (modules), on an opt-in basis. This would be enormously useful to measure the impact of certain features/scripts, and it would allow people to disable party that they don't need.

Sooner or later, it would also make sense to explore reworking FGNasalSys (the SGSubsystem integrating the Nasal engine and implementing it in FlightGear's main loop), so that it can support multiple instances - at that point, it would technically be possbile to use a different instance of the Nasal engine for aircraft specific scripts using a proxy interpreter, and only run vital scripts (core, GUI) in the main thread - and everything else in a different context (think aircraft/scenery).

At that point, these steps could be considered the groundwork to also support other scripting languages using some generic FGScriptingSys baseclass that may run interpreter instances on a worker thread, with an adapter to support different IPC implementation to get/set properties and run fgcommands - which would mean that features like bombable, or advanced weather, would be running outside the main thread - so that their implementation language does not matter at all.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11376
Joined: Tue Mar 25, 2008 8:40 am

Re: Nasal must go

Postby www2 » Sat Oct 08, 2016 3:18 pm

@Hooray I agree what you sat in the devils-advocate mode.
Hooray wrote in Sat Oct 08, 2016 2:15 pm:[list]
[*] It has much better, and much more, end-user documentation (in dozens of languages, possibly even Elvish :D ), which we don't have to write/maintain ourselves

If Elvish not exist in the python docs, one of us can be translate to that languages. ;)
I wait for the Klingon translation. :lol:

The decoupling Nasal from FG we need to create a new subsystem call FGScryptSys that manager the FGstd library for interface to the impotent functions like the proptree, navcash and some of the other function where scripting languages like Nasal and python need only translate the api to his own interface using a template.

For cleaners what i say in this post is more to the anti nasal camp where nasal can use the best.
www2
 
Posts: 260
Joined: Thu Apr 16, 2009 1:58 pm

Re: Nasal must go

Postby Hooray » Sat Oct 08, 2016 3:34 pm

www2 wrote in Sat Oct 08, 2016 3:18 pm:The decoupling Nasal from FG we need to create a new subsystem call FGScryptSys that manager the FGstd library for interface to the impotent functions like the proptree, navcash and some of the other function where scripting languages like Nasal and python need only translate the api to his own interface using a template.


You can really only make progress on the Python front if you accept that Nasal is there to stay until Python works reasonably well without causing tons of regressions, absent that, there will be (at least) two co-existing scripting solutions, which is likely to complicate the situation even further - especially in the light of the mess that $FG_ROOT/Nasal and aircraft-side scripts have become, despite Nasal being an unpopular niche language.
Otherwise, supporting Python would only make the shortcomings in FlightGear's legacy architecture even more prominent, quite possibly within just a few months, because Python has a much lower barrier to entry, so that people will be able to create all sorts of functionality, and in the process continue to clutter the main loop and make it even more non-deterministic than it is already.

The first step really is coming up with a mode to entirely disable nasal using something like --disable-nasal, and the next step is to rework the Nasal initialization sequence, to move away from the hard-coded assumption that the whole system is up and running and that all modules are always loaded - at that point, a non-Nasal mode can make actual progress, because FlightGear would no longer necessarily depend on Nasal - sooner or later, FGNasalSys could be reworked to support multiple instances, so that aircraft scripts can be treated, and executed, separately - i.e. separate from other simulator scripts (think GUI, scenery), and it would become much more straightforward to run certain parts on helper threads, or even introduce private property trees for certain purposes (think one private property tree for the FDM/AP, one for Canvas textures (MFDs)).

All of this could greatly benefit from the Phi/mongoose groundwork that Torsten completed, because it would provide for a sane mechanism to get/set properties and run fgcommands, without requiring Nasal inevitably.

However, we really don't want to have another scripting interpreter running inside the main loop/thread - that would only add even more work to our todo lists.

<SCNR mode="on">PS: I kinda like the whole notion of an "impotent navcash" which sums up the situation rather well if you ask me :D (no offense, I don't even speak French!) </SCNR>
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11376
Joined: Tue Mar 25, 2008 8:40 am

Re: Nasal must go

Postby www2 » Sat Oct 08, 2016 4:52 pm

Hooray wrote in Sat Oct 08, 2016 3:34 pm:
www2 wrote in Sat Oct 08, 2016 3:18 pm:<SCNR mode="on">PS: I kinda like the whole notion of an "impotent navcash" which sums up the situation rather well if you ask me :D (no offense, I don't even speak French!) </SCNR>

No offense from me I speak no world French even i have a French name.

For the FGScriptSys we can do this simultaneous with FGPythonSys, where FGPythonSys enable Python and FGScriptSys for the basic API between FGFS and a scripting language and on a later moment we can port FGNasalSys to FGScriptSys.
www2
 
Posts: 260
Joined: Thu Apr 16, 2009 1:58 pm

Re: Nasal must go

Postby Thorsten » Sat Oct 08, 2016 5:25 pm

I guess the difference is largely between a fantasy world in which one can simply 'switch' things without a huge cost in manpower, bugs, regressions, you name it and the real world in which these things actually happen and it's darn difficult to get anything bug-free and in which regressions continue to cripple us.

Would FG be in a better shape if the scripting language would not be Nasal? Possibly. Do the benefits of any change outweigh the cost? I haven't seen anything to convince me of a yes - if you guys would bother to spend a week learning Nasal, everything could be there at a tiny fraction of the cost in manpower and bugs. To call for huge architecture change because you can't learn a new language is not efficient in my book, and this makes only sense in a fantasy world in which there's infinite manpower and zero switching cost.

And for that simple reason - not going to happen.

Okay, enough of my time spent here - you go on spinning fantasies, I have to develop real stuff.
Thorsten
 
Posts: 11193
Joined: Mon Nov 02, 2009 8:33 am

Re: Nasal must go

Postby curt » Sat Oct 08, 2016 6:04 pm

I think the difference here Thorsten is that the original poster is making a case for Python and *hoping* to attract support from the development community. I don't think we need to call that a fantasy world, it's just someone making a case for an idea that they have, and someone who wishes to talk through the idea out loud.

You are right though that anything like this takes real effort. Many of the seasoned developers understand the level of effort this would require and the impact in terms of breaking many things in the short term. And when I say short term, I mean until the first fully working release of the new system with the new scripting engine which could be a full year or more of dedicated effort by someone. You are right to suggest that the payoff may not be worth the effort or pain. But it's ok to discuss this without automatically telling people they live in a fantasy world and trying to shut down the whole conversation.

It is fair to point out something that a lot of end users may not fully realize. In an open-source project, it is almost always the person with the idea that ends up doing the work. It is actually quite rare to convince someone else to volunteer a year of their life for your own idea. (Exceptions might be if it is a really really good idea that someone else has already pondered and your input is the tipping point, or maybe it is a quick fix or an easy change.) Again, it's fair to discuss it, it's fair to lobby for your idea, but the original poster needs to understand the magnitude of what they are suggesting, and if they aren't willing or able to step up to the plate themselves, they can't feel bad about not convincing someone else to do the work for them. That's just how it is when everyone is volunteering out of their limited spare time.

It does sound like you (Thorsten) have spent more than enough of your own spare time discussing this. I don't think you have to worry about someone unilaterally deleting nasal support in favor of python and cutting off all your projects at the knees. No need to defend turf here. So if you've said your opinion on the subject, then you are better off focusing on real stuff like you say.

From my own perspective, thinking purely on language syntax + interpreter engine footprint + seamless integration with flightgear + language capability/completeness: nasal may still be the overall winner. Nasal is brilliant in a number of ways. But to be fair, those are not the only considerations. Don't be too quick to discount the community of knowledge, the documentation, the library support, ease of installing new libs with new functionality ... there are many reasons to consider a choice.

For example: I might take a slightly longer way home from work so I don't have to take a left turn at an uncontrolled intersection during a busy time at rush hour. Technically the first route is shorter, maybe even on average quicker, but perhaps I also consider stress levels and safety as part of the overall equation. That isn't a direct analogy to nasal vs. python, just pointing out that a person's final decision may weigh additional more subjective factors, and that a normal thing that we humans do.

Best regards,

Curt.
Aerospace Engineering and Mechanics
University of Minnesota
curt
Administrator
 
Posts: 1174
Joined: Thu Jan 01, 1970 12:00 am
Location: Minneapolis, MN

Re: Nasal must go

Postby Hooray » Sat Oct 08, 2016 6:20 pm

Thorsten kinda hit the nail on the head. He could have been more polite - but sometimes your opinion can be unpopular, while your still right - and sometimes you can be wrong, and still be very popular.

This being the "infamous merito-crazy" we are, we also tend to weigh feedback based on who's making certain statements, and especially requests and suggestions that require tons of work...

You/we cannot expect any contributors familiar with Nasal, and its internals, to immediately jump on the Python bandwaggon - but for the whole thing to work out, you need to get all parties involved. First and foremost, this means understanding the goals and priorities of all the people involved in "FlightGear scripting" (in general, unrelated to any particular solution).

Next, you need to understand what the legacy functionality, i.e. functionality that we simply "inherit" and that must (mostly) continue to work "as is", without causing tons of work - imagine for instance how someone like Thorsten is going to respond if I start telling him that his weather/ALS/shuttle code needs to be rewritten in Python, or that it has been automatically transformed, but that roughly 500-1000 lines of code need to be manually reviewed/debugged and tested, or that he needs to install some Python libraries to make everything work again.

Thus, the first step really is understanding who's got any stakes in FlightGear scripting, and to what degree, in what form and how that translates into workload/priorities (for instance, advanced weather being part of the base package is arguably going to be considered more important than some 3rd party aircraft outside fgaddon, or than the bombable addon - yet, the corresponding contributors will definitely yell at us if we break something that previously used to work fine with Nasal).

So, realistically, "porting scripts" isn't going to be a relevant discussion anytime soon.

If I was seriously interested in seeing support for other scripting languages, such as Python, my priorities would be different:

I would try to review what we have in terms of functionality, and who's involved, what must continue to work - with Python it seems, two important key players would be Curt and bugman, so that it'd be good to have them on your team. For them to be on your team, you must understand their goals, requirements and priorities.

Next, you would need to understand the priorities of those who have major Nasal stakes (e.g. the MFD/Canvas folks), who will also not be too happy if some commits revert much of their work to some pre-OSG/mid-2006 state.

And then there is the most obvious challenge, that of aircraft scripting - no matter if that means $FG_ROOT, FGAddon, fgmembers or any 3rd party hangars - this is the sort of stuff that is usually beyond the control of those making such changes, so that it is easy to break tons of aircraft without ever noticing it, because there is no good way to automatically "test" an aircraft and its features in isolation.

Thus, realistically, anything involving a new/additional scripting language will need to be aligned well within the constraints of the current environment, which means keeping Nasal for the time being, but detangling the mess this has become over time, and recognizing why this is so.

For instance, massive fgaddon/aircraft scripting and fgdata addons developed in scripting space are primarily a sympton of middleware developers wanting functionality that core developers don't prioritize developing (e.g. bombable or a dedicated weather system).

So, the surprising adoption rate that Nasal is, and has been, seeing is merely a symptom of the disparity for the functionality gap between what the core developer community can/wants to implement, and what middleware contributors, and end-users want to see/use FlightGear for, regardless of the reasons (e.g. some contributors don't appreciate having to deal with the devel list, while some core developers don't particularly appreciate having to deal with the forum/end-users), as long as this disparity exists, there will be an incentive for such "modules" or "plugins" to be developed, and it makes sense that scripting be used for this.

In other words, the use of scripting in FlightGear's base package, and aircraft/scenery in particular, is kinda symptomatic for the kind of modding functionality, and community, that FlightGear could -in theory- cultivate over time, which is surprisingly massive given the niche nature of Nasal as a language and extension mechanism, especially compared to more mainstream languages like Python or Lua.

Consequently, Thorsten's comments regarding the potential proliferation of addons and its ramifications is absolutely valid, and given how FlightGear scripting in its current "primitive" form has impacted FlightGear's evolution (think having a scripted weather system instead of a hard-coded one, think having a scriptable 2D drawing API for creating hardware accelerated MFDs, think having a scripted dogfighting addon etc), the ramifications of providing a more established scripting solution need to be carefully discussed from a variety of angles (think security, package/dependency management, threading, I/O etc).

What has been happening to many FlightGear subsystems over the course of the last 10+ years is that its developers recognized that they're falling victim to what an "architecture astronaut" would describe as the "singleton fallacy", aka, subsystems that were originally designed, developed and maintained with the implicit and hard-coded assumption of there ever being only a single entity/subsystem managing certain entities.

More specifically, this means in non-coding terms that simulation related concepts like that of an aircraft, FDM, autopilot etc are implemented in a way that prevents the simulation from ever having more than one instance of said component, which is kinda like the difference between using one sheet of paper per telephone number or creating a list to keep multiple phone numbers, or even maintaining a full phone book.

With FlightGear, there is actually a long and impressive history of this Singleton Fallacy being incrementally recognized, discussed and then addressed over time, such as e.g. FlightGear being originally designed with there only ever being ONE:

  • CPU
  • aircraft
  • joystick
  • FDM instance
  • autopilot
  • route manager
  • view
  • fgfs window
  • fgfs instance
  • machine running fgfs
  • weather engine
  • renderer
  • scenery/terrain engine

These days, this is no longer a safe assumption to be made, and in fact it's rather restricting one that is turning out to complicate FlightGear's future evolution rather significantly, because the real world has evolved very differently - with multicore and multi-gpu hardware being available at Walmart's, it's no longer farfetched to imagine a distributed setup of FlightGear running spread across multiple computers, with multiple instances, multiple windows, each showing different (possibly independent) views.

When it comes to scripting, FlightGear is also designed for having a single mainloop that sequentially updates its subsystems and a single subsystem implement scripting support, all of which end up updating a single monolithic property tree.

This is unnecessarily complicating matters for people interested in more sophisticated use-cases, one of which would be using alternative scripting solutions.

But for any of that to become an actual option, FlightGear needs to evolve beyond the point where there is only a single scripting interpreter, in the form of just one Nasal execution context, because that's hugely chaotic from a scripting standpoint (just imagine all JavaScript code in all your browser tabs would be running in the same context, i.e. facebook, twitter and ebay code running all in one context).

We kinda saw the same thing happening with Microsoft Windows which used to have the implicit assumption of there only ever being a single user/account, whereas Linux evolved differently, as a multi-user system, carefully designed to support this use-case.

So far, FlightGear has -in many parts- accidentally been developed into a rather monolithic singleton-monster that has tons of hard-coded assumptions that make it look more like a product created by Microsoft than opensource afficionados - not unlike the first internet browsers, where you had to open a new instance to view another website (no support for tabs back then).


And this shows particularly in the reset/re-init department, i.e. the whole initialization sequence is a rather massive mess that is complicating any efforts to re-architect FlightGear to support other use-cases (or "startup modes").

This is one of the reasons why it is so important to deal with Nasal first prior to dealing with FGPythonSys: FlightGear has the hard-coded assumption that there will always be a full simulator session up and running, i.e. it really is much more akin to what Microsoft Windows used to look like in the early 90s than what Linux looked like (it already having an incremental, and configurable startup sequence using a combination of scripts and so called "run-levels").

If you don't recognize that this is the case and why that is the case, i.e. the legacy codebase that we have to deal with, you are only going to add one more problematic subsystem to FlightGear, with the major exception that this particular subsystem would be a massive technology enabler to help de-skilling (lowering the barrier to entry) contributing even more, with tons of resources available that would enable people to go literally crazy and use an extremely popular language in all kinds of places in FlightGear, without understanding the ramifications of doing so.

Almost certainly, this degree of scripting support would not just cause a proliferation of addons and optional modules, but it would also cause tons of challenges on the security front, and package/dependency management (imagine having to install Python modules to run a certain aircraft/scenery tile or AI scenario etc).

Over time (as in 3-5 releases), FlightGear would turn into a bloated, bug-ridden and fragile application that runs slower than molasses, lagging like hell, where deploying FlightGear resources would become a nightmare quickly. Just imagine for a second Python had been used instead of Nasal, and Thorsten had used all sorts of Windows-specific Python modules to implement his contributions (weather, ALS, earthview, shuttle), simply because his nvidia GPU only works under Windows, and he cannot do any testing under Linux.

In theory freedom of choice is a great thing, in practice that also depends on the repercussions - just look at the components where freedom of choice is actively encouraged and cultivated (3D models, FDMs, weather, GUIs, scenery engines, Rembrandt vs. ALS etc).

This sort of "freedom" always has repercussions - and that may be acceptable for end-user functionality, but it is hugely problematic when this freedom competes with other mechanism to develop the program in a consistent fashion - just look at what Nasal adoption has done to core development, and how FlightGear middleware development has been literally boosted by making scripting available to people who no longer need to patch/rebuild fgfs. In fact, look at the Canvas system it has literally KILLED OFF any core development of glass cockpit gauges in its entirety. And at the same time, Nasal scripting and the Canvas system don't translate into immediate benefits for ongoing core development other than the end-user/forum community no longer having to deal with the devel list to implement certain functionality (and vice versa), and looking back in time, things turned out rather crazy looking at the messy situation.

Meanwhile, Nasal is massively complicating certain core development efforts, such as the reset/re-init effort, because it's a feature that is there to stay, and that must continue to work - but it's encouraging such a "free form" of contributing that the core engine does not even "know", let alone "understand", what a script is really doing (in terms of functionality/dependencies) - which is very much empowering for middleware contributors but a PITA for anybody wanting to modernize the simulation to better formalize and standardize certain features (as in not loading unneeded modules, keeping the garbage collector "pressure" low, or running scripts on different threads).

At the end of the day this also means that this freedom has repercussions beyond just being a PITA for core developers to deal with, because it is this very freedom of choice that is having repercussions on performance and complexity of the simulation.

Adding another, and potentially more popular, scripting option to the equation using the same approach is a particularly ill-informed move, especially because this is not just free-form functionality that is added, but functionality that can be used to add even more free-form functionality and so on and so on.

No matter if you hate Nasal or not, you cannot realistically make any progress with Python without also looking at Nasal and its shortcomings, and the very first steps will involve reviewing what is needed to make it incrementally optional so that parts of Nasal can be disabled/detangled and moved to other cores.

And for that to happen it would make sense to identify a common subset of people interested in FlightGear scripting.

Absent that, this will just be another "pie-in-the-sky" effort - that even if it should end up in the commit logs, it will be an optional feature that doesn't integrate well with the rest of FG, and that merely adds to it being even more fragile than it already is.

Thus, this whole idea has much more to do with being willing to make concessions and find compromises than doing actual coding, because the people interested in non-Nasal FlightGear scripting will inevitably need to team up with Nasal folks (experts!) to make all this work, without breaking tons of Nasal functionality. And for that to happen, there need to be benefits for the Nasal folks, such as groundwork that translates into improvements to the general FlightGear scripting integration - so that it can support multiple instances, languages, threads and use IPC to talk to the rest of FlightGear, quite possibly introducing multiple private property trees for certain subsystems to make the whole thing tangible.

But that is aking to asking two countries at war to start peace negotiations ...
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11376
Joined: Tue Mar 25, 2008 8:40 am

PreviousNext

Return to Nasal

Who is online

Users browsing this forum: No registered users and 0 guests