Board index FlightGear Development Nasal

Nasal must go

Nasal is the scripting language of FlightGear.

Re: Nasal must go

Postby Johan G » Fri Mar 17, 2017 7:26 pm

Forgive him but he, like many other, are tired of seeing the same arguments repeated over the years with often only superficial comparisons done. While one could argue that the arguments would have a good foundation as they are often repeated, the amount of repetition does not really say much about the arguments.

Also, Thorsten have spent a lot of time* trying to profile Nasal's bottlenecks and trying to find the more effective workarounds to make complicated code to not lag.

* Frankly much more time than anyone else I can think of. On the other hand some of his contributions are among the more complex things done in Nasal.
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5492
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

Re: Nasal must go

Postby adamstyl » Fri Mar 17, 2017 9:09 pm

Hi Johan G,

There is nothing to forgive, I'm not here to start a cyber fight. Just to share my POV. I tried to explain to the OP (if he's still reading) that there are other things beyond "python rocks nasal suck$". For good or for bad that was the choice made, and the project "married" it. Divorcing is difficult. And yes I also expressed my opinion that a more mainstream scripting language could have given the room and opportunity of experimentation which now doesn't exist. There are no implications for no-one's work and nothing hidden behind the my words.
adamstyl
 
Posts: 6
Joined: Fri Mar 17, 2017 5:30 pm

Re: Nasal must go

Postby Thorsten » Fri Mar 17, 2017 9:25 pm

Everyone is entitled to his opinion.

I've just come to value positions based on long experience with the code and tangible test data much more than opinions.

There's for instance no shortage of people who explain in the forum how FG rendering really needs to be done, but there is a shortage of people who can actually come up with the code.

Translation: It's easy to talk about your favourite scripting language having a much better performance than Nasal. However, it should give you pause that people who are familiar with the way Nasal and FG interact never do so but consistently point towards the real issues.
Thorsten
 
Posts: 10844
Joined: Mon Nov 02, 2009 8:33 am

Re: Nasal must go

Postby adamstyl » Fri Mar 17, 2017 9:55 pm

Low doer to talker ratio. Known story. The main reason I develop my radar sim alone.

Thorsten wrote in Fri Mar 17, 2017 9:25 pm:Translation: It's easy to talk about your favourite scripting language having a much better performance than Nasal. However, it should give you pause...


You still think it's about me? You are missing the big picture. And I already wrote that I hate scripting languages. Now you're the one not reading.
adamstyl
 
Posts: 6
Joined: Fri Mar 17, 2017 5:30 pm

Re: Nasal must go

Postby Thorsten » Sat Mar 18, 2017 6:48 am

Regardless of whether you hate scripting or not (which is a question of taste which I don't really care about), you made a performance argument (and I do care about arguments). For your convenience, let me quote it back to you:

if js was chosen back then, now (and assuming for a moment there are no licensing issues) V8 could have been harnessed to FG and scripting performance could have rocketed sky high.


I replied to that argument as I consider it far more relevant for the discussion than your personal likes/dislikes, and I highlighted the general theme that claiming that 'if X had happened or would happen, Y would be much improved' is relatively cheap - but there's usually no evidence offered that the claim as made is true.

You'll perhaps understand that as you don't seem overly familiar with the FG code itself, I won't take your opinion on what would improve its performance particularly seriously and will continue to listen to what the people who have actually worked with the code for a long time have to say instead.
Thorsten
 
Posts: 10844
Joined: Mon Nov 02, 2009 8:33 am

Re: Nasal must go

Postby Hooray » Sat Mar 18, 2017 7:18 am

I don't think Thorsten was being negative at all - adamstyl made a number of valid observations (and of which have been previously mentioned in this and other threads), but also made a number of invalid obvervation, all of which have been previously addressed and discussed here, too - so it is easy to get the impression that adamstyl didn't actually read up on the topic at all.

I belong to the camp of people who actually have replaced Nasal once (for all the reasons previously mentioned), and I know how to completely disable the whole subsystem (patched posted here and in the wiki).
Equally, I understand how the FlightGear/SimGear side of this works, and how all of this relates to the FlightGear main loop, and most of its subsystems working in terms of so called "SGSubsystems" - all of which is to say that Thorsten is right: we could have v8/lua/python support within a few days of work - however, the real bottleneck really is the FlightGear main loop and its way of implementing subsystems - Nasal is hardly relevant in the mainstream world obviously, however that could also be said to be an advantage here - because despite Nasal's niche role, it still is very accessible - but a more mainstream language like Python, JavaScript or Lua would make FlightGear's architectural issues much more prominent within a few months.

None of that is to say that I oppose any efforts related to supporting other/additional languages, potentially with the goal to phase out Nasal entirely at some point in the future.
However, as has been previously said by a bunch of folks intimately familiar with the FlightGear/SimGear code base, the Nasal engine and C++ in general: if/when Nasal is moved to a dedicated worker thread(s), all of its runtime issues (think garbage collection overhead) would go away immediately - that involves the whole HLA/RTI (SimKit) development - and that development could also greatly benefit FGPythonSys or similar efforts.

In other words, it's a ROI issue - i.e. how much time/expertise and manpower we have to invest to address a certain problem - it could be spent implementing Python/Lua or JavaScript support, but that would only make FlightGear architectural issues more prominent - what is really needed is a way to implement and run asynchronous subsystems.

Ironically, Nasal was designed with this in mind - right from the get-go, unlike FlightGear. Which is to say you can write asyny/multi-threaded scripts in Nasal without using a GIL (Global Intepreter Lock).

Obviously, Lua/Python support could be great technology enablers - but we're ignoring the real issue, which has nothing to do with Nasal at all.

Besides, I also find it slightly disrespectful that we're having to rehash the same arguments here, at least the poster could have said that he didn't spend any time looking up what's been previously said in this thread - having to summarize the discussion and having to re-present the facts is slightly annoying.

Had adamstyl looked up the history of Nasal or even just the beginning of this topic, he would have realized that his stated conclusions/facts are false - e.g. that prior to Nasal, FlightGear did have experimental support for other scriptign languanges - including JavaScript (prototyped by Erik back then IIRC). Equally, while the argument about Nasal's lack of relevance does remain valid, Nasal isn't solely used by FlightGear (search the forum for AlgoScore).

Speaking in general, if were to dedicate XX % development resources to "fix Nasal", that would hardly be about Nasal the language/the engine - but about FlightGear's main loop, at which point you don't even need to touch any Nasal engine/VM code (or scripts).

The real issue is that these architectural issues are there and that we have been encouraging a fairly free-form scripting style, which is conflicting with this architectural design - however, Nasal already supports threads - and while none of its existing SG/FG extension functions are thread-safe - the Nasal library itself is intended to be thread-safe by design. Which means that it would be possible to write asynchronous Nasal code that runs outside the FlightGear main loop.

Ultimately, the real bottleneck is the way FGNasalSys is implemented as a subsystem on top of timers and listeners, which is a highly fragile (and non-deterministic) way to implement features/subsystems inside the FlightGear main loop.

Thus, if people are serious about supporting Python, Lua or JavaScript - the first step involves getting FlightGear to build, and run, without any Nasal code running: http://wiki.flightgear.org/Howto:Disable_Nasal_entirely

The next step involves reviewing incremental subsystem re-/initialization: http://wiki.flightgear.org/Howto:Reset/ ... leshooting
This may also involve formalizing the subsystem dependencies between Nasal code and native C++ subsystems (think geodinfo requiring the terrain/scenery subsystems) http://wiki.flightgear.org/Initializing_Nasal_early

At that point, it would become possible to come up with a highly restricted startup mode that does not involve much/any Nasal code.

And then your are approaching what FGPythonSys is currently facing: the realization that supporting the language is one thing, but that fixing up FlightGear's architectural issues is a completely different challenge - and that has zero to do with any Nasal code or the Nasal engine per se.

If in doubt, do a forum search for "FGPythonSys" (or just check its summary in the wiki) - and maybe ask for bugman's opinion, as he is actually experimenting with Python support in FlightGear.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11326
Joined: Tue Mar 25, 2008 8:40 am

Re: Nasal must go

Postby adamstyl » Sat Mar 18, 2017 11:19 pm

Thorsten wrote in Sat Mar 18, 2017 6:48 am:Regardless of whether you hate scripting or not (which is a question of taste which I don't really care about)

If you don't care why do you read?! And why bother answering? Sometimes it's as simple as that!

Thorsten wrote in Sat Mar 18, 2017 6:48 am:I replied to that argument as I consider it far more relevant for the discussion than your personal likes/dislikes, and I highlighted the general theme that claiming that 'if X had happened or would happen, Y would be much improved' is relatively cheap - but there's usually no evidence offered that the claim as made is true.

There is no thesis (claim) in anything, there is hypothesis. Learn the difference. I don't know the quality of your English but in the text that you've quoted the word could appears twice. Are you familiar with the word's meaning?*

Thorsten wrote in Sat Mar 18, 2017 6:48 am:You'll perhaps understand that as you don't seem overly familiar with the FG code itself, I won't take your opinion on what would improve its performance particularly seriously and will continue to listen to what the people who have actually worked with the code for a long time have to say instead.

Again don't listen to me. You are not compelled to, and besides I wasn't addressing to you I was addressing to the OP. But for Christ's sake don't start a flame war.

I don't know if you are professional developer, but you are certainly amateur with your manners. You are rude, ironic, selfish and definitely I'm glad I never had colleagues with your attitude.
One question: If everything has been discussed, your Majesty is the only one to possess the one and only Truth, why don't you ask for the thread to be locked?

*could: past of can; used to indicate possibility possible: that may exist or happen, but that is not certain or probable.


Hooray wrote in Sat Mar 18, 2017 7:18 am:I don't think Thorsten was being negative at all

No he was rude

but also made a number of invalid obvervation

What is invalid observation? That this is an in-house language? That it's not maintained? That it's not evolved? That it's not used by anyone else? And yes I've heard you about AlgoScore but if you compare it to the millions of users of js or python then again it's a drop in the ocean.

Let me put it another way: if a mainstream language has been used then in http://wiki.flightgear.org/Portal:Developer you wouldn't be having "Fixing the Garbage Collector (the GC in Nasal needs to be improved)" under help needed section. Why? Because someone else solves these problems for you.

so it is easy to get the impression that adamstyl didn't actually read up on the topic at all.

A few words for that: I was reading several articles in the wiki about FG scripting. My eyes were blistering. I've also skimmed through the thread. Sorry for having a life and not having the time to read everything and in every detail things that others spent years to write. I've landed on that page looking for an experimental V8 adoption in FG that I remember I was reading a few years ago somewhere. I was looking if something happened with that but the thread is gone. I'm sorry for not having enough time to "google harder".

Now it seems that my mistake wasn't to land to this thread, it was to explain the OP (if it's still not clear) that things in real software development aren't as simple as that.

I belong to the camp of people who actually have replaced Nasal once (for all the reasons previously mentioned), and I know how to completely disable the whole subsystem (patched posted here and in the wiki).[...]we could have v8/lua/python support within a few days of work

At last I've learned something useful from someone!

a more mainstream language like Python, JavaScript or Lua would make FlightGear's architectural issues much more prominent within a few months.

I don't know where do you base that assumption.

I also find it slightly disrespectful that we're having to rehash the same arguments here[...]- having to summarize the discussion and having to re-present the facts is slightly annoying.

Disrespectful??? Rehash? Summarize? Have I explicitly asked "ah someone rehash and summarize because I'm lazy reader"?

Had adamstyl looked up the history of Nasal

And where exactly can I do that? The language's web site is down... I visited this http://wiki.flightgear.org/How_the_Nasal_GC_works. And it's full of forum quotes that seem irrelevant to the How the Nasal GC works thing. Sorry but I won't read it. It's repelling.

Bottom line, I said something that seems to have outraged some people, that having V8 harnessed, it might have been faster. I might be wrong. Good. Did anybody try it? Is there a benchmark, comparison whatever? I was looking for it... If there is please have the good kindness to point me there and I will gladly read. Otherwise we are just rumbling. Thanks for the lengthy answer and your time, although from the 3rd person I understand you were not addressing to me...
adamstyl
 
Posts: 6
Joined: Fri Mar 17, 2017 5:30 pm

Re: Nasal must go

Postby Thorsten » Sun Mar 19, 2017 6:47 am

There is no thesis (claim) in anything, there is hypothesis.


I think the vast majority of readers will agree that this

I don't believe that nasal performs faster than V8 which compiles to machine code. ]

is a claim that V8 performs faster than Nasal. Otherwise, what's the point of posting it?

(If you like semantics, it's an expression of a rather firm belief, to be contrasted with e.g. 'Could it be that Nasal does not perform faster...?' which is an expression of a hypothesis).

don't start a flame war.


Hm, we get from you things like

* No he was rude
* , but you are certainly amateur with your manners. You are rude, ironic, selfish and definitely I'm glad I never had colleagues with your attitude.

Most readers will readily recognize these as ad hominem rather than a reply to a point made in a debate. Kindly just refrain from speculations about the character of other forum members, that's not how this community works.

In contrast, you'll look in vain for any negative remark about your character in my postings. Go figure who is about to start a flamewar.

I wasn't addressing to you I was addressing to the OP.


You're posting in a public forum, everyone can read what you write. If you want to talk to someone without others replying, may I suggest using the PM function instead? Otherwise, I suggest you live with people replying to you if they disagree with what you say.
Thorsten
 
Posts: 10844
Joined: Mon Nov 02, 2009 8:33 am

Re: Nasal must go

Postby Thorsten » Sun Mar 19, 2017 7:02 am

*could: past of can; used to indicate possibility possible: that may exist or happen, but that is not certain or probable.


Doing liguistics as a hobby, I can't resist pointing out that 'could' is not necessarily a past form, and definitely not in the sense you want to use it.

Compare: _I could go to the library now._
with: _I could do differential equations ten years ago (but today I no longer can)._

In the first sentence, it manifestly doesn't carry a past meaning as evidenced by the expression of time. Usually both _can_ and _could_ in the former context are classified as modal verbs.
Thorsten
 
Posts: 10844
Joined: Mon Nov 02, 2009 8:33 am

Re: Nasal must go

Postby Hooray » Sun Mar 19, 2017 10:05 am

@Adam: You sound like you want to argue, whereas I previously stated quite clearly that share certain *assertions* you are making, as in adopting a more mainstream language like Python, Lua or v8/ECMAScript (nodeJS) would mean that certain problems (you mentioned the mark/sweek GC) would simply not exist, equally compared to Nasal, v8 definitely has a faster runtime profile, mainly because of the fact that it is creating machine code and as a fairly sophisticated code generator/optimizer.

However, like I -and others- stated previously - those are rather low-level Nasal issues, you literally need to write tons of systems-level code in Nasal to actually encounter any of those - what is really limiting FlightGear's evolution of scripting support is the legacy main loop and its way of having a single sequential vector with subsystems whose ::update(double dt) method is invoked accordingly - it is this that is causing stuttering, and that requires the FDM/AP to be running carefully interleaved, and it is also this that means that Nasal code may impact/affect not only the framerate/spacing (latency), but also other crucial systems (think FDM/AP).

And yes, that is exactly why I previously said it would be annoying to have the feeling that we're required to rehash previously stated comments, especially those backed up with facts.

You mentioned a number of wiki articles and how they're suffering from extensive quoting, here is the breaking news: I am the right person to complain to, because I happen to be the culprit copying relevant information over to the wiki, usually trying to retain references to the topics where certain things have been said - so that people interested in working on certain features, can more easily find the corresponding information. Besides, I am also the one who originally added the "Need for help (GC)" section that you are referring to (but that doesn't mean anything) - so you are preaching to the choir, because I am well aware of all of this, and quite a bit more.

So, let me reiterate: adopting v8/Python or Lua would only make FlightGear's architectural issues much more prominent than they already are - Nasal is quite a bit more prepared for working in a non SGSubsystem based architecture than is currently the case - however, Nasal's niche nature is masking/shielding and isolating the project from an even higher rate of adoption, which is not necessarily a bad thing because the majority of active core developers consider the sheer rate of adoption that Nasal scripting has been seeing highly problematic for a variety of more or less valid reasons - supporting Python or Lua according to the FGNasalSys runtime model would not be difficult, but it would be highly problematic due to the repercussions that would entail.

We have already exceeded the point where we can simply stop telling people to use Nasal (or Canvas!) in a certain way - however, this fgdata/fgaddon level scripting is more often than not highly problematic for people wanting to make progress in the core development department (imagine wanting to be able to save/load and resume flights, do proper simulator re-initialization, doing proper multi-instance state synchronization/replication for multiplayer or FSWeekend/LinuxTag-like setups).

These use-cases are next to impossible to support without also having a well-defined interface in place that all extensions mechanisms use - however, Nasal is so accessible and so easy to use that people will simply use it without paying much attention to core developer requirements - this problem would only be magnified by people using Python or Lua.

Obviously, that is not say that Python or Lua support would be bad - or that the Nasal/GC would exist today if we had access to Lua/Python - but the real issue isn't unlike JavaScript running in a tabbed browser with one thread per tab: You don't want flightgear.org JavaScript code to affect your home banking and vice versa.

In the case of FlightGear, the project has failed to come up with a properly isolated execution model that supports multiple execution contexts (think aircraft code vs. scenery code vs. gui code vs. systems code) and potentially multiple threads (despite Nasal having threading support) - bringing Python/Lua to the table would lower the barrier even more (think better tooling, documentation, examples) so that development of 3rd party aircraft/scenery addons would surely explode within months, causing a real issue for people developing the C++ code (Curt and others acknowledged on various ocassions that being the case)

In reality, Nasal is being widely used by content developers because it's way easier to use than JavaScript and will get the job done - but that also means that the solutions that people come up with may often conflict with goals that core developers have, but that would be no different should a different language be adopted - unless that language happens to actually run outside the main loop using an IPC mechanism and well-defined extension mechanisms so that there is no way for such scripts to interfere with the rest of the simulation in any adverse way.


Bottom line, I don't think anyone is outraged at all - Thorsten and I were just expecting that you would more carefully state your opinions/assertions or try to back up your statements with data/facts, so that we can review those and make up our own minds. With your kind of background (C#/Windows, right?) your help could definitely be useful for anything involving Windows testing/development or even just troubleshooting - equally, if you disagree with the quality of the docs (or the wiki specifically), you are invited to help clean up such articles by reviewing/improving such pages.

To explain myself: Let's assume you don't have much of a background in 3D rendering (which I don't), however you see something useful that may be of interest to people wanting to get involved, and you cannot realistically expect them to search the archives for postings made 5+ years ago (heck, because they may actually have a life) - but you may lack the time/insight (or motivation) to actually rewrite useful information (that may become out of date over time) - thus, the easiest way for me to preserve such information is to copy it over to the wiki, where it may linger until someone "adopts" such additions by reviewing/copy-editing and improving/maintaining them - e.g. see: http://wiki.flightgear.org/Howto:Shader ... discussion

Now, if you are familiar with 3D programming and effects/shaders in FlightGear, you won't find any of that useful and even consider it an asthetically unpleasant addition to the whole article - Thorsten keeps pointing that out, but most people not having Thorsten's background may actually find that helpful.

Equally, there are a number of core developers who have stopped documenting their additions to the core code in the form of dedicated README.* files added to $FG_ROOT/Docs, and who more often than not simply announce their additions well after the fact, without ever providing/updating or maintaining much if any documentation at all, so that quoting the archives may be the most straightforward way to "bootstrap" new articles by allowing wiki contributors to help review those additions and update the wiki accordingly, e.g. see: http://wiki.flightgear.org/FlightGear_Qt_launcher

And then there are "side-efforts" like FGPythonSys whose developers actively discourage quoting, but apparently still seem too appreciate that their efforts are being summarized that way until they find the time/motivation to actually create dedicated documentation, which can be seen be repeated postings on the forum/devel list linking back to such quote-based summaries: http://wiki.flightgear.org/FGPythonSys

Like you said, it's a pain/gain thing, and all of us have a life - which is why quoting is such an easy way to retain potentially useful information, until it is reviewed and properly edited/cleaned up.

For the full background on quoting, and why it is being used despite being considered problematic by most people, see: http://wiki.flightgear.org/FlightGear_wiki:Instant-Refs
(Ironically, that's using code written in JavaScript ...)

PS: Here's some friendly advice: don't try to argue with Thorsten, it will just end badly - if in doubt, look at my own posting history and any arguments between him and myself :D
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11326
Joined: Tue Mar 25, 2008 8:40 am

Re: Nasal must go

Postby Hooray » Sun Mar 19, 2017 1:28 pm

explain the OP (if it's still not clear) that things in real software development aren't as simple as that.


Anybody wanting to understand how subsystems in FlightGear work, and how the Nasal scripting interpreter specifically works (and how that relates to supporting other subsystems, including those supporting other scripting languages like JavaScript, Python or Lua) check:


http://wiki.flightgear.org/The_FlightGear_Main_Loop
http://wiki.flightgear.org/Callbacks
http://wiki.flightgear.org/Howto:Create_new_subsystems

http://api-docs.freeflightsim.org/simge ... ystem.html
http://api-docs.freeflightsim.org/simge ... Group.html

https://sourceforge.net/p/flightgear/fl ... s.cxx#l406

This should provide sufficient pointers to be able to appreciate some of the subtleties without getting lost in pointless arguments, i.e. helping people realize that not all issues are "magically solved" by adopting a "more mainstream, non-Nasal, language" like Python/JavaScript whatever.

Someone with a professional background in software development should have no trouble skimming through these pages to see that the challenges that lie ahead have little to do with Nasal, if in doubt, refer to: http://wiki.flightgear.org/Technical_Re ... _Simulator
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11326
Joined: Tue Mar 25, 2008 8:40 am

Previous

Return to Nasal

Who is online

Users browsing this forum: No registered users and 1 guest