Board index FlightGear Development Nasal

Why not use Erlang instead of NASAL?

Nasal is the scripting language of FlightGear.

Re: Why not use Erlang instead of NASAL?

Postby abassign » Sat Mar 18, 2017 11:41 am

geed wrote in Sat Mar 11, 2017 5:01 pm:@abassign, How far did you get in analyzing Erlang? Please try to implement a real world application or solve a real world problem with it and I would like to know how you evaluate the language or the environment afterwards. This is not meant as a joke or to mock you, please Test Erlang and come back and present the problem you solved with the solution you came up with. I would be extremely interested in your approach and what analysis you made and how you relate that to a possible implementation as a scripting language in Flightgear.


Meanwhile, I'm sorry for the delay in the response.
Erlang is a language to manage applications with highly fragmented and driven by events on multi-CPU systems. I would say is just perfect for simulation applications such as FGFS!
FGFS is fundamentally based on a hierarchical data structure (Property Tree or PT), structure in which Erlang (As LISP, from which derives ...), it has a correct and complete syntax for access to these particular data structures.
When building a complex plane (777-200, SU15, Mig15, F14, A10 etc ...) we observe an immediate explosion of subsystems, such as the electrical system, the hydraulic system, instrumentation (especially if the analog type) etc . All these objects interact with each other through messages that are currently managed by PT. I have often noted that such "explosion" of tasks leads to a real slow system performance, regardless of the problem of slowness due to the graphics card.
Certainly an approach based only on messages (Where Erlang is great) would reduce much stress and allow to operate on a real distribution of the workload across multiple CPUs.

Unfortunately we live in a real world and profoundly change the data structure based on the PT, is not currently feasible.

At this point I feel that if it were possible to implement a bind, or build a SOLID C ++ interface with PT. This would open the possibility of a much more extensive use of other languages such as Java, javscript, Erlang, Python etc ... to be inserted directly into the folder that contains the plane's data.

Now it becomes more complicated, a bind is efficient if you do directly on the system if it is implemented and not through a high-level access code (TCP-UDP). You can also create bindings using special system calls but are dependent heavily of the OS installed on.

Personally I like the idea of an application operating at the TCP level, the reason is that in the current multi-CPU systems, TCP transactions are handled by separate task in multi-CPU mode and then have a weight reduction of the frame rate, very low.

Now what I am saying is true for any language that will want to interface with FGFS:

1. Through the XML it must be possible to activate / deactivate an external interpreter (Erlang, Python, Java, javscript, LISP, etc.) which itself loads the program to run it becomes an independent task of FGFS.
2. The interpreter needs to know what IP address to start the conversation and have a series of commands that allow you to build a dynamic transmission protocol between the data in the PT. Because the PT is very complex it is useful to assume that each task will take charge of getting what he wants and be then the connection system that distributes data in an appropriate manner.

Better to make an example:

1. In FGFS an XML declaration that opens an external interpreter (eg Python, Erlang, LISP, Java, JavaScript ...) and activates the TCP connection channel.
2. The interpreter runs the application.
3. The application for example require to read the altitude and speed of the aircraft and will send back the value of the throttle.
4. The application then opens a data read/write channel, to the PT. In this way, the PT can build a data packet send with an unique and short TCP transaction. These channels are no more than a data structure to be built / dynamically destroy that allow a high read / write speed. Personally I believe incorrect use more sophisticated methods (and therefore slow ...), because it makes no sense. The important thing is that the individual tasks have the ability to see the PT data in reading and in writing with an optimal speed of less than 10 ms.
5. From now all the processing load is the responsibility of tasks that have been opened by the interpreter. When individual tasks performed processing, they send their responses (eg the throttle value) to the output channel, which, every 5-10 ms, is sent to the PT to update it.

The solution I have proposed is designed for external tasks, with other languages, on different machines (!), Using only a few commands and a few transactions.
Transactions will be faster if they are few per unit of time and only with the data that we need!
Developer of the program https://wiki.flightgear.org/Julia_photoscenery_generator
FDM developer of the G91R1B aircraft https://wiki.flightgear.org/FIAT_G91R1B
JSBSim collaborator
abassign
 
Posts: 947
Joined: Mon Feb 27, 2012 6:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2020.4
OS: Ubuntu 20.10

Re: Why not use Erlang instead of NASAL?

Postby Hooray » Sat Mar 18, 2017 4:05 pm

Honestly, with obvious exceptions like props.ns (which could, and arguably should, be re-implemented on top of Nasal/CppBind), scripting in FlightGear is not primarily restricted because of Nasal or any of its shortcomings, but because of the existing/legacy main loop architecture and the existing threading "model".

What you have in mind would obviously work, but it would directly affect ongoing developments like HLA/RTI, and even without looking at such "pie-in-the-sky" developments, there is other heavily related work, e.g. Torsten's Phi/mongoose approach, that could be directly leveraged to implement support for 3rd party scripting solutions running in separate threads/processes and only ever interfacing with the fgfs process using an IPC abstraction - for starters, that could indeed by the mongoose layer that Torsten implemented to expose the property tree and fgcommands.

That would work "right now" - and it would take any C++ changes - people could write Python/Lua or JavaScript/V8 code that directly interfaces with FlightGear using this IPC/RPC mechanism, which could then obviously be extended over time.

Besides, if/when Nasal's HTTP bindings are reviewed and extended accordingly, that would also be a very promising way to implement asynchronous Nasal scripts that only ever work via AJAX/JSON (the IPC mechanism in use really being just an abstraction detail that could be easily replaced at some point, e.g. using Richard's Emesary framework).


http://wiki.flightgear.org/Howto:Making ... from_Nasal
http://wiki.flightgear.org/Emesary
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Why not use Erlang instead of NASAL?

Postby abassign » Sat Mar 18, 2017 7:51 pm

HLA/RTI,this is a wonderful solution, but it allows to be able get a refresh rate of 120 Hz? I have doubts about it ... I think it takes something very, very light.

When I proposed Erlang, as I immediately said, was to make a provocation, just like to say "any language" that one would like to use to build your own creature. Then it will be the use that will decide the dominant language and the final architecture will adopt the most fans.

Having a lightweight protocol for "extend" to other processes, the tree of the properties (PT) I think it is the best solution and the fastest to implement.

Meanwhile, what is the properties tree (PT) ? From the point of view of a modern programming language it is a sort of hash table coupled with index (B-tree). I worked for years with these structures, are simple and efficient and allow for different views, without changing the data, using different indices. It is not said that the implementation in FGFs is precisely that, but it can be reported to that.

"/consumables/fuel/tank/capacity-gal-us" <-> 61582f8d

For example:

The external program make e request:

12|0de3fed1|"/consumables/fuel/tank/capacity-gal-us"|61582f8d

where:

"|" pipe separate the fields
12 -> command type code (for example read the field)
0de3fed1 -> id caller
"/consumables/fuel/tank/capacity-gal-us" -> the PT field
61582f8d -> caller md5 field

after the caller make another request:
12|0de3fed1|"/sim/frame-rate"|d8cbcb52

The client send a request:
00|0de3fed1

The system respond:
61582f8d|256<CR>
d8cbcb52|25<CR>

This is very compact form, is possible to write the data from the client to FGFS:

01|0de3fed1|61582f8d|340

where, for example, the code: 01 -> write command to single field

The client send a request:
00|0de3fed1

where, for example, the code: 00 -> read command to all configurated field

The system respond:
0de3fed1|61582f8d|340<CR>
0de3fed1|d8cbcb52|25<CR>

The protocol ... I prefer the UDP ... is short and fast, but if we are outside the network better TCP.
With this methodology it is possible to use as a coprocessor even very small machines, for example Arduino (think for example the possibility to use an input device of a control system). Raspberry etc ...

Eventually my request to do a telnet to remember who asked the question and associated with a path to the PT's the equivalent hasch code.
The rest of the workload is done by the client. The only care is to be able to start / close the client with the program from XML, so that we should first start the client and then start FGFS. The caller id allows the client to know if it is he who has to answer, which is very useful for example to find out if it is to rebuild the request, for example is it restart
the FGFS program.

I wonder if someone gives me a hand on C ++ FGFS to do a test and see how fast it can work, I work on the client side.
Developer of the program https://wiki.flightgear.org/Julia_photoscenery_generator
FDM developer of the G91R1B aircraft https://wiki.flightgear.org/FIAT_G91R1B
JSBSim collaborator
abassign
 
Posts: 947
Joined: Mon Feb 27, 2012 6:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2020.4
OS: Ubuntu 20.10

Re: Why not use Erlang instead of NASAL?

Postby Hooray » Sat Mar 18, 2017 8:18 pm

See the implementation of the props/telnet protocol and specifically its "subscribe" command, it could be easily extended to experiment with a scheme like the one you have in mind (to be potentially replaced with something UDP based or using Torsten's mongoose groundwork):

http://wiki.flightgear.org/Telnet_usage ... nsubscribe
http://wiki.flightgear.org/Improved_J66 ... uggestions

Also note that the multiplayer protocol is already using a table of property IDs mapped to their string equivalents, which you are hinting at - it just happens to be using UDP and XDR for transmission/encoding purposes respectively, so the concept is already in use - and the idea is sound, it just isn't widely done elsewhere (yet).

And regarding your idea specifically, you should take a look here:
http://wiki.flightgear.org/Distributed_ ... sage_Types
http://wiki.flightgear.org/Remote_Properties


This is about implementing PUB/SUB on top of the property tree using some (arbitrary) form of RPC/IPC: https://en.wikipedia.org/wiki/Publish%E ... be_pattern

(and yes, the property tree is basically just that: a hash table)

PS: See google's protocol buffers


EDIT 12/2017:
I've just been asked about a question posed on the JBW forum regarding Nasal interfacing, suggesting that my response on the forum would not answer the question correctly: http://thejabberwocky.net/viewtopic.php ... =70#p15307

I went through all the documents mentioned, and they tell me all the stuff i do not and want to know. They are not design documents on interfacing something with fg.

- The first document describes property tree. Nice to know, but it does not tell me about whether that is all the binding i need for a scripting language.

- The others are about how to use nasal. Totally irrelevant. Yet again spent time on nothing.

- The last one is about nasal internals. Also totally irrelevant. Lost time again.

The crucial part is missing: How is nasal as a whole hung into the fg C core? By what api? Is this the only way to hang a scripting language into fg, or are there other api's usable? What are the preferred api for hanging a scripting language in fg? Where is the api documentation? Or is there no api interface and is nasal entangled spaghetti with fg C core?

Or is a standard embedding of a scripting language into C sufficient for fg to understand it?

https://docs.python.org/2/extending/embedding.html

I highly doubt it. So where is the embedding documentation? The embedding design of nasal would be nice for starters.


This stuff is actually covered in the repositories:

https://sourceforge.net/p/flightgear/fl ... Nasal.html
https://sourceforge.net/p/flightgear/fg ... format=raw

Nasal as such is integrated (read: initialized) using a conventional SGSubsystem: http://api-docs.freeflightsim.org/simge ... Group.html

Pretty much all of the runtime stuff is invoked in the form of bindings, timers and/or listeners.

And yeah, that's pretty much what is needed to integrate another scripting language with FlightGear, at least using the FGNasalSys based approach.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Previous

Return to Nasal

Who is online

Users browsing this forum: No registered users and 1 guest