Board index FlightGear Development New features

FGPython an propose for Python as an nasal alternative

Discussion and requests for new features. Please note that FlightGear developers are volunteers and may or may not be able to consider these requests.

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 12:30 am

Sorry, still too abstract for my taste - your whole notion of how XML and properties relate to each other is problematic/lacking, i.e. in the context of SimGear, FlightGear - i.e. XML and properties forming the existing IPC mechanism for PODs (strings, integers, floats, doubles, booleans) - so there is no reason why this should be affected (or even touched/changed) by anybody wanting to support/use Python in FG.

I suggest you look at the postings made by www2 and bugman in this thread to fill you in on the priorities - and then check the posting where I posted links on how to embed a python interpreter in a C++ program

The last thing is true, but can be trivially implemented by mapping osgDB read/write access to an RPC/IPC service (probably using RTI)
Last edited by Hooray on Thu Dec 31, 2015 12:32 am, edited 1 time in total.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby abassign » Thu Dec 31, 2015 12:32 am

@Hooray

if it's too obscure, like COBOL


However Cobol is not dark, it is instead only obscure to us programmers of 2015, what were the technology in the decade 1950-1960. During those years were using electromechanical calculating machines that were programmed with techniques compatible with COBOL.
Developer of the program https://wiki.flightgear.org/Julia_photoscenery_generator
FDM developer of the G91R1B aircraft https://wiki.flightgear.org/FIAT_G91R1B
JSBSim collaborator
abassign
 
Posts: 947
Joined: Mon Feb 27, 2012 6:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2020.4
OS: Ubuntu 20.10

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 12:35 am

I mentioned COBOL specifically because I had the impression that you must have some kind of background in the early days of computing, i.e. possibly embedded systems and microprocessors - because many of your concerns make only sense from that standpoint, and many of the higher level concepts you are bringing up are either obsolete meanwhile, or simply not relevant on an average computer system having 8+ cores, 16+ gb of RAM and 2048 MB of VRAM. Still, you manage to use quite a bit of vocabularly and acronyms that make your postings sound interesting - unfortunately, I fail to see any SG/FG related specifics, at which point things are simply way too abstract for my taste.

Hopefully, www2 and bugman can better handle your feedback though
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby abassign » Thu Dec 31, 2015 12:41 am

@Hooray

Sorry, still too abstract for my taste - your whole notion of how XML and properties relate to each other is problematic/lacking, i.e. in the context of SimGear, FlightGear - i.e. XML and properties forming the existing IPC mechanism for PODs (strings, integers, floats, doubles, booleans) - so there is no reason why this should be affected (or even touched/changed) by anybody wanting to support/use Python in FG.


In fact I am not interested what there is under the hood of the engine, I'm interested in a way to interface with that engine, if you do not understand then you're not the right person to answer my question. If you do not want to understand I suspect that try to sink this discussion in order to do nothing.
So instead of hasty responses try to meditate on what I have written, if you are interested in deepening ok, if you're interested in the topic, please do not respond not to waste time to any of us.
Developer of the program https://wiki.flightgear.org/Julia_photoscenery_generator
FDM developer of the G91R1B aircraft https://wiki.flightgear.org/FIAT_G91R1B
JSBSim collaborator
abassign
 
Posts: 947
Joined: Mon Feb 27, 2012 6:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2020.4
OS: Ubuntu 20.10

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 12:55 am

You are ignoring actual problems and inventing new ones where there aren't any at all. I am just pointing this out so that you may reconsider if/how you want to spend your time on this without first reading up on a few more details.

Obviously, I may be missing something - so if www2, bugman or Curt can see the potential in your last couple of postings, I will just shut up for now and watch the whole thing unfold, I am just failing to see how this is going to happen given what I have read so far, i.e. your postings specifically.

Besides, there seems to be a tendency among people to think that they have figured it all out, no matter if it's spaceflight, dogfighting/combat support or now Python integration, without them necessarily having the background, or even just sufficient information, to tackle such tasks. At which point either they may be wrong, or the community may not yet be ready for their ideas - however, I don't think that we'll have that many Darwins, Cantors or Boltzmanns around here - otherwise, my suggestion would be to write down your ideas and publish/preserve them for the future [1] :D

[1]: Canvas took over 5 years to materialize, the OSG port is still in progress, and HLA took 7+ years to materialize ... so please don't hold your breath
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby abassign » Thu Dec 31, 2015 1:09 am

@Hooray

I mentioned COBOL specifically because I had the impression that you must have some kind of background in the early days of computing, i.e. possibly embedded systems and microprocessors - because many of your concerns make only sense from that standpoint, and many of the higher level concepts you are bringing up are either obsolete meanwhile, or simply not relevant on an average computer system having 8+ cores, 16+ gb of RAM and 2048 MB of VRAM. Still, you manage to use quite a bit of vocabularly and acronyms that make your postings sound interesting - unfortunately, I fail to see any SG/FG related specifics, at which point things are simply way too abstract for my taste.


I answer only because I begin to understand something of what goes through your head. The programming of complex systems requires a fair amount of planning and the reduction of the problems into smaller pieces and only then, when you have clear ideas, you can proceed to write code. I do not know your job, but your approach seems rather expensive and difficult to manage at the level of the working group.

In this specific case I see an XML code that is written on a support, I understand that this XML is actually an interface to the "engine" (whatever it is) and that if I transcribe an XML equivalent of the support I get a result equivalent. Look, so it can not be different from this, otherwise FGFS is a joke! Then we have NASAL that allows you to do things that XML can not do or are too expensive (XML, for your information, is a complete language that you can write any program that goes through your head ... only that sometimes;) is too broad and a good procedural language can be a good thing).

As you may have noticed someone says NASAL sucks as a language, not only, but for example, Python does a lot more things. I think up and according for a variety of information (including you) say:
Attention is not convenient to combine Python XML, since it is proabile generated bottlenecks.
Since I have a brain with a normal coefficient of intelligence, I think about a few hours and I say:
Maybe there is a solution that makes everyone happy!
We divide the problem into smaller pieces and we make the most of what there is. We create an intermediate interface (is a normal process in the object-oriented programming ...) that do not see an external program plus an XML, but a series of calls to equivalent procedures. Not only that, but I assume that if the interface is used, no longer need to go through the XML parser, but directly calls the parser executes.
Obviously at this point it has generated a protocol that can be published and those who want to do a little work can convert it to code written, for example, in Python, ADA, Java, Ruby etc ...
Developer of the program https://wiki.flightgear.org/Julia_photoscenery_generator
FDM developer of the G91R1B aircraft https://wiki.flightgear.org/FIAT_G91R1B
JSBSim collaborator
abassign
 
Posts: 947
Joined: Mon Feb 27, 2012 6:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2020.4
OS: Ubuntu 20.10

Re: FGPython an propose for Python as an nasal alternative

Postby curt » Thu Dec 31, 2015 1:14 am

Just a couple things to keep in mind ...

Any time you divide up the code into multiple independent pieces, you have to account for how those pieces will communicate. Some pieces need to exchange a whole lot of data with other pieces, and this can become a practical bottleneck. Applications that multi-thread really well typically are applications that have many independent pieces that don't need a lot of frequent intercommunication.

There is always overhead and latency when exchanging data between independent programs.

Here's something I don't believe a lot of people think much about when it comes to threading and parallel processing discussions here.

Let's presume you have split up the code into a bunch of independent modules all running at their own rates on their own clocks and happily exchanging data back and forth. Life is good, right? But think about a quick example of shipping a package from a mail order company to a customer. How long does that take? How reliable and predictable is the shipping time?

Let's say new orders are sent to the warehouse in bundles twice a day at specific times, let's say 8am and 12pm. If I place my order at 12:01pm, it won't go to the warehouse for picking/packing until 8am the next day ... ouch! Now let's say fedex shows up every day at 4pm to pick up all the outgoing packages. What if there were lots of packages today and my package wasn't ready by 4pm? It might have to wait another whole day. Now the package is in the hands of fedex which does a pretty good job because they've thought this all through very carefully and their business depends on doing things well, but once in a while they miss a connection or there are weather delays, things are never perfect. I could continue on with additional steps ... often there are many many steps required to pass data from one end and the other. Maybe this is a package that is given to the local post office for final delivery. There is another opportunity to just miss the mail truck loading deadline in the morning. And maybe I can't check my mail box until 6pm each evening after work. So how many days and hours will it be from the time I place my order until the time I can pull it out of my mailbox at home?

What if each step didn't care about the other steps and so each stage made up it's own schedule ... sometimes random, sometimes on even intervals, but with no synchronization or coordination in the timing ... how does that affect the predictability of package delivery times?

How do we handle this? One strategy would be for each step in the process to be eagerly waiting to accept the package from previous step as soon as it is ready, process it immediately, and hand it off to the next step. That sound great, right? That's exactly what a single threaded program does.

Another idea would be to carefully schedule the timing of each stage so that the next stage doesn't have to wait very long ... i.e. the company has all their boxes packed up and ready to go by 3:30pm, the fedex guy shows up around 4pm, he has to get all his packages back to the central fedex distribution location by 8pm, etc. This is a pretty practical real world solution, but then all these things need to run in synchronization from the same time source. In the world of threads and parallel processing this can sometimes be much trickier to accomplish correctly and efficiently. And as soon as you have it working, Asok the intern pushes a change to git and breaks the whole thing accidentally, but no one notices util two months later after many other changes have been so no one really knows what broke or who broke it or why it broke. Let's just reboot and keep going.

Yet another idea would be to speed up all the loops, send new orders to the warehouse 4x a day instead of twice a day. Have the fedex guy come twice a day, fedex could double their aircraft fleet and double their flights. Still not fast enough? Double everything again! The problem is that even though speeding up the rates can improve the end-to-end time, it doesn't improve the predictability or consistency of delivery times. This doesn't address the problem of one loop just missing the next loop and adding a huge delay to the delivery. In the world of computers, often we are already running as fast as we can.

Timing and multi-threading is very very very .... <time passes> ... very tricky to do correctly and efficiently, even more so when there are many hands in the soup. That doesn't mean we should avoid all threads and all parallel processing, but FlightGear and OSG's existing use of threading is a perfect case study for what is dangerous and what can go wrong. And the problem is when the happy little reader/writer example in the computer science text book meets all the crazy demands of a real world application, ideology often has to go out the window in favor of more convoluted solutions, and this can get really hard. Just look at politics ... every few years we go back and forth between political parties and ideologies, each side promising their view point will finally fix all the problems, but no one ever actually seems to fix any of the problems (they may claim they've made vast improvements, but uhhh, most intelligent people see what is going on ...) So there is a huge need for practical minded people that understand how the ideology works, but can flex in intelligent ways to meet the complexity the real world throws at us ... those people are rare and probably burned out quickly. :-)
curt
Administrator
 
Posts: 1168
Joined: Thu Jan 01, 1970 1:00 am
Location: Minneapolis, MN

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 1:15 am

@abassign:thanks for explaining top-down/bottom-up design, but there are so many things you are confusing here, that you are ridiculing yourself, e.g.:

abassign wrote in Thu Dec 31, 2015 1:09 am:XML, for your information, is a complete language that you can write any program that goes through your head ... only that sometimes;) is too broad and a good procedural language can be a good thing


XML is a mark-up language, it's not a programming language.
Consider using google to learn more about the differences.

Honestly, you are getting lost in irrelevant details here - and I really hope that www2, bugman or Curt will see any merits in what you write, because otherwise you are about to kill the whole Python effort before it even starts.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby curt » Thu Dec 31, 2015 1:18 am

xml can be a complete programming language ... jsbsim's config files have dipped their toes in these waters. But generally it's agreed that xml is designed for representing structured data and it's a bad idea to abuse it for programming purposes.
curt
Administrator
 
Posts: 1168
Joined: Thu Jan 01, 1970 1:00 am
Location: Minneapolis, MN

Re: FGPython an propose for Python as an nasal alternative

Postby abassign » Thu Dec 31, 2015 1:20 am

@Hooray

You are ignoring actual problems and inventing new ones where there aren't any at all. I am just pointing this out so that you may reconsider if/how you want to spend your time on this without first reading up on a few more details.


What for you is secondary to other, no less smart than you, can be a big problem for many reasons. However you never asked why seven years of work on HLA/RTI is currently not worth anything ? Create beautiful pieces of program without a clear strategy can lead to these results. And now that someone is saying, Finally it is possible use HLA/RTI if you add a module that permitting their use ... does not seem a bad thing, is not it ?
Developer of the program https://wiki.flightgear.org/Julia_photoscenery_generator
FDM developer of the G91R1B aircraft https://wiki.flightgear.org/FIAT_G91R1B
JSBSim collaborator
abassign
 
Posts: 947
Joined: Mon Feb 27, 2012 6:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2020.4
OS: Ubuntu 20.10

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 1:28 am

curt wrote in Thu Dec 31, 2015 1:18 am:xml can be a complete programming language ... jsbsim's config files have dipped their toes in these waters. But generally it's agreed that xml is designed for representing structured data and it's a bad idea to abuse it for programming purposes.


So have XSLT and other XML based languages, and you can also make up a programming language using whitespaces, stones, matches or even toilet paper- i.e. it's all left to the way you interpret your "code". If in doubt, I suggest to refer to wikipedia instead of having this discussion here and now - there's a clear difference between a generic markup language and a programming language, which is not say that you cannot express markup in code or code in markup (in fact, even in FG bindings/fgcommands are attempting to do that, too)

Otherwise, I will try to remain silent for a while to see if anything constructive comes out of this, because this is getting more and more pointless unfortunately.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby japreja » Thu Dec 31, 2015 5:20 am

Rather than FlightGear embedding another language, FlightGear would probably benefit much better with a type of loadable module support. Nasal could become one of those modules along with Python and maybe Perl. Much in the same manner that Apache and Apache Modules work. I think that would greatly broaden the horizon.

Scripting would be just one type of module, the GUI could be another type which might make it easier to support QT or even Java, not to mention that the FDM code could become a separate module, one for each supported FDM. So in just discussing Python support we may be limiting ourselves and/or FlightGear .

Should we be discussing Module support instead?
Would that greatly reduce the core FlightGear code?
Has module support been discussed before?

hmmmm... It would "probably" take the same amount of work just implementing Python.
Jorge
japreja
 
Posts: 334
Joined: Fri May 08, 2015 12:05 am
Location: MT, USA
OS: Windows 10 Pro 64bit

Re: FGPython an propose for Python as an nasal alternative

Postby Thorsten » Thu Dec 31, 2015 8:11 am

@Curt:

I get the sense that you don't want to budge until you are convinced. That is fair, but you have structured the questions in a way that makes it nearly impossible to be convinced. Any of the answers or suggestions that have been presented are interpreted in the most diminished way possible.


Ah... interesting. No, you're completely misunderstanding my position.

I've learned that in evaluating novel ideas, it makes sense that one party assumes a critical perspective, the other party tries to 'sell' the idea. The idea is that through critical questions and the resulting need to argue a position, an initially vague notion of 'wouldn't it be better' can be sharpened to an idea that avoids the worst pitfalls up-front and is actually convincing. Or not. So my idea here is to get an accurate picture of what the advantages would be such that I can form an educated position.

I do know that usually adding more things into the fray, making more independent components to talk to each other generically leads to unexpected consequences. I've seen that so often that nobody can really convince me that we won't see this as down side. The relevant question for me is - will this be balanced (or even more than balanced) by advantages?

See, knowing hardly anything, it could be that all this is just about familiarity - people are familiar with a certain syntax, so they just don't want to adapt to something new. Fair enough, but wouldn't convince me. It could be that this is about a misconception of what are performance bottlenecks, like the recurring theme that dds textures render faster - in this case, the misconception should be cleared up. It could be that this is about something genuinely useful. People frequently argue all sorts of things for all sorts of reasons.

So yeah, I think it'd be a good thing if someone would convince me, because then he'd be forced to think hard about pros and cons himself and probably eventually make a better implementation that way. There's a reason that proposals get evaluated before they're done in reality as well. This isn't fundamentally about convincing me - but think - if you can't even theoretically, in a vision sketched, get me excited by the potential - how good is your idea really?

(Finally - you do make some very good points with complex AP logic - that was actually the kind of case I was hoping to see.)

@abassign:

I think it's important to come off from the couple XML-NASAL, which proved difficult to manage for large projects, such as SU15 or F15 etc ...


I am actually managing one of these 'huge' projects and I don't see the difficulty you're mentioning. It's (yet another) red herring.

You can write good and efficient code in Nasal, in GLSL, in C++ or in Python, and you can write a poorly structured design in any of those as well.

Frankly, it's clear from several of your postings that you don't have any hands-on experience in rendering - and yet you frequently feel the need to comment how it 'really' should be done. I've never seen your 'huge' aircraft project - and yet you feel the need to lecture the people who do them how they 'really' work. It's clear from your examples that you're not aware how and where performance bottlenecks arise in FG - and yet you frequently point out what 'really' needs to be done to remove those. At the same time, when these things are explained to you by whom ever, it doesn't seem to have any effect.

This is an attitude I find personally very problematic. I can respect people whose assertions are backed up by hands-on experiences or theoretical knowledge, even if there is scant evidence and even if their experiences they differ from mine - but I can't respect such off-hand assertions of 'evident truth' backed up by nothing.

You're replacing evidence by wishful thinking and that's never a good idea.


Let's see how the Conservative Party, currently in power, responds to my proposal, I'm curious;) and I also hope that it can avoid continuing its offensive manner of analyzing the problem.


If you find it offensive that people point out to you where you are wrong, you should perhaps not participate in such a discussion. If you would care to bolster your proposal by more evidence and less wishful thinking, you'd find a warmer reception.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: FGPython an propose for Python as an nasal alternative

Postby bugman » Thu Dec 31, 2015 12:52 pm

Time for a trump card :) Here is one point that will make it all worthwhile. The concept is called software prototyping.

Let's for a moment forget if abassign, www2, or someone else has the coding skills to do this. You shouldn't judge someone until you've seen what they really can do. Let their code speak. There are also others capable of doing this. Let's also not put up an impossible-to-surmount wall - the requirement of knowing most of the FG internals - as most of the core developers will not be able to get over this wall. You also learn as you go, which is probably how all core developers started.

For the idea of prototyping, you could set up a builtin Python package which provides a framework to create a new FlightGear subsystem. There's no point arguing if this is doable or not, because I know that anyone who knows Python, C, and the FG internals could probably embed Python, add read+write access to the property tree, and create this subsystem framework package in an extremely short timeframe (with real dedication, a week would not be unreasonable). I could probably do this myself in this timeframe, if it wasn't for my other projects (1, 2). You can expose as much of the C++ internals as you wish with the Python-C API.

Once you have a Python interface that allows for code prototyping, then this opens up the FlightGear internals to a much larger group of people. For example uni students, as a large number already use Python for prototyping ideas. Once the idea is tested, the code can then be ported to C++. Or if the performance impact is low, the Python subsystem can be integrated into the core without modification.

Python is the perfect prototyping language for lower level languages, as it is one of the easiest to learn. Syntax-wise it is the simplest language. The prototyping potential of FG embedded Python is such an awesome feature, that it makes any arguments against it moot. It trumps the downsides, though I'm yet to be convinced that there are downsides to such an 'optional' feature.

Regards,
Edward
bugman
Moderator
 
Posts: 1808
Joined: Thu Mar 19, 2015 10:01 am
Version: next

Re: FGPython an propose for Python as an nasal alternative

Postby Thorsten » Thu Dec 31, 2015 1:42 pm

Two questions, Edward? Is it really that much better for prototyping than Nasal (and why)? The same argument has been made frequently for Nasal.

Also, suppose I had written AW in Python and had used a large number of modules. How optional is a subsystem really once people need it? De-facto, it'd probably require a lot of people to install all my libs.

It reminds me on the wording of making Qt5 an optional dependency - de-facto it's not really as there's no alternative provided and very few people will write their own or have a two-monitor setup and use Phi.

So yeah, suppose that goes through - I will be able to run a gutted FG without GUI and without the need to install a random collection of Python libs - but won't I de-facto be forced to have both if I really want to use it?
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

PreviousNext

Return to New features

Who is online

Users browsing this forum: No registered users and 5 guests