Board index FlightGear Development Aircraft

Citation-X route/fix display fail solved

Questions and discussion about creating aircraft. Flight dynamics, 3d models, cockpits, systems, animation, textures.

Re: Citation-X route/fix display failed

Postby SurferTim » Sun Aug 22, 2021 2:25 pm

Thanks for that. I'll take a serious look. I'm not certain I am up to that yet. My brain is still on fire from the troubleshooting.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1653
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: Citation-X route/fix display failed

Postby Hooray » Sun Aug 22, 2021 2:27 pm

come on, according to your avatar, you're even on Linux - so applying a patch that's 5 lines of code, can even be done without using any shell/terminal :lol:

Here's how to test the patch by manually editing a single file in simgear:

  • go to $SG_SRC/canvas/elements/CanvasPath.cxx
  • navigate to/search the line containing static const VGubyte shCoordsPerCommand[]
  • prepend/insert a new line before that one
  • containing: static OpenThreads::Mutex _mutex;
  • next, navigate to the line containing osg::State* state = renderInfo.getState();
  • prepend/insert a new line containing OpenThreads::ScopedLock<OpenThreads::Mutex> lock(_mutex);

recompile sg+fg, test and report back :D
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Citation-X route/fix display failed

Postby SurferTim » Sun Aug 22, 2021 2:38 pm

My brain does not work exactly as it used to. What my avatar doesn't say is I'm 71 and I had a stroke a few years ago. When my brain cools off, I'll take a look. Thanks again for your patience. :)

Edit: I use FG as therapy. It makes my brain work, and sometimes too much.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1653
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: Citation-X route/fix display failed

Postby V12 » Sun Aug 22, 2021 3:12 pm

I had some crashes on R7 and 12 threads (8 physical cores with hyperthreading enabled). I tested 16 threads, FG was very unstable, but N.Y. scenery was loaded extremly fast. All thread count higher than 1 led to the instability.
Then I disabled HT, FG stability was significant better, but not perfect, even with 2 database threads.
Fly high, fly fast - fly Concorde !
V12
 
Posts: 2757
Joined: Thu Jan 12, 2017 5:27 pm
Location: LZIB
Callsign: BAWV12

Re: Citation-X route/fix display failed

Postby wkitty42 » Sun Aug 22, 2021 4:23 pm

hyper-threading! i couldn't remember the proper name of it... thanks for the hint :)
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9070
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Citation-X route/fix display failed

Postby V12 » Sun Aug 22, 2021 7:38 pm

And one interesting fact from P3D world - with disabled HT, P3D 5.2HF1 on AMD Ryzen 7 3700X is almost stutters free. With enabled, sim stutters. I played with affinity mask, but without effect.
FG with enabled HT stutters same as with HT disabled. But in general, on my machine is more stable without HT than with it. Few CTDs on scenery load with splash screen, almost none CTD in flight.
Fly high, fly fast - fly Concorde !
V12
 
Posts: 2757
Joined: Thu Jan 12, 2017 5:27 pm
Location: LZIB
Callsign: BAWV12

Re: Citation-X route/fix display failed

Postby Hooray » Sun Aug 22, 2021 7:40 pm

please keep in mind that HT (Hyperthreading) is essentially a huge fake - i.e. a single processor core pretending to be two cores.
Thus, for the sake of this discussion -or the larger fgfs vs. multicore debate- the whole HT thingy should be only of marginal interest, if any at all.

When dealing with a multi-threaded application, you don't want "fake cores", you need real ones - expecting HT to perform well, is akin to expecting integrated GPUs to perform well :lol:

Otherwise, you'll literally have thousands of context switches per frame to emulate the real core/gpu, or missing functionality.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Citation-X route/fix display failed

Postby wkitty42 » Sun Aug 22, 2021 10:34 pm

yep! that's exactly why i disabled hyper-threading on my other system, the Ryzen 7 with RTX2060... that system is dedicated to gaming/simming :)
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9070
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Citation-X route/fix display failed

Postby V12 » Mon Aug 23, 2021 4:57 am

I know very long that HT is fake, years ago when first HT enabled CPUs entered to the market I worked with multithreaded apps (3DS MAX, ProEng, ANSYS etc), there was not any performance benefit from that fake core. Only system response under full load was better.
HT is first thing, what I disable on the customer's PC, when they observed any problems with games. For heavy apps I leave HT active.
Fly high, fly fast - fly Concorde !
V12
 
Posts: 2757
Joined: Thu Jan 12, 2017 5:27 pm
Location: LZIB
Callsign: BAWV12

Re: Citation-X route/fix display failed

Postby SurferTim » Mon Aug 23, 2021 11:21 am

My brain is back (kinda). It is hard to believe this fix is dependent on a 71 year old stroke victim, but ok.

I found the file, but before I make changes, is running the download_and_compile.sh script going to overwrite these changes?

Edit: If you want to follow along, the file in question is
/fgmeta/simgear/simgear/canvas/elements/CanvasPath.cxx

Too late. I recompiled the changes and the FLICKERING IS GONE!!. Gonna take a flight later after I see if it crashes after running on the ramp.
@Hooray: At the risk of being premature: THANK YOU VERY MUCH! I know I can be an a**hole sometimes. I've always been like that. :)
@clm76: Your nav display works fine. Thanks again. :)

Made it from Destin FL to Las Vegas NV without a problem.

There is a drawback. With FG2020.4.0
The frame rate on my Citation without the nav panel is around 55fps.
The frame rate on my Citation with just the canvas nav panel (MFD only) added is around 35fps.
The frame rate on the Patten Citation (canvas PFD and MFD) is around 25fps.
Just FYI.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1653
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: Citation-X route/fix display fail solved

Postby Hooray » Mon Aug 23, 2021 4:50 pm

SurferTim wrote in Mon Aug 23, 2021 11:21 am:I found the file, but before I make changes, is running the download_and_compile.sh script going to overwrite these changes?

Edit: If you want to follow along, the file in question is
/fgmeta/simgear/simgear/canvas/elements/CanvasPath.cxx
Too late.


Speaking in general, there is no way to "break" anything here, you can easily revert all local changes, so you don't need to be worried at all.


SurferTim wrote in Mon Aug 23, 2021 11:21 am:I recompiled the changes and the FLICKERING IS GONE!!. Gonna take a flight later after I see if it crashes after running on the ramp.
@Hooray: At the risk of being premature: THANK YOU VERY MUCH! I know I can be an a**hole sometimes. I've always been like that. :)

Thanks for reporting back, glad to hear. That being said, you've probably now witnessed firsthand that this was trivial to try, which was my whole point a couple of days ago when I posted the patch: Based on the questions I asked, and the answers posted in that thread, we did have an assumption about the origin of/reason for the issue, and also a potential patch to see if that "fixes" the issue or not.

However, the point was not literally coming up with a definite fix, but rather sharing a potential workaround, to see how well that works for others, to hopefully help us better understand the issue - i.e. if it goes away or not, or if new issues show up.
And like I also said previously, depending on your settings, you may well run into other threading issues, unrelated to the Canvas.Path issue.

Given that you seem to be regularly testing on a multi-screen setup, I would definitely suggest you get involved in regular testing, and especially: please consider using the issue tracker to file bug reports. The point being, these "multi-screen and multi-gpu" setups don't receive much testing, so it's important to have people involved who can help us with testing things.
Otherwise, you should not be surprised if things break again some time in the future: many folks don't have access to a multi-screen/multi-gpu system, which is why new features may not have been tested on such hardware.

Assuming this patch fixes the issue for you locally, you have now also witnesses that the process can work pretty well: tons of Q&A on the forum among a handful of users, and then someone shares a proof-of-concept patch, and encourages people to get involved in testing.
That's basically how open source works: don't underestimate your role.

Made it from Destin FL to Las Vegas NV without a problem.
There is a drawback. With FG2020.4.0
The frame rate on my Citation without the nav panel is around 55fps.
The frame rate on my Citation with just the canvas nav panel (MFD only) added is around 35fps.
The frame rate on the Patten Citation (canvas PFD and MFD) is around 25fps.
Just FYI.


In a way, that's to be expected, the change you added to CanvasPath.cxx basically tells OSG to "stop & wait", rather than run the code from different threads (which was causing problems).
There are undoubtedly better ways to address the issue. Which is why I said, we need more testing/feedback and then evaluate what's still missing.
But as has been said already, some devs are considering to replace the offending "shiva" component in FlightGear (which is indeed a 3rd party library/service, which was written by someone not involved in the FG project at all).

Basically, what I posted a couple of days ago, was a long-hanging fruit to try - based on the backtraces shared by merspieler and wkitty42.
This isn't to say that this is the definite workaround/fix

There are ways to fix such performance issues, but unfortunately most ND/PFD code is hardly optimized from a property I/O standpoint - if in doubt, I'd suggest using the ufo and the FG1000 (debug menu) - that's a fair stress test for the Canvas system (including MapStructure/Canvas.Path specifically), and it has much better optimized property I/O than the majority of aircraft have.

PS: I also apologize if any of my remarks offended you in any way - obviously, we don't know anything about the age of fellow contributors, or medical conditions - but as you can see, neither your age nor your medical background prevented you from applying the patch and testing it, so kudos for that :D
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Citation-X route/fix display fail solved

Postby SurferTim » Mon Aug 23, 2021 4:57 pm

No offense. That would have been me doing the offending. By the time you brought up the patch, my brain was overheated. I had to stop for the day.
I expected it to eat a bit more resources, just not quite that much. :shock:
At least it works satisfactory. I bet the frame rate on a single monitor is far better.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1653
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: Citation-X route/fix display fail solved

Postby Hooray » Mon Aug 23, 2021 5:06 pm

There is another long-standing performance issue, that's also related to the Canvas system, because of the way FlightGear handles scene cameras (near/far scheme), Canvas displays (cockpit avionics) are rendered twice per display, which is unnecessary and wasteful: https://wiki.flightgear.org/Post_Flight ... ges#Canvas

Review Tim's original suggestion[27]: Another [Canvas] optimization is to use a pre-render camera that isn't in the scene graph to do the Canvas rendering. Otherwise the Canvas textures are rendered at least twice, in the near and far cameras. [28][29] Tim Moore’s canvas-rendering-on-a-pre-camera fix, again should give us a nice perf win for Canvas-intensive acft[30], since within each camera pass (far camera, near camera), all the passes of a technique are run.[31] Also, Mathias stated once: that it would be good to be able to specify a completely different scenegraph in some subcameras, i.e. for having panel like instruments on an additional screen/display for example.[32]


Speaking in general, if you care about the multi-screen/multi-gpu use-case (and overall multi-threading) in FlightGear, please get involved in regular testing, specifically:



If in doubt, tinker with using additional OSG pager threads, too: https://wiki.flightgear.org/Howto:Using ... er_Threads
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Citation-X route/fix display fail solved

Postby SurferTim » Mon Aug 23, 2021 5:18 pm

I'm up to a challenge now and then. Wouldn't want to make a habit out of it tho.
What should I expect if I run that firs test? I run FG with a desktop script.
Code: Select all
./run_fgfs.sh --launcher

Should I append those parameters here?
I put it in the launcher in the settings section. Started ok. Shows "Composite Viewer" on a/c start.

Programming, debugging, and troubleshooting is work. :(
Flying is joy. :D
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1653
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: Citation-X route/fix display fail solved

Postby Hooray » Mon Aug 23, 2021 6:13 pm

SurferTim wrote in Mon Aug 23, 2021 5:18 pm:I'm up to a challenge now and then. Wouldn't want to make a habit out of it tho.


Generally, if you care about your specific use-case (multiple screens), it's a good idea to test things every now and then and report back if something goes haywire.

You could say, "Composite Viewer" is all about emulating multiple screens by opening multiple windows, shown on a single screen (or possibly different ones), thus it's relevant to your use-case (typically issues in CV mode - like the CanvasPath issue- will also show up with your multi-screen config and vice versa):

Image

What should I expect if I run that firs test? I run FG with a desktop script.
Code: Select all
./run_fgfs.sh --launcher

Should I append those parameters here?
I put it in the launcher in the settings section. Started ok. Shows "Composite Viewer" on a/c start.

Yes, the launcher should do - what it will do is enable "composite viewer" mode - which means, support for multiple independent windows per fgfs instance.
What the test will do, is open a handful of windows, which will also start a few OSG threads, and then display a canvas dialog.
If you open the FG1000, too - you are basically stress testing the Canvas system in multi-threading mode.

For starters, this is the sort of thing that one would want to do with a simple aircraft (e.g. ufo) and without any scenery (ocean/sea location).

That is why I posted those screen shots in the other thread: they showed the osg stats, another window, the suspected culprit (canvas map) and the threading mode (as part of the stats):

Image


Programming, debugging, and troubleshooting is work. :(
Flying is joy. :D


Like I said, you could then fly a fixed pattern/circle, use the route manager or a pre-recorded flight - so you don't have to sit next to the PC while doing this, even just returning after 5,10,15 minutes (or even after a couple of hours, i.e. overnight), should tell you whether things are working well or not - if FlightGear seems to have once you return, you know that there is a problem.

So, I would say, you don't need to much in terms of debugging/troubleshooting - rather, even just testing (=running) fgfs regularly in multi-threaded mode (multiple screens/windows), using different combinations of settings (aircraft/locations), is a good way to ensure that FlightGear still supports your use-case.

But if you should find something, you can file a ticket, report it via the devel-list or discuss the issue on the forum.
As you have seen, there are people like wkitty42 and merspieler who don't mind rolling up their sleeves to fill the gaps, once they know that there /might/ be an actual problem.


Performance is best investigated using the built-in OSG stats (debug menu): https://wiki.flightgear.org/OSG_on_screen_stats
Image
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

PreviousNext

Return to Aircraft

Who is online

Users browsing this forum: No registered users and 1 guest