Board index FlightGear Development Canvas

Multi-threaded FlightGear/OSG and Canvas flickering solved

Canvas is FlightGear's new fully scriptable 2D drawing system that will allow you to easily create new instruments, HUDs and even GUI dialogs and custom GUI widgets, without having to write C++ code and without having to rebuild FlightGear.

Re: NDCanvas flickering problem

Postby Hooray » Mon Aug 16, 2021 5:28 pm

SurferTim wrote in Sun Aug 15, 2021 7:33 pm:Edit2: Got the minimum settings to work. Moved to KDTS airport. Selected Equipment -> Map (Canvas). Started flickering, then crashed back to the desktop.
Is that what you wanted?


Does that mean, it's literally crashing (segfault) ? If so, that would be good - maybe someone could share a backtrace ?
Is this also happening when using a native Canvas GUI dialog, rather than the PUI placement ?

Also, that's interesting: Does that mean, that the flickering starts immediately when a Canvas widget is shown in the main window, despite it not being shown in any of the other windows ? If so, that would suggest that it's not necessarily culling related.

In general, people interested in helping troubleshoot this, should ideally check next:

  • is this issue in any way affected by using -or not using- the built-in Qt5 launcher ?
  • can the issue be reproduced for non-Canvas avionics using any of the od_gauge MFDs (navdisplay, agradar, wxradar) ?
  • also, please share screen shots showing the OSG stats
  • it's also helpful to share your exact setup so that other users can easily reproduce your config and see if they can trigger the problem (that's basically the requirement to run fgfs inside a debugger)


PS: I think this is unrelated to any specific airport or aircraft, so this should work with the minimal startup profile - if the issue isn't showing up in places without scenery, that would suggest that it's specific to certain map elements / canvas primitives, because those are only renderered if scenery/airports are close (which would also be good to know).
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: NDCanvas flickering problem

Postby SurferTim » Mon Aug 16, 2021 5:39 pm

I've posted my setup. I can upload a video to YouTube. If I use the minimum start stuff, it will be really short. It starts flickering almost immediately and will crash within seconds.

When I started with the minimum settings, it put me way out in nowhere. When I selected the canvas map there, it didn't crash, at least right away, but it wasn't displaying any vors or airports. There weren't any around. It showed a dark brown background with minimal info. One time I ended up just NNW of the Marshall Islands.

It was when I selected an airport that had a few vors and airports did it flicker and crash.

I'm about to be visited by tropical storm Fred, so my FG computer is off today. Power fail and lightning strikes are always a possibility.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1709
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: NDCanvas flickering problem

Postby Hooray » Mon Aug 16, 2021 6:00 pm

Believe it or not, but that's a really good description - and it suggest that we might be "lucky" in that the issue might be specific to certain Canvas elements (=rendering primitives), VORs being based on OpenVG (Shiva), would be in line with my previous working thesis, that certain elements (or element specific culling) could play a role here.

I would suggest to ignore the Canvas map for now and see if you can trigger the same issue (crash/flickering) by running a self-contained Canvas snippets using only certain rendering primitives (e.g. Canvas.Text and Canvas.Image specifically), if those don't crash/flicker, you've nailed down the issue down to OpenVG (Canvas.Path) handling, which would not be far-fetched at this point.

But again, checking if the issue disappears without using the integrated UI, would also go a long way, because then we know whether it's related to the creation of the OpenGL context by Qt5.

Sharing your config really is helpful, but I don't think you need any airports/scenery - you should be able to trigger the issue/crash by just executing one of the Canvas examples:

If these two work without triggering the issue, native OSG elements work properly:

https://wiki.flightgear.org/Canvas_snippets
Code: Select all
var (width,height) = (320,160);
var title = 'My new Window';

# create a new window, dimensions are WIDTH x HEIGHT, using the dialog decoration (i.e. titlebar)
var window = canvas.Window.new([width,height],"dialog").set('title',title);


# adding a canvas to the new window and setting up background colors/transparency
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));

# Using specific css colors would also be possible:
# myCanvas.set("background", "#ffaac0");

# creating the top-level/root group which will contain all other elements/group
var root = myCanvas.createGroup();

# path is relative to $FG_ROOT (base package)
var path = "Textures/Splash1.png";
# create an image child for the texture
var child = root.createChild("image")
    .setFile(path)
    .setTranslation(100, 10)
    .setSize(130, 130);


Code: Select all
var (width,height) = (320,160);
var title = 'My new Window';

# create a new window, dimensions are WIDTH x HEIGHT, using the dialog decoration (i.e. titlebar)
var window = canvas.Window.new([width,height],"dialog").set('title',title);


# adding a canvas to the new window and setting up background colors/transparency
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));

# Using specific css colors would also be possible:
# myCanvas.set("background", "#ffaac0");

# creating the top-level/root group which will contain all other elements/group
var root = myCanvas.createGroup();


var myText = root.createChild("text")
      .setText("Hello world!")
      .setFontSize(20, 0.9)          # font size (in texels) and font aspect ratio
      .setColor(1,0,0,1)             # red, fully opaque
      .setAlignment("center-center") # how the text is aligned to where you place it
      .setTranslation(160, 80);     # where to place the text



If the following snippet triggers the issue, the issue is specific/related to Canvas.Path handling:

Code: Select all
var (width,height) = (320,160);
var title = 'My new Window';

# create a new window, dimensions are WIDTH x HEIGHT, using the dialog decoration (i.e. titlebar)
var window = canvas.Window.new([width,height],"dialog").set('title',title);


# adding a canvas to the new window and setting up background colors/transparency
var myCanvas = window.createCanvas().set("background", canvas.style.getColor("bg_color"));

# Using specific css colors would also be possible:
# myCanvas.set("background", "#ffaac0");

# creating the top-level/root group which will contain all other elements/group
var root = myCanvas.createGroup();

# change the background color
myCanvas.set("background", "#ffaac0");

# creating the top-level/root group which will contain all other elements/group
var root = myCanvas.createGroup();

var filename = "/Nasal/canvas/map/Images/boeingAirplane.svg";
var svg_symbol = root.createChild('group');
canvas.parsesvg(svg_symbol, filename);

svg_symbol.setTranslation(width/2,height/2);

#svg_symbol.setScale(0.2);
#svg_symbol.setRotation(radians)


EDIT: I tested these snippets locally in a non-Qt5 build, and cannot reproduce the issue at all - no flickering/no crashes whatsoever ...

Image

Also, MapStructure is working as expected, too:
Image

(pay attention to the osg-stats and the threading modes shown there)
Image
Image

PS: I cannot say for sure whether the issue is really related to the Qt5 integration, but the observation would be in line with advice and recommendations on the osg-users list, and also in line with comments shared by Edward, Emilian and others, who expressed previously concerns about OSG/OpenGL contexts not created by OSG, especially in the context of multi-threading. Personally, I cannot reproduce the issue in a non-Qt5 build.
And I suppose, Jules and Fernando might also be testing/developing using a non-Qt5 build (?)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: NDCanvas flickering problem

Postby SurferTim » Mon Aug 16, 2021 7:17 pm

I started my FG computer during a lull, and started FG without the launcher. The canvas map still flashes and crashes.

I wasn't able to do any more than that. More storm bands coming soon. :)
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1709
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: NDCanvas flickering problem

Postby Hooray » Mon Aug 16, 2021 7:25 pm

I tested on 64 bit Linux using a Nvidia GPU and Lubuntu 20.04 (openbox, with LXQt, Driver Version: 450.119.04) - however, my binary does not contain any Qt5 support at all, so I configured/built fgfs without Qt5 support/integration. Not sure if that's a red herring or not, let's see what others are reporting - but as you can see in the screen shots posted above, I cannot currently reproduce the issue - i.e. the Canvas seems to be working properly in CompositeViewer mode for me, including the ND/MapStructure specifically.

We'll see if merspieler can come up with any new data points to disprove/support the assumption that it might be related to Qt5.

Anyway, for the time being, not using the integrated launcher seems to be the primary difference ?

If in doubt, let's use the forum/wiki to share our exact fgfsrc/startup config, so that others can more easily test things and report back here.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: NDCanvas flickering problem

Postby SurferTim » Mon Aug 16, 2021 7:30 pm

Does your config setup start multiple windows?
If I don't use multiple windows, it doesn't flash and crash.
If I don't use multiple threads, it doesn't flash and crash.
If I use multiple windows and multiple threads...flash...flash...crash.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1709
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: NDCanvas flickering problem

Postby Hooray » Mon Aug 16, 2021 7:36 pm

Right, as you can see in those screen shots, there are multiple CompositeViewer (CV) windows being shown - for different views.
It is however correct, I am not setting those up during startup, but clone them manually after booting into fgfs (via the View menu).
Also, yes, I am using multi-threaded FGFS/OSG in CompositeViewer mode (again, you only need to refer to the osg stats in the screen shot, upper left corner, to see for yourself - that is why I suggested to also include osg stats in your screen shots, so that we can review all relevant info).
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: NDCanvas flickering problem

Postby SurferTim » Mon Aug 16, 2021 7:39 pm

My bad. I am not talking about CV windows. I am configuring multiple windows, each with one camera in
--config=/home/tim/config.xml

First window and camera is the left monitor.
Second window and camera is the center monitor.
Third window and camera is the right monitor.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1709
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: NDCanvas flickering problem

Postby Hooray » Mon Aug 16, 2021 7:49 pm

Yes, but merspieler already confirmed that he's seeing the same issue in compositeviewer mode, which makes sense because it happens to use the same CameraGroup APIs - using --prop:/sim/rendering/multithreading-mode=CullThreadPerCameraDrawThreadPerContext, I am still not seeing any crashes or excessive flickering, but I do see individuals symbols flickering every once in a while (especially those using OpenVG/ShivaVG), which would suggest that it could be culling related when multiple threads are being used.

However, no stability issues whatsoever (but I am also not testing any cockpit placements, only the PUI placement and native Canvas GUI dialogs), i.e. map-canvas.xml and the snippets I posted above.

Image

PS: Is this specific to having more than one additional view/window ?? Please do include osg-stats in future screen shots, thank you!
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: NDCanvas flickering problem

Postby SurferTim » Mon Aug 16, 2021 8:04 pm

If I see any flickering of vors, airports, fixes, that is a time bomb. I thought everything was fine for 30 minutes on one flight. Just a little flickering. Nothing to be concerned about. Then I found myself staring at my desktop.

If I use one window, no flashing or crashing.
If I use one thread, no flashing or crashing.

If I start multiple windows in my config xml file, then try to display canvas panels on those, flash and crash.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1709
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: NDCanvas flickering problem

Postby Hooray » Mon Aug 16, 2021 8:10 pm

I have been testing multiple CV windows on conjunction with aggressive OSG threading while displaying different Canvas dialogs, rendering all supported Canvas primitives, including a MapStructure map (which is the back-end of the Canvas ND), and couldn't reproduce this for the time being.

So, let's see if merspieler or someone else shows up with any useful info at this point.

PS: It's actually not bad if the issue can be reproduced in both, CameraGroup configs and CompositeViewer windows, since the issue then isn't specific to the new CV mode. For now (and neither the Qt5 integration layer), I still suppose what you are seeing could be related to culling from multiple OSG threads:

https://groups.google.com/g/osg-users/c ... rhn_wKe88J
In certain threading modes osgUtil::Renderer uses 2 SceneViews to cull and
render 2 consecutive frames concurently. Robert used to call it
doublebuffering, first of these SceneViews culls and renders odd frames and
second culls and renders even frames. Technically its possible to have them
working concurently. For example first SceneView may be culling the graph
while second SceneView is outputing(drawing) RenderStage.

This shouldn't be done when some resources of the scene graph are shared and
they change with each frame. If some of these resources are used by both
SceneViews there might be race conditions - it will be possible that some of
them will be changed by first SceneView traversals while also being accessed
by second SceneViews traversals.

What it means for PSSM ? I bet that PSSM:cull changes some shared resources
(like Texgen or some Uniforms for example) in the same time as Draw in
Second View uses them. Classic race condition - sometimes cull wins
sometimes draw wins. Hence some flicker.

DataVariance set to DYNAMIC could be used to serialize SceneViews traversals
and may potentially solve your issue.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: NDCanvas flickering problem

Postby SurferTim » Mon Aug 16, 2021 8:17 pm

Hooray wrote in Mon Aug 16, 2021 8:10 pm:So, let's see if merspieler or someone else shows up with any useful info at this point.

I agree. merspieler is the user who came up with the one thread stops the crashing workaround.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1709
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: NDCanvas flickering problem

Postby merspieler » Mon Aug 16, 2021 8:20 pm

TL;DR

what do you need from me?
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: NDCanvas flickering problem

Postby SurferTim » Mon Aug 16, 2021 8:22 pm

I think Hooray wants to check with you to see if you are still experiencing flickering and crashing if using multiple threads and multiple windows in your config xml file.
Fly low. Fly slow. Land on a dime. Twin Otter. https://github.com/SurferTim/dhc6p
My other aircraft is a Citation-X https://github.com/SurferTim/CitationX
PirateAir videos at https://www.youtube.com/user/SurferTim850
User avatar
SurferTim
 
Posts: 1709
Joined: Sun Dec 09, 2018 6:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2020.4.0
OS: Ubuntu 18.04

Re: NDCanvas flickering problem

Postby Hooray » Mon Aug 16, 2021 8:27 pm

Based on what we've seen and discussed so far, I still think that it's primarily Canvas.Path (specifically ShivaVG) that we need to look at - aggressive OSG threading is ... aggressive :D

Which is to say, the Canvas.Path element is not even using native OSG code, and the flickering shown in the screen shots, suggests that it's something related to Canvas.Path handling.

Once you have a simple test case that can reliably reproduce the issue, it would make sense to run the test inside a gdb session, and you'll probably see something related to $SG_SRC/canvas/elements/shiva - which is where the 3rd party sources related to ShivaVG reside.

Shiva itself cannot be considered thread-safe, but may be invoked from multiple threads in fgfs, once fgfs is being used in conjunction with compositeviewer:

https://github.com/ileben/ShivaVG/blob/master/README
No multi-threading support has been implemented yet.

This also pointed out in the Qt 4 docs, when shiva was used:

https://doc.qt.io/archives/qt-4.8/openvg.html
The paint engine is not yet thread-safe, so it is not recommended for use in threaded Qt applications that draw from multiple threads. Drawing should be limited to the main GUI thread.


For now, I suppose that the shiva stuff is called by C++ code from multiple threads - otherwise, seeing flickering specific to map elements implemented via OpenVG (Canvas.Path) would not make sense, which is why I asked previously if text/image are affected or not - because according to the screen shots, it's primarily vector graphics that are affected ?

If in doubt, please review my posting above, the one containing all the screen shots and the code snippets to see for yourself if you can reproduce the crash by executing the first 2 snippets via the Nasal console.

Let us know if you need help doing that.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

PreviousNext

Return to Canvas

Who is online

Users browsing this forum: No registered users and 4 guests