Multi-threaded FlightGear/OSG and Canvas flickering solved

[Hooray]

I thought the developers have done a good job with the multi-threading. I haven't had a problem with it since I've been here until these canvas displays, and I've had problems with them from the start.
[...]
I see it as the "ugly stepchild" is not FG threading, but canvas.

That's a misconception unfortunately ...

Like I pointed out already 3 days ago (see the posting dated Sun Aug 15, 2021 11:45 am), the Canvas system is using a 3rd party library to render vector graphics, and this component (called "Shiva") isn't thread-safe.

This component is neither provided by the FlightGear team, nor by the Canvas developer/s.

FlightGear itself makes only very marginal use of multi-threading, most threading is provided by OpenSceneGraph itself.

What you are now running into is actually symptomatic of that - since the type of threading you're wanting to use, isn't even provided by FlightGear, but rather implicitly by OpenSceneGraph (OSG).

Unfortunately, for that to work properly, C++ developers need to make use of certain coding patterns/protocols.

And in the case of Shiva, that isn't trivial, because Shiva itself isn't designed to be used from multiple threads - but your specific use-case, will trigger the implicit creation of multiple threads by OSG itself.

Basically, using your threading mode, OSG will simply assume that all FG related code (including Shiva) is safe to be called from multiple threads, which is simply wrong - and which is not a factor as long as people aren't calling Shiva from multiple threads.

In the backtrace I shared with you yesterday, you can see that there are other threading issues, unrelated to the Canvas system, or to the Shiva component specifically.

FlightGear itself is pathetic when it comes to multi-threading, and there's a long standing history of that, including literally hundreds of postings by some of the most senior contributors - for a basic summary of the issue, refer to (these being quotes/comments from core devs involved in the decision making at the time):

https://wiki.flightgear.org/Multi-Threa ... FlightGear
https://wiki.flightgear.org/Technical_R ... _Simulator
https://wiki.flightgear.org/High-Level_Architecture

Fixing your issue specifically entails disabling threading for the Shiva component, by marking the Canvas.Path drawable as osg::Object::DYNAMIC and probably using explicit OpenThreads based synchronization. Either way, that won't solve other threading issues - and as you can tell by looking at the wiki, a number of core developers have been sharing that FlightGear is facing serious multithreading issues, for several years (look at comments by Torsten Dreyer, who was the primary core dev at the time involved in testing on actual multi-core/multi-GPU hardware, who complained about multithreading issues long before the Canvas system got introduced in FlightGear):

https://wiki.flightgear.org/Howto:Activ ... PU_support

[Hooray]

basically, open $SG_SRC/canvas/elements/CanvasElement.cxx and look for something called "getOrCreateStateSet" - there should be some kind of conditional block, and we'd need to set the data variance to DYNAMIC there, but not just on the drawable itself, but the associated state set.
[...]
Once that is working, we have a test case - and we can then clone a handful of views in CV mode, and keep the whole thing running over night - if Canvas.Text/Canvas.Image don't cause any flickering/crashing, then it'd be worth trying fixing up Canvas.Path to use an explicit Object::DYNAMIC StateSet to tell OSG to wait and not call the code from multiple threads.
[...]
An interim solution would be using explicit synchronization (locks/mutexes) to tell OSG not to use threading for Canvas.Path based drawables - according to the docs, that should be possible by setting the data variance to osg::Object::DYNAMIC - but for the drawable itself that is already being done, thus CanvasPath.cxx will probably need a review to add explicit OSG/OpenThread based synchronization (which is probably something best discussed with Fernando, Richard and James on the devel list/issue tracker).

I guess I need a patch to test that...

For the sake of troubleshooting and to start experimenting with potential approaches/fixes: You would need to look up the Geode and StateSet of each Canvas element and register its data variance as "DYNAMIC" - in addition, open CanvasPath.cxx and locate the drawImplementation() callback - add a mutex/lock to it, to see if you can prevent it from being called from multiple OSG threads. Next, start testing, tinkering with different settings, to see if the flickering or crash persists.

For testing purposes, use CullThreadPerCameraDrawThreadPerContext in conjunction with the stress test posted here: Multithreaded CompositeViewer and Canvas flickering and keep the whole thing running with the Map canvas dialog or the development extensions/canvas map active. If you can trigger a segfault, please share a backtrace (gdb with symbols)


Something along these lines (which will need to be tested and adapted as needed, depending on user feedback - it woud probaly make sense to move the mutex into global scope though)

Code: Select all

diff --git a/simgear/canvas/ODGauge.cxx b/simgear/canvas/ODGauge.cxx
index 8e9f8c2b..74ed41f0 100644
--- a/simgear/canvas/ODGauge.cxx
+++ b/simgear/canvas/ODGauge.cxx
@@ -241,6 +241,8 @@ namespace canvas
     updateStencil();
 
     osg::StateSet* stateSet = camera->getOrCreateStateSet();
+    stateSet->setDataVariance(osg::Object::DYNAMIC);
+
     stateSet->setMode(GL_LIGHTING, osg::StateAttribute::OFF);
     stateSet->setMode(GL_CULL_FACE, osg::StateAttribute::OFF);
     stateSet->setMode(GL_FOG, osg::StateAttribute::OFF);
diff --git a/simgear/canvas/elements/CanvasElement.cxx b/simgear/canvas/elements/CanvasElement.cxx
index 1ac4d5e6..2b2a766d 100644
--- a/simgear/canvas/elements/CanvasElement.cxx
+++ b/simgear/canvas/elements/CanvasElement.cxx
@@ -811,18 +811,25 @@ namespace canvas
 
     auto geode = new osg::Geode;
     geode->addDrawable(_drawable);
+    geode->setDataVariance(osg::Object::DYNAMIC);
     _scene_group->addChild(geode);
+    getOrCreateStateSet();
   }
 
   //----------------------------------------------------------------------------
   osg::StateSet* Element::getOrCreateStateSet()
   {
+    osg::StateSet* ss = nullptr;
     if( _drawable.valid() )
-      return _drawable->getOrCreateStateSet();
+      ss = _drawable->getOrCreateStateSet();
     else if( _scene_group.valid() )
-      return _scene_group->getOrCreateStateSet();
-    else
-      return nullptr;
+      ss =  _scene_group->getStateSet();
+   
+    if (ss) {
+      ss->setDataVariance(osg::Object::DYNAMIC);
+      SG_LOG(SG_GL, SG_ALERT, "canvas::Element: setting up dyn variance for state set");
+    }
+    return ss;
   }
 
   //----------------------------------------------------------------------------
diff --git a/simgear/canvas/elements/CanvasPath.cxx b/simgear/canvas/elements/CanvasPath.cxx
index 90057c85..0638912c 100644
--- a/simgear/canvas/elements/CanvasPath.cxx
+++ b/simgear/canvas/elements/CanvasPath.cxx
@@ -215,7 +215,7 @@ namespace canvas
         _path_element(path)
       {
         setSupportsDisplayList(false);
-        setDataVariance(Object::DYNAMIC);
+        setDataVariance(osg::Object::DYNAMIC);
 
         setUpdateCallback(new PathUpdateCallback());
       }
@@ -391,6 +391,9 @@ namespace canvas
         if( _attributes_dirty & PATH )
           return;
 
+   static OpenThreads::Mutex _mutex;
+   OpenThreads::ScopedLock<OpenThreads::Mutex> lock(_mutex);
+
         osg::State* state = renderInfo.getState();
         assert(state);
 
@@ -461,6 +464,7 @@ namespace canvas
         if( was_blend_enabled )   glEnable(GL_BLEND);
         if( was_stencil_enabled ) glEnable(GL_STENCIL_TEST);
         if( blend_func )          blend_func->apply(*state);
       }
 
       osg::BoundingBox getTransformedBounds(const osg::Matrix& mat) const
@@ -592,6 +596,7 @@ namespace canvas
 
       Path *_path_element;
 
       mutable VGPath    _path {VG_INVALID_HANDLE};
       mutable VGPaint   _paint {VG_INVALID_HANDLE};
       mutable VGPaint   _paint_fill {VG_INVALID_HANDLE};



[merspieler]

That patch certainly fixed the flickering... I've had a differnet issue but that might be me, messing the patch up (git complained about the patch being invalid, so I patched it by hand, might have overlooked something?)

[Hooray]

it should fix both, the flickering and the crash (race condition), because it's using a mutex to tell OSG to stop and wait.

If in doubt, please post another gdb backtrace

[merspieler]

It didn't crash... only other stuff was wrong (tho with mangohud I still could trigger crashes... but that's not a default use case):

1st, before adding the cloned view, the PFD already doesn't look right:


2nd: When ever I add or close a clone view, the Upper ECAM goes black.

I can bring it back to life by clicking on it and thus opening the canvas in the popup which also triggers this log message: "187.04 [ALRT]:general Canvas constructor: node=/canvas/by-index/texture[15]"

Would be great if someone else could test this with the A320.

[Hooray]

To be honest, I am not sure I understand your bug report ...

Are you the only person to see this, how about other people using the same aircraft/settings (and binary/version)?
Is the issue specific to any particular aircraft/instrument and how about other Canvas displays ?
How about Canvas dialogs (other placements) ?
Are you running any other OpenGL-hooking apps ?

If you say something isn't looking right before cloning any additional views, why do you think the issue is related to CompositeViewer/the Canvas ??


PS: Is the ECAM even implemented as a Canvas ??

[merspieler]

> Are you the only person to see this, how about other people using the same aircraft/settings (and binary/version)?
I didn't hear from anyone else with this yet (A320 3D branch, SG commit 6d891eae7e5a666567ba47bf958bbda37a5822c1 with your patch applied)

> Is the issue specific to any particular aircraft/instrument and how about other Canvas displays ?
Thie issues seems to be bound the the PFD and UECAM respectively, the ND, LECAM and MCDU look just fine now.

> How about Canvas dialogs (other placements) ?
the issue on the canvas map is fixed by the patch

> If you say something isn't looking right before cloning any additional views, why do you think the issue is related to CompositeViewer/the Canvas ??
What I'm saying is that those issues are regressions from the patch as before, without clone view, it worked all just fine.

> Are you running any other OpenGL-hooking apps ?
as said before, mangohud is disabled for those tests

> PS: Is the ECAM even implemented as a Canvas ??
yes

[Hooray]

I didn't hear from anyone else with this yet (A320 3D branch, SG commit 6d891eae7e5a666567ba47bf958bbda37a5822c1 with your patch applied)

And you could reproduce the issue reliably ?
Can you create/save a pre-recorded flight (fgtape) and can you repro the issue that way ?
How about tinkering with other threading models while replaying the flight, does that have any impact ?

Also, which patch exactly are you using - the one from the other thread, or this thread ?
The point being, in this thread, I also added explicit osg::Object::DYNAMIC attributes to a bunch of Canvas elements - whereas in the other thread, I told SurferTim to just try the mutex only. That is why it would be relevant which patch you're referring to ?

Thie issues seems to be bound the the PFD and UECAM respectively, the ND, LECAM and MCDU look just fine now.

Can you specifically name the elements on the PFD/UECAM that are affected, or rather their underlying canvas elements ? (text, image, path)

What I'm saying is that those issues are regressions from the patch as before, without clone view, it worked all just fine.

I am not sure I can parse that response ... did you see the same issue without the patch (with or without composite viewer) ?

as said before, mangohud is disabled for those tests

You have previously reported apparent bugs that ended up being issues due to OpenGL hooking/injection - which is why I was specifically asking whether you are also using other OpenGL injection tools (profilers/debuggers etc), and if you are sure that the issue can also be reproduced without them being present on the system ?
No offense, but I don't quite see how this could be related to the patch specifically ? It being "instrument specific" seems also weird to me ...

In other words, there are no problems with the PFD/ECAM if you are using the unpatched binary in conjunction with CV (enabled) ?
If so, I'd suggest to try the mutex-only patch posted in the other thread - since that's apparently what's working for SurferTim

[Johan G]

One post was split of to the new topic Multithreading and TerraSync scenery download.

[merspieler]

And you could reproduce the issue reliably ?

100% of the time.

Also, which patch exactly are you using - the one from the other thread, or this thread ?

Yours from this thread

Can you specifically name the elements on the PFD/UECAM that are affected, or rather their underlying canvas elements ? (text, image, path)

The pfd has no actual element, that's missing...the AI isn't clipped but extended over the entire canvas(the elements are that big but are usually clipped) the UECAM was the whole canvas, once the canvas came bake, it showed as usual.

did you see the same issue without the patch (with or without composite viewer) ?

No, those issues appeared with the patch hence a regression...

In other words, there are no problems with the PFD/ECAM if you are using the unpatched binary in conjunction with CV (enabled) ?

There are no issues with it, until I add the clone view which causes the issue this thread is about.

If so, I'd suggest to try the mutex-only patch posted in the other thread - since that's apparently what's working for SurferTim

Which other patch?

[Hooray]

I was referring to SurferTim's other thread, where I posted the mutex-only variant of the same patch.

However, for now, you can probably ignore all patches if you are using the latest nightly (or if you are building from source).
Since the recent commits moving Canvas cameras out of the scene graph into the viewer are apparently "fixing" this issue by implicitly adding OSG-level serialization due to shared context usage, regardless of the chosen OSG threading mode.

If in doubt, please revert any local changes and test the latest sources from git (sg+fg), and do report back (devel-list/issue tracker) if there are any remaining/new issues.

[merspieler]

On latest next, it crashes, as soon as I hit "Add Clone View" with the following backtrac:

Code: Select all

fgfs: ../../subprojects/imgui-1.81/imgui.cpp:3389: ImGuiIO& ImGui::GetIO(): Assertion `GImGui != __null && "No current context. Did you call ImGui::CreateContext() and ImGui::SetCurrentContext() ?"' failed.
[Thread 0x7ffe0979f700 (LWP 1422679) exited]
--Type <RET> for more, q to quit, c to continue without paging--c

Thread 144 "fgfs" received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffe467fc700 (LWP 1422548)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50   ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt full
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
        set = {__val = {4096, 140737290516076, 4222451712, 140734939420800, 140734939420901, 140734939420800, 140734939420800, 140734939421007, 140734939421100, 140734939420800, 140734939421100, 0, 0, 0, 0, 0}}
        pid = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
#1  0x00007ffff42fb537 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x7fff68125880, sa_sigaction = 0x7fff68125880}, sa_mask = {__val = {0, 140734939420800, 207, 0, 0, 0, 21474836480, 140728898420738, 140730081195120, 140737291647952, 140737291632936, 0, 15018367828010315520,
              140737291616216, 140737154715648, 140737291632936}}, sa_flags = -134633696, sa_restorer = 0xd3d}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007ffff42fb40f in __assert_fail_base (fmt=0x7ffff4464128 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x7ffff7f9aaf8 "GImGui != __null && \"No current context. Did you call ImGui::CreateContext() and ImGui::SetCurrentContext() ?\"",
    file=0x7ffff7f9a720 "../../subprojects/imgui-1.81/imgui.cpp", line=3389, function=<optimized out>) at assert.c:92
        str = 0x7fff68125880 "@\n\022h\377\177"
        total = 4096
#3  0x00007ffff430a662 in __GI___assert_fail (assertion=0x7ffff7f9aaf8 "GImGui != __null && \"No current context. Did you call ImGui::CreateContext() and ImGui::SetCurrentContext() ?\"", file=0x7ffff7f9a720 "../../subprojects/imgui-1.81/imgui.cpp", line=3389,
    function=0x7ffff7f9e96c "ImGuiIO& ImGui::GetIO()") at assert.c:101
No locals.
#4  0x00007ffff7ef4674 in ImGui::GetIO() () from /usr/lib/mangohud/lib/x86_64-linux-gnu/libMangoHud.so
No symbol table info available.
#5  0x00007ffff7e9c675 in MangoHud::GL::ImGui_ImplOpenGL3_CreateFontsTexture() () from /usr/lib/mangohud/lib/x86_64-linux-gnu/libMangoHud.so
No symbol table info available.
#6  0x00007ffff7e9d91d in MangoHud::GL::ImGui_ImplOpenGL3_CreateDeviceObjects() [clone .isra.0] () from /usr/lib/mangohud/lib/x86_64-linux-gnu/libMangoHud.so
No symbol table info available.
#7  0x00007ffff7e9dcfd in MangoHud::GL::ImGui_ImplOpenGL3_NewFrame() () from /usr/lib/mangohud/lib/x86_64-linux-gnu/libMangoHud.so
No symbol table info available.
#8  0x00007ffff7e9de4e in MangoHud::GL::imgui_render(unsigned int, unsigned int) () from /usr/lib/mangohud/lib/x86_64-linux-gnu/libMangoHud.so
No symbol table info available.
#9  0x00007ffff7e9f92b in do_imgui_swap(void*, void*) [clone .part.0] () from /usr/lib/mangohud/lib/x86_64-linux-gnu/libMangoHud.so
No symbol table info available.
#10 0x00007ffff7e9faaf in glXSwapBuffers () from /usr/lib/mangohud/lib/x86_64-linux-gnu/libMangoHud.so
No symbol table info available.
#11 0x00007ffff76752e1 in osgViewer::GraphicsWindowX11::swapBuffersImplementation() () from ../../openscenegraph/lib/libosgViewer.so.162
No symbol table info available.
#12 0x00007ffff72a14ed in osg::SwapBuffersOperation::operator()(osg::GraphicsContext*) () from ../../openscenegraph/lib/libosg.so.162
No symbol table info available.
#13 0x00007ffff731f24d in osg::OperationThread::run() () from ../../openscenegraph/lib/libosg.so.162
No symbol table info available.
#14 0x00007ffff72a1a7a in non-virtual thunk to osg::GraphicsThread::run() () from ../../openscenegraph/lib/libosg.so.162
No symbol table info available.
#15 0x00007ffff70ed32f in OpenThreads::ThreadPrivateActions::StartThread(void*) () from ../../openscenegraph/lib/libOpenThreads.so.21
No symbol table info available.
#16 0x00007ffff666aea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
        ret = <optimized out>
        pd = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140730081199872, 7676043309297021483, 140737488339806, 140737488339807, 140730081196992, 8396800, -7675747542676950485, -7676064004012002773}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0,
              cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
#17 0x00007ffff43d3def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.


Looking at the frozen image, you can see that some canvases have already been initialized:


I've tried it again and that time the PFD and ND already showed up before the crash.

[Hooray]

Your BT says ImGui, so that's probably unrelated to the Canvas?

If so, it's best to discuss such issues on the devel list/issue tracker.

[merspieler]

clean rebuild solved at least the instant crash...

But it still shows the flickering, some canvases, this time UECAM and DCDU (small displays above the MCDUs) where just black and when I've clicked the UECAM to bring up the popup, it crashed again:

Code: Select all

realloc(): invalid next size
--Type <RET> for more, q to quit, c to continue without paging--c

Thread 85 "fgfs" received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffe4a7fc700 (LWP 1488525)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50   ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt full
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
        set = {__val = {4096, 38069568, 140734267129888, 3456, 140734279744192, 2, 131072, 140737292406481, 140734279048336, 131072, 140734279048336, 93825053153648, 28, 140737168782295, 140734279048336, 140737292399052}}
        pid = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
#1  0x00007ffff44bb537 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x1, sa_sigaction = 0x1}, sa_mask = {__val = {140737292647233, 1, 0, 0, 140737170749034, 39671, 1492721456, 0, 93825888384372, 93825894334568, 140737163070715, 93825888458456, 93825888507728, 0, 0, 0}},
          sa_flags = 1249881708, sa_restorer = 0x0}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007ffff4514768 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff4622e2d "%s\n") at ../sysdeps/posix/libc_fatal.c:155
        ap = {{gp_offset = 24, fp_offset = 0, overflow_arg_area = 0x7ffe4a7faea0, reg_save_area = 0x7ffe4a7fae30}}
        fd = <optimized out>
        list = <optimized out>
        nlist = <optimized out>
        cp = <optimized out>
#3  0x00007ffff451ba5a in malloc_printerr (str=str@entry=0x7ffff462116d "realloc(): invalid next size") at malloc.c:5347
No locals.
#4  0x00007ffff451f9bc in _int_realloc (av=av@entry=0x7ffff4654b80 <main_arena>, oldp=oldp@entry=0x55558b30e700, oldsize=oldsize@entry=144, nb=272) at malloc.c:4564
        newp = <optimized out>
        newsize = <optimized out>
        newmem = <optimized out>
        next = 0x55558b30e790
        remainder = <optimized out>
        remainder_size = <optimized out>
        __PRETTY_FUNCTION__ = "_int_realloc"
        nextsize = <optimized out>
#5  0x00007ffff4520c56 in __GI___libc_realloc (oldmem=0x55558b30e710, bytes=256) at malloc.c:3235
        ar_ptr = 0x7ffff4654b80 <main_arena>
        nb = 140737292082401
        newp = <optimized out>
        hook = <optimized out>
        oldp = 0x55558b30e700
        oldsize = 144
        __PRETTY_FUNCTION__ = "__libc_realloc"
#6  0x0000555556635fe4 in shVector2ArrayReserveAndCopy (newsize=32, a=0x55558a5ac960) at /games/flightgear-new/simgear/simgear/canvas/ShivaVG/src/shArrayBase.h:190
        newitems = <optimized out>
        newitems = <optimized out>
#7  shVector2ArrayReserveAndCopy (a=0x55558a5ac960, newsize=32) at /games/flightgear-new/simgear/simgear/canvas/ShivaVG/src/shArrayBase.h:181
        newitems = 0x0
#8  0x0000555556636106 in shVector2ArrayPushBackP (a=a@entry=0x55558a5ac960, item=item@entry=0x7ffe4a7fafb0) at /games/flightgear-new/simgear/simgear/canvas/ShivaVG/src/shArrayBase.h:240
No locals.
#9  0x0000555556625daf in shPushStrokeTri (p3=0x55558f884c90, p2=0x7ffe4a7fafb8, p1=0x7ffe4a7fafb0, p=0x55558a5ac900) at /games/flightgear-new/simgear/simgear/canvas/ShivaVG/src/shGeometry.c:387
No locals.
#10 shStrokeCapRound (p=p@entry=0x55558a5ac900, c=c@entry=0x55558f884c90, start=start@entry=1, t=<optimized out>, t=<optimized out>) at /games/flightgear-new/simgear/simgear/canvas/ShivaVG/src/shGeometry.c:482
        a = 6
        ang = 1.57079637
        cosa = <optimized out>
        sina = <optimized out>
        p1 = {x = 484.823517, y = 747.75238}
        p2 = {x = 484.98111, y = 747.269653}
        steps = 12
        tt = {x = -0.357207924, y = 1.91193235}
#11 0x0000555556627eca in shStrokePath (c=c@entry=0x7fff7894d300, p=p@entry=0x55558a5ac900) at /games/flightgear-new/simgear/simgear/canvas/ShivaVG/src/shGeometry.c:718
        w = 1.94501495
        mlimit = 4
        vertsize = 10
        contourStart = <optimized out>
        contourLength = 13
        start = <optimized out>
        end = <optimized out>
        loop = <optimized out>
        close = <optimized out>
        segend = <optimized out>
        i1 = <optimized out>
        i2 = <optimized out>
        v1 = <optimized out>
        v2 = <optimized out>
        p1 = 0x55558f884c90
        p2 = 0x55558f884ca8
        d = {x = -0.982991099, y = -0.183653042}
        t = {x = 0.357207924, y = -1.91193235}
        dprev = {x = 0, y = 0}
        tprev = {x = 0, y = 0}
--Type <RET> for more, q to quit, c to continue without paging--c
        norm = 19.2886505
        cross = <optimized out>
        mlength = <optimized out>
        l1 = {x = 483.426392, y = 745.000549}
        r1 = {x = 482.711975, y = 748.824402}
        l2 = {x = 464.46582, y = 741.45813}
        r2 = {x = 463.751404, y = 745.281982}
        lprev = {x = 0, y = 0}
        rprev = {x = 0, y = 0}
        dashIndex = 0
        dashLength = <optimized out>
        strokeLength = 0
        dashSize = 2
        dashPattern = 0x7fff788fffc0
        dashOn = 1
        dash1 = {x = -1.89763227e-32, y = 3.0611365e-41}
        dash2 = {x = <optimized out>, y = <optimized out>}
        dashL1 = {x = 1.40129846e-45, y = 0}
        dashR1 = {x = 1.40129846e-45, y = 0}
        dashL2 = {x = 1.40129846e-45, y = 0}
        dashR2 = {x = 6.24843628e+13, y = 3.0611365e-41}
        nextDashLength = <optimized out>
        dashOffset = <optimized out>
#12 0x000055555663590c in vgDrawPath (path=0x55558a5ac900, paintModes=<optimized out>) at /games/flightgear-new/simgear/simgear/canvas/ShivaVG/src/shPipeline.c:396
        p = 0x55558a5ac900
        mi = {m = {{0, 0, 0}, {0, 0, 0}, {0, 0, 2.80259693e-44}}}
        mgl = {1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1}
        fill = 0x7fff60124870
        stroke = 0x7fff40bc7f00
        rect = <optimized out>
        context = 0x7fff7894d300
#13 0x00005555564c4b3c in simgear::canvas::Path::PathDrawable::drawImplementation (this=0x55555d7f3610, renderInfo=...) at /games/flightgear-new/simgear/simgear/canvas/elements/CanvasPath.cxx:453
        state = 0x55558f8a9260
        was_blend_enabled = true
        was_stencil_enabled = false
        blend_func = 0x55558acf8ab0
        err = <optimized out>
        __FUNCTION__ = "drawImplementation"
#14 0x00005555566cd610 in osg::Drawable::drawInner (renderInfo=..., this=0x55555d7f3610) at /games/flightgear-new/install/openscenegraph/include/osg/Drawable:276
No locals.
#15 osg::Drawable::draw (renderInfo=..., this=0x55555d7f3610) at /games/flightgear-new/install/openscenegraph/include/osg/Drawable:617
        state = <optimized out>
        useVertexArrayObject = <optimized out>
        state = <optimized out>
        useVertexArrayObject = <optimized out>
        contextID = <optimized out>
        vas = <optimized out>
        setVASProxy = {_state = <optimized out>}
        contextID = <optimized out>
        globj = <optimized out>
#16 osg::Drawable::draw (this=0x55555d7f3610, renderInfo=...) at /games/flightgear-new/install/openscenegraph/include/osg/Drawable:549
        state = <optimized out>
        useVertexArrayObject = false
#17 0x00007ffff7a0da8e in osgUtil::RenderLeaf::render(osg::RenderInfo&, osgUtil::RenderLeaf*) () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#18 0x00007ffff7a08598 in osgUtil::RenderBin::drawImplementation(osg::RenderInfo&, osgUtil::RenderLeaf*&) () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#19 0x00007ffff7a08fa3 in osgUtil::RenderBin::draw(osg::RenderInfo&, osgUtil::RenderLeaf*&) () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#20 0x00007ffff7a085db in osgUtil::RenderBin::drawImplementation(osg::RenderInfo&, osgUtil::RenderLeaf*&) () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#21 0x00007ffff7a12d60 in osgUtil::RenderStage::drawImplementation(osg::RenderInfo&, osgUtil::RenderLeaf*&) () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#22 0x00007ffff7a08fa3 in osgUtil::RenderBin::draw(osg::RenderInfo&, osgUtil::RenderLeaf*&) () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#23 0x00007ffff7a11f13 in osgUtil::RenderStage::drawInner(osg::RenderInfo&, osgUtil::RenderLeaf*&, bool&) () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#24 0x00007ffff7a13c0d in osgUtil::RenderStage::draw(osg::RenderInfo&, osgUtil::RenderLeaf*&) () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#25 0x00007ffff7a0e03b in osgUtil::RenderStage::drawPreRenderStages(osg::RenderInfo&, osgUtil::RenderLeaf*&) () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#26 0x00007ffff7a1f4f2 in osgUtil::SceneView::draw() () from ../../openscenegraph/lib/libosgUtil.so.162
No symbol table info available.
#27 0x00007ffff77f0b0f in osgViewer::Renderer::draw() () from ../../openscenegraph/lib/libosgViewer.so.162
No symbol table info available.
#28 0x00007ffff7461a81 in osg::GraphicsContext::runOperations() () from ../../openscenegraph/lib/libosg.so.162
No symbol table info available.
#29 0x00007ffff74e124d in osg::OperationThread::run() () from ../../openscenegraph/lib/libosg.so.162
No symbol table info available.
#30 0x00007ffff7463a7a in non-virtual thunk to osg::GraphicsThread::run() () from ../../openscenegraph/lib/libosg.so.162
No symbol table info available.
#31 0x00007ffff72ad32f in OpenThreads::ThreadPrivateActions::StartThread(void*) () from ../../openscenegraph/lib/libOpenThreads.so.21
No symbol table info available.
#32 0x00007ffff682aea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
        ret = <optimized out>
        pd = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140730148308736, -5458565547996675119, 140737488339870, 140737488339871, 140730148305856, 8396800, 5459439657709798353, 5458550263774381009}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
#33 0x00007ffff4593def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.


[Hooray]

That doesn't make sense, what's your exakt SG/FG version that you're building ?
Are you sure that you re-pulled and rebuilt SG+FG ?
Using shared GL contexts, should implicitly serialize most OSG threading that we'd normally want ...

Again, it's important to actually build the proper sources - without stuff like mangohud/imgui etc - the ossues you have been sharing are unrelated to FlightGear itself.