Board index FlightGear Development Effects and shaders

Canvas:View development

An exciting "new" option in FlightGear, that includes reflections, lightmaps, the particle system etc.. A lot is yet to be discovered/implemented!

Canvas:View development

Postby Hooray » Sat Sep 02, 2017 1:27 pm

So, based on F-JJTH's groundwork, Icecode GL has been prototyping a Canvas-enabled version that renders arbitrary view-manager views to an offscreen texture that can be added to a canvas - the idea being to use this for implementing features that are currently not supported, tail-cams (A380) being one of the most important ones. Another use-case would be "mirrors" (which are going to be more complicated, because I don't think the view-manager can currently be told to render a mirror way, can it?)

http://wiki.flightgear.org/Howto:Canvas ... ra_Element
Image

This isn't restricted to showing just a single view, it can show arbitrary views - and it can also be told to disable the sykdome:
Image

Apart from that, there are other ideas such as rendering synthetic terrain views, using a customizable effect/shader - probably in conjunction with draw-masks per view to enable/disable scenery features (think weather/environment, AI traffic, random buildings) - aka stuff that you would not expect to show up on your PFD:

Gear view in cockpit computer
Hooray wrote:I do think that this work would nicely fit into the bigger picture of supporting G1000-style avionics built in Canvas space, as more and more PFDs do render some kind of synthetic terrain view - for example:
Image

Image

As has been mentioned previously, once this new element is added, we will need to revisit the whole idea to also support custom shaders/effects to customize the appearance of the display, as well as support draw-masks (osg::Switch nodes) for things like the skydome, AI objects, 3D models, random buildings etc - so that these can be specifically excluded in such "views", while a proper tailcam would obviously still want to render those - so we need to make this configurable per view element



This thread is intended to gather/archive community feedback related to this, because so far we've been conducting all talks in private - but we realize that it may be better to get more people involved in help refine the requirements of a corresponding View element - even though the initial version is likely going to be just that: a Canvas element to render view-manager views to a texture.

Ideas/feedback appreciated !
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas:View development

Postby Thorsten » Sat Sep 02, 2017 6:17 pm

As has been mentioned previously, once this new element is added, we will need to revisit the whole idea to also support custom shaders/effects to customize the appearance of the display


I'm not sure I understand how that's supposed to work and deliver what displays might need. I guess this requires some thought.

If you render for instance the FLIR view, this isn't 'just' a post-processing of the final pixel - the temperature info is added per draw, and that information would get lost if you render to texture and then try to post-process - you'd need to render into a buffer, storing all required info in extra buffers to make that kind of thing work.

It should conceptually be possible to re-configure effects by a kind of 'overlay' before rendering the cam view - in that way, quality level or IR filter could be activated, while the rest of the effects hierarchy (rendering framework selection etc.) would remain intact - but that'd be prone to misuse.

Dependent on the size of the cam texture, the cam view might be reasonably cheap on the fragment shader at least...

I don't have a good answer at the moment, but we should start thinking soon.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Canvas:View development

Postby Hooray » Sat Sep 02, 2017 7:37 pm

actually, there is no hurry at all - while what we have looks pretty good, it will require someone more familiar with OSG/OpenGL to fit into the bigger picture of things - fortunately, Stuart offered to help review this thing.

Regarding your concerns specifically: The initial version of this is unlikely to have any configurable support for effects/shaders other than what the fgfs binary currently has to offer. As you may remember, the skydome effect needed to be explicitly added for the view to work at all.

Then again, if you are aware of a simple view-related effect/shader that we could use for testing purposes, that would obviously be useful to identify potential issues.
Ideally, that would be something that is neither specific to Rembrandt nor to ALS. The point being that the Canvas view element has been developed without any of these frameworks in mind, and that we fully realize that this may indeed require further thought.

But the initial focus will really just be to implement and provide the building blocks that enable aircraft developers to add tailcam-related functionality to their aircraft without having to know any C++/OSG details.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas:View development

Postby icecode » Sat Sep 02, 2017 8:45 pm

As Hooray said, for now we are emphasizing in implementing camera views in Canvas. We also keep in mind that this might be an important groundwork for what might be RTT support for effects/shaders. For that we obviously need a link between Canvas and Effects. It's kind of a good thing that both systems are completely separated and don't know of each other, we are given more freedom when it comes to joining them. This link is kind of delicate and has to be well planned out. For example Hooray (and also Thorsten, but I'm not too sure) suggested per-element effects. I'd be kind of useful to have that, but would that give us the flexibility we need? Shaders can recieve many input textures and modern shaders support MRT (Multiple Render Targets) too. It makes sense to come up with some kind of well defined link between Canvas and Effects that allows for inputting/outputting canvases via effects. This would allow the same things that per-element effects would allow but with more flexibility and less overlapping functionality. I don't feel like I know the FG codebase well enough to think of elegant and future-proof ways to establish this link though.
icecode
 
Posts: 708
Joined: Thu Aug 12, 2010 1:17 pm
Location: Spain
Version: next
OS: Fedora

Re: Canvas:View development

Postby Hooray » Sat Sep 02, 2017 8:55 pm

I don't think Thorsten stated any preference regarding supporting effects per canvas/placement or at the element level - but I did indeed suggest the latter, because I think it is more flexible/powerful and easier to integrate in a holistic fashion by extending sc::Element to have an effects sub-tree for each element. For starters, this could simply take a filename and some attribute mapping.

Then again, I also don't claim to know enough about the effects framework and especially its limitations in this department - one of the more commonly made suggestions is actually supporting UBOs (Uniform Buffer Objects): http://wiki.flightgear.org/Uniform_Buffer_Objects

This is a suggestion independelty made by several long-term core developers, so it's probably worth investigating - but TheTom mentioned already a while ago, that the effects framework would need changing to support UBOs.

Personally, I was tinkering with a scheme to treat/register any Canvas texture (as a whole) as a "material" using SGMaterialLib, at that point all the effects machinery would be available to Canvas space without the Canvas system having to know anything about effects/shaders. And actually it was Thorsten who discouraged pursuing that approach, because there's an increasing trend to go all in on GLSL/shader-based land-classes. I cannot seem to find the corresponding topics, but I think this was discussed on the devel list.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas:View development

Postby Thorsten » Sun Sep 03, 2017 6:37 am

The initial version of this is unlikely to have any configurable support for effects/shaders other than what the fgfs binary currently has to offer.


I'm already lost with the meaning of that sentence.

So, let's try to get us all on the same page:

When we render a scene, there's multiple effects used (different for sky, trees, panel, terrain,...). What effects precisely are used depends on the predicate sections of the effects which select for rendering framework, quality level etc.

Now, I assume if you render the same scene for a different view, you can either leave all settings 'as is' - or you can re-run some logic. For instance, you could (for this view) switch the filters and IR view flags on, in which case the normal scene would be ALS visual and the camera scene would be ALS IR.

That is certainly possible with the FG binary, the question is more whether you actually re-apply the effect settings (with possible additions) or whether you just use whatever settings govern the main camera.

The second thing that's possible is if you bind the canvas to a 3d surface, you can run a shader of your liking over the surface. However, all that shader can work with is what's on the canvas texture - there's just pixel color available, we wouldn't have any distinction between sky and terrain for instance (for synthetic terrain). That works right now.

Now, you kind of sound like you imagine a third processing stage which somehow involves also effects and shaders, and I'm kind of scratching my head what that would be. But it's clearly not what I have in mind (or would consider useful at this point).

I think out of the box we can do cameras, mirrors, black/white and noisy camera images, light amplified camera images etc. just by running a dedicated effect over the surface onto which the cam is rendered (might require some support for popup windows to appear there as well) - but synthetic terrain and FLIR could not be done that way unless you've already taken care of my first item.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Canvas:View development

Postby Hooray » Sun Sep 03, 2017 7:26 am

Actually, all you say is entirely correct - however, what we have currently are only slaved views, slaved to the main view/window. There are no independent views at all.
To be sure we're on the same page, it would be great if you could post an effects snippet that is specific to a certain view - I would then post a screenshot showing that, so that you can see for yourself how the whole scheme works out.

Like we said previously, there is no support for custom effects/shaders at this point, simply because all the texture will receive is a standard view rendered to a texture.
Everything else will require additional work in the future.

As a matter of fact, Icecode GL came up with a more elaborate scheme that would expose additional osg data structures in the form of dedicated Canvas elements, to make these things possible - but at this point, I agree that we are really only working towards tailcam-like functionality. The other stuff will have to wait until that works, and is integrated/committed - in the meantime, we can work out a scheme to actually have our cake and eat it.

On the other hand, would it be of any help to tune up the AW/ALS settings at KSFO and display a few slave views in different Canvas dialogs to see what it is currently doing ?

PS: All the questions you are posing would become increasingly relevant in anything covering a CompositeViewer/PagedLOD approach with totally independent views - so if/when this should be discussed on the devel list, we will definitely need to work out a more elaborate scheme covering all the aspects you mentioned.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas:View development

Postby Thorsten » Sun Sep 03, 2017 7:32 am

To be sure we're on the same page, it would be great if you could post an effects snippet that is specific to a certain view -


I don't think we can do that right now (or at least I don't know how) - it would only exist dependent on your implementation of the cam canvas (if IcecodeGL was wondering to render cam view at lower quality level, that would be the same problem).

I believe the way FG assembles effects it would be conceptually easy to include (but I may be wrong, it's been a while since I looked at the C++ part of effects).
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Canvas:View development

Postby Hooray » Sun Sep 03, 2017 7:36 am

I may be wrong, but I thought I had seen screenshots running view-specific effects.
Anyway, it will be interesting to see if and how this evolves in the future - tail-cams are really just the beginning, supporting osg::Cameras for RTT/MRT purposes could have a great impact on any efforts requiring support for custom texture baking using cascades with different effects/shaders.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas:View development

Postby Thorsten » Sun Sep 03, 2017 7:39 am

I may be wrong, but I thought I had seen screenshots running view-specific effects.


You can have a Nasal logic 'if view switches to XX, change properties y and z to switch on the IR filter' which would do that.

But I believe that would switch both main and cam view to IR.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Canvas:View development

Postby Hooray » Sun Sep 03, 2017 7:45 am

ah I see, obviously that would not show up when just having a single full-screen view.
On the other hand though, I'd be kinda surprised if that's really the case, because these features were both developed by the same person, in pretty much the same timespan: Tim Moore. He is the one who came up with the effects framework, but also the one who came up with the CameraGroup stuff, which is designed to allow separate windows/views to be set up, which includes support for doing all sorts of offset views rendering to a texture according to README.multiscreen: https://sourceforge.net/p/flightgear/fg ... ultiscreen

I think it was Torsten who posted photos and screenshots of rather elaborate view setups at FSWeekend/LinuxTag (?)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas:View development

Postby icecode » Tue Sep 12, 2017 12:12 am

So I've been experimenting with effects/shaders support for Canvas and I got something kind of dirty and primitive but it works:

Image

It works similarly to how Rembrandt does its buffers. There is a new texture type called "canvas" that allows shaders to access any canvas texture via a texture unit, just like you'd access a normal texture from the hard drive. This removes (most) limitations of addPlacement, which can only substitute the "base" texture (unit 0). The posibilities are endless, it just needs some care and work. At the moment we could have shadow mapping outside Rembrandt just by creating a new view placed at the Sun and make it render the depth buffer to a canvas. Then every ALS shader could access this canvas and do the shadow comparison thingy. Planar reflections in the water should be kind of trivial as well, it'd just require the model-view matrix to be multiplied by a reflection matrix and the water shader could just paste the result over the water surface (with some more fancy calculations to make it pretty of course).
icecode
 
Posts: 708
Joined: Thu Aug 12, 2010 1:17 pm
Location: Spain
Version: next
OS: Fedora

Re: Canvas:View development

Postby Necolatis » Tue Sep 12, 2017 12:29 am

Nice!
"Airplane travel is nature's way of making you look like your passport photo."
— Al Gore
User avatar
Necolatis
 
Posts: 2232
Joined: Mon Oct 29, 2012 1:40 am
Location: EKOD
Callsign: Leto
IRC name: Neco
Version: 2020.3.19
OS: Windows 10

Re: Canvas:View development

Postby Thorsten » Tue Sep 12, 2017 6:28 pm

Now that sounds pretty interesting... :mrgreen:

Then every ALS shader could access this canvas and do the shadow comparison thingy.


Actually, every ALS shader could choose to access this canvas - so one can run the more detailed opacity map technique in cockpit and shadow mapping outside for instance. :mrgreen:
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Canvas:View development

Postby Hooray » Wed Sep 13, 2017 8:37 pm

I agree completey, this is looking relly good and very promising.
I hope that we can work out a way to share these experimental patches so that others contributors interested in related functionality can take a look and tinker with this scheme - for instance, psadro_gm's work on procedurally draped scenery/airports could benefit from this, but also anything involving photo-scenery or runtime texture baking in general.

In the mid-term it may make sense to add a dedicated Canvas placement for opening osg windows, as well as a geo-referenced placement to put a canvas texture onto the terrain mesh using lat/lon tuples:

http://wiki.flightgear.org/Photoscenery ... otoscenery
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Next

Return to Effects and shaders

Who is online

Users browsing this forum: No registered users and 1 guest