Board index FlightGear Development Canvas

Canvas + effects/shaders

Canvas is FlightGear's new fully scriptable 2D drawing system that will allow you to easily create new instruments, HUDs and even GUI dialogs and custom GUI widgets, without having to write C++ code and without having to rebuild FlightGear.

Canvas + effects/shaders

Postby Hooray » Fri Feb 21, 2014 3:28 pm

Soo ... as some of you may have noticed, I started extending the "canvas development" article and added stuff from various places to document what we've been talking about regarding potential future additions.

Next, I got a message here from someone asking what would be involved in adding shader support to FG, so I took everything that I could find and added it to the wiki, too.

So I ended up writing some pseudo code - mostly copy&paste, based on some OSG tutorials and existing FG stuff to explain what needs to be done.
And I was going to add it to the wiki so that others can develop it further.

However, the funny thing is: I just tried it with the OSG greyscale fragment shader they're using in the tutorial - and well, it actually works:
Image

Here's the same image with a fragment shader applied that turns every pixel red:
Image

Here's a more sophisticated example:
Image

It's less than 30 lines of code actually to run a fragment shader from $FG_ROOT/Shaders. I am currently applying the shader to all canvases via the canvas constructor.
The code took just a few minutes to put together - so it's really just a "prototype" currently, and would need to be modified to use our effects framework in SimGear obviously

So this would need to be extended:

If nobody else is interested in picking this up, I am going to generalize things a bit and use PropertyObject/PropertyBasedElement to generalize shader handling and push everything to a gitorious branch.

Otherwise, I'd hope that we could decide how to best proceed from here - supporting per-canvas shaders seems trivial and doable actually, integrating things with the effects framework is something I am probably not gonna do myself, because I am not familiar with it, and it would take me a while to get up to scratch with things (time being precious).

But I am sure that i4dnf, Zan or Stuart could help with the effects related side of things, while Tom could handle the Canvas integration ?

Also, I was hoping to do per-element rendering to sub-textures, so that we could support effects per canvas::element, rather than just per canvas

So, how do we want to proceed ?
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11354
Joined: Tue Mar 25, 2008 8:40 am

Re: Canvas + effects/shaders

Postby Hooray » Sat Feb 22, 2014 10:56 pm

Okay, I am slowly starting to understand how to use the Effects framework, looking at Stuart's commits is really helpful and instructive here :D

So, the next question is how to ideally proceed with the canvas side of things. Adding just support for per-canvas effects is simple, but not very flexible/scaleable.
The next straightforward option would be only supporting effects per Canvas::Image, so that people could reference another canvas via canvas:// in order to apply effects/shaders to it.
That would be fairly simple to support and not require very much code - but obviously at the cost of indirection, i.e. we would need some Nasal wrappers, because each "effect/shader" would inevitably be using two textures, the original canvas, and then the other canvas referencing it via CanvasImage


So, I was thinking of having a new element called "CanvasEffect" that implements Canvas::Group but also the CanvasImage interface, that way it would be pretty flexible, because each CanvasEffect would render to its own texture (set to DYNAMIC), so that effects/shaders can be easily applied - but also so that sub-elements could be added because it would also be a group, thoughts/better ideas ?
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11354
Joined: Tue Mar 25, 2008 8:40 am

Re: Canvas + effects/shaders

Postby TheTom » Tue Feb 25, 2014 11:41 am

I don't think another Element is needed. Applying shaders to CanvasElements should allow for all possible scenarios. A shader on a group would apply to all child elements (while rendering) and for post processing shaders one would have to apply the effect to a CanvasImage which is rendered inside another canvas.
TheTom
 
Posts: 321
Joined: Sun Oct 09, 2011 10:20 am

Re: Canvas + effects/shaders

Postby Hooray » Tue Feb 25, 2014 11:58 am

okay, so we would need to extend CanvasElement then to support effects - according to some of your postings, you played with that already a while ago using code from SGMaterialAnimation apparently, as in 2 years ago ? Is that something that we should reuse ?

Specifically, I've been in touch with some of our effects/shader guys, and we're wondering if we could directly register a Canvas/Effects texture as a texture available to effects, which is something that Thorsten and Tim talked about a while ago, being a limitation in effects that they would like to get rid of:

http://www.mail-archive.com/flightgear- ... 37873.html
Tim Moore wrote:It would be nice if the Effects framework had a way to load arbitrary textures and make them available to effects.
I don't know if there is a better way to create your texture offline than write C++ code in simgear.


Now, obviously, this all predates Canvas - but if we could not just support glsl/effects in Canvas, but also expose those canvas textures to the effects system, that would bring a lot of possibilities, quoting Thorsten:
Thorsten wrote:It'd sure be interesting to learn how to do a few things outside the shader. Cloud shadows would be an example - we know where all the clouds are in Nasal, so the math to compute a texture from that isn't very hard (and even fast) - so if we could generate that texture and pass it to the terrain shader, we'd have fuzzy cloud shadows outside of Rembrandt. Which'd be cool, because Rembrandt can't really do it, a shadow-drawing cloud pass would be way too expensive.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11354
Joined: Tue Mar 25, 2008 8:40 am

Re: Canvas + effects/shaders

Postby Johan G » Tue Feb 25, 2014 2:56 pm

Hooray wrote in Tue Feb 25, 2014 11:58 am:...if we could not just support glsl/effects in Canvas, but also expose those canvas textures to the effects system, that would bring a lot of possibilities, quoting Thorsten:
Thorsten wrote:It'd sure be interesting to learn how to do a few things outside the shader. Cloud shadows would be an example - we know where all the clouds are in Nasal, so the math to compute a texture from that isn't very hard (and even fast) - so if we could generate that texture and pass it to the terrain shader, we'd have fuzzy cloud shadows outside of Rembrandt. Which'd be cool, because Rembrandt can't really do it, a shadow-drawing cloud pass would be way too expensive.

I hope I am not the only one who see a potential for aircraft shadows outside Rembrandt using this technique. :wink:

The big drawback with the animated shadows used by some aircraft is that they really only work well on flat terrain. But if a shadow texture can be passed to the terrain shader in a similar way to how Thorsten describes using it for cloud shadows, possibly with the addition of scaling a shadow texture in x and y depending on pitch and roll, we could have at least half descent shadows with a lot less performance drop framerate-wise then if using Rembrandt.
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5529
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

Re: Canvas + effects/shaders

Postby Thorsten » Tue Feb 25, 2014 7:15 pm

Huh? Aircraft shadows?

If you want just the aircraft shadow, there's other techniques like shadow volume one could try. Textures are a sort of limited commodity, since the GPU has just 8 texture units, and for terrain we will many of them with the various overlay textures already. And the aircraft can't go on the cloud shadow texture, since the cloud would have a resolution of 10s of meters per pixel, that'd be very blurry and far too coarse for the aircraft.
Thorsten
 
Posts: 11106
Joined: Mon Nov 02, 2009 8:33 am


Return to Canvas

Who is online

Users browsing this forum: No registered users and 0 guests