Board index FlightGear Development Effects and shaders

PBR or Physically-based rendering

An exciting "new" option in FlightGear, that includes reflections, lightmaps, the particle system etc.. A lot is yet to be discovered/implemented!

PBR or Physically-based rendering

Postby Icecode GL » Sun Jul 09, 2017 7:48 am

For those not familiar with the term this post should help. Basically, PBR is a recent trend in the computer graphics industry that aims to better represent the behavior of light and surfaces through a more less unified set of rules. Both raytracers and real time applications benefit from this workflow (see the recent Principled BSDF in Blender Cycles or Unreal Engine 4). This is kind of a big deal since previously every application had its own way to represent materials and how they interact with light. One of the premises of PBR is that every material description should be independent of the lighting conditions. This consistency simplifies work for both the artist and the coder, allowing a material to be rendered in a path tracer and a game engine using the same assets.

So what can FlightGear benefit from? Currently the ALS framework does a great job at doing PBR on its own way (in fact it was ahead of its time). But as I said before, the main advantage of PBR is the consistency across applications. Dirt or metal scratches don't have to be procedurally generated like ALS does currently and can be drawn instead by the artist in Blender and previewed directly there. Metal and dielectric surfaces can be combined in the same material via a map, same with glossiness, allowing for shiny metal aircraft fuselages with diffuse details.

The amount of documentation is huge due to the recent popularity of the topic, so implementing the PBR shader itself is not too hard. A quick implementation of Unreal Engine 4's presentation for SIGGRAPH 2013 already gives good results:
Image
(Code here).

So what do you think? Is it worth merging PBR and ALS in FG?


More information:
- https://disney-animation.s3.amazonaws.com/library/s2012_pbs_disney_brdf_notes_v2.pdf
- http://filmicworlds.com/blog/everything-is-shiny/
- https://docs.unrealengine.com/latest/INT/Engine/Rendering/Materials/PhysicallyBased/
Icecode GL
 
Posts: 514
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby erik » Sun Jul 09, 2017 8:51 am

I think that due to the inherent differences between blender and FlightGear-ALS the best approach would be to extend the blender exporter to generate the desired textures (and possibly the effect files). If this can be done using a new shader: fine but I for one still use AC3D and would like to see the current shaders being supported.

Erik
erik
 
Posts: 1520
Joined: Thu Nov 01, 2007 1:41 pm

Re: PBR or Physically-based rendering

Postby Icecode GL » Sun Jul 09, 2017 8:58 am

I for one still use AC3D and would like to see the current shaders being supported.


AC3D is FG's native (and probably only supported) mesh format, so that doesn't have to change. Other shaders won't be affected, just the model ones (model ultra ALS mainly).

There are also many other 3rd party tools to generate these textures, both in Blender and in other applications such as Substance Painter.
Icecode GL
 
Posts: 514
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby erik » Sun Jul 09, 2017 9:07 am

I see you point, and while I probably will never use it like the idea. But what I was trying to say is that there are more ways to get the same result. One is to directly implement PBR features in FlightGear, the other is to export PBR related information to be used by the current PBR-like implementation ALS.

Erik
erik
 
Posts: 1520
Joined: Thu Nov 01, 2007 1:41 pm

Re: PBR or Physically-based rendering

Postby Thorsten » Sun Jul 09, 2017 9:37 am

The presence of thousands of existing 3d models forces us to not discontinue the present rendering strategy (unless someone is keen on converting all of them manually...). So this has to be opt-in.

One important thing I've learned over the years is that for such things as the model shader, it's the aircraft maintainers and scenery modelers who decide whether something works. The most brilliant idea is not going to take off if it doesn't resonate with how people work.

Look at the procedural rock for instance - I think it's a fairly ingenious bit of technology I'm fairly proud of how many different types of rock it can represent, yet it's not really widely used, people don't seem comfortable with it.

So personally I like the concept a lot (I guess it's an open secret that I don't have a high opinion of Blinn-Phong schemes) - but the question is - will aircraft maintainers adopt a different workflow?

Note that I don't think you can make anything look really different from what we can do (more or less) right now - just the language to describe what is what is different (and much more to my taste).

Dirt or metal scratches don't have to be procedurally generated like ALS does currently and can be drawn instead by the artist in Blender and previewed directly there.


We do them procedurally because we want the ability to create them runtime, not because we can't do them design-time.

So what do you think? Is it worth merging PBR and ALS in FG?


I would propose to offer an effect model-pbr.eff as an alternative to model-combined.eff. This could be used by anyone who wants to work with this and gather experience. My understanding is there is no easy fallback to a fixed pipeline (?), which is going to be tricky from a POV of offering backward compatibility.
Thorsten
 
Posts: 10740
Joined: Mon Nov 02, 2009 8:33 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Sun Jul 09, 2017 11:22 am

One important thing I've learned over the years is that for such things as the model shader, it's the aircraft maintainers and scenery modelers who decide whether something works. The most brilliant idea is not going to take off if it doesn't resonate with how people work.


That's something I forgot to include in my OP but I completely agree with. Open source projects generally run that way so it's hard to generalize stuff and do big changes to certain features. Keeping this in mind, you also have to consider that this PBR thing is quite mature, backed up by giants in the industry. My bet (and most people's bet) is that PBR will be an standard for at least a few years. FG will have to adapt with time, same way it has until now. The sooner we implement the technology, the sooner people might start using it.

will aircraft maintainers adopt a different workflow?


Some won't at first if they don't like to learn/adopt new stuff. But the truth is that this scheme is the product of big industries with thousands of artists. One of the premises of PBR is its simplicity for the artist, so it should be easier to understand than any other schemes.

Note that I don't think you can make anything look really different from what we can do (more or less) right now


I have to disagree here, but anyway the most important thing I value about PBR is its unification, not on being more or less realistic.

I would propose to offer an effect model-pbr.eff as an alternative to model-combined.eff. This could be used by anyone who wants to work with this and gather experience.


Yeah, that would be the cleanest and less obstructive option.
Icecode GL
 
Posts: 514
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Hooray » Sun Jul 09, 2017 2:45 pm

My bet (and most people's bet) is that PBR will be an standard for at least a few years. FG will have to adapt with time, same way it has until now. The sooner we implement the technology, the sooner people might start using it.


To be fair, and without wanting to argue with anybody around here, the same has been said about other developments like Rembrandt/deferred rendering or HLA (High Level Architecture), or even osgEarth at some point. More recently, the whole idea of supporting Python in FlightGear was quite popular around here.

Equally, a number of efforts to use the FlightGear Canvas system for implementing a simple and portable GUI in FlightGear were declared "dead" by some folks, simply because such efforts conflicted with the idea to use Qt for much of the future UI in FlightGear.

3+5 years later, it is obviously up to you to make up your mind if/how these developments (or even just debates) have affected FlightGear as a whole, especially in comparison to its evolution over time in the past 2 decades.

Then again, I found Thorsten's response much more positive and supportive than I was actually expecting, given his focus on procedurally-created textures. So this is definitely thought-provoking and interesting, but I would suggest to tread very carefully here, there are far too many "pie-in-the-sky" efforts that have been used to shut down other efforts, even though those would have been able to come up with suitable temporary compromises. One of the most recent examples is Richard's work on updating the MP system, which is long overdue, but not because nobody was interested in doing so, but because tons of experienced contributors were "waiting" for HLA to materialize, because it'd be an "industry standard" and "the way to go".

Again, I am certainly not trying to argue with anyone - but please keep in mind how many such examples exist around here, where this community literally allowed "the perfect to be the enemy of the good (enough, for now)".

Besides, Thorsten's scripted Advanced Weather system is one of the most obvious and most successful examples of a system that was critically discussed (dissected) by many around here, yet it's also come to cause FlightGear's environment modelling to be mentioned in many reviews to critical acclaim, i.e. people were giving a damn if the underlying code was C++ or something else (Nasal/GLSL), or if its structure was in any way "standard" - despite tons of criticism voiced on the devel list by people "in the know".

Anyway, having, keeping and providing/extending several options seems to have always served FlightGear as a project very well, which is to say that optionally supporting PBR may be a good thing, but let's not expect to see everybody jump on the bandwagon, lest we face another HLA, Rembrandt or Qt5 dilemma.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Sun Jul 09, 2017 3:08 pm

To be fair, and without wanting to argue with anybody around here, the same has been said about other developments like Rembrandt/deferred rendering or HLA (High Level Architecture), or even osgEarth at some point. More recently, the whole idea of supporting Python in FlightGear was quite popular around here.


And Fred wasn't wrong, deferred rendering was the way to go. In fact the standard lately is a hybrid approach combining both forward and deferred renderers using a G-buffer and several framebuffers. Deferred rendering didn't take off in FG, but that doesn't mean deferred rendering wasn't the way to go. Without access to any render to texture techniques in FG I (personally) feel very limited when it comes to implementing certain features, often requiring a workaround to fit forward rendering and most of the time not even a workaround is possible. Maybe it could have been implemented better, maybe it could have been better supported by the community or maybe it could have taken off if it didn't offer 20 fps on a gaming rig. My point is that any feature's success is really unpredictable and more dependent on how the community welcomes it.

PBR is infinitely simpler to implement than deferred rendering, that's why in my opinion it's worth to try out and see how it goes.
Icecode GL
 
Posts: 514
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Hooray » Sun Jul 09, 2017 3:25 pm

Icecode GL wrote in Sun Jul 09, 2017 3:08 pm:Without access to any render to texture techniques in FG I (personally) feel very limited when it comes to implementing certain features, often requiring a workaround to fit forward rendering and most of the time not even a workaround is possible.


I don't disagree with that, and Fred also kinda acknowledged that at the time - and the original discussion on the devel list (it all pre-dating Canvas), involved generalizing Tim's effects framework (which does include support for rendering a camera to a buffer) and using Zan's "newcameras" approach to "have our cake and eat it", which is to say that Zan came up with an XML configurable rendering pipeline where even individual cameras and FBO/RTT contexts could be set up - the videos and images he posted on the forum were showing scenery views rendered to the surface of a cube and other fancy stuff.

Most of that code hasn't been touched in years, so someone familiar with SG/FG, and C++/OSG should still be able to make heads and tails of Zan's approach.
Zan's topic branches are also still available at: https://gitorious.org/fg/zans-flightgea ... newcameras

Some of the surrounding context from the devel list (mainly comments from Fred, Tim, Mathias and Lauri) can be found on the wiki, too:

http://wiki.flightgear.org/Supporting_m ... _.28Zan.29


As you can see, Matthias, Fred and Lauri were originally hoping to unify and integrate their approaches using Tim's groundwork - but nothing actually materialized by now.
Since the addition of the Canvas system, that would actually be the most logical candidate to serve as the integration mechanism to allow rendering stages to be specified/configured and set up without having to touch any C++: http://wiki.flightgear.org/Canvas_Devel ... ng_Cameras

You will also see that long-term contributors like James and AndersG repeatedly mentioned that being able to do custom RTT passes would be useful for a number of purposes: http://wiki.flightgear.org/Canvas_Devel ... 2F_Shaders

It's not that this is generally too difficult actually - in fact, I am not much of a graphics coders myself, but I actually managed to hook up the effects/shader system to the Canvas system several years ago, and things were looking promising actually - once someone takes that approach and integrates it with the groundwork done by Fred (Rembrandt stages) and Zan (XML/property configurable), we are all set to do pretty much arbitrary passes and rendering stages, all rendering to different FBO/RTT contexts represented in the form of a Canvas texture:

Image

Doing that would also greatly facilitate any efforts involving PagedLOD refactoring and/or adopting the CompositeViewer machinery provided by OSG: http://wiki.flightgear.org/CompositeViewer_Support

Sorry, I realize that some may consider this a form of hijacking the thread - but it just seemed to me that you were unaware of these efforts and existing patches/branches.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Sun Jul 09, 2017 3:44 pm

Sorry, I realize that some may consider this a form of hijacking the thread - but it just seemed to me that you were unaware of these efforts and existing patches/branches.


I'm actually quite aware, I've been closely following FG rendering even if I wasn't actively involved with it. In fact, as I said in another thread (the shadows one I believe), I'd be more than happy to finally unify the mess that is rendering in FG and allow framebuffers to be created as Canvases etc. etc. The problem is that I don't have the knowledge or time to do so. It's also not rewarding, specially if we consider ALS's maturity and how most shaders would have to be rewritten to fit another framework.
Icecode GL
 
Posts: 514
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Hooray » Sun Jul 09, 2017 4:02 pm

Yeah, that's also why I originally reached out to Thorsten when my experiments began to involve writing actual shaders, because I could really only handle the C++/OSG part, and wasn't that interested in learning GLSL.

I don't think the pain/gain ratio is that bad actually - it's not that existing features must inevitably be ported to provide new infrastructure/hooks.
Thorsten and others mentioned on various ocassions how procedural texturing would be empowered by having support for adding passes/stages dynamically, i.e. without having to touch/rebuild any C++ code.

So there is use-cases beyond Rembrandt/ALS, even though it is admittedly obvious that -at least for the time being- anything involving ALS/Thorsten is most likely to become the "killer application" :wink:

And yeah, I am aware of the other thread, and I also remember our FGRadar experiment, or I would not have taken the time to post these pointers.

Speaking in general, the FlightGear project has a tendency to attract incredibly clever folks, but the project is not particularly good at recognizing the fact that many of these people are only able to join the project (or contribute to it) on a temporary basis, so that we end up with excellent ideas and concepts in various places that are hardly/poorly integrated over time, which is why things like effects/shaders, Rembrandt, Canvas etc know nothing about eachother in terms of mutually shared APIs and building blocks.

Yet, all a Canvas really is, is a RTT context (FBO), and to become usable as such for Rembrandt or effects/shaders, all that is needed is to look at the functionality not yet exposed/integrated - e.g. things like specifying/changing the color depth of the texture, the texture format etc - this is stuff that isn't currently needed obviously, but it'd be trivial to expose such things at the mere cost of binding a handful of additional properties per Canvas texture to a corresponding listener/method that handles the update/creation event.

Given the constraints of the project, many people are also not particularly good at recognizing that they're actually working towards features with overlapping functionality/requirements - and that applies even in seemingly conflicting pairings like Rembrandt/ALS - as in procedural texturing, or osgEarth (rendering a moving map to a texture) and so on.

Even something as "conceptually different" as osgEarth support would have benefited from using a Canvas FBO as the integration mechanism.

It's literally taken months for the project to come up with a draft for a so called "project roadmap", and the items on it are hardly in line with reality - it seems much more like a wishlist of some long-term contributors, whereas the facts (commits) are proving other things to be much more relevant. Otherwise, it is hardly explicable how things like the Qt5/HLA effort made it onto that list (despite their pie-in-the-sky reputation), whereas the whole reset/re-init effort wasn't even mentioned.

Besides, as far as I recall. that "project roadmap" got never actually published anywhere, except for the wiki, despite many requests and reminders (?) - thus, when it comes to long-term strategies (think adopting deferred rendering/PBR), it seems generally not a good idea to proceed that way. Wish lists simply don't work, as long as it isn't also made explicitly clear that the implementation/success of these efforts depend solely on the involvement of a single person.


Looking at how things change over time, these wish lists don't work at all and it would have made more sense to state architectural goals rather than personal milestones, and confine those goals to a certain time frame/release cycle, or people will look at these "roadmaps" and wonder what is really going on here.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Sun Jul 09, 2017 5:46 pm

I think you are oversimplifying things. Just have a look at renderer.cxx or CameraGroup.cxx. Maybe after a few hours I can begin to understand about 10% of what is written there, only to forget everything the next day I have the time to sit down and continue. You might remember from FGRadar how I documented things even before any code was there, just to know what I was doing/going to do. Perhaps that's too much, but the truth is that the rendering code is messed and rushed, which isn't good considering that the entire simulator depends on it. Zan had it much easier in 2010 when the rendering stuff didn't have Rembrandt all over the place and everything had much more structure and design considerations. Even if the best OSG/OpenGL/GLSL/C++ programmer comes today to fix rendering in FG he will run away as soon as he sees what awaits him, or at least hesitate a lot. And even if he comes up with a solution, he can wait another year before it gets submitted upstream.

Rewriting is the best option, but there are many other subsystems that depend on the current code. Patching it further will just increase its complexity and won't fix the mess. I dunno, I'm not an expert AT ALL, but I don't think the solution to this whole thing is to just sit a few hours a day and write code. And I'd dare to say that there is no solution, at least not a near perfect one.
Icecode GL
 
Posts: 514
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Hooray » Sun Jul 09, 2017 6:21 pm

I don't disagree that there is exactly the kind of complexity you mentioned (especially CameraGroup stuff and Rembrandt hacks all over the place) - however, like I said - that was all prior to the Canvas days. These days, Tim, Mathias, Fred, Zan and even poweroftwo (osgEarth) would definitely approach their projects differently, by recognizing that they have certain overlapping/shared requirements, and that these could be tackled in a holistic fashion without touching much of the legacy code in question - simply because all of these features are primarily about rendering to a configurable FBO/RTT context.

As a matter of fact, the distorted screen mode is already rendering to a framebuffer - it just isn't used/exposed outside C++ space obviously.

It's very much akin to the introduction of the property tree by David M. - back then it was a novel concept, and people were concerned about using it - e.g. due to performance considerations (they were even using raw pointers to speed up variable access then) - these days, that's a no-brainer obviously. But it's time to recognize that the Canvas system is doing to FlightGear rendering what the property tree did to state/variable sharing - you can now implement arbitrary FBO/RTT rendering without much work, and if something is missing (or too slow), think rendering 3D models or rendering GeoTIFF/DEM heightmaps, it can be added as a new model. Equally, if something is to be shown outside the main fgfs window, it's just a new Canvas placement that needs to be added.

All the concepts are well understood, and the infrastructure is readily there to be picked up - just like David's original work on the property tree subsystem contained support for listeners that weren't really picked up for the better part of a decade, despite tied properties generally considered to be bad thing - which was explicitly communicated by him, too.

These days, you can have an empty OSG/OpenGL-enabled fgfs window up and running without much of an effort, you can disable scenery/terrain, and even the skydome -at which point you could theoretically also implement support for alternate scenery/terrain engines, or even completely alternate renderer schemes akin to Rembrandt.

So this isn't over-simplifying things at all - former core developers like David M. and Erik suggested to do this over a decade ago, it just never materialized - it's only since the introduction and adoption of the Canvas system that this has actually become feasible.

Even Curt himself suggested on various ocassions that the scenery/terrain engine and the renderer infrastructure be treated as a "plugin" akin to the FDMShell machinery that encapsulates the differences between YASim/JSBSim: viewtopic.php?f=5&t=25081&p=292221&#p292221


Someone embarking on a journey to implement a feature like osgEarth, the effects system, Rembrandt without using the Canvas system as the underlying foundation would waste tons of time and sacrifice tons of functionality - the Canvas system literally has become the property tree for rendering, i.e. there is no conceptual difficulty to tie together all rendering functionality in FlightGear (even conflicting ones like Rembrandt and ALS) using just a handful of concepts (a Canvas, a group, a Canvas element, placements and events)


Thus, if we want to have our cake and eat it, it's time to recognize that there is so much groundwork that has been done by dozens of contributors over the course of the last 2 decades, that we're literally standing on the shoulders of giants - which is to say, there is really no technical hurdle when it comes to porting/re-integrating existing legacy functionality like the GUI, HUD, 2D panels or Rembrandt/osgEarth by going through the Canvas system to serve as the common subsystem providing FBO/RTT support.

And at that point, scenery/TG folks like psadro_gm and others would collaborate despite possibly working on different projects - just like the SGPropertyNode implementation has always been actively maintained by the FlightGear project, simply because it's the foundation, or rather "nervous system", of the whole thing.

Honestly, looking at all the needs for dynamically using RTT, there should hardly be any need for hard-coded FBO use in FlightGear, it could certainly be provided by the Canvas system, which would be extended as needed over time - just like the property node system has been extended, too.

It would no longer matter if you're interested in implementing fancy procedural rendering schemes, Rembrandt-like schemes, a new terrain engine or a completely new renderer - the barrier to entry would be massively lowered, and the code base unified (and hardened !) in the whole process: http://wiki.flightgear.org/Supporting_m ... _renderers

So there really is no oversimplification here - it's already happening as we're speaking, exactly in line with the impact that the property tree had on the whole FlightGear code base - the only question is how long it is taking until people will recognize that the Canvas system is no necessarily confined to just 2D rendering only, but that it can just well serve as the FBO/RTT provider for arbitrary purposes, and that it can also deal with non-2D use-cases (scenery, 3D models, GeoTIFF) at the mere cost of adding a new Canvas element sub-class - and that its notion of virtual "placements" nicely maps onto the few remaining use-cases not yet covered, as in window management (OS/OSG windows), and streaming to/from a Canvas (think for screen capturing).

It's not exactly rocket science to see what's already going on - the property tree will obviously always remain the nervous system of FlightGear, and given that the Canvas system is merely an abstraction mechanism on top of the property tree, it will continue re-enforce its significance to serve as the primary mechanism to tie rendering components together that primarily need to deal with different FBO/RTTs.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: PBR or Physically-based rendering

Postby Thorsten » Sun Jul 09, 2017 6:30 pm

I think a PBR model effect is a very reasonable goal to reach for - an overhaul of the whole rendering framework however is not unless you come with serious manpower or the one 'must have' application.
Thorsten
 
Posts: 10740
Joined: Mon Nov 02, 2009 8:33 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Sun Jul 09, 2017 6:37 pm

Unfortunately I have to agree with Thorsten. We would all love to have an unified rendering framework, but being realistic that isn't going to happen any time soon. Integrating the PBR shader on a deferred renderer is trivial, so there is that. If the day finally comes, PBR efforts won't be in vain.
Icecode GL
 
Posts: 514
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Next

Return to Effects and shaders

Who is online

Users browsing this forum: No registered users and 2 guests