Board index FlightGear Development Effects and shaders

PBR or Physically-based rendering

An exciting "new" option in FlightGear, that includes reflections, lightmaps, the particle system etc.. A lot is yet to be discovered/implemented!

Re: PBR or Physically-based rendering

Postby Hooray » Sun Jul 09, 2017 6:38 pm

Thorsten wrote in Sun Jul 09, 2017 6:30 pm:...unless you come with serious manpower or the one 'must have' application.


point well taken - however to be fair, neither seems to have worked out too well in the context of the OSG port, which never got really "completed" - despite very valid "must have" arguments (think effects/shaders) and quite a bit of manpower along the way.

It still is a thought-provoking discussion, and kinda reminiscent of how we're having debates on requiring a scriptable "2D API" over the course of several years (to no avail, despite core developers sharing the view) - and how much of an effort that would be, how much manpower that would require, how slow scriptable 2D drawing would be- and ultimately it turned out that the whole thing took shape through the forum and the wiki only, primarily inspired by this - and according to the history, it took "only" 2 years "from brain fart to prototype" 8) (compare that to the OSG effort, Rembrandt, Qt UI etc)

Likewise, integrating the osgEarth scenery engine into FG was considered a multi-year mammooth task by many around here (and especially on the devel list), until poweroftwo proved all of us wrong - implementing the whole thing on his own, with much less of an impact on the code base than even the core developers expected (see James' and Stuart's comments in the archives).

And Zan was basically half-way there anyway, it's just kinda unfortunate that Rembrandt took place simultaneously, and that the Canvas system materialized so much later.
Anyway, all of these 3 efforts were handled by a single individual only :shock:
(Not unlike the OSG port which got initiated and largely implemented by Mathias only)

Getting back on topic, I don't think that Rembrandt is being actively maintained these days, I guess your best bet is trying to reach out to i4dnf and coordinate any PBR/shader related efforts with him.

PS: I will report the topic to have it split
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11100
Joined: Tue Mar 25, 2008 8:40 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Sun Jul 09, 2017 7:45 pm

Getting back into topic, implementing the shader in the OP in FG:
Image

The next step would be to get rid of that fixed color light and fully integrate the ALS lighting calculations. The problem is that the reflectance equation expects the light source intensity (which is usually greater than 1) while the ALS lighting outputs values between 0 and 1. I'm not too sure about how to proceed here.
Icecode GL
 
Posts: 432
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Thorsten » Mon Jul 10, 2017 5:05 am

The problem is that the reflectance equation expects the light source intensity (which is usually greater than 1) while the ALS lighting outputs values between 0 and 1. I'm not too sure about how to proceed here.


ALS lighting computation outputs 0-1 per channel for diffuse illumination, so the intensity goes to ~1.7. That's supposed to be in perception space.

The relationship between rendered intensity I_R and physical intensity I_P in Lux assumed by ALS is something like

I_R = 0.14 * log(I_P)

http://www.science-and-fiction.org/rend ... sl_12.html

(I don't vouch for the constant factor right now...) - you can use that equation to get physical parameters of the light within the perception model used.
Thorsten
 
Posts: 9539
Joined: Mon Nov 02, 2009 8:33 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Tue Jul 11, 2017 12:37 pm

My description was too vague, I'm sorry. Let's see if I can make sense of this whole thing...

The rendering equation expects the spectral radiance of every wavelength of the incoming light, but since we are working in a trichromatic environment the integral can be approximated as a sum of the radiances of pure red, green and blue. Directional light sources don't have an attenuation factor, so we can say the radiance is equal to the radiant intensity. And since the radiant intensity is constant regardless of the angle we look at it, we can say radiance is finally equal to the radiant flux of the light source.

Radiant flux is measured in Watts in the SI, so plugging in a vec3 consisting of the radiant flux of red, green and blue into the rendering equation should output the reflected light power, also in Watts. Assuming everything I said is correct (which I doubt), how do you translate the 0-1 value per channel outputted by ALS to the radiant flux per wavelength? I don't think you can just use the 0-1 value, so I have no idea what to do.
Icecode GL
 
Posts: 432
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Thorsten » Tue Jul 11, 2017 12:53 pm

Well, since (1,1,1) is solar light at noon, we know it's equal to the solar constant, i.e. 1.361 kilowatts per square meter. At the same time, you know that the physical intensity I_P is supposed to be const * exp(I_R) with I_R the rendered (rgb) values.

So you can equate

const. * exp(I_R = 1) = 1361 W/m^2 / 3

and go from there by fixing const. Actually, dividing the solar constant by three is simplistic, you can weigh it with wavelength and consider the Planck distribution of sunlight, but I seriously doubt you need anywhere near that accuracy.

And once const. is fixed, you can use the equation for any other value of I_R.
Thorsten
 
Posts: 9539
Joined: Mon Nov 02, 2009 8:33 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Tue Jul 11, 2017 4:19 pm

Well, since (1,1,1) is solar light at noon, we know it's equal to the solar constant


I was overthinking things, that makes perfect sense. Thanks! Now everything works as intended and is physically accurate (I hope).
Icecode GL
 
Posts: 432
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Necolatis » Sat Jul 15, 2017 7:39 am

Just want to say, that I am very excited with the possibility of PBR coming to FG. I looked into it a while ago. Seems much easier to get realistic looking surfaces with than what we have now. And it seems many websites have tables of surfaces for what values goes into albedo, roughness, metallic etc. to make the modelers life easier. I really hope we at some point get this into Flightgear. :)
"Airplane travel is nature's way of making you look like your passport photo."
— Al Gore


Hangar: https://sites.google.com/site/fghangar/
User avatar
Necolatis
 
Posts: 1833
Joined: Mon Oct 29, 2012 12:40 am
Location: EKOD
Callsign: Leto
IRC name: Neco
Version: 2018.1.1
OS: Windows 10 Pro

Re: PBR or Physically-based rendering

Postby Thorsten » Sat Jul 15, 2017 9:06 am

Nice - so we have the first tester. :mrgreen:

@Icecode GL, let me know if you need any help with the ALS specifics. It *should* be fairly easy with the material you have, but just in case.
Thorsten
 
Posts: 9539
Joined: Mon Nov 02, 2009 8:33 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Sat Jul 15, 2017 10:32 am

The main issue is "translating" between PBR in games and PBR in ALS. Games approach the situation in a much more artistic way. For example they usually apply arbitrary values greater than 1 to llght sources so artists can get something that looks good. Then they use that HDR framebuffer and apply yet another artistic filter via tonemapping to get a LDR image to be displayed on screen. ALS has an entirely different philosophy, but I think I'm fine. Those files from the shadows stuff really help too.
Icecode GL
 
Posts: 432
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Icecode GL » Sat Jul 15, 2017 1:47 pm

So an idea has been roaming my mind for a while. The entire ALS implementation revolves around Blinn-Phong: light intensities are calculated for the ambient, diffuse and specular components and many "hacks" are required to get physically accurate indirect lighting. A few months ago I read about IBL (Image Based Lighting), which basically tries to emulate indirect lighting by considering every pixel in a cube map to be a light emitter. It's a very crude approximation to global illumination but the results are surprisingly good with an unexpected decent performance.

It's obvious that FG doesn't support anything related to RTT (yet), and even if it did, rendering a cube map every frame is not viable. Most (if not all) of the lighting comes from the sky. Instead of sampling from a cube map, we could sample from the skydome shader itself. I have my doubts about the performance of that approach, but then again the current ALS model lighting calculations aren't too cheap either. The advantages are really interesting and would definitely be more physically accurate by default, without many tweaks.
Icecode GL
 
Posts: 432
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Thorsten » Sat Jul 15, 2017 1:56 pm

We're (sort of) doing that already, sky-light illuminates your models and there's an irradiance distribution for ambient light computed at high quality level.

I've played with it some and decided there's no point in making it yet more accurate since the changes it does are rather subtle already and the current implementation is very fast.

I think we have one advantage over game engines - we have very good control over lighting in 99% of cases, we usually know the plane we're trying to render is outside and the instrumentation is inside the cockpit, and so we never need to consider the possibility much that we move from a dimly lit space into a narrow lane out into a brightly lit plaza.

Whereas for the two situations we do have, we can know the lighting pretty well using analytical formulae.

Whether you call that still Blinn-Phong is in the eye of the beholder - I really don't.
Thorsten
 
Posts: 9539
Joined: Mon Nov 02, 2009 8:33 am

Re: PBR or Physically-based rendering

Postby Thorsten » Sat Jul 15, 2017 2:42 pm

Afterthought:

but then again the current ALS model lighting calculations aren't too cheap either. The advantages are really interesting and would definitely be more physically accurate by default, without many tweaks.


I actually believe the illumination is fairly good in physical accuracy, and I don't think of what the shaders do as 'tweaks' - they're case by case custom-computed approximations to real light scattering optimized for our particular needs. And while the computations aren't too cheap, they're as expensive as absolutely needed.

You have to consider what we're trying to do here - which is trying to render models our contributors come up with such that users with decent graphics cards will get acceptable framerate and a realistic outcome.

As for the first - it's not that I wouldn't know how to code a physically very accurate shader based on double-differential photon scattering cross sections - I do - it's just that I'm probably the only one who readily knows what kind of input data this shader would require model-side. Thus it wouldn't help us much because contributors could not use the technology for lack of physics background.

The salient observation is that a PBR implementation has never been requested as a feature from contributors - plenty of other things have.

In actual reality, we're getting all sorts of models with each contributor trying to implement his own lighting philosophy - we even have people who try to 'correct' the ALS lighting model by using emissive animations dependent on daytime. Which is to say - we have very limited influence on the input we get, unlike in a game development team, we can't send models back to the art department, they have to work 'as is'.

And as for the second goal, it's not that I wouldn't know how to make, say, the ambient lighting more realistic - but the lighting computations are intentionally 'realistic enough' rather than 'as realistic as I can make them'.. I'm not getting enthusiastic about the latest and greatest in rendering technology that's possible, I take a good look at an FG scene and a real photograph in comparison and ask 'What is the major discrepancy we need to address to make them more similar, and what is the framerate-friendliest way to achieve that?'

You can find rather frequent comments that ALS 'surprisingly' out-performs the Classic renderer - I smile every time I see them, because that's a direct result of a philosophy of simply stopping when things are 'good enough' and not trying to go for 'realistic, no matter the performance'. I'd like to keep it that way. Gives us some leeway to implement the next framerate-gobbling feature we really want if GPUs get faster...
Thorsten
 
Posts: 9539
Joined: Mon Nov 02, 2009 8:33 am

Re: PBR or Physically-based rendering

Postby Icecode GL » Sat Jul 15, 2017 3:31 pm

I'm not doubting your physics background, or in other words, I know you could do very physically accurate shaders with unlimited resources. The thing is that we don't have unlimited resources. The only major industry that spends millions trying to get physically accurate results in real time is the game industry, so it makes sense to borrow some things from there. Again, I'm not doubting that you are doing a better job than those actually getting paid to move the industry forward. Although you have to consider that these techniques keep evolving with time and they are the result of the works of several research teams, and consequently they are proven to work and give good results with decent performance. Apart from that, these techniques have really good documentation that anyone familiar with the industry can understand. The truth is that if you ever decide to take a break from FG, ALS will become unmaintained and no one will be able to improve it or update it (just look at $FG_SOURCE/Viewer or the Effects framework in general). Physics don't change much in 10 years, computer graphics do.

To implement a PBR workflow in FG, you need to comply with several standards. ALS lighting calculations could be adapted to work (in fact I already did that), but they weren't meant to work. Image Based Rendering perfectly complies with the Cook-Torrance BRDF, which is the standard these days. It gives good results, is performance friendly and is much simpler than the whole ALS lighting model. Apart from that, everything is much more readable and extensible. No duplicated code or tens of uniforms, just the skydome shader. As you said, we get many different kinds of models and we need to be able to light them all accurately. Well, the entire point of PBR and IBL is to do that: get realistic results on every model under any environment.

In the end it's all a matter of personal preference, so I don't expect you to agree. You are probably right and any improvements are redundant, I'm just tossing ideas around.
Icecode GL
 
Posts: 432
Joined: Thu Aug 12, 2010 12:17 pm
Location: Spain
Callsign: icecode
Version: GIT
OS: Arch Linux

Re: PBR or Physically-based rendering

Postby Thorsten » Sat Jul 15, 2017 4:53 pm

In the end it's all a matter of personal preference


Let's say I'm disputing particular points here, and I'm trying to explain why.

One of them is whether certain things should be considered hacks, or whether adhering to standards is a meaningful operation.

Point being, one can code general-purpose standard solutions or special-purpose solutions tailored to the problem at hand. Simple example - a water wave shader can reasonably assume that the surface normal always points up (if it doesn't in reality you don't get waves). Now, that's not true from orbit where Earth curvature matters, and not true for other terrain types - so knowing you're rendering water from an aircraft altitude, you can simplify your particular problem, otherwise you solve the more general more complicated problem.

I wouldn't call this a hack - in my book it's a clever optimization, making use of what you know about the problem (we teach students to organize calculations that way!).

So there's a choice - adhere to standards, code standard general purpose solutions - or tailor a solution to the problem. The second tends to be messy, but (at times much) faster.

What choice you make is to some degree a personal preference - but one should spell out the choice clearly I think.

It gives good results, is performance friendly and is much simpler than the whole ALS lighting model.


The ALS lighting model pays performance for deriving two pixel-position dependent colors - sunlight and skylight, and you would pay the same performance in any rendering scheme that delivers what ALS does in terms of lighting the scene. You can't compute low sun colors in a landscape much cheaper, that works only if you give up position-differential light.

Although you have to consider that these techniques keep evolving with time and they are the result of the works of several research teams, and consequently they are proven to work and give good results with decent performance.


I have never developed games, but I would think the situation is somewhat different.

I imagine as a game developer I would not have the need to take care of 15 year old legacy content and make sure it runs, there wouldn't be a need for a sane migration path all the time, I could simply bump up hardware requirements rather than making sure 10 year old computers are not left behind, I could generally work within a streamlined effort rather than the various optional developments FG does and I had control over the input by the art department in terms of hard requirements.

Take a short look at how many planes actually implemented even the simple Rembrandt requirement of registering transparent surfaces and how long it took to get to that point and you get a feeling for what I mean.

If I were in the situation a modern game developer is, I would do several things differently - but I am not, and there's no point pretending I am. I try to work with the cards I've been dealt, not to change the game I'm playing.

Which is why I think the argument of industry standards is misleading - we have no industry-standard art department to go along with the effort, nor any way to validate or enforce adherence to standards.

The truth is that if you ever decide to take a break from FG, ALS will become unmaintained and no one will be able to improve it or update it


The truth is that it's really well documented in painstaking detail what it does and that there are people around who have been submitting patches or added simple things. Sure, they might have to tinker for a week or two, but I don't necessarily buy your argument.

Whereas the 'industry' solution deferred rendering is not potentially but really unmaintained, so there's that - didn't really seem to help.
Thorsten
 
Posts: 9539
Joined: Mon Nov 02, 2009 8:33 am

Re: PBR or Physically-based rendering

Postby Hooray » Sat Jul 15, 2017 5:04 pm

The pattern is a recurring one, i.e. the whole debate about "industry standards" being potentially "better" in comparison to what FlightGear as a community tends to come up with - I think we've seen some rather long debates about Nasal vs. Python and how using Nasal may have affected FlightGear for the worse ...
Then again, there are other examples to be found where adopting an industry standard obviously has been a good thing in the sense that it was a huge technology enabler (think OpenSceneGraph).
I am not familiar with the maintenance status of Rembrandt at all, but I seem to recall that folks like i4dnf were indeed submitting fgdata related Rembrandt patches at some point, which probably is not so much worse compared to the ALS situation ? Then again, ALS is purely fgdata as far as I know, whereas Rembrandt actually does require a C++ developer to maintain the parts that most people complain about (e.g. performance)

Looking at the way Rembrandt has been integrated, it was actually fairly bold to get it committed at all - it's tremendously complicating matters for anybody interested in dealing with the setup of the rendering pipeline and with the rendering main loop as a whole - the pre-Rembrandt status wasn't exactly that comfy either, but it was at least easier to understand how things were hanging together. Making heads and tails of the CameraGroup stuff nowadays is quite an undertaking to say the least, which is why it is particularly unfortunate that Rembrandt isn't been maintained these days - then again, Fred did actually create a fairly long article describing Rembrandt, it just seems that most of the integration bits are missing.

Overall, my impression is that Rembrandt no longer being maintained has primarily to do with the performance issues many people have reported, including a number of core developers, who stated that they were hesitant to jump on that band wagon, because of its performance issues.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11100
Joined: Tue Mar 25, 2008 8:40 am

PreviousNext

Return to Effects and shaders

Who is online

Users browsing this forum: No registered users and 5 guests