Board index FlightGear Development Effects and shaders

optimizing random trees: strong effect on performance ?

An exciting "new" option in FlightGear, that includes reflections, lightmaps, the particle system etc.. A lot is yet to be discovered/implemented!

optimizing random trees: strong effect on performance ?

Postby abassign » Thu Nov 12, 2015 9:14 am

@Thorsten ? ... I'm sorry, I do not have enough imagination :( to understand your subtle consideration

I want to point out that an important optimization you might have with the trees which, when activated, lead to a strong deterioration in the performance. I think not all have an interest in trees displayed by the FGFS, but if you are flying VFR, I think they are rather important for the realism of the flight especially in phases of take-offs and landings. Optimizing the rendering of the trees could greatly improve the performance of the system (remember @Thorsten did an excellent optimization with the shadows of the trees). About a year ago it has been done a great job of optimizing the rendering of the terrain, and now also with LOD of 600 km it is possible to have excellent performance.
Even the clouds have a very high cost of processing, I used a cloud cover to 1.0, you can reduce this parameter, but obviously this is a test and it seems useful to bring the system to the highest quality levels. I wonder if there are better techniques to create clouds following the methodological @Thorsten track.
abassign
 
Posts: 767
Joined: Mon Feb 27, 2012 5:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2018.3
OS: Linux Mint 19. x

Re: Su-15

Postby Thorsten » Thu Nov 12, 2015 9:34 am

I want to point out that an important optimization you might have with the trees which, when activated, lead to a strong deterioration in the performance.


By all means optimize them if you know how to. I'm afraid however simply stating 'I want X to run faster' won't do anything.
Thorsten
 
Posts: 10587
Joined: Mon Nov 02, 2009 8:33 am

Re: Su-15

Postby abassign » Thu Nov 12, 2015 10:10 am

@Thorsten
To optimize the display of the trees I think we need to think with the same technique as was done with the ground in order to increase the LOD.
The tree is currently made of two pieces, it is possible, over a certain distance, only use one turning in front of the observation point as we do with the clouds? Obviously mine is just an idea. Someone pointed out the cost in the rendering of transparencies, the trees are full of transparency, it is true, then it is possible to use the same tree, but without transparency when they are far enough ? I remember that you did a great job with the far clouds, is it possible apply the same principle to the far trees (perhaps through a parameter that everybody can operate his tastes ... ;) )...
I have not sufficient knowledge of the techniques of rendering to give a certain guideline :( ...
abassign
 
Posts: 767
Joined: Mon Feb 27, 2012 5:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2018.3
OS: Linux Mint 19. x

Re: optimizing random trees: strong effect on performance ?

Postby Thorsten » Thu Nov 12, 2015 11:06 am

Right...

Let me ask you straight: A number of very clever people have been over the tree code. There's design by Tim Moore who designed the FG effect system, there's work by Stuart Buchanan in, both did the OpenGL part of it in the core, I have brooded over the ALS part of it in the GLSL shaders.

Do you honestly think that if there were a straightforward way of optimizing the vegetation without side effects, we wouldn't have done it already? That we deliberately let users put up with slow code till a user actually tells us to optimize? That we didn't each try a number of different schemes already? Or that we somehow don't realize that vegetation costs performance?

Or what is the actual reason you feel you need to tell us where to optimize?

it is true, then it is possible to use the same tree, but without transparency when they are far enough


It sure is possible - then you get a pretty tree picture on a white rectangular texture sheet.

The tree is currently made of two pieces, it is possible, over a certain distance, only use one turning in front of the observation point as we do with the clouds?


Doing that, you will discover that turning one tree to face you is more expensive than simple having two and not turning either. Has been tried already and discarded for that reason.

I have not sufficient knowledge of the techniques of rendering to give a certain guideline


Then you should perhaps trust that the people who do rendering for FG are neither habitually stupid nor mean and have worked hard to make the trees what they are now, and that stating 'This should be faster in my opinion' is not useful unless you at least are aware of what has been tried before and have a new idea.
Thorsten
 
Posts: 10587
Joined: Mon Nov 02, 2009 8:33 am

Re: optimizing random trees: strong effect on performance ?

Postby Hooray » Thu Nov 12, 2015 11:12 am

You are certainly right, but on the other hand, we should also not "habitually assume" that all existing code is necessarily peer-reviewed and highly optimized, just because it has been around for so long and touched by a number of developers - for instance, just look at psadro_gm's LOD work, quoting Stuart:

Subject: Level of detail
stuart wrote:You've managed to solve in a couple of days what I've been struggling with for months, plus found a massive memory saving. I'm massively impressed.

Thanks again.

-Stuart
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11312
Joined: Tue Mar 25, 2008 8:40 am

Re: optimizing random trees: strong effect on performance ?

Postby Thorsten » Thu Nov 12, 2015 12:02 pm

Look, it's obviously a difference if I have an idea of the matter at hand and ask 'Did we try that and that approach perhaps? I think it might help for that and that reason.' or just state 'I think X needs to be optimized - so you go ahead and do it please.'

There's a record of trying to optimize tree rendering both in the commit history of the effect as well as in the forum in various discussions, and there's just no reason to assume people would be unaware that trees take lots of performance and need to be pointed to it.

So yes - maybe there is a clever way without side effects - then tell me and we'll try to make it work.
Thorsten
 
Posts: 10587
Joined: Mon Nov 02, 2009 8:33 am

Re: optimizing random trees: strong effect on performance ?

Postby Hooray » Thu Nov 12, 2015 12:13 pm

As you know, I am not in a good position to help with any of this, and quite frankly: you almost certainly don't want me to even just look at this, because you could not handle the sheer amount of PMs with questions about lower-level details that are completely eluding me when it comes to effects, shaders and the underlying 3D math (again, just how many different coordinate systems are currently used in shader space??):lol:

however, if I were interested in this, I would inspect the osg-stats with different shader settings and run an OpenGL/GLSL profiler to determine where most time is spent - probably posting my findings/screen shots on the forum so that you can take a look.
Next, I would use the minimal startup profile (over terrain) and dump the scene graph to a file, and then run the OSG optimizer on the root node to see what it can do, and then compare the two .osg files to see if any nodes/osg::StateSets can be flattened.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11312
Joined: Tue Mar 25, 2008 8:40 am

Re: optimizing random trees: strong effect on performance ?

Postby Hooray » Thu Nov 12, 2015 5:50 pm

PINTO wrote:@Thorsten
So people shouldn't let the devs know when there's a component of Flightgear that has a negative impact on their flightsimming experience while using FG?
Right. How will that attitude help attract younger people, people without much computer science experience, or people who are looking at the FG community determining if they want to be apart of it?
I in fact have coding experience, from Python/Lua all the way down to assembly. I will not, however, be helping out this project (in coding) in the least, other than some minor aircraft tweaking, largely due to how you treat people who come to these forums offering insight and suggestions.


Personally, I doubt that this is the reason for not getting involved in graphics stuff.
There are frankly many other factors that are a much greater barrier to entry.

I mean, Thorsten may be the primary developer involved in rendering/graphics who is active on the forum, but there are others, too - such as Stuart (who originally wrote the random vegetation code IIRC).

Also, if you disagree with the way Thorsten is handling such "feature requests", you are obviously invited to deal with them differently, especially should you have to contribute anything specific to the matter at hand.

That said, people who are active on the forum may not be as active on the devel list (and vice versa), which is to say that someone who seems to have a certain standing/reputation on one platform, may not be overly active on the other - and especially on the devel list, you will find others familiar with C++/OSG and OpenGL coding, who should feel quite comfortable reviewing any patches/feedback you may have to help improve the performance of rendering related code like the random trees.

I don't disagree that we may sometimes appear pretty harsh, or even blunt, but you also need to look at this from the perspective of those who literally spend hundreds, if not thousands, of hours contributing to an OSS project, dedicating their skills, expertise and spare time - the worst, and most disrespectiful, thing someone can do to those people is to waste their time, because "spare time" is exactly the "currency" that counts in a volunteer-based project like FlightGear, where you can only contribute your skills and expertise if you have a sufficient degree of time, too.

Most of the time spent on the forum could be easily said to be a waste of people's time unfortunately, due to the nature of the typical exchange with end-users - for instance, look at my number of postings (right-hand side), it's somewhere around 8k, right - now let's imagine that each of those postings took on average 3-4 minutes to write - which is roughly 25k minutes, in other words, all that time could be spent elsewhere.

And now look at how many contributors are actually actively involved in the forum.
Personally, I don't mind dealing with people's ideas, feature requests - but the exchange should be fruitful/constructive, i.e. contain either sufficient information to be actionable (i.e. bug-reports), demonstrate sufficient follow-up to understand an issue, or sufficiently informed to have a brainstorming of what's possible.

So far, I have yet to see anything like this in this thread - which is unfortunate but true, and regardless of your background in python/lua and assembly language, I don't see anything suggesting that you would be any different (without meaning to offend you).

There is something like a signal/noise ratio, and people can usually spot pretty quickly if you are worth dealing with or not (sorry if that sounds pretty blunt, too) - either you are making a bug report, a feature requests or a suggestion - but you need to behave accordingly, especially if you want to be taken seriously.

If I were in abassign's position, I would team up with others interested in benchmarking and profiling FG, hoping to come up with findings that are sufficiently actionable for Thorsten and others to act upon those.

To be fair, you WILL also find postings by Thorsten and myself about "Rembrandt being too slow", along the lines of "something is wrong.... and needs fixing" - which is also not particularly informed admittedly, so there's that, too - and keep in mind that others like F-JJTH suggested that people getting bad performance on Rembrandt, would simply be incapable of configuring their computers properly.

If you think there is an actual issue/bug with a certain feature, please provide much more data - if in doubt, start a new thread, and ask how to do that, i.e. something along the lines of "feature XYZ seems to have an odd impact on my system (specs) when using the following settings (details), anybody else seeing this - is anybody interested in working out what is going on here? "

I am sure that you could be helpful without having to be a glsl/graphics expert.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11312
Joined: Tue Mar 25, 2008 8:40 am

Re: optimizing random trees: strong effect on performance ?

Postby PINTO » Thu Nov 12, 2015 6:02 pm

I wrote that post in haste, so I apologize. I have since deleted it.
And yes, that is 100% the reason I will not be contributing to the main source code.
Yes, I have contributed hundreds of man-hours to open source projects before. I know that time is precious and resources are scarce. I do, too, wish that bug reports could be helpful all the time, but more often that not (in my experience), it won't be. I don't think having a flippant attitude is justified, however.
Actively developing the MiG-21bis (link to repo) (link to thread)

http://opredflag.com is an active flightgear dogfighting community (using a system that isn’t bombable)
User avatar
PINTO
 
Posts: 935
Joined: Wed Oct 21, 2015 6:28 pm
Callsign: pinto
Version: 2016.3.0
OS: Win10

Re: optimizing random trees: strong effect on performance ?

Postby Hooray » Thu Nov 12, 2015 6:09 pm

look, if you disagree with another contributor (or their attitude), you can always deal with others instead.
Personally, I find it pathetic to just attribute this to a single person's behavior, no matter if it's justified or not.

Besides, should you have the background you mentioned, there are many other areas to which you could contribute, without ever having to deal with Thorsten (or myself for that matter).
For instance, the scenery guys seem to be a fairly nice/agreeable bunch of folks (hi, psadro_gm !)
Equally, you will find that other contributors have decided not to deal with certain contributors, while still contributing (i4dnf, vitos etc)

The very instant you make your involvement depend on someone else's attitude/behavior, you are also empowering the very people you are disagreeing with - no matter if you are right or not.

The best, and most compelling, way to disagree with someone in an open source project is to prove them "wrong" on technical grounds - and even if you should manage pulling that off, Thorsten belongs to those who would appreciate that (that is, after a few days have passed)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11312
Joined: Tue Mar 25, 2008 8:40 am

Re: optimizing random trees: strong effect on performance ?

Postby Thorsten » Thu Nov 12, 2015 6:16 pm

So people shouldn't let the devs know when there's a component of Flightgear that has a negative impact on their flightsimming experience while using FG?


Well, try the difference between

'I've noticed that whenever the random lights come on at night, my framerate goes down by 20% - is anyone else experiencing this?'

and

'I want to point out that an important optimization you might have with the trees'

and see whether you can spot a difference.

Got it?

In the first case, you're not assuming a priori that someone else screwed up and are not graciously pointing the 'clueless' developers the right direction which apparently only you happen to know.


Basically stating you want FG to run faster is null information - I don't know a single person who wants it to run slower, and even if you find such a person, there is framerate throttling. Stating you want FG to be visually more appealing is null information - don't we all?

If you've spent a few weeks with the tree code, testing half a dozen different strategies to squeeze a few more fps out of it, and you get told offhand by someone who has no real track record of contributing anything or of debugging rendering that you just have to optimize the code, you might come to think of that as either disrespectful or just clueless about the history and the problem involved.

I guess it boils down to thinking before posting:

What is the actual information I have (and what have I silently assumed)?
Is there reason to assume that others do not already have this information, i.e. is it worth telling?
Do I understand the problem well enough to give a recommendation, or should I better just state an observation, or perhaps only ask a question?

Having said that - we all get that wrong now and then, nobody is perfect. But it can't hurt to try.


I will not, however, be helping out this project (in coding) in the least, other than some minor aircraft tweaking, largely due to how you treat people who come to these forums offering insight and suggestions.


I guess we may disagree what an actual insight is - in my book 'feature X is too slow' is a null information as pointed out above, and 'you need to optimize it' is not a suggestion either. You will find that posting an actual insight makes a lot of difference and often people go to some length to follow up on it.
Thorsten
 
Posts: 10587
Joined: Mon Nov 02, 2009 8:33 am

Re: optimizing random trees: strong effect on performance ?

Postby PINTO » Thu Nov 12, 2015 6:23 pm

Thorsten wrote in Thu Nov 12, 2015 6:16 pm:"I've noticed that whenever the random lights come on at night, my framerate goes down by 20% - is anyone else experiencing this?'

To be fair, the actual quote was:
I want to point out that an important optimization you might have with the trees which, when activated, lead to a strong deterioration in the performance.

I don't want to argue semantics, but moving the comma-parenthetical to another spot, and remove the percentage from your example, and basically it's the same thing. He wasn't stating he wanted trees to be faster per se, but rather that they have contribute to a large degradation of frames per second on his machine. He didn't get any numbers, but a simple "Can you post the fps you get? Preferably with screenshots?" would've been better, and taken less posting time, and garnered more information, than that wall of text you did post.

FWIW, I leave trees off because of the sometimes severe performance degredation. Having them on .5 can lead to a 50% drop in my performance (~25fps to around 12fps).
Actively developing the MiG-21bis (link to repo) (link to thread)

http://opredflag.com is an active flightgear dogfighting community (using a system that isn’t bombable)
User avatar
PINTO
 
Posts: 935
Joined: Wed Oct 21, 2015 6:28 pm
Callsign: pinto
Version: 2016.3.0
OS: Win10

Re: optimizing random trees: strong effect on performance ?

Postby Hooray » Thu Nov 12, 2015 6:26 pm

You should probably better ask for tests that you could run, and screenshots/settings to share in order to illustrate the problem
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11312
Joined: Tue Mar 25, 2008 8:40 am

Re: optimizing random trees: strong effect on performance ?

Postby stuart » Thu Nov 12, 2015 8:30 pm

Hi abassign,

I've spent quite a bit of time looking at ways to optimize the tree rendering. I'm fairly confident that we've tried most of the "low hanging fruit" at one time or another and kept the things that make a worthwhile difference: things like reducing the number of quads, reducing the density of trees further away, and applying LoD techniques and Quadtrees. There may be further tuning that could be done to the latter two techniques to eek out further performance at the cost of some visual effects, but I'm not hopeful.

The one area that might make a big difference is the use of Impostors, which basically replaces a whole bunch of the scene-graph with a texture. Unfortunately the OSG Impostor class doesn't seem to work very well, and it's not clear if it works well with shader-rendered objects like the trees and clouds.

I'm also not clear that tree rendering is a limiting factor for most people's systems.

The LoD loading the Hooray mentions was a different case and a much more substantial change.

Best regards,

-Stuart
G-MWLX
User avatar
stuart
Moderator
 
Posts: 1467
Joined: Wed Nov 29, 2006 9:56 am
Location: Edinburgh
Callsign: G-MWLX

Re: optimizing random trees: strong effect on performance ?

Postby Hooray » Thu Nov 12, 2015 8:56 pm

FYI: it's a coincidence that you were involved in both parts of the code, I only mentioned it to support the notion that some of our end-users may have a valid point despite not being able to find the right words or even just provide actionable information to begin with.

I could have just as well mentioned Torsten's bugfix that fixed the leaking effects listeners, or the memory leak in the routePath implementation - both of these were also discussed on the forum, long before anybody had a clue what was going on, and we would post knee-jerk reactions that people would not know what they were talking about, only to be proven wrong 18+ months later.

Which is not to say that I believe there is anything wrong with the tree code - I never looked at it, and don't even have it enabled usually. I am just suggesting that people try to provide actionable information so that we can determine if there is a problem or not.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11312
Joined: Tue Mar 25, 2008 8:40 am

Next

Return to Effects and shaders

Who is online

Users browsing this forum: No registered users and 1 guest