## Optimization (a metaphor)

Discussion and requests for new features. Please note that FlightGear developers are volunteers and may or may not be able to consider these requests.

### Optimization (a metaphor)

(not sure into which subtopic this goes, feel free to move around).

We're quite frequently discussing about how to make FG faster. That's not as straightforward as it looks, so... let's take a step back and investigate a situation which is perhaps more familiar. Suppose we want to manufacture a car and optimize the assembly process.

The situation

We're a modern company, which means we have 'just in time' management for parts, we are unable to store half-assembled parts for instance - everything arrives at the processing plants when it's needed, everything gets picked up when finished.

We have two plants - plant 1 and plant 2. Plant 1 is a traditional assembly-line setup - it creates the car without engine. The tasks it has to do are

a) assemble frame
b) attach wheels
c) attach seats
d) attach doors
e) paint
f) put engine in as soon as it arrives

Plant 2 is a very modern setup in which the engine mechanical parts and the electronics are modules which can be put together more or less in parallel. Team A assembles the mechanical parts, and any member of the team just grabs two non-attached modules, connects them and hands the finished part into a basket, to be picked up by the next person. Team B takes any of the (half-assembled) structural parts and attaches electronics and sensors. Team A does mechanical parts only, team B electronics only, but each team member does whatever work needs to be done at any given point. However, electronics can only be attached when the structure for some part of the engine is done - there's no attaching to a structure which isn't built yet.

The bottleneck

Suppose plant 1 takes the following times to assemble the car:

assemble frame: 24 minutes / attach wheels: 4 minutes / attach seats: 4 minutes / attach doors: 2 minutes paint: 10 minutes / engine: 2 minutes

Team A in plant 2 takes 58 minutes to assemble a complete engine, team B takes 20 minutes to assemble the electronics.

When will a car be finished?

It's fairly straightforward to get the answer for the assembly line in plant 1 since each task is done after the other, so we can just add the numbers and find 46 minutes.

In plant 2, adding the numbers would be wrong, since team B starts working almost as soon as team A has assembled the first modules, so by and large they work in parallel, and team B needs to do only very little work after team A finished, so it actually takes some 58 minutes.

Yet, building the car then takes 60 minutes, because after 44 minutes the assembly line in plant a needs to stop and wait for the engine to arrive at 58 minutes, then the engine is put in which takes the last two minutes.

Making a car is obviously an analogy for computing a FG frame, plant 1 is the CPU, plant 2 is the graphics card, and while the CPU does tasks in sequence, the GPU internally does things in parallel - but the fragment shader (team B) needs to wait for the vertex shader to be finished (team A), and the frame needs to wait for all tasks to be completed.

Some things that do not work

A consultant might have the bright idea that while we can't paint the car before the doors are on, we can attach wheels and seats simultaneously. Each task takes 4 minutes, but we're not doing them in sequence now, we're doing them in parallel, so we save 4 minutes of the total.

Does that mean we can now roll out cars every 56 minutes? No - it means we finish the assembly line work after 40 minutes, then wait 18 minutes for the engine to arrive, and roll the car out after 60 minutes.

That's utilizing multiple CPUs better in a situation where the GPU is the limiting factor.

***

Another consultant might have the idea to hire more people for team B to get the engine done faster. So team B now only takes 10 minutes to attach the electronics. Yet - team B can't finish before team A does, so it still takes 58 minutes to do the engine and an hour to make a car.

That's streamlining fragment shader operations or getting more fragment processing cores in a situation that's vertex-shader limited.

What does work

In fact, the only thing that will in this setup get cars out faster is to hire more people for team A. If team A is able to assemble the engine structure in 15 minutes, and team B works in parallel but now takes 20 minutes, we're going to see an engine after 20 minutes. Then plant 2 can enjoy a break while the engine gets delivered to plant 1, waits there for 24 minutes, then is put in for two minutes, getting the car ready after 46 minutes. After which plant 2 can start to produce a new engine.

That's an upgrade of the graphics card to give it better vertex processing capability.

In fact, now the situation is different. Hiring more people for team B now does something, i.e. a whole engine can be assembled in 15 minutes. But this doesn't influence the speed at which we roll out cars, all it does is changing the time the engine waits in plant 1 from 24 to 29 minutes - the bottleneck is still in plant 1.

GPU update helps only if you're rendering-limited

But the other solution, i.e. attaching wheels and doors in parallel, now actually has an effect, and we really save four minutes.

Parallelizing helps if you don't need to wait for a task that is longer or needs to be serial.

What to target

The next vastly overpaid consultant has a look at how things in plant 1 are done and finds that attaching doors is really slow, and by investing a million, the car company can make it four times faster than it is now.

His colleague looks at assembling the frame, and finds that well, there's a modest optimization potential - it can be done 20% faster by investing a million.

Well, we only have a million to spare, thus what should we do?

In the first case, we're saving 75% of 2 minutes, or 1:30. In the second case, we're saving 20% of 24 minutes - or almost 5 minutes. So despite the relative increase being vastly larger in the first case, we should go for the second case.

A modest performance increase in a relevant subsystem that actually takes lots of time usually by far outweighs a dramatic performance increase in an irrelevant subsystem that hardly takes time to execute anyway - thus valuable developer time is better spent slightly improving the things that take lots of time than dramatically improving things that don't take measurable time anyway.

Final words

So, next time you feel like pointing at a random FG subsystem and claim we absolutely need to make this faster because it drags everyone's framerate, think of the car manufacturing process, how non-trivial it is to optimize and how many optimizations actually don't help, and how what helps may also be different from one setup to the next.
Thorsten

Posts: 11375
Joined: Mon Nov 02, 2009 8:33 am

### Re: Optimization (a metaphor)

Brilliant analogy. Thanks.

Let's hope the correct FGers read it.

Vic
Time flies like an arrow
Fruit flies like a banana

VicMar

Posts: 2061
Joined: Sun Apr 06, 2008 5:53 pm
Location: Lancing. UK (EGKA)
Callsign: VicMar
Version: 2018.3.1
OS: OS X 10.12.6

### Re: Optimization (a metaphor)

So it seems the only case in which optimization in CPU work(like running in parallel etc) would be beneficial is, when it takes less than or equal the time for pipeline related work(vertex+fragments shaders) to be performed.

but i would guess CPU related work comes in many forms, for example Nasal, although is performed on the CPU, is still binded to the graphical frame(where i qouted you mentioning that <15 fps is unstable for AP if it used Nasal), but when does it actually penalize the graphical performance? (amount of time from relative tasks, to vertex shader to buffer), is it only when using the Canvas subsystem? since it is rendering related?

quoting what you said in the "What does work" sub-section to provide context:

Thorsten wrote in Wed Nov 25, 2015 7:59 am:In fact, now the situation is different. Hiring more people for team B now does something, i.e. a whole engine can be assembled in 15 minutes. But this doesn't influence the speed at which we roll out cars, all it does is changing the time the engine waits in plant 1 from 24 to 29 minutes - the bottleneck is still in plant 1.

You then mentioned that "GPU update helps only if you're rendering-limited",

edit: as in rendering (plant 2) takes more time then CPU (plant 1)

But for the inverse, If we have both a good CPU and GPU, plant 1(CPU) doesn't just wait 5 minutes longer after the faster GPU Update (from plant 2's engine work), but we can actually roll the car "frame/chassis" with the engines attached in less time now.

Thank you for this analogy, i think it is the better way to provide analogies such as this for the layman, instead of repeating techno speak for the uninitiated.

*this been edited many times as i re-read.
Last edited by hamzaalloush on Wed Nov 25, 2015 5:24 pm, edited 1 time in total.
hamzaalloush

Posts: 632
Joined: Sat Oct 26, 2013 9:31 am
OS: Windows 10

### Re: Optimization (a metaphor)

hamzaalloush wrote in Wed Nov 25, 2015 5:00 pm:So it seems the only case in which optimization in CPU work(like running in parallel etc) would be beneficial is, when it takes less than or equal the time for pipeline related work(vertex+fragments shaders) to be performed.

Only if fps is your only concern

Regards,
Edward
bugman
Moderator

Posts: 1736
Joined: Thu Mar 19, 2015 9:01 am
Version: next

### Re: Optimization (a metaphor)

Hi Edward,

As i mentioned, CPU related work comes in many form, we can improve Nasal (or run in parallel) not only for rendering related activities, but also for others, as well as FDM, system or whatever else, but my focus was for rendering related activities, since i asked the question weather Canvas would be the sub-system responsible for any graphical performance issues and not pure Nasal(Canvas in this case would be one of the tasks on CPU side that Thorsten represented as the "Frame" department), and we are discussing why HLA wouldn't be beneficial to fps(my main concern in this thread) if you are already rendering limited, i hope this made sense.

*also edited many times...
hamzaalloush

Posts: 632
Joined: Sat Oct 26, 2013 9:31 am
OS: Windows 10

### Re: Optimization (a metaphor)

I'm aware of the discussions. I'm just pointing out that there are contexts where fps doesn't matter, for example the headless concept. And developers writing new C++ code, in general they care or should care about the speed of their code independent of the rest of the system. Using the car analogy, if you are part of a team assigned to design and build a single component of the car, then you make sure your part is well designed, taking the whole into account (i.e. not needlessly wasting CPU cycles). But if you are chasing maximising fps, then you need to quantify how much influence the parts have on the whole.

Anyway, considering the single part, it is good to remember that Nasal is a standalone scripting language which just so happens to be embedded into FlightGear.

Regards,
Edward
bugman
Moderator

Posts: 1736
Joined: Thu Mar 19, 2015 9:01 am
Version: next

### Re: Optimization (a metaphor)

And developers writing new C++ code, in general they care or should care about the speed of their code independent of the rest of the system.

I don't think it's that simple. Often (at least in rendering) you can trade

* memory consumption against performance
* slower general purpose code with low maintenance load against faster special purpose code optimized for a particular task
* clear structure against optimized mess

and what I would choose is different dependent on whether the bit I'm looking at happens to be performance critical or not.
Thorsten

Posts: 11375
Joined: Mon Nov 02, 2009 8:33 am

### Re: Optimization (a metaphor)

VicMar wrote in Wed Nov 25, 2015 9:08 am:Brilliant analogy. Thanks.

+1

Also, you have some good points above. It often seem engineering is more about trade-offs than about sheer optimization, but then again well though out trade-offs is optimization in its own right.
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator

Posts: 5546
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

### Re: Optimization (a metaphor)

Like I mentioned already in private, that's indeed brilliant and well-suited to bring across a whole number of non-trivial points. But I would hope that this can be revisited/refined with a focus on how being CPU/GPU bound relates to this analogy, as well as the impact on HLA/RTI (it primarily helping with CPU-bound issues, as well as distributing processes), but also overall resource utilization (think RAM/VRAM), and how swapping/paging relates to this, i.e. using more memory than the operating system/process can address.

I do agree with bugman's point, especially regarding standalone systems, and how their performance relates to the whole system, i.e. the whole relationship between asynchronous updates (OSG cull/update), vs. the sequential main loop, where only parts are threaded (think sound), with others running interleaved (FDM/AP).

If this metaphor could be reviewed/edited accordingly, we could point people to the article, and end a bunch of debates, including not just debates on "optimizing" fg, but also on having to profile it first, and why understanding the performance of a system in isolation is useful, and its inputs/outputs affect other subsystems
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray

Posts: 11475
Joined: Tue Mar 25, 2008 8:40 am

### Re: Optimization (a metaphor)

@Hooray,

Is there any chance you could explain your meaning without adopting techno-speak, so those of us who appreciate what is being discussed but don't understand the lingo can keep up?

Another analogy perhaps?
Time flies like an arrow
Fruit flies like a banana

VicMar

Posts: 2061
Joined: Sun Apr 06, 2008 5:53 pm
Location: Lancing. UK (EGKA)
Callsign: VicMar
Version: 2018.3.1
OS: OS X 10.12.6

### Re: Optimization (a metaphor)

the idea was to refine the metaphor so that it can be also used to cover other issues not commonly understood by most people, such as how the number of CPUs/GPUs and the amount of VRAM/RAM relates to the whole "assembly line" metaphor, but also how ongoing core development efforts -like HLA- relate to it
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray

Posts: 11475
Joined: Tue Mar 25, 2008 8:40 am

### Re: Optimization (a metaphor)

DOH!!
Time flies like an arrow
Fruit flies like a banana

VicMar

Posts: 2061
Joined: Sun Apr 06, 2008 5:53 pm
Location: Lancing. UK (EGKA)
Callsign: VicMar
Version: 2018.3.1
OS: OS X 10.12.6

### Re: Optimization (a metaphor)

The question is, if it's worth it, the very instant you need analogies and metaphors, it is even more unlikely that the people reading it will directly apply this knowledge to benefit the FG project.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray

Posts: 11475
Joined: Tue Mar 25, 2008 8:40 am

### Re: Optimization (a metaphor)

The question is, if it's worth it, the very instant you need analogies and metaphors,
You may have a point there, but if people are trying to understand what is being discussed I consider it extremely bad practice to continue to talk in code/riddles.
it is even more unlikely that the people reading it will directly apply this knowledge to benefit the FG project
That is most likely to make interested FGers stop reading the thread. It could even be put down to arrogance on your part.

I've no intention to fall out with you but - Do you really think it's your way or nothing? Perhaps it's time to realise many FGers have limited coding experience and not treating them as stupid because of that fact might make your epistles better reading.

Personally, I find a lot of your postings very informative but having to find translation for obscure acronims can be a turn off.

Vic
Time flies like an arrow
Fruit flies like a banana

VicMar

Posts: 2061
Joined: Sun Apr 06, 2008 5:53 pm
Location: Lancing. UK (EGKA)
Callsign: VicMar
Version: 2018.3.1
OS: OS X 10.12.6

### Re: Optimization (a metaphor)

point well taken, but let's say there is a fancy FDM related debate going on and I keep asking about AoA, VS, fpa, stall etc - I would obviously be unlikely to apply the corresponding knowledge for the benefit of the project, and I could just as well "google" (research) the terms elsewhere if I am truly interested in learning more.
Otherwise, I am really just wasting the time of those who are trying to accomplish something, without investing my own time to fill in any gaps.

Look at someone like Thorsten: You are unlikely to understand his ALS related light equations unless you have a strong background in math and physics, so he can also tell immediately if/how certain postings are going to materialize in the form of contributions.

(Note: I also don't have a degree in math/physics, so all this would apply just as much to myself as it would apply to others, and I am fine with that)

PS: And please don't be worried about "falling out", in fact, you would be in most excellent company
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray

Posts: 11475
Joined: Tue Mar 25, 2008 8:40 am

Next