What rendering infrastructure are we actually missing? [...] when the details and parametrisation of the problem are completely lacking
Great question. I've tried to explain the problem as best as I could in the OP, but I guess I oversimplified and ended up confusing both end-users and people who are looking for the juicy details.
Let's start with multi-threading. This is not particularly related to real-time applications, but it's definitely one of the biggest reasons why we are seeing a paradigm shift in 3D graphics. CPU clock speeds are hitting their physical limits, so manufacturers started increasing the number of cores. OSG uses OpenGL, which is an ancient API, but has been kept alive by extensions and numerous patches. However, we have reached a point where the paradigm shift is so big that you can no longer patch the behemoth until it does what you want. That's why Vulkan appeared, DirectX 11 has almost been completely rewritten and Apple developed Metal. It is not debatable that we are seeing a huge change - if we weren't, these companies wouldn't have spent millions in developing these new APIs.
FG exclusively uses a single core (yes, it may use several cores for very specific jobs, but FG is basically single-threaded). Clock speeds aren't getting faster so we can't take advantage of new hardware, we are essentially stuck with 5/10 year old hardware even if you just bought a brand new Intel i9. However, we keep adding features to FlightGear. Aircrafts are getting more complex systems by the day, models have huge triangle counts etc. This wasn't a problem 10 years ago because while the simulator kept getting more complex, newer hardware helped us keep the performance stable. Time helped us, now it doesn't.
This is all known by the community already, I've seen some of the discussions in the multi-core wiki article. A solution would be to rewrite FG using
VulkanSceneGraph or something similar, while carefully rewriting the SGSubsystem loop to be multithreaded via a job dispatching system (or something like HLA). That's a mammoth task on its own, and unless a team of developers comes to the project and manifests their interest in rewriting FG, it won't happen any time soon. But let's assume this is all done. We have rewritten FG to take advantage of newer hardware. Now let's write a rendering pipeline for it.
Let's take concrete examples, HDR and PBR for example, since you mentioned them. HDR just means that shaders output the reflected light from the object to the viewer in absolute linear values. Later, these values are tone mapped and adjusted for the display range of the monitor. Legacy material definitions (ambient, diffuse and specular) don't make sense in such a system, so we need to use PBR. Note I'm using PBR to describe a material system where the material BRDF is described using physical values, the parametrization is not important. Is the team that moved FG to
VulkanSceneGraph also going to change every material definition of every aircraft in FG? You mentioned you don't see why this can't be optional. Well, you can't have two systems outputting to the same framebuffer: one outputting physical luminous intensity and the other outputting an arbitrary 0-1 color value. How do you merge them? Would it even look good? And if you are not convinced yet, what about lights? Do you define lights in the PBR style or in the legacy style? PBR lights won't be compatible with legacy materials, and legacy lights won't be compatible with PBR materials.
You could argue that this is all optional and we don't need HDR and PBR. You could argue that we have much bigger problems. And okay, that's what Thorsten thinks, but that's not the subject of the topic. I like 3D modelling and I sometimes (try to) do some art. But in this project I'm only interested in the core rendering, so I don't care if I'm told that the main issue is that we don't have enough 3D artists. I care about the barriers and obstacles we will be facing in the rendering side of FlightGear. I tried to show what the new Microsoft Flight Simulator is capable of, and how we can learn from it. Many game companies (I don't like making comparisons with the game industry, but they are the main creators of real-time applications) are opting to rewrite their engines to add these features and to pave the way for even newer features. If they are spending money in doing this, it is very likely that the reasons are more than justified, and perhaps we, as a community project, need to think about it.