I would not say that a benchmark comparing Rembrandt with non-Rembrandt is necessarily meaningless, as long as the majority of features in each mode is disabled, so that conclusions can be drawn about the impact of certain features - IIRC, that is basically how Richard came up with the hypothesis that Rembrand't performance issues may be in some way due to OSG-level issues involving camera/vsync-limits:
http://wiki.flightgear.org/FlightGear_N ... _and_VsyncNow, I don't know if you, or anybody else familiar with the effects/GLSL and OSG side of things, ever got around to looking at those findings, but it would definitely make sense to revisit this at some point, simply because Rembrandt may currently be the main component in FG using slaved cameras to this degree, but as you know, we do have other cameras/components making fairly extensive use of OSG cameras that should not necessarily be vsync limited, e.g. Canvas coming to mind - and the other thing being CompositeViewer support:
http://wiki.flightgear.org/CompositeViewer_SupportAdmittedly, I belong to the camp of those people (like Thorsten) with fairly powerful hardware, who are still not able to get very impressive frame rates - a few months ago, F-JJTH was politely suggesting that we simply would not understand how to configure our systems properly. From my standpoint, I never got sufficiently motivated to look at these issues beyond exchanging osg-stats with Thorsten.
But like Richard's tests have shown, it is indeed possible to come up with tests/benchmarks and come up with useful data, or even a working hypothesis.
So personally I would hope that we don't simply disregard such information or shut down such efforts too prematurely, especially given the very low number of active developers familiar with effects, shaders and the OSG/C++ side of all this (largely unmaintained code).
Looking back in time, some fairly significant bugs were discovered, and subsequently fixed during the last 1-2 release cycles, many of which could have been identified/fixed earlier, had we paid more attention to end user feedback, and not just disregarded things.
I am familiar with hamzaalloush's benchmarking efforts, and I really hope that we can grow a library of tests and extend fgfs to enable end-users to make better bug reports, even unrelated to ALS/Rembrandt in particular.