if you have any idea how to arrive at a realistic quality control without these side effects, let me know.
This is a very tough question on which i will not claim to have the answer. But let's say i have somewhat an idea of the direction to look for an answer.
I think it is not feasible to try to say: this 737 really flies like a 737. Then we need pilots for every model in fg. But it is feasible to judge if a plane behaves somewhat like a plane or like nothing or is totally nature defying. To test this in a reasonable objective manner a checklist could be used. If such list is explanatory enough, almost anyone could test a plane.
When i test a plane i already use such a list in the back of my mind.
- climb: Does the plane gradually get a harder time climbing when getting in thinner air? I have had planes that climed like mad and whould not stop climbing. That got me into "space" floating at fl 2000. Others climbed like mad and suddenly just stopped climbing at a certain fl.
- desc: Is the plane able to desc without speeding up? I have had planes that speeded up at the slightest angle of desc (without power of course).
- config change: Does anything reasonable happen with a config change? For flaps that would be in the first settings more lift than drag, and in the last few settings a lot drag for little extra lift. I have had planes where nothing happened. I could just flick through all flap settings without any effect. Another one put all the drag in the first 2 settings and after that nothing more.
- speed: Are take off and landing speeds reasonable? A jetliner that needs way over 200 kts to get off the ground is somewhat unreasonable. Same goes for landing. You must be able to get near 140 kts without stalling. Also cruise speed is interesting. Some subsonic jets happily fly faster than the speed of sound in level flight.
- balance: Check the trim setting in cruise. Is it near center, then it is fine. Is it fully at the end of the envelope, then it is rather bad.
- cross wind: Is the rudder effective enough to counter cross wind? Does the plane want to vane into the wind at high speeds on the take off run? Some developers have just glued the wheels to the ground so there is no such effect whatsoever.
If such a checklist could be agreed upon then there still can be disagreement, but i think it can be solved quite quickly. If a tester says there is no such-and-such effect and the developer says there is, but very little, then the tester or anyone else can verify that. There is no way to judge how big the amount of effect should be. Only if it is there or not.
About sensitivities of developers i can say: I am not thinking about throwing away a plane all together. Just removing it from being advertized on the main download page. It should still sit in it's version control repository waiting to be fixed so it can be advertized again.
Kind regards, Vincent