Board index FlightGear Development Aircraft

Aircraft Rating System

Questions and discussion about creating aircraft. Flight dynamics, 3d models, cockpits, systems, animation, textures.

Re: Aircraft Rating System

Postby mr_no » Wed May 02, 2012 1:00 am

Let me say one thing about ratings: MEH!
It's all a matter of taste. Usually a user will like something other than a developer, and two users or two developers will not agree. Users will often like an unfinished model that is fun to fly for some reason and developers will debate of how realistic or well modeled/textured it is. Relax. Just have fun. :|
Mosquito-XE JT-5B-autogyro Extra-300s STOL-Ch701
User avatar
mr_no
 
Posts: 360
Joined: Thu Jan 19, 2012 3:20 pm

Re: Aircraft Rating System

Postby helijah » Wed May 02, 2012 1:23 am

mr_no wrote in Wed May 02, 2012 1:00 am:Let me say one thing about ratings: MEH!
It's all a matter of taste. Usually a user will like something other than a developer, and two users or two developers will not agree. Users will often like an unfinished model that is fun to fly for some reason and developers will debate of how realistic or well modeled/textured it is. Relax. Just have fun. :|


absolutely :)

FG is a hobby before anything else. And this is how I would continue to consider it. :p

Regards Emmanuel
Some planes (and other) for FlightGear
http://helijah.free.fr
and
http://embaranger.free.fr
User avatar
helijah
 
Posts: 1338
Joined: Wed Dec 27, 2006 1:35 pm
Location: Chartres (France)
Callsign: helijah
IRC name: helijah
Version: GIT
OS: GNU/Linux

Re: Aircraft Rating System

Postby hvengel » Fri May 04, 2012 10:23 pm

F-JJTH wrote in Thu Apr 12, 2012 6:44 pm:
fredb wrote in Wed Apr 11, 2012 5:41 pm:What about an additional flag or rating to signal aircraft that was converted to run with rembrandt (working lights and shaders, transparent surfaces verified) ?

Why not ;)


This is not a "rating" and does not appear to be something that belongs in the rating system.

In the Wiki aircraft can be given certain designators that show that they have been an "Aircraft of the month" or are "Bombable" among other things. I think this is probably how Rembrandt compatibility should be shown. See http://wiki.flightgear.org/FlightGear_N ... ruary_2012 for more details.
hvengel
Retired
 
Posts: 1127
Joined: Sun Dec 24, 2006 5:35 am
Location: Minden Nevada

Re: Aircraft Rating System

Postby fredb » Fri May 04, 2012 10:45 pm

So you suggest that one should look in the wiki to see if an aircraft will be properly rendered instead of having the information directly available in the launcher ?
User avatar
fredb
 
Posts: 753
Joined: Fri Dec 01, 2006 11:41 am
Location: Paris, France

Re: Aircraft Rating System

Postby Hooray » Fri May 04, 2012 10:53 pm

Introducing additional XML tags is cheap and simple, I think it would make sense to add more meta information to the aircraft-set.xml file - including rembrandt-related information. Launchers like fgrun could then be a little smarter and automatically check the aircraft-set.xml file once the --enable-rembrandt is selected, so that they can show a warning if incompatible (non ported) aircraft are selected.

The wiki is great, but the whole point of the aircraft status tag was that smart frontends could be created (launchers, download sites etc) - the wiki information is redundant, just because it cannot parse the aircraft-set.xml directly.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Aircraft Rating System

Postby Philosopher » Sun Aug 19, 2012 4:53 am

I agree with fredb and Hooray: it should be a rating, since, from what I hear, it is just as important as the other ratings and doesn't really cost anything to implement. Besides, who looks at aircaft on the wiki?

I also have two more opinions: that it shouldn't be used to calculate alpha/beta/production status (because it isn't so much as is it a finished model as whether work has been done bleeding-edge-ing-ly* lately & to avoid updating launchers), and that it should have 3 values:
• 0: No work has been done, and things will (probably) look bad with Rembrandt enabled.
• 1: Some work has been done so that it won't look bad, but it won't take advantage of all the features Rembrandt has to offer.
• 2: The aircraft is fully Rembrandt-ready and will look great.

*Bleeding-edge-ing-ly: had to make it into an adjective ;)
Philosopher
 
Posts: 1593
Joined: Sun Aug 12, 2012 7:29 pm

Re: Aircraft Rating System

Postby HJ1AN » Tue May 27, 2014 6:37 am

Apologies for bumping an old thread but I feel it's related..

Why does the Ogel "aircraft" have a full rating on the web site? This alongside the Sopwith Camel, Hurricane and TU154B (4 in total).
User avatar
HJ1AN
 
Posts: 374
Joined: Sat Jul 25, 2009 5:45 am
Callsign: HJ888
Version: 3.4
OS: OS X

Re: Aircraft Rating System

Postby hvengel » Tue May 27, 2014 4:15 pm

HJ1AN wrote in Tue May 27, 2014 6:37 am:Apologies for bumping an old thread but I feel it's related..

Why does the Ogel "aircraft" have a full rating on the web site? This alongside the Sopwith Camel, Hurricane and TU154B (4 in total).


I have wondered about the Ogel ratings my self. How can an aircraft model exactly replicate something that does not exist? Of course it is a fantasy so I guess that it meets the specifications of that fantasy. Not sure what should happen but I think it is a hole in the rating system that aircraft that don't exist IRL are rated using what was intended to be an objective system to compare FG aircraft to their real world counter parts. In this case it is used to, instead, compare the FG model to an aircraft that only exists in FG. So the Ogel is compared to itself so it will always be a 5* model no matter what. So for aircraft like this the rating is totally meaningless. I also think most FG users will know this so it may not be an issue to get worked up about other than the noise factor. So as long as we don't see too many new 5* fantasy FG aircraft appearing I don't think this is a significant issue.
hvengel
Retired
 
Posts: 1127
Joined: Sun Dec 24, 2006 5:35 am
Location: Minden Nevada

Re: Aircraft Rating System

Postby HJ1AN » Wed May 28, 2014 2:25 am

Well, in my mind, the thing would not fly at all, so the FDM shouldn't be rated as accurate (does it actually fly, I never tried). There are also some other fantasy planes that are - i think - rated quite appropriately. Anyways, it's not a big issue just something I noticed.
User avatar
HJ1AN
 
Posts: 374
Joined: Sat Jul 25, 2009 5:45 am
Callsign: HJ888
Version: 3.4
OS: OS X

Re: Aircraft Rating System

Postby Thorsten » Wed May 28, 2014 6:44 am

Well, once we open Pandora's box, knowing now how YaSim does things internally, I keep wondering how YaSim aircraft should even conceptually be able to reach a number above 3 for the FDM. But let's just not go there, having the system at all was a hard fight, and it selects on something useful.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Aircraft Rating System

Postby Michat » Wed May 28, 2014 8:02 am

Apart of the technical aspects of the controversial Rating System and FDM accuracy, etc..

Should be a Category that could represent the users experience - User's choice- Voted by users. From 0-5.

And what about to show a ranking orderer by aircraft flight time? Results can be used to choose aircraft of the month, year, decade, or all the times.

Aircraft Users feedbacks (likes), dynamic results, and some tracks of connected aircrafts - users, could be shown at the home page.

In my opinion web page should reflect what is happens in FG in real time , airspace, wiki, forum, position objects- web tools new contributions adds, general community users: pilots, developers, airlines, hangars, tracker....... should have that you call ¿modules in php? I mean those boxes with that dynamic info.

More click interactivity to the users, less exposing our best human resources to a troll-chat roulette.

Please accept my comments like laminar winds.
User avatar
Michat
 
Posts: 1226
Joined: Mon Jan 25, 2010 7:24 pm
Location: Spain
Version: 191b
OS: MX 21 Fluxbox oniMac

Re: Aircraft Rating System

Postby Thorsten » Wed May 28, 2014 8:28 am

In my opinion web page should reflect what is happens in FG in real time , airspace, wiki, forum, position objects- web tools new contributions adds, general community users: pilots, developers, airlines, hangars, tracker....... should have that you call ¿modules in php? I mean those boxes with that dynamic info.


We may all agree on that being a neat idea, but that still doesn't conjure up the manpower to do it. If you volunteer, just drop Curt a mail, and I'm sure he'll be happy to let you have a go.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Aircraft Rating System

Postby Hooray » Wed May 28, 2014 10:56 am

HJ1AN wrote in Tue May 27, 2014 6:37 am:Why does the Ogel "aircraft" have a full rating on the web site?


it's a fictional "aircraft" - besides, what else would you like to see improved :?:
And obviously the rating system was not conceived with fictional aircraft in mind.
Obviously, the bluebird is another example, but much more developed
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Aircraft Rating System

Postby hvengel » Wed May 28, 2014 5:27 pm

Michat wrote in Wed May 28, 2014 8:02 am:Apart of the technical aspects of the controversial Rating System and FDM accuracy, etc..

Should be a Category that could represent the users experience - User's choice- Voted by users. From 0-5.

And what about to show a ranking orderer by aircraft flight time? Results can be used to choose aircraft of the month, year, decade, or all the times.

Aircraft Users feedbacks (likes), dynamic results, and some tracks of connected aircrafts - users, could be shown at the home page.

In my opinion web page should reflect what is happens in FG in real time , airspace, wiki, forum, position objects- web tools new contributions adds, general community users: pilots, developers, airlines, hangars, tracker....... should have that you call ¿modules in php? I mean those boxes with that dynamic info.

More click interactivity to the users, less exposing our best human resources to a troll-chat roulette.

Please accept my comments like laminar winds.


On the surface a good idea but there are a number of very real issues with it that need to be considered. We have a very wide range of aircraft types and to be really useful this type of system would need to rank each aircraft against it's peers and not against all aircraft of every type. If I want to find the very best jet fighter then I would like to see jet fighters ranked as a peer group. Same thing for jet airliners, flying boats, air ships, modern aerobatic aircraft, WWII fighters, single engine GA... Otherwise the user based ranking is not particularly meaningful and some TYPES of aircraft are likely to dominate the ranking list not because the TYPE has more very good examples but for other reasons. For example, we seem to have lots of people who are into jet air liners (nothing wrong with that but this is not my thing) and there is a possibility that some kind of metric like total flight hours could end up being dominated by airliners because these tend to be used for longer flights (hours in duration) where as something like a modern aerobatic aircraft might typically be flown for 15 to 20 minutes at a time. So an extremely nice aerobatic aircraft that has a following and should show up in any short list of great aircraft could actually rank much lower than a mediocre jetliner because of the differences in how these types are typically operated, In addition I have read posts here where airliner pilots will go on auto pilot and let the aircraft basically fly itself for hours well doing things like eating lunch or taking a shower - should those hours count toward a rank and if not how would this be accounted for?

Also many pilots never or seldom use the multiplayer system. How would we account for this usage?

User ratings also tend to be very subjective. Users may like the way an aircraft with an unrealistic FDM flys and give it a high ranking but then turn around and give a low ranking to a similar aircraft that actually has a very accurate FDM. There are other issues with user supplied rankings. I have seen posts here where users complained about certain aircraft with known good FDMs being "totally un-flyable" where it turned out they they had totally messed up controller setups that were causing the problems. After sorting out the controller setup they found the "totally un-flyable" aircraft to be a very nice flyer. Had these users not fixed their controller issues would they have ranked that aircraft with an undeserved very low score? How often do users have issues with their controller setups that are undiagnosed? Some very advanced FDMs are difficult to impossible to fly if the user does not have a nearly complete controller setup. How often will these aircraft be given low rankings by users because they lack the necessary controller setup to really enjoy the aircraft (IE. I can't fly this thing with a mouse so I will rank it as a 0).

Over all the current rating system seems to be working at least for those aircraft where the dev is being an honest actor which I think is the case far more often than not. Aircraft devs have for the most part tried to use the system in a more or less objective way. Some have been perhaps a little too generous but most are very conservative and tend to error toward giving lower ratings. For example, when I look at the ratings for my aircraft if something (like the external model) is in between say a 3 and a 4 I will rate is as a 3.

In addition, the published rating criteria for each of the four areas is very objective in nature with each having a list of very specific criteria needed for each rating level that aircraft devs are able to easily interpret. Aircraft devs can look at the criteria and do a good rating job in perhaps 15 minutes because of their knowledge of the aircraft. In addition, the aircraft devs are, for the most part, the most qualified person to do this rating because they are generally by far the most knowledgeable person about that particular aircraft who is part of the FG community and they are well aware (way more so than the vast majority of users) how close it is to the real thing and what work remains to be done. The better aircraft are done by devs that have spend hundreds if not thousands of hours researching their aircraft and these folks are very aware of every flaw, every missing feature, every detail that is not perfect in their aircraft and they are generally very upfront about these issues. Other users, on the other hand, would likely need dozens and perhaps even hundreds of hours effort to do a good job of using the rating system on a higher end aircraft model because it would take that long to get and understand the documentation for the real aircraft and then run tests to verify that the model was to specifications.

The Ogel is a very unusual case as it is related to the rating system. It along with other fantasy/imaginary aircraft don't fit the rating system since it was not designed for rating these aircraft. Since there are so few of these it is probably at most a minor issue.
hvengel
Retired
 
Posts: 1127
Joined: Sun Dec 24, 2006 5:35 am
Location: Minden Nevada

Re: Aircraft Rating System

Postby Jabberwocky » Wed May 28, 2014 6:19 pm

Easy solution: Call the user rating "Fun Factor" and you get exactly that, how much fun have people with lfying that plane.
A rating by flight hours makes no sense. Long range haulers like me pile up hours like crazy, only because we fly from Phoenix to Munich for Weisswürschtl.
Jabberwocky
Retired
 
Posts: 1316
Joined: Sat Mar 22, 2014 8:36 pm
Callsign: JWOCKY
Version: 3.0.0
OS: Ubuntu 14.04

PreviousNext

Return to Aircraft

Who is online

Users browsing this forum: D-ECHO and 13 guests