Board index FlightGear Development Aircraft

Aircraft model/cockpit rating [see first post]

Questions and discussion about creating aircraft. Flight dynamics, 3d models, cockpits, systems, animation, textures.

Aircraft model/cockpit rating [see first post]

Postby Thorsten » Mon Nov 15, 2010 1:26 pm

In previous discussions the usefulness of having some aircraft rating system was pointed out. Rather than more discussion with no follow-up action, I plan to go ahead and introduce a partial rating system of the aircraft 3d model and start using the system to assign a rating to aircraft models in GIT. Obviously, this system is as useful as people agree with my visual preferences what a 'good' and a 'bad' cockpit is - I'm not defending those, nor am I arguing that the system is completely objective, nor does it represent an official Flightgear rating or have any official consequences for the aircraft status as shown on the website(if enough people find it useful, maybe it will eventually...).

The rating is based on the lower of exterior and interior. For all practical applications, that means it will be the cockpit, as the cockpit is 90% of the work of the modeller. This is also rather useful, as typically you see the cockpit most of the time when you fly around.

The rating is a number from 0 to 10, derived as follows. A base number between 0 and 4 is assigned by visual comparison with standard cockpit images and criteria. This number is multiplied with 2 and modified by +- 1 to account for small features which are better/worse than the standard. Finally, a +2 is assigned to truly outstanding cockpits.

The criteria and visual comparison cockpits to determine the base number are as follows:

0: doesn't have a visible model

1: has rendering errors, large empty spaces, is incomplete

Image

2: is 2d, is 3d but largely untextured, lacks details, appears flat

Image

3: is detailed, has some 3d effects

Image

4: uses photorealistic texturing

Image

(I think most people will agree that we're observing an increase in realism here and that it is possible to judge similarity to one of the above pictures, and that the whole procedure is not grossly unfair).

I have started on the aircraft list in GIT, and I'll begin a list of aircraft and model/cockpit ratings soonish - maybe some will find it useful.


Edit: The list so far (thumbnails linked to aircraft code):

----10 ----

f-14b,IAR80,sopwithCamel,MiG-15bis

---- 9 ----

A-10, ask13,Cub,ec135,lightning

---- 8 ----

aerostar700, alouette2,b1900d, bleriot-XI,CitationX,Crusader,dhc2W,long-ez,pa22-160,pa25-250-CIII,sea-vixen,ZLT-NT

---- 7 ----

787, A-6E, a4f, an2,bocian,Bravo,c150,c172p,C684,ch47,ch53e,Citation-II,DC-6B,Dragonfly, f16,f4u,K5Y1,katana,lionceau,OV10_NASA,MD11,pa28-161,R22,rallye-MS893,s76c,SenecaII,Submarine_Scout,ZivkoEdge540,747-400,deperdussin

---- 6 ----

777-200ER, A380, A6M2, alphajet, B-2, B-1B,ar-234-C,b29, beech99-yasim, bf109g,dhc6,dr400-120,ec130,ercoupe,f104,fokker100,fokker50,fw61,g109,g115,j7w2,l39,OH-1,p51d,PC-7,PC-9, stieglitz,vulcanb2,z50lx,Hansajet

---- 5 ----

ask21,b26,beaufighter, bo105,bugatti,c172r,c182,c310,c310u3a,dc3,dhc3W,f106,Fi-156,flash2a,fw190a8,geebee,harrier,He162,hm-14,ho-ix,hsp75,hunter,hurricaneIIb,ju52,Ki-84,m33,MD11,me209v1,pa100,PC-6,pc7,pterodactyl,s6b,seafireIIIc,seahawk, spitfireIIa,spitfireIX,stampe,stearman,T4-jsbsim,typhoon,v22,velocity,xb35,Concorde,Nordstern

===== 'alpha' status below =====

---- 4 ----

707, 727-230, 737, 737-100, 747, a24, a26, a4, apache, as332, avenger,b24,b25,br761,buccaneer,bv170,c119,c130,c160,ca12,dh88,dh89,dolphin,f15,f6f,F80C,fk9mk2,gloster-meteor,H-21C,HondaJet,hornet,lancaster,MB326,MC-15,mc72,me262,me262hg3,Mirage_F1,mosquito,p1101_NASA, p47d-30,p59,p61,pa18,pa32,pittss1c,pond,potez630,quickie2,s38,s58,Short_Empire,skyranger,sr71,starship,stirling, superwal,trimotor,uh1,vega,737-300

---- 3 ----

A300, A320, aircrane, Albatross,AllegroW,b17,beech99-v1-uiuc,c-2a,caravelle,cl415,couzinet70,d510,d520,do335,E3B,f183d,f7f,f-86f,gripen,h1,h4,HUP,hv220,i16,ju87, KC135,la5,lancair235,mirage2000,ms406,nf104a,P180-YASim,pt22,RafaleB17,ryan-sosl,s11,S-51,s55,sgs233,skyvan,spadvii,stiletto,T38,uh60,vg33,victor,xf11,fkdr3

---- 2 ----

Alouette-III, ant20,b36d,bell222x,breguet19,burnelli,bv141,dauphin,do26,dox,g38,il2,k7,ka50,Lynx,mi12,Mig-29, MRJ90-STD,nord2501,nord2502,pogo,SU-37,T37,tu154,YS-11,X15

---- 1 ----

767, AN-225,B-52F,BAC-TSR2,CanberraBI8,ComperSwift,dc2, dh91,F-117,j22,jaguar,km,L-1011-500,late631,SaabJ350eDraken,spaceshipone,sr20,superfrelon,tigre,TU-114,vmx22, YF-23

---- has no model ----

dhc8
NTPS

---- did not start (GIT 15.11.2010) ----
747-200
asw20
Aerocar
airwaveXtreme150
Blackbird-A
c182rg
Catalina
colditz
F-5B
jetman
marchetti
N2501
N2502
P-38L
wrightFlyer1903
Last edited by Thorsten on Tue Nov 30, 2010 10:26 am, edited 40 times in total.
Thorsten
 
Posts: 10089
Joined: Mon Nov 02, 2009 8:33 am

Re: Aircraft model/cockpit rating

Postby Upkeep » Mon Nov 15, 2010 1:39 pm

Personally I would find this very useful. I think it would be an excellent start. I agree with Thorsten's way of assessing the cockpits and I think it is the right way to go about having some sort of classification. Regards. Upkeep.
Upkeep
 
Posts: 286
Joined: Sun Nov 22, 2009 5:21 pm

Re: Aircraft model/cockpit rating

Postby rafa23189 » Mon Nov 15, 2010 5:15 pm

Yes! definitively a great idea! Rating encourages people to improve designs. Adding textures to cockpits (floor, seats, panel) is easy and makes it look much better.
rafa23189
 
Posts: 79
Joined: Tue Jan 26, 2010 10:51 pm

Re: Aircraft model/cockpit rating

Postby Algernon » Mon Nov 15, 2010 6:42 pm

rafa23189 wrote:Adding textures to cockpits (floor, seats, panel) is easy and makes it look much better.

Is it really?! I've found adding textures - especially photo textures - to a surface to be utterly baffling.

I'm not wild about the idea of rating aircraft on the model alone - while it's naturally important to an enjoyable flying experience, there are other factors which count, and are more important to some people, such as a good functioning autopilot for instance... so lets hope people remember looks aren't everything! :)
Algernon
FGUK - A FlightGear community in the United Kingdom and Republic of Ireland
User avatar
Algernon
 
Posts: 498
Joined: Sun Jun 27, 2010 3:55 pm
Callsign: G-ALGY
Version: 3.0
OS: W7U

Re: Aircraft model/cockpit rating

Postby Hooray » Mon Nov 15, 2010 7:25 pm

Algernon wrote:I'm not wild about the idea of rating aircraft on the model alone - while it's naturally important to an enjoyable flying experience, there are other factors which count, and are more important to some people, such as a good functioning autopilot for instance... so lets hope people remember looks aren't everything! :)


There is no reason why you should not be able to also do this for all sorts of other aspects (exerior, fdm, effects, sounds, systems, failures).
I think this is certainly a good start.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11263
Joined: Tue Mar 25, 2008 8:40 am

Re: Aircraft model/cockpit rating

Postby Buckaroo » Mon Nov 15, 2010 8:33 pm

Algernon wrote:
rafa23189 wrote:Adding textures to cockpits (floor, seats, panel) is easy and makes it look much better.

Is it really?! I've found adding textures - especially photo textures - to a surface to be utterly baffling.


It is not easy. It takes a signficant level of knowledge and study of the aircraft, skill with 3D and 2D tools, a fair amount of artistic talent, and considerable time. To do it well it is definitely not easy.

I dislike these ratings schemes, especially by people with no hand in the development of the aircraft in question, but my position is already stated in the mentioned previous discussion (page 4).

-Buck
Callsign: Buckaro(o)
Author: Lockheed 1049H Constellation, Grumman Goose, MD-81, Edgley Optica, Velocity XL RG, YASim Guide
User avatar
Buckaroo
 
Posts: 475
Joined: Fri Jan 18, 2008 6:45 am
Location: Bloomington IN USA
Callsign: Buckaro(o)
Version: 2.10
OS: Windows & Linux

Re: Aircraft model/cockpit rating

Postby Gijs » Mon Nov 15, 2010 8:49 pm

rafa23189 wrote:Rating encourages people to improve designs.

Hm, it won't encourage me if I was given a low score after hours of work...
Unfinished aircraft are mostly not a matter of "needs en encouragement", but a matter of "needs extra spare time/man power".

Rather than a "rating" I would call this a "status indication". That way, you won't downgrade somebody's work, you will just say it ain't (close to) finished yet. But before we turn this thread in a copy of the previous thread I would like to see some more examples by Thorsten. I think we can say FlightGear's spirit is "don't dream, just do (it yourself)" after all ;)
Airports: EHAM, EHLE, KSFO
Aircraft: 747-400
User avatar
Gijs
Moderator
 
Posts: 9306
Joined: Tue Jul 03, 2007 2:55 pm
Location: Amsterdam/Delft, the Netherlands
Callsign: PH-GYS
Version: Git
OS: Windows 10

Re: Aircraft model/cockpit rating

Postby Algernon » Mon Nov 15, 2010 9:25 pm

Hm, it won't encourage me if I was given a low score after hours of work...
Unfinished aircraft are mostly not a matter of "needs en encouragement", but a matter of "needs extra spare time/man power".

Nicely put, Gijs. That is my main concern - there are three of us working on the Eurofighter at the moment, and whilst some of the ongoing work is cosmetic, most of the hard work required to model a cutting edge fighter jet is not. Furthermore, it's nowhere near finished, and I for one would find it pretty discouraging if people started giving the aircraft low ratings whilst it's still very much a work in progress, or based on cosmetic factors when in fact there is a great deal of work going on beneath the surface.
Algernon
FGUK - A FlightGear community in the United Kingdom and Republic of Ireland
User avatar
Algernon
 
Posts: 498
Joined: Sun Jun 27, 2010 3:55 pm
Callsign: G-ALGY
Version: 3.0
OS: W7U

Re: Aircraft model/cockpit rating

Postby Upkeep » Mon Nov 15, 2010 10:13 pm

I agree with Algernon's comments. I really like Gijs's idea of having a Status, which implies more a stepping stone. I think this is the way to go from my point of view. Rgds Upkeep.
Upkeep
 
Posts: 286
Joined: Sun Nov 22, 2009 5:21 pm

Re: Aircraft model/cockpit rating

Postby El Flauta » Tue Nov 16, 2010 4:27 am

Remember that this post refferst about cockpit development, not aircraft development.

The think that you calls just "cosmetic" is for some people -like me- one of the important factors to feel like inside a true plane. If you was sometime inside to a real aircraft cockpit, you will understand me.

In the other side, is quite sad to see -less than two months before 2011- graphics which was well for the early 90's on some planes. Why make just half things, if we can to it really good? :wink:
Vive FlightGear! Have you a Ñ on your keyboard? Spain-LatinAmerica FlightGear community!
--
PZL M18B Dromader
CASA C-101 Aviojet
Cessna 337G Skymaster
User avatar
El Flauta
 
Posts: 429
Joined: Fri Mar 14, 2008 12:09 am
Location: SCVM, Chile
Callsign: CC-FLT
Version: 3
OS: Windows 7 SP1

Re: Aircraft model/cockpit rating

Postby Thorsten » Tue Nov 16, 2010 8:22 am

I'm not wild about the idea of rating aircraft on the model alone - while it's naturally important to an enjoyable flying experience, there are other factors which count, and are more important to some people, such as a good functioning autopilot for instance... so lets hope people remember looks aren't everything!


Neither am I - which is why I called the exercise 'a partial rating system'. I have the full scheme taking into account FDM (0-10), systems (0-10) and extras (0-5) ready, but obviously while I can judge the model visual quality within 30 seconds, experimenting with the systems and judging AP costs me maybe 20 minutes and doing benchmark testing with the FDM costs some research time and probably more than 3 hours of flying. On the other hand, an aircraft is not 'ready' if the cockpit is incomplete, regardless of the value of the FDM. So while I plan to extend the rating scheme beyond the model quality, I will do this only for the aircraft which score 5 or more for the model, and I will try to get some help.

I dislike these ratings schemes, especially by people with no hand in the development of the aircraft in question


That's precisely the point - a rating by the people who had a hand in the development is hardly objective.

Hm, it won't encourage me if I was given a low score after hours of work...


That's not what it's primarily for. It's for users which face 300+ aircraft to download and want some more info.

Rather than a "rating" I would call this a "status indication".


I cannot judge the status of the development, because I have no way of knowing if an aircraft is actively developed or not, short of trying to contact each and every developer. It's up to the developer to make known what he thinks the status of the aircraft development is.

In the end it is a number expressing the quality of visual detail. It doesn't really matter to me how you call that number - so pick any name you like.

Nicely put, Gijs. That is my main concern - there are three of us working on the Eurofighter at the moment, and whilst some of the ongoing work is cosmetic, most of the hard work required to model a cutting edge fighter jet is not. Furthermore, it's nowhere near finished, and I for one would find it pretty discouraging if people started giving the aircraft low ratings whilst it's still very much a work in progress, or based on cosmetic factors when in fact there is a great deal of work going on beneath the surface.


If it is work in progress, and I happen to observe work in progress, then there should be nothing discouraging about me (or anyone else) saying it is not ready. I forsee tensions rather in the case you think it's ready and I think it's not. Well - can't please everyone.

In my view, having an imperfect scheme is better than discussing forever without a consensus what the perfect scheme should be. Of course you may prefer none at all - but does that so much harm to you that those of us who would like to see some scheme implemented should refrain?
Thorsten
 
Posts: 10089
Joined: Mon Nov 02, 2009 8:33 am

Re: Aircraft model/cockpit rating [see first post]

Postby i4dnf » Wed Nov 17, 2010 9:01 am

That's a huge load of work, Thorsten, but maybe long overdue. I too think it will be useful in the long run. It's always a good exercise to see how others perceive your work, as I know that after a while one can get so personal with their creation that one thinks it's the best thing since hot water ;).
Even better if you could gather a "team" that does this, so that the results come a bit faster.


Suggestion: Couldn't this be moved to the cockpit subforum and made a sticky?
i4dnf
Retired
 
Posts: 745
Joined: Wed Sep 09, 2009 7:17 am
Location: LRBS
Callsign: YR-I4D
Version: GIT
OS: Gentoo Linux ~amd64

Re: Aircraft model/cockpit rating

Postby Algernon » Wed Nov 17, 2010 10:16 am

Thorsten wrote:If it is work in progress, and I happen to observe work in progress, then there should be nothing discouraging about me (or anyone else) saying it is not ready. I forsee tensions rather in the case you think it's ready and I think it's not. Well - can't please everyone.

Well, you can't argue with that! :) I'll be interested to see how this scheme works out, as I do agree there needs to be some system to provide good standards benchmarks.
Algernon
FGUK - A FlightGear community in the United Kingdom and Republic of Ireland
User avatar
Algernon
 
Posts: 498
Joined: Sun Jun 27, 2010 3:55 pm
Callsign: G-ALGY
Version: 3.0
OS: W7U

Re: Aircraft model/cockpit rating [see first post]

Postby Hooray » Wed Nov 17, 2010 10:42 am

All aircraft in FlightGear have obviously VERY different strengths and weaknesses. And obviously different developers and users care for different things.
So the only right thing to do is to start classifying aircraft according to what's important TO YOU.

Many people will find the cockpit very important, many will agree that the FDM is very important, too - others may say that exterior visuals and effects are very important. Basically, you will not find many people that agree completely on all factors.

So whenever anybody feels that his/her work is not being judged fairly, because some "important areas" are getting neglected, that person should feel invited to come up with a rating scheme to highlight the strengths in other areas.

If on the other hand people want to replace/refine an existing rating system with something "better", they should obviously also be willing to do the necessary work (i.e. provide patches) so that all aircraft using the previous system can be ported accordingly.

If all people did that, we would quickly end up with people proposing all sorts of schemes for:
* cockpits
* FDM
* visuals/effects
* system modeling
etc

There are already aircraft (like e.g. the bluebird) which internally use a very similar rating scheme: http://gitorious.org/fg/fgdata/blobs/ma ... d-base.xml
Code: Select all
<sim>
   <aero>bluebird</aero>
   <status>early-production</status>
   <aircraft-version>10.4</aircraft-version>
   <author>S Andreason</author>
   <aircraft-status>
      <fdm>beta</fdm>
      <exterior>early-production</exterior>
      <cockpit>early-production</cockpit>
      <instrumentation>early-production</instrumentation>
      <procedures>alpha</procedures>
      <system-modeling>beta</system-modeling>
      <failure-modeling>beta</failure-modeling>
      <checklist-support>alpha</checklist-support>
      <tutorial-support>alpha</tutorial-support>
      <documentation>alpha</documentation>
      <usability>early-production</usability> 
      <eye-candy>production</eye-candy>
   </aircraft-status>
   <type-of-aircraft>
      <is-futuristic type="bool">true</is-futuristic>
      <is-hovercraft type="bool">true</is-hovercraft>
      <is-ufo type="bool">true</is-ufo>
   </type-of-aircraft>


This is MUCH more informative and useful than the existing scheme already, and it could be processed easily by GUI tools like fgrun.

From an implementation point of view it would however seem to make sense to use the existing XML scheme, so that these ratings can be maintained in git and also be processed by software tools (such as fgrun) easily:

viewtopic.php?f=4&t=8441&p=87039&hilit=meta#p85688
http://wiki.flightgear.org/index.php/Fo ... aft_Status
http://wiki.flightgear.org/index.php/Si ... Deployment
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11263
Joined: Tue Mar 25, 2008 8:40 am

Re: Aircraft model/cockpit rating [see first post]

Postby Thorsten » Wed Nov 17, 2010 11:49 am

Let me comment a bit more on some random points:

First, it seems to me there is a fundamental (and unfair) mismatch between what a developer wants and what a user perspective (from which ratings are done) does. A developer usually wants some appreciation for hard work. A user wants a finished product which looks and feels great, and if it does suit his fancy, he expresses appreciation. Here's the problem: I spent the better part of 5 months coding work to make something disappear from the weather system. Now it's largely gone and doesn't bother me any more - but do you really think that any future user is ever going to express his gratitude that it's not there? Of course not - he'll never notice, which is just the point. He'll just notice that cloud texture X looks spectacular, which is nice, but that's 5 hours of work instead of 5 months. Also with cockpit design - it's obviously much easier to create a 'wow!' effect when you have a glider cockpit with 5 instruments, rather than the Concorde with 200. But a user simply isn't interested in the time it took to get something going - if I can't start a plane because it creates an error, I don't appreciate the hours gone into that plane. If someone spent 100 hours to get a nice cockpit, and the guy next door spent 1000 hours to get a great cockpit for the same plane, I will use the great cockpit and not appreciate 100 hours's work. From the perspective of a developer, that's not fair, but that's how it is, and once we can accept that users think that way, we can move on.

Second, although there is a danger that rating airplanes by visual appearance neglects work done for the FDM, I find that there's often a correlation. Planes with realistic systems (say, a real engine startup procedure) need buttons and gauges - so immediately the cockpit design is an issue, and all planes with good systems I know rate at least 5 on the visual scale, usually 7+. The connection with the FDM is less obvious, but I guess anyone who bothers to model a cockpit down to checklist pictures and wear and tear on the floor also spends enough time on the FDM to get at least a decent result. The planes with very high quality FDMs I know are not always the ones with highest quality visuals - but in almost all cases high quality FDM planes have devent cockpits and high quality model planes have devent FDMs. So, focusing on better-than-average visuals actually does seem to do a reasonable job in tagging also better-than-average FDMs.

Third, even with the number of planes processed so far, one benefit came readily to me - I have discovered some high-quality work which I would have simply missed otherwise because I don't know the real-world airplane and there's just nothing else around to single them out . So that's what I hope the rated list ultimately does - inform users that there are really great aircraft which usually do not feature very prominently anywhere to have a look at.
Thorsten
 
Posts: 10089
Joined: Mon Nov 02, 2009 8:33 am

Next

Return to Aircraft

Who is online

Users browsing this forum: wrg and 12 guests