Board index FlightGear Development Aircraft

Aircraft Rating System

Questions and discussion about creating aircraft. Flight dynamics, 3d models, cockpits, systems, animation, textures.

Re: Aircraft Rating System

Postby HHS » Fri Oct 07, 2011 11:12 am

@turtle:
File size is good to know, but with increasing internet speed and cheaper disk space, *it looses priority for those who doesn't use GIT, but just the official Release. But file size doesn't say which parts of an aircraft is well done, we have older models with large size, but poor modeling, fdm and systems.

Like Gijs said, showing file size is more a matter of the aircraft download page, not the rating system itself. A good php-programmer should be able to make it happen easily.

The aircraft 2.4. page has been updated, but many aircraft still show an old version though being updated several times since the last time- so here you can see the disadvantage of the old system. Some developer simply forgot to update the version number. And of course, some aircraft hasn't been updated since a whole time.

The -set.xml-file is the heart of all aircraft models. An aircraft would never work without this file, and contains a whole lot informations which the sim needs to work. It contains all paths to the 3d-model, path to the system files, path to the fdm, nasal scripts etc...
And it contains also informations like authors name, licence, which fdm etc. FGRun and other launchers reads this file, and so does the script which automatically builds the Aircraft Download page whenever we have a final release.

So it makes sense to include the rating of an aircraft in the -set.xml-file.


You must be a new user not yet familiar with the concepts of FGFS and as I can see on your requests and questions you didn't read the discussions related to this topic.

I advise you to read this: viewtopic.php?f=4&t=9680&hilit=Rating
It explains very good why we have decided for a new rating system. That does not mean that other reviews are useless. Thorsten R. has an own system, which he uses and from time to time he publish a new review, and they are informative as well and very helpfull.

*EDIT: I have added a part: File size isn't that important who uses a official release and just downloads the aircraft he is interested in.
But it looks different for those who uses the developers version (Known as "GIT"), who have to download about 4GB datas and every single aircraft in there.
Just to make it clear!
Up, up and away
User avatar
HHS
Retired
 
Posts: 3624
Joined: Thu Jul 19, 2007 8:09 am
Version: GIT

Re: Aircraft Rating System

Postby Gijs » Fri Oct 07, 2011 4:09 pm

turtle wrote in Fri Oct 07, 2011 8:22 am:
Gijs wrote in Fri Oct 07, 2011 6:00 am:Luckily one can get the filesize automatically with PHP, it some other language. No need to add that kind of info to -set.xml

No excuse for laziness when it's a free. :( If you guys don't want to do it, then let me or another volunteer do it.

I think you misunderstood my post. I agree we should show the filesize on the downloadpage (just look at what I did for the livery database, where I actually have control over, as oposed to the website), but we don't need to write that down in the -set.xml file. The download-page can retrieve the filesizes itself, all automatically.

Why is the aircraft information coming from the -set.xml file? Seems like a lot less work if developers would just send you guys (in charge of hosting the website) the information

That is exactly what the current system does: developers add the info the their aircraft's -set.xml file. And that info is then copied to the website.
Airports: EHAM, EHLE, KSFO
Aircraft: 747-400
User avatar
Gijs
Moderator
 
Posts: 9377
Joined: Tue Jul 03, 2007 2:55 pm
Location: Amsterdam/Delft, the Netherlands
Callsign: PH-GYS
Version: Git
OS: Windows 10

Re: Aircraft Rating System

Postby Hooray » Fri Oct 07, 2011 7:39 pm

turtle wrote in Fri Oct 07, 2011 5:00 am:
HHS wrote in Tue Oct 04, 2011 4:17 pm:@turtle:

As Stuart said on the first page of this thread we have already a Aircraft Rating System: http://wiki.flightgear.org/Formalizing_Aircraft_Status
We had a very long discussion about how the new system will look, and how it will be used. File size was one possibility we considered but we dumped it for good reasons, as it doesn't give the inputs the user wish.

What?!? :shock:
How is not knowing how long it is going to take you to download a file before clicking on it useful? I honestly cannot comprehend you guys actually agreeing otherwise. The first thing people ask themselves before they download something is, "how long is it going to take?" This is just one of the uses why adding File Size: as part of the requirement for aircraft would be beneficial.


You can add arbitrary information to each *-set.xml file, however the time required to download a file will obviously be a matter of your connection speed, so it would probably be more useful to simply introduce "categories":
A: < 10 MB
B: < 15 MB
C: < 20 MB

You can add this info easily to each aircraft, but like Gijs said already, it makes more sense to simply use php and show the size automatically, so there would be no need to maintain the info inside the set.xml file.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11335
Joined: Tue Mar 25, 2008 8:40 am

Re: Aircraft Rating System

Postby El Flauta » Sat Oct 08, 2011 12:22 am

An incomplete aircraft with 20 liveries could have a bigger filesize than any aircraft with just one livery, but fully simulated with tons of Nasal or xml. File size, personally, doesn't have a true relation with quality.
Vive FlightGear! Have you a Ñ on your keyboard? Spain-LatinAmerica FlightGear community!
--
PZL M18B Dromader
CASA C-101 Aviojet
Cessna 337G Skymaster
User avatar
El Flauta
 
Posts: 429
Joined: Fri Mar 14, 2008 12:09 am
Location: SCVM, Chile
Callsign: CC-FLT
Version: 3
OS: Windows 7 SP1

Re: Aircraft Rating System

Postby hvengel » Sat Oct 08, 2011 7:58 am

El Flauta wrote in Sat Oct 08, 2011 12:22 am:An incomplete aircraft with 20 liveries could have a bigger filesize than any aircraft with just one livery, but fully simulated with tons of Nasal or xml. File size, personally, doesn't have a true relation with quality.


In fact Nasal and XML files tend to be very small compared to things like textures (like those used for liveries) and an aircraft with many liveries but very incomplete FDM and systems would have a large download size in spite of it's poor overall quality. Even a very complex aircraft with extensive XML and Nasal will have perhaps 20 to 40 K of of Nasal and XML files and the size of the Nasal and XML files is probably a good indicator of aircraft quality. Overall file size is at best a very rough indicator of aircraft quality. As another example of things can have a large affect file size that are not a direct indicator of aircraft quality I thought about including an actual (public domain) pilots manual with the P-51D and this would have added about 54 meg to the download. This manual would be useful to virtual pilots but is not a significant quality enhancement. But I decided to not do this because of concerns of download bandwidth. Someone could include a similar manual with a very incomplete aircraft and have a large download size.
hvengel
Retired
 
Posts: 1128
Joined: Sun Dec 24, 2006 4:35 am
Location: Minden Nevada

Re: Aircraft Rating System

Postby turtle » Sun Oct 09, 2011 7:42 am

HHS wrote in Fri Oct 07, 2011 11:12 am:You must be a new user not yet familiar with the concepts of FGFS and as I can see on your requests and questions you didn't read the discussions related to this topic.

I advise you to read this: http://www.flightgear.org/forums/viewto ... lit=Rating
It explains very good why we have decided for a new rating system. That does not mean that other reviews are useless. Thorsten R. has an own system, which he uses and from time to time he publish a new review, and they are informative as well and very helpfull.

I am not new to FGFS, so not sure why you would make that assumption. The question in regards to the set.xml file was because I did not know why it was chosen for the aircraft download page (I imagined it was for aircraft information). I have been on and off FG since 1.9. I recently became a regular user since 2.4 does not crash 80% of the time like the previous versions did on my PC (Linux and Windows). The only thing I can say I am a new to, is what exactly is inside each files for the aircraft (model not included). The rest I have at least looked over while tweaking the file permissions.

In regards to the file size, I think don't see how the results would not be more beneficial than the proposed rating system on this thread. Some of you are arguing that the plane could be have a ton of liveries but the actually plane's behavior and model is very inaccurate. However, having personally downloaded from 50 to 75 (I lost count after 40) aircraft I can say this assumption has not yet showed up for me. Even if the information on the aircraft download page is out-dated, the download still implies that bigger file size aircraft are more developed than those who have a small file size. I will agree with the statement that the file size is not giving people information on what is good in the aircraft or not. But how would we the users who are download know this? Had it not been for someone making up a list rating the aircraft by the system, I would have not know this actually existed. In other words, is there going to be an easier way to rate the aircraft than someone making up a list once in a while? I don't want to talk about GIT because my focus is on helping the basic or new users (most experience we have with aircraft is flying them in FG), and GIT as I understand it is more for developers and advanced users.
User avatar
turtle
 
Posts: 124
Joined: Wed Jun 22, 2011 10:41 pm

Re: Aircraft Rating System

Postby HHS » Sun Oct 09, 2011 10:41 am

turtle wrote in Sun Oct 09, 2011 7:42 am:I am not new to FGFS, so not sure why you would make that assumption.
...
The only thing I can say I am a new to, is what exactly is inside each files for the aircraft (model not included). The rest I have at least looked over while tweaking the file permissions.


:wink:

Well, there is a difference between bloody user it seems you had been and a developer who makes use of FlightGear's achitecture and concepts to create Addons like aircraft and does understand the meaning of the different files.

In regards to the file size, I think don't see how the results would not be more beneficial than the proposed rating system on this thread. Some of you are arguing that the plane could be have a ton of liveries but the actually plane's behavior and model is very inaccurate. However, having personally downloaded from 50 to 75 (I lost count after 40) aircraft I can say this assumption has not yet showed up for me. Even if the information on the aircraft download page is out-dated, the download still implies that bigger file size aircraft are more developed than those who have a small file size. I will agree with the statement that the file size is not giving people information on what is good in the aircraft or not. But how would we the users who are download know this? Had it not been for someone making up a list rating the aircraft by the system, I would have not know this actually existed. In other words, is there going to be an easier way to rate the aircraft than someone making up a list once in a while? I don't want to talk about GIT because my focus is on helping the basic or new users (most experience we have with aircraft is flying them in FG), and GIT as I understand it is more for developers and advanced users.


50 -75 of about 300+ aircraft? Not really representative. Again: I advise you to read the mentioned link!! It was a long but good discussion and I and others here are convinced that this system is pretty good already.

Bigger file size doesn't mean that it is more developed!
Example needed?

The Caravelle: 18.8 Mb unpacked, just three instruments (at least you need six to fly properly), a basic 3d-exterior model, a basic fdm far away from being realistic, but 7 liveries, 6 of them are high-resolution.

The 777-200ER: 13.4 Mb unpacked, most advanced airliner with a complete flightdeck (Panel, Overhead, centerpedestal, simple flightbag), detailed exterior model, just two liveries, advanced systems simulations like TCAS, Flight Guidance system, Brakesystem, a more advanced fdm etc...

The 737-300: 27MB unpacked; 4 liveries, realistic fdm, but flightdeck isn't completed, no advanced systems modeling, just a simple basic Autopilot...

There are many more examples like that !!

In other words, is there going to be an easier way to rate the aircraft than someone making up a list once in a while?

Hä?

The rating system is a tool given to the Aircraft authors, but it of course users can rate the aircraft they want themselves and publish the resultat.
The Aircraft Author or voluntary user uses this tool, rates the work, and publish it maybe on the wiki aircraft page of their work or/and in the -set.xm-file. There is no easier way than that.

The only things which is still needed, is that the Aircraft Download Page makes use of this system, shows the ratings and allows to filter. This means some php-programming.
Up, up and away
User avatar
HHS
Retired
 
Posts: 3624
Joined: Thu Jul 19, 2007 8:09 am
Version: GIT

Re: Aircraft Rating System

Postby fredb » Fri Nov 11, 2011 7:29 pm

Soon on your screen :

image removed

Edit: obsolete screenshot removed. See below
Last edited by fredb on Sat Nov 26, 2011 10:01 am, edited 1 time in total.
User avatar
fredb
 
Posts: 756
Joined: Fri Dec 01, 2006 10:41 am
Location: Paris, France

Re: Aircraft Rating System

Postby Sealbhach » Fri Nov 11, 2011 8:15 pm

This is great, much more information there. For me, I'm always keen to know if a plane has a realistic flight model, but there's no way of knowing at present. Hopefully that is what is meant by "modéle de vol" there.

.
Sealbhach
 
Posts: 942
Joined: Wed Jun 30, 2010 9:17 am

Re: Aircraft Rating System

Postby hvengel » Thu Nov 24, 2011 10:39 pm

Sealbhach wrote in Fri Nov 11, 2011 8:15 pm:This is great, much more information there. For me, I'm always keen to know if a plane has a realistic flight model, but there's no way of knowing at present. Hopefully that is what is meant by "modéle de vol" there.

.


Yes that is correct "modéle de vol" is the FDM rating.

The new rating system includes a rating for FDM, systems, cockpit and external model quality. These are 0 to 5 for each category and the FG wiki page that describes these rating has fairly detailed descriptions of what a 5 (or 0 or 1 or 2 ...) rating in each category means. So you can look there to see what these should indicate about the model.

By the way 5's are extremely difficult to achieve in any category so even aircraft with mostly 4's is a very nicely done model that is the result of a huge amount of work by it's creator(s).

It is good to see this feature coming to fgrun. Now we need to get this info exposed/used on the download page.
hvengel
Retired
 
Posts: 1128
Joined: Sun Dec 24, 2006 4:35 am
Location: Minden Nevada

Re: Aircraft Rating System

Postby fredb » Fri Nov 25, 2011 7:17 am

The rating display has been changed to show stars instead of bars :

Image
User avatar
fredb
 
Posts: 756
Joined: Fri Dec 01, 2006 10:41 am
Location: Paris, France

Re: Aircraft Rating System

Postby helijah » Tue Dec 13, 2011 7:10 am

Hey all

At this time (13-12-2011) there are, in my hangar, with aircraft rating system :

Aichi D3A
Airspeed Horsa
Ask 21 Mi
B25 Mitchell
Bleriot 5191
BV 170
Cessna 208 Caravan
Do 335
Do X
Douglas Dc 3
Gloster Meteor
Gloster E28/39 "Whittle"
Grob 102
Lancair 235
Late 29
Lionceau APM 20
M.B. 5
Me 262
Piper Pa 32
I 16
R 44
P 92
Tu 95
Zlin 50

Regards Emmanuel
Some planes (and other) for FlightGear
http://helijah.free.fr
and
http://embaranger.free.fr
User avatar
helijah
 
Posts: 1038
Joined: Wed Dec 27, 2006 12:35 pm
Location: Chartres (France)
Callsign: helijah
IRC name: helijah
Version: GIT
OS: GNU/Linux

Re: Aircraft Rating System

Postby Pierre.Mueller » Tue Dec 13, 2011 11:13 am

As I'm just a stupid user, I'm happy to see such an improvement like the rating system.
In the past days I went though the list of those aircraft in the FGFS-repo which has been rated already. Not ready yet, I can see some things now.

I noticed that some aircraft are higher rated as they should be. Probably due to the system and the guidelines not being really sufficient and clear enough.
Most of the times it seems to work, but I found until now three aircraft, which should have less stars.

One is the WWII Fighter Martin Baker MB5. With a bit use of Google you can find a lot of informations about the MB5.

Exterior model rating with 4 stars and cockpit rating with 2 stars seems right to me.
But systems rating seems wrong. According to this, the fuel tanks has the wrong positions and volumes. But it has a simple startup procedure.
I'm not sure about giving 0 or 1 star here?

The fdm has a rating of 3 stars- so I would expect a "FDM tuned for rate of climb and cruise Pilot Operating Handbook (PoH) performance numbers"
What I found says that the FlightGear's version is much, much too powerful!
I got more than 5000-8000fpm as climbrate- even all wrong fuel tanks filled up and at hot weather at high altitudes and at a speed between 150-200ktn.

But Maximum rate at 7000ft should be around 4000fpm.

Top speed at 20.000ft should be 460mph or 403ktn True AirSpeed.
With the FGFS-version i got more than 460ktn easily!

So I would give just 1 star for the fdm.

At the end Stars I would give:
1
2
1
4

8 stars= alpha

Sources:
http://johnmarlinsmb5replica.mysite.com/index_1.html
http://airplanesandrockets.com/airplanes/martin-baker-mb5-1971-aam.htm
http://www.martin-baker.com/Sub-Navigation/History/Martn-Baker-MB5.aspx
http://forums.x-plane.org/index.php?automodule=downloads&showfile=14943

The other is the SenecaII.
Here as well Google is my friend, and the climb rates and speeds mentioned seems to match pretty well. 5 stars from me as well.
Systems seems to be 99% complete, but when entering the aircraft I hear a sound: the gyro. On a cold and dark aircraft I woulden't expect it. The cockpit seems to be one of the finest in FlightGear- just two dummy instruments on the right side of the panel, otherwise all instruments, knobs, switches, avionics available and fully working.
5 and 4 Stars are o.k. (not entirely sure on the systems rating due to the gyro sound)

But now we come to the exterior model. Not very attracting. :(
The whole fuselage is edgy, while the original has smooth outlines. Window glases are hardly noticeable. At least the shape shows more or less that it is a Piper Seneca. Just a very simple livery.
But the system doesn't say anything about how "accuracy" is defined. And not about the quality of the 3d-models. So I tend to give 3 stars as well, but with some headaches.

So I would give:
5
4 or 5?
4
3 (here I'm uncertain)

16-17stars = production

Sources:
http://www.fargojet.com/school/view_aircraft.php?id=9

The other one is the R44. Here luckily I have access to the PoH and in the past I had some discussions about the R22/R44 with their pilots.
We begin with the exterior. From the side it does look like the real one. But from the front the profil looks like blown up. Was here a wide angle photo used as reference? And there is a second upper window on the pilot side- it shouldn't be there.

Again the same question: how is accuracy defined? With this it doesn't look accurate enough to me for 4 stars.

But the main rotor and tail rotor rods has been modeled, so it could be maybe still a candidate of 4 stars or even more?- but the animations of it does look not right- the swashplate of the main rotor is wobbling when rotor is turning, but should be only moving on pilot inputs. The rods on the tail rotor aren't animated.
So 4 stars with a big interrogation mark, but tending to 3

The cockpit otherwise does look really nice. Not difficult as the real R44 does not have much instruments. Everything is there, obviously taken from the R22. Only photorealistic textures are missing, so 4 stars from me.

About the systems: Startup is quite correct, but I don't see any "generic failure modelling (Vne, +ve/-ve G, gear limits)". And, even worse, while testing the fdm, I noticed that there had been a so called CAS/SAS-system added. A eletronic helper which you only find on bigger commercial helicopters to stabilize the helicopter a bit- but not on the R44! To get 3 stars there shouldn't be any unrealistic systems, but with the missing limits and the unrealistic CAS/SAS, i can only give 2 stars! :(

Now we come to the fdm... :|
Ouch! Sorry, but the fdm is completly wrong. As an american heli, it has a counterclockwise rotor, and so the heli should rotate clockwise. (actio - reactio).
But when I lift the collective, the heli begins to spin counterclockwise, I have to step into the right pedal. That's not correct!

Maybe just a bug, I went on. The R44 is known as quite sensitive to inputs, a bit less maybe than her smaller sister R22, but still enough for needing an extra Awareness training in RL. Against FlightGear's version of it is really simple too fly. Just keep the stick centered, and the heli hovers like a flying carpet. Even without SAS there aren't many inputs needed to keep the heli stable. It reacts really slow on inputs, feels sluggish.
Again, a wrong behaviour.

Climb rates seems to less for me, only speeds seems more or less right. While adjusting Load and balance, I noticed that the FGFS R44 has 3 fuel tanks. The virtual engine doesn't need to consume any fuel, again that's wrong. The real R44 has only one main fuel tank, and one auxiliary fuel tank.

I take a look into the R44-yasim.xml-file. Many numbers and values I don't understand, but there are some I do. Like the rotor rpm- too high, as the R44 has a 100% rpm of about 400. (and not 495 as in the file). Or the weight of the main blades. It seems much to high for me, it has the same value than the bigger BO105.

So I'm not sure if it uses a fdm of an other aircraft.
So I would give only 0-1 stars.

My resultat for the R44:
0-1
2
4
4? But I tend to 3

9- 11stars = beta

At least, the R44 has some nice sounds, I guess they are realistic and probably recorded from the real thing. And with this, we come a big missing thing on this rating system: what's with the sound?

Overall Conclusion:
So I think, the rating system is already quite good, but maybe needs some more exact definitions.

When can we speak of accuracy?
What is, when a one of two needed things for a rating is not true?
And what's about sound? Sound is important as well!

Large posting, especially as it is my very first posting here.
Maybe the authors of the mentioned aircraft here won't like it. But I do think a good rating system and correctly used can help users to find their aircraft and aircraft authors to improve their aircraft.

Kind regards

Pierre Mueller
Pierre.Mueller
 
Posts: 33
Joined: Wed Nov 23, 2011 10:06 pm

Re: Aircraft Rating System

Postby Hooray » Tue Dec 13, 2011 11:27 am

WELCOME PIERRE (getting educated by Gijs!)

obviously, different people have different backgrounds and expectations - so while the existing rating system may not suit everybody's needs, it is indeed a very good start. If you disagree with a rating, or if you'd like to create your own review (or even rating system!), then that's just fine.

And we do appreciate your feedback. But the truth is, creating and establishing the "new" rating system has been long overdue and still is work in progress. It really isn't yet widely adopted or supported.


Some people have thus created their own aircraft reviews here and posted them to the forum, the wiki or the FG main website - given the detail of your reviews, I would seriously suggest to also post your reviews in the wiki.

The new rating system isn't perfect, but it is far better than what we used to have.
If you have something to share (bug reports, feature requests), it is always a good idea to get in touch with the developers/maintainers of the aircraft, that can usually be easily done using the issue tracker: http://flightgear-bugs.googlecode.com/
Last edited by Hooray on Tue Dec 13, 2011 11:40 am, edited 2 times in total.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11335
Joined: Tue Mar 25, 2008 8:40 am

Re: Aircraft Rating System

Postby Gijs » Tue Dec 13, 2011 11:38 am

Welcome Pierre! Also on behalf of Hooray ;)

I would advise you to contact the aircraft' authors and share your doubts about their ratings. I've just added a "issue type" to our issue tracker: Type-StatusRating. Feel free to create an entry for each of the aircraft you disagree with. Generally developers are very happy to receive feedback, especially when supported by good/extensive arguments.

Cheers,
Gijs
Airports: EHAM, EHLE, KSFO
Aircraft: 747-400
User avatar
Gijs
Moderator
 
Posts: 9377
Joined: Tue Jul 03, 2007 2:55 pm
Location: Amsterdam/Delft, the Netherlands
Callsign: PH-GYS
Version: Git
OS: Windows 10

PreviousNext

Return to Aircraft

Who is online

Users browsing this forum: No registered users and 4 guests