Board index FlightGear Development Documentation

Automatic documentation generator

Discussion of the FlightGear documentation, how it can be improved and coordination of people working on it.

Automatic documentation generator

Postby rubdos » Sun Jan 11, 2015 11:41 am

Hello flightgear-people,

First of all, do not consider my past on this forum. I've changed my mind about some things ;-)
I'm pretty active on the #flightgear IRC channel and people there told me to start discussing my idea here and perhaps on the devel mailinglist.

I've written a simple python script which iterates over all Airplanes in FGDATA and does some stuff with it. Basically, it automatically generates LaTeX files, which are then compiled into a PDF.

This is useful for eg. having your aircraft documentation on (e-)paper and just having documentation bundled.

Current features:
0) It takes the name of the aircraft and puts it in a title, together with a thumbnail and a table of contents.
1) It takes all files ending in "checklist.xml", it parses them for some commonly used ways of writing checklists for flightgear, it makes it a bit human readable and then writes them into a pdf.
2) It takes the subdirectory Docs of the aircraft. If it contains extra LaTeX sources, it copies them over and includes them.

Extending this simple script with more alike features can render some pretty nice .pdf files.

Planned extra features:
- Enable airplane developers to easily add documentation, without the knowledge of LaTeX, by adding additional .xml or alike files to the Docs folder, i.e.
* Aircraft specifications (max takeoff weight, cruise speed, max passengers, range)
* Procedures
- Having a website (or a subdomain of flightgear.org) where you can search and download the precompiled .pdf files.

I added some extra files in the Docs folder of the 777 series to have some example https://cloud.glycos.org/public.php?service=files&t=8f66c6a18171d88bc8b8530ab4394913.

EDIT: the red bounding boxes won't be there when printed; they're just indicators for having links within the pdf. Pretty useful if you ask me.

Please give me your opinion and extra ideas.
User avatar
rubdos
 
Posts: 17
Joined: Sun Aug 28, 2011 5:48 pm
Location: Brussels
Callsign: FTL579
Version: git
OS: Linux

Re: Automatic documentation generator

Postby Hooray » Sun Jan 11, 2015 11:50 am

Hi,

just briefly:
It takes all files ending in "checklist.xml", it parses them for some commonly used ways of writing checklists for flightgear, it makes it a bit human readable and then writes them into a pdf.

you don't need to look for the file name - you can simply use the checklists node in the -set.xml file - you just need to make sure to honor the include attribute in those PropertyList XML files:

http://wiki.flightgear.org/Aircraft_Checklists
http://wiki.flightgear.org/PropertyList_XML_File


It takes the subdirectory Docs of the aircraft. If it contains extra LaTeX sources, it copies them over and includes them.

it would make sense to introduce optional tags (e.g. docs) and use those for aircraft specific docs, e.g.:

Code: Select all
<docs>
<file>Docs/foo.tex</file>
<file>Docs/bar.tex</file>
</docs>


That way, there's a cleaner method - and you'd only need to document it on the wiki and get in touch with active aircraft developers to establish this as a "best practice".


Enable airplane developers to easily add documentation, without the knowledge of LaTeX, by adding additional .xml or alike files to the Docs folder

I'd suggest to keep on using PropertyList-XML files using the include attribute for the sake of compatibility with the rest of SG/FG.
You should also be aware of Catalog metadata: http://wiki.flightgear.org/Catalog_metadata

For procedures, please see: viewtopic.php?f=6&t=24703&p=225601&hilit=#p225601

Overall, this could be pretty useful - we've had a few related attempts in the past (so it would make sense to coordinate things with other contributors/committers), e.g:


Subject: Canvas API documentation
Hooray wrote:Adrian's nasal_api_doc.py script in $FG_SRC/scripts/python should automatically create updated API documentation based on the nasal source code in $FG_ROOT/Nasal (html output).


(this may not be directly applicable here, but could be useful for processing $FG_ROOT files, so I am just mentionig this here because it's also written in python - so you could borrow useful code from related scripts)

Subject: READ-ME (text-file) formatting request

Hooray wrote:the deeper issue here is improving accessibility and usability which is 100% legit - and is exactly the reason why I am supportive of the idea of using a LaTex wrapper for the various README.* files in $FG_ROOT that compiles into a simple PDF that will automatically be opened by the correct viewer, i.e. acrobat reader, without being a PITA to maintain for fgdata contributors, who would still end up committing updates to the various README files.

So please, do not try to turn this into a MS vs. NIX issue - which it really isn't at all - I'd argue that it has taken all of us more time to respond here than simply provide a LaTex wrapper to get this thing started, and the topic closed for good :D

Here's a little Nasal helper that create a simple LaTex stub:
Code: Select all
var path = getprop("/sim/fg-root") ~ "/Docs";
var doc_path = func(file) return path ~ file;

var latex_begin = "
\\documentclass[12pt]{article}

\\usepackage[english]{babel}
\\usepackage[utf8x]{inputenc}
\\usepackage{amsmath}
\\usepackage{graphicx}
\\usepackage{sverb}

\\title{FlightGear READMEs}
\\author{FlightGear Community}

\\begin{document}
\\maketitle

% automatically created section

";


var files = [
 "README.introduction","README.electrical","README.sound", "README.3DClouds", "README.fgjs", "README.JSBsim","README.minipanel","README.submodels",
 "README.xmlparticles", "README.airspeed-indicator", "README.flightrecorder","README.jsclient","README.multiplayer", "README.systems","README.xmlsound",
 "README.checklists","README.gui","README.multiscreen", "README.tutorials","README.xmlsyntax","README.commands","README.hud","README.layout",
 "README.osgtext","README.yasim","README.conditions","README.properties","README.wildfire", "README.digitalfilters","README.IO","README.logging","README.protocol","README.xmlhud", 
 "README.effects","README.Joystick","README.materials","README.scenery","README.xmlpanel" ];

print( latex_begin );
foreach (var f; files) {
   var section =  "\\section{"~f~"}\n";
   var content =  "\\verbinput{"~doc_path(f)~"}\n";
   print (section, content);
}
print("\\end{document}");


(you would probably want to order the README files a little and maybe provide more descriptive titles - but for starters, that shoudl work - you could also directly write the whole thing to disk using io.nas obviously)


(Stuart ended up rewriting this in perl or python - but it's doing the same thing)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11378
Joined: Tue Mar 25, 2008 8:40 am

Re: Automatic documentation generator

Postby rubdos » Sun Jan 11, 2015 12:11 pm

I would suggest dedictating some subdomain (let's say, docs.flightgear.org) for downloading such documents, with some subcategories like you said:

For procedures, please see: viewtopic.php?f=6&t=24703&p=225601&hilit=#p225601


Such as:
- airport diagrams (generated from the geo-data)
- aircraft documentation

and so forth.

I'll have a look at the other things you proposed and I'll try to implement reading the -set.xml file. (I didn't do that yet :oops: )
Other than that I'll make a proof of concept for the documentation-site. I'll keep this thread posted.
User avatar
rubdos
 
Posts: 17
Joined: Sun Aug 28, 2011 5:48 pm
Location: Brussels
Callsign: FTL579
Version: git
OS: Linux

Re: Automatic documentation generator

Postby onox » Sun Jan 11, 2015 12:33 pm

Very cool, rubdos :)

Could you make a specific python script that only displays the checklists? Treat each checklist as a chapter and put each new chapter on a separate page.

Perhaps you could also add .pdf files that each contain SIDs and STARs (user has to point your python script to these files or to the directory containing these files) and add them together to your single .pdf file.
Last edited by onox on Sun Jan 11, 2015 12:36 pm, edited 1 time in total.
onox
Retired
 
Posts: 431
Joined: Fri Jun 20, 2014 2:45 pm

Re: Automatic documentation generator

Postby rubdos » Sun Jan 11, 2015 12:34 pm

onox wrote in Sun Jan 11, 2015 12:33 pm:Very cool, rubdos :)

Could you make a specific python script that only displays the checklists? Treat each checklist as a chapter and put each new chapter on a separate page.


I was thinking of doing this. Now that there's a request, I'll certainly do :)
User avatar
rubdos
 
Posts: 17
Joined: Sun Aug 28, 2011 5:48 pm
Location: Brussels
Callsign: FTL579
Version: git
OS: Linux

Re: Automatic documentation generator

Postby rubdos » Sun Jan 11, 2015 3:53 pm

Hooray wrote in Sun Jan 11, 2015 11:50 am:Hi,

just briefly:
It takes all files ending in "checklist.xml", it parses them for some commonly used ways of writing checklists for flightgear, it makes it a bit human readable and then writes them into a pdf.

you don't need to look for the file name - you can simply use the checklists node in the -set.xml file - you just need to make sure to honor the include attribute in those PropertyList XML files:

http://wiki.flightgear.org/Aircraft_Checklists
http://wiki.flightgear.org/PropertyList_XML_File



I just implemented reading the -set.xml files, which will now also generate documentation per Aircraft type (e.g. 777-300ER, 777-200LR, ...) instead of just per aircraft directory (777, 737, c172p, ...)

It reads the include attribute and parses it into the checklist. I'll do the docs tag later.
User avatar
rubdos
 
Posts: 17
Joined: Sun Aug 28, 2011 5:48 pm
Location: Brussels
Callsign: FTL579
Version: git
OS: Linux

Re: Automatic documentation generator

Postby Hooray » Sun Jan 11, 2015 5:30 pm

sounds good to me - great job/swift response! :D
onox wrote in Sun Jan 11, 2015 12:33 pm:Perhaps you could also add .pdf files that each contain SIDs and STARs (user has to point your python script to these files or to the directory containing these files) and add them together to your single .pdf file.


Concerning TPPs, I'd suggest to check out a handful of related discussions/efforts, especially:

http://wiki.flightgear.org/Airport_Diagram_Generator

A project to create a source of free geo-referenced instrume


Subject: 777 EFB: initial feedback

Hooray wrote:At least for the US, there's a ton of data freely available that can be extracted from such charts procedurally using a bit of scripting and OCR or regex. The corresponding data could be put into a database and updated regularly, according to the AIRAC cycle. But something like that should really be a central web service - in FG, we now have the means to easily download data via HTTP, and make web service requests. In other words, we could populate some kind of central navdb with our existing data, update the whole thing with the kind of data provided by the scripted discussed at Subject: A project to create a source of free geo-referenced instrume.

At that point, we would have a web service that could run a cron job to regularly update the whole thing - maybe in combination with some kind of wiki-like front-end to allow contributors to update/edit and maintain entries for different countries and navaids, as well as procedures.

The navdb would then be md5-hashed and fetched on demand, so that the navcache can be updated/rebuilt accordingly.


Subject: Why does Flightgear use X-Plane airport data?
Hooray wrote:I am not sure if I agree: There are so many advantages when reusing the X-Plane data and their format, they have a HUGE community of contributors - which FG simply doesn't have.

On the other hand, some web-based frontend to allow people to easily contribute and maintain airport/navdb data, would be great to have in my opinion:
Pretty much like a "GIS Wiki", possibly based on PSQL or even git - so that people could go to a website, open an airport/navaid and file merge requests, review entries and help maintain all the data there.

The new scenemodels submission process is actually a very good example here.

Something like this could be very useful, not just for airports (runways, taxiways, aprons etc), but also for navaids and terminal procedures (SIDs/STARs and IAPs).

And I guess we could even get the X-Plane community to contribute to something like this.
The whole work flow could be moved to a website, so that it would be easier contribute.

Using git as the backend (i.e. plain text files) would have the advantage, that we'd also get a revision history automatically.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11378
Joined: Tue Mar 25, 2008 8:40 am

Re: Automatic documentation generator

Postby stuart » Sun Jan 11, 2015 8:17 pm

Hi,

This all sounds great.

For reference, the current Latex creator that parses and packages up all the Docs/README files is available as part of the "getstart" git repository: https://www.gitorious.org/fg/getstart/

The specific script is getstart/bin/makereadmepdf.pl

I re-run that script for each release and check in the resulting PDF into FGDATA. If you'd like to do so, we could include your script in the getstart repository and I would re-run it for each release as well, so each release would include a set of Aircraft PDF files as well.

As an aside, it would be worth including the source repository (generally fgaddon) for each aircraft, so a use could read the PDF and know where to download the aircraft from.

Personally, I think it's worth keeping the per-Aircraft documentation separate from the SID/STAR, and the former would be more immediately useful than the latter.

-Stuart
G-MWLX
User avatar
stuart
Moderator
 
Posts: 1469
Joined: Wed Nov 29, 2006 9:56 am
Location: Edinburgh
Callsign: G-MWLX

Re: Automatic documentation generator

Postby Hooray » Sun Jan 11, 2015 8:19 pm

stuart wrote in Sun Jan 11, 2015 8:17 pm:I re-run that script for each release and check in the resulting PDF into FGDATA. If you'd like to do so, we could include your script in the getstart repository and I would re-run it for each release as well, so each release would include a set of Aircraft PDF files as well.


You might want to also include Philosopher's tex file and add it to the build system: https://gitorious.org/fg/fgdata/source/ ... al-doc.tex
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11378
Joined: Tue Mar 25, 2008 8:40 am

Re: Automatic documentation generator

Postby rubdos » Mon Jan 12, 2015 3:16 pm

stuart wrote in Sun Jan 11, 2015 8:17 pm:Hi,

This all sounds great.

For reference, the current Latex creator that parses and packages up all the Docs/README files is available as part of the "getstart" git repository: https://www.gitorious.org/fg/getstart/

The specific script is getstart/bin/makereadmepdf.pl

I re-run that script for each release and check in the resulting PDF into FGDATA. If you'd like to do so, we could include your script in the getstart repository and I would re-run it for each release as well, so each release would include a set of Aircraft PDF files as well.

As an aside, it would be worth including the source repository (generally fgaddon) for each aircraft, so a use could read the PDF and know where to download the aircraft from.

Personally, I think it's worth keeping the per-Aircraft documentation separate from the SID/STAR, and the former would be more immediately useful than the latter.

-Stuart


I did a merge request on fgdata already, I put the generator script in the Docs directory.

I don't really get what you say about including the source repository; you mean adding the pdf to the zip-download of the Aircraft.

I agree on the SID STAR matter. That should be separate from the aircraft.

Next thing I want to try is creating an automated build on my VPS. I have one problem there: I only have 10 GB disk space left and the fgdata git distribution is about 17 GB. Gotta invent some form of git-mount :p
User avatar
rubdos
 
Posts: 17
Joined: Sun Aug 28, 2011 5:48 pm
Location: Brussels
Callsign: FTL579
Version: git
OS: Linux

Re: Automatic documentation generator

Postby rubdos » Tue Jan 13, 2015 1:54 pm

I sent a mail to flightgear-devel regarding my merge-request, but it seems like it got rejected by a moderator :O

onox wrote in Sun Jan 11, 2015 12:33 pm:Could you make a specific python script that only displays the checklists? Treat each checklist as a chapter and put each new chapter on a separate page.


I implemented this, it's in the pull request.

onox wrote in Sun Jan 11, 2015 12:33 pm:Perhaps you could also add .pdf files that each contain SIDs and STARs (user has to point your python script to these files or to the directory containing these files) and add them together to your single .pdf file.

It sounds like this is a different though related issue. This is a bit more difficult I think, because it'd be preferable to have charts of the airspace on the pdf? It's not impossible, but a lot more work to get that sort of things automated.
User avatar
rubdos
 
Posts: 17
Joined: Sun Aug 28, 2011 5:48 pm
Location: Brussels
Callsign: FTL579
Version: git
OS: Linux

Re: Automatic documentation generator

Postby onox » Tue Jan 13, 2015 2:23 pm

Yeah, on second thought it's not logical to have it in the same .pdf as the aircraft docs. Maybe I can use some LaTeX to just stitch together many .pdf files so that I don't have to open many separate pdf files when approaching an airport.
onox
Retired
 
Posts: 431
Joined: Fri Jun 20, 2014 2:45 pm

Re: Automatic documentation generator

Postby rubdos » Tue Jan 13, 2015 2:50 pm

onox wrote in Tue Jan 13, 2015 2:23 pm:Yeah, on second thought it's not logical to have it in the same .pdf as the aircraft docs. Maybe I can use some LaTeX to just stitch together many .pdf files so that I don't have to open many separate pdf files when approaching an airport.


You can use LaTeX, or just some pdf stitching program. Having several LaTeX pdf files open won't hurt though; LaTeX creates very efficient pdf files. e.g. I compiled the c++17 docs some time ago. Over a (two?) thousand pages, lots and lots of text, just over 1 MB.

Personally, I'd use some e-reader or extern tablet for that sort of things and I'd print the procedures for my favorite aircraft or real paper.

(I've created a little copyright war on IRC regarding that topic right now)

If there's any free/libre GPL compatible source of SID and STAR data in FlightGear, I'd have no problem creating some extra lines python that make up a map and draw something useful for SID and STAR diagrams (leveling up my knowledge of Tikz in LaTeX :wink: ). That would make a free and legal source for SID and STAR sheets, even for commercial use.

Simple things like this can have a great impact on flightgear. I think it would improve the flightgear ecosystem very well. Imagine a single website (docs.flightgear.org? :D) on which you can find all this sort of information.
User avatar
rubdos
 
Posts: 17
Joined: Sun Aug 28, 2011 5:48 pm
Location: Brussels
Callsign: FTL579
Version: git
OS: Linux

Re: Automatic documentation generator

Postby rubdos » Fri Jan 16, 2015 1:11 pm

I have made a little webinterface for downloading the pdf-files. @onox: the checklists are there.
Currently only the fgdata Aircraft are parsed, didn't have time for the fgaddon aircraft yet. Please have a look: http://fgfsdocs.glycos.org/
User avatar
rubdos
 
Posts: 17
Joined: Sun Aug 28, 2011 5:48 pm
Location: Brussels
Callsign: FTL579
Version: git
OS: Linux

Re: Automatic documentation generator

Postby Hooray » Fri Jan 16, 2015 1:36 pm

also check out jlmcgraw's postings, e.g.: viewtopic.php?f=6&t=25168
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11378
Joined: Tue Mar 25, 2008 8:40 am

Next

Return to Documentation

Who is online

Users browsing this forum: No registered users and 0 guests