Board index FlightGear Support 3rd Party Repositories

Cloning fgdata with GIT submodules

Cloning fgdata with GIT submodules

Postby IAHM-COL » Thu Feb 05, 2015 7:31 pm

IMPORTAT INFORMATION

The FGDATA has succesfully been transferred to the NEXT GENERATION FGDATA
Now also available (NOT IN TEST MODE) with Submodules

The location of FGMEMBERS remains unchanged at: https://github.com/FGMEMBERS
The fork of the Official FGDATA Fork can now be obtained at: https://sourceforge.net/p/fgdata/submod ... next/tree/

Read the Notification here:

viewtopic.php?f=28&t=25314&start=45#p234592

_______________________________________________________________________________

@ALL Flightgear Devs (Core, Aircraft and Other)

Introduction:
Fetching FGdata has been done with a repository hosted in gitorious, for as long as I can remember. (Ive heard it was different before).
This was a very cool way to fetch all the contents of the mainstream FGDATA, including an astonishing amount of FG aircraft in very varied states of development. That are there for us to play with. Literally. Meaning, flying them, tweaking them, forking them, and in the good cases, sharing our updates with the FG world via GIT MERGE REQUEST

The Problem:
The size of this monolithic FGDATA was huge, naturally causing several headaches
1. Git cloning it. Was a time for 2 or 3 coffes. Yes, you get all in one hit. but a loooong hit
2. Git repos size limitation. Happily gitorious allowd that big thing. other repo hosts may be less friendly... but although accepting they seem to be complaining to the dev-core's
3. Git traffic in a huge repo also a problem
4. A project like that large and monolithic, and full of binaries is in summary hard to keep/maintain/develop etc

https://gitorious.org/fg/fgdata/source/ ... 52a366de09:

The proposed Solution
Currently, the solution being executed is, to keep a BASE PACKAGE fgdata, without most of the aircraft, in gitorious. Then move every aircraft away into another monolithic huge repository. And to avod git hosting frictions, they decided to migrate (Back?!) to SVN, CVS done right!, in sourceforge.

http://sourceforge.net/p/flightgear/fgaddon/HEAD/tree/
viewtopic.php?f=28&t=24045

IAHMCOL cries loud, request a reversal of decision, suggest a few alternatives
I may seem very childish and negative when it comes to adopting SVN.

IAHM-COL wrote in Tue Feb 03, 2015 5:41 pm:Legitimate cry
I am almost subversive against subversion!


Maybe right. Also questioning a few things like, adopting SVN for the Aircrafts, adopting GIT for the rest; Finally when I am interested in submit a few modifications of a plane, (EF-Typhoon), I found out that logically the master repo, the SVN is under siege by the core devs. Logically cause noone wants to let anyone accidentally damage years of work. But under siege anyways. And for those, that doubt that there is some sort of controlling siege in place look at the answer of FF-JJTH to the request of a new inclusion:

http://sourceforge.net/p/flightgear/tickets/3/

Does that foster new collaborations?! What is the problem with allowing a new plane that even other devs can progress? Is every other plane in FG a prime-status of dev such that another plane will be soooo much below the threshold ? (these are retorical questions since I hardly see someone changing my opinion here, too)

So I found now the Aircrafts not only move to SVN, but also are now under siege and protected for good.

In the Git world that type of politics has few place. An user forks, modifies, and pulls a request. The request is evaluated, and hopefully if it does not break something luckily goes in, so every other FG member has access to a new "drone" or "plane" or whatever. If it s not good, maybe the base for a new collaboration. A new project. A new moving forward.


The SVN moves sounds like a taken decision
The point behind, the discussions already took place. Its too late to cry on running milk.

A git approach

Fork.
From where, the gitorious fgdata will loose all its aircraft in a menacing near future.

So I decided to step ahead and make a new Proposal for FGDATA management.
Last edited by IAHM-COL on Mon Mar 09, 2015 9:12 pm, edited 2 times in total.
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Thu Feb 05, 2015 7:46 pm

Forking FGDATA BASE and bringing Aircrafts as Submodules

I questioned already whether this approach has been considered. No answer so far.

IAHM-COL wrote in Tue Feb 03, 2015 8:54 pm:If the goal was to "simplify" fgdata cloning, in the first place, and leaving the plethora of aircrafts appart, was working this as submodules ever considered?!


Why submodules?

submodules are rather criticized heavily around the web. Maybe they really bad, or hard to manage, etc.
But for us they behave quite like ring to the finger.

A submodule is an independent git repository that lives in the tree of the superrepository. It can be developed independently, but the superrepository has a pointer to a particular commit, thus always checking out to a certain stage.

Every aircraft in FG independently is a rather small repo. Not a huge monolithic monster. Including submodules (of any size) have very little if any impact on the superrepo's size

and to my knowledge there are not limits in how many submodules a super repo can have.

So it solves all the problems on a rather sneaky way!

FGDATA repo

Let it be the Base. It contains only those aircraft that are base to FG. Like it is being intended.
So
Code: Select all
$ git clone <urltofgdata.repo.git> fgdata


It effectively clones JUST the base. and it does not automatically initialize the 470+ additional aircraft. Just the base ones.
ITs a git meaning it can be cloned, modified, pull, push, forked, merged, rebased, etc,etc


The "Add-on" Aircrafts

Let them be submodules. So every addon aircraft is a git on its own, which means can be independently downloaded, forked and developed by anyone (regardless of how capable that one is.. no every one is a patten)
The pull requests that are worthy can be then committed to the fgdata, making an effective "Airplane update"

More importantly, no one needs to clone the whole fgdata, since one can init submodules with a parameter.

example

Code: Select all
$ git submodule init A300


Initializes the aircraft A300.


Code: Select all
$ git submodule update


updates every initialized submodule. Meaning any aircraft that has been initialized as above!

Anyone can control what subset of Aircrafts will or will not download to his local.


(again, even so, any additional aircraft can be independently cloned outside fgdata, if so desired)

Getting the whole list of 470+ aircrafts

What a hard endeavour will be to initialize all aircraft?!

Not really,
More so

Code: Select all
git submodule init


Without a parameter, means, initialize all existing submodules!

That is every single babe in the FGDATA!

when you run the submodule update... for the first time, maybe, as expected time to go for 2 or 3 coffees ;)
But when back, one is done, and one has a complete working, modular, cute FGDATA ver 3.5.0
Last edited by IAHM-COL on Thu Feb 05, 2015 9:00 pm, edited 1 time in total.
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Thu Feb 05, 2015 7:52 pm

Wait a minute... Did you say you step forward?

Yes, Kinda

This is what I am doing right now

I cloned last FGDATA.
I forked the FGDATA base (I did not bother to keep history here)
Created a new FGDATA git clone somewhere else (kind of like ONOX before)

https://github.com/FGDATA/fgdata

And I am currently working into pushing every submodule of each aircraft up.

The "add-on" aircraft individual repo's I forked them here:

https://github.com/FGMEMBERS

I invite you all to join the resistance :D

Any FG member that is currently in the developers list of FGDATA will gain access to this if he/she contacts me with his Github login
Additionally I will be happy to include him/her in the FGMEMBERS group, wich is an automatic way to gain access to all aircraft.

(Any non-dev core that will willing to modify planes can fork and pull request, as always, or by request, may be joined to FGMEMBERS)

(pull + push + admin)
And since this is a git. Feel free to fork, and spread the word :D

Feel free to modify and pull request.

ETC

LETS FOSTER A COOPERATIVE ENVIRONMENT

And in the meantime, lets lobby for a free FG. A Git versioned one :D
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Thu Feb 05, 2015 7:54 pm

FINAL WORD

I am building a cron job that fetches the SVN every X Frequency, takes the Diff, and updates this Modular FGDATA
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Thu Feb 05, 2015 8:26 pm

Mmmm.. I think I dont need to loose history of the git

Let me test something here, for an hour or two
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby AndersG » Thu Feb 05, 2015 8:49 pm

You certainly don't need to lose the history - use git filter-branch to create the per-aircraft repositories from fgdata/git. That was done a already few years back but in the end the change came to nothing that time..

You could consider git-svn to pull upstream changes into your per-aircraft submodules (probably cherry-picking them across to a "pure" git branch rooted in the old history).
Last edited by AndersG on Thu Feb 05, 2015 8:51 pm, edited 1 time in total.
Callsign: SE-AG
Aircraft (uhm...): Submarine Scout, Zeppelin NT, ZF Navy free balloon, Nordstern, Hindenburg, Short Empire flying-boat, ZNP-K, North Sea class, MTB T21 class, U.S.S. Monitor, MFI-9B, Type UB I submarine, Gokstad ship, Renault FT.
AndersG
 
Posts: 2524
Joined: Wed Nov 29, 2006 10:20 am
Location: Göteborg, Sweden
Callsign: SE-AG
OS: Debian GNU Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Thu Feb 05, 2015 8:51 pm

nice hint :D
Thanks Anders :D

Right now trying to repush all fgdata"base" with the history too.

When I get the base, then I can attempt getting the per-aircraft, as you are suggesting :D

.... on the edge...
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Thu Feb 05, 2015 11:44 pm

@ANDERS

That will be doing the job, indeed
But it is very very painfully slow to preserve the history of 1 single craft. Or the base. Let alone start extracting the whole 470 histories.

Iam really close to give up in attempting to preserve that history. For that we all have the official repos, I guess.

Most importantly, the history after a git filter-branch is a completely rewritten one, which means, that such actually changes.What I am planning now to do is go ahead and try to create an FGDATA with a rewritten history that lacks everything in Aircraft directory, then repopulate an "Historyless" Aircraft directory.
That should get the job done for most cases in my fork.


Besides, given that most aircrafts in my fork are submodules, these may be able to be repopulated with the history re-extracted at a later date.

I am really wondering how are the dev-cores going to ditch the now-addon aircrafts out of fgdata while avoiding a history rewrite ?!

PS: I reconstructed the repo without history. I apologize to the history and the if anyone feels his/her contribution is not acknowledge by the lack of history. I will be looking forward to see how the official git repo will respond, and they may be able to provide a modular but complete FGDATA :D
Or at least a version of the FGDATA minus Aircrafts that conserves the history

IHCOL
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby AndersG » Fri Feb 06, 2015 9:47 pm

IAHM-COL wrote in Thu Feb 05, 2015 11:44 pm:Most importantly, the history after a git filter-branch is a completely rewritten one, which means, that such actually changes.


Yes, that's one of git's best features - that the sequence of commit IDs in a branch is thoroughly tied to the corresponding content. Change the content and the commit ID changes. However, git filter-branch preserves the meta information about each commit - and that's enough from my perspective to call it preserving the history in this context. Removing the fg-addon aircraft and their history from the master branch in fgdata would also be a filter-branch operation.
Callsign: SE-AG
Aircraft (uhm...): Submarine Scout, Zeppelin NT, ZF Navy free balloon, Nordstern, Hindenburg, Short Empire flying-boat, ZNP-K, North Sea class, MTB T21 class, U.S.S. Monitor, MFI-9B, Type UB I submarine, Gokstad ship, Renault FT.
AndersG
 
Posts: 2524
Joined: Wed Nov 29, 2006 10:20 am
Location: Göteborg, Sweden
Callsign: SE-AG
OS: Debian GNU Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Fri Feb 06, 2015 10:31 pm

Yes indeed
Thanks for your comments AndersG
They are rather pointing me in the right direction

I had initialized all git repos now, basically with an historyless commit. Now I am working in isolating each aircraft with the filter-branch option, to rescue the relevant history, to finalize by pushing force the new history.
I hope that works.

As you point out, in this context (the addon aircrafts) it is a nice approach to recover that history.

On the other hand, the base package is of a concern to me.

The ideal will be Forking a git repo that does not have those add-on aircrafts. And then top the submodular aircraft.
The advantage of doing so, is that by being effectively a fork, I can just pull from the official repo, push to my fork.
But I can't do that, simply because the official repo still has all those crafts! :(.... which means, if I filter-branch=it to get a clean base, that "history change" is dramatic enough to not allow me to update my fork from the official repo anymore ....

Furthermore, since removing the addons is an effort to effectively reduce fgdata size, I am wondering what steps the dev-core people is planning to take~?

I will first concentrate my efforts in effectively isolating the aircrafts. Because they are the submodule base to build my fork later.
I am still looking forward to a few things from the dev core:

1) Do they think my option is viable for the mainstream? (I mean opposite to SVN the crafts?)
2) Will they be ok to initiate conversations about moving in this direction? ---I will be satisfied enough of seeing that some talk is generated, and if after, a consensus says the official repo is SVNed for aircrafts, that will be accepting. But right now, I wonder if I am at least stirring some curiosity here?!

In any case, if I have to generate a fork, I will be looking forward to be able to fork a "addonless" fgdata, that is already effectively reduced in size, so the mainstream and my fork can have a shared history, so I can at least pull to update. If that is the case, I can look to either use git svn to fork -clone everything done on the SVN side. A middle scenery is scripting some way to use the svn diff to update the aircraft repos.

In the super-best case scenerio for me, it will be that the dev-core people agree with going submodular as well.
Look, in my opinion, even if someone wants to maintain a separate SVN-distro of addons appart, that is not incompatible with going modular as well. Basically, the SVN group just will be avoiding to initialize any submodule.

Again, Anders, thanks for the great pointers.

I just finished initalizing all of the 469 repos, and even in a few hours initialize the "::TEST::" version of the submodular fgdata GIT. Now into rescuing history and force update these histories in the already initialized repos.

Best,
IHCOL

Aircraft Repos: https://github.com/FGMEMBERS
:TEST: FGdata forked submodular repo: https://github.com/FGDATA/fgdata
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Sat Feb 07, 2015 3:21 am

Hi ALL

The Modular FGDATA for test purposes is completely uploaded now.
I call this the "TEST 2" because is not quite a fork yet. It is more an snapshot of the fgdata on version 3.5.0 as it stood on Feb*05*2015

See more details above of my request for a conversation on the future of storing aircraft in FG, but I animously invite everyone to test ride this method using the test FGDATA provided: Here are the basic guidelines

1. Required:
A git software, ideally version 1.8 or more for having all functionality with submodules. Examples below are for the command line (or the git-bash) that comes with your git software. Using GUI versions all this must be possible, but I had not tested it

2. Initial FGDATA cloning.
This is done equally as the current FGDATA git is cloned. But there is a twist.
  1. Current mainstream FGDATA: Its a huge repository that will bring all together a collection of about 485 aircraft. It brings down the complete suit and may take a significant time. Depending on your connection speed of course
  2. This Testing FGDATA; Its a modular repository. Cloning brings down ONLY what is known as the "BASE PACKAGE". It includes all folders and directories complete, as they stood on the snapshot taken for the example. BUT, in the Aircraft folder, it only includes 21 base folders, which includes the few initial aircraft of every "base packaged" Flightgear. The C172p, the F14, The 777, the B1900d, the UFO, are among this download aircraft. Bringing this base package creates a functionally complete flightgear, without the burning on downloading 469 additional aircraft in the first shot. It significantly reduces the "downloading time"

Code: Select all
$ git clone https://github.com/FGDATA/fgdata.git
or
$ git clone git@github.com:FGDATA/fgdata.git


2. Installing new Aircraft

As I mentioned, this FGDATA is modular, and the cloning did not bring all aircraft. To bring an aircraft, its submodule needs to be initialized. Then the initialized modules updated.

Code: Select all
$ git submodule init Aircraft/707
$ git submodule update


Such code will initialize your repo to also have the amazing lake of constanze 707. then by updating it, it brings the aircraft to your local repo of FGDATA
You can initialize as many aircraft as wanted. No limit. All Initialized submodu2. Initial FGDATA cloning. les are updated with the single update command simmultaneously!

(one can make a bash script to initialize multiple aircraft, or so)


3. NAH! I want 469 additional aircraft to the less command approach.

Well again, there is a nice feature: submodule init without parameters will initialize all submodules in a shot

Code: Select all
$ git submodule init
$ git submodule update

or
$git submodule update --init
for an abbreviated form that initializes and updates the whole package


4. Uninstalling modules

For the end user, after you initialize an Aircraft, the GIT proper way of ditching it out is:

Code: Select all
$ git submodule deinit Aircraft/707
$git submodule update

(Why would someone do that goes beyond my understanding but oh well... this is for test purposes rite?!)

Having an option of deinstalling submodules is a cool advantage to reduce the size of one own's fgdata repo that is not that easy in the monolithic current approach.

Also
Code: Select all
$ git submodule deinit   .   #there is a point at the end there!
$git submodule update


Will actually very efficiently get you back to the basic package!

5. Forewarning
This repository is currently just a test frame for a possible implementation of modularity in the FGDATA git.
DO NOT USE for a main installation, and reffer to the official channel in https://gitorious.org/fg/fgdata/source/ ... 32dcff9727: for reliability purposes!!

The most important thing to note, is that the test fgdata is currently just an snapshot of version 3.5.0 an Feb*05*2015, not a real fork. So that FGDATA lacks all branches, and previous history of the official channel. And in addition, remote cannot be properly redirected to be able to fetch from the official one (just yet!)

The only way to use the FGDATA provided in this test with an actual flightgear installation is to
1. change the file version to the proper string, example from 3.5.0 to 3.3.0
2. point your fglauncher to the FGDATA obtained here (with fgrun, fgx, or the command line --fg-root option)
3. I tested this FGDATA with FG 3.2, 3.3 and 3.5, and it workd. I tested it with FG 2.12 and it did not, logging in creating nav data infinetely

6. This method of fetching FGDATA
Maintains a reduced size of the base package in the repository.
Submodules initialized do not effectively increase the size of a repo, with the exception of a few lines of code in .gitmodule and .git/config lines

So, it accomplishes the main goal of reducing FGDATA size, but maintains aircraft development in an git state
Furthermore, Git submodules can point to ANY repo, so these could be even pointing to aircraft repositories in development by any FG member, in any place, as much as they agree with conform with GPL.

7. Give it a test ride to the test :D

There is lots of red-tags concerns online about modules, but I think they are actually quite fit for the FGDATA part of fligthgear project. And a very powerful mode we could actually get to manage and develop our aircraft.
Every Aircraft in a submodule is effectively a git somewhere else, and that somewhere else can be developed on its own.
The FGDATA is actually just having a pointer to a certain commit on the development tree of such aircraft. This can be move to a different commit later, but it has the advantage that if a given plane is being developed, assuming no-one destroys the "history" of that plane, the status of such plane in FGDATA is the same until the commit pointed to is updated!.

A major suggestion is to never "develop" a submodule inside the main tree (FGDATA). Instead having an independent "development clone". So FGDATA does not get "mess up" with changes in its tree. Things may need to be figured out in more detail, but in general terms, I think, we can practice and experience with this test frame I created.

Furthermore, I am currently trying to rescue the "git history" of each individualized aircraft; figuring out the filter-branch option Anders has kindly pointed me too ....

Best to all
IHCOL
Last edited by IAHM-COL on Wed Feb 11, 2015 2:23 am, edited 1 time in total.
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Sat Feb 07, 2015 8:48 pm

AndersG wrote in Thu Feb 05, 2015 8:49 pm:You certainly don't need to lose the history - use git filter-branch to create the per-aircraft repositories from fgdata/git.



Great
I finally figured this one out
Time to prepare the script and recurse
It is slow, but not too much... I think

The most important part is after git filter, is to create a clone that ONLY has the new object set, so it is effectively a small repo with only the aircraft
(as opposed to a huge 20G repo for every aircraft!)

This is the webpage I kind of followed:
http://www.pither.com/simon/blog/2009/0 ... repository

This is the result for the first aircraft in the list:
https://github.com/FGMEMBERS/14bis

Now we can see the proper Commit's list here
https://github.com/FGMEMBERS/14bis/commits/master

8)
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Sun Feb 08, 2015 3:15 am

well once one figures things out, it becomes easier

And since all those aircraft gits were intialized already, I am looping over an script that is effectively bringing up the history of each aircraft. one at a time. It looks kinda cool seeing those gits being updated. I hope the script does not break somewhere... but we'll see:

For the curious ones this is how my script looks like:


Code: Select all
#!/bin/bash

FGDATA=~/local/source/gitfg/f2/fgdata
TEMP=~/local/source/gitfg/tempA
PLANE=Aircraft/$1

echo $1
echo $PLANE
cd $FGDATA
git branch extract
echo "extracting"
git filter-branch -f --subdirectory-filter Aircraft/$1 extract
echo "extracted"
cd $TEMP/$1
echo $PWD
git init
echo "git inited"
echo "pulling"
git pull $FGDATA extract
git remote add origin git@github.com:FGMEMBERS/$1.git
git remote -v
git push --force -u origin master
echo $1 >> ~/local/source/repoUpdated.txt
echo "pushed"
#do not delete quite yet!
#rm * -Rf
#rm .git -R
cd $FGDATA
git branch -D extract
echo "done"
cd $FGDATA/..
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Sun Feb 08, 2015 9:43 pm

The script was recursively executed on every "NOT BASE" aircraft in the FGDATA repository.

This succesfully

1. Extracted the Aircraft from the monolithic FGDATA
2. Rewrote a history containing only every commit relative to the each plane
3. Reduced the per-aircraft git repository to the appropriate smaller size
4. Forced a push that completely updated the FGMEMBER history, now thus all history being "in this context" as AndersG put it, rescued it

These planes and their commit history can be visited, cloned, branched, etc, at:
https://github.com/FGMEMBERS/

Note: A few planes appeared only in the SVN but not to be present in the lastest FGDATA I pulled, thus I don't have a proper repository to pick the history as of yet.
If one of you know a git repository that contains such history do not doubt to point it to me and I ll update them

Planes in the SVN and the FGMEMBERS FGDATA not present in the mainstream FGDATA as of FEB2015. Thus still their history is not recovered.
Code: Select all
‘Caproni-C22J/’
‘Cirrus-SR22/’ 
‘Dassault-Mystere-IV/’
"Harrier-GR1/’
‘Jodel-D140/’
‘rah-66/’
‘Sky-sportster/’
‘Su-25/’
‘Tecnam-P2006T’
‘Tecnam-P2010’
‘Tecnam-P92’
‘Tu-SB2bis-M103/’
‘XP-67/’
‘Yak-130/’
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Re: Cloning fgdata with GIT submodules

Postby IAHM-COL » Sun Feb 08, 2015 9:49 pm

Notes about FGDATA test with submodules

The rebasing of all aircraft in FGDATA has not taken place. Thus the FGDATA still points to the history-less initialization commit.
I am unsure how that will work at the moment, and the rewritten history may have broken the FGDATA until all the aircraft submodules are rebased, and point to the latest commit.

Feel free to test it, but expect a MODULAR FGDATA TEST major update soon, rendering current repo outdated.

For the Aircraft repos, you can consider them now "stable" , and thus I encourage everyone to use them for development and submit development pull requests too :D
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it? Probably not, because if they don’t recognise their freedoms, they’ll let their freedoms fall
User avatar
IAHM-COL
Retired
 
Posts: 4057
Joined: Wed Aug 08, 2012 6:40 pm
Location: Homey, NV (KXTA) - U.S.A
Callsign: HK-424D or ICAO4243
Version: 3.7-git
OS: Linux

Next

Return to 3rd Party Repositories

Who is online

Users browsing this forum: No registered users and 4 guests