Board index FlightGear Development Scenery

Copy Terrasync objects into custom scenery  Topic is solved

Questions and discussion about enhancing and populating the FlightGear world.

Re: Copy Terrasync objects into custom scenery

Postby wkitty42 » Sun Sep 13, 2020 4:55 pm

there's also TerraMaster which is a java GUI for downloading scenery tiles... just click on the tile you want and it'll download if your copy is non-existent/different than what the server has... the tricky part (the last time i ran it on linux) was being able to point it to the .fgfs directory which is normally hidden... that was when i found out how to move my terrasync anywhere and tell FG the new location to find the files...
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 6551
Joined: Fri Feb 20, 2015 3:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 14.04.5

Re: Copy Terrasync objects into custom scenery

Postby montagdude » Sun Sep 13, 2020 7:56 pm

wkitty42 wrote in Sun Sep 13, 2020 4:55 pm:there's also TerraMaster which is a java GUI for downloading scenery tiles... just click on the tile you want and it'll download if your copy is non-existent/different than what the server has... the tricky part (the last time i ran it on linux) was being able to point it to the .fgfs directory which is normally hidden... that was when i found out how to move my terrasync anywhere and tell FG the new location to find the files...

That's what I tried to use at first and ended up with some empty stg files. I did notice a bunch of http 429 errors, but since some of the objects downloaded fine, I assumed it was not a problem. I still found TerraMaster useful for figuring out which tiles I had flown over with the UFO, though.
montagdude
 
Posts: 225
Joined: Tue Dec 31, 2019 6:04 am

Re: Copy Terrasync objects into custom scenery

Postby montagdude » Wed Sep 16, 2020 11:10 am

It worked! One tile in downtown NYC had 467,000 buildings and required me to format and use a 32 GB swap partition on an external HDD just to complete it. But the Terrasync objects and osm2city buildings are happily coexisting now. (Looks good, doesn't it?)

Image
montagdude
 
Posts: 225
Joined: Tue Dec 31, 2019 6:04 am

Re: Copy Terrasync objects into custom scenery

Postby Johan G » Wed Sep 16, 2020 12:20 pm

montagdude wrote in Wed Sep 16, 2020 11:10 am:Looks good, doesn't it?

It sure does! :D
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5896
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

Re: Copy Terrasync objects into custom scenery

Postby pb321 » Wed Sep 16, 2020 9:56 pm

So, in the end, how did you make it work? Was the one tile the problem all along. Did processing stop at the tile? How did you recognize that was the problem? It's looking good!
pb321
 
Posts: 267
Joined: Sun Nov 27, 2016 4:08 pm
Version: 2020.2.1
OS: Windows 10 Pro

Re: Copy Terrasync objects into custom scenery

Postby montagdude » Thu Sep 17, 2020 12:59 am

pb321 wrote in Wed Sep 16, 2020 9:56 pm:So, in the end, how did you make it work?
I have this in my osm2city params.ini file:

Code: Select all
PATH_TO_SCENERY = "/home/dello/Projects/FGScenery_development/PA_NJ_NYC/output"
PATH_TO_OUTPUT = "/home/dello/Projects/FGScenery_development/PA_NJ_NYC/output"

Prior to running osm2city, that path contained two subdirectories: Terrain (TerraGear output) and Objects (copied over from Terrasync for the region I'm doing). Then I ran osm2city. It uses the contents of the output/Terrain directory for elevation and output/Objects to check for overlap with static objects. That's why it didn't try to put a building where the Statue of Liberty belongs or to attempt to rebuild the One World Trade Center, for example. In the end you get buildings filled in everywhere that there isn't already a static one from Terrasync.

pb321 wrote in Wed Sep 16, 2020 9:56 pm:Was the one tile the problem all along. Did processing stop at the tile? How did you recognize that was the problem? It's looking good!
I knew it was a problem because I'm always checking how many tiles completed, and that one never did. As it's running, I check how many have completed like this:

Code: Select all
grep "Finished tile" *.log | wc -l

And then compare that with the total number of tiles that the log file reports. You need to run it with the --logtofile option to get these files. Sometimes you get an osm2city-exceptions.log file when it runs out of memory, but in my case I didn't. I think it might have something to do with the fact that I'm using swap on my computer, but I'm not sure. In my case, when it runs out of memory, the process is killed and osm2city forks a new process (with its own log file), but the new one never does any work, and osm2city hangs once all the other tiles are completed. My solution was to add a lot more swap space and then reprocess just that one tile.
montagdude
 
Posts: 225
Joined: Tue Dec 31, 2019 6:04 am

Re: Copy Terrasync objects into custom scenery

Postby pb321 » Thu Sep 17, 2020 4:58 pm

Thanks for the explanation. That helps a lot. Now have to figure out the parallel commands in Windows or run osm2city on a Linux machine. Looking forward to seeing the scenery!
pb321
 
Posts: 267
Joined: Sun Nov 27, 2016 4:08 pm
Version: 2020.2.1
OS: Windows 10 Pro

Re: Copy Terrasync objects into custom scenery

Postby montagdude » Sat Sep 19, 2020 11:43 am

One kind of tedious part about this approach is that if you want to see the objects that osm2city puts in the Details output directory, you have to move them to the Objects directory, but the stg files have the same names. So you can't just move them over, or you will overwrite the original stg files. I think I should write some sort of script to copy all the ac files while concatenating the stg files.
montagdude
 
Posts: 225
Joined: Tue Dec 31, 2019 6:04 am

Re: Copy Terrasync objects into custom scenery

Postby montagdude » Sun Sep 20, 2020 4:29 am

montagdude wrote in Sat Sep 19, 2020 11:43 am:One kind of tedious part about this approach is that if you want to see the objects that osm2city puts in the Details output directory, you have to move them to the Objects directory, but the stg files have the same names. So you can't just move them over, or you will overwrite the original stg files. I think I should write some sort of script to copy all the ac files while concatenating the stg files.

Here is the script I came up with. It will copies everything from output/Details and output/Objects into output/Objects_new, concatenating stg files where necessary. You can then replace the original output/Objects directory with output/Objects_new when you're satisfied that it's all good.

Code: Select all
#!/usr/bin/env python
#
# Copies output/Details and output/Objects into output/Objects_new, concatenating any
# duplicate stg files.

import os
import sys
import shutil
import pathlib

if __name__ == "__main__":

    if not os.path.isdir(os.path.join("output", "Objects")):
        sys.stderr.write("output/Objects not found.\n")
        sys.exit(1)
    if not os.path.isdir(os.path.join("output", "Details")):
        sys.stderr.write("output/Details not found.\n")
        sys.exit(1)

    # Create output directory
    try:
        os.mkdir(os.path.join("output", "Objects_new"))
    except OSError:
        sys.stderr.write("output/Objects_new must not already exist.\n")
        sys.exit(1)

    # Copy files and directories from output/Objects
    for root, dirs, files in os.walk(os.path.join("output", "Objects")):
        for fname in files:
            splitroot = root.split(os.path.sep)
            # Gives relative path of parent directory under output/Objects
            relroot = os.path.join(splitroot[2], splitroot[3])
            # Create the parent directory structure under the output directory (like mkdir -p)
            destroot = os.path.join("output", "Objects_new", relroot)
            pathlib.Path(destroot).mkdir(parents=True, exist_ok=True)
            # Copy the file
            shutil.copy(os.path.join(root, fname), destroot)

    # Copy files and directories from output/Details
    for root, dirs, files in os.walk(os.path.join("output", "Details")):
        for fname in files:
            splitroot = root.split(os.path.sep)
            relroot = os.path.join(splitroot[2], splitroot[3])
            destroot = os.path.join("output", "Objects_new", relroot)
            if not os.path.isdir(destroot):
                pathlib.Path(destroot).mkdir(parents=True, exist_ok=True)
            # If the file exists and is an stg, append rather than overwriting. If not
            # an stg, issue a warning.
            if os.path.isfile(os.path.join(destroot, fname)):
                if not fname.endswith(".stg"):
                    print("Warning: {:s}/{:s} already exists - not overwriting!".format(
                          destroot, fname))
                else:
                    print("Appending to {:s}/{:s}.".format(destroot, fname))
                    f1 = open(os.path.join(destroot, fname), 'a')
                    f2 = open(os.path.join(root, fname))
                    for line in f2:
                        f1.write(line)
                    f1.close()
                    f2.close()
            else:
                shutil.copy(os.path.join(root, fname), destroot)
montagdude
 
Posts: 225
Joined: Tue Dec 31, 2019 6:04 am

Re: Copy Terrasync objects into custom scenery

Postby montagdude » Sun Sep 20, 2020 4:30 am

And I just need to share one more screenshot. :) This will definitely be going into the screenshots thread next month, since I already submitted one for this month. It just looks so dang gorgeous - thanks osm2city! Unfortunately, my laptop can only manage 3-5 fps when flying over Manhattan because of all the buildings.

Image
montagdude
 
Posts: 225
Joined: Tue Dec 31, 2019 6:04 am

Re: Copy Terrasync objects into custom scenery

Postby Johan G » Sun Sep 20, 2020 7:22 am

Wow! :shock: 8) Beat that, other flightsim. :wink:
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5896
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

Previous

Return to Scenery

Who is online

Users browsing this forum: No registered users and 4 guests