Board index FlightGear Development Scenery

Next World Scenery build

Questions and discussion about enhancing and populating the FlightGear world.

Re: Next World Scenery build

Postby Nick R » Sun Jun 25, 2017 6:12 am

pommesschranke wrote in Fri Jun 16, 2017 9:33 pm:Torsten:

I use the scripts by NickR to make my custom scenery terrain:

https://gitlab.com/fg-scenery-build-scr ... ree/master

Thanks for linking to my scripts although I haven't used them in almost two years I think. Do they still work? I never really documented the scripts anywhere on the Wiki which I probably should have so users who are looking for scripts can find them.

Off topic: I have been away from flightgear for over a year I think even though I really like the FlightGear project and would like to be more active with it. Hopefully I will have more time in a few weeks, but maybe I first need a computer upgrade.
Hangar: fgpipistrel.org
Modelling the Pipistrel Virus SW (github, website)
Nick R
 
Posts: 173
Joined: Tue Nov 26, 2013 4:50 pm
Location: Stettler, AB, Canada
Callsign: NickR
Version: 2017.2.1
OS: Linux

Re: Next World Scenery build

Postby Torsten » Mon Jun 26, 2017 11:12 am

Just a little update on my (slow) progress:
- I managed to compile the toolchain, the required code changes to compile with latest simgear are pushed to the ws20 branch. Note: there is still the need to add "-std=c++11" to the cmake command line and to run cmake twice before executing make.
- I have created a Jenkins job to build TerraGear: https://scenery.flightgear.org/jenkins/job/TerraGear/
- I have downloaded the complete SRTM3 and SRTM1 data set
- I ran gdalchop over each separate landmass/continent
- I started to run terrafit over the entire globe. This crashed after a couple of hours. Bad news: I compiled without debug information and have no usable stack trace. Will recompile as RelWithDebInfo and restart this step.

Used disk space so far is approx. 100GB

Torsten
flightgear.org - where development happens.
User avatar
Torsten
 
Posts: 648
Joined: Fri Feb 01, 2008 10:22 pm
Location: near Hamburg, Germany
Callsign: offline
Version: next
OS: Linux

Re: Next World Scenery build

Postby wkitty42 » Mon Jun 26, 2017 8:25 pm

Torsten wrote in Mon Jun 26, 2017 11:12 am:Note: there is still the need to add "-std=c++11" to the cmake command line and to run cmake twice before executing make.

is this running cmake twice step needed every time or only the first time it is built?

do you think you'll be able to find where, in the cmake files, to add the "-std=c++11" option?
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9148
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Next World Scenery build

Postby Torsten » Tue Jun 27, 2017 8:18 am

wkitty42 wrote in Mon Jun 26, 2017 8:25 pm:is this running cmake twice step needed every time or only the first time it is built?

Just once for the first build. Once it's compiled, your are done. Or if you hack the code, you do
- cmake -blah -foo -bar
- cmake -blah -foo -bar
- make
- <hack-hack-hack>
- make
- <hack--hack-hack>
- make

wkitty42 wrote in Mon Jun 26, 2017 8:25 pm:do you think you'll be able to find where, in the cmake files, to add the "-std=c++11" option?

I think I am too lazy for that :-)

Torsten wrote in Mon Jun 26, 2017 11:12 am: started to run terrafit over the entire globe. This crashed after a couple of hours.

And that is because terrafit has some serious memory leaks. Running it over the entire globe make the heap explode and the OS kills the program.
Simply restarting terrafit helps, it skips the already processed data and after a couple of restarts it went through.

Torsten
flightgear.org - where development happens.
User avatar
Torsten
 
Posts: 648
Joined: Fri Feb 01, 2008 10:22 pm
Location: near Hamburg, Germany
Callsign: offline
Version: next
OS: Linux

Re: Next World Scenery build

Postby D-ECHO » Tue Jun 27, 2017 8:27 am

Great news @Torsten!
D-ECHO
 
Posts: 2460
Joined: Sat May 09, 2015 1:31 pm
Pronouns: Bea (she/her)
Version: next

Re: Next World Scenery build

Postby wkitty42 » Tue Jun 27, 2017 4:53 pm

@Torsten: oh... i was asking so as to try to figure out how to adjust the download_and_compile.sh script... we should be able to add the define to the terragear build line but that doesn't seem to work... i really don't understand why cmake has to be run twice, either... it sounds almost like there's a recursion that gets half done the first time and finally completed the second time... my method of running the dnc script is to run configure every time and then let it do its cmake/make thing... in the configure section is where i see what flags are being used and the std one that was added is not showing up there...
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9148
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Next World Scenery build

Postby wkitty42 » Tue Jun 27, 2017 7:27 pm

i've added another cmake round to my dnc script and yes, building terragear goes further but it just doesn't make sense that cmake has to be run twice... here's the output from both cmake executions...
Code: Select all
1st execution /home/myuser/flightgear-dev/next/install/cmake/bin/cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX:PATH=/home/myuser/flightgear-dev/next/install/terragear -DCMAKE_PREFIX_PATH=/home/myuser/flightgear-dev/next/install/simgear; -DCMAKE_CXX_FLAGS=-std=c++11 ../../terragear/
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE 
-- Boost version: 1.54.0
-- Found the following Boost libraries:
--   system
--   thread
--   chrono
--   date_time
--   atomic
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8")
-- SimGear include directory: /home/myuser/flightgear-dev/next/install/simgear/include
-- found SimGear version: 2017.3.0 (needed 3.1.0 or higher)
-- looking for static SimGear libraries
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- found SimGear libraries
-- Performing Test SIMGEAR_COMPILE_TEST
-- Performing Test SIMGEAR_COMPILE_TEST - Success
-- Found SimGear: optimized;/home/myuser/flightgear-dev/next/install/simgear/lib/libSimGearScene.a;debug;/home/myuser/flightgear-dev/next/install/simgear/lib/libSimGearScened.a;optimized;/home/myuser/flightgear-dev/next/install/simgear/lib/libSimGearCore.a;debug;/home/myuser/flightgear-dev/next/install/simgear/lib/libSimGearCored.a;optimized;/home/myuser/flightgear-dev/next/install/simgear/lib/libSimGearCore.a;debug;/home/myuser/flightgear-dev/next/install/simgear/lib/libSimGearCored.a (Required is at least version "3.1.0")
-- Found GDAL: /usr/lib/libgdal.so (Required is at least version "2.0.0")
-- Found TIFF: /usr/lib/x86_64-linux-gnu/libtiff.so (found version "4.0.3")
-- Found CGAL 4.2
-- Build type: Release
-- USING CXXFLAGS = '-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2  -frounding-math -O3 -DNDEBUG'
-- USING EXEFLAGS = ' -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,--as-needed '
-- Targetting Unix Makefiles
-- Using /usr/bin/c++ compiler.
-- Requested component: Core
-- Requested component: MPFR
-- Requested component: GMP
-- Looking for unistd.h
-- Looking for unistd.h - found
-- Looking for sys/time.h
-- Looking for sys/time.h - found
-- Looking for windows.h
-- Looking for windows.h - not found
-- Looking for rint
-- Looking for rint - not found
-- Configuring done
-- Generating done
-- Build files have been written to: /home/myuser/flightgear-dev/next/build/terragear

Code: Select all
2nd execution /home/myuser/flightgear-dev/next/install/cmake/bin/cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX:PATH=/home/myuser/flightgear-dev/next/install/terragear -DCMAKE_PREFIX_PATH=/home/myuser/flightgear-dev/next/install/simgear; -DCMAKE_CXX_FLAGS=-std=c++11 ../../terragear/
-- Boost version: 1.54.0
-- Found the following Boost libraries:
--   system
--   thread
--   chrono
--   date_time
--   atomic
-- SimGear include directory: /home/myuser/flightgear-dev/next/install/simgear/include
-- found SimGear version: 2017.3.0 (needed 3.1.0 or higher)
-- looking for static SimGear libraries
-- found SimGear libraries
-- Performing Test SIMGEAR_COMPILE_TEST
-- Performing Test SIMGEAR_COMPILE_TEST - Success
-- Found CGAL 4.2
-- Build type: Debug
-- USING CXXFLAGS = '-std=c++11 -g'
-- USING EXEFLAGS = ' -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,--as-needed '
-- Targetting Unix Makefiles
-- Using /usr/bin/c++ compiler.
-- Requested component: Core
-- Requested component: MPFR
-- Requested component: GMP
-- Configuring done
-- Generating done
-- Build files have been written to: /home/myuser/flightgear-dev/next/build/terragear

it is just weird that the second cmake execution actually says that it is going to build a debug version and it is using the "-std=c++11" CXX flag whereas the first one doesn't seem to see that information in the options...

i've looked and looked and looked in the cmake files in terragear and just can't find anything that would even hint as to where to look :(

FWIW: it does make it a lot further but then cries about these...
Code: Select all
[ 51%] Building CXX object src/Prep/OGRDecode/CMakeFiles/ogr-decode.dir/ogr-decode.cxx.o
/home/myuser/flightgear-dev/next/terragear/src/Prep/OGRDecode/ogr-decode.cxx: In function ‘int main(int, char**)’:
/home/myuser/flightgear-dev/next/terragear/src/Prep/OGRDecode/ogr-decode.cxx:655:59: error: ‘GDAL_OF_VECTOR’ was not declared in this scope
     poDS = (GDALDataset*) GDALOpenEx( datasource.c_str(), GDAL_OF_VECTOR, NULL, NULL, NULL );
                                                           ^
/home/myuser/flightgear-dev/next/terragear/src/Prep/OGRDecode/ogr-decode.cxx:655:92: error: ‘GDALOpenEx’ was not declared in this scope
     poDS = (GDALDataset*) GDALOpenEx( datasource.c_str(), GDAL_OF_VECTOR, NULL, NULL, NULL );
                                                                                            ^
/home/myuser/flightgear-dev/next/terragear/src/Prep/OGRDecode/ogr-decode.cxx:667:29: error: ‘class GDALDataset’ has no member named ‘GetLayerByName’
             poLayer = poDS->GetLayerByName( argv[i] );
                             ^
/home/myuser/flightgear-dev/next/terragear/src/Prep/OGRDecode/ogr-decode.cxx:677:30: error: ‘class GDALDataset’ has no member named ‘GetLayerCount’
         for (int i=0;i<poDS->GetLayerCount();i++) {
                              ^
/home/myuser/flightgear-dev/next/terragear/src/Prep/OGRDecode/ogr-decode.cxx:678:29: error: ‘class GDALDataset’ has no member named ‘GetLayer’
             poLayer = poDS->GetLayer(i);
                             ^
make[2]: *** [src/Prep/OGRDecode/CMakeFiles/ogr-decode.dir/ogr-decode.cxx.o] Error 1
make[1]: *** [src/Prep/OGRDecode/CMakeFiles/ogr-decode.dir/all] Error 2
make: *** [all] Error 2

i don't know where further discussion of this should take place...
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9148
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Next World Scenery build

Postby Torsten » Tue Jun 27, 2017 7:29 pm

This line
Code: Select all
include ( ${CGAL_USE_FILE} )

overrides the CMAKE_CXX_FLAGS on the first run. Just guessing that it does some configuration for CGAL, writes the state to CMakeCache.txt and on the second run skips this configuration step (and the command line CMAKE_CXX_FLAGS get through). That's a bit creepy but it's working.

Here is another insight I just learned the hard way:
When running gdalchop for an entire continent like Eurasia, make sure to set the hard and soft limit for open files to 20,000 or more. This is because gdalchop opens each hgt file for reading before processing.
Code: Select all
ulimit -Hn 20000
ulimit -Sn 20000

You might need to increase the hard limit in /etc/security/limits.conf (as root)
flightgear.org - where development happens.
User avatar
Torsten
 
Posts: 648
Joined: Fri Feb 01, 2008 10:22 pm
Location: near Hamburg, Germany
Callsign: offline
Version: next
OS: Linux

Re: Next World Scenery build

Postby statto » Wed Jun 28, 2017 3:45 am

It doesn't make sense to me why gdalchop doesn't work procedurally, but perhaps it's faster.

Thank you for your work on this.

I have random land cover data for whenever the full and final run happens.
Custom Scenery available from http://www.stattosoftware.com/flightgear
statto
 
Posts: 2106
Joined: Fri Jan 25, 2008 10:57 pm

Re: Next World Scenery build

Postby wkitty42 » Wed Jun 28, 2017 6:04 pm

Torsten wrote in Tue Jun 27, 2017 7:29 pm:This line
Code: Select all
include ( ${CGAL_USE_FILE} )

overrides the CMAKE_CXX_FLAGS on the first run. Just guessing that it does some configuration for CGAL, writes the state to CMakeCache.txt and on the second run skips this configuration step (and the command line CMAKE_CXX_FLAGS get through). That's a bit creepy but it's working.

you're right... it is very creepy... i've been looking for anything i can find that identifies that file but have not been successful yet... i've grepped tons of files looking but nothing :(
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9148
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Next World Scenery build

Postby Nick R » Wed Jun 28, 2017 6:20 pm

Just grep everything recursively:
Code: Select all
grep -r pattern /path/to/grep/recursively
Hangar: fgpipistrel.org
Modelling the Pipistrel Virus SW (github, website)
Nick R
 
Posts: 173
Joined: Tue Nov 26, 2013 4:50 pm
Location: Stettler, AB, Canada
Callsign: NickR
Version: 2017.2.1
OS: Linux

Re: Next World Scenery build

Postby wkitty42 » Wed Jun 28, 2017 6:27 pm

i've done that, nick... no luck... no clue if it is in a .cmake, .txt, .in, or similar file and those are just the few that i have found related to cmake... some are in the build directory... some are in the project's source directory...
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9148
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Next World Scenery build

Postby Nick R » Wed Jun 28, 2017 6:31 pm

Where is CGAL installed? Have you tried the CMAKE_MODULE_PATH?
Hangar: fgpipistrel.org
Modelling the Pipistrel Virus SW (github, website)
Nick R
 
Posts: 173
Joined: Tue Nov 26, 2013 4:50 pm
Location: Stettler, AB, Canada
Callsign: NickR
Version: 2017.2.1
OS: Linux

Re: Next World Scenery build

Postby Torsten » Wed Jun 28, 2017 8:23 pm

I have added
Code: Select all
message(STATUS "CGAL_USE_FILE is ${CGAL_USE_FILE}")

just above
Code: Select all
include ( ${CGAL_USE_FILE} )

and my output is
Code: Select all
-- CGAL_USE_FILE is /usr/lib64/CGAL/UseCGAL.cmake


Torsten
flightgear.org - where development happens.
User avatar
Torsten
 
Posts: 648
Joined: Fri Feb 01, 2008 10:22 pm
Location: near Hamburg, Germany
Callsign: offline
Version: next
OS: Linux

Re: Next World Scenery build

Postby Torsten » Wed Jun 28, 2017 8:28 pm

statto wrote in Wed Jun 28, 2017 3:45 am:I have random land cover data for whenever the full and final run happens.

Good to know, I'll get back to you when I am starting with landcover.

I just fired genapts850 for our current apt.dat to check if it works and compare some self generated airports to our shipped ones.

What version of apt.dat would we use to build our next gen scenery from?

Torsten
flightgear.org - where development happens.
User avatar
Torsten
 
Posts: 648
Joined: Fri Feb 01, 2008 10:22 pm
Location: near Hamburg, Germany
Callsign: offline
Version: next
OS: Linux

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 9 guests