Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby merspieler » Wed Aug 07, 2019 6:09 pm

I've got trouble with the latest version as well.

Code: Select all
SpawnPoolWorker-1 root INFO     Time used in seconds for processing aerodromes: 0.007571
^CTraceback (most recent call last):
  File "../../osm2city/build_tiles.py", line 301, in <module>
    pool.join()
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 556, in join
    self._worker_handler.join()
  File "/usr/lib/python3.7/threading.py", line 1044, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.7/threading.py", line 1060, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt


After `processing aerodromes` it consumes 100% CPU but doesn't continue with any work. Looking at the back trace it seems to me like something wants to get a lock but doesn't get it.

Version I've reverted to for now is e9a1885c55431c830930dd2a057111fe8ced12ec
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2228
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: osm2city.py development

Postby vanosten » Sat Aug 31, 2019 4:41 pm

merspieler wrote in Wed Aug 07, 2019 6:09 pm:I've got trouble with the latest version as well.
..
After `processing aerodromes` it consumes 100% CPU but doesn't continue with any work. Looking at the back trace it seems to me like something wants to get a lock but doesn't get it.
...


There seems to be only a problem with a particular airport, which uses a different layout than default. No problems discovered anywhere else. Otherwise please tell me exactly what is going on.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Sat Aug 31, 2019 4:50 pm

Over the last weeks plus the final today I have pushed changes to how roads and railways are processed. The result in general should be less z-fighting at junctions (visible stuff) plus better logic for z-separation between ways (code stuff).

If you find that things got worse, then please tell me exactly where and how.

PS: there are IMO still two major annoyances:
* The roads do not take the underlying digital elevation model (DEM) into account - elevation probing is done at "random" nodes based on OSM data instead of at edges of the DEM's triangle. This leads to parts of roads being under the terrain. The solution is to put nodes where the OSM way and the DEM triangles intersect.
* Where the terrain has a certain rate of change in steepness, the smoothing of the ways is not smooth enough. The solution to this is IMO several passes through the network graph.

If someone would try to do that, please PM me.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby saul » Sat Aug 31, 2019 8:14 pm

Hi Vanosten,

I have tested the last commit on Arlanda airport in Sweden. I still have the same issue that I reported before with terminals, parking lots, and other buildings not being generated. The issue is strange because there are some buildings that are in fact created as you can see in the pictures below. As far as I can tell, the only thing I have not done from the instructions is to index the tags in hstore. The version of FG that I am using is 2019.1.1 (stable).

https://imgshare.io/image/iXJPe
https://imgshare.io/image/iXHOw

Best regards,
Saul
saul
 
Posts: 40
Joined: Tue Nov 26, 2013 10:57 pm

Re: osm2city.py development

Postby vanosten » Sun Sep 01, 2019 7:50 am

Thanks for the pictures and the airport - now I have something to play with, once I get time.

Unfortunately Project3000 (http://wiki.flightgear.org/Project3000) has been left alone for quite some time and not been ported to Python 3 (I have talked to the owner). I would love to take some of the code and merge it into osm2city, so airports generically can be handled better. That would be a good project for someone to help with osm2city.

I introduced a new parameter OVERLAP_CHECK_PAVEMENT, which is by default set to False. Overlap checking for buildings is only done for runways and helipads, not for pavements. Please test and show pictures no matter whether it works or not.

Not even if investing a lot of hours into code and making it more complicated will solve the issue of FG scenery and OSM not being a perfect match. And I believe that even rudimentary terminal buildings created for FG scenery will look better after few hours of work than osm2city buildings).

PS: I never do indexing. The time it takes to index the database you might not even get in speedup later.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby legoboyvdlp » Sun Sep 01, 2019 3:28 pm

I ported dsf2stg to Py3, not P3000 yet. Want me to send the code?
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: osm2city.py development

Postby saul » Tue Sep 03, 2019 10:22 pm

Hi Vanosten,

Many thanks for the change! Now I can see the terminal buildings, control tower, etc. Here below are a couple of pictures with OVERLAP_CHECK_PAVEMENT = False:

https://imgshare.io/image/ylZIF
https://imgshare.io/image/ylEH9

For reference, I also run the scripts with OVERLAP_CHECK_PAVEMENT = True:

https://imgshare.io/image/ylM2n
https://imgshare.io/image/yl3KY

The buildings are there, but the streets that lead to the terminals and parking lots are still missing. Maybe is the same issue?

Thanks again and best regards,

Saul
saul
 
Posts: 40
Joined: Tue Nov 26, 2013 10:57 pm

Re: osm2city.py development

Postby vanosten » Sat Sep 07, 2019 7:46 am

I replaced parameter OVERLAP_CHECK_PAVEMENT with OVERLAP_CHECK_PAVEMENT_BUILDINGS_INCLUDE and OVERLAP_CHECK_PAVEMENT_ROADS_INCLUDE. I.e. unless you add a specific ICAO code to the list of one of the parameters, there will be no overlap check against pavement (e.g. APRONs). I hope this gives enough control with sensible defaults in most situations.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby saul » Wed Sep 11, 2019 8:53 pm

Many thanks!

I have completed the processing of Sweden without issues. I checked the largest airports in the country and Wow! Now the airports look nicer and full of life! I will be updating the Sweden repository soon.

I have one question. I want to process the whole Scandinavia (should be around 10 GB when finished). What is the best option?
1) Create a database and push the osm data of all the countries in it so that at the end a single directory with all the objects are created
2) Create each country in a separate directory and merge the objects in a single directory.
3) Create each country in a separate directory and just pass them as scenery options when starting fg.

I ask this because I dont know what would happen if two countries create files for the same locations (the rectangles during processing will certainly overlap areas of neighbor countries).
saul
 
Posts: 40
Joined: Tue Nov 26, 2013 10:57 pm

Re: osm2city.py development

Postby vanosten » Thu Sep 12, 2019 7:38 pm

I never tried. But my recommendatiin would be to merge osm files with OSMOSIS, then upload the merge into a database and then generate the whole thing. Generally you need to think in and respect tile boundaries.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Fri Oct 25, 2019 4:05 pm

I have pushed some changes, which among others should improve the way roofs are handled, especially in European style cities.

With the help of Stuart we are now much closer to having the majority of buildings displayed as shader buildings, which leads to better visuals, faster loading and less disk space requirements. You can give it a try by installing the latest FlightGear nighties and having a look at Northern Holland (e.g. EHAM) in the Netherlands: https://my.pcloud.com/publink/show?code ... iQ1RLM2JCy (ca. 440 zip, ca. 2 GB on disk).

(Yes, the shader buildings look better than the other buildings. I am still looking for someone to help with better textures for osm2city.)
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby wkitty42 » Sat Oct 26, 2019 12:33 am

i tried this but something in Buildings is causing my FG to crash...

Code: Select all
Error: caught signal 11:
  /lib/x86_64-linux-gnu/libc.so.6 : ()+0x36cb0
  ./fgfs : std::vector<std::string, std::allocator<std::string> >::operator[](unsigned long) const+0x10
  ./fgfs : simgear::SGBuildingBin::SGBuildingBin(SGMaterial const*, bool)+0x53
  ./fgfs : simgear::SGBuildingBin::SGBuildingBin(SGPath const&, SGMaterial const*, bool)+0x4f
  ./fgfs : simgear::ReaderWriterSTG::_ModelBin::DelayLoadReadFileCallback::readNode(std::string const&, osgDB::Options const*)+0x5e6
  ../../openscenegraph/lib/libosgDBd.so.161 : osgDB::Registry::readNode(std::string const&, osgDB::Options const*, bool)+0x9b
  ../../openscenegraph/lib/libosgDBd.so.161 : osgDB::DatabasePager::DatabaseThread::run()+0xd0f
  ../../openscenegraph/lib/libOpenThreadsd.so.21 : OpenThreads::ThreadPrivateActions::StartThread(void*)+0xcd
  /lib/x86_64-linux-gnu/libpthread.so.0 : ()+0x8184
  /lib/x86_64-linux-gnu/libc.so.6 : clone()+0x6d

i know it is Buildings because i removed all of the directories and put them back one at the time until it crashed again...

i note that in buildings there are .txt files and there's also reference in the ac files to a Textures directory that i don't find but i may be looking in the wrong place for them...
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9123
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: osm2city.py development

Postby slawekmikula » Sat Oct 26, 2019 7:24 am

Same here. Sorry for little more info. Graphics card: Intel. In fg log nothing substantial.
slawekmikula
 
Posts: 128
Joined: Sun Feb 19, 2017 10:31 am

Re: osm2city.py development

Postby vanosten » Sat Oct 26, 2019 8:15 am

Could you please do https://osm2city.readthedocs.io/en/late ... perimental and tell me, whether it helps. Probably we should add it as default to FGData.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby slawekmikula » Sat Oct 26, 2019 9:07 am

Yes, that was it (at least in my case). Now FG loads up correctly without crash.

Scenery looks nice. Don't know if it is exact urban density but some screens. Intel card, UFO.

Image

Image

One issue (or some observation). I'm flying with UFO. In some areas there lights on the buildings are not everywhere as there is some shadows from the clouds. But when moving with the UFO the shadow (or parts of buildings which ale lit) is moving with the UFO position as UFO is casting a shadow (or is light emitter). Last screen shows areas with different light value across buildings in the view. Sorry, don't know how more precise describe this in english ;) For me (YMMV) it looks strage and I was looking what caused the shadow.

Image
slawekmikula
 
Posts: 128
Joined: Sun Feb 19, 2017 10:31 am

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 3 guests