Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby montagdude » Thu Sep 03, 2020 8:57 pm

merspieler wrote in Thu Sep 03, 2020 3:12 pm:Yeah, I agree... in Feb - March it worked better, memory wise...

The commit I've posted above might be at fault.

Could you please checkout commit 30a43ac33855fd094a9fe680c9c2fad81c9eea79 and try again, building the entire tile.
If that works, please try again, this time with commit 61464a366939cd2e4af41574a6871fdcda3f9601.
I'd assume the first build succeedes and the second fails.

I successfully built the entire tile with 30a43ac33855fd094a9fe680c9c2fad81c9eea79. Didn't come close to using up all the memory. I will try next with 61464a366939cd2e4af41574a6871fdcda3f9601.

Don't have time to narrow down the problem area further, but here is the smallest box I got:
top=39.96875 left=-75.18750 bottom=39.93750 right=-75.12500

If you want to test it, first load this area into a database and then run osm2city on the tile with boundaries *-75.250_39.875_-75.000_40.000.
montagdude
 
Posts: 266
Joined: Tue Dec 31, 2019 7:04 am

Re: osm2city.py development

Postby merspieler » Thu Sep 03, 2020 9:16 pm

Can you give me the osm.pbf file, to make sure, I test on the very same dataset.
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: osm2city.py development

Postby montagdude » Thu Sep 03, 2020 9:26 pm

Sure, what's a good way to transfer it?
montagdude
 
Posts: 266
Joined: Tue Dec 31, 2019 7:04 am

Re: osm2city.py development

Postby montagdude » Fri Sep 04, 2020 12:59 am

montagdude wrote in Thu Sep 03, 2020 8:57 pm:
merspieler wrote in Thu Sep 03, 2020 3:12 pm:Yeah, I agree... in Feb - March it worked better, memory wise...

The commit I've posted above might be at fault.

Could you please checkout commit 30a43ac33855fd094a9fe680c9c2fad81c9eea79 and try again, building the entire tile.
If that works, please try again, this time with commit 61464a366939cd2e4af41574a6871fdcda3f9601.
I'd assume the first build succeedes and the second fails.

I successfully built the entire tile with 30a43ac33855fd094a9fe680c9c2fad81c9eea79. Didn't come close to using up all the memory. I will try next with 61464a366939cd2e4af41574a6871fdcda3f9601.

Don't have time to narrow down the problem area further, but here is the smallest box I got:
top=39.96875 left=-75.18750 bottom=39.93750 right=-75.12500

If you want to test it, first load this area into a database and then run osm2city on the tile with boundaries *-75.250_39.875_-75.000_40.000.

The tile also completes with commit 61464a366939cd2e4af41574a6871fdcda3f9601, so I don't think that's the problematic commit.
montagdude
 
Posts: 266
Joined: Tue Dec 31, 2019 7:04 am

Re: osm2city.py development

Postby montagdude » Fri Sep 04, 2020 1:23 am

Philadelphia looks nice when the tile actually completes, although I'm skeptical about that one really tall thin building.

Edit: actually, it's the right height to be the Comcast Technology Center, it's just not quite the right shape. So that's okay.

Image
montagdude
 
Posts: 266
Joined: Tue Dec 31, 2019 7:04 am

Re: osm2city.py development

Postby merspieler » Fri Sep 04, 2020 8:46 am

My scenery near YMEL has a too tall building... it tops out at FL150 (ground is almost sea level :D
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: osm2city.py development

Postby merspieler » Fri Sep 04, 2020 10:45 am

Bad news... that tile of yours went as well over 75GB (before I killed it) on the current version... so there's defenetly something wrong
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: osm2city.py development

Postby montagdude » Fri Sep 04, 2020 12:02 pm

Was that with this latest commit from yesterday?

Code: Select all
commit edc2dde5a05b83fdf18742c2248c426bfd88a96d (origin/master, origin/HEAD, master)
Author: Rick Gruber-Riemer <rick@vanosten.net>
Date:   Thu Sep 3 20:04:13 2020 +0200

    Possible contribution to solve https://gitlab.com/osm2city/osm2city/-/issues/140. Missed removal of reference
montagdude
 
Posts: 266
Joined: Tue Dec 31, 2019 7:04 am

Re: osm2city.py development

Postby merspieler » Fri Sep 04, 2020 12:27 pm

nope, I didn't include that yet... will re run it

This would be a good resource for tracking down those memory leaks:
https://medium.com/zendesk-engineering/ ... 24d0518774
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: osm2city.py development

Postby merspieler » Fri Sep 04, 2020 2:42 pm

It succeeded... the fix works, thanks @vanosten
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: osm2city.py development

Postby montagdude » Fri Sep 04, 2020 2:52 pm

Great, then I will build my stuff with the latest master. I'm going to start over since I had to redo some TerraGear stuff anyway.

I agree, thanks @vanosten.
montagdude
 
Posts: 266
Joined: Tue Dec 31, 2019 7:04 am

Re: osm2city.py development

Postby montagdude » Tue Sep 08, 2020 12:06 pm

It's strange, I'm still getting an issue with this same tile using the latest master. I've done most of the state of Pennsylvania now, and this is the only place where this or any other exception has occurred. As I mentioned in a previous post, this tile also built successfully with commit 61464a366939cd2e4af41574a6871fdcda3f9601 from May.

Code: Select all
******* Exception in tile 1712251 - to reprocess use boundaries: -75.25_39.875_-75.0_40.0 ******* at 2020-09-07_104413 - 
Traceback (most recent call last):
  File "/home/dello/Projects/FGScenery_tools/osm2city/build_tiles.py", line 143, in process_scenery_tile
    buildings.process_buildings(the_coords_transform, my_fg_elev, blocked_apt_areas, my_stg_entries,
  File "/home/dello/Projects/FGScenery_tools/osm2city/osm2city/buildings.py", line 1129, in process_buildings
    write_buildings_in_meshes(coords_transform, buildings_in_meshes, stg_manager, stats)
  File "/home/dello/Projects/FGScenery_tools/osm2city/osm2city/buildings.py", line 1034, in write_buildings_in_meshes
    building_lib.write(os.path.join(path_to_stg, file_name + ".ac"), cl.objects,
  File "/home/dello/Projects/FGScenery_tools/osm2city/osm2city/building_lib.py", line 1766, in write
    b.write_to_ac(ac_object, cluster_elev, cluster_offset, roof_mgr, face_mat_idx, roof_mat_idx, stats)
  File "/home/dello/Projects/FGScenery_tools/osm2city/osm2city/building_lib.py", line 1267, in write_to_ac
    number_prev_ring_nodes += self._write_faces_for_ac(ac_object, self.polygon.exterior,
  File "/home/dello/Projects/FGScenery_tools/osm2city/osm2city/building_lib.py", line 1313, in _write_faces_for_ac
    tex_coord_bottom, tex_coord_top = _calculate_vertical_texture_coords(self.body_height, self.facade_texture)
  File "/home/dello/Projects/FGScenery_tools/osm2city/osm2city/building_lib.py", line 1598, in _calculate_vertical_texture_coords
    if t.v_can_repeat:
AttributeError: 'NoneType' object has no attribute 'v_can_repeat'


I can submit the osm.pbf, script, and params.ini if that would help debug.
montagdude
 
Posts: 266
Joined: Tue Dec 31, 2019 7:04 am

Re: osm2city.py development

Postby rustemy » Tue Sep 08, 2020 3:21 pm

Have you stumbled on the now legendary "Melbourne Monolith"?

Image

There is/was an issue with data from OpenStreetMap that has also caught out Microsoft.

There is an article on engadget.com that explains what happened. A typo created a 212-story monolith in ‘Microsoft Flight Simulator’
rustemy
 
Posts: 24
Joined: Fri Jun 21, 2013 2:15 pm
Version: PPA-Edge
OS: Linux Mint 20.2 Uma

Re: osm2city.py development

Postby merspieler » Tue Sep 08, 2020 4:24 pm

montagedude: Know issue, see: https://gitlab.com/osm2city/osm2city/-/issues/160

rustemy:
I know... in the fg scenery it's root top is at FL150
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: osm2city.py development

Postby vanosten » Sat Sep 19, 2020 9:50 am

StuardB has provided a patch with some heuristics to make smaller buildings in the same area look similar - for building lists only (FLAG_STG_BUILDING_LIST = True). It seems to work quite well in the UK and I have a bit mixed results in Switzerland.

If you want to give it a try, then set the new parameter BUILDING_TEXTURE_GROUP_RADIUS_M to something like 2000.

PS: I created https://gitlab.com/osm2city/osm2city/-/issues/162, because something similar should also work for buildings in meshes etc. Instead of distance real topology data from landuse.py might be needed.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 3 guests