osm2city.py development

[pb321]

Ah...OK. Thank you, @wkitty42. This is helpful information.

[ivan.baricic]

First, .. thanks a lot for this marvelous tool. This is my very first post in the forum, .. after hours playing with the FlightGear, .. I would like to contribute with something to this great product too, .. not only consume it. The learning curve at the very beginning is a bit long (shallow), I mean, .. to get oriented in all the forums, Wikis, source code repositories etc, .. but after a few houres spent here, .. I feel, I am oriented towards the right direction

My environment (where I play and "develop"):

• Xubuntu 17.10

Successfully managed to:

• download and compile the latest (2017.4) release of the FlightGear (thanks to the download_and_compile.sh script from the SourceForge).
• enhance the Phi user interface with new map (https://www.opentopomap.org; Widgets, Map.js) (which I favour over the default OSM one).
• download offline the scenery tiles using TerraMaster for my region of interest
• "install" the suite of the osm2city.py scripts, following the great documentation (http://osm2city.readthedocs.io/en/latest/index.html), including
  
  • PostgreSQL 9.6 (with pgAdmin3)
  • Osmosis
• download the OSM (*.pbf) data from http://download.geofabrik.de/ and process them with Osmosis.
• run successfully the generation scripts (build_tiles.py etc) using excerpts from the data, for the region inside of Tunisia and Algeria (where I am "flying"; and where I found the scenery really very sparse).

Found a minor bug in the https://gitlab.com/fg-radi/osm2city/blob/master/pylons.py (process_pylons) when C2P_PROCESS_POWERLINES == True && C2P_PROCESS_POWERLINES_MINOR == False && C2P_PROCESS_AERIALWAYS == False (there are no req_keys set to osmparser.fetch_osm_db_data_ways_keys(req_keys) => wrong SQL query in osmparser.fetch_db_way_data). I would fix it, but I do not know, what is to-do in this case.

Some suggestions, which tools do you use / favour for debugging the Python code, under Linux, are welcome.
There are a plenty of them, I know, .. but I am a "newbie" in this area, .. I use to develop mostly on Windows (Eclipse, Java, C#, databases, GIS, etc).
Nowadays I use only the Sublime Text editor with some plugins on Ubuntu. Thanks!

[paju1986]

I'm regenerating my spain scenary with the latest updates and improvements. let's see how it goes

[vanosten]

Found a minor bug in the https://gitlab.com/fg-radi/osm2city/blob/master/pylons.py (process_pylons) when C2P_PROCESS_POWERLINES == True && C2P_PROCESS_POWERLINES_MINOR == False && C2P_PROCESS_AERIALWAYS == False (there are no req_keys set to osmparser.fetch_osm_db_data_ways_keys(req_keys) => wrong SQL query in osmparser.fetch_db_way_data). I would fix it, but I do not know, what is to-do in this case.

I will look into it between x-mas and new year. There is just a lot of stuff going on at work and I am in the middle of some other changes in osm2city.

Some suggestions, which tools do you use / favour for debugging the Python code, under Linux, are welcome.

If you do not mind using commercial tools, then I would suggest to start with the community edition of PyCharm.

[paju1986]

I found the same exception and a few more when regenerating spain, i have a work around for most of those and i am testing now. when done i will post the code that i changed

[paju1986]

OK... Spain scenery regenerated. I found some exceptions on the way.

First, a typo on a variable name on pylons.py line 363. it should be:

Code: Select all

self.pylon_model = 'generic_chimney_01.xml'

Second, the bug ivan.baricic was talking about, my workaround is to add this line on the line 1875 of pylons.py

Code: Select all

req_keys.append('power')

Thrid. a division by zero error on

Code: Select all

tex_y12 = self.facade_texture.y((self.body_height + self.roof_height_pts[i]) /
                                                self.body_height * tex_coord_top_input)


of buildings_lib.py workaround is to add:

Code: Select all

if (self.body_height) == 0:
                self.body_height = 1
            if (tex_coord_top_input == 0):
                tex_coord_top_input = 1

before of that line.

and fourth. parsing strings into float errors because some weirds tags on method _parse_building_levels of buildings_lib.py. workaround capture the exceptions:

Code: Select all

def _parse_building_levels(tags: Dict[str, str]) -> float:
    proxy_levels = 0.
    if 'building:levels' in tags:
        if ';' in tags['building:levels']:
            try:
                proxy_levels = float(utils.osmparser.parse_multi_int_values(tags['building:levels']))
            except ValueError:
                proxy_levels = 0
        else:
            try:
                proxy_levels = float(tags['building:levels'])
            except ValueError:
                proxy_levels = 0
    if 'levels' in tags:
        if ';' in tags['levels']:
            try:
                proxy_levels = float(utils.osmparser.parse_multi_int_values(tags['levels']))
            except ValueError:
                proxy_levels = 0
        else:
            try:
                proxy_levels = float(tags['levels'])
            except ValueError:
                proxy_levels = 0
    return proxy_levels


Regards!

[vanosten]

Could you please submit the osm2city-exceptions.log?

[paju1986]

Sorry but I deleted it

[vanosten]

The newest version of osm2city has a new flag BUILDING_USE_SHARED_WORSHIP, which places worship buildings using shared objects fro the scenery database at the right place and as far as possible right angle / right size. There are a few geometry glitches, but it looks ok right now. Unfortunately the existing shared objects could use a bit of improvement and cover basically exclusively christian (catholic / protestant) buildings.

The idea is that places of worship tend to be landmarks and therefore might look more authentic with shared objects than buildings with generic textures. Og course if there is a static object, then even better.

A generated scenery at Lucerne / Switzerland looks like follows (4 churches):

[paju1986]

Great job, but why the textures of those buildings are just a blank color with no windows or something?

[vanosten]

Great job, but why the textures of those buildings are just a blank color with no windows or something?

Because the shared models are not modelled with a nicer texture. If someone improves the shared models, then they will also look nicer when used in scenery (no matter whether standard scenery or osm2city).

[paju1986]

Hello.

When I regenerated spain I found weird road "spikes" in many places, any ideas?














Parameters used:

PREFIX = "spain"
PATH_TO_SCENERY = "/media/alberto/00CCD25B5B0F574C/osm2city/fgfs_terrasync"
PATH_TO_OUTPUT = "/media/alberto/00CCD25B5B0F574C/osm2city/fg_customscenery/spain"
PATH_TO_OSM2CITY_DATA = "/media/alberto/00CCD25B5B0F574C/osm2city/development/osm2city-data"

BOUNDARY_WEST = -9.779014
BOUNDARY_EAST = 5.098525
BOUNDARY_NORTH = 44.14855
BOUNDARY_SOUTH = 35.154153

NO_ELEV = False
FG_ELEV = '/usr/games/fgelev'
FG_ELEV_CACHE = False

DB_HOST = "localhost"
DB_PORT = 5432
DB_NAME = "pgsnapshot"
DB_USER = "alberto"
DB_USER_PASSWORD = "alberto"
OVERLAP_CHECK_CONVEX_HULL=True
OVERLAP_CHECK_CH_BUFFER_STATIC=20
POINTS_ON_LINE_DISTANCE_MAX=20
PROBE_FOR_WATER=True
TEXTURES_REGIONS_EXPLICIT = ["de", "gb", "us"]
FLAG_2017_2 = True
BUILDING_FORCE_EUROPEAN_INNER_CITY_STYLE=True
MAX_SLOPE_ROAD=0.15

[vanosten]

No, I have not seen that kind of "spikes". I know that there are bugs in roads.py - e.g. sometimes the texture is flipped upside down in the middle.

The most hated one by myself currently is that often on some parts of roads the texture appears to be darker, like it is upside-down - happens a lot for highways when the one side is fine and the opposite direction is darker. Unfortunately I have not been able to resolve the problem, because at least to me the normals look correct.

My focus currently (apart from actually flying one in a while) is to get the texture-map for buildings changed, such that repeating of textures is not necessary and I could start using larger texture maps. I.e. unless someone can point to an obvious error/root cause, I will probably not actively look at roads.py for many weeks to come.

I someone like you would be interesting to help out, then I would be very glad to use time on (voice) chat to introduce to how the code works.

That being said: could you please tell me the coordinates for the pictures above in lat/lon decimals (e.g. using the information in the lower left corner of https://scenery.flightgear.org/map/)?

[vanosten]

I have added a new command line argument -o to enable saving logging output to a file instead of just stderr. See the docs for more information.

I also added an optional command line argument -m NUMBER, which defines the maximum number of tasks per worker. If you see resource problems (e.g. RAM), which you had not before, then please let me know and set -m 1 to mimic the previous behavior (I have not tested on Windows).

[radi]

My focus currently (apart from actually flying one in a while) is to get the texture-map for buildings changed, such that repeating of textures is not necessary and I could start using larger texture maps.

Vanosten,

I haven't been active in a while, so my vote probably counts little... just out of curiosity, why do you want to get rid of the repeating textures feature, and what currently limits texture size? When I initially designed how textures would be used, I specifically wanted to have repeating textures (obviously), because some buildings in my area were really long and therefore called for repeating textures, and so do very tall buildings. I never came across actual downsides of this approach.

(I kind of feel I've asked this before; please excuse my fading memory )