feat(maps): Consolidating all country maps (and TS) into the Jupyter notebook workflow.

[rusackas]

SUMMARY

Ok, so our little world of maps is a a heterogenous mess :) Time to clean house! This PR does the following:

• Makes various updates to the Jupyter notebook to make it run reliably (there were some deprecated method and whatnot)
• Adds a few countries (Afghanistan, Albania, Algeria) as a test
• Takes stock of the maps in Superset that were NOT part of the notebook, and adds them
  • many countries, like Austria, existed in geojson, but were not part of the notebook. Now they're generated by the notebook (which might have unintended consequences)
  • a couple of countries (france_regions and italy_regions, namely) can't be made by the notebook. These are now copied over by the notebook from a new directory of "custom" geojson files at run time
• geojson files now consistently use underscores instead of spaces in filenames
• The typescript file that creates the select menu is now generated automatically. You can pretty much just add a country name to the notebook and let it rip to open a new country PR.
• Some countries updated by the notebook (e.g. Sri Lanka) have notably different characteristics - these might generate some breaking changes unintentionally, but seem more correct, and better fit Superset's patterns.
• Note: The france_regions map seems to be causing a browser meltdown before or after this PR. I'm not sure what's up with that.

TODO (in this PR or subsequent ones)
• Figure out what's broken with the france_regions map
• Add documentation about this notebook to the CONTRIBUTING file
• Add codeowners to the relevant files to prevent more diverging maps
• Add more countries!

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

I built a dataset of every airport in the world, and went through most, if not all, of the new maps. All work well except Sri Lanka, which I haven't gotten working with any dataset... there's something strange about their region codes.

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (863f8bb) 69.18% compared to head (f69849a) 69.17%.
Report is 216 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #26300      +/-   ##
==========================================
- Coverage   69.18%   69.17%   -0.01%     
==========================================
  Files        1946     1949       +3     
  Lines       75988    80251    +4263     
  Branches     8479     9732    +1253     
==========================================
+ Hits        52570    55514    +2944     
- Misses      21228    22320    +1092     
- Partials     2190     2417     +227     

Flag	Coverage Δ
hive	53.68% <ø> (ø)
javascript	57.63% <100.00%> (+1.10%)	⬆️
mysql	78.08% <ø> (ø)
postgres	78.17% <ø> (ø)
presto	53.63% <ø> (ø)
python	82.85% <ø> (ø)
sqlite	76.83% <ø> (ø)
unit	55.80% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rusackas]

/testenv up

[github-actions]

@rusackas Ephemeral environment spinning up at http://52.37.45.250:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

[villebro]

As a Finn this caught my eye - apparently Åland Islands has been replaced by Åland in the dataset. Can you replace the following snippet:

finland_aland = df_admin0_10m.loc[
    df_admin0_10m.name_en.isin(['Åland Islands']),
    [x for x in df_admin0_10m.columns if x in df.columns]
]
finland_aland = finland_aland.merge(pd.DataFrame(
    data={
        "name_en": ["Åland Islands"],
        "name_fi": ["Ahvenanmaan maakunta"],
        "iso_3166_2": ["FI-01"],
    },
), on="name_en", how="left")

with

finland_aland = df_admin0_10m.loc[
    df_admin0_10m.name_en.isin(['Åland']),
    [x for x in df_admin0_10m.columns if x in df.columns]
]
finland_aland = finland_aland.merge(pd.DataFrame(
    data={
        "name_en": ["Åland"],
        "name_fi": ["Ahvenanmaan maakunta"],
        "iso_3166_2": ["FI-01"],
    },
), on="name_en", how="left")

This will ensure that the Finnish southwestern archipelago looks correct.

Before:

After:

[rusackas]

@villebro done!

[qleroy]

@rusackas

About france_regions the meltdown is due to the projection used, I added detailed explanation in the PR that added this #25676

and @dmeaux proposed a fixed geojson file #26848

[qleroy]

To consolidate the geojson generation with the notebook,
I suggest to explicitly project the data with the WGS84 (EPSG:4326) project so that it is consistent,
https://geopandas.org/en/stable/docs/reference/api/geopandas.GeoDataFrame.to_crs.html

This could also be a step in the documentation to note that it is required to provide files projected with WGS84

[qleroy]

Also, I would love that we include overseas islands for France !
I don't understand why there were removed in the first place
while the flying islands were moved closer for other countries.

[villebro]

LGTM!

[villebro]

@qleroy would you be open to contributing the floating islands for the French maps? Also the other improvement ideas you presented. That would be awesome! 🇫🇷

[rusackas]

Happy to keep iterating on this, and indeed would love help with any of it! Meanwhile, let's merge this swath of it, so others can fork and contribute :) Thanks!

[github-actions]

Ephemeral environment shutdown and build artifacts deleted.

[qleroy]

@villebro @rusackas
I just updated the France map to include overseas islands !
#26995

[rusackas]

Love it! Thanks for the follow-through @qleroy

[zufolo441]

Hi all,
is there a way to view deck.gl polygons same way as country maps, without any background map?
I used to use country maps, but now I have some maps in PostGIS database and they show in different ways....

thanks!