Data Publishing

Local Ecosystem Accounts, Finskala Naturregnskap, map results will be published in the dataset Finskala Naturregnskap on NINA’s map catalogue for curated maps. To publish maps, the data manager of the project should submit the import schema to Miljødata (see the step-by-step guide).

The Local Ecosystem Accounts Dataset is structured as follows: maps are categorized by project, and within each project, map layers are further organized according to the components of the Ecosystem Accounting Framework:

Curated Maps represent a collection of maps that are carefully selected and organized. NINA’s map catalogue is designed to facilitate the sharing of your final map results with both internal and external users. It’s important to note that this portal is not intended for storing interim or temporary data products.

If you have feedback on the map portal, please raise an issue in the GitHub repositories or contact the developers. Links to repositories can be found under Guide for Developers.

Step-by-step guide for submitting data

  1. Organize your data
  • Create a folder nina_map_catalog within your project folder (P:/)
  • Group your data into Ecosystem Accounting Framework groups: services, extent, condition, misc.
  • Raster data:
    • save each raster as a single-band GeoTIFF into the nina_map_catalog folder.
    • use underscore as spaces (air_pollution_0301.tif)
  • Vector data:
    • save all vector layers of one group into one GeoPackage and save it in the nina_map_catalog folder.
    • %project_name%_service.gpkg
    • %project_name%_extent.gpkg
    • %project_name%_condition.gpkg
    • %project_name%_misc.gpkg
  1. Fill out the import schema

    • Download import schema
    • Ensure to read the README-sheet and fill out all required (*) fields.
    • Important: Check the data source of each data layer. If the data source is not owned by NINA, provide the WMS link to the original source and the link of where users can download this data (for example GeoNorge). For example, we do not add AR5 data into the gpkg’s! Instead we show the map as a WMS layer and point the users to NIBIO for downloading the data.
  2. Layer styling

    • save your vector layer style inside your Geopackage.
    • save your raster colormap inside a .txt file: “datasetName-style.txt”

    Note: it is recommended to save your layer style inside your Geopackage using QGIS or create a QGIS style file. It is also possible to share your style using another file format (e.g. arcgis pro style file, yaml- python- or R-script with styling info.) We try to make the styling as similar as possible, but know that the MapLibre library we use is not as advanced as for example ArcGIS Pro. So customized colormaps might look slightly different in the NINA Maps Portal.

  3. Request for publishing a new project within the EA NINA map portal

    • Send the following information to Miljødata:
      • the filled out import-schema
      • the path to your cleaned and organized data: P:/…/nina_map_catalog
      • a qgis file, a arcgispro layer file, an agol app, an R or python file that displays your preferred styling of the map layers.

Example structure of nina_map_catalog folder:

P:/.../nina_map_catalog
├── NINA_naturregnskap_metadata_importskjema_PROSJEKTNAVN.xlsx    (import schema)
├── *%project_name%*_layer_styling.qgz                            (qgis project with preferred styling)
├── *%project_name%*_condition.gpkg                               (vector layers grouped by EA framework)  
├── *%project_name%*_extent.gpkg
├── *%project_name%*_misc.gpkg
|    ├── study_area
|    └── vector_layer
├── *%project_name%*_service.gpkg
├── air_pollution_0301.tif                                        (single band raster layers)
└── raster_layer.tif

Guide for developers

Developers:

Documentation: https://ninanor.github.io/nina-catalogue

NINA’s catalogue includes a Map Module for storing maps, accesible at maps.nina.no, and a Datasets Module for storing data and metadata, accesible via PyGeoAPI. For the Local Ecosystem Accounts dataset (Finskala Naturregnskap), maps are uploaded to the Map Catalogue. In the future, these maps may also be uploaded as a dataset thus be accesible via the PyGeoAPI.

Below is a brief guide for setting up a local installation of the Catalogue on your machine. For detailed information and inquiries, please refer to the documentation or contact the developers. This is meant for development and testing, the production environment is managed by Niccolò Cantù.

Installation instructions

Prerequisites:

  • Linux OS
  • Docker

GitHub repositories:

Repository Description Technologies
nina-catalogue Backend architecture Django and PostgreSQL
nina-map-explorer Frontend map application MapLibre with PM Tiles
naturregnskap_maps_uploader API, specifically designed for uploading map data and metadata to the Local Ecosystem Accounts dataset using the import schema. Python

Installation steps:

  1. Install the backend architecture

    • Follow the instructions in the README of the nina-catalogue
      • Create secrets in the secrets folder

        cd setup
        docker compose up --build
        cd ..
      • start all services with the map profile

        # source the helper functions
        . helpers.sh
        
        # alias in helpers.sh
        # run the docker-compose file in the dev profile
        # env variables are loaded from secrets 
        dpcli_maps up --build
      • an admin user is already created with the following credentials

        email: admin@nina.no
        password: admin            

        These can be used to login to the Django admin: localhost:8000/admin/

      • start the Django admin interface

        xdg-open "http://$(hostname):8000/admin"
      • generate a token for your user in Django admin GUI: Tokens -> Add token


Django Admin Interface
Django Admin GUI

To be able to display rasters in the browser it’s necessary to use instead of localhost your VDI’s hostname. Run hostname in your terminal and replace localhost with this value, also when opening it in the browser.

  1. Open the frontend map application

         ```{bash}
         xdg-open "http://$(hostname):8000"
         ```

Errors in the web application can be inspected in the browser console using: right-click -> inspect -> console or ctrl+shift+j.

  1. Install the Naturregnskap API

    • Follow the instructions in the README of the naturregnskap_maps_uploader
      • Install the Python environment

          ```{bash}   
          python3 -m venv .venv
          source .venv/bin/activate
        
          pip install -e .
        
          touch .env
          ```
      • Configure the API with the Django backend

        • Copy the token generated in Django admin (step 1)
        • Add the token to the .env file
        AUTH_TOKEN=my_token
        DEBUG=True

Now you have a local installation of the NINA Catalogue and the following services are available:

Service URL
Metadata catalogue localhost:8000
Django admin localhost:8000/admin/
API documentation localhost:8000/api/docs/
Map application localhost:8000
Documentation localhost:3001
TiTiler documentation localhost:8000/titiler/

How to create a map and upload data to the NINA Catalogue

Below are the steps to create a map for the Local Ecosystem Accounts dataset and upload the data to the NINA Catalogue from scratch. You only need to create a map once, and then you can upload data to the same map multiple times using the API.

  1. Django admin - Create a Map

    • Django admin -> Maps -> Add map
    • Fill out the form and save. An error will be shown if the form is not filled out correctly.
    • The field Extras can be used to directly pass a JSON object to the MapLibre Frontend. For example, to set the initial map view, you can pass the following JSON object: {"center": [10.7, 59.9]}.
  2. Django admin - Add background layer to the map.

    • add raster source
      • Maps > Raster sources

      • Set tile-URL, zoom and size in extras:

        {
        "tiles": ["https://a.basemaps.cartocdn.com/light_nolabels/{z}/{x}/{y}{r}.png"], 
        "maxZoom": 20, 
        "tileSize": 256
        }
    • add source to the Map as Layer
      • Maps > Layers > Add layer
      • Set style: {"type": "raster"}
      • check basemap box
  3. Prepare your data

    • Prepare the raster data
      • raster_operations.sh
        • mosaic rasters (if raster is split into multiple files)
        • clip the raster to your study area, set a NoData value, and set the coordinate system
      • generate_cog.sh
        • convert a GeoTIFF to a Cloud Optimized GeoTIFF (COG)

          # source the helper functions
          source raster_operations.sh
          source generate_cog.sh
          
          # mosaic rasters
          mosaic_rasters /path/to/input_dir /path/to/mosaic_output.tif
          
          # clip the raster
          clip_raster input.tif input.gpkg clip_layer output.tif NODATA EPSG 
          
          # convert to COG (using default method, see the script for more options)
          to_cog /path/to/input.tif default
    • Prepare the vector data
      • generate_pmtiles.sh
        • convert the GPKGs to PMTiles using GDAL >= 3.8.

        • MAXZOOM and MINZOOM influence the level of detail in the map and the size of the tiles (higher values = more detail, larger tiles).

        • docs: gdal.org/drivers/vector/pmtiles.html.

          source generate_pmtiles.sh
          generate_pmtiles /path/to/input.gpkg /path/to/output.pmtiles MINZOOM MAXZOOM
  4. Style your data

  5. Naturregnskaps API - Upload map layers and metadata

    1. Check the import schema (provided by the projects data manager)

    2. Check the style file

    3. Upload the data and metadata using the API

      command: naturregnskap_maps_uploader_start http://server:port map_slug --wd /path/to/data/

      example: naturregnskap_maps_uploader_start http://localhost:8000 localEA --wd ~/temp/nina-map-catalog

      The API will read the import schema and upload the data to the NINA Catalogue.