Welcome to the Spatialdata.gov.scot Metadata Portal’s documentation!

About this Guide

This guide sets out detailed instructions on the creation, publishing and maintenance of standard compliant metadata published on the spatialdata.gov.scot metadata portal. It has been written using Read The Docs, and can be navigated through by scrolling or clicking the sections in the navigation bar. The subsections can be explored by clicking the plus sign next to each section. A PDF copy of this guidance can be exported by clicking on ‘v: latest’ at the bottom of the navigation panel, which will display the options to download the document in PDF and other formats.

The portal is based on the open source GeoNetwork platform. Documentation on the GeoNetwork framework is also linked to within some pages of spatialdata.gov.scot. While this is more generic guidance, it may be of use if you can’t find your answer within this document.

SpatialData.gov.scot

SpatialData.gov.scot (previously referred to in short as the SSDI) provides a catalogue service where users can find, share, and reuse spatial data published by Scottish public sector organisations. Information available is not confined to just environmental data, but includes a variety of themes such as health, transport and administration. Datasets are discoverable by text, map and programmatically based searches, can be previewed on a map interface, and can be accessed directly via web service links recorded in the metadata.

In addition to the catalogue service, the portal also provides editing and validation tools necessary for data publishers registered on the site to create and maintain the metadata they publish. The portal employs the UK GEMINI metadata specification to ensure metadata is recorded in a consistent manner, enhancing the access to and reusability of the data.

Good quality metadata is key to understanding whether data is fit for purpose, has value, and is reliable - particularly if it is to inform policy decisions. SpatialData.gov.scot fosters the development of good quality metadata, and underpins a range of websites and applications that aim to interrogate and analyse available spatial data.

Any queries about the portal can be directed to: SSDI.metadata@gov.scot

SpatialData.gov.scot and INSPIRE

In order to effectively use spatial data in the delivery of national outcomes, spatial data managed by public sector bodies should be collected and shared in a manner that is open, minimises duplication and encourages re-use. A common way to achieve this is through the development of a Spatial Data Infrastructure (SDI) - which is essentially a framework of connected standards, spatial datasets, metadata and tools for visualisation and analysis that allow for spatial data to be used efficiently. The main components of an SDI are:

  • Repositories (to store the information)
  • Catalogue services (to discover the information)
  • Web services (to access the information)
  • Desktop GIS/Web Apps (to interrogate the information)

The EU Inspire Directive aims to create a spatial data infrastructure for the European Union to enable the sharing of and access to environmental spatial information across Europe and assist in policy-making across boundaries. This EU SDI will be based on the infrastructures established and operated by the individual member states. The Directive came into force on 15 May 2007 with full implementation expected by 2021. This directive has since been transposed into Scottish law) in 2009.

Scotland, in partnership with other administrations of the UK, works to fulfill its INSPIRE obligations by developing its own spatial data infrastructure compliant to the standards specified by the Directive. The Scottish SDI aims to apply the principles of INSPIRE, not just to environmental data, but to all spatial data maintained by the Scottish public sector.

SpatialData.gov.scot fulfills the discovery/catalogue service component of the SDI and feeds into the UK public data publishing platform Find open data (data.gov.uk) (which in turn, feeds into the EU INSPIRE Geoportal). Harvest of the metadata on spatialdata.gov.scot to data.gov.uk is performed automatically on a daily basis, therefore record amendments should appear on data.gov.uk within 24 hours. Users should contact SSDI.metadata@gov.scot if they believe their records are not being harvested correctly.

Further information on INSPIRE, such as data specifications and implentation guidance, can be found on the INSPIRE knowledge base pages. Information relevant to Scotland is also circulated on Knowledge Hub, within the Scottish SDI Strategy Group, SDI/Inspire forum.

Metadata Standards on SpatialData.gov.scot

Metadata on the portal is compliant to the UK GEMINI specification, which defines a core set of metadata elements for UK geospatial data, and is compatible with the INSPIRE requirements for metadata. The current version of the standard is UK Gemini 2.3. Users should consult Annex 2 of this guidance to upgrade their existing metadata to Gemini 2.3 if the have not already done so.

The MEDIN Discovery Metadata Standard is a profile of UK Gemini specific to marine data. It can be thought of as a superset of Gemini, including the same core elements, but extending where necessary to include additional information relevant to the marine environment (e.g. marine keyword term lists). MEDIN format metadata uploaded to spatialdata.gov.scot will retain these additional elements. It is recommended that Scottish orgnaisations maintaining marine datasets with MEDIN format metadata submit to both spatialdata.gov.scot and the MEDIN portal.

Metadata Basics

What is metadata?

Metadata is essentially information about data. It describes, among other things, who collected it, why it was collected, the methods used to collect it and how it can be used. Metadata provides all the information a data user needs to assess whether the data resource is fit for use. Metadata allows users to:

  • Know who created the data and for what purpose
  • Know when the data was created
  • Know the geographic extent or coverage of the data (for geospatial data)
  • Understand how the data was created and manipulated
  • Understand the limitations of the data with respect to use
  • Understand any legal limitations to access and use of the data
  • Determine tools needed to view, manipulate, and use the data
  • Know how to access the data

Metadata is important to both the dataset provider and other potential users. High quality metadata allows dataset owners to keep an inventory of datasets they have created, with the ability to locate the data and recall the circumstances and context under which it was created and analysed. When publicly available, metadata enables other interested parties the ability to discover, understand and use the data to its fullest potential.

What are metadata standards?

A metadata standard is a specification which establishes a common understanding of the meaning (or semantics) of data, to ensure correct and proper use and interpretation by its owners and users. Many standards begin as schemas, which group metadata elements into sets designed for a specific purpose (e.g. for geospatial data) to enable the best possible description of a data resource in that context. For every element the name and semantics are specified, along with any rules on how content must be formulated and represented, and where element values are to be identified from controlled lists.

The development of such schemas tends to be controlled through community consensus combined with a formal processes for submission, approval and publishing of new elements. Schemas that are developed and maintained by standards organisations (e.g. ISO) or organisations that have taken on such responsibility (e.g. Dublin Core Metadata Initiative) are called metadata standards.

The UK Geospatial Metadata Interoperability Initiative (GEMINI ) is the UK’s geographic metadata standard. It was originally produced through a collaboration between the Association for Geographic Information (AGI), the e-Government Unit (eGU) and the UK Data Archive. An application profile of ISO 19115, UK GEMINI is conformant with the INSPIRE Technical Guidance and designed to meet the requirements of INSPIRE in a UK context.

Why create metadata?

Metadata adds significant value to any organisation’s data holdings, and failure to create it can lead to hidden costs generated at a later date. Undocumented data soon loses its value as staff change and institutional knowledge is lost. Subsequent staff with little understanding of the contents and use of the data and may lose confidence in results generated by it.

Beyond the individual organisation, data resources are a major national asset. Information of what data resources exist within different organisations, particularly within the public sector, is required to improve efficiencies and reduce data duplication. Metadata standards enable interoperability between organisations and platforms. Metadata from a variety of sources can be integrated into other technical systems or machine read by compatible ones. Adoption of metadata standards greatly increases the potential for data discoverability.

Data can only be useful if it can be interpreted and understood, and can only be used if it can be found. Creating and maintaining standard compliant metadata secures the financial investment that organisations put into data by:

  • Retaining institutional knowledge
  • Enabling better decision making through the use of appropriate data
  • Avoiding costly duplication of effort through lack of knowledge about data
  • Minimising the risk of unintentional use through good documentation
  • Increasing the value of data by unlocking its potential for re-use
  • Supporting collaboration through discoverability and simplified access
  • Inspiring trust in the data source

When effort and resource is put into the provision of standard compliant metadata, organisations will be better placed to find, assess and make use of the data that is available to them – particularly in times of national emergency - saving time, money and lives.

Why use UK Gemini?

UK GEMINI is the designated standard for geographic metadata in the United Kingdom. It ensures consistency with respect to metadata across the UK through content guidance and the use of UK specific terms and code lists. UK Gemini also simplifies the INSPIRE Technical Guidance making it easier for organisations to achieve compliance.

While ensured compliance to regulations is important, the key benefit is the interoperability the standard allows across a range of platforms. Metadata in the Gemini format can be shared and consumed easily by a number of platforms and third party applications. Gemini is a requirement for the data.gov.uk portal and, with version 2.3, can now be consumed by the ESRI ArcGIS platform without the need of additional plugins.

Searching for Metadata

This section will guide users and guests to the site on how to search and filter metadata provided by data publishers to spatialdata.gov.scot.

Accessing the portal

The portal can be accessed from the following link:

https://www.spatialdata.gov.scot

The home page of the portal comprises a basic search bar at the top, and also provides the option to browse by topic category (default) or by INSPIRE theme. Visitors can also browse the newest entries to the site or the most popular using the tabs lower down the page. Tabs in the header menu give access to the main search page, map viewer and sign in page (for registered users). Clicking the portal name on the header menu will return the user to the home page.

userdoc_fig_2_1_1_Home

Figure 2.1.1: Home page

When logged in, registered users of the site will be presented with further buttons on the header menu for button_contribute and button_adminconsole (for user and site administrators). The button_contribute menu gives options to add a new record, import new records, manage directory, and batch editing.

Basic searching

Quick searches on the portal can be performed directly from the home page using the search box and clicking the search button (button_search_icon).

Note: wildcard characters, such as the asterisk key, can also be used.

userdoc_fig_2_2_1_BasicSearch

Figure 2.2.1: A basic search from the home page

This will take the user to the main search panel, showing the resulting records in the centre of the page, with additional filtering options to the left. The user can also access the main search panel by clicking button_search on the top menu of the home page. To the bottom right of the page is a small map view showing the geographic extents of the queried results. This can be minimised by clicking the collapse button on the right of the map.

userdoc_fig_2_2_2_SearchResults

Figure 2.2.2: Search results displayed on the main search panel

Advanced searching

Other search options are available from the vertical ellipses icon to the right of the main search box. This expands to show What and When options. What allows users to search by Categories, Keywords, and Resource Contact. When allows users to limit the search temporally based on the metadata change date of an interactive resource (service) or dataset record.

To perform an advanced search:

1| Click button_search to access the search page.

2| Click the button_search_advanced next to the search text box to access advanced options.

3| Provide additional search criteria (e.g. constraining by category, keyword, contacts or time).

4| Click button_search_icon to run the search based on the selected options. Results will be displayed in the list below.

Note: search options can be cleared by clicking button_search_reset.

userdoc_fig_2_3_1_AdvancedSearch

Figure 2.3.1: Advanced search options

Filter and sort options

Search results can be limited using the options available in the left-hand panel. Each search filter shows the number of records returned next to it. Filtering options available are as follows:

  • Type of resource (i.e. datasets or services)
  • Licence Type
  • Topics
  • INSPIRE Theme
  • Keywords
  • Contact for the resource
  • Year
  • Format
  • Available Actions
  • Representation Type (i.e. raster or vector)
  • Update Frequency
  • Scale
  • Resolution

At the top right of the search results list is a menu to sort results by title, relevance, date of change, rating, popularity, and scale. At the top left of the results list users can select all results, all results on the current page, or none to deselect. Upon selection users can choose to export the results to a zip, pdf or csv file. Note that registered users will be able to perform additional actions, such as publish or delete, on selected records.

userdoc_fig_2_4_1_SortOptions userdoc_fig_2_4_1_SelectOptions

Figure 2.4.1: Sort and select options

Viewing Metadata and Data

This section will guide users and visitors to the site on viewing metadata and previewing data using the interactive map.

Viewing metadata records

Clicking on the title of a record (from either the search results or those presented on the main page) will allow the user to view the metadata record. Below the record title, the abstract provides a brief description of what the dataset (or service) is and what it is used for. The spatial extent of the resource can be seen in the map to the right. Access to the data itself can be gained from the options listed in the Download and links section.

userdoc_fig_3_1_1_DefaultViewA

Figure 3.1.1: Default record view

Scrolling down the page, the About this resource and Technical information headings provide details on topic categories and keywords, licencing information, contacts, maintenance frequency, coordinate systems and data formats are provided. Information on the history of a resource and how it was created can be found in the lineage section.

Note that the default view for a metadata record displays a summary of the information in the record. To change the display view of the metadata record:

1| Click on button_view_display at the top right of the record.

2| Choose ‘Full view’ to display a detailed view of the record.

For detailed descriptions of each of the elements included in the metadata record, consult the Metadata Elements section.

Exporting metadata records

Metadata records can be exported from spatialdata.gov.scot via the download icon menu at the top right of the record. To export a record:

1| Click on button_view_download at the top right of the record.

2| Choose the format to export (ZIP, PDF, XML, or RDF). The permalink option will provide a URL for the record that can be copy and pasted.

userdoc_fig_3_2_1_ExportMetadata

Figure 3.2.1: Export metadata menu

Previewing data in the map

Spatial data can be previewed in the interactive map using the web services referenced in the metadata. This capability can be accessed either from the search results page, the metadata record view, or from the interactive map tab in the header menu.

To add a layer to the map from the search results page:

1| Click on button_view_globe visible at the bottom right of the record.

2| If there are multiple layers in the service, click the desired layer or choose to add all layers to the map.

userdoc_fig_3_3_1_AddtoMapSearch

Figure 3.3.1: Adding a layer to the map via the search results page

To add a layer from the main map page:

1| Click on button_map in the header menu to navigate to the main map page.

2| On the right-hand side of the map, click button_map_addlayer at the top of the menu to open the Add a Layer dialog.

3| On the Search tab of the Add a Layer dialog, enter text in the search box to find a layer. Alternately, copy and paste a URL for a service into the WMS, WMTS, or WFS boxes on the Services tab. KML can be added to the map via the Files tab.

4| Click button_map_addicon beside the layer to add it to the map. If adding from a URL, click on the layer name to add it to the map.

Note: when adding straight from the map page, using the search tab, you may see layers with no name. This is caused by the web service layer names not being referenced properly in the associated resources part of the dataset metadata record. Contributors to spatialdata.gov.scot should update their metadata records so their resources in the dataset metadata are named with the layer name as it appears in the service (i.e. as listed in the GetCapabilities request). See the Associated Resources section for guidance on how to set up associated resources.

userdoc_fig_3_3_3_AddtoMapMapPage

Figure 3.3.3: Adding a layer to the map via the map interface

To add a layer to the map from the metadata record view:

1| Scroll to the Downloads and links section of the record. Services that can be added to the map will have button_view_addtomap beside them.

2| For view services (i.e. WMS), Click on button_view_addtomap. The user will be automatically taken to the map page.

3| For download services (i.e. WFS), select the layer from the drop down menu then click on button_view_addtomap.

userdoc_fig_3_3_2_AddtoMapRecordView

Figure 3.3.2: Adding a layer to the map from the metadata record view

Note: with respect to WFS, users may experience occasions where the layer appears to have been added to the map, yet does not display. This occurs when the Access-Control-Allow-Orgin header (CORS access policy) is missing from the remote server, and thus the request is blocked by the browser. This is out of the control of the portal as it is dependent on the organisations’ server configuration, however, an error message shall be displayed.

Managing layers in the map

Available tools for the map are arranged down the right-hand side of the page.

To manage layers displayed in the map:

1| Click button_map_managelayers on the right-hand side of the map. This will display a list of layers currently added to the map.

Note: those with scale-dependent visibility that are not currently visible will be greyed out. Layers that cannot be added for some other reason, such as an error with the service, will display the layer name in red next to an error icon (button_map_exclamationicon).

2| To toggle a layer on or off, click on the layer name or check/uncheck the box.

3| Click the cog icon (button_map_cogicon) to access a menu to zoom to the extent of the layer, select syle (if available), adjust the opacity, or remove the layer from the map.

4| To toggle the layer legend on or off, click button_map_legendicon.

userdoc_fig_3_4_1_ManageLayers

Figure 3.4.1: Manage layers menu

Other map functions

Other functionality available from the tabs on the main map page is as follows:

button_map_default Default Map:
 via the manage layers tab, allows the user to reload the default map settings (removes all layers, etc.).
button_map_measure Measure:
 allows the user to draw a line on the map to measure it’s length. Single-click to create a node and double-click to finish.
button_map_graticules Projection Switcher:
 change the projection of the map to one of the supported projections in the dropdown list.
button_map_zoomin button_map_zoomout Zoom in and out:
 zoom in or out on the map.
button_map_zoomextent Zoom to initial map extent:
 zoom to the full extent of the map.
button_map_graticules Graticule:
 adds a grid to the map (in lat/long).

Registered Users

This section will guide registered users on how to sign in and manage their accounts.

Becoming a registered user

To add or edit metadata on SpatialData.gov.scot, your organisation must be registered on the site. The portal does not currently allow self registration, and therefore interested parties should contact (SSDI.Metadata@gov.scot). Currently, user accounts are created for organisations as a whole rather than individuals.

Registered user obligations

When an organisation becomes a registered user and submits metadata to SpatialData.gov.scot, they commit to the following operational and service requirements:

  • Maintaining the discovery metadata added by the organisation:
    • by assuring its quality against appropriate standards
    • by reviewing it, updating it regularly, and ensuring it is updated when new versions of data, data series or on-line services are published.
    • maintaining metadata file identifiers (GUID) throughout the life of the metadata resource (new GUIDs should only be created when new resources are published, to prevent duplicate metadata resources).
  • Adopting a process of continuous improvement for the resources published by the organisation, based on the feedback received.
  • Compliance with the INSPIRE Metadata Regulation operational requirements, as set out in this guide.
  • To publish INSPIRE compliant view services for a given dataset or data series and reference them appropriately in the metadata, as set out in this guide.
  • Legal compliance with the INSPIRE Directive when publishing data that comes under an INSPIRE Theme and contains features listed in the INSPIRE Data Specifications.
  • Compliance with the INSPIRE Network Services Regulation when publishing INSPIRE Network Services.

Signing in

To sign in to an existing account:

1| Click the button_signin in the header menu.

2| Enter your organisations’ account user name and password.

3| Click the button_signin_confirm button.

Note: user names and passwords are case sensitive.

userdoc_fig_4_3_1_SignIn

Figure 4.3.1: Signing in to a registered account

To recover a forgotten password:

1| Click the button_signin_forgotten on the right-hand side of the sign in page.

2| Enter the account user name.

3| Click the button_signin_sendlink button. This will send an email to the address registered to that user account providing a link to reset the password.

Note: user names are case sensitive. The password recovery function requires that a valid email address (accessible by the user) is registered with the user account in question. If you have not registered an email address or do not have access to the relevant mailbox, you will not be able to receive password reset emails.

userdoc_fig_4_3_2_ResetPassword

Figure 4.3.2: Resetting a forgotten password

User account management

Once logged in, users can manage their account details by clicking on their profile name displayed in the header. This will take the user to the account admin page. From this page users can enter contact details and reset passwords.

To reset an account password:

1| Click on the user profile name in the header. By default the button_account_manageusers page should be presented. Click the user name on the left hand side under ‘Users’ to display the user account information.

2| Click the button_account_resetpass button.

3| Enter a new password in the Password text box. Repeat the password in the box below.

4| Click the button_account_resetpassconfirm button to confirm the change.

Note: passwords are case sensitive and must be at least 8 characters long.

When amending details, users should click the button_account_save button to save changes.

userdoc_fig_4_4_1_ManageAccount

Figure 4.4.1: The account management page

Note that there is also a button_account_managegroups tab. Generally each user account is an administrator for a group of the same name. On this tab users can manage group names and contacts as wells as assign logos for the group (which will appear on record view) - groups can be thought of as the ‘publishing body’. Groups can have multiple user accounts associated with them, and users may be a part of a number of groups.

Users and groups

SpatialData.gov.scot employs the concept of users, groups and profiles. Users are those accounts registered on the portal, and can be part of one or more groups (which are collections of users corresponding to logical units, such as teams within an organisation). The group and profile (or role) of a user defines what tasks they can perform on the portal or on specific metadata records.

The rights associated with the various profiles are as follows:

1) Administrator Profile:
 
The Administrator has special privileges that give access to all available functions. These include:
  • Full rights for creating new groups and new users
  • Rights to change users/groups profiles
  • Full rights for creating/editing/deleting new/old metadata
  • Perform system administration and configuration tasks.
2) User Administrator Profile:
 
The User Administrator is the administrator of his/her own group with the following privileges:
  • Full rights on creating new users within the own group
  • Rights to change users profiles within the own group
  • Full rights on creating/editing/ deleting new/old data within the own group
3) Content Reviewer Profile:
 
The content reviewer is the only person allowed to give final clearance on the metadata publication on the Intranet and/or on the Internet:
  • Rights on reviewing metadata content within the own group and authorising its publication
4) Editor Profile:
 
The editor works on metadata with following privileges:
  • Full rights on creating/editing/ deleting new/old data within the own group
5) Registered User Profile:
 
The Registered User has more access privileges than non-authenticated Guest users:
  • Right to download protected data

Note: profiles (or roles) are hierarchicaland based on inheritance. This means that a user with an Editor profile can create and modify new metadata records, but can also use all functions a Registered user can use.

Normally, organisations registered on the site are given the role of Content Reviewer for the purpose of managing their metadata records.

Creating Discovery Metadata

This section will guide the user through the process of creating discovery metadata records on spatialdata.gov.scot. Discovery metadata can be added to the portal by importing an existing metadata XML file, creating a new record from a template, duplicating an existing record, or harvesting records from other metadata portals.

Importing existing metadata

Prior to following the steps below to import an existing metadata record, please review Annex 1.

To import an existing XML metadata record:

1| Click the button_contribute button in the header menu and choose button_contribute_import. This will take the user to the Import new records menu board.

2| Select where to import the record from. Options are from your computer (default), a URL, by copy/paste, or from a server.

3| Select Metadata from the type of record dropdown.

4| Indicate how the unique identifier for the record (UUID) should be treated. If you are uploading in order to refine or change an existing record, then choose the Overwrite metadata with the same UUID option. If you are uploading to create a new metadata record then choose the Generate UUID for inserted metadata option. If the XML contains a UUID that does not already exist on the portal but you would like to keep this UUID then the None option should be selected.

Note: The UUID is a generated automatically by whatever internal system (i.e. ESRI ArcGIS) was used to write and manage your metadata. In xml, it can be found at the top of the page under the file identifier section. Within the portal, the UUID or File Identifier can be found at the bottom of the metadata record display. This UUID, together with the metadata date stamp, are used to ensure records are unique within spatialdata.gov.scot, data.gov.uk and other metadata catalogues.

5| Under the Apply XSLT conversion menu, select the appropriate translation to convert the metadata to the UK Gemini format.

Note: For an xml file that is already in Gemini 2.3 format and recognised as such (see Annex 1), the Apply XSLT conversion option can be set to ‘None’. Select ‘GEMINI 2.2 to GEMINI 2.3’ to convert an existing Gemini 2.2 record to 2.3, select ‘ESRI to GEMINI 2.3 to convert records created using the ESRI ArcGIS platform. Users providing data in MEDIN format should select the MEDIN to GEMINI conversion.

6| Optionally, select whether to validate the record on import.

Note: it is recommended that the validate option be left un-ticked, and users perform a validation of the record after import. If the validation option is ticked and an error is found, the record will fail to import.

7| Optionally, select whether to publish the record on import.

8| Leave the Assign to current catalog option ticked.

9| Assign the record to a user group.

10| Assign the record to the appropriate licence type (i.e. OGL, NCGL, INSPIRE or other). This will allow for filtered searching based on the licence conditions.

Note: Licence types can also be amended at a later date. See assigning a licence category.

11| Click the button_contribute_importconfirm button.

userdoc_fig_5_1_1_ImportMetadata

Figure 5.1.1: Importing an existing XML metadata record

When the import operation has finished, an Import Report will be displayed on the right-hand side of the screen, indicating whether or not the import has been successful. Following a successful import, users should verify that their records are valid. Consult the Validating metadata section for details. Amendments to the record may be required to make it fully Gemini compliant.

Creating metadata from a template

Basic templates are available for users to create metadata directly on spatialdata.gov.scot.

To create a metadata record using an available template:

1| Click the button_contribute button in the header menu. This will enable the Contribute board, where all records the user has access to will be listed.

2| Click the button_contribute_addrecord button.

3| Choose the type of record to be created based on the type of resource that is being documented, i.e. a dataset, series or service record. When selected available templates for that type are displayed.

Note: templates for dataset and service metadata are slightly different. Metadata for a service has extra elements for the spatial data service type (i.e. view, download, etc.) and for coupled resources. You must choose the correct template for the type of record being documented as it is not possible to change once the record has been created.

4| Click on the template to be used.

5| Click the button_contribute_create button to create the new record. This will automatically take the user to the editing board where elements of the new record can be modified. See Editing a metadata record for guidance on documenting the resource.

Note: if the user is an editor for multiple groups or an administrator to the site, they may be presented with multiple templates of the same name. Additionally, they will be required to select which group folder to create the metadata record in.

userdoc_fig_5_2_1_CreateTemplate

Figure 5.2.1: Creating a metadata record from a template

Duplicating existing metadata

One of the easiest ways to create metadata on the portal is to duplicate an existing record. This is particularly useful for organisations where many of the metadata elements are the same for all records (e.g. points of contact, etc.)

To create a metadata record by duplicating and existing record:

1| Click the button_contribute button in the header menu. This will enable the Contribute board, where all records the user has access to will be listed.

2| Click the duplicate icon (button_contribute_duplicate) next to the record name.

3| Click the button_contribute_create button to create the new record. This will automatically take the user to the editing board where elements of the new record can be modified. See Editing a metadata record for guidance on documenting the resource.

Note: if the user is an editor for multiple groups or an administrator to the site, they will be required to select which group folder to create the duplicate metadata record in.

userdoc_fig_5_3_1_Duplicate

Figure 5.3.1: Duplicating an existing metadata record

Harvesting metadata

Spatialdata.gov.scot has the capability to automatically consume metadata from external services on a scheduled basis. Organisations that maintain their own spatial data portals which are interested in having their records automatically harvested should contact SSDI.Metadata@gov.scot.

Creating directory metadata

Directory records are a means of automatically populating fields such as those for the point of contact section of a metadata record with pre-written values. This is useful if you have to create multiple records that share the same point of contact.

To create a directory entry:

1| Click the button_contribute button in the header menu.

2| Click either button_contribute_managedirectory from the contribute drop down, or the button below the header of the Contribute board. This will bring the user to the manage directories page.

3| On the entries tab are previously created directory entries. Under the Actions drop menu, users can choose to edit or duplicate an existing record. Alternately, users can create a blank directory entry using a template by first clicking on the Templates tab, and then choosing duplicate from the Actions menu of the ‘TEMPLATE TO DUPLICATE’ entry.

4| Amend the entry as appropriate and click button_contribute_savedirectory to save changes.

5| Click button_contribute_saveclosedirectory to save and exit.

Note: The individual name field is not mandatory for the UK Gemini standard, however, for directories it can be used to identify teams/business areas if you have a number of different contact details within an organisation (e.g. GIS, Planning, etc.)

userdoc_fig_5_5_1_DirectoryEntry

Figure 5.5.1: Creating a directory entry

Editing Metadata

This section will guide the user through editing a metadata record on spatialdata.gov.scot, compliant to the UK Gemini standard. In most cases, the basic functionality provided by the default view of an editing sessions will be sufficent for the users need, however, in some instances additional functionality from the advanced view may be required. For detailed information on completing each of the individual metadata elements, refer to the Metadata Elements section.

Starting an editing session

Users will be able to edit any records they have contributed to the portal, or if they have appropriate privileges to edit records submitted by others (e.g. administrators).

To begin an editing session:

1| Sign in using your account credentials.

2| Either search for a particular record direct from the home page or click button_contribute and search or scroll for the record.

Note: if the ‘only my records’ box at the top left of the contribute page is checked, this will limit the record list to only those belonging to that user.

3| Click the pencil icon (button_contribute_pencil) visible in the search results list, or next to the record name on the contribute board. This will enter the user into an editing session.

Note: users can also start an editing session from the record view page by clicking button_view_edit at the top of the page.

Editing basics

Once in an editing session, the user is presented with the simple default editing view. In the header menu, buttons are available for modifying the categories, group that the record belongs to, validation, cancelling the edit session, and saving the record. On the right side of the page, the Associated resources panel is visible, showing all related online resources and service records related to the record being edited.

The form displays the metadata elements grouped into relevant sections. The user can navigate through the form using the table of contents visible in the lower right of the screen (click button_edit_toc to display). Clicking an element section will take the user to the relevant section of the form. The user can collapse all elements of the form by clicking the button_edit_collapse button visible just below the header. Clicking on a section title will re-expand that section. The button_edit_arrowup and button_edit_arrowdown buttons allow the user to place the section further up or down the document order. Note that UK Gemini is only concerned with whether the elements (or sections) are present and correct, not with the order they are displayed. For detailed guidance on filling out each element, users should consult the Metadata Elements section.

Mandatory fields (e.g. Abstract) are highlighted with a red asterisk. Note that for sub-elements, this only indicates that the sub-element is mandatory within its context. For example, the vertical extent is an optional element, but if the record requires one, then the minimum and maximum values are mandatory.

Where a button_edit_plus button is displayed, the user can click this to add another occurrence of an element or section. For example, a record could have more than one alternative title. When the user hovers over a section and a button_edit_delete appears, clicking the cross will delete that element or section.

userdoc_fig_6_2_1_DefaultEdit

Figure 6.2.1: The default view of the editing session interface

Changing the editing view

Sometimes it will be necessary for the user to change the view to enable additional functionality. For example, to add a new occurrence of an element when the button_edit_plus is not visible in the default view.

To change the view of editing session:

1| Click on button_edit_view on the top right of the editing board.

2| From the dropdown menu, choose either INSPIRE, simple, full or XML. Advanced (full) view is a more detailed version of the default (simple) view, where as XML view allows the user to edit the raw XML.

Note: users can also enable tool tips from the view menu.

userdoc_fig_6_3_1_AdvancedEdit userdoc_fig_6_3_1_XMLEdit

Figure 6.3.1: Advanced editing and XML view

Assigning a licence category

Records can be classified based on their licence type (i.e. Open Government, Non-commercial Government, INSPIRE or other) using the assign category function. This classification can be done either in an editing session, via the contribute board, or when a record is imported into spatialdata.gov.scot.

To assign or change the licence category in an editing session:

1| Click on button_edit_category in the header.

2| From the dropdown menu, check the appropriate licence type (Open Government, Non-commercial, INSPIRE or other).

3| Click anywhere to dismiss the menu.

userdoc_fig_6_4_1_LicenceCategory

Figure 6.4.1: Assigning a licence category in an editing session

To assign or change the licence category via the contribute board:

1| Click on button_contribute in the header.

2| In the record list, select the record(s) for which the licence category will assigned or amended.

Note: the licence category can be updated for multiple records at a time.

3| Click the dropdown labelled ‘selected’ at the top left of the record list and choose ‘Update licence’ from the list. This will display the licence categories dialog.

2| From the menu, check the appropriate licence type (Open Government, Non-commercial, INSPIRE or other).

3| Click button_contribute_replace to apply the changes.

The licence category can also be assigned using the appropriate option when importing a record. For those organisations which harvest their metadata records into the portal, a default licence type can be selected for all records, though any records that differ from the default setting would need to be amended post harvest.

Associated resources

The Associated Resources panel allows users to add online resources to their metadata and create relationships between dataset and service records (i.e. coupled resources). When in an editing session, the associated resources panel can be found at the top right of the page.

To add an online resource to dataset metadata where a corresponding service record exists:

1| Click on the button_edit_addassocresource button at the top of the Associated Resources panel.

2| From the dropdown menu select ‘Link to a service’. This will display the Link resources dialog.

3| Search using the free text box and click on the service record in the results list to select it (when selected it will be highlighted in blue).

4| This will display the layers available in the service. Click on the appropriate layer to select it (when selected it will be highlighted in green).

5| Click on the button_edit_linkservice to link the service to the dataset metadata. This creates a link between the dataset and service records and adds an online resource element to the dataset record.

6| If required, amendments can be made by clicking on the pencil icon beside the newly added online resource in the associated resources panel. Alternately scroll to the element in the main editing board and amend manually.

7| Click button_edit_updatelink.

Note: For web services (i.e. WMS and WFS) to correctly display in the map, you must ensure that the protocol and layer name are correctly entered for the online resource. For dataset records the URL for for a WMS or WFS should not include the GetCapabilities request. While these elements are not required for Gemini compliance, the are needed to enable previewing on the map.

userdoc_fig_6_5_1_AssocDatasetService

Figure 6.5.1: Adding an online resource to a dataset by associating with a service record

To add an online resource to dataset metadata where no corresponding service record exists:

1| Click on the button_edit_addassocresource button at the top of the Associated Resources panel.

2| From the dropdown menu, select the first option, ‘Link and online resource’. This will display the Link resources dialog.

3| Ensure the ‘Add online resource’ radio button is clicked.

4| Add the URL to ensure it points at the service (e.g. www.mapdata.scot/ServiceName/MapServer/WMSServer?). For dataset records the URL for for a WMS or WFS should not include the GetCapabilities request.

5| Set the correct protocol for the service from the drop down (e.g. OGC-WMS Web Map Service). This will then display the layers available in the service.

6| Click on the appropriate layer to select it (it will highlight in green).

7| Click button_edit_addlink.

Note: For web services (i.e. WMS and WFS) to correctly display in the map, you must ensure that the protocol and layer name are correctly entered for the online resource (steps 5 and 6 above). While these elements are not required for Gemini compliance, the are needed to enable previewing on the map.

userdoc_fig_6_5_2_OnlineResourceDataset

Figure 6.5.2: Adding an online resource to a dataset record

To add an online resource (e.g. WMS or WFS) to a service metadata record:

1| Click on the button_edit_addassocresource at the top of the Associated Resources panel.

2| From the dropdown menu, select the first option, ‘Link and online resource’. This will display the Link resources dialog.

3| Ensure the ‘Add online resource’ radio button is clicked.

4| Select the appropriate protocol for the online resource (e.g. for a WMS, select OGC-WMS Capabilities service)

5| Enter the URL for the service.

Note: A service may contain multiple layers and these will be displayed when the service URL is entered. As this is a service record, it is not necessary to select a single layer to link to (this is only required for dataset records).

6| Optionally, provide a name and description for the resource.

7| Click button_edit_addlink.

userdoc_fig_6_5_3_OnlineResourceService

Figure 6.5.3: Adding an online resource to a service record

Validating metadata

Once all elements for a new record have been completed or changes have been made to an existing record, it should be validated against the UK Gemini standard rules.

To validate a metadata record:

1| In an editing session, click on the button_edit_validate button in the header bar.

2| Errors will be highlighted in the validation panel. Click the red thumbs-down icon (button_edit_thumbsdown)to see details of the error(s) organised by schematron.

3| Correct errors as necessary and repeat steps 1 and 2 until no errors are returned by the validator.

Note: users can save an invalid record and return at a later date to correct.

userdoc_fig_6_6_1_ValidationPanel

Figure 6.6.1: Validation panel

The validation check should be repeated once amendments have been made until no errors are detected. Error descriptions given by the validator are often complex and difficult for the lay user to understand. Consult the Metadata Elements section for guidance on how to complete specific elements.

Saving and exiting an editing session

As the user works through the editing form, it is recommended that progress be saved periodically as the session will time-out after a period of inactivity.

To save a metadata record:

1| Click button_edit_save to save the record and continue editing, or click button_edit_saveclose to save the record and exit the editing session.

OR

1| Click button_edit_cancel to exit the editing session without saving the record.

Users should note that a saved record will be stored on the server only (not locally). The server is periodically backed up, though it is recommended that users store master copies of their metadata records within their own systems. To export copies of metadata created on the portal, consult the exporting metadata section.

Publishing metadata

Once a metadata record has been successfully validated, it can then be published, making it publicly viewable and available for harvesting by other data portals, such as data.gov.uk.

To publish a metadata record:

1| Click on button_contribute in the header and locate the metadata record in the list below.

2| Click on the button_contribute_locked next to the record name to access the Privileges dialog.

3| To make publicly viewable, tick the ‘Set all privileges’ box next to the row marked ‘All’ in the group column.

4| Click on button_contribute_replace to update the privileges for the record.

Once the record is published, the padlock icon displayed on the contribute menu should appear unlocked (button_contribute_unlocked).

userdoc_fig_6_8_1_PrivilegesPanel

Figure 6.8.1: Publishing via the contribute page

Alternatively, there is a shortcut method to publish a record directly from the record’s page on spatialdata.gov.scot:

1| Click on the button_view_managerecord button at the top right of the record view.

2| From the dropdown menu, click on button_view_publish to publish the record.

Note: the privileges dialog can also be accessed from this dropdown menu.

userdoc_fig_6_8_2_PrivilegesRecordView

Figure 6.8.2: Publishing via record view

Batch editing

Administrators will have the privileges to perform batch editing on records. See the Administrator Guidance for more information.

Deleting metadata

Users with the appropriate privileges will be able to edit or delete records from the portal.

To delete a metadata record:

1| Click on button_contribute in the header and locate the metadata record in the list below.

2| Click on the button_contribute_delete in the row for the record.

3| A conformation prompt will be displayed. Click ‘OK’ to confirm the delete.

Note: the record may still appear in the list until the page is refreshed.

userdoc_fig_6_10_1_DeleteRecordConfirm

Figure 6.10.1: Confirmation requested to delete a metadata record

Alternatively, the user can delete a record directly from the record’s page by clicking the button_view_delete button.

Users should note that when deleting a record that had previously been published, that record will not be deleted from other portals which may have harvested the record, such as data.gov.uk. Users should contact SSDI.Metadata@gov.scot to have records deleted from data.gov.uk.

Metadata Elements

This section provides guidance with respect to each of the UK Gemini elements. Section headings refer to the label present in the editing form of spatialdata.gov.scot, however, UK Gemini and INSPIRE equivalents are also provided. Links to full element guidance from the AGI are given, which includes information on known common errors. Images are provided showing example entries for each of the elements.

Users may also wish to consult the full UK Gemini 2.3 specification.

Defining INSPIRE compliance

To define within the metadata if a dataset is compliant with INSPIRE, users should ensure the Keyword section is completed. If the dataset is covered by INSPIRE (e.g. parts of it are within the Scope of an INSPIRE Annex data specification), then the first keyword should be from the GEMET INSPIRE themes keyword list.

If another controlled vocabulary is used as the first keyword, such as the Integrated Public Sector Vocabulary (IPSV), then the dataset is not covered by INSPIRE. That said, the dataset and service metadata will still be required to comply with the operation and service commitments, as set out in the Registered user obligations section.

Abstract

Gemini:

Abstract

INSPIRE:

Resource abstract

ISO19115:

MD_Identification.abstract

Obligation:

Mandatory

occurrence:

One

Definition:

A brief narrative summary of the data resource.

Guidance:

Provide a clear and concise statement of the content of the dataset and its purpose (i.e. the ‘what’ and ‘why’ aspect of the dataset). General background information should not be included, and unexplained abbreviations and jargon should be avoided. The abstract description should be understood by non-experts.

Note: with Gemini 2.3 there is now a requirement for the abstract to be at least 100 characters long, and to be different from the title element. Validation will display an error if these conditions are not met.

userdoc_fig_7_1_1_Abstract

Additional information

Gemini:Additional information
INSPIRE:supplemental information
ISO19115:MD_Identification > MD_DataIdentification.supplementalInformation
Obligation:Optional
occurrence:One
Definition:Other descriptive information about the data resource.
Guidance:Record relevant information that does not clearly belong in any other element. A URL may be entered to reference other documents. Do not use it to describe how to access the resource (this should be in the resource locator element), or for any restrictions on access or use (use the limitations on public access and use constraints elements). If this element is not visible in the editor, switch to advanced editing view, and the Identification info tab, and use the “Additional information” button at the bottom of the page.

userdoc_fig_7_26_1_SupplementalInformation

Alternative title

Gemini:

Alternative title

INSPIRE:

n/a

ISO19115:

MD_DataIdentification.citation > CI_Citation.alternateTitle

Obligation:

Optional

occurrence:

Many

Definition:

Any short name, other name, acronym or alternative language title by which the data resource is known.

Guidance:

This element should be used to record alternative titles by which the data resource is known, such as historical names, commonly used abbreviations and acronyms. It can also be helpful to include title with spatial or temporal coverage if applicable. Entries should remain short and concise.

If the element is not visible in the default view of the editing form, it can be added by switching to advanced view and clicking the button_edit_plus button.

userdoc_fig_7_2_1_AlternativeTitle

Character encoding

Gemini:Character encoding
INSPIRE:Character encoding (metadata/2.0/req/isdss/character-encoding)
ISO19115:MD_DataIdentification.characterSet
Obligation:Conditional (mandatory if not UTF-8)
occurence:Many
Definition:The character encoding used in the dataset.
Guidance:This element enables data in non-standard character encodings to be handled. By default the encoding is set to UTF8, but other options are available using the dropdown list. The majority of users will not need to change this element.

userdoc_fig_7_30_1_CharacterEncoding

Data format

Gemini:Data format
INSPIRE:Data encoding
ISO19115:MD_Distribution.distributionFormat > MD_Format
Obligation:Mandatory
occurrence:Many
Definition:The format(s) in which the digital data is available.
Guidance:The name of the format and the version can be selected from common options listed in the drop down menu. Otherwise, entries should be recognised formats for data transfer, either standard or proprietary. If the data is not transferable (e.g. view only), then this should be stated. Recommended best practice is to select a value from a controlled vocabulary, such as PRONOM.

userdoc_fig_7_7_1_DistributionFormat

Data quality

Gemini:

Data quality

INSPIRE:

Topological consistency. Other types are specified in the data specifications

ISO19115:

DQ_DataQuality.report->DQ_Element

Obligation:

Conditional. Consult Gemini Guidance.

occurrence:

Many

Definition:

Quantitative aspects of data quality. To allow quantitative description of the completeness, consistency, accuracy, and correctness of the data

Guidance:

This element comprises two sub-elements:

1| Scope - the specific data to which the data quality information applies.

2| Conformity - the product specification being assessed against (e.g. conformance to INSPIRE).

At least one data quality element must be scoped to the entire data set or series that the metadata refers to. Other quality measures, for example relating to specific feature types may be provided in addition. To add additional Data Quality elements, switch to advanced view and click the button_edit_plus button at the base of the page.

Data quality - Conformity

Gemini:

Conformity

INSPIRE:

Conformity - degree

ISO19115:

DQ_DataQuality > DQ_Element.result > DQ_ConformanceResult

Obligation:

Mandatory

occurrence:

Many

Definition:

A sub-element of Data quality, this is the degree of conformity with the product specification or user requirement against which the data is being evaluated (i.e. degree of conformity to INSPIRE).

Guidance:

At least one conformity statement shall be to an INSPIRE specification, even if to simply say that the data set is not conformant or not tested. Template snippets for INSPIRE Spatial data sets, Network services and Spatial data services have been provided. Use the Search for a quality report… search box at the bottom of the element to choose the appropriate one.

To record that a data set or service has not been evaluated, for the Degree choose ‘Not evaluated’. Otherwise choose ‘Conformant’ or ‘Not conformant’ as appropriate.

To record conformance against an additional report, switch to the Advanced editing view, change to the Quality tab and add a new report using the plus button within the appropriate Data quality element.

Data quality - Quality scope

Gemini:

Quality Scope

INSPIRE:

metadata/2.0/req/datasets-and-series/one-data-quality-element

ISO19115:

DQ_DataQuality.scope

Obligation:

Mandatory

occurrence:

Many

Definition:

A sub-element of Data quality, this is specific scope to which the data quality assessment applies.

Guidance:

At least one Data quality element scoped to the entire dataset or series to which the metadata record refers. For a dataset, set quality scope to ‘dataset’, for a dataset series, set quality scope to ‘series’.

If using a Data quality element to describe something other than the whole dataset or series, then provide additional data quality elements with their scope set appropriately. Switch to the Advanced editing view, change to the Quality tab, and Data Quality info element using the plus button at the base of the page. Set the quality scope to the appropriate level using the dropdown list.

userdoc_fig_7_4_1_ConformanceResult

Dataset language

Gemini:Dataset language
INSPIRE:Resource language
ISO19115:MD_DataIdentification.language
Obligation:Conditional
occurrence:Many
Definition:The language used within the dataset (assuming the data resource contains text/written information, e.g. in attribute tables).
Guidance:Select a language from the drop down menu listing entries from the ISO 639-2 code list. For INSPIRE compliance, this has to be an official language of the European Community, of which English (eng) is the only one in common use across the UK (and is the default). For non-INSPIRE metadata records, it can be any ISO 639-2 three letter code, of which the relevant entries for the UK are English (eng), Welsh (cym), Gaelic (Irish) (gle), Gaelic (Scottish) (gla), Cornish (cor), Ulster Scots (sco).

userdoc_fig_7_13_1_DatasetLanguage

Dataset reference date

Gemini:

Dataset reference date

INSPIRE:

Date of publication

ISO19115:

MD_Identification.citation > CI_Citation.date

Obligation:

Mandatory

occurrence:

Many

Definition:

Reference date for the data resource, indicating the date of publication, creation and/or last revision. Note that for services, this date relates to the creation, publication, and/or revision of the service, not the dataset(s) within it.)

Guidance:

This element is a notional date of publication of the data resource. It is different from the temporal extent, which is the date of currency of the resource. Multiple dataset reference dates can be defined, but there must only be one of type creation and only one of type revision.

Dates may be to any degree of precision, from year (YYYY) to full date and time. The extended date format (YYYY-MM-DD) should be used, where YYYY is the year, MM the month, and DD the day. If required, time (HH:MM:SS, where HH is the hour, MM the minute, and SS the second) may be added, with ‘T’ separating the two parts.

userdoc_fig_7_6_1_ReferenceDate

Extent - Geographic bounding box

Gemini:Bounding box
INSPIRE:Geographic bounding box
ISO19115:MD_DataIdentification.extent > EX_Extent > EX_GeographicExtent > EX_GeographicBoundingBox
Obligation:Mandatory
occurrence:Many
Definition:Rectangle enclosing the extent of the data resource described in latitude and longitude, to enable the resource to be located geographically. It has four sub-elements: west bounding longitude, east bounding longitude, south bounding latitude, and north bounding latitude.
Guidance:A pre-defined extent can be chosen from the drop down menu, or bounding coordinates can be manually entered in the corresponding boxes. The west bounding longitude should be less than the east bounding longitude, and the north bounding latitude must be greater than the south.

userdoc_fig_7_8_1_GeogBoundingBox

Extent - Geographic description

Gemini:Extent
INSPIRE:n/a
ISO19115:MD_DataIdentification.extent > EX_Extent > EX_GeographicExtent > EX_GeographicDescription.geographicIdentifier
Obligation:Optional
occurrence:Many
Definition:The geographical extent of the data resource relative to an administrative hierarchy. Note that it is the coverage of the data resource, not the individual objects in the data resource. Thus if the data resource was national parks in Scotland, the extent would be ‘Scotland’, even though many parts of Scotland do not have National Parks. Sub-elements of the geographic description are the code identifying the extent and the designating authority (optional).
Guidance:Enter a pre-defined extent of country or Local Authority using one of the controlled lists available in the drop down above the bounding box element (e.g. Geonames). Note that this element may also be populated using a linked data register entry, for example http://statistics.gov.scot/id/statistical-geography/S92000003.

userdoc_fig_7_9_1_GeogDescription

Extent - Temporal extent

Gemini:

Temporal extent

INSPIRE:

Temporal extent

ISO19115:

EX_Extent > EX_TemporalExtent.extent

Obligation:

Mandatory (dataset), Conditional (service)

occurrence:

Many

Definition:

This is the date(s) that describes the time period covered by the content of the dataset. It may refer to (a) the period of collection of the data or (b) the date at which it is deemed to be current. Temporal extent is the date of the validity of the data, and is different from the dataset reference date (i.e. publication).

Guidance:

Enter a Begin date and/or End date as required. Click on button_edit_calendar to activate the calendar selector. To add additional temporal extents, use the advanced editor view.

2| Scroll to the extent section (identification tab) and click the button_edit_plusdrop button next to the Temporal element label.

3| From the dropdown menu, select Temporal Extent. This will present a further button_edit_plusdrop button labeled Extent.

4| From the dropdown menu, select the extent type from either time edge, instant, node or period.

Note: if the temporal extent is composed of a begining and an end date, select time period. If it is a single date (e.g. census day) choose time instant. The time edge and node options are not often used.

5| If the time period option is chosen, use the button_edit_plusdrop buttons to add begining and end dates.

Enter a date, or two dates defining the duration of the period, as defined by BS ISO 8601. Dates may be to any degree of precision, from year (YYYY) to full date and time. The extended date format (YYYY-MM-DD) should be used, where YYYY is the year, MM the month, and DD the day. If required, time (HH:MM:SS, where HH is the hour, MM the minute, and SS the second) may be added, with ‘T’ separating the two parts. Periods are recorded as {fromdate/todate} (e.g. 2006-04-01/2007-03-31). Either the from date or the to date (but not both) of the time period may be left blank to indicate uncertainty.

userdoc_fig_7_10_1_TemporalExtent

Extent - Vertical extent

Gemini:

Vertical extent information

INSPIRE:

n/a

ISO19115:

MD_DataIdentification.extent > EX_Extent > EX_VerticalExtent

Obligation:

Optional

occurrence:

Many

Definition:

Describes the vertical domain (height range) of the data resource. The element is composed of the minimum value, maximum value and the vertical coordinate reference system (recorded as a name or code from a recognised thesaurus, i.e. EPSG Geodetic Parameter Registry).

Guidance:

This element should be completed only where the vertical extent is relevant (e.g. geology, mining, etc.). If it is relevant, then all sub-elements are mandatory.

To add the vertical extent:

1| In default (simple) view, scroll to the extent element and click button_edit_verticalextent.

2| Enter a minimum value in metres.

3| Enter a maximum value in metres.

4| Choose the appropriate vertical CRS from the recommended values dropdown list.

userdoc_fig_7_11_1_VerticalExtent

File identifier

Gemini:

File identifier

INSPIRE:

metadata/2.0/rec/common/fileIdentifier

ISO19115:

MD_Metadata.fileIdentifier

Obligation:

Mandatory

occurrence:

One

Definition:

Unique identifier for this metadata file.

Guidance:

To support the operation of UK Location and INSPIRE, discovery metadata records must include a File Identifier for the resource. This will be auto-generated by a metadata creation tool (e.g. the metadata portal itself or an external metadata editor) and once created cannot be changed.

File identifier should not be confused with the UK GEMINI2 metadata item Resource identifier, which identifies the data resource being described.

userdoc_fig_7_31_1_FileIdentifier

Hierarchy level name

Gemini:Hierarchy level name
INSPIRE:Hierarchy level
ISO19115:MD_Metadata.hierarchyLevelName
Obligation:Mandatory for series and services, optional for datasets
occurrence:One
Definition:The name of the hierarchy level for which the metadata is provided (eg dataset, series, service).
Guidance:Set to ‘service’, ‘series’, or ‘dataset’ as appropriate. Note that in UK Gemini this element is conditional for datasets.

userdoc_fig_7_19_1_MetadataHierarchyLevel

Keyword

Gemini:

Keyword

INSPIRE:

Keyword

ISO19115:

MD_Identification.descriptiveKeywords > MD_Keywords

Obligation:

Mandatory

occurrence:

Many

Definition:

Terms covering the subject of the data resource that are more specific than those entered under topic category. Ideally, these will be standardised keywords originating from a controlled vocabulary, so that resources can be identified in any search.

Guidance:

It is recommended that keyword values be taken from a standardised subject vocabularies, such as General Environmental Multi-Lingual Thesaurus (GEMET) or the Integrated Public Sector Vocabulary (IPSV), and the formal citation provided (including the date, version and any amendments where appropriate). This will enable other users to perform more efficient searches and eliminate resources that are of no interest more easily.

If the dataset is covered under INSPIRE, then the first keyword should be from the General Environmental Multi-Lingual Thesaurus (GEMET) - INSPIRE Spatial Data Themes list. Service records must include a keyword from the INSPIRE Classification of spatial data services code list.

To add new keywords from a controlled vocabulary to the editing form:

1| Click the button_edit_selectthesaurus button below the keywords element.

2| Select the desired thesaurus (e.g. GEMET - INSPIRE themes, version 1.0). This will add a search box for the thesaurus.

3| Click to select the relevant keyword. The citiation for the originating vocabulary will be pre-populated.

Note: alternately, free text keywords can be added by clicking the button_edit_addkeyword button.

userdoc_fig_7_12_1_Keywords

Limitations on public access

Gemini:

Limitations on public access

INSPIRE:

Limitations on public access

ISO19115:

MD_Identification.resourceConstraints > MD_LegalConstraints.accessConstraints & otherConstraints

Obligation:

Mandatory

occurrence:

Many

Definition:

Restrictions imposed on access to the data resource for security and other reasons (i.e. who can see the data).

Guidance:

Limitations on public access is different from the use constraints element, which describes limitations on using the data, such as fees or licencing restrictions, rather than the access to it. A data resource can be openly accessible (which all INSPIRE data should be), but have restrictions on its use such as licensing, fees, or usage limitations.

For INSPIRE purposes, the Limitations on public access dropdown box must be set to ‘other restrictions’. The anchor element below must then be populated with an appropriate label from the INSPIRE code list for Limitations on public access. Change the anchor label to the appropriate entry from the code list, and ensure the URL points to the corresponding code list URI.

Article 13 of the Directive contains a list of cases where limitations on public access can be set. With regards to providing the metadata for the datasets and services through discovery services, the limitations on public access can be set on the base of reasons of international relations, public security or national defence. Concerning providing View, Download or Transformation Services, or e-commerce services, limitations on public access can be set on the base of the following reasons:

  • the confidentiality of the proceedings of public authorities, where such confidentiality is provided for by law;
  • international relations, public security or national defence;
  • the course of justice, the ability of any person to receive a fair trial or the ability of a public authority to conduct an enquiry of a criminal or disciplinary nature;
  • the confidentiality of commercial or industrial information, where such confidentiality is provided for by national or Community law to protect a legitimate economic interest, including the public interest in maintaining statistical confidentiality and tax secrecy;
  • intellectual property rights;
  • the confidentiality of personal data and/or files relating to a natural person where that person has not consented to the disclosure of the information to the public, where such confidentiality is provided for by national or Community law;
  • the interests or protection of any person who supplied the information requested on a voluntary basis without being under, or capable of being put under, a legal obligation to do so, unless that person has consented to the release of the information concerned;
  • the protection of the environment to which such information relates, such as the location of rare species.

This element shall only include information regarding access to the resource (not the use of the data, which is documented under the Use constraints section). When Member States limit public access to spatial data sets and spatial data services under Article 13 of Directive 2007/2/EC, this metadata element shall provide information on the limitations and the reasons for them. If there are no limitations on public access, this metadata element shall indicate that fact.

userdoc_fig_7_14_1_LimitationsPublicAccess

Lineage

Gemini:Lineage
INSPIRE:Lineage
ISO19115:DQ_DataQuality.lineage > LI_Lineage.statement
Obligation:Mandatory
occurrence:One
Definition:A sub-element of data quality that should provide information about the events or source data used in the creation of the data resource. This will be useful in determining whether the data is fit for purpose.
Guidance:The lineage differs from the abstract in that it covers ‘how’ the dataset was created as opposed the ‘what’ and ‘why’ of the dataset. A brief technical description should be given noting any sources and processes used. Any procedures or protocol associated with the update of the dataset should also be noted, along with notes on previous updates.

userdoc_fig_7_15_1_Lineage

Maintenance information

Gemini:Maintenance information
INSPIRE:Maintenance information
ISO19115:MD_MaintenanceInformation.maintenanceAndUpdateFrequency
Obligation:Optional
occurrence:One
Definition:Information about the scope and frequency of updating. Note that this identifies how often the updated data resource is made available to the user (for instance a data resource may be updated continuously, but released to the user only monthly).
Guidance:Choose the appropriate frequency from the drop down list. If the update cycle is unknown, please choose ‘Unknown’ from the list.

userdoc_fig_7_16_1_UpdateFrequency

Metadata date stamp

Gemini:Metadata date
INSPIRE:Metadata date
ISO19115:MD_Metadata.dataStamp
Obligation:Mandatory
occurrence:One
Definition:The date on which the metadata was last updated.
Guidance:This element is not editable and is set by the editor when the file is saved. It is used by spatialdata.gov.scot and data.gov.uk to determine if metadata with the same field identifier (UUID) has been changed. A single date is specified in the extended format YYYY-MM-DD, where YYYY is the year, MM is the month and DD is the day.

userdoc_fig_7_18_1_MetadataDate

Metadata language

Gemini:Metadata language
INSPIRE:Metadata language
ISO19115:MD_Metadata.language
Obligation:Mandatory
occurrence:One
Definition:The language used to document the metadata. The purpose of this element is to identify the language used in a multi-lingual metadata service, for example in the INSPIRE geo-portal.
Guidance:Selected a language from the drop down menu listing entries from the ISO 639-2 code list. For INSPIRE compliance, this has to be an official language of the European Community, of which English (eng) is the only one in common use across the UK (and is the default in the SSDI). For non-INSPIRE metadata records, it can be any ISO 639-2 three letter code, of which the relevant entries for the UK are English (eng), Welsh (cym), Gaelic (Irish) (gle), Gaelic (Scottish) (gla), Cornish (cor), Ulster Scots (sco).

userdoc_fig_7_20_1_MetadataLanguage

Metadata point of contact

Gemini:

Metadata point of contact

INSPIRE:

Metadata point of contact

ISO19115:

MD_Metadata.contact > CI_ResponsibleParty

Obligation:

Mandatory

occurrence:

Many

Definition:

This element records the details of the organisation(s) responsible for the creation and maintenance of the metadata record. The structure of this element is the same as the Point of contact element. There are eight sub-elements:

  • Organisation name
  • Position name (i.e. job role or position of the responsible person or business area)
  • Voice (i.e. telephone number)
  • Facsimile (i.e. facsimile number)
  • Address (i.e. postal address as defined by Royal Mail)
  • Electronic mail address (i.e. email address)
  • Resource locator (i.e. web address of the organisation)
  • Role (of the responsible party with respect to the metadata)
Guidance:

Of the eight sub-elements, only the organisation name, email address and role are mandatory. All other sub-elements are optional. With regards to the organisation name, this should be provided in full without abbreviations. In terms of the role, for INSPIRE purposes this must be set to point of contact.

For position name, a general job title (e.g. Data Manager) should be identified rather than individuals which are subject to change without notice and difficult to maintain. Likewise, email addresses should be provided for branch or team (i.e. shared) mailboxes where possible rather than for individuals.

If the user has stored contact details in a directory entry on the portal, details can be auto-populated by searching for the contact in the search box below the element.

userdoc_fig_7_17_1_MetadataContact

Metadata standard name

Gemini:Metadata standard name
INSPIRE:N/A
ISO19115:MD_Metadata.metadataStandardName
Obligation:Optional in GEMINI but required in spatialdata.gov.scot
occurence:One
Definition:Name of the metadata standard or profile used, cited with a reference to the appropriate register entry.
Guidance:This element is required in spatialdata.gov.scot and must be set to UK GEMINI when importing metadata records into the portal. For records created within the portal from templates, this element will be automatically filled in, and will not be editable in the default editing view.

Metadata standard version

Gemini:Metadata standard version
INSPIRE:N/A
ISO19115:MD_Metadata.metadataStandardVersion
Obligation:Optional in GEMINI but required in spatialdata.gov.scot
occurence:One
Definition:Version of the metadata standard (profile) used.
Guidance:This element is required in spatialdata.gov.scot and must be set to 2.3 when importing metadata records into the portal. For records created within the portal from templates, this element will be automatically filled in, and will not be editable in the default editing view.

userdoc_fig_7_32_1_MetadataStandard

Parent identifier

Gemini:Parent identifier
INSPIRE:N/A
ISO19115:MD_parentidentifier.scope
Obligation:Optional
occurrence:One
Definition:File identifier of the metadata that is a parent to this child metadata. It supports parent-child relationships in metadata and allows navigation from a dataset record to the series record of which it forms a part.
Guidance:Only to be used if the dataset is part of a series. To add a parent identifier, click button_edit_addparent and add the Unique Identifier for the parent record.

userdoc_fig_7_33_1_ParentIdentifier

Resource identifier

Gemini:Resource identifier
INSPIRE:Unique resource identifier
ISO19115:MD_DataIdentification.citation > CI_Citation.identifier
Obligation:Mandatory (datasets), Optional (services)
occurrence:Many
Definition:A value uniquely identifying the data resource within your organisation (i.e. how your organisation references the dataset).
Guidance:The Code element is mandatory, and should be a recognisable identifier within the context of your organisation (i.e. a unique code of numbers, letters or combination of referring to a single dataset). If it is not guaranteed to be unique across the UK and the wider EU (INSPIRE), then the Codespace sub-element must be specified; this should be the internet domain of the data provider/owner.

userdoc_fig_7_3_1_CitationIdentifier

Resource locator

Gemini:Resource locator
INSPIRE:Resource locator
ISO19115:MD_Distribution > MD_DigitalTransferOptions.online > CI_OnlineResource.linkage
Obligation:Conditional
occurrence:Many
Definition:Location (address) for on-line access to the resource using a Uniform Resource Locator (URL). This element should point to where the dataset may be accessed, and may be different from where it may be ordered online (which should be included in the web address of the distributor).
Guidance:This element should primarily be used to enter URLs for web services (i.e. WMS, WFS, etc.), however, links to web pages offering more information or other services (e.g. interactive mapping applications) can also be added. To add an online resource, follow the instructions provided in the Associated resources section. Once an online resource has been added, it can be edited as normal in the editing form. Note that for web services to be displayed in the interactive map, the protocol, layer name and description must be entered. The layer name must match that as defined in the GetCapabilities request of the service.

userdoc_fig_7_21_1_OnLineResource

Resource type

Gemini:Resource type
INSPIRE:Resource type
ISO19115:MD_Metadata.hierarchyLevel
Obligation:Mandatory
occurrence:One
Definition:Scope to which the metadata applies (i.e. dataset, series, service).
Guidance:Identify whether resource is a dataset or a series (collection of datasets with a common specification). For metadata created on spatialdata.gov.scot, this element is set by the choice of template and should not be changed.

userdoc_fig_7_5_1_HierarchyLevel

Responsible organisation

Gemini:

Responsible organisation

INSPIRE:

Responsible party

ISO19115:

MD_Identification.pointOfContact

Obligation:

Mandatory

occurrence:

Many

Definition:

This element records the details of the organisation(s) responsible for the creation, maintenance and distribution of the data resource. The structure of this element is the same as the Metadata contact element. There are eight sub-elements:

  • Organisation name
  • Position name (i.e. job role or position of the responsible person)
  • Voice (i.e. telephone number)
  • Facsimile (i.e. facsimile number)
  • Address (i.e. postal address as defined by Royal Mail)
  • Electronic mail address (i.e. email address)
  • Resource locator (i.e. web address of the organisation)
  • Role (of the responsible party with respect to the resource)
Guidance:

Of the eight sub-elements, only the orgnaisation name, email address and role are mandatory. All other sub-elements are optional. With regards to the organisation name, this should be provided in full without abbreviations. In terms of the role, if a responsible party is both the creator, publisher and distributor of the resource then the role should be set to Publisher. If the resource was created by a party other than the provider this should also be recorded, using the role value Originator.

For position name, a general job title (e.g. Data Manager) should be identified rather than individuals which are subject to change without notice and difficult to maintain. Likewise, email addresses should be provided for branch or team (i.e. shared) mailboxes where possible rather than for individuals.

If the user has stored contact details in a directory entry on the portal, details can be auto-populated by searching for the contact in the search box below the element.

userdoc_fig_7_22_1_PointofContact

Spatial reference system

Gemini:

Spatial reference system

INSPIRE:

n/a

ISO19115:

MD_ReferenceSystem.referenceSystemIdentifier > RS_Identifier.code

Obligation:

Mandatory

occurrence:

Many

Definition:

Identifier, name or description of the system of spatial referencing, whether by coordinates or geographic identifiers, used in the data resource.

Guidance:

The reference system should be recorded referencing a well known common register (e.g. EPSG Geodetic Parameter Registry) in a URL link format. For INSPIRE purposes the first entry must be one of the reference systems listed in Annex D.4 (Default Coordinate Reference Systems) of the INSPIRE Metadata Technical Guidelines. For this reason it is recommended that ETRS89 is included as the first entry, and if desired British National Grid as a secondary entry. Failure to do so will result in a validation error.

The box below the element can be used to search for and add additional reference systems.

userdoc_fig_7_23_1_ReferenceSystem

Spatial representation type

Gemini:Spatial representation type
INSPIRE:Spatial representation type
ISO19115:MD_DataIdentification.spatialRepresentationType
Obligation:Mandatory
occurrence:Many
Definition:The method used to spatially represent geographic information.
Guidance:This element uses the MD_SpatialRepresentationTypeCode from ISO 19115. Available options are vector (the defualt), grid (for images and coverage data), tin (surface data), and text, table (for datasets with an indirect spatial reference).

userdoc_fig_7_34_1_SpatialRepresentationType

Spatial resolution - Distance

Gemini:Spatial resolution
INSPIRE:Spatial resolution
ISO19115:MD_Identification.spatialResolution > MD_Resolution.distance
Obligation:Conditional
occurrence:Many
Definition:A distance measure of the granularity (in metres), providing an indication of how detailed the data is. It is equivalent to the ground sample distance and should not be confused with the scale of a map (which is purely a display attribute).
Guidance:Enter values that are real numbers, greater than 0, and specified in metres. Commonly used distances can be added from the recommended values drop down next to the element. For data captured in the field, it is the precision at which the data is captured (this may be the accuracy for topographic surveys, or the average sampling distance in an environmental survey). For data taken from maps, it is the positional accuracy of the map, while for image data it is the resolution of the image.

userdoc_fig_7_24_1_SpatialResDistance

Spatial resolution - Equivalent scale

Gemini:Equivalent scale
INSPIRE:Equivalent scale
ISO19115:MD_Identification.spatialResolution > MD_Resolution.equivalentScale > MD_RepresentativeFraction.denominator
Obligation:Optional
occurrence:Many
Definition:The level of detail expressed as the scale denominator of a comparable hardcopy map or chart.
Guidance:Where the data is captured from a map, the scale of that map should be recorded as a positive integer. Note that distance is the preferred expression for spatial resolution. The equivalent scale should only be given when the distance cannot be determined.

userdoc_fig_7_25_1_SpatialResEqScale

Title

Gemini:Title
INSPIRE:Resource title
ISO19115:MD_DataIdentification.citation > CI_Citation.title
Obligation:Mandatory
occurrence:One
Definition:The name given to the data resource.
Guidance:This should be the formal or product name if one exists. Otherwise the title should be created that is short, encapsulates the subject, temporal and spatial coverage of the data resource, and does not contain terms or jargon that make it incomprehensible.

userdoc_fig_7_27_1_Title

Topic category

Gemini:

Topic category

INSPIRE:

Resource topic category

ISO19115:

MD_DataIdentification.topicCategory

Obligation:

Mandatory (datasets), N/A (services)

occurrence:

Many

Definition:

Describes the main theme(s) of the data resource, using topic categories in accordance with ISO 19115 standard.

Guidance:

Select from the drop down list one or more categories that most closely represent the topic of the data resource. This element is only meant to represent the general theme of the information. Keywords should be used to provide greater detail on the nature of the dataset. While more than one topic category can be applied, only a limited number of most relevant should be chosen (e.g. topographic maps should not be classified as farming). Multiple topic categories can be added.

Note that the choice of topic category will dictate which categories the data resource is listed under on the Browse by topics section of the homepage.

userdoc_fig_7_28_1_TopicCategory

Use constraints

Gemini:

Use constraints

INSPIRE:

Conditions applying to access and use

ISO19115:

MD_Identification.resourceConstraints > MD_LegalConstraints.useConstraints & otherConstraints

Obligation:

Mandatory

occurrence:

Many

Definition:

Restrictions and legal constraints on using the data resource. This can be entered as a free text statement, or link to a URL containing the information.

Guidance:

Use constraints are different from limitations on public access which describe limitations on access to the data. A data resource can be openly accessible (which all INSPIRE data should be), but have restrictions on its use such as licensing, fees, or usage limitations.

This element comprises two sub-elements: the ‘use constraints’ which should be set to other restrictions, and ‘other constraints’ which can be free text or a URL link.

1| useConstraints- should contain an MD_RestrictionCode element with code list value “otherRestrictions”

2| otherConstraints- a free text element describing the usage restrictions or link to a URL, as below.

To link to a URL in the otherConstraints element, click the button_edit_plusdrop button and choose either the anchor option. All records on spatialdata.gov.scot should document the licencing arrangements for the data resource using the anchor type. Where possible, this should link to a URL such as the Open Government Licence, Non-Commercial Government Licence, or INSPIRE End User Licence. Users can further categorise their records in terms of licence type by following the guidance in the assigning a licence category section.

userdoc_fig_7_29_1_UseConstraints

Annex 1: Requirements for metadata to be correctly identified as UK Gemini

Records that are imported into SpatialData.gov.scot (rather than created within the portal using the available templates) must include the following XML elements to ensure that the record is correctly identified as UK GEMINI upon import.

Note: Using the default to-gemini23 xsl transformation when importing will do this for you.

Root element

The root element should include a link to the gml 3.2 namespace and should not include xsi:schemalocation as this is handled internally:

<gmd:MD_Metadata xmlns:gmd="http://www.isotc211.org/2005/gmd"
              xmlns:srv="http://www.isotc211.org/2005/srv"
              xmlns:gts="http://www.isotc211.org/2005/gts"
              xmlns:gco="http://www.isotc211.org/2005/gco"
              xmlns:gml="http://www.opengis.net/gml/3.2"
              xmlns:xlink="http://www.w3.org/1999/xlink"
              xmlns:gmx="http://www.isotc211.org/2005/gmx"
              xmlns:gsr="http://www.isotc211.org/2005/gsr"
              xmlns:gss="http://www.isotc211.org/2005/gss"
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

Metadata standard name and version

The correct metadata standard name and version should be included. Note these are case-sensitive:

 <gmd:metadataStandardName>
    <gmx:Anchor xlink:href="http://vocab.nerc.ac.uk/collection/M25/current/GEMINI/">UK GEMINI</gmx:Anchor>
</gmd:metadataStandardName>
 <gmd:metadataStandardVersion>
    <gco:CharacterString>2.3</gco:CharacterString>
 </gmd:metadataStandardVersion>

Annex 2: Upgrading to UK Gemini 2.3

SpatialData.gov.scot has now implemented UK GEMINI 2.3. All new metadata created on the portal will conform to version 2.3, and metadata publishers are strongly encouraged to upgrade their existing 2.2 records as soon as possible. This section provides guidance on how users can upgrade their existing metadata via a number of methods. For detail on what has changed from version 2.2 to 2.3, please see information provided by the Association of Geographic Information (AGI).

The conversion tools will make the following changes to 2.2 records to make them compliant with Gemini 2.3:

  • Add or otherwise update the MetadataStandardName to UK GEMINI and the MetadataStandardVersion to 2.3.
  • Change the encoding of the Access Constraints and Use Limitations elements to meet the new standard, either making a best guess at moving your existing constraints into the correct encoding, or adding the default “no limitations” constraints as appropriate.
  • Adds the default INSPIRE Conformance Report for datasets or services if one does not already exist.
  • Replaces any references to http://www.opengis.net/gml with http://www.opengis.net/gml/3.2.
  • Adds a default Character Encoding of UTF8.

Updating through an editing session

Existing records on SpatialData.gov.scot can be converted to Gemini 2.3 through an editing session. Before users begin they should ensure that their existing metadata record contains the following tags which identify it as Gemini 2.2:

<gmd:metadataStandardName>
        <gco:CharacterString>UK GEMINI</gco:CharacterString>
</gmd:metadataStandardName>
<gmd:metadataStandardVersion>
        <gco:CharacterString>2.2</gco:CharacterString>
</gmd:metadataStandardVersion>

To convert an existing record to Gemini 2.3:

1| Follow the steps to begin an editing session.

2| On the right hand side of the editing board, run the suggestions wizard.

Note: if the if the record is currently recognised as Gemini 2.2, the following suggestion should appear: Metadata Schema is Gemini 2.2. Run this task to convert to Gemini 2.3.

3| Click the suggestion to run the transformation script.

Note: the record will be converted to Gemini 2.3 and the editing view will change to show the embedded 2.3 guidance.

4| Ensure the record is valid by clicking button_edit_validate in the header bar. See validating metadata for more information.

Note: The conversion process will not automatically create valid metadata. Users must ensure that the record validates and correct any errors.

5| Click button_edit_saveclose to save and close the record.

Updating records on import

Metadata records can also be converted from 2.2 to 2.3 on import into SpatialData.gov.scot.

1| Follow steps 1 to 4 of importing an existing XML metadata record.

2| For the XSLT transformation, select GEMINI 2.2 to GEMINI 2.3 from the drop down menu.

3| Follow the remaining steps to import the record.

4| Users should ensure the imported metadata is valid. See validating metadata for more information.

Updating harvested records

Some organisations maintain their own spatial data portals and have their metadata harvested on a scheduled basis by SpatialData.gov.scot. Depending on what version of metadata is being supplied, an xslt conversion may be required on each harvest. Publishers who maintain their records via harvest should contact SSDI.metadata@gov.scot with any queries.

Updating through a batch process

Administrator users on SpatialData.gov.scot will have the ability to convert records to Gemini 2.3 via a batch process. See the administrator guidance for more information.

Common errors

The following is a list of common errors encountered when converting from Gemini 2.2. to 2.3 that cannot be addressed by the transformation script.

  • Abstract Length: Gemini 2.3 requires and abstract with a minimum length of 100 characters.

Annex 3: Best Practice Guide

Good quality metadata is key to understanding whether data is fit for purpose, has value, and is reliable - particularly if it is to inform policy decisions. Further, it is noted that most users search for data through general search engines such as Google. Since data is often not readable by search engines it is generally recognised that searches are performed on the metadata, rather than the data itself. This means that improvements to the metadata content can have the most significant impact on data discovery.

This section aims to provide users with best practice guidance on how to improve their metadata quality to improve data discoverability.

  • Title
  • Abstract
  • Links in records:
    • http rather than https
    • 404s
    • Protocol

This section is currently under development.

Introduction

Overview

This documentation is intended to provide guidance to administrators of the SSDI Catalogue. It assumes familiarity with the user guidance, particularly around the creation and validation of metadata.

The administrative functionality within the catalogue is accessed via the Admin console, on the top menu when you log in with an administrator profile. Additional functionality, such as downloading catalogue-wide reports, is available from the advanced search page and also the editing dashboard.

image0

Figure 1.1.1: SSDI catalogue main page, showing admin console link

image1

Figure 1.1.2: SSDI catalogue admin console, for a full administrator

The level of functionality available in the admin console is dependent on the privileges of the logged in user. The above image shows the admin console for a full Administrator. This page also provides a quick “at a glance” view of the number of records available, and the numbers of datasets and services.

Searches

Advanced Workflow

The editing dashboard can be used to perform administrative functions on individual or batch metarecords. Tools for acting on individual records are represented by icons to the right of the metadata title. The validation status of a record is indicated by the colour of it’s icon:

  • Green- validated and passing checks
  • Red- validated but not passing checks
  • Grey- not yet validated
  • White- a template or other record that should not be validated

image-5

Figure 2.1.1: Administrative functions that act on individual records, accessed from the editing dashboard

In order from left to right these are:

  • Privileges- set privileges on which groups can access or edit this record
  • Edit- access the metadata editor for this record
  • Delete this record
  • Duplicate record
  • Create a child record

image-6

Figure 2.1.2: Administrative functions that act on multiple records

To act on multiple metadata records, use the select function and choose the appropriate function from the dropdown list:

  • Export (ZIP)- exports the records in MEF format
  • Export (PDF)- exports as a combined PDF- note this will disregard any templates
  • Export (CSV)- exports as a combined CSV- note this will disregard any templates
  • Update privilges- note that this will UPDATE privileges rather than APPEND to existing ones
  • Publish/Unpublish- add or remove the “publish” privilege from the “All” group
  • Transfer Ownership
  • Update categories
  • Validate
  • Delete

Metadata and Templates

The Metadata and Templates button provides access to four subsections, as shown in the image below:

image2

Figure 2.2.1: Metadata and Templates subsections

Metadata and Templates Standards available

This shows the metadata standards or profiles loaded onto the system. Selecting one or more of the standards activates the options to Load templates and Load samples for the selected standards.

Note that the templates have been already loaded for iso19139.gemini22

Formatter

Advanced

This allows advanced administrators/system maintainers to customise how metadata is displayed within Geonetwork. For further information on this topic see the Geonetwork help documentation at http://geonetwork-opensource.org/manuals/trunk/eng/users/customizing-application/creating-custom-view.html?highlight=formatter

Schematron

Advanced

This section allows advanced administrators/system maintainers to see which schematron rules are in place for specific schemas and to edit them as required. It is not recommended that changes are made in this section as it will impact on how records are validated.

Metadata Identifier Templates

Advanced

By default this functionality is disabled in the admin settings, and identifiers are generated automatically by geonetwork when a record is created or imported.. When enabled in the admin settings, this section allows the customisation of the identifier. The default is a randon UUID string, but identifier templates allow a pre-configured format with a template for rendering any user-generated elements. For example:

  • scot.gov::{IDCODE} would render the IDCODE as an element to be completed during metadata creation
  • scot.gov::{SLA}:{ID} would render both SLA and ID as elements to be completed during metadata creation

When this setting is enabled, additional options are available when creating a record from a template. When importing records, the existing file identifier element is used as the UUID. An error is triggered if this is not unique in the catalogue.

Harvesting

Harvesting allows you to consume metadata from external services on a scheduled basis. The harvester section has two subsections, as shown in the image below. It also shows any existing harvester nodes set up on the system, and their current status (running or paused).

image3

Figure 3.1.1: Harvesting Section

Harvester Settings

From this section you can see the list of available harvesting nodes, and clone an existing node or add a new one.

To add a new harvester node, either clone an existing one by selecting it from the dropdown Clone list in the above image, or click the “Harvest from” button to access the dropdown list of harvester types. The configuration options available will depend on the type of node selected.

image4

Figure 3.2.1: Common harvester node configuration options, top half of screen

image5

Figure 3.2.2: Common harvester node configuration options, bottom half of screen

Common configuration options are:

  • Node name and logo: a unique name and optional logo for this harvesting node
  • Group: The group that the harvester should belong to
  • Schedule: Should the harvester run repeatedly to a schedule or just run the once
  • Delete: delete this harvester node and all related records
  • Remove records: delete records but leave harvester in place
  • Save: save changes to harvester configuration
  • Harvest: run this harvest
  • Category: set the category that the harvested records should be added to
  • Validate records before import``: Reject invalid records before importing them into the catalogue. This has the following options``:
    • Accept all metadata without validation (this option will import all records regardless of validity)
    • Accept metadata that are XSD valid (this option will cause the harvest to fail on the first invalid record)
    • Accept metadata that are XSD and Schematron valid (this option will cause the harvest to fail on the first invalid record)

Below the configuration options is a report on the harvester history if available, and the number of records harvested in the last harvest. Note that if the configuration changes at all these will be reset.

For information on the configuration options for the different harvester node types, see the Geonetwork documentation at http://geonetwork-opensource.org/manuals/trunk/eng/users/user-guide/harvesting/index.html

Harvester Reports

This section allows you to see the harvester history for all nodes and export the results as a CSV file.

image6

Figure 3.3.1: Exporting harvest history as a CSV

Statistics and Status

This section provides a range of metrics and information on the health of the Geonetwork installation and the activity on it. It is split into five subsections.

Status

This subsection should be the first port of call if there are any problems with the Catalogue. If the catalogue is under a low load and is performing optimally then the status page should look like the one below:

image7

Figure 4.1.1: Status page showing catalogue performing optimally

image7a

Figure 4.1.2: Status page showing error, with detailed error message accessed by clicking on red link

Report any errors shown here to Astun Technology.

The metrics links provide advanced information on the status of the system and should not generally be needed. The Activity “Export (zip)” provides a log file from the server. This may be requested by Astun Technology support staff. This download could be large and may take some time. The Thread Status link provides advanced information on the java installation, and should not generally be needed.

Search Statistics

This is an extensive subsection showing the following, all additionally available for export as CSV:

  • Search statistics, such as the number of catalogue views by day or month
  • Statistics for the CSW endpoint
  • Number of searches by timeperiod and type, shown as a graph
  • Types of services for which a search is triggered, such as export to MEF, RDF, keyword search in portal, shown as a Pie Chart
  • IP address for each search
  • Search fields and terms
    • This allows the choice of the type of search, which then displays the criteria used and the number of searches
    • Clicking on a criteria shows the breakdown of number of searches per term for that criteria
    • For example, to see a breakdown of search keywords, choose “Search” from the dropdown box and then click the “Keyword” link in the list below (see Figure 4.2.3)

image8

image9

image10

Figures 4.2.1/2/3: Various elements of the search statistics interface

Content Statistics

This subsection provides more information on content searches. The following options are available, for export as CSV:

  • Catalog content statistics, such as the number of harvested records, total number of records, number of public records and so on
  • Metadata records popularity, showing the most popular records searched for (clicking the blue eye icon to the right of a record will take you to it)
  • Best Rated, showing the most highly rated records (if local star ratings are being used)
  • Record statistics, this is a dropdown list with a number of available metrics, for example (but not limited to):
    • Year
    • Format
    • INSPIRE theme(s)
    • Contact for the resource

image11

Figure 4.3.1: Catalog content statistics and Most popular records

  • A number of pie charts, showing the following metrics, where hovering over a slice will provide the actual number of records:
    • Category
    • Owner
    • Workflow Status
    • Validation Status

image12

Figure 4.3.2: Record statistics by Year

Information

This subsection provides information on the Geonetwork installation on the server. This section is for advanced administrator/system maintainer use.

It comprises the following sections:

  • Catalog information- where the files and folders are located on the server
  • Database- the connection status and connection string for connecting to the database
  • System information- the version of java in use and the amount of memory being consumed
  • Index- information about the configuration of the search index

Versioning

If metadata versioning is enabled on the server, which it is not, by default, then this section provides information about this process. This section is for advanced administrator/system maintainer use only.

Reports

This section contains various downloadable reports on user activity in the catalogue. Each provide the option to choose a date range, and the option to filter by group. The resulting reports are available for download as a CSV. The following reports are available:

  • Updated metadata- can be used to find records that have been, or not been, updated within a specific timeframe
  • Internal metadata- can be used to find records that are only available within their group
  • Metadata file uploads- shows records for which data has been attached (not used within SSDI)
  • Metadata file downloads- shows records for which data has been downloaded (not used within SSDI)
  • Users access- shows user names, emails and last login dates

image13

Figure 5.1.1: The reporting section

Classification Systems

This section provides access to the Thesauri and other directories used within the catalogue. It is split into three subsections.

Thesaurus

Thesauri in rdf (skos) format can be managed or added here. It is also possible to interrogate the existing thesauri loaded into the catalogue.

image14

Figure 6.1.1: The thesaurus subsection showing the customised Scottish Regions thesaurus

Selecting a thesaurus from the list provides further information about it. The settings here should not be changed as they will affect the working of the catalogue.

It is possible to add additional thesauri by clicking the “Add thesaurus” link. The options are as follows:

  • New thesaurus- build one from scratch in Geonetwork
  • From local file- upload a thesaurus in rdf (skos) format from your local hard drive
  • From URL- provide a link to a compatible thesaurus online

Directories

Directories are snippets of information that can be pre-completed and inserted into metadata records. They are derived from subtemplates that are part of metadata standards. The available subtemplates are shown as tabs across the top of the Directory subsection- for Gemini 2.2 these are Organizations and contacts and Online resources.

Selecting Organizations and contacts shows the metadata snippet for adding a Points of Contact subtemplate. To create a new record based on this template, click the “Duplicate” icon (image14c) to the right of the template name.

image14a

Figure 6.2.1: The Organizations and contacts interface

Along the top of the entry window are, from left to right:

  • Group dropdown- which group should have ownership of this subtemplate
  • Sharing settings- which groups should have visibility of this subtemplate
  • Save
  • Save and close

It’s also possible to change the view that is used for completing this subtemplate, from default to xml view by clicking the blue “eye” symbol (image14d).

Once saved, the contact will be available using the “search for a contact” dialogue when creating or editng a metadata record.

image14b

Figure 6.2.2: Adding a point of contact snippet when editing a record

The dropdown box next to the contact name allows the user to choose the element where this snippet should be inserted.

Categories

This subsection lists the categories that records can be added to, and provides the option to add new categories or delete existing ones.

image15

Figure 6.3.1: Category list

Selecting a category from the list brings up a dialogue where it’s display name can be edited in each of the languages enabled in the catalogue, along with the options to save changes or delete the category.

image16

Figure 6.3.2: Selecting a category and displaying additional options

To add a new category, click the “New category” button shown in Figure 6.3.1 above. Add an unique name for the category and save it. To change it’s display name in any of the available languages, select it from the category list and edit as shown in Figure 6.3.2 above.

image17

Figure 6.3.3: Adding a new category

Standards

This section should only be used by advanced administrators and system maintainers. It lists the metadata standards or profiles that are loaded into the installation, along with the option to remove a standard, update it, or add a new one.

When adding a new standard or updating an existing one, there are three options:

  • Provide the path to the folder containing the standard on the server filesystem
  • Provide the UUID of a metadata record that contains a schema archive as an attached online resource
  • (Not shown in Figure 7.1.1) manually add or update the standard on the server in the schema_plugins location (found in the System Information panel) and restart the server

image18

Figure 7.1.1: The standards interface

Users and groups

This section is where existing users and groups are managed, and new ones are added. There are two subsections, described below.

Manage groups

This subsection allows you to list and edit the existing groups in the catalogue, and add new ones. Groups are listed on the left, and selecting a group brings up additional options, shown below, with the option to save any changes or delete this group.

image19

Figure 8.1.1: The groups list and top half of the groups editing interface

image20

Figure 8.1.2: The middle section of the groups editing interface

image21

Figure 8.1.3: The lower section of the groups editing interface (intermediate section on translations for this group name not shown)

To add a new group, click the blue “New group” button below the list (not shown in figure 8.1.1). This brings up the same dialogue shown in figures 8.1.1-8.1.3.

Manage Users

This subsection allows you to list and edit the existing users in the catalogue, and add new ones. Users are listed on the left, and selecting a user brings up additional options, shown below, with the option to save any changes or delete this user. For exsiting users there is also the option to reset the password.

The settings for editing or creating a user are as follows:

  • Enable (default is yes)- allow this user to log in, or not
  • User name (mandatory)- the name the user should use to log in
  • Password (mandatory, not shown for existing users)
  • Name (mandatory)
  • Surname (mandatory)
  • Organisation (optional)
  • Address fields (optional)
  • Is an administrator (default is no)- if checked the user will be a full administrator for the whole catalogue with full access to all functionality for all groups
  • Profiles per group- if not a full administrator then the user needs to be assigned at least one role or profile within the catalogue. Note that a user can be part of multiple groups and have different profiles per group
    • Registered User: can download protected data
    • Editor: has rights to create/delete/edit metadata within their group
    • Reviewer: has rights to authorise publication of metadata within their own group
    • User administrator: has rights to administer users, and create/delete/edit metadata within their group
  • Records owned by this user (if any)

image22

Figure 8.2.1: The users list and top section of user editing interface

image23

Figure 8.2.2: The middle section of the user editing interface

image24

Figure 8.2.3: The lower section of the user editing interface

Settings

This section should be used only by advanced administrators and system maintainers. This section is where the main configuration of for the installation is set up. It contains a number of subsections, as described below, but in general settings should only be changed by experienced staff as they can have an adverse effect on the running of the site.

Settings

The settings subsection provides access to the main configuration for the installation.

image25

Figure 9.1.1: The settings panel

The various elements are described in detail in the Geonetwork documentation at http://geonetwork-opensource.org/manuals/trunk/eng/users/administrator-guide/configuring-the-catalog/index.html so not all elements are described below, but there are some sections that may be useful:

  • Log levels- there are two of these, accessed via dropdown lists. The first is at the top of the settings panel, next to the “Save settings” button. The second is in the Catalog server subsection. The default for both is “Production” and this level should only be changed if increased logging is specifically requested. In that case, change to “Dev” and save settings, but remember to set it back to “Production” for normal use!
  • Catalog description- the Catalog name and Organisation are used throughout the catalogue and can be changed as required
  • Organisation- this is also used throughout the catalog and can be changed as required
  • Catalog- shows the version of Geonetwork in use
  • Catalog server- shows the URL and protocol (http or https) being used. Changing these values will trigger a change to internal URLs within the metadata records
  • Metadata Search Results- limits how many records can be selected in a single operation. Increasing this value may have a negative impact on the performance of the server
  • Catalog Service for the Web (CSW)- should this be enabled (default is yes). Note that there is a known issue where the option to set a contact to be used for GetCapabilities does not work. This is being investigated, and can be set elsewhere. In general the default settings for this section should be used.
  • User self-registration- if this option is set (default is no) then visitors to the catalogue can register as users (with the Registered User profile)
  • User feedback- deprecated in version 3.0.0 onwards
  • Search statistics- if enabled (default is yes) then Geonetwork will save statistics on searches in the database
  • INSPIRE Directive configuration- this section enables INSPIRE options in the CSW response from the catalogue and enables the INSPIRE search options in the advanced search panel.
  • Harvesters- allow editing on harvested records- if this is set (default is no) then records harvested from a remote location can be edited. Note that, if set, any changes would be overwritten by subsequent harvests
  • Harvester- this section contains settings to alert if a harvest has succeeded or failed. It is not configured by default
  • Metadata configuration- this section contains settings on which views are available, which is the default, and whether group logos should be used for records. The default values are generally sufficient
  • Metadata workflow- this section includes sections on the publication of invalid metadata. If the publication of invalid metadata is disabled, it is also possible to automatically set the status of invalid records to unpublished, and to force validation when a metadata record is saved.

The remaining settings are generally only configured on installation and should only be changed by experienced administrators or system maintainers.

Sources

This subsection is only used when a number of different sources are used for loading the data, such as harvesting nodes.

CSW

This subsection provides more detailed options for configuring the CSW service for the catalogue. In general the default options are sufficient.

The Contact setting allows the choice of a user within the catalogue to populate the contact information in the CSW GetCapabilities request. A specific user “Metadata Contact”, has been set up for this purpose.

The CSW Service information setting allows the configuration of some of the GetCapabilities elements, such as the Title, Abstract and so on. Fields are selected using a dropdown list, the language is chosen (default is English), and then the Value for the given setting can be added.

image27

Figure 9.3.1: The CSW settings interface

Virtual CSW

This subsection allows the configuration of Virtual CSW endpoints. These allow for different CSW URLs to be used within the catalogue, such as for different groups or metadata categories.

To configure a new Virtual CSW endpoint, click the blue “New Virtual CSW” button and then configure the following settings:

  • Name- This will be used as the URL endpoint for the virtual CSW. This must begin with csw-
  • Description
  • Filters/Query- the filter that should be used to decide on the records that should be published under this endpoint. Choose from the dropdown list, or use the advanced query option. The figure below shows an example endpoint for records belonging to the Aberdeen group

image28

Figure 9.4.1: Virtual CSW endpoint set up for Aberdeen group

Once a virtual endpoint has been saved, it’s capabilities can be checked using the link shown in figure 9.3.1

Note that the default contact information, as configured in the CSW subsection, will be used for the GetCapabilities request

CSW test

This subsection allows you to test various CSW requests on the standard endpoints or any virtual endpoints configured. The endpoints are displayed as radio boxes and the available requests are accessible from a dropdown list. Where it is necessary to adjust a parameter, such as a search term or record ID, this can be done in the XML section. To run the request click the green “Send CSW request” button. The response will appear below.

image29

Figure 9.4.2: Results of CSW “GetRecordById” request to the standard catalogue CSW endpoint, for record with ID 8fc49f46-2d6b-45b0-abd9-79e8aa2233cb

Map servers

This subsection allows you to set up the catalogue as an OGC publisher for WMS/WFS/WCS. In order to configure this section a Geoserver or Mapserver instance must be installed on the server. Not currently implemented in this installation.

Tools

This section contains some tools that can be run when maintenance is needed on the server. The subsections are as follows:

Index admin

In general it is not necessary to run these tools unless transferring a large number of metadata records, changing the catalogue’s appearance, or if unexpected search results are reported. The tools are as follows:

  • Rebuild index- this rebuilds the lucene search index, used in all operations that perform searches on the metadata
  • Optimize index- this is run nightly but can be run to re-optimise the index if odd search results are spotted
  • Rebuild index configuration- rebuild the index configuration if changes have been made to the catalogue, such as to the search views
  • Clear the XLink cache- if directories are added or changes have been made to the catalogue that would affect the structure of xlinks (such as to the URL or protocol), clear the cache
  • Clear formatter cache- if changes have been made to the catalogue display (schematron views, or text strings), clear the cache

image30

Figure 10.1.1: The index admin interface

Batch process

This subsection can be used to make xsl transformations on multiple records at once, such as to update contact information in bulk, change internal URLs or update keywords. It is wise to request a database backup before performing a batch process as there is no way to undo the transformation once it has been started.

The processes are defined per schema, and new processes can be added as required. The records that the process should be applied to can be filtered by Group, Owner, Category or custom search term, further filtered to include/exclude templates, and then selected using the dropdown selection dialog.

The available processes are accessed via a dropdown list, which then prompts you to supply the required parameters, such as the Old URL and the New URL. To run the process, click the blue “Run” button. The results of the process will be shown below. Note that the process may take a long time if there are a large number of records to search.

image31

Figure 10.2.1: The batch processing interface, showing the configuration of the URL replacer for metadata records owned by Transport Scotland

Transfer ownership

This subsection allows you to change the owner (user) of a set of records. Choose the user from the dropdown list, noting that this list contains only users with profiles of editor and above.

In the “Target group and editor” list choose the group that your new editor belongs to. The “Target editor” dropdown list will then show all users with editor profile and above in that group, and additionally full administrators for the catalogue. Select the appropriate editor and then click the blue “Transfer” button.

image32

Figure 10.3.1: The transfer ownership interface, configured for transferring records currently owned by the Fife Council user to the Admin user.

Note that changing the ownership of a record applies only to users and does not affect group privileges. This tool is also available in the Search panel as one of the available actions for selected results.

Editing online documentation

The source for the online documentation is written in ReStructuredText (rst) syntax and stored in a public repository on GitHub at https://github.com/AstunTechnology/ssdidocs. Every time this source documentation is updated, it is regenerated into html by ReadTheDocs and the generated version is available at http://scottish-sdi-metadata-portal.readthedocs.io/.

There are two sections to the documentation at present, represented by two separate rst files:

Editing the documentation will require a GitHub account.

ReStructured Text Syntax

ReStructured Text documents should be written in a text editor rather than a word processing application

A guide to syntax can be found at http://docutils.sourceforge.net/docs/user/rst/quickref.html

An online editor and previewer can be found at http://rst.ninjs.org/

Edit on github

The documentation can be edited online by either clicking the “Edit on GitHub” link, or navigating to the URLs given above. Note that the procedure outlined here seems complicated but in most cases it’s a button-pressing exercise.

Click the edit icon for the document you wish to edit.

image33

Figure 11.1.1.: Editing in GitHub

The editing interface has two tabs, one for editing and one for previewing changes. You can use shortcuts like ctrl-f to find text in the document that you wish to change. Check your changes using the preview tab, and then in the “Propose file change” section briefly outline your change.

image34

Figure 11.1.2: Detailing your changes

Click the green “Propose change” button and then in the following window, quickly review your changes and then click the green “Create pull request” button.

image35

Figure 11.1.3: Reviewing the pull request

image36

Figure 11.1.4: Creating the pull request

In the following window, if necessary expand on your explanation of the changes and then click the green “Create pull request” button.

The final window provides one further opportunity to make comments about your changes, but there is no requirement to do so. An email is sent to the owner of the repository, who will review the changes you have made and accept or reject them. If the change is accepted you will be notified by email but need take no further action- the new documentation will be automatically updated and pushed to ReadTheDocs. If the change is rejected for some reason then you will be notified of the reason why and the pull request will be closed without your changes being accepted.

Note that you will not be able to add new images using this method, you’ll need to edit locally (or provide Astun with the images).

Editing locally

Editing the files locally requires the installation of a Git client on your local computer. Details on installation and configuring git are beyond the scope of this document, but downloads and instructions for windows can be found at https://git-for-windows.github.io/.

You will need a GitHub account to edit files locally.

Advanced Troubleshooting

Changes to stylesheets and layout configuration occasionally require the GeoNetwork cache to be rebuilt. This may result in partially displayed or completely blank pages. If the “Clear Formatter Cache” tool in the Admin Console does not resolve the issue, or is not reachable, then running the following two services will resolve the problem. You will need to reload the problematic page once you have run these two services.

Note that the services don’t return a result, just a blank page. Note also that the pages will take a long time to load after these services have been run, as the cache is being completely rebuilt.

Using Zeppelin for Analytics

What is Zeppelin?

Apache Zeppelin is a user-configurable data dashboard that connects to the back-end PostgreSQL database where the SSDI Metadata records are stored. It allows catalogue adminstrators to query the metadata alongside the other data stored in the database, using standard Structured Query Language (SQL). A read-only user is used for the database connection to ensure that the data cannot be edited or deleted.

Logging in

It can be accessed by clicking the link Analytics (admin only) in the SSDI catalogue, and will open in a new tab. To log in (this must be done separately to the SSDI catalogue), click the Login buttin in the top right and use your catalogue username and password.

Once you have logged in, click the SSDI Analytics link on the left.

Getting Started

Zeppelin notebooks comprise a series of paragraphs with the following elements:

  • An optional title
  • Buttons to run or refresh the query, hide visual elements, and more configuration options
  • The SQL statement, otherwise known as the editor (this may be hidden)
  • An optional dynamic entry form such as a dropdown list or text box
  • A set of charting options, such as table, bar graph or piechart, along with data download options
  • The results, otherwise known as the output

SSDI Catalog Database Structure

There are three schemas in the SSDI database. To refer to a table, view (saved query) or function in any schema other than the public one, it is necessary to use the form schema.object.

audit

This holds the functions and tables required for the audit history plugin. The logged_actions table holds the actual audit history.

custom

Holds any custom functions or views (database queries) added to the core GeoNetwork database.

The key views are:

  • metadata_xml: this is a copy of the main metadata table in the public schema, with the difference that the data_xml column contains the actual stored metadata in xml format rather than as a blob. This allows xpath functionality to be used, so that the metadata itself can be queried.
  • all_records: this is a simplified conversion of the actual metadata into database columns. It is not all-encompassing as the xml structure doesn’t map exactly to a flat table format.

If all_records does not include the necessary information, it is possible to update it, but this must be done in the PostgreSQL database itself, not in zeppelin.

Other views have been created to simplify the SQL used in the zeppelin paragraphs.

public

This holds the core GeoNetwork tables. The key tables are:

  • metadata: holds the actual metadata records along with ownership, schema, changedate and harvesting information (amongst other things)
  • groups: lists the groups
  • users: lists the users

Relationships between tables are done with id fields. The naming convention is to name the column id in the primary table, and tableid in the secondary table. So for example, the usergroups table that contains information on which users are members of which groups has the following columns:

  • userid: the id from the user table
  • groupid: the id from the group table
  • profile: the user’s profile within the given group

Some tables also have translation tables, which have a des suffix. This contains the display name for the given object in the different languages that GeoNetwork supports. Again, the id field is used to link the primary table to the translation table, but the naming convention is not always consistent. As an example, the groupsdes table which shows the display name for groups has the following columns:

  • iddes: the id from the group table
  • langid: the language shortcode
  • label: the display name in the given language

See below on how to use zeppelin to understand the tables in order to build your own queries.

Constructing new paragraphs or queries

So it’s possible to construct new queries, the following paragraphs can be used to understand the structure of the database and tables:

  • Get tables in public, custom and audit schemas : This lists the tables and views in the audit, custom and public schemas. It is primarily used to get the correct table name for use in the query below.
  • Find the columns and data types for a given table: This lists the columns and data types for the table entered into the dynamic text box. Note that this should be entered without a schema prefix.

Basic multi-table queries

If you wish to construct a query containing columns from two or more tables or views then the general approach is to use table aliases and joins on the id fields. For example, to construct a query showing usernames, the display name for the groups they belong to, and their profiles in that group, then the following syntax would be used:

SELECT u.username, s.profile, g.label
FROM users u
JOIN usergroups s ON u.id=s.userid
JOIN groupsdes g ON s.groupid = g.iddes
WHERE g.langid = 'eng'
ORDER BY username, label, profile;

Queries involving xml

To query the stored metadata in custom.metadata_xml it is necessary to use XPath syntax within PostgreSQL. XPath is a query language for selecting nodes from an XML document. To use it you will need to interrogate the XML of a record, either by viewing it in the XML view when editing, or downloading it as XML and opening it in a text editor. Learning XPath is beyond the scope of this guide, but the above link gives a primer on the syntax.

PostgreSQL has a function called xpath for interrogating XML. It takes the following parameters:

  • xpath: the Xpath for the xml element you want
  • xml: the xml it should interrogate
  • nsarray: an array of namespaces referenced in the XPath query. These are the prefixes before the metadata elements, such as gmd:fileIdentifier or gco:CharacterString. Each namespace in the XPath query needs to have both the prefix and the namespace URI. These can be found in the root element of a metadata record if you view it as XML.

The xpath function returns an array of values. These must be concatenated into a string to use them in queries. The PostgreSQL array_to_string function is used for this. It takes the following parameters:

  • array: the array to convert
  • delimiter: the character used to delimit the array values when converted to a string
  • null string (optional): the optional character used in place of any NULL values found in the array

The following example extracts the uuid and title from the metadata records, using the data_xml column from custom.metadata_xml as the xml source. If there were multiple titles, they would be separated by commas:

SELECT uuid, array_to_string
        (
                xpath
                (
                        '/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title/gco:CharacterString/text()',  metadata_xml.data_xml,
                        '{{gco,http://www.isotc211.org/2005/gco},{gmd,http://www.isotc211.org/2005/gmd}}'
                ),
        ',')
AS title FROM custom.metadata_xml;

(note that the above query will result in some blank rows- these are generally template records)

Adding new queries to Zeppelin

To add a new query or paragraph, scroll to the very bottom of the notebook and start typing your SQL query at the prompt. When you are ready to run it, click the run this paragraph button. If there are any errors in your query they will be shown in the output.

To give your query a title, click the settings button and select show title, then type the title in the text area that appears (generally pre-filled with “Untitled”).

Privacy on the SSDI

This notice explains how we use any personal information we collect about you when you use this website. The site, spatialdata.gov.scot, is provided by the Scottish Government (the data controller), however, we use a third party, Astun Technology, for the hosting and management of the metadata catalogue (the data processor).

This privacy notice is kept under regular review and any updates will appear on this page. This notice was last updated on July 8th, 2020.

What is collected and how is it used?

When you browse the site, your IP address is collected and cookies are used to monitor your session. Additionally, if you register as a contributer to the site you will be required to provide your email address. This information is used purely for the purposes of maintaining the portal, improving it based on how you use it, and notifying contributors of any changes or service interruptions. The information can be viewed by authorised people in the Scottish Government.

As noted above, we use a third party, Astun Technology, for the hosting and management of the metadata catalogue. To view further details about what is collected on the site and how it is used please see their metadata privacy policy. Astun Technology does not retain, share, store or use this information for any secondary purposes.

We will not share your information with other organisations for commercial purposes and we won’t pass on your details to other parties, unless required to do so by law.

Access to your information

You are entitled to view, amend, or delete the personal information that we hold about you. For more information on accessing the information we hold about you, please contact us.

Be aware that most modern web browsers allow users to control cookies through the browser settings. To find out more about cookies, including how to see what cookies have been set and how to manage and delete them, visit www.aboutcookies.org or cookies.insites.com.

Contact us

Please contact us via email at SSDI.metadata@gov.scot if you have any questions with regards to your privacy when using the SSDI metadata portal or information we hold about you.

Documentation Licence

© Crown copyright 2019. You may use or re-use this documentation (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence 3.0. See www.nationalarchives.gov.uk/doc/open-government-licence.

Attribution

When you use this information under the Open Government Licence, you should include the following attribution:

Spatialdata.Gov.Scot Metadata Portal Documentation, Scottish Government and Astun Technology, 2019, licensed under the Open Government Licence www.nationalarchives.gov.uk/doc/open-government-licence.
http://www.nationalarchives.gov.uk/images/infoman/ogl-symbol-41px-retina-black.png

Accessibility

This website is hosted and maintained by Astun Technology on behalf of the Scottish Government. We want SpatialData.gov.scot to be accessible and usable for as many people as possible.

How accessible this website is

We know some parts of this website are not fully accessible:

  • Not all non-text content is available in text form, for example not all images have “alt” attributes
  • Not all content can be rendered using assistive technology
  • Some elements such as links are distinguishable only by colour
  • Not all pages have titles or descriptions that adequately describe their purpose
  • Not all links have a purpose that can be determined by the link text alone
  • Entering data into controls does not always have a predictable effect, nor are consistent instructions provided on their use

Feedback

If you cannot access any part of this site or want to report an accessibility problem, please tell us.

AbilityNet has advice on making your device easier to use if you have a disability.

Enforcement procedure

The Equality and Human Rights Commission enforces the accessibility regulations (the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018).

If you’re not happy with how we respond to your feedback, contact the Equality Advisory and Support Service. They are an independent advice service. They will advise you on what to do next.

Compliance statement

The Scottish Government commits to making its websites accessible, in accordance with the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018. This accessibility statement applies to SpatialData.gov.scot and was last reviewed on 5 August 2020.

The site was reviewed using the SiteImprove Chrome plugin in August 2020. This website is not yet compliant with the Web Content Accessibility Guidelines version 2.1 (WCAG) A success criteria due to the non-compliances listed above.