Setting Up an OGC-Compliant Metadata Catalog with PyCSW

Introduction

A Spatial Data Infrastructure (SDI) is more than a collection of spatial databases and web maps. It also needs a reliable way for people and applications to discover what datasets exist, understand their characteristics, evaluate their quality, and locate the resources required to access them.

Without a metadata catalog, an organization may maintain hundreds of valuable datasets across PostgreSQL databases, file servers, remote sensing repositories, and web mapping services, yet users have no consistent mechanism for finding them.

The Open Geospatial Consortium (OGC) addresses this problem through standards for discovering and querying metadata. The Catalogue Service for the Web (CSW) defines interfaces through which applications can search catalog records and retrieve descriptions of geospatial resources. These interfaces support interoperable discovery across GIS software, institutional catalogs, and regional or international data infrastructures. See the official OGC Catalogue Service standard.

PyCSW is an open-source Python implementation of the CSW standard. It publishes metadata records through HTTP endpoints and supports metadata formats and profiles used in geospatial information management, including Dublin Core, FGDC, and ISO metadata profiles.

In this guide, we will deploy a PyCSW metadata server, configure a repository, ingest XML metadata records, test the service, and connect the catalog to QGIS. The result is a reusable foundation for publishing searchable geospatial metadata without building a catalog engine from scratch.

Version note: PyCSW’s released 2.6.x series uses default.cfg and commands such as pycsw-admin.py -c setup_db -f default.cfg. The newer 3.0 development documentation uses YAML configuration and updated command names. This guide follows the established default.cfg workflow and identifies the newer syntax where it differs. Check the PyCSW releases before deploying, and do not mix commands and configuration files from different versions.

1. System Prerequisites and Architecture for setting up an OGC-Compliant metadata catalog with PyCSW

A basic PyCSW deployment requires four components:

1. PyCSW application: Processes catalog requests and returns metadata responses.
2. Metadata repository: Stores indexed records and their searchable properties.
3. Web server or WSGI server: Exposes the application over HTTP.
4. GIS clients: Discover and query catalog records, for example through QGIS MetaSearch.

Recommended architecture

For development, SQLite is a convenient repository because it does not require a separate database server. For a larger institutional SDI, PostgreSQL with PostGIS can support a more scalable deployment and spatial database integration.

<escape>[Image 1: Architecture diagram of PyCSW catalog server connecting databases to web/desktop clients]</escape>

Alt text: System architecture diagram illustrating PyCSW metadata server parsing XML records from PostGIS and serving OGC CSW endpoints to QGIS and web map portals.

Software requirements

Component	Purpose
Python 3.x	Runs PyCSW and its supporting libraries
PyCSW	Implements the metadata catalog service
SQLite or PostgreSQL/PostGIS	Stores searchable metadata records
WSGI server or Apache	Serves the application over HTTP
QGIS with MetaSearch	Searches catalog records from a desktop GIS

The PyCSW installation documentation provides additional information about the supported Python environment and deployment models.

For this tutorial, the main workflow uses a Linux environment, a Python virtual environment, the PyCSW source distribution, and SQLite. The same metadata and request-testing principles also apply to a PostGIS-backed service.

2. Step One: Install PyCSW

Create a directory for the application and prepare an isolated Python environment.

sudo mkdir -p /opt/pycsw
sudo chown "$USER":"$USER" /opt/pycsw
cd /opt/pycsw

python3 -m venv .venv
source .venv/bin/activate

Download the PyCSW source for a specific release. This provides the sample configuration and WSGI entry point alongside the application code.

git clone --branch 2.6.3 \
  https://github.com/geopython/pycsw.git source

cd source

pip install -e .
pip install -r requirements-standalone.txt

If you deploy a different release, confirm that its source, dependencies, and administration commands match the configuration used here. Pinning a tested release helps prevent unexpected changes during deployment.

Create a writable directory for the repository and copy the sample configuration.

mkdir -p /opt/pycsw/data

cp default-sample.cfg default.cfg

This gives you a working configuration template rather than requiring you to reconstruct every supported setting manually.

For a straightforward PyPI-based installation, you can also install a released package with pip install pycsw. In that case, obtain the matching configuration template from the corresponding release and ensure that your WSGI deployment can locate it.

3. Step Two: Configure the PyCSW Metadata Server

The default.cfg file defines the service URL, logging settings, metadata profiles, organization details, repository connection, and transaction permissions.

Open it in a text editor:

nano default.cfg

Update the main sections with values appropriate for your organization.

Configure the server

[server]
home=/opt/pycsw/source
url=http://localhost:8000/
mimetype=application/xml; charset=UTF-8
encoding=UTF-8
language=en-GB
maxrecords=100
profiles=apiso
loglevel=INFO

The settings have different roles:

• home points to the PyCSW source directory in this deployment.
• url defines the advertised service URL. It must match the address through which clients actually access the service.
• language identifies the service language and country.
• maxrecords limits the number of records returned in a response.
• profiles=apiso enables the ISO metadata application profile for compatible CSW requests.
• loglevel controls the amount of diagnostic information recorded.

For production, replace the localhost URL with the public HTTPS endpoint, such as https://catalog.example.org/csw, and configure the web server routing to match.

Configure metadata provider information

The catalog itself should have meaningful service-level metadata so that clients can identify the organization responsible for the service.

[metadata:main]
identification_title=Institutional Geospatial Metadata Catalog
identification_abstract=Searchable catalog of institutional geospatial datasets and services
identification_keywords=GIS, geospatial data, metadata, SDI
identification_keywords_type=theme

provider_name=Example Mapping Agency
provider_url=https://catalog.example.org

contact_name=Geospatial Data Administrator
contact_position=SDI Administrator
contact_email=gis@example.org
contact_country=Kenya
contact_role=pointOfContact

Replace these example values with real institutional information. Use a contact mailbox that will remain valid when personnel change.

Configure the repository

For the local SQLite installation, set:

[repository]
database=sqlite:////opt/pycsw/data/records.db
table=records

The four slashes in sqlite:////opt/pycsw/data/records.db are intentional: this is an absolute Unix filesystem path encoded as a SQLAlchemy connection URL.

PostgreSQL is another option:

[repository]
database=postgresql://catalog_user:YOUR_PASSWORD@localhost:5432/pycsw
table=records

Create the database and account separately, grant only the necessary privileges, and avoid storing production credentials in a publicly accessible configuration file. Use an environment-specific configuration or protected secret management instead.

For PostGIS deployments, enable the PostGIS extension before initializing the PyCSW repository if native PostGIS geometry support is required. Consult the PyCSW administration documentation for database-specific details.

Protect catalog transactions

The sample configuration normally disables metadata transactions.

[manager]
transactions=false
allowed_ips=127.0.0.1

Keep transactions disabled unless clients genuinely need to insert, update, or delete records remotely. Enabling them introduces additional security and operational considerations.

For production, the default configuration should be treated as a starting point rather than a complete security policy. Restrict database access, protect application files, and ensure that the web server cannot serve default.cfg as a downloadable document.

4. Step Three: Initialize the Metadata Repository

A catalog repository stores more than the original XML. PyCSW extracts searchable properties, maintains record identifiers, and makes metadata available through its supported query interfaces.

From the PyCSW source directory, activate the intended Python environment and initialize the database.

cd /opt/pycsw/source
source /opt/pycsw/.venv/bin/activate

pycsw-admin.py -c setup_db -f default.cfg

The command creates the required repository structures and indexes for the configured database.

If initialization fails, check the following:

• The database directory exists and is writable.
• The active Python environment contains the correct PyCSW installation.
• The configuration file references the intended database.
• PostgreSQL credentials and permissions are correct, if applicable.
• You are not accidentally pointing at an already initialized repository.

A repository setup command is not an everyday refresh operation. Back up any existing records before rebuilding or replacing repository structures.

The newer PyCSW 3.0 development documentation uses the updated equivalent:

pycsw-admin.py setup-repository --config default.yml

This is a different command and configuration format; use it only when following the matching newer release workflow.

5. Step Four: Prepare and Ingest XML Metadata Records

Once the repository exists, populate it with XML metadata files.

PyCSW supports metadata records based on established formats, including:

• ISO 19115 / ISO 19139: Detailed descriptions of geospatial datasets, including identification, extent, quality, lineage, and responsible parties.
• FGDC CSDGM: A metadata standard historically used by many United States geospatial agencies.
• Dublin Core: A general-purpose descriptive metadata vocabulary that can support resource discovery.

Compatibility depends on the metadata schema, the enabled profile, and the type of request made by the client. For ISO-oriented discovery through CSW 2.0.2, the apiso profile is especially relevant.

Organize the metadata directory

For example:

metadata/
├── administrative_boundaries.xml
├── roads_network.xml
├── landcover_2025.xml
└── elevation_model.xml

Each file should be well-formed XML and contain a valid, unique record identifier. Metadata should describe the resource accurately, include appropriate spatial or temporal information, and provide valid links to data or related services when available.

A catalog record does not automatically publish the underlying raster, vector dataset, or web map service. It makes the resource discoverable and can contain links through which clients access the resource.

Load the records

Run the following command:

pycsw-admin.py -c load_records \
  -f default.cfg \
  -p /opt/pycsw/source/metadata \
  -r

In this example, change the final directory to the actual location of your metadata XML files.

The -r option processes subdirectories recursively. To load a single file, pass that file’s path instead. The -y option can be used to overwrite existing records with matching identifiers, but use it only when an update is intended.

After ingestion, review the console output and logs for malformed XML documents or records that could not be parsed.

Validate metadata before publication

A successful import is not a substitute for metadata quality assurance.

Check that each record has a meaningful title, an abstract or description, a responsible organization, a unique identifier, appropriate keywords, and the correct geographic extent. Validate ISO metadata against the intended schema or profile, particularly when records will be harvested by external catalogs.

For an institutional SDI, you can implement automated metadata generation and ISO standards compliance with Python upstream so that newly published spatial datasets arrive with consistent descriptions and identifiers.

6. Step Five: Start the Catalog Service

For an initial test, PyCSW’s included WSGI entry point can run a small server on port 8000.

cd /opt/pycsw/source
source /opt/pycsw/.venv/bin/activate

python pycsw/wsgi.py

If successful, the application reports that it is serving requests on port 8000.

This development server is suitable for initial testing, not direct production exposure. In production, use a properly configured WSGI server such as Gunicorn or Apache mod_wsgi, usually behind a reverse proxy providing HTTPS, request logging, and other operational controls.

For example, with Gunicorn installed in the same environment:

pip install gunicorn

export PYCSW_CONFIG=/opt/pycsw/source/default.cfg

gunicorn pycsw.wsgi:application \
  --bind 127.0.0.1:8000 \
  --workers 2

This makes the WSGI application available locally for a reverse proxy to expose at the intended public path. Verify the public endpoint and the server.url setting before allowing external clients to use it.

7. Step Six: Test the OGC CSW Endpoints

The next step is to confirm that PyCSW responds correctly to standard catalog requests. These tests check the HTTP service and whether indexed records can be discovered.

Test GetCapabilities

GetCapabilities returns information about the service, its operations, supported formats, and capabilities.

curl -G "http://localhost:8000/" \
  --data-urlencode "service=CSW" \
  --data-urlencode "version=2.0.2" \
  --data-urlencode "request=GetCapabilities"

Inspect the XML response. Confirm that the operation names appear correctly and that the service advertises the expected endpoint and metadata profile.

A successful HTTP response alone does not establish complete standards conformance. The XML structure, supported operations, profile declarations, and behavior under representative requests must also be checked.

Test GetRecords

Use GetRecords to search the catalog.

curl -G "http://localhost:8000/" \
  --data-urlencode "service=CSW" \
  --data-urlencode "version=2.0.2" \
  --data-urlencode "request=GetRecords" \
  --data-urlencode "resultType=results" \
  --data-urlencode "typeNames=csw:Record" \
  --data-urlencode "elementSetName=summary" \
  --data-urlencode "maxRecords=10"

The response should contain matching records and basic descriptive fields. If no records are returned, confirm that ingestion succeeded, the request targets the right repository, and the requested record type and profile are supported.

Test GetRecordById

To retrieve one complete record, use its actual identifier from the metadata source.

curl -G "http://localhost:8000/" \
  --data-urlencode "service=CSW" \
  --data-urlencode "version=2.0.2" \
  --data-urlencode "request=GetRecordById" \
  --data-urlencode "id=YOUR_METADATA_IDENTIFIER" \
  --data-urlencode "elementSetName=full"

Replace YOUR_METADATA_IDENTIFIER with a real identifier rather than the title or filename unless that is explicitly the record’s identifier.

The full response lets you inspect the metadata elements returned to clients and confirm that the record contains useful information.

For formal testing, use the OGC compliance and conformance resources, including the applicable CSW test suite. Standards compliance should be verified against the specific CSW version and profile required by your organization.

8. Step Seven: Connect the Catalog to QGIS MetaSearch

QGIS includes the MetaSearch catalog client, which can interact with OGC CSW services and supported OGC API – Records services.

<escape>[Image 2: QGIS MetaSearch plugin connecting to live PyCSW metadata catalog]</escape>

Alt text: Screen capture of QGIS MetaSearch plugin querying and discovering spatial datasets live from an OGC-compliant PyCSW catalog endpoint.

Configure the catalog connection

1. Open QGIS and enable MetaSearch through the Plugin Manager if it is not already available.
2. Navigate to the MetaSearch interface, usually under the Web menu.
3. Open the Services tab and select New.
4. Enter a descriptive service name, such as Institutional PyCSW Catalog.
5. Enter the base service URL, for example https://catalog.example.org/csw, and select OGC CSW 2.0.2.
6. Save the connection, select it, and use the Search tab to query records.

For a CSW 2.0.2 connection, QGIS generally requires the base catalog URL rather than a complete GetCapabilities request URL. See the official QGIS MetaSearch documentation.

Search using a dataset title, theme keyword, or another supported query property. Inspect the results and confirm that returned records provide appropriate descriptions, extents, and resource links.

If the connection fails, test the endpoint with GetCapabilities first. Then check the public URL, reverse-proxy path, HTTPS certificate, firewall configuration, and any access restrictions. A server that responds locally may still be inaccessible to remote desktop clients.

9. Integrate with GeoNode and Other SDI Portals

PyCSW can also participate in broader metadata discovery infrastructures.

GeoNode uses PyCSW as a catalog component, enabling discovery of geospatial metadata associated with the platform. In the standard GeoNode configuration, this integration is managed by GeoNode itself; a separate standalone PyCSW server is not necessarily required.

See the PyCSW GeoNode integration documentation.

For an existing SDI, distinguish between two architectures:

• Standalone PyCSW: Metadata is imported into a dedicated PyCSW repository.
• Integrated catalog: PyCSW exposes metadata managed by another application, such as GeoNode, using the supported repository mapping or integration mechanism.

Do not create an independent duplicate catalog unless there is a clear operational need. Duplicate repositories introduce synchronization issues when metadata changes in the source system.

For federation, configure approved remote catalog endpoints and consider how identifiers, access permissions, response times, and duplicated records will be managed. Distributed discovery can extend a local catalog’s reach, but it does not guarantee that remote datasets are current or accessible.

10. Security, Maintenance, and Operational Best Practices

A metadata catalog can be publicly searchable without making its management functions publicly writable.

Protect configuration files. Keep database credentials and service configuration outside publicly served directories. Explicitly deny HTTP access to .cfg files and other sensitive deployment files.

Restrict metadata transactions. Leave transactions disabled unless a controlled publishing workflow requires them. If writes are enabled, use authentication and network restrictions appropriate to the deployment.

Use HTTPS in production. Terminate TLS at the reverse proxy or web server, and ensure that the advertised server.url uses the correct public HTTPS endpoint.

Back up the repository and metadata sources. SQLite deployments need consistent database backups; PostgreSQL deployments should follow a tested database backup and restoration plan. Keep source XML files so that the catalog can be rebuilt when necessary.

Monitor logs and response behavior. Watch for failed metadata imports, database connectivity problems, malformed requests, long-running searches, and unexpected response sizes.

Maintain stable identifiers. Metadata identifiers should persist across revisions of the same resource where appropriate. Duplicate or inconsistent identifiers complicate record updates and cross-catalog harvesting.

Control record growth. Review the need for database optimization and index maintenance as the catalog expands. PyCSW’s administration utilities include commands for optimizing certain supported database backends and rebuilding database indexes.

Revalidate after upgrades. Test GetCapabilities, GetRecords, GetRecordById, profile-specific queries, and client interoperability whenever the application, database, or reverse-proxy configuration changes.

11. SEO Interlinking and Further Reading

Suggested internal links:

• Automated Geospatial Metadata Generation with Python — for generating standardized metadata records as part of publishing pipelines.
• Docker-Based GIS Infrastructure Deployment — for containerized deployment, service orchestration, and reproducible environments.
• Spatial Database Security with PostgreSQL and PostGIS — for account privileges, connection management, and database hardening.

Replace these suggested paths with the actual URLs on your website.

Recommended external resources:

• PyCSW Documentation — installation, configuration, administration, and profile support.
• PyCSW Releases — current release information and upgrade reference.
• OGC Catalogue Service Standard — formal catalog interoperability standards.
• QGIS MetaSearch Documentation — client setup and catalog discovery.
• OGC Conformance Testing — test resources for validating supported standards.

Conclusion

PyCSW provides a practical foundation for an OGC CSW metadata catalog within a Spatial Data Infrastructure. By configuring the service endpoint, initializing SQLite or PostGIS storage, ingesting valid XML metadata, and testing standard CSW requests, GIS administrators can make institutional datasets discoverable through compatible applications and portals.

A successful deployment involves more than getting the server to respond. Records must be meaningful and consistently identified, metadata profiles must match client expectations, configuration files must be protected, and the public endpoint must be correctly advertised.

The key maintenance principle is to treat the catalog as an operational SDI service, not a one-time installation. Automate metadata creation, validate incoming records, monitor repository health, test interoperability after updates, and keep the service aligned with the OGC standards required by your users.