Automating Spatial ETL Pipelines with Apache Airflow and GeoPandas

1. Introduction: Why automate spatial ETL?

When you are automating spatial ETL pipelines Apache Airflow GeoPandas, building a repeatable workflow that retrieves geospatial data becomes essential. To automate spatial ETL with Apache Airflow and GeoPandas is to build a repeatable workflow that retrieves geospatial data, validates and transforms it, and publishes the processed results to a spatial database on a defined schedule.

Many GIS workflows still depend on someone manually downloading updated datasets, repairing geometries, reprojecting layers, calculating spatial attributes, and exporting the results. That approach can work for a one-off analysis, but it becomes difficult to maintain when the same operation must run every hour, every day, or whenever an upstream dataset changes.

Consider a few practical scenarios:

• A web map that displays the latest road closures and construction sites.

• A flood-monitoring dashboard that combines newly published satellite-derived flood extents with administrative boundaries.

• An enterprise database that refreshes land parcels, building footprints, or utility assets overnight.

• A transportation dashboard that calculates updated road lengths and service coverage from newly delivered vector data.

In each case, manual processing introduces delays and opportunities for error. A scheduled pipeline can standardize the workflow, log each run, retry temporary failures, and make refreshed data available to downstream applications.

Three technologies provide a flexible foundation for this approach.

Airflow is an orchestration platform, not a GIS processing engine. GeoPandas performs the spatial transformation, while PostGIS provides the persistent publication layer. Airflow’s TaskFlow API connects these operations into a dependency-aware workflow.

This guide builds a daily pipeline that downloads a polygon dataset, validates and transforms it, calculates polygon area, and publishes the resulting features to PostGIS.

2. Architecture overview: How the pipeline works

A reliable geospatial data pipeline has three primary stages: extraction, transformation, and loading.

Figure 1. A scheduled, three-stage spatial ETL pipeline. Airflow orchestrates the tasks; the processing code executes the geospatial operations.

2.1 Understanding DAGs and tasks

Airflow represents workflows as DAGs (Directed Acyclic Graphs). Each node represents a task, while each directed edge describes a dependency.

For the pipeline in this guide, the dependency graph is:

download_vector
       |
       v
transform_vector
       |
       v
load_to_postgis

The tasks execute in sequence because the transformation needs the downloaded file, and the loading stage needs the transformed output.

The main Airflow concepts are:

• DAG: Defines the workflow, schedule, dependencies, and run-level settings.

• Task: A unit of work that Airflow can execute and monitor independently.

• Operator or TaskFlow function: Defines what a task does. The @task decorator is convenient for Python-based spatial processing.

• XCom: Passes small pieces of information between tasks, such as a file path, object-storage key, or record count.

• Retry policy: Specifies how Airflow should respond when a task fails.

For new Airflow 3.x DAGs, this tutorial uses the public airflow.sdk TaskFlow interface. Airflow’s current documentation describes TaskFlow as a way to define Python tasks and automatically establish dependencies when task outputs are passed to other tasks.

2.2 What runs where?

A key architectural decision is separating orchestration from computation.

The scheduler decides when work should run. A worker executes the task. The GeoPandas process loads features into memory and performs the transformations. The database server stores the resulting features and handles publication.

This distinction matters when using Docker or distributed workers. A local path such as /opt/airflow/data/parcels.gpkg is only reliable between tasks if both tasks can access the same shared storage. With distributed workers, use shared persistent storage or object storage and pass its URI between tasks, rather than passing the dataset itself through XCom.

3. Step-by-step implementation

Step 1: Prepare the environment

The example assumes a deployment with:

• Apache Airflow 3.x.

• Python with GeoPandas and Shapely installed on the workers that execute spatial tasks.

• PostgreSQL with PostGIS enabled.

• A downloadable polygon dataset in GeoPackage (.gpkg) format.

• A PostGIS target table created in advance.

• Shared storage mounted at /opt/airflow/data for this single-environment example.

GeoPandas supports vector file input and PostGIS integration. Its to_postgis() method requires SQLAlchemy, GeoAlchemy2, and a PostgreSQL driver.

Install the geospatial and database dependencies in the Airflow worker image or its Python environment:

pip install geopandas shapely sqlalchemy geoalchemy2 \
    "psycopg[binary]" requests

Use a dependency lockfile or pinned, tested versions for production deployments. Native geospatial libraries, including GDAL and PROJ dependencies used by the geospatial stack, should be tested in the same container image that runs the DAG.

Configure these environment variables through the Airflow deployment or its secrets-management mechanism:

SPATIAL_SOURCE_URL=https://example.org/data/parcels.gpkg
SPATIAL_DATA_DIR=/opt/airflow/data

POSTGIS_SQLALCHEMY_URL=postgresql+psycopg://etl_user:REPLACE_ME@postgis:5432/gisdb

The source URL is illustrative and must be replaced by a real GeoPackage download endpoint. Do not commit production database passwords to source code or a public repository.

Step 2: Write the GeoPandas transformation script

The transformation script is the spatial processing component of the pipeline. Keeping it separate from the DAG makes it easier to test locally, maintain, and reuse.

The example assumes that the input contains polygon features and a stable, unique feature_id field. It optionally uses a name field. Change these fields and the geometry rules to match the actual source dataset.

The script performs five operations:

1. Reads the downloaded vector dataset.

2. Checks its coordinate reference system and required identifiers.

3. Removes null or empty geometries and repairs invalid ones.

4. Calculates polygon area using a projected equal-area coordinate system.

5. Writes a cleaned GeoPackage in EPSG:4326 for downstream web mapping.

GeoPandas provides GeoSeries.make_valid() for repairing invalid geometries. Because a repair can change the geometry type, the script also enforces the expected polygon geometry contract rather than silently publishing incompatible shapes.

Create spatial_transform.py:

from pathlib import Pathimport loggingimport geopandas as gpdfrom shapely.geometry import MultiPolygonlogger = logging.getLogger(__name__)# Global equal-area CRS suitable for this area-calculation example.AREA_CRS = "EPSG:6933"# Web-friendly output CRS.OUTPUT_CRS = "EPSG:4326"def transform_vector(    input_path: str,    output_path: str,) -> dict:    """    Validate polygon features, calculate area, and export a GeoPackage.    Expected input:      - A vector file readable by GeoPandas.      - A defined CRS.      - A non-null, unique feature_id.      - Polygon or MultiPolygon features after geometry repair.    """    input_path = Path(input_path)    output_path = Path(output_path)    if not input_path.is_file():        raise FileNotFoundError(            f"Input vector file does not exist: {input_path}"        )

Save the script in a directory that is available to the Airflow worker, such as /opt/airflow/dags/spatial_transform.py, or install it as a Python package in the worker image. The task below imports it at execution time.

Important CRS considerations

The area calculation deliberately uses two coordinate systems for different purposes:

• EPSG:6933: An equal-area projected CRS suitable for the illustrative area calculation. The resulting area_m2 values represent square metres.

• EPSG:4326: The output CRS for storing longitude/latitude geometries commonly used by web mapping clients.

Do not calculate area directly from longitude and latitude coordinates: the result would be in angular units, not square metres. For local engineering or cadastral work, select a suitable local projected CRS and validate its accuracy for the region and measurement requirements. Also avoid using one projection indiscriminately for datasets spanning large or multiple projection zones.

to_crs() transforms coordinates; it does not discover the correct source CRS. That is why the script fails when the source CRS is unknown instead of guessing.

Step 3: Configure the Apache Airflow DAG

The DAG connects the download, transformation, and database publication steps. Each task does one job, and Airflow controls the execution order.

Create spatial_etl_dag.py:

from datetime import timedeltafrom pathlib import Pathimport loggingimport osimport pendulumimport requestsfrom airflow.sdk import dag, taskDATA_DIR = Path(    os.environ.get("SPATIAL_DATA_DIR", "/opt/airflow/data"))DB_SCHEMA = "analytics"TARGET_TABLE = "spatial_features"STAGING_TABLE = "spatial_features_stage"logger = logging.getLogger(__name__)@dag(    dag_id="automated_spatial_etl",    description="Download, transform, and publish polygon data to PostGIS",    schedule="0 2 * * *",  # Daily at 02:00 UTC    start_date=pendulum.datetime(2026, 1, 1, tz="UTC"),    catchup=False,    max_active_runs=1,    default_args={        "owner": "spatial-data-platform",        "retries": 2,        "retry_delay": timedelta(minutes=5),        "execution_timeout": timedelta(minutes=45),    },    tags=["gis", "geopandas", "postgis", "spatial-etl"],)def automated_spatial_etl():    @task    def download_vector() -> str:        """Download a GeoPackage without publishing partial files."""        source_url = os.environ.get("SPATIAL_SOURCE_URL")        if not source_url:            raise ValueError("SPATIAL_SOURCE_URL is not configured.")        raw_dir = DATA_DIR / "raw"        raw_dir.mkdir(parents=True, exist_ok=True)        destination = raw_dir / "source.gpkg"        temporary = raw_dir / "source.gpkg.part"        try:            response = requests.get(                source_url,                stream=True,                timeout=(10, 120),            )            response.raise_for_status()            bytes_written = 0            with temporary.open("wb") as file:                for chunk in response.iter_content(                    chunk_size=1024 * 1024                ):                    if chunk:                        file.write(chunk)                        bytes_written += len(chunk)            if bytes_written == 0:                raise ValueError("Downloaded source file is empty.")            # The final filename becomes visible only after the            # download completes successfully.            os.replace(temporary, destination)            logger.info(                "Downloaded %d bytes to %s",                bytes_written,                destination,            )            return str(destination)        except Exception:            temporary.unlink(missing_ok=True)            logger.exception("Source download failed.")            raise    @task    def transform_vector(input_path: str) -> str:        """Execute the separate GeoPandas processing module."""        from spatial_transform import transform_vector as process_vector        output_path = DATA_DIR / "processed" / "spatial_features.gpkg"        result = process_vector(            input_path=input_path,            output_path=str(output_path),        )        logger.info("Transformation metrics: %s", result)        return str(output_path)    @task    def load_to_postgis(output_path: str) -> None:        """Load staging data and publish it in one database transaction."""        import geopandas as gpd        from geoalchemy2 import Geometry        from sqlalchemy import create_engine, text        database_url = os.environ.get(            "POSTGIS_SQLALCHEMY_URL"        )        if not database_url:            raise ValueError(                "POSTGIS_SQLALCHEMY_URL is not configured."            )        if not Path(output_path).is_file():            raise FileNotFoundError(                f"Transformed file not found: {output_path}"            )        engine = create_engine(            database_url,            pool_pre_ping=True,            connect_args={"connect_timeout": 15},        )        try:            gdf = gpd.read_file(                output_path,                layer="spatial_features",            )            if gdf.empty:                raise ValueError(                    "Refusing to publish an empty dataset."                )            expected_columns = {                "feature_id",                "name",                "area_m2",                "geometry",            }            if not expected_columns.issubset(gdf.columns):                raise ValueError(                    "Transformed data does not match the "                    "expected output schema."                )            # Write to a dedicated staging table first.            gdf.to_postgis(                name=STAGING_TABLE,                schema=DB_SCHEMA,                con=engine,                if_exists="replace",                index=False,                chunksize=5000,                dtype={                    "geometry": Geometry(                        geometry_type="MULTIPOLYGON",                        srid=4326,                    )                },            )            # The target table must be provisioned in advance with            # matching feature_id, name, area_m2 and geometry columns.            # TRUNCATE and INSERT run in the same transaction.            with engine.begin() as connection:                connection.execute(                    text(                        "TRUNCATE TABLE "                        "analytics.spatial_features"                    )                )                connection.execute(                    text("""                        INSERT INTO analytics.spatial_features                            (feature_id, name, area_m2, geometry)                        SELECT                            feature_id, name, area_m2, geometry                        FROM analytics.spatial_features_stage                    """)                )            logger.info(                "Published %d features to %s.%s",                len(gdf),                DB_SCHEMA,                TARGET_TABLE,            )        except Exception:            logger.exception("PostGIS publication failed.")            raise        finally:            engine.dispose()    # TaskFlow creates dependencies through returned task outputs.    raw_file = download_vector()    processed_file = transform_vector(raw_file)    load_to_postgis(processed_file)automated_spatial_etl()

Place both Python files in the DAGs directory or ensure spatial_transform.py is installed on the worker’s Python path.

How Airflow executes this DAG

The pipeline runs daily at 02:00 UTC. The catchup=False setting prevents the scheduler from automatically creating a series of historical runs for every missed scheduled interval before the DAG was enabled. The configured max_active_runs=1 prevents overlapping runs of this DAG.

The TaskFlow dependencies are established by passing raw_file to transform_vector() and processed_file to load_to_postgis(). Each returned value is a small string, not the actual spatial dataset. This is an important design choice because XCom is intended for metadata and small task results rather than large GeoDataFrames.

Step 4: Prepare the PostGIS target table

Before the first run, create the database schema and target table. The following SQL assumes that PostGIS is installed and enabled in the target database.

CREATE EXTENSION IF NOT EXISTS postgis;

CREATE SCHEMA IF NOT EXISTS analytics;

CREATE TABLE IF NOT EXISTS analytics.spatial_features (
    feature_id TEXT PRIMARY KEY,
    name TEXT NOT NULL DEFAULT '',
    area_m2 DOUBLE PRECISION NOT NULL,
    geometry geometry(MULTIPOLYGON, 4326) NOT NULL
);

CREATE INDEX IF NOT EXISTS
    spatial_features_geometry_gix
ON analytics.spatial_features
USING GIST (geometry);

The table uses a stable feature identifier, a text label, a numeric area field, and a MultiPolygon geometry in EPSG:4326.

The GiST index supports many spatial query patterns, including intersection and containment searches. This helps downstream applications query geographic features without repeatedly scanning the entire table. The index is useful only when the relevant SQL query and spatial predicate can take advantage of it.

Create a dedicated database user with only the privileges required by the pipeline. The user needs appropriate access to the target schema and target table, as well as permissions to create, replace, and write to the staging table. Production privilege design should be tested against the actual PostgreSQL ownership and grant model.

Step 5: Understand the staging and publication strategy

A common ETL failure occurs when a pipeline overwrites a production table before validating whether the replacement data loaded successfully.

The example avoids that specific failure mode by loading first into a staging table and then publishing within a single database transaction.

Processed GeoPackage
        |
        v
analytics.spatial_features_stage
        |
        v
BEGIN TRANSACTION
        |
        +--> TRUNCATE analytics.spatial_features
        |
        +--> INSERT complete staging dataset
        |
        v
      COMMIT

PostgreSQL’s transactional behavior means that if the INSERT fails, the transaction can roll back the preceding TRUNCATE, leaving the previous published rows in place.

There is still an important limitation: the staging load itself is not atomic with publication, and a single shared staging table is unsuitable for concurrent pipelines that write to that same table. The example avoids concurrent runs within this DAG, but an enterprise platform with parallel ingestion should use run-specific staging tables or isolated staging schemas.

Also, this is a full-refresh design: features missing from the latest source are removed from the target upon successful publication. It is appropriate when the upstream download represents a complete authoritative snapshot. It is not suitable without modification for incremental feeds, historical tracking, or change-only updates.

For production datasets with strict availability requirements, consider a versioned publication strategy, database-side validation, and an atomic view or table swap where appropriate. Preserve the previous published version until the replacement has passed validation.

4. Handling failures, retries, and logging

Automated spatial ETL can fail for reasons that are unrelated to the spatial transformation itself. The download server may be unavailable, a file may be corrupted, an input schema may change, a geometry may fail validation, or the database may become temporarily unreachable.

The goal is not to retry everything blindly. It is to distinguish between failures that may resolve on a subsequent attempt and failures that require intervention.

Airflow supports task-level retry settings, timeouts, and execution logs. Its documentation also recommends making tasks repeatable because retries may execute a task more than once.

4.1 Retry only when the failure can reasonably be transient

The example uses two retries with a five-minute delay. This is a reasonable starting point for a small daily job, but the right settings depend on source availability, service-level objectives, and task duration.

For more refined control, assign different policies to different tasks:

• Download tasks may benefit from exponential backoff for transient network errors or rate limits.

• Database tasks may retry transient connection failures, but authentication and permission errors should be investigated rather than retried indefinitely.

• Deterministic validation errors, such as a missing identifier field, should fail quickly and produce a clear diagnostic.

Airflow 3.3 also documents exception-specific retry policies, allowing teams to choose different retry actions based on the exception type.

4.2 Write logs that explain the outcome

A useful log should answer three questions:

1. What operation was attempted?

2. What happened?

3. How much data was processed?

The transformation script already records feature counts, geometry-repair counts, discarded empty geometries, and the output path. The DAG logs downloaded bytes and published feature counts.

For production, extend those metrics with:

• Source modification time or ETag, where available.

• Download checksum and dataset version.

• Source, rejected, and published row counts.

• Processing duration and task exit status.

• Geometry validity results and unexpected geometry-type counts.

• Database publication timestamp and total load duration.

Never log database passwords, authorization headers, or sensitive connection strings. Use Airflow’s task logs and configured secret backends for credentials and operational access.

4.3 Prevent invalid data from replacing good data

Task success should not mean merely that a Python function returned without raising an exception. Add data-quality gates before publication.

For example, reject a feed when:

• The output contains fewer features than an agreed minimum.

• Unique identifiers are missing or duplicated.

• Coordinates fall outside the expected geographic extent.

• The proportion of repaired or rejected geometries exceeds a defined threshold.

• Required attributes are unexpectedly null.

• The incoming dataset is stale or has not changed when an update was expected.

For operational monitoring, record the last successful publication time separately from the time the most recent attempt began. This makes it possible to detect when a pipeline is repeatedly failing while downstream applications continue to display old data.

5. Best practices for production spatial pipelines

The initial implementation is suitable for demonstrating scheduled vector ETL, but enterprise workloads introduce additional challenges: memory consumption, large files, concurrent processing, schema changes, and data availability requirements.

5.1 Manage GeoPandas memory consumption

GeoPandas typically holds a GeoDataFrame in memory while executing operations. A dataset’s memory footprint can be substantially larger than its compressed file size because of geometry objects, attributes, indexing, and intermediate copies created during processing.

For example, a source GeoPackage of a few hundred megabytes should not automatically be assumed to fit comfortably in a container with the same amount of available memory.

Several spatial operations can temporarily increase memory usage:

• Reading an entire vector dataset into a GeoDataFrame.

• Calling make_valid() on many complex geometries.

• Reprojecting a large dataset, creating additional geometry arrays.

• Creating temporary copies for filtering, indexing, or spatial joins.

• Loading a transformed file into memory again during publication.

The sample pipeline uses a full in-memory transformation for clarity. For larger datasets, change the execution strategy rather than relying exclusively on a larger container.

Practical memory optimization strategies

For example, a bounded read reduces unnecessary attribute loading:

import geopandas as gpdgdf = gpd.read_file(    "/opt/airflow/data/raw/source.gpkg",    columns=["feature_id", "name", "geometry"],)

The columns argument is useful when the source has many attributes and the processing needs only a few of them. A bbox or mask can also restrict spatial extent when the source’s format and read engine support the relevant filtering behavior. GeoPandas documents these options in its vector-file I/O interface.

Chunking without breaking the analysis

For a large source file, process independent subsets and write their results to intermediate GeoPackages or Parquet files. Assemble the results before loading them into the publication table.

Large source dataset
       |
       +--> Chunk 1 --> Validate and transform --> Output 1
       |
       +--> Chunk 2 --> Validate and transform --> Output 2
       |
       +--> Chunk 3 --> Validate and transform --> Output 3
                                                   |
                                                   v
                                          Merge / consolidate
                                                   |
                                                   v
                                            PostGIS staging

Chunking must be designed around the operation. Independent feature-level validation and area calculations are generally easier to partition than spatial joins, dissolves, or connected-network calculations, which may require relationships across chunk boundaries.

Avoid using arbitrary row limits as a substitute for a sound partitioning strategy. A correct spatial partition should preserve feature completeness and account for boundary effects where relevant.

5.2 Configure Docker resources and worker concurrency

GeoPandas-heavy processes should run in containers with memory limits matched to workload requirements.

Important operational practices include:

• Give spatial workers sufficient memory headroom for intermediate geometry arrays.

• Keep worker concurrency low enough that simultaneous tasks cannot exhaust available memory.

• Separate large geospatial transformations from lightweight DAG scheduling and monitoring processes.

• Set task execution timeouts based on expected processing duration.

• Monitor container memory consumption, CPU utilization, and worker restarts.

• Test with representative large datasets, including complex geometries and worst-case attribute sizes.

Do not assume that every task in a spatial DAG requires the same resources. A 200 MB HTTP download may be I/O-bound, while geometry repair on the same source may consume substantially more CPU and memory.

A dedicated queue or appropriately configured worker pool can isolate heavy processing. In distributed deployments, the same dependency versions and native geospatial libraries must be available on any worker that executes the processing task.

5.3 Choose the correct data-transfer mechanism

Moving datasets between Airflow tasks deserves particular attention.

For a small, single-host setup, a shared volume is straightforward:

Task A writes /opt/airflow/data/raw/source.gpkg
                         |
                         v
Task B reads the same shared file

In a distributed deployment, the two tasks may execute on different machines. A path stored in XCom cannot make a worker-local file visible elsewhere.

A more scalable pattern is to write the extracted data to object storage:

Source API
    |
    v
Extract task --> Object storage: raw/run_id/source.gpkg
                         |
                         v
Transform task --> Object storage: processed/run_id/output.gpkg
                         |
                         v
Load task --> PostGIS

Pass object keys or storage URIs between tasks. The actual data should remain in the shared storage service or be downloaded by the worker that needs it. Use run-specific locations so concurrent or retried executions cannot unintentionally overwrite each other’s files.

5.4 Let PostGIS handle database-native spatial work

Not every transformation needs to happen in Python. If the source is already in PostGIS, performing suitable operations in SQL can avoid moving large geometries into a Python process.

For example, area calculations can be performed in the database with ST_Area. The geometry overload uses the units of the geometry’s spatial reference system, whereas the geography overload returns square metres for area calculations.

A database-native calculation might look like this:

SELECT
    feature_id,
    ST_Area(
        ST_Transform(geometry, 6933)
    ) AS area_m2
FROM analytics.source_parcels;

This example assumes that geometry has a valid, correctly assigned source SRID and represents geometries that can be transformed into EPSG:6933.

Use GeoPandas when the workflow benefits from Python libraries, custom logic, file-based inputs, or integration with the wider Python ecosystem. Use PostGIS when data is already in the database and the work can be executed efficiently there. Many enterprise workflows use both.

5.5 Make every task safe to rerun

Failures may occur after a task has started but before Airflow receives confirmation of its completion. A retry might therefore repeat an operation that partially succeeded.

A reliable design must account for these repetitions.

For this pipeline:

• Download to a temporary file and rename it only after a completed download.

• Reprocess the same source deterministically.

• Validate the transformed file before publication.

• Load to staging before changing the published target.

• Publish in a transaction rather than leaving a partially refreshed target.

• Preserve run identifiers and metrics so repeated runs can be traced.

The source feature_id must be stable across refreshes. Generating a new random identifier every time a feature is processed undermines deduplication, change detection, and reproducible updates.

For incremental ETL, an upsert strategy keyed by a stable identifier is generally more appropriate than truncating and rebuilding the entire published table. If features can be deleted in the source, the pipeline must also define how those deletions are propagated.

5.6 Test the pipeline before scheduling it

An enterprise workflow should be tested at three levels:

Unit tests: Test geometry repair, CRS assumptions, area calculations, duplicate identifiers, and expected output schemas with small fixtures.

Integration tests: Run a complete ETL cycle against a test PostGIS database. Confirm that geometry types, SRIDs, indexes, constraints, and transaction behavior match the target design.

Operational tests: Simulate a failed download, a database outage, unexpected geometry types, and an empty feed. Verify retry behavior and confirm that the last valid published dataset remains accessible.

A useful final acceptance test is a repeat run with the exact same input. The result should be consistent, with no duplicate features or unintended schema changes.

6. Deployment checklist and troubleshooting

Before enabling the DAG in a production environment, verify the following:

Common errors and their solutions

Test the DAG in a non-production environment before enabling its daily schedule. In particular, test failure recovery against a populated target table rather than an empty one.

7. Conclusion

Automating spatial ETL with Apache Airflow and GeoPandas replaces a repetitive manual GIS process with a scheduled, observable, and repeatable data workflow.

The architecture separates responsibilities: Airflow coordinates task execution, GeoPandas and Shapely handle vector transformation, and PostGIS stores the published spatial data for applications and analysis. Validation, staged loading, transactional publication, and controlled retries protect data quality and availability.

For small and medium datasets, a GeoPandas-based pipeline is often a practical starting point. As volumes grow, shared storage, partitioned processing, database-native spatial SQL, dedicated workers, and more sophisticated publication strategies become increasingly important.

The central principle is simple: automate the processing, but make data quality and safe publication part of the automation itself. A spatial pipeline is only reliable when it can recover from failures without replacing good data with incomplete or invalid results.

Further reading

• Apache Airflow: TaskFlow API — building Python workflows and task dependencies.

• Apache Airflow: Best Practices — designing repeatable and reliable tasks.

• GeoPandas: Input/output guide — reading vector files and interacting with spatial databases.

• GeoPandas: to_postgis() reference — loading GeoDataFrames into PostGIS.

• PostGIS: ST_Area — calculating polygon areas using geometry or geography types.