Introduction to Cloud-Native Geospatial Formats: COGs & GeoParquet

1. Introduction: Why cloud-native geospatial formats matter

Traditional GIS workflows were designed around files stored on local disks, but cloud-native geospatial formats address these bottlenecks. Formats such as standard GeoTIFF and ESRI Shapefile remain useful, but their conventional storage and access patterns can become bottlenecks when applications must serve large datasets to web maps, process millions of features, or analyze imagery across distributed computing environments.

Consider a web application displaying satellite imagery covering an entire country. A conventional workflow might download a large raster before rendering a small map window. Similarly, an application querying a few attributes from a large Shapefile may have to read several associated files and process more data than necessary.

These inefficiencies become significant when datasets reach gigabytes or terabytes and hundreds of users need simultaneous access.

Cloud-native geospatial formats address this problem by aligning data organization with how cloud storage and distributed systems work. Two important examples are Cloud Optimized GeoTIFF (COG) for raster data and GeoParquet for vector data.

The key architectural distinction is that cloud-native does not simply mean storing a file in Amazon S3, Azure Blob Storage or Google Cloud Storage. It means organizing the data so applications can access and process the necessary portions without repeatedly transferring or scanning the entire dataset.

2. Cloud Optimized GeoTIFFs (COGs)

2.1 What makes a GeoTIFF cloud-optimized?

A GeoTIFF combines raster pixel values with georeferencing information. It can represent satellite imagery, digital elevation models (DEMs), land-cover classifications and temperature surfaces.

A conventional GeoTIFF may store raster data in strips or other layouts that are inefficient for retrieving small spatial windows over HTTP. A COG uses a more deliberate internal organization to support partial access.

Three mechanisms are central:

2.2 How partial raster access works

Suppose a satellite image occupies 8 GB in cloud storage, but a web map needs only a small urban area.

A simplified COG request workflow is:

The client might transfer only a small fraction of the 8 GB object, depending on the area, resolution, compression and tile layout. The exact amount is not guaranteed: metadata requests, network latency, cache behavior and the number of tiles also affect performance.

This architecture is particularly useful for interactive satellite imagery viewers, elevation services, environmental monitoring dashboards and large-area remote-sensing applications.

2.3 Converting a GeoTIFF into a COG

GDAL provides a dedicated COG driver that handles tiling, overview generation and file organization. The examples below use the GDAL command-line interface.

Step 1 — Check your GDAL installation

gdalinfo --version
gdalinfo --format COG

The COG driver is available in GDAL 3.1 and later. Check the capabilities of your installed build before running the conversion.

Step 2 — Convert the raster

gdal_translate input.tif output_cog.tif \
  -of COG \
  -co COMPRESS=DEFLATE \
  -co BLOCKSIZE=512 \
  -co OVERVIEW_RESAMPLING=AVERAGE

What these options do:

• -of COG selects the Cloud Optimized GeoTIFF driver.

• COMPRESS=DEFLATE applies lossless compression, preserving the original pixel values.

• BLOCKSIZE=512 sets the internal tile dimensions to 512 × 512 pixels.

• OVERVIEW_RESAMPLING=AVERAGE uses average resampling to create reduced-resolution overviews when they are generated.

For categorical rasters, such as land-cover classes, consider OVERVIEW_RESAMPLING=NEAREST or MODE to avoid creating misleading class values. For continuous imagery, average resampling is often suitable, although the correct choice depends on the data and analytical objective.

Step 3 — Inspect and validate the result

gdalinfo output_cog.tif

On GDAL versions that support the COG validation subcommand, run:

gdal driver cog validate output_cog.tif

Validation helps confirm that the output follows the expected COG layout. A successful conversion alone should not replace inspection of the raster’s CRS, dimensions, bands, nodata values and pixel values.

2.4 Python workflow for COG creation

If the conversion is part of an automated raster-processing pipeline, GDAL’s Python bindings can be used directly.

from osgeo import gdalsource_path = "input.tif"output_path = "output_cog.tif"gdal.UseExceptions()options = gdal.TranslateOptions(    format="COG",    creationOptions=[        "COMPRESS=DEFLATE",        "BLOCKSIZE=512",        "OVERVIEW_RESAMPLING=AVERAGE",    ],)result = gdal.Translate(    output_path,    source_path,    options=options,)if result is None:    raise RuntimeError("COG conversion failed")result = None  # Close the output datasetprint(f"Created: {output_path}")

This approach is useful for scheduled imagery ingestion, batch conversion of satellite scenes, and repeatable preprocessing pipelines. Ensure the Python GDAL bindings are installed and compatible with the GDAL library on your system.

For remote hosting, place the completed COG in object storage and configure the server to support byte-range requests and appropriate cross-origin resource sharing (CORS) when browser clients need direct access.

3. GeoParquet: A cloud-native vector data format

3.1 Extending Apache Parquet for geospatial data

Apache Parquet is a column-oriented data format designed for efficient storage and analytical processing. GeoParquet extends Parquet with standardized metadata describing geometry columns, geometry encodings and coordinate reference systems.

A Shapefile stores vector features and attributes in a record-oriented structure, typically spread across multiple associated files. GeoParquet instead takes advantage of Parquet’s columnar architecture while preserving the geometry needed for spatial analysis.

For example, imagine a building dataset containing these columns:

An application that calculates average building height may need only height_m, not every other attribute and geometry. Columnar storage allows compatible query engines to read the necessary columns rather than scanning every field.

3.2 Why columnar storage matters

GeoParquet is especially useful for municipal asset inventories, building footprints, road networks, mobility records, environmental observations and large vector datasets used in statistical analysis.

However, columnar storage does not automatically guarantee fast spatial queries. Performance also depends on row-group sizing, spatial ordering, available bounding-box metadata, partitioning and whether the query engine can apply spatial filters efficiently.

3.3 Converting a Shapefile to GeoParquet using GDAL

The GDAL vector conversion utility, ogr2ogr, can convert many common vector formats.

Step 1 — Check your installation

ogr2ogr --version
ogrinfo --formats

Check that the Parquet driver is available. GDAL’s Parquet driver supports GeoParquet output, with GeoParquet 1.0 support in GDAL 3.8 and GeoParquet 1.1 support in GDAL 3.9.

Step 2 — Convert the vector data

ogr2ogr \
  -f Parquet \
  buildings.parquet \
  buildings.shp

This command reads the Shapefile and writes its features, attributes and geometry to a Parquet file using the available GeoParquet support.

You can also reproject the data during conversion. For example, to convert to WGS 84 longitude and latitude:

ogr2ogr \
  -f Parquet \
  buildings_wgs84.parquet \
  buildings.shp \
  -t_srs EPSG:4326

Use the appropriate target CRS for your project. For accurate local area, distance or engineering calculations, retain or choose a suitable projected CRS rather than assuming longitude and latitude are always appropriate.

Step 3 — Verify the output

ogrinfo -so -al buildings.parquet

Inspect the layer schema, feature count, geometry type and spatial reference. If the file is intended for another application, verify that the receiving software supports the GeoParquet specification version and geometry encoding in the output.

3.4 Converting with Python and GeoPandas

For Python-based geospatial analysis, GeoPandas provides a straightforward alternative.

Install the required packages in your Python environment:

python -m pip install geopandas pyarrow

Then convert the dataset:

import geopandas as gpdinput_path = "buildings.shp"output_path = "buildings.parquet"gdf = gpd.read_file(input_path)# Confirm the source coordinate reference systemprint("Source CRS:", gdf.crs)print("Feature count:", len(gdf))if gdf.crs is None:    raise ValueError("Source CRS is missing; establish it before conversion.")# Write a GeoParquet filegdf.to_parquet(output_path, index=False)print(f"Saved {len(gdf)} features to {output_path}")

The CRS check is important: assigning a CRS to data whose coordinates are unknown can introduce spatial errors. If reprojection is required, use gdf.to_crs(...) with a verified target CRS before writing the file.

GeoPandas writes geometry and associated metadata through its Parquet support. The exact GeoParquet version and encoding depend on the installed library versions, so check compatibility when exchanging files across different systems.

3.5 Querying GeoParquet with DuckDB

Conversion is only the first step. The main advantage appears when analytical tools can query the resulting dataset efficiently.

DuckDB is an analytical database engine that can query Parquet files directly without requiring you to load the entire dataset into a separate database first.

Install DuckDB and its spatial extension:

python -m pip install duckdb

Example query:

INSTALL spatial;
LOAD spatial;

SELECT
    building_type,
    COUNT(*) AS building_count,
    AVG(height_m) AS mean_height
FROM read_parquet('buildings.parquet')
GROUP BY building_type
ORDER BY building_count DESC;

This example aggregates building attributes by type. It illustrates how columnar data can support analytical workloads without requiring every field to be processed.

For spatial queries, the Spatial extension provides geometry functions. The exact functions available depend on the installed DuckDB version and the geometry representation being read. For instance, a query can use a spatial intersection predicate to select buildings intersecting a study-area polygon.

One important distinction: efficient attribute queries do not necessarily imply equally efficient spatial queries. For large datasets, examine spatial filtering support, geometry decoding costs, row-group statistics and spatial partitioning.

3.6 GeoParquet in Apache Spark

Apache Spark is suited to distributed processing when datasets or workloads exceed the practical capacity of a single machine.

Spark can read Parquet files using its standard DataFrame API:

from pyspark.sql import SparkSessionfrom pyspark.sql import functions as Fspark = (    SparkSession.builder    .appName("BuildingAnalytics")    .getOrCreate())buildings = spark.read.parquet("buildings.parquet")summary = (    buildings    .groupBy("building_type")    .agg(        F.count("*").alias("building_count"),        F.avg("height_m").alias("mean_height")    )    .orderBy(F.desc("building_count")))summary.show()

This code demonstrates distributed attribute analysis. Spark does not automatically provide every geospatial operation merely because the input is GeoParquet. Geometry-aware processing generally requires a compatible spatial library, such as Apache Sedona, and an appropriate GeoParquet reader or geometry conversion workflow.

For production pipelines, consider partitioning data by meaningful administrative areas or time periods, and avoid creating thousands of tiny Parquet files. File size, partition design and data distribution can have a significant impact on query performance.

4. Designing a cloud-native geospatial architecture

COG and GeoParquet solve different problems, so they work particularly well together.

A practical architecture separates storage from compute:

• Storage layer: Holds durable raster and vector objects.

• Access layer: Uses HTTP range requests, supported cloud filesystem interfaces or compatible APIs.

• Processing layer: Runs conversion, filtering, spatial joins and other analyses.

• Application layer: Delivers maps, dashboards, reports and analytical results.

Not every GIS client can read every remote format directly. Verify driver support, authentication, CORS configuration and range-request behavior for the intended application.

5. Summary comparison: COG vs. GeoParquet vs. legacy formats

These are general tendencies rather than absolute performance rankings. A well-organized conventional dataset can outperform a poorly configured cloud-native one for a particular task, and application support remains a critical factor.

6. Final recommendations

For modern GIS storage and application development, choose the format based on the data and access pattern:

• Use COGs when users need interactive access to large satellite images, elevation rasters or other gridded datasets.

• Use GeoParquet when storing large vector datasets for column-oriented queries, data engineering and scalable analysis.

• Retain Shapefiles or standard GeoTIFFs when appropriate for compatibility with existing software and established workflows; migrate when the benefits justify the conversion and validation effort.

• Use both formats together when a system combines raster imagery with vector features, such as urban analytics, land-use monitoring or environmental risk mapping.

The broader shift is from treating geospatial data as files that must be downloaded to treating it as structured, selectively accessible cloud data. COGs optimize access to raster pixels; GeoParquet aligns vector data with modern analytical storage and compute engines.

For further technical details, consult the official specifications and documentation:

• GDAL Cloud Optimized GeoTIFF driver

• GDAL GeoParquet driver

• GeoParquet specification

• GDAL vector conversion utility