{"id":1440,"date":"2026-10-25T08:37:07","date_gmt":"2026-10-25T08:37:07","guid":{"rendered":"https:\/\/helpwithmygisassignment.com\/blog\/?p=1440"},"modified":"2026-10-25T08:37:27","modified_gmt":"2026-10-25T08:37:27","slug":"automated-spatial-qa-qc-pipelines-python","status":"publish","type":"post","link":"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/","title":{"rendered":"Automated Spatial QA\/QC Pipelines: Detecting and Repairing Invalid Geometries with Python"},"content":{"rendered":"<div><\/div>\n<div>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_89 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Automated_Spatial_QAQC_Pipelines_Detecting_and_Repairing_Invalid_Geometries_with_Python\" >Automated Spatial QA\/QC Pipelines: Detecting and Repairing Invalid Geometries with Python<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#1_Introduction\" >1. Introduction<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#2_Prerequisites_Python_Libraries_and_Environment\" >2. Prerequisites: Python Libraries and Environment<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Install_the_required_libraries\" >Install the required libraries<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Step_1_Load_raw_vector_data_and_run_topology_checks\" >Step 1: Load raw vector data and run topology checks<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Step_2_Identify_geometry_errors_and_log_their_types\" >Step 2: Identify geometry errors and log their types<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Step_3_Repair_invalid_geometries\" >Step 3: Repair invalid geometries<\/a><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Preferred_method_shapelymake_valid\" >Preferred method: shapely.make_valid()<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Alternative_method_buffer0\" >Alternative method: buffer(0)<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Step_4_Filter_small_polygon_fragments_and_export_clean_data\" >Step 4: Filter small polygon fragments and export clean data<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#3_Building_an_Automated_QAQC_Function\" >3. Building an Automated QA\/QC Function<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Complete_reusable_script\" >Complete reusable script<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Understanding_the_outputs\" >Understanding the outputs<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Robustness_adjustment_for_completely_rejected_datasets\" >Robustness adjustment for completely rejected datasets<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#4_Exporting_QAQC_Results_to_PostGIS\" >4. Exporting QA\/QC Results to PostGIS<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#5_Additional_Topology_Checks_for_Production_Pipelines\" >5. Additional Topology Checks for Production Pipelines<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Detect_overlapping_polygons_and_coverage_errors\" >Detect overlapping polygons and coverage errors<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Handle_duplicate_vertices_and_unclosed_rings_carefully\" >Handle duplicate vertices and unclosed rings carefully<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-19\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#Harden_the_pipeline_for_scheduled_execution\" >Harden the pipeline for scheduled execution<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-20\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/#6_Summary\" >6. Summary<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"Automated_Spatial_QAQC_Pipelines_Detecting_and_Repairing_Invalid_Geometries_with_Python\"><\/span>Automated Spatial QA\/QC Pipelines: Detecting and Repairing Invalid Geometries with Python<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<h2><span class=\"ez-toc-section\" id=\"1_Introduction\"><\/span>1. Introduction<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-1592\" src=\"https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_px3c1wpx3c1wpx3c-1024x572.jpg\" alt=\"Automated Spatial QA\/QC Pipelines Python\" width=\"1024\" height=\"572\" srcset=\"https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_px3c1wpx3c1wpx3c-1024x572.jpg 1024w, https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_px3c1wpx3c1wpx3c-300x167.jpg 300w, https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_px3c1wpx3c1wpx3c-768x429.jpg 768w, https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_px3c1wpx3c1wpx3c.jpg 1376w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/p>\n<p>When building Automated Spatial QA\/QC Pipelines Python workflows, spatial datasets often look correct on a map while containing geometric defects. Spatial datasets often look correct on a map while containing geometric defects that compromise spatial analysis, database imports, and automated GIS workflows. A polygon may cross its own boundary, a boundary ring may be malformed, or overlapping features may create unintended slivers. Duplicate vertices, empty geometries, inconsistent geometry types, and invalid polygon holes can also undermine downstream processing.<\/p>\n<p>These problems become particularly costly in automated workflows. An invalid parcel boundary can cause a spatial overlay to fail; a malformed administrative polygon can produce incorrect containment results; and inconsistent geometries can interrupt bulk imports into <a href=\"https:\/\/helpwithmygisassignment.com\/blog\/spatial-window-functions-recursive-ctes-postgis\/\">PostGIS<\/a> or other spatial databases. When a pipeline processes thousands of features, manually inspecting every record is neither efficient nor reproducible.<\/p>\n<p>Automated spatial QA\/QC (quality assurance and quality control) addresses this problem by systematically validating geometries, recording the reasons for failure, applying controlled repairs, and exporting only features that meet defined acceptance criteria.<\/p>\n<p>Python provides a practical toolkit for implementing this process:<\/p>\n<ul>\n<li>\n<div>\n<div>\n<p>GeoPandas manages vector datasets as geospatial data frames, supports spatial file I\/O, and integrates with pandas.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Shapely performs geometric operations, validates topology, explains invalidity, and repairs many invalid geometries.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Fiona provides an interface to vector data formats supported by GDAL, making it useful for understanding and troubleshooting data ingestion.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<\/ul>\n<p>A robust pipeline should do more than call a repair function. It should preserve the original data, log every detected problem, distinguish repaired geometries from rejected features, verify the repair outcome, and maintain an auditable record of what changed.<\/p>\n<p>This tutorial builds a reusable Python workflow that reads a raw vector dataset, detects invalid geometries, diagnoses their failure reasons, attempts repairs, screens suspicious polygon fragments, and exports the resulting datasets to GeoPackage or PostGIS.<\/p>\n<p>The central principle is simple: a geometry that can be repaired is not automatically a geometry that should be accepted. Repair must be followed by validation and application-specific quality checks.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"2_Prerequisites_Python_Libraries_and_Environment\"><\/span>2. Prerequisites: Python Libraries and Environment<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Use Python 3.10 or newer in a virtual environment. The examples below target Shapely 2.x, which provides the <code>shapely.make_valid()<\/code> and <code>shapely.is_valid_reason()<\/code> functions used throughout the pipeline.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Install_the_required_libraries\"><\/span>Install the required libraries<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<div>\n<div>\n<pre dir=\"ltr\"><code># Create and activate a virtual environment on Windows.\r\npython -m venv .venv\r\n.venv\\Scripts\\activate\r\n\r\n# Install the spatial data processing libraries.\r\npython -m pip install --upgrade pip\r\npython -m pip install geopandas shapely fiona pyproj pandas<\/code><\/pre>\n<\/div>\n<\/div>\n<p>On Linux or macOS, activate the environment with:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>source .venv\/bin\/activate<\/code><\/pre>\n<\/div>\n<\/div>\n<p>GeoPandas uses a vector-data I\/O engine such as Fiona or Pyogrio to read and write supported formats. Fiona is included explicitly here because it is useful when you need to control or troubleshoot the underlying vector file reader. It handles data access; Shapely handles geometric operations. See the <a href=\"https:\/\/geopandas.org\/en\/stable\/docs\/reference\/io.html?utm_source=chatgpt.com\" target=\"_blank\" rel=\"noopener\">GeoPandas I\/O documentation<\/a> and <a href=\"https:\/\/fiona.readthedocs.io\/en\/stable\/README.html?utm_source=chatgpt.com\" target=\"_blank\" rel=\"noopener\">Fiona documentation<\/a>.<\/p>\n<p>Verify the installed libraries:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>import geopandas as gpdimport shapelyimport fionaprint(\"GeoPandas:\", gpd.__version__)print(\"Shapely:\", shapely.__version__)print(\"Fiona:\", fiona.__version__)<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>The tutorial focuses on polygon and multipolygon datasets, such as land parcels, building footprints, land-use zones, and administrative boundaries. The repair workflow deliberately preserves polygonal output rather than allowing a repaired polygon to silently become a line or point feature.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Step_1_Load_raw_vector_data_and_run_topology_checks\"><\/span>Step 1: Load raw vector data and run topology checks<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-1593\" src=\"https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_ciqgosciqgosciqg-1024x572.jpg\" alt=\"\" width=\"1024\" height=\"572\" srcset=\"https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_ciqgosciqgosciqg-1024x572.jpg 1024w, https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_ciqgosciqgosciqg-300x167.jpg 300w, https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_ciqgosciqgosciqg-768x429.jpg 768w, https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_ciqgosciqgosciqg.jpg 1376w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/p>\n<p>Suppose the source dataset is <code>raw_parcels.gpkg<\/code>, containing a polygon layer named <code>parcels<\/code>. Start by reading the layer and checking its coordinate reference system, geometry types, empty features, and validity.<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>import geopandas as gpdimport shapelyINPUT_FILE = \"raw_parcels.gpkg\"INPUT_LAYER = \"parcels\"# Use Fiona as the vector-data reading engine.try:    gdf = gpd.read_file(        INPUT_FILE,        layer=INPUT_LAYER,        engine=\"fiona\"    )except Exception as exc:    raise RuntimeError(        f\"Could not read {INPUT_FILE}, layer {INPUT_LAYER!r}\"    ) from exc# A missing CRS prevents reliable interpretation of coordinates.if gdf.crs is None:    raise ValueError(        \"Source data has no CRS. Establish the correct CRS \"        \"before performing spatial QA\/QC.\"    )print(\"Features:\", len(gdf))print(\"CRS:\", gdf.crs)print(\"Geometry types:\")print(gdf.geom_type.value_counts(dropna=False))# Distinguish missing geometry from empty geometry.missing_mask = gdf.geometry.isna()empty_mask = gdf.geometry.is_empty# Topological validity is a separate test.valid_mask = gdf.geometry.is_validprint(\"Missing geometries:\", missing_mask.sum())print(\"Empty geometries:\", empty_mask.sum())<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p><code>geometry.is_valid<\/code> evaluates whether a geometry satisfies the geometry engine&#8217;s validity rules. It does not certify that a feature is geographically accurate, correctly classified, free of duplicates, or consistent with neighboring polygons.<\/p>\n<p>That distinction matters. A perfectly valid parcel may be in the wrong county, while two valid administrative polygons may overlap when they should form a non-overlapping coverage.<\/p>\n<p>For broader data cleaning, track at least four categories:<\/p>\n<div>\n<div>\n<table>\n<tbody>\n<tr>\n<th>QA category<\/th>\n<th>Example check<\/th>\n<th>Why it matters<\/th>\n<\/tr>\n<tr>\n<td>Structural integrity<\/td>\n<td>Missing or empty geometry<\/td>\n<td>Prevents unusable spatial records<\/td>\n<\/tr>\n<tr>\n<td>Topological validity<\/td>\n<td>Self-intersections, invalid holes<\/td>\n<td>Prevents invalid geometric operations<\/td>\n<\/tr>\n<tr>\n<td>Schema integrity<\/td>\n<td>Unexpected geometry type<\/td>\n<td>Prevents incompatible database loads<\/td>\n<\/tr>\n<tr>\n<td>Spatial consistency<\/td>\n<td>Overlaps, gaps, duplicate features<\/td>\n<td>Detects dataset-level errors<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<h3><span class=\"ez-toc-section\" id=\"Step_2_Identify_geometry_errors_and_log_their_types\"><\/span>Step 2: Identify geometry errors and log their types<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-1594 size-large\" src=\"https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_1sgc4a1sgc4a1sgc-1024x559.jpg\" alt=\"\" width=\"1024\" height=\"559\" srcset=\"https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_1sgc4a1sgc4a1sgc-1024x559.jpg 1024w, https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_1sgc4a1sgc4a1sgc-300x164.jpg 300w, https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_1sgc4a1sgc4a1sgc-768x419.jpg 768w, https:\/\/helpwithmygisassignment.com\/blog\/wp-content\/uploads\/2026\/10\/Gemini_Generated_Image_1sgc4a1sgc4a1sgc.jpg 1408w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/p>\n<p>A Boolean validity flag tells you which records need attention, but it does not explain the problem. Shapely&#8217;s <code>is_valid_reason()<\/code> function returns a diagnostic string for invalid geometries. The alternative <code>shapely.validation.explain_validity()<\/code> serves a similar purpose. See the <a href=\"https:\/\/shapely.readthedocs.io\/en\/stable\/manual.html?utm_source=chatgpt.com#diagnostics\" target=\"_blank\" rel=\"noopener\">Shapely geometry validation documentation<\/a>.<\/p>\n<p>Create a diagnostic report before modifying the dataset:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>import pandas as pdimport shapely# Preserve a source identifier for tracing features back to the input.gdf = gdf.copy()gdf[\"_qa_source_id\"] = gdf.index.astype(str)# Identify invalid, missing, or empty geometries.invalid_mask = ~gdf.geometry.is_validproblem_mask = (    gdf.geometry.isna()    | gdf.geometry.is_empty.fillna(False)    | invalid_mask)problems = gdf.loc[problem_mask].copy()diagnostics = []for _, row in problems.iterrows():    geom = row.geometry    if geom is None:        reason = \"Missing geometry\"    elif geom.is_empty:        reason = \"Empty geometry\"    elif not geom.is_valid:        reason = shapely.is_valid_reason(geom)    else:        reason = \"Other QA issue\"    diagnostics.append({        \"source_id\": row[\"_qa_source_id\"],        \"geometry_type\": (            None if geom is None else geom.geom_type        ),        \"valid\": (            False if geom is None else geom.is_valid        ),        \"error_reason\": reason,    })diagnostic_df = pd.DataFrame(diagnostics)diagnostic_df.to_csv(\"geometry_errors.csv\", index=False)print(diagnostic_df.head(20))<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>Typical diagnostic messages include:<\/p>\n<ul>\n<li>\n<div>\n<div>\n<p><code>Self-intersection<\/code> \u2014 a polygon boundary crosses itself.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p><code>Ring Self-intersection<\/code> \u2014 a ring intersects itself, possibly creating an invalid hole or shell.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p><code>Hole lies outside shell<\/code> \u2014 an interior ring lies outside the exterior polygon.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p><code>Too few points<\/code> \u2014 a ring or geometry does not have enough points to define the expected shape.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<\/ul>\n<p>The precise wording depends on the geometry engine and version. Treat the reported text as a diagnostic, not as a permanent error-code standard.<\/p>\n<p>What about duplicate vertices and unclosed rings? These need special attention. Repeated consecutive coordinates do not necessarily make a polygon invalid. Likewise, some file readers or geometry constructors may automatically close polygon rings during ingestion, while malformed source files can fail before <code>is_valid<\/code> is reached. A validity check is therefore only one component of a complete spatial QA\/QC pipeline.<\/p>\n<p>For a separate duplicate-vertex cleanup pass, Shapely provides <code>remove_repeated_points()<\/code>. Use it cautiously: coordinate changes can affect topology, and any modified geometry should be validated again. Do not remove vertices merely because they are duplicated somewhere in a ring; repeated coordinates may have a legitimate geometric role.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Step_3_Repair_invalid_geometries\"><\/span>Step 3: Repair invalid geometries<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Shapely provides two common approaches to repairing invalid polygons: <code>make_valid()<\/code> and the traditional <code>buffer(0)<\/code> technique.<\/p>\n<h4><span class=\"ez-toc-section\" id=\"Preferred_method_shapelymake_valid\"><\/span>Preferred method: <code>shapely.make_valid()<\/code><span class=\"ez-toc-section-end\"><\/span><\/h4>\n<div>\n<div>\n<pre dir=\"ltr\"><code>import shapelyinvalid_geom = gdf.loc[    ~gdf.geometry.is_valid].geometry.iloc[0]# Repair an invalid polygon using Shapely.repaired_geom = shapely.make_valid(invalid_geom)print(\"Original type:\", invalid_geom.geom_type)print(\"Original valid:\", invalid_geom.is_valid)print(\"Repaired type:\", repaired_geom.geom_type)print(\"Repaired valid:\", repaired_geom.is_valid)print(\"Repaired empty:\", repaired_geom.is_empty)<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p><code>make_valid()<\/code> attempts to produce a valid representation of the input geometry. Its result may be a <code>MultiPolygon<\/code> or a <code>GeometryCollection<\/code>, rather than a single <code>Polygon<\/code>. For instance, a self-crossing bow-tie polygon may be split into multiple polygon components. In other cases, repair can return polygonal and lower-dimensional components together. These are documented behaviors, not necessarily repair failures. See <a href=\"https:\/\/shapely.readthedocs.io\/en\/stable\/manual.html?utm_source=chatgpt.com#validation-make-valid\" target=\"_blank\" rel=\"noopener\">Shapely&#8217;s <code>make_valid()<\/code> documentation<\/a>.<\/p>\n<p>This creates an important design decision: if your target dataset must contain only polygons, you need to extract and verify the polygonal components while logging any discarded line or point remnants.<\/p>\n<h4><span class=\"ez-toc-section\" id=\"Alternative_method_buffer0\"><\/span>Alternative method: <code>buffer(0)<\/code><span class=\"ez-toc-section-end\"><\/span><\/h4>\n<div>\n<div>\n<pre dir=\"ltr\"><code># Legacy fallback for certain invalid polygon cases.# This is not guaranteed to preserve the intended geometry.fallback_geom = invalid_geom.buffer(0)print(\"Fallback type:\", fallback_geom.geom_type)print(\"Fallback valid:\", fallback_geom.is_valid)<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>A zero-distance buffer can clean certain self-crossing or self-touching polygon configurations, but it may produce a different result from <code>make_valid()<\/code>. Shapely documents this technique as a sometimes-useful cleanup approach rather than a universal repair method. See the <a href=\"https:\/\/shapely.readthedocs.io\/en\/stable\/manual.html?utm_source=chatgpt.com#object.buffer\" target=\"_blank\" rel=\"noopener\">Shapely buffer documentation<\/a>.<\/p>\n<p>For a repeatable data pipeline, use <code>make_valid()<\/code> first and consider <code>buffer(0)<\/code> only as a controlled fallback. Log which method succeeded so the result remains auditable.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Step_4_Filter_small_polygon_fragments_and_export_clean_data\"><\/span>Step 4: Filter small polygon fragments and export clean data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Repairing geometry validity is not the same as assessing whether the resulting polygon is useful. A repaired polygon can have a negligible area, or a self-intersecting source polygon can become a multipolygon whose parts need additional review.<\/p>\n<p>For this tutorial, we will use an area threshold to identify tiny polygon fragments. A small area is a screening criterion, not proof of a sliver. A small building footprint or land parcel may be legitimate. True sliver detection can also use shape compactness, width, overlap with neighboring features, and the positional accuracy expected from the source data.<\/p>\n<p>Area calculations must use an appropriate projected coordinate reference system with metre units. Do not calculate polygon areas directly from longitude and latitude.<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code># Example only: use a suitable projected CRS for your study area.AREA_CRS = \"EPSG:32637\"# A threshold of 1 square metre is illustrative, not universal.MIN_AREA_M2 = 1.0# Reproject a copy for area measurement.projected = gdf.to_crs(AREA_CRS)# Calculate area in square metres.gdf[\"area_m2\"] = projected.geometry.area.to_numpy()# Flag very small polygons for review.sliver_candidates = gdf.loc[    gdf[\"area_m2\"] &lt; MIN_AREA_M2].copy()print(\"Small polygon candidates:\", len(sliver_candidates))<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>This example demonstrates the area test; in the complete pipeline below, apply it to the repaired geometries instead of the raw data. The example CRS is suitable only for areas where the selected projection is appropriate. Choose a local projected CRS for your own study region.<\/p>\n<p>Do not delete candidates without preserving an audit trail. A production workflow should export accepted geometries, quarantine rejected features, and create a report explaining each decision.<\/p>\n<p>GeoPackage is a convenient delivery format for the cleaned data:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code># Export accepted polygons to a GeoPackage.cleaned_gdf.to_file(    \"cleaned_parcels.gpkg\",    layer=\"parcels_clean\",    driver=\"GPKG\",    index=False)<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>GeoPackage supports layers in a single file and can store spatial metadata. GeoPandas also supports writing vector data to PostGIS through <code>to_postgis()<\/code>, which requires SQLAlchemy, GeoAlchemy2, and a PostgreSQL driver. See the <a href=\"https:\/\/geopandas.org\/en\/stable\/docs\/user_guide\/io.html?utm_source=chatgpt.com\" target=\"_blank\" rel=\"noopener\">GeoPandas file I\/O guide<\/a> and <a href=\"https:\/\/docs.geopandas.org\/en\/stable\/docs\/reference\/api\/geopandas.GeoDataFrame.to_postgis.html?utm_source=chatgpt.com\" target=\"_blank\" rel=\"noopener\"><code>to_postgis()<\/code> reference<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"3_Building_an_Automated_QAQC_Function\"><\/span>3. Building an Automated QA\/QC Function<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The next step is to combine ingestion, diagnostics, repairs, area screening, and export into a reusable function.<\/p>\n<p>The function below implements several safeguards:<\/p>\n<ul>\n<li>\n<div>\n<div>\n<p>Validates input CRS and polygon geometry types.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Records the original validity reason for each feature.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Tries <code>make_valid()<\/code> before considering <code>buffer(0)<\/code>.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Extracts polygonal components from repaired geometry collections.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Rechecks the validity of repaired geometries.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Rejects geometries that cannot be repaired into valid polygonal output.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Applies an optional area threshold using an appropriate projected CRS.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Exports clean data, a rejection layer where possible, and a CSV audit report.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div>\n<div>\n<p>Uses logging and exception handling to make failures visible.<\/p>\n<\/div>\n<\/div>\n<\/li>\n<\/ul>\n<h3><span class=\"ez-toc-section\" id=\"Complete_reusable_script\"><\/span>Complete reusable script<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Save the following as <code>spatial_qa_qc.py<\/code>.<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>from pathlib import Pathimport loggingimport geopandas as gpdimport pandas as pdimport shapelyfrom pyproj import CRS# ------------------------------------------------------------# 1. Configure logging# ------------------------------------------------------------logging.basicConfig(    level=logging.INFO,    format=\"%(asctime)s | %(levelname)s | %(message)s\")logger = logging.getLogger(\"spatial_qa_qc\")# ------------------------------------------------------------# 2. Extract polygonal components from a geometry# ------------------------------------------------------------def polygonal_only(geom):    \"\"\"    Return valid polygonal parts as a Polygon or MultiPolygon.    Lower-dimensional components such as LineStrings and Points    are excluded because this pipeline targets polygon datasets.    Returns None if no polygonal component exists.    \"\"\"    if geom is None or geom.is_empty:        return None    # Preserve ordinary polygonal geometries without alteration.    if geom.geom_type in {\"Polygon\", \"MultiPolygon\"}:        return geom    parts = []    def collect_polygons(item):        # Collect individual polygon components.        if item.geom_type == \"Polygon\":            parts.append(item)        elif item.geom_type == \"MultiPolygon\":            parts.extend(list(item.geoms))        # Recursively inspect GeometryCollections.        elif item.geom_type == \"GeometryCollection\":            for subgeom in item.geoms:                collect_polygons(subgeom)    collect_polygons(geom)    if not parts:        return None    if len(parts) == 1:        return parts[0]    # Combine the polygon pieces within this one source feature.    # The caller will verify the result's validity.    return shapely.union_all(parts)# ------------------------------------------------------------# 3. Attempt controlled geometry repair# ------------------------------------------------------------def repair_polygon(geom):    \"\"\"    Return (repaired_geometry, method, error_message).    The result is accepted only if it is a non-empty, valid    Polygon or MultiPolygon. Otherwise, return None.    \"\"\"    errors = []    def validate_candidate(candidate):        if candidate is None or candidate.is_empty:            return None        # Repair may return a GeometryCollection. Retain only        # polygonal components for this polygon-only workflow.        candidate = polygonal_only(candidate)        if candidate is None or candidate.is_empty:            return None        if not candidate.is_valid:            return None        if candidate.geom_type not in {\"Polygon\", \"MultiPolygon\"}:            return None        return candidate    # Method 1: use the dedicated Shapely repair function.    try:        candidate = shapely.make_valid(geom)        result = validate_candidate(candidate)        if result is not None:            return result, \"make_valid\", \"\"        errors.append(\"make_valid produced no acceptable polygon\")    except Exception as exc:        errors.append(f\"make_valid failed: {exc}\")    # Method 2: try buffer(0) only as a fallback.    # It can alter geometry, so its use is recorded explicitly.    try:        candidate = geom.buffer(0)        result = validate_candidate(candidate)        if result is not None:            return result, \"buffer(0)\", \"; \".join(errors)        errors.append(\"buffer(0) produced no acceptable polygon\")    except Exception as exc:        errors.append(f\"buffer(0) failed: {exc}\")    return None, \"failed\", \"; \".join(errors)# ------------------------------------------------------------# 4. Run the complete QA\/QC workflow# ------------------------------------------------------------def run_spatial_qa_qc(    input_path,    output_dir,    area_crs,    layer=None,    id_field=None,    min_area_m2=None):    \"\"\"    Clean a polygon vector dataset and save:      - cleaned.gpkg: accepted polygons      - rejected.gpkg: rejected source features, if writable      - qa_audit.csv: per-feature decisions and diagnostics    Parameters    ----------    input_path : str or Path        Input vector dataset path.    output_dir : str or Path        Directory where outputs will be written.    area_crs : str        Suitable projected CRS with metre units, used for        area-based checks.    layer : str, optional        Input layer name, e.g. for a GeoPackage.    id_field : str, optional        Stable input attribute to include in the audit report.    min_area_m2 : float, optional        Minimum accepted polygon area in square metres.        Set to None to disable automatic area filtering.    \"\"\"    input_path = Path(input_path)    output_dir = Path(output_dir)    output_dir.mkdir(parents=True, exist_ok=True)    if min_area_m2 is not None and min_area_m2 &lt; 0:        raise ValueError(\"min_area_m2 cannot be negative.\")    # Check that the area CRS uses projected metre units.    metric_crs = CRS.from_user_input(area_crs)    if not metric_crs.is_projected:        raise ValueError(            \"area_crs must be a projected CRS with metre units.\"        )    units = [        axis.unit_name.lower()        for axis in metric_crs.axis_info[:2]    ]    if not units or not all(        \"metre\" in unit or \"meter\" in unit        for unit in units    ):        raise ValueError(            \"area_crs must use metre units for area screening.\"        )    # Read the source without overwriting it.    try:        raw = gpd.read_file(            input_path,            layer=layer,            engine=\"fiona\"        )    except Exception as exc:        logger.exception(\"Failed to read input dataset.\")        raise RuntimeError(            f\"Could not read source dataset: {input_path}\"        ) from exc    if raw.crs is None:        raise ValueError(            \"Input CRS is missing. Establish the correct CRS first.\"        )    if raw.empty:        raise ValueError(\"Input dataset contains no features.\")    geom_col = raw.geometry.name    if \"_qa_source_row\" in raw.columns:        raise ValueError(            \"Input already contains reserved column '_qa_source_row'.\"        )    if id_field is not None and id_field not in raw.columns:        raise ValueError(f\"ID field not found: {id_field}\")    logger.info(\"Loaded %s features.\", len(raw))    logger.info(\"Source CRS: %s\", raw.crs)    accepted_records = []    rejected_records = []    audit_rows = []    # Enumerating rows creates a unique run-level source reference    # even where the original DataFrame index contains duplicates.    for position, (_, row) in enumerate(raw.iterrows()):        source_row = str(position)        geom = row[geom_col]        original_type = (            None if geom is None            else getattr(geom, \"geom_type\", type(geom).__name__)        )        original_valid = (            False if geom is None            else bool(                hasattr(geom, \"is_valid\") and geom.is_valid            )        )        original_reason = \"Missing geometry\"        if geom is not None and hasattr(geom, \"is_empty\"):            if geom.is_empty:                original_reason = \"Empty geometry\"            elif geom.is_valid:                original_reason = \"Valid Geometry\"            else:                original_reason = shapely.is_valid_reason(geom)        external_id = (            str(row[id_field])            if id_field is not None            else source_row        )        audit = {            \"source_row\": source_row,            \"source_id\": external_id,            \"original_type\": original_type,            \"original_valid\": original_valid,            \"original_reason\": original_reason,            \"repair_method\": \"none\",            \"output_type\": None,            \"output_valid\": False,            \"area_m2\": None,            \"status\": None,            \"details\": \"\"        }        # Keep a copy of the original row for the rejection layer.        original_record = row.to_dict()        original_record[\"_qa_source_row\"] = source_row        # Reject missing or empty geometries.        if (            geom is None            or not hasattr(geom, \"geom_type\")            or geom.is_empty        ):            audit[\"status\"] = \"rejected_missing_or_empty\"            rejected_records.append(original_record)            audit_rows.append(audit)            continue        # This script explicitly targets polygon datasets.        if geom.geom_type not in {\"Polygon\", \"MultiPolygon\"}:            audit[\"status\"] = \"rejected_non_polygon_input\"            audit[\"details\"] = (                \"Expected Polygon or MultiPolygon input.\"            )            rejected_records.append(original_record)            audit_rows.append(audit)            continue        # Preserve valid geometries; repair only invalid ones.        if geom.is_valid:            fixed = geom            method = \"unchanged\"            repair_error = \"\"        else:            fixed, method, repair_error = repair_polygon(geom)        audit[\"repair_method\"] = method        audit[\"details\"] = repair_error        # Reject features that cannot be repaired adequately.        if fixed is None or not fixed.is_valid:            audit[\"status\"] = \"rejected_repair_failed\"            rejected_records.append(original_record)            audit_rows.append(audit)            continue        # Preserve original attributes and store the repaired geometry.        cleaned_record = row.to_dict()        cleaned_record[geom_col] = fixed        cleaned_record[\"_qa_source_row\"] = source_row        accepted_records.append(cleaned_record)        audit[\"output_type\"] = fixed.geom_type        audit[\"output_valid\"] = bool(fixed.is_valid)        audit[\"status\"] = (            \"valid_unchanged\"            if method == \"unchanged\"            else \"repaired\"        )        audit_rows.append(audit)    # Construct output frames using the original geometry column and CRS.    accepted = gpd.GeoDataFrame(        accepted_records,        geometry=geom_col,        crs=raw.crs    )    audit_by_row = {        record[\"source_row\"]: record        for record in audit_rows    }    # --------------------------------------------------------    # Area-based screening after geometry repair    # --------------------------------------------------------    final_records = []    if not accepted.empty:        projected = accepted.to_crs(metric_crs)        areas = projected.geometry.area.to_numpy()        for position, (_, row) in enumerate(accepted.iterrows()):            record = row.to_dict()            source_row = str(record[\"_qa_source_row\"])            area = float(areas[position])            audit = audit_by_row[source_row]            audit[\"area_m2\"] = area            record[\"qa_area_m2\"] = area            # If configured, quarantine undersized polygons.            if (                min_area_m2 is not None                and area &lt; min_area_m2            ):                audit[\"status\"] = \"rejected_below_min_area\"                audit[\"details\"] = (                    f\"Area {area:.6f} m\u00b2 is below \"                    f\"threshold {min_area_m2:.6f} m\u00b2.\"                )                rejected_records.append(record)            else:                final_records.append(record)    cleaned = gpd.GeoDataFrame(        final_records,        geometry=geom_col,        crs=raw.crs    )    rejected = gpd.GeoDataFrame(        rejected_records,        geometry=geom_col,        crs=raw.crs    )    # --------------------------------------------------------    # Verify final output validity    # --------------------------------------------------------    if not cleaned.empty:        if (            cleaned.geometry.isna().any()            or cleaned.geometry.is_empty.any()            or not cleaned.geometry.is_valid.all()        ):            raise RuntimeError(                \"Final QA failed: invalid or empty geometry remains.\"            )        # Remove temporary area measurement from the final data;        # the audit CSV retains the calculated area.        if \"qa_area_m2\" in cleaned.columns:            cleaned = cleaned.drop(columns=\"qa_area_m2\")        clean_path = output_dir \/ \"cleaned.gpkg\"        try:            cleaned.to_file(                clean_path,                layer=\"cleaned\",                driver=\"GPKG\",                index=False            )            logger.info(\"Clean dataset written to %s\", clean_path)        except Exception as exc:            logger.exception(\"Could not write clean GeoPackage.\")            raise RuntimeError(                f\"Failed to write {clean_path}\"            ) from exc    else:        logger.warning(            \"No features passed QA\/QC; no clean GeoPackage written.\"        )    # Export rejected source features separately where possible.    if not rejected.empty:        rejected_path = output_dir \/ \"rejected.gpkg\"        try:            rejected.to_file(                rejected_path,                layer=\"rejected\",                driver=\"GPKG\",                index=False            )            logger.info(                \"Rejected features written to %s\", rejected_path            )        except Exception:            # Some rejected geometries may not be serializable.            # Keep the audit CSV even when this export fails.            logger.exception(                \"Could not write rejection geometries. \"                \"Use the audit CSV to identify affected records.\"            )    # Always produce an audit report.    audit_df = pd.DataFrame(audit_rows)    report_path = output_dir \/ \"qa_audit.csv\"    try:        audit_df.to_csv(report_path, index=False)    except Exception as exc:        logger.exception(\"Could not write QA audit report.\")        raise RuntimeError(            f\"Failed to write {report_path}\"        ) from exc    logger.info(        \"Finished: input=%s, accepted=%s, rejected=%s\",        len(raw),        len(cleaned),        len(rejected)    )    return cleaned, rejected, audit_df# ------------------------------------------------------------# 5. Example execution# ------------------------------------------------------------if __name__ == \"__main__\":    cleaned_gdf, rejected_gdf, audit_df = run_spatial_qa_qc(        input_path=\"raw_parcels.gpkg\",        layer=\"parcels\",        output_dir=\"qa_output\",        # Select a projected CRS appropriate for the study area.        # EPSG:32637 is an example for suitable locations in UTM 37N.        area_crs=\"EPSG:32637\",        # Replace with your actual source attribute, if available.        id_field=None,        # Example threshold only; validate against your dataset.        min_area_m2=1.0    )    print(\"\\nQA\/QC summary:\")    print(audit_df[\"status\"].value_counts(dropna=False))<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<h3><span class=\"ez-toc-section\" id=\"Understanding_the_outputs\"><\/span>Understanding the outputs<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>After a successful run, the output directory should contain:<\/p>\n<div>\n<div>\n<table>\n<tbody>\n<tr>\n<th>File<\/th>\n<th>Contents<\/th>\n<\/tr>\n<tr>\n<td><code>cleaned.gpkg<\/code><\/td>\n<td>Features that passed the repair and acceptance rules<\/td>\n<\/tr>\n<tr>\n<td><code>rejected.gpkg<\/code><\/td>\n<td>Features rejected or quarantined for review, if writable<\/td>\n<\/tr>\n<tr>\n<td><code>qa_audit.csv<\/code><\/td>\n<td>Source identifiers, original validity reasons, repair methods, final statuses, and area measurements<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<p>A clean feature may be marked <code>valid_unchanged<\/code> or <code>repaired<\/code>. A feature can be rejected for a missing geometry, unexpected geometry type, failed repair, or area below the configured threshold.<\/p>\n<p>The audit report intentionally records repair method and acceptance status separately. This lets an analyst distinguish a polygon that was repaired successfully from one that was already valid and passed unchanged.<\/p>\n<p>Note that the script treats the area threshold as an optional business rule. Set <code>min_area_m2=None<\/code> to disable it, or configure a threshold appropriate to the mapping scale, legal requirements, and expected size of legitimate features.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Robustness_adjustment_for_completely_rejected_datasets\"><\/span>Robustness adjustment for completely rejected datasets<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>One edge case is worth handling before production use: if every feature is rejected, a GeoDataFrame constructed from an empty list of records may not have the geometry column defined. To support datasets where zero features pass QA\/QC, add this helper inside <code>run_spatial_qa_qc()<\/code>, after validating the source dataset:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>def build_output_frame(records):    \"\"\"Build a GeoDataFrame even when no records were accepted.\"\"\"    if records:        return gpd.GeoDataFrame(            records,            geometry=geom_col,            crs=raw.crs        )    # Preserve the original schema and active geometry column.    empty = raw.iloc[0:0].copy()    empty[\"_qa_source_row\"] = pd.Series(dtype=\"object\")    return empty<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>Then replace the three GeoDataFrame constructors in the main function with:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>accepted = build_output_frame(accepted_records)# ...perform the area screening as shown above...cleaned = build_output_frame(final_records)rejected = build_output_frame(rejected_records)<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>This allows the pipeline to finish its audit reporting even when every feature is quarantined. The export section can then skip creating an empty clean GeoPackage while still reporting why no features passed.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"4_Exporting_QAQC_Results_to_PostGIS\"><\/span>4. Exporting QA\/QC Results to PostGIS<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>For multi-user GIS applications, a spatial database is often a more useful destination than a standalone file. After verifying that the cleaned output meets your acceptance criteria, you can load it into PostGIS.<\/p>\n<p>Install the additional dependencies:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>python -m pip install sqlalchemy geoalchemy2 \"psycopg[binary]\"<\/code><\/pre>\n<\/div>\n<\/div>\n<p>Store the PostgreSQL connection string in an environment variable named <code>POSTGIS_URL<\/code>. For example, a connection string may follow this pattern:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>postgresql+psycopg:\/\/USER:PASSWORD@HOST:5432\/DATABASE<\/code><\/pre>\n<\/div>\n<\/div>\n<p>Do not commit real credentials to a source-code repository.<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>import osimport loggingfrom sqlalchemy import create_enginelogger = logging.getLogger(\"spatial_qa_qc\")database_url = os.environ.get(\"POSTGIS_URL\")if not database_url:    raise RuntimeError(        \"POSTGIS_URL is missing from the environment.\"    )engine = create_engine(database_url)try:    if cleaned_gdf.empty:        logger.warning(            \"No accepted features are available for PostGIS.\"        )    else:        cleaned_gdf.to_postgis(            name=\"parcels_clean\",            con=engine,            schema=\"staging\",            if_exists=\"fail\",            index=False        )        logger.info(            \"Clean spatial data successfully loaded into PostGIS.\"        )except Exception:    logger.exception(\"PostGIS export failed.\")<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>The <code>staging<\/code> schema must already exist, and the database user must have suitable permissions. The <code>if_exists=\"fail\"<\/code> setting prevents this example from silently replacing an existing table.<\/p>\n<p>After the import, perform a database-side validity check. If your geometry column is called <code>geometry<\/code>, run:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code>SELECT\r\n    id,\r\n    ST_IsValid(geometry) AS is_valid,\r\n    ST_IsValidReason(geometry) AS validity_reason\r\nFROM staging.parcels_clean\r\nWHERE NOT ST_IsValid(geometry);<\/code><\/pre>\n<\/div>\n<\/div>\n<p>Replace <code>id<\/code> and <code>geometry<\/code> with the actual column names in your table. A successful query returning no rows indicates that no invalid geometries were found by this check. It does not establish that the features are geographically correct or free from overlaps.<\/p>\n<p>PostGIS provides another layer of validation close to the system that will consume the data. This is useful when the pipeline feeds spatial joins, database constraints, web services, or other automated geoprocessing tasks.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"5_Additional_Topology_Checks_for_Production_Pipelines\"><\/span>5. Additional Topology Checks for Production Pipelines<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Individual geometry validity is necessary, but it is not sufficient for every spatial dataset. A production pipeline should also test the relationships between features and preserve a clear distinction between structural defects and business-rule violations.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Detect_overlapping_polygons_and_coverage_errors\"><\/span>Detect overlapping polygons and coverage errors<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Consider a dataset of adjacent administrative areas. Every polygon might pass <code>geometry.is_valid<\/code>, yet the dataset could still contain unintended gaps or overlaps.<\/p>\n<p>In versions of GeoPandas and GEOS that support polygon coverage validation, you can run:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code># Run this after individual geometries have been validated.if hasattr(cleaned_gdf.geometry, \"is_valid_coverage\"):    coverage_ok = cleaned_gdf.geometry.is_valid_coverage()    print(\"Coverage valid:\", coverage_ok)    if not coverage_ok:        # Inspect edges associated with coverage errors.        bad_edges = cleaned_gdf.geometry.invalid_coverage_edges()        print(\"Coverage issues:\", (~bad_edges.is_empty).sum())<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>Use coverage validation only where the features are expected to form a polygon coverage. For example, a dataset of neighboring administrative boundaries may be expected to have no overlaps, whereas a dataset of building footprints may legitimately contain gaps. The acceptance rules must reflect the spatial model.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Handle_duplicate_vertices_and_unclosed_rings_carefully\"><\/span>Handle duplicate vertices and unclosed rings carefully<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Repeated consecutive vertices may be removable without materially changing a polygon, but repeated coordinates are not automatically evidence of invalid topology. Shapely supports <code>remove_repeated_points()<\/code> for this purpose. If you add it to your cleaning workflow, record that the coordinates were changed and validate the result again.<\/p>\n<p>Unclosed rings require a different approach. A malformed source file may fail during ingestion before any Shapely validity function can inspect its geometry. The main pipeline therefore catches read failures, but it cannot repair a feature that never reached the in-memory GeoDataFrame. Such cases need source-file inspection or a separate ingestion-recovery procedure.<\/p>\n<p>A useful optional diagnostic for any suspicious polygon is:<\/p>\n<div>\n<div>\n<pre dir=\"ltr\"><code># Only run this when a feature has actually been identified# as having repeated consecutive coordinates.candidate = shapely.remove_repeated_points(invalid_geom)if not candidate.is_empty:    print(\"Candidate valid:\", candidate.is_valid)    print(\"Candidate type:\", candidate.geom_type)<\/code><\/pre>\n<\/div>\n<div><\/div>\n<\/div>\n<p>This is a diagnostic example, not a replacement for the main repair function. Check the output before accepting it, particularly if the geometry was already invalid.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Harden_the_pipeline_for_scheduled_execution\"><\/span>Harden the pipeline for scheduled execution<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>For recurring data deliveries, incorporate the workflow into a scheduled job or continuous integration process. Useful operational controls include recording the input filename and processing timestamp, retaining the original source, tracking repaired and rejected feature counts, and failing the job when the proportion of invalid features exceeds an agreed tolerance.<\/p>\n<p>For instance, a pipeline can reject an entire delivery if too many records require repair. This prevents a severely degraded source dataset from being accepted simply because every individual feature was eventually converted into a valid geometry.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"6_Summary\"><\/span>6. Summary<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>An automated spatial QA\/QC pipeline turns geometry validation from a manual troubleshooting exercise into a repeatable data engineering process.<\/p>\n<p>GeoPandas manages vector datasets, Shapely identifies and repairs invalid geometry, and Fiona provides a reliable vector-data access option. Together, they support a workflow that detects topology failures, records error reasons, applies controlled repairs, checks repaired results, screens small polygon fragments, and exports accepted data to GeoPackage or PostGIS.<\/p>\n<p>The most important engineering practices are to preserve the original data, validate every repaired result, use a suitable projected CRS for area calculations, and maintain an audit trail for rejected features. Do not assume that every invalid polygon can be repaired safely or that every small polygon is a sliver.<\/p>\n<p>A reliable spatial data cleaning pipeline does not merely make geometries valid. It makes its decisions traceable, its acceptance criteria explicit, and its outputs suitable for the next stage of analysis.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Automated Spatial QA\/QC Pipelines: Detecting and Repairing Invalid Geometries with Python 1. Introduction When building Automated Spatial QA\/QC Pipelines Python workflows, spatial datasets often look correct on a map while containing geometric defects. Spatial datasets often look correct on a map while containing geometric defects that compromise spatial analysis, database imports, and automated GIS workflows. &#8230; <a title=\"Automated Spatial QA\/QC Pipelines: Detecting and Repairing Invalid Geometries with Python\" class=\"read-more\" href=\"https:\/\/helpwithmygisassignment.com\/blog\/automated-spatial-qa-qc-pipelines-python\/\" aria-label=\"Read more about Automated Spatial QA\/QC Pipelines: Detecting and Repairing Invalid Geometries with Python\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":1592,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,4],"tags":[],"class_list":["post-1440","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","category-gis-articles"],"_links":{"self":[{"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/posts\/1440","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/comments?post=1440"}],"version-history":[{"count":4,"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/posts\/1440\/revisions"}],"predecessor-version":[{"id":1596,"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/posts\/1440\/revisions\/1596"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/media\/1592"}],"wp:attachment":[{"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/media?parent=1440"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/categories?post=1440"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/helpwithmygisassignment.com\/blog\/wp-json\/wp\/v2\/tags?post=1440"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}