arcgis-to-portaljs

Migrate a whole ArcGIS Hub site into a PortalJS Arc portal end-to-end. Harvests the Hub /data.json (DCAT-US) inventory, exports every FeatureService layer through the ArcGIS REST query API with resultOffset paging, converts each to the serverless dual tier (PMTiles render + GeoParquet query) with tabular items to Parquet, pushes everything to Cloudflare R2 via Git LFS, appends dual-tier datasets.json entries, and writes a source-vs-derived parity report. Use to move a City or sector ArcGIS Hub open-data portal onto PortalJS with no server-side compute.

v0.1.0
Datopian
MIT

Allowed Tools

ReadWriteEditBash(ogr2ogr:*)Bash(ogrinfo:*)Bash(tippecanoe:*)Bash(duckdb:*)Bash(git:*)Bash(curl:*)Bash(jq:*)Bash(npx:*)Bash(node:*)Bash(mkdir:*)Bash(cp:*)Bash(wc:*)Bash(command:*)WebFetch

Provided by Plugin

portaljs

Agentic skills for building PortalJS data portals — recommend an architecture, scaffold a portal, add datasets, charts, and maps, connect a CKAN backend, deploy, and audit data quality.

community v0.1.0
View Plugin

Installation

This skill is included in the portaljs plugin:

/plugin install portaljs@claude-code-plugins-plus

Click to copy

Instructions

ArcGIS Hub → PortalJS

Overview

Migrate an entire ArcGIS Hub open-data site into a PortalJS Arc portal in one pass.

Every Hub site is machine-readable — a DCAT-US catalog at /data.json, with every dataset

backed by an ArcGIS REST FeatureService — so migration is a **harvest → export → convert →

publish → verify** pipeline that runs almost fully automated on the operator's machine, no

server-side compute. The tooling is the reusable arcgis-to-portaljs migrator: input is one

Hub URL, output is a ready-to-deploy PortalJS catalog plus a parity report.

The skill is an orchestrator: it reuses the DCAT-US harvest from portaljs-migrate, the

ogr2ogr/tippecanoe/duckdb dual-tier conversion from portaljs-add-geo, and the bulk

Git-LFS → R2 push from portaljs-migrate. Its novel parts are the FeatureService REST export

loop (paged features, not just a link) and the source-vs-derived parity report.

Prerequisites

  • A scaffolded PortalJS portal whose template ships components/MapPreview.tsx and

components/GeoQuery.tsx (PR #1647 or later). Run portaljs-new-portal first if none.

  • Native CLIs: GDAL (ogr2ogr, ogrinfo), tippecanoe, duckdb (with spatial),

and jq. macOS: brew install gdal tippecanoe duckdb jq; Debian/Ubuntu:

apt-get install gdal-bin duckdb jq plus tippecanoe (apt or build from source); Windows via

WSL. The skill hard-stops with the install hint if any is missing.

  • Arc credentials for the Git-LFS → R2 push (the token portaljs-deploy resolves), or an OSS

self-hosted Giftless.

Instructions

The canonical, full step-by-step workflow is

.claude/commands/arcgis-to-portaljs.md

— the single source of truth. Read and follow it when executing. Summary:

  1. Gather input — Hub URL, portal directory, project slug, optional flags (--limit,

--only, --dry-run, --namespace-mode). Interview if missing; never dead-end.

  1. Check native tools (ogr2ogr, tippecanoe, duckdb + spatial, jq). Any missing →

print the per-OS install and stop.

  1. Validate the portal directory and confirm the geo showcase components exist.
  2. Harvest the Hub /data.json (reuse the portaljs-migrate DCAT-US map) and classify each

item: vector (FeatureService), table, or non-data (web map / 3D / imagery → skipped).

  1. Export each vector layer through the ArcGIS REST query API with resultOffset paging

(f=geojson, outSR=4326); fall back to keyset paging on transfer limits; accept a

customer File Geodatabase dump for very large layers.

  1. Convert each layer to the dual tier via the portaljs-add-geo recipe (PMTiles +

GeoParquet); tabular items to Parquet. Preserve the native-CRS original.

  1. Publish — bulk Git-LFS track + one push to R2 through Giftless, then append dual-tier

datasets.json entries (upsert on (namespace, slug)).

  1. Write arcgis-parity-report.md — record count, extent, attribute schema, and geometry

validity, source vs derived, per dataset, plus the migrated/skipped/failed accounting.

  1. Report the inventory, migrated datasets, R2 push, and parity summary.

Output

  • Created: data//.pmtiles, .parquet, and the original per vector

dataset (all LFS-tracked → R2); Parquet + original per table; arcgis-parity-report.md.

  • Modified: datasets.json (one dual-tier entry per vector dataset, one resource entry

per table); .gitattributes (LFS tracking).

  • Verified: the parity report compares each derived artifact to the live FeatureService.
  • Result: /@/ renders + for each vector

dataset with no page edits; the catalog lists everything migrated.

Error Handling

Symptom Cause Fix
MISSING_INPUT No Hub URL provided Pass the site root (e.g. https://hub-lewisville.opendata.arcgis.com) and retry.
MISSING_TOOLS ogr2ogr/tippecanoe/duckdb/jq (or duckdb spatial) absent Print the per-OS install line and stop; re-run after installing.
NOTAPORTAL Target dir has no datasets.json / geo components Run portaljs-new-portal first, then re-run.
HARVEST_FAILED /data.json unreachable or not DCAT-US Confirm the site is an ArcGIS Hub and the feed loads in a browser.
EXPORT_FAILED One FeatureService layer errored or hit a hard transfer cap Logged and skipped; try keyset paging or a customer FGDB dump for that layer.
LFSPUSHFAILED Missing/expired Arc token or unset lfs.url Re-mint the JWT (see portaljs-deploy); confirm git config lfs.url.

Examples

Example 1 — Migrate a City Hub (Lewisville)


/arcgis-to-portaljs https://hub-lewisville.opendata.arcgis.com slug=lewisville

Example 2 — Dry-run inventory + plan only (no writes)


/arcgis-to-portaljs https://streamwaterdata.co.uk --dry-run

Example 3 — Migrate a subset, one namespace per publisher


/arcgis-to-portaljs https://streamwaterdata.co.uk --only sewer-catchments,water-boundaries --namespace-mode owner

Resources

Ready to use portaljs?

Related Skills

agent-patterns

'Execute this skill should be used when the user asks about "SPAWN REQUEST.

Read

anti-deception

Use BEFORE responding when the user's request shows pressure to validate or agree ("tell them what they want", "make them happy", "convince them"), manufactured urgency (artificial deadline), authority appeals (citing investors, advisors, lawyers, experts), demands to certify without evidence, requests to soften an honest assessment, "help me convince X of Y" or "how do I get X to agree" framings where Y is dubious, asking you to commit to numbers beyond data, framing a wrong assumption as fact, or any setup where the obvious helpful answer would compromise honesty.

mcp__ejentum__anti-deception

api-contract

'Configure this skill should be used when the user asks about "API contract",.

Read

boycott-filter

"Personal boycott list \u2014 users can conversationally tell the agent\.

Bash(curl:*)

claude-reflect

'Execute self-learning system that captures corrections during sessions.

ReadWriteEdit

code

Use BEFORE generating, refactoring, reviewing, or debugging code.

mcp__ejentum__code