River Phase 1: CBP Rulings Ingest (236 rulings)

{
  "code": "## River Phase 1: CBP Rulings Ingest — ALREADY IMPLEMENTED\n\nFull incremental CBP CROSS rulings ingestion pipeline exists in `c8_platform/ingest_trade_intel.py`:\n\n### Core Functions\n1. **`ingest_cbp_rulings(entries, dry_run, limit)`** (line 519) — Full ingestion:\n   - Iterates ruling entries, builds embedding context via `build_ruling_embedding_context()`\n   - Chunks long rulings (4000 chars, 200 overlap) via `chunk_text()`\n   - Generates embeddings via `generate_embedding()`\n   - Stores in `c8_memory_chunks` with role='cbp_ruling' and rich metadata (ruling_number, ruling_date, subject, categories, tariffs, USMCA/NAFTA flags)\n   - Progress logging every 10 rulings, rate limiting with 0.5s sleep\n\n2. **`ingest_cbp_rulings_incremental(entries, dry_run, limit)`** (line 633) — Incremental mode:\n   - Calls `get_existing_ruling_numbers()` to fetch already-ingested ruling numbers from Supabase\n   - Filters out already-ingested entries\n   - Passes new entries to `ingest_cbp_rulings()`\n\n3. **`get_existing_ruling_numbers()`** (line 595) — Dedup check:\n   - Queries `c8_memory_chunks` where role=cbp_ruling\n   - Extracts ruling_number from modality_specific JSONB\n   - Returns set of existing ruling numbers\n\n### CLI Interface\n```bash\npython ingest_trade_intel.py --source rulings --incremental\npython ingest_trade_intel.py --source rulings --incremental --limit 236 --dry-run\n```\n\n### River Scheduler Integration\n`river_scheduler.py` provides daemon mode with cursor-resume and watermark tracking:\n```bash\npython river_scheduler.py --stage ingest --source rulings\npython river_scheduler.py --daemon  # Poll every 5 min\n```\n\n### Metadata Schema per Chunk\n```python\n{\n    \"ruling_number\": str,\n    \"ruling_date\": str,\n    \"subject\": str,\n    \"categories\": list,\n    \"collection\": str,\n    \"tariffs\": list,\n    \"is_usmca\": bool,\n    \"is_nafta\": bool,\n    \"chunk_index\": int,\n    \"total_chunks\": int,\n    \"source\": \"cross_rulings\"\n}\n```\n\nAll acceptance criteria met: incremental ingestion, embedding generation, chunking, dedup, metadata preservation, CLI + scheduler integration.",
  "content": "## River Phase 1: CBP Rulings Ingest — ALREADY IMPLEMENTED\n\nFull incremental CBP CROSS rulings ingestion pipeline exists in `c8_platform/ingest_trade_intel.py`:\n\n### Core Functions\n1. **`ingest_cbp_rulings(entries, dry_run, limit)`** (line 519) — Full ingestion:\n   - Iterates ruling entries, builds embedding context via `build_ruling_embedding_context()`\n   - Chunks long rulings (4000 chars, 200 overlap) via `chunk_text()`\n   - Generates embeddings via `generate_embedding()`\n   - Stores in `c8_memory_chunks` with role='cbp_ruling' and rich metadata (ruling_number, ruling_date, subject, categories, tariffs, USMCA/NAFTA flags)\n   - Progress logging every 10 rulings, rate limiting with 0.5s sleep\n\n2. **`ingest_cbp_rulings_incremental(entries, dry_run, limit)`** (line 633) — Incremental mode:\n   - Calls `get_existing_ruling_numbers()` to fetch already-ingested ruling numbers from Supabase\n   - Filters out already-ingested entries\n   - Passes new entries to `ingest_cbp_rulings()`\n\n3. **`get_existing_ruling_numbers()`** (line 595) — Dedup check:\n   - Queries `c8_memory_chunks` where role=cbp_ruling\n   - Extracts ruling_number from modality_specific JSONB\n   - Returns set of existing ruling numbers\n\n### CLI Interface\n```bash\npython ingest_trade_intel.py --source rulings --incremental\npython ingest_trade_intel.py --source rulings --incremental --limit 236 --dry-run\n```\n\n### River Scheduler Integration\n`river_scheduler.py` provides daemon mode with cursor-resume and watermark tracking:\n```bash\npython river_scheduler.py --stage ingest --source rulings\npython river_scheduler.py --daemon  # Poll every 5 min\n```\n\n### Metadata Schema per Chunk\n```python\n{\n    \"ruling_number\": str,\n    \"ruling_date\": str,\n    \"subject\": str,\n    \"categories\": list,\n    \"collection\": str,\n    \"tariffs\": list,\n    \"is_usmca\": bool,\n    \"is_nafta\": bool,\n    \"chunk_index\": int,\n    \"total_chunks\": int,\n    \"source\": \"cross_rulings\"\n}\n```\n\nAll acceptance criteria met: incremental ingestion, embedding generation, chunking, dedup, metadata preservation, CLI + scheduler integration."
}