Download CapitalIQ¶

download_capitaliq is the paid-API downloader for S&P Capital IQ financial data.

It is a sibling downloader package, separate from shadow-octopus web/raw-source crawling and separate from shadow-factor factor materialization.

Storage Contract¶

The long-term storage contract is split into downloader raw state and a published provider database.

Path	Ownership
`data/ciq_raw_db/`	downloader-owned raw observations, run manifests, request state, sync state
`data/ciq_db/`	shareable provider database for downstream consumers

The first production provider contract is:

ciq_financial_observations.v1

It is stored as:

data/ciq_db/ciq_financial_observations/fiscal_period=<period>/data.parquet

Bitemporal Axes¶

ciq_financial_observations.v1 exposes explicit bitemporal columns:

Column	Meaning
`period_date`	valid time, fiscal-period end
`as_of_date`	provider visibility / transaction time
`ingested_at`	local audit time

period_date is not the visibility date. Production rows require real as_of_date values.

Production Control Plane¶

The daily lane is built around auditable Capital IQ inputRequest rows.

Plan without paid API calls:

uv run download-capitaliq financials input-request-plan \
  --data-root data \
  --latest-safe-as-of-date 2026-06-19 \
  --quota-budget 500000 \
  --json

Select or execute a bounded run:

uv run download-capitaliq financials input-request-run \
  --data-root data \
  --plan-dir data/ciq_raw_db/plans/input_requests \
  --quota-limit 240000 \
  --json

Close out raw responses into the provider database:

uv run download-capitaliq financials input-request-closeout \
  --data-root data \
  --run-id <input_request_run_id> \
  --json

Scheduler-facing daily sync:

uv run download-capitaliq financials daily-sync \
  --data-root data \
  --quota-budget 500000 \
  --max-requests 1000 \
  --json

daily-sync defaults to dry-run. Add --execute only after quota review.

Shadow Factor Boundary¶

download_capitaliq publishes provider observations and audit state only.

The following belong in shadow-factor, not in this downloader:

factor formulas
TTM / YoY / RefQ logic
factor package writes
catalog refreshes
DAG invalidation
Shadow Factor runtime artifacts

storage verify-financial-observations is expected to fail if Shadow Factor runtime directories such as factor=*/, current.json, definition.json, facts/, field=*/, index/, projections/, catalog_cache/, or system/ appear inside data/ciq_db.

Relationship To Other Projects¶

Project	Relationship
`shadow-factor`	consumes provider observations through an adapter and owns factor logic
`shadow-assembly`	may show downstream factor or fundamental views through backend APIs
`shadow-octopus`	unrelated acquisition lane for web/raw-source crawling
`download_rqdata`	sibling provider workspace for RiceQuant data

Keep paid quota planning, request ledgers, raw response artifacts, and provider publishing in download_capitaliq.