Shadow Lighthouse HTTP API¶

The HTTP service exposes source-local and multi-source read surfaces over local Lighthouse indexes.

Start the multi-source server:

uv run shadow-lighthouse --config config.example.toml serve \
  --host 127.0.0.1 \
  --port 8766

Start a single-source server:

uv run shadow-lighthouse --config config.example.toml serve \
  --source cninfo_announcements \
  --host 127.0.0.1 \
  --port 8766

Multi-Source Endpoints¶

Endpoint	Purpose
`GET /health`	Service health and initialized source names
`GET /sources`	Source summaries, index presence, and document counts
`GET /documents`	Merged reverse-time document stream
`GET /documents/search-page`	Paginated document search for application browsing
`GET /documents/{id}`	Find a document across sources
`GET /objects/{sha256}`	Find documents referencing one object
`GET /search?q=...`	Fan-out FTS search across source-local indexes
`GET /news/search?q=...`	Search the global news index
`GET /documents/search?q=...`	Search the Meilisearch-backed document index when configured
`GET /tables/search?q=...`	Fan-out structured table search
`GET /document-evidence/status`	Document evidence store status
`GET /document-evidence/search?q=...`	Search externally submitted document evidence
`POST /document-evidence/targets/lease`	Lease document evidence work for local workers
`POST /document-evidence/results`	Submit parser/OCR/evidence results

GET /documents accepts:

Parameter	Meaning
`source`	Optional repeated source filter
`start`	Inclusive lower bound for `published_at`
`end`	Inclusive upper bound for `published_at`
`limit`	Result limit, default `50`, max `500`

Example:

curl "http://127.0.0.1:8766/documents?source=cninfo_announcements&limit=24"

Source-Scoped Endpoints¶

All source-local surfaces are also available under /sources/{source}:

Endpoint	Purpose
`GET /sources/{source}/status`	Source status
`GET /sources/{source}/documents`	Source-local reverse-time documents
`GET /sources/{source}/documents/{id}`	Source-local document lookup
`GET /sources/{source}/objects/{sha256}`	Source-local object lookup
`GET /sources/{source}/objects/{sha256}/file`	Serve original object bytes
`GET /sources/{source}/issuers/{query}/documents`	Documents for issuer name/code/id
`GET /sources/{source}/issuer-counts`	Per-issuer document/object counts
`GET /sources/{source}/search?q=...`	Source-local FTS search
`GET /sources/{source}/tables/search?q=...`	Source-local table row search
`GET /sources/{source}/documents/{id}/artifacts`	List derived artifacts for a document
`GET /sources/{source}/documents/{id}/evidence`	List document evidence runs and items
`GET /sources/{source}/documents/{id}/research-catalog`	Product-safe research report metadata/catalog
`POST /sources/{source}/documents/{id}/pdf-download-request`	Request PDF materialization
`GET /sources/{source}/documents/{id}/pdf-download-status`	Check PDF availability
`GET /sources/{source}/documents/{id}/pdf-viewer`	Product-safe PDF viewer manifest

Single-Source Endpoints¶

When serve --source <name> is used, the source prefix is omitted:

Endpoint	Purpose
`GET /status`	Source status
`GET /documents`	Source-local documents
`GET /documents/{id}`	Document lookup
`GET /objects/{sha256}`	Object lookup
`GET /objects/{sha256}/file`	Serve original object bytes
`GET /issuers/{query}/documents`	Documents for issuer name/code/id
`GET /issuer-counts`	Per-issuer counts
`GET /search?q=...`	FTS search
`GET /tables/search?q=...`	Table row search

Document IDs¶

Lighthouse accepts raw source ids such as:

announcement:1225267495

It also accepts AI-facing ids for CNInfo documents:

cninfo:announcement:1225267495

Multi-source document lookup returns a list because the caller may not know which source owns the id. Source-local lookup returns one document object.

Document Search Page¶

GET /documents/search-page is the application-facing document browsing endpoint.

Common parameters:

Parameter	Meaning
`source`	Optional repeated source filter
`q`	Text query
`start`, `end`	Published-time bounds
`record_type`	Source record type, such as `research_report`
`pdf_only`	Require a PDF object
`pdf_availability`	PDF state filter
`broker`, `category`, `industry`	Research report metadata filters
`security_code`, `stock_code`	Issuer/security filters
`has_catalog`	Require explicit research catalog metadata
`recommendation_bucket`	Research label bucket when labels are indexed
`min_value_score`	Minimum research value score
`limit`, `offset`	Pagination

Example:

curl "http://127.0.0.1:8766/documents/search-page?source=gangtise_foreign_reports&record_type=research_report&limit=20"

Research metadata-first browsing should not require pdf_only=true. A report can be visible before its PDF bytes are materialized.

PDF Request Flow¶

For metadata-first research reports, Lighthouse records product-safe PDF requests and availability state. Octopus still performs the actual object download/materialization.

Request a PDF:

curl -X POST \
  "http://127.0.0.1:8766/sources/gangtise_foreign_reports/documents/gangtise_report%3A454901220168568832/pdf-download-request"

Poll status:

curl \
  "http://127.0.0.1:8766/sources/gangtise_foreign_reports/documents/gangtise_report%3A454901220168568832/pdf-download-status"

Read the viewer manifest after availability:

curl \
  "http://127.0.0.1:8766/sources/gangtise_foreign_reports/documents/gangtise_report%3A454901220168568832/pdf-viewer"

The response exposes browser-safe file/viewer paths. Lower-level operator CLI output may still include local filesystem paths.

Artifact Content¶

Derived artifacts can be listed and fetched through document routes:

curl "http://127.0.0.1:8766/sources/manual_media/documents/manual:call:1/artifacts"
curl "http://127.0.0.1:8766/sources/manual_media/documents/manual:call:1/artifacts/transcript/content"

Use these routes for OCR text, transcripts, extracted table JSON, and other derived content written by Lighthouse extractors.

Document Evidence¶

Lighthouse stores externally produced document evidence, but does not parse PDFs itself.

Worker-facing flow:

POST /document-evidence/targets
POST /document-evidence/targets/lease
POST /document-evidence/results
GET  /document-evidence/runs
GET  /document-evidence/search?q=...
GET  /sources/{source}/documents/{source_record_id}/evidence

POST /document-evidence/results is idempotent by idempotency_key within a source. Replaying the same payload returns the stored run; reusing the same key with a different payload returns 409.