Download RQData¶

download_rqdata is the unified RiceQuant downloader workspace.

It owns license-aware data acquisition, local parquet storage, refresh scripts, and quality checks for RiceQuant-backed data. The main consumer-facing read surface is shadow-ingest, which reads over download_rqdata/data instead of creating a second database.

Responsibility¶

Use download_rqdata when you need to:

• configure and rotate RQData license keys
• download or refresh RiceQuant datasets
• maintain the local parquet data tree
• inspect storage coverage and data quality
• decide whether a RiceQuant data domain belongs in the maintained universe

Do not use download_rqdata to define factors, expose public SDK contracts, or run strategy execution.

Data Scope¶

The maintained scope is driven by direct usefulness to:

• stock research
• stock factors
• stock strategies
• disclosures and corporate actions
• trading liquidity
• necessary index, ETF, futures, or options data for hedge and execution research

Explicitly excluded domains include:

• convertible bonds
• most fund product datasets such as NAV, ranking, rating, fee, subscription/redemption, QDII, bond-fund, money-market-fund, and fund-manager product analytics

The local data/fund_db keeps only the allowed fund subset such as public fund holdings, allocation, ETF baskets, instrument lists, and category membership.

Forbidden API Boundary¶

The repository records forbidden RQData APIs for excluded convertible-bond and fund-product domains. Do not add new refresh jobs, automated maintenance jobs, or downstream dependencies for those APIs unless the business scope changes and the boundary is updated first.

Examples of forbidden families:

• rqdatac.convertible.*
• fund NAV, fee, rating, ranking, manager, transition, transaction status, and structured-fund service APIs

Allowed fund APIs are narrow and limited to holdings, allocation, ETF components, instruments, and category membership needed by stock research.

Relationship To Shadow Ingest¶

download_rqdata/data
  -> shadow-ingest catalog.py
  -> shadow-ingest operations.py
  -> SDK / FastAPI / Flight

download_rqdata is the provider workspace. shadow-ingest is the read-only access layer and public dataframe contract.

The maintained shadow-ingest service runs:

• FastAPI on 30110
• Arrow Flight on 30100

Common Operations¶

License checks:

python cli.py license check
python cli.py license list
python cli.py license rotate

Daily stock data:

python cli.py stock-cn stock
python cli.py stock-cn raw
python cli.py stock-cn adj
python cli.py stock-cn all

Quality checks:

python cli.py quality tick
python cli.py quality tick --start 2024-01-01 --end 2024-12-31

Consumer Boundary¶

Downloader internals, refresh checkpoints, and license handling should not leak into user-facing product code.