Market Discovery Phase 3: API Clients

This post covers Phase 3 of ADR-017 - implementing the API clients that fetch market listings from Polymarket and Kalshi for automated discovery.

The Problem

Phase 1 and 2 established storage and matching. Now we need data sources:

1. Polymarket - Gamma API at gamma-api.polymarket.com
2. Kalshi - Trade API at api.elections.kalshi.com/trade-api/v2

Both APIs have: - Pagination (different styles) - Rate limits (different thresholds) - Different response schemas

Design: DiscoveryClient Trait

We define a common trait for both platforms:

#[async_trait]
pub trait DiscoveryClient: Send + Sync {
    async fn list_markets(
        &self,
        limit: Option<u32>,
        cursor: Option<&str>,
    ) -> Result<DiscoveryPage, DiscoveryError>;

    fn platform_name(&self) -> &'static str;
}

This allows the scanner (Phase 4) to enumerate markets from either platform interchangeably.

Rate Limiting

Both APIs have rate limits we must respect:

Platform	Limit	Implementation
Polymarket	60 req/min	Token bucket
Kalshi	100 req/min	Token bucket

We implement a token bucket rate limiter:

struct RateLimiter {
    tokens: AtomicU64,
    last_refill: Mutex<Instant>,
    max_tokens: u32,
}

impl RateLimiter {
    async fn acquire(&self) -> Option<Duration> {
        // Refill tokens based on elapsed time
        let elapsed = last.elapsed();
        let refill = (elapsed.as_secs_f64() / 60.0 * max_tokens) as u64;

        // Try to consume a token
        if tokens > 0 {
            tokens -= 1;
            return None; // Success
        }

        // Return wait time
        Some(Duration::from_secs_f64(60.0 / max_tokens))
    }
}

If rate limited, we return DiscoveryError::RateLimited with the retry time.

Pagination Strategies

Polymarket: Offset-based

GET /markets?limit=100&offset=0
GET /markets?limit=100&offset=100
...

We use the offset as the cursor, incrementing by page size.

Kalshi: Cursor-based

GET /markets?limit=100&status=open
→ { markets: [...], cursor: "abc123" }

GET /markets?limit=100&cursor=abc123
→ { markets: [...], cursor: null }

We pass through the cursor directly.

Response Mapping

Each API returns different schemas that we map to DiscoveredMarket:

Polymarket Gamma API

struct GammaMarket {
    condition_id: String,    // → platform_id
    question: String,        // → title
    outcomes: String,        // JSON array → outcomes
    end_date: String,        // → expiration
    volume_24hr: f64,        // → volume_24h
    active: bool,            // Filter: skip if false
    closed: bool,            // Filter: skip if true
}

Kalshi Markets API

struct KalshiMarket {
    ticker: String,          // → platform_id
    title: String,           // → title
    expiration_time: String, // → expiration
    volume_24h: i64,         // Cents → dollars
    status: String,          // Filter: only "open"/"active"
}

Key transformations: - Kalshi volume is in cents, converted to dollars (/ 100.0) - Inactive/closed markets are filtered out before returning - Missing fields use sensible defaults

Error Handling

pub enum DiscoveryError {
    Http(reqwest::Error),           // Network failures
    Parse(String),                  // JSON parsing
    RateLimited { retry_after_secs: u64 },  // 429 responses
    ApiError { status: u16, message: String },  // Other HTTP errors
}

The scanner (Phase 4) can handle these appropriately - retrying on rate limits, logging API errors.

Test Strategy

We use wiremock for HTTP mocking:

#[tokio::test]
async fn test_list_markets_success() {
    let mock_server = MockServer::start().await;

    Mock::given(method("GET"))
        .and(path("/markets"))
        .respond_with(ResponseTemplate::new(200)
            .set_body_json(mock_response()))
        .mount(&mock_server)
        .await;

    let client = GammaApiClient::with_base_url(&mock_server.uri());
    let page = client.list_markets(Some(10), None).await.unwrap();

    assert_eq!(page.markets.len(), 2);
}

Test Coverage

Phase 3 adds 8 tests (30 total for discovery):

Module	Tests	Focus
polymarket_gamma.rs	4	Success, pagination, rate limit, mapping
kalshi_markets.rs	4	Success, cursor pagination, rate limit, mapping

What's Next

Phase 4 will implement the scanner and approval workflow:

• DiscoveryScannerActor for periodic discovery runs
• ApprovalWorkflow for human review (FR-MD-003)
• Integration with MappingManager.verify_mapping()

Council Review

Phase 3 passed council verification with confidence 0.87. Key findings:

• No unsafe code
• Proper rate limiting prevents API abuse
• 30-second timeout prevents hanging
• No credentials hardcoded
• Closed/inactive markets filtered out

---

Implementation: arbiter-engine/src/market/discovery_client/ | Issues: #44, #45 | ADR: 017