Market Discovery Phase 4: Scanner & Approval Workflow

This post covers Phase 4 of ADR-017 - the scanner actor for periodic discovery and the safety-critical human approval workflow.

The Problem

Phase 1-3 established storage, matching, and API clients. Now we need:

1. Automated Discovery - Periodic scanning of both platforms
2. Human Approval - Safety gate preventing automated mappings from entering trading

This phase implements FR-MD-003 (human confirmation required) and FR-MD-004 (auto-discover markets).

Safety-First Design

FR-MD-003 is SAFETY CRITICAL. The approval workflow enforces:

1. Warning Acknowledgment - Cannot approve candidates with semantic warnings without explicit acknowledgment
2. Audit Logging - All decisions logged with full context
3. MappingManager Integration - Approved candidates go through existing safety gate

pub fn approve(&self, id: Uuid, acknowledge_warnings: bool) -> Result<Uuid, ApprovalError> {
    let candidate = self.get_candidate(id)?;

    // SAFETY CHECK: Require warning acknowledgment if warnings exist
    if !candidate.semantic_warnings.is_empty() && !acknowledge_warnings {
        return Err(ApprovalError::WarningsNotAcknowledged);
    }

    // Create verified mapping through the existing safety gate
    let mapping_id = {
        let mut manager = self.mapping_manager.lock().unwrap();
        let id = manager.propose_mapping(/*...*/);
        manager.verify_mapping(id);  // MappingManager safety gate
        id
    };

    // Update status and log decision
    // ...
}

Scanner Actor

The DiscoveryScannerActor implements the Actor trait for periodic discovery:

pub enum ScannerMsg {
    Scan,           // Trigger a scan
    ForceRefresh,   // Ignore cache
    Stop,           // Graceful shutdown
    GetStatus(tx),  // Query status
}

Deduplication

The scanner prevents duplicate candidates:

async fn is_duplicate_candidate(&self, candidate: &CandidateMatch) -> Result<bool, ScanError> {
    let storage = self.storage.lock().await;

    // Check pending candidates
    let pending = storage.query_candidates_by_status(CandidateStatus::Pending)?;
    for existing in pending {
        if existing.polymarket.platform_id == candidate.polymarket.platform_id
            && existing.kalshi.platform_id == candidate.kalshi.platform_id
        {
            return Ok(true);
        }
    }

    // Also check approved candidates
    let approved = storage.query_candidates_by_status(CandidateStatus::Approved)?;
    // ...
}

Scan Flow

1. Fetch markets from Polymarket (with pagination)
2. Fetch markets from Kalshi (with cursor pagination)
3. Store all markets in SQLite
4. Run similarity matching
5. Deduplicate against existing candidates
6. Store new candidates with Pending status

Approval Workflow

The ApprovalWorkflow provides the human interface:

// List candidates awaiting review
let pending = workflow.list_pending()?;

// Approve (must acknowledge warnings if present)
let mapping_id = workflow.approve(candidate_id, true)?;

// Reject with reason (required)
workflow.reject(candidate_id, "Different settlement criteria")?;

Rejection Requires Reason

To maintain audit trail quality, rejections require a non-empty reason:

pub fn reject(&self, id: Uuid, reason: &str) -> Result<(), ApprovalError> {
    if reason.trim().is_empty() {
        return Err(ApprovalError::ReasonRequired);
    }
    // ...
}

Audit Trail

Every decision is logged with full context:

let entry = AuditLogEntry {
    timestamp: Utc::now(),
    action: AuditAction::Approve,  // or Reject
    candidate_id: id,
    polymarket_id: candidate.polymarket.platform_id.clone(),
    kalshi_id: candidate.kalshi.platform_id.clone(),
    similarity_score: candidate.similarity_score,
    semantic_warnings: candidate.semantic_warnings.clone(),
    acknowledged_warnings: acknowledge_warnings,
    reason: None,  // or Some("...") for rejections
    session_id: self.session_id.clone(),
};
storage.append_audit_log(&entry)?;

Test Coverage

Phase 4 adds 10 tests (40 total for discovery):

Module	Tests	Focus
scanner.rs	5	Finding candidates, deduplication, threshold, storage, graceful stop
approval.rs	5	List pending, approve w/o warnings, warning acknowledgment, reject, verified mapping

Critical Safety Test

#[test]
fn test_approve_requires_warning_acknowledgment() {
    // Add candidate WITH warnings
    let candidate = setup_candidate(&storage, true);
    let workflow = ApprovalWorkflow::new(storage, mapping_manager);

    // Try to approve WITHOUT acknowledging warnings - MUST FAIL
    let result = workflow.approve(candidate.id, false);
    assert!(result.is_err(), "SAFETY VIOLATION: Should require warning acknowledgment");

    match result {
        Err(ApprovalError::WarningsNotAcknowledged) => {
            // Correct error type
        }
        Ok(_) => panic!("SAFETY VIOLATION: Approved without acknowledging warnings!"),
        // ...
    }
}

What's Next

Phase 5 will implement CLI integration:

• --discover-markets - Trigger discovery scan
• --list-candidates - List pending/approved/rejected
• --approve-candidates - Approve by ID
• --reject-candidates - Reject with reason

Council Review

Phase 4 passed council verification with confidence 0.91 (Safety focus). Key findings:

• FR-MD-003 enforcement verified
• Warning acknowledgment required
• Audit logging on all decisions
• Integration with MappingManager.verify_mapping() confirmed
• Deduplication prevents duplicate reviews

---

Implementation: arbiter-engine/src/discovery/{scanner,approval}.rs | Issues: #46, #47 | ADR: 017