This appendix covers implementation patterns, recommended architecture, and operational best practices.
A) Integration Options Overview
A1) Scheduled Exports (Bucket-Based)
Pattern: Prism → Cloud Bucket → Your Ingestion Pipeline → Warehouse/BI
Prism generates export files at a defined cadence (e.g., daily)
Files are delivered to your cloud bucket (S3 / Azure Blob / GCS)
Your pipeline loads data into staging tables and transforms downstream
Why teams choose this
Avoids API pagination/retry/rate-limit management
Easy to trigger ingestion from bucket events
Common configuration choices
Format: CSV is typically easiest
Lookback: often “last 30 days” exported daily (supports backfill + late updates)
Schema control:
“Respect selected columns” if you want to prevent schema drift
“Include all columns” if your pipeline can handle evolving schemas
A2) API (Direct Connection)
Pattern: Your Middleware / ETL Job → Prism APIs → Warehouse
Typical use:
GraphQL: list calls, fetch call metadata, fetch computed metrics
REST: fetch transcripts (per-call)
Key behavior to plan for
“Index/list of calls” responses do not include full transcripts due to size
Transcript retrieval is commonly one request per call
B) Authentication and Access
B1) Service account (recommended)
Create a dedicated Prism user:
Name: “API User” / “BI User”
Permission: Company Admin
Ensure access to all locations
Generate an API token for that user and store it securely (vault/secret manager).
Why this matters
Tokens are scoped to the user who creates them
Service accounts avoid disruptions when an employee leaves or changes roles
C) Recommended Architecture (API Ingestion)
C1) Middleware ingestion layer
Most teams implement a small ingestion service that:
Pulls data from Prism (GraphQL + REST)
Handles pagination and retries
Normalizes JSON into tabular schemas
Loads into the warehouse
Destination examples
BigQuery: load raw JSON into staging → dbt transforms to analytics tables
Snowflake/Redshift: similar staging + transform pattern
C2) Data landing strategy
Recommended:
Land raw JSON (immutable/staging)
Transform into curated tables (flattened, typed, indexed)
Build reporting models on curated tables
This approach makes schema changes, debugging, and reprocessing much easier.
D) Data Freshness, Buffering, and Updates
D1) Analysis completion window (important)
Calls/interactions trigger many inferences. Data is often ready quickly, but some fields can finalize later.
Best practice: ingest with a 24-hour buffer
Example: on Jan 12 run, ingest through Jan 11 (not “today”).
D2) Why data can change
Example scenario:
Call initially marked “not booked”
A follow-up converts the patient later that day
Original call record may update to reflect the booking outcome
D3) Lookback strategy
Even with buffering, use a rolling lookback window (commonly 7–30 days) to:
Capture late updates
Ensure data consistency after follow-ups
Simplify incremental loads
E) Pagination, Rate Limits, and Reliability
E1) Pagination
When pulling a list of calls, treat it as a paged/scroll workflow:
Page through results deterministically
Persist checkpoint state (cursor / last timestamp / last ID)
Deduplicate by call ID
E2) Transcript fetching at scale
Transcript retrieval is commonly “per call,” so:
Implement concurrency limits (don’t fire unlimited parallel requests)
Add retry with exponential backoff
Cache already-fetched transcripts by call ID + last-updated timestamp (if available)
E3) Rate limiting
Rate limits exist but are generally liberal. Still implement:
Backoff on 429/5xx
Max retries and dead-letter handling
Circuit breaker behavior for prolonged failures
F) Monitoring, Alerts, and Notifications
F1) Integration monitoring (recommended)
Auth failures (token revoked/expired)
API failures (5xx spikes)
Ingestion pipeline failures (load jobs failing)
Volume anomalies (e.g., “no calls for 2 days”)
Data drift (unexpected schema changes)
F2) Prism technical notifications
Add one or more technical contacts in Prism to receive notifications about:
Integration disruptions
Significant data changes
Potential platform-side issues
G) Implementation Checklist
Scheduled Exports
Bucket created (S3/Blob/GCS)
Credentials generated + added in Prism
Export configured (filters, columns, cadence, lookback, format)
Event trigger or scheduled ingestion job created
Staging + transform models deployed
API
Prism service account created + Company Admin + all locations
API token created and stored securely
GraphQL query patterns defined (calls list, calls detail, metrics)
REST transcript retrieval implemented (per call)
Buffering + rolling lookback implemented
Retry/backoff + monitoring + alerting implemented