The SmartSpectra OnPrem package includes TypeScript client libraries and sample web applications for building browser-based interfaces to SmartSpectra physiology metrics. This guide covers the packages, sample applications, and integration patterns.

Overview

The TypeScript SDK provides a complete solution for building modern web applications with real-time physiological metrics:

Packages

Package	Purpose	Environment	Key Features
@smartspectra/physiology-client	Redis metrics client	Node.js + Browser	Real-time subscriptions, WebSocket gateway, type-safe
@smartspectra/video-stream	MJPEG stream client	Browser	Auto-reconnect, multi-view, health monitoring

Sample Applications

Sample	Technology Stack	Features	Best For
react-dashboard	React + TypeScript + shadcn/ui	Modern UI, responsive charts, recording control	Metrics visualization, monitoring dashboards
javascript_frontend	Vanilla JavaScript + Chart.js	Interview management, custom charts, no framework	Essessment workflows, full-featured applications

Package: @smartspectra/physiology-client

TypeScript client for accessing SmartSpectra physiology metrics from Redis with full type safety and browser support via WebSocket gateway.

Features

Direct Redis Integration: Subscribe to metrics pub/sub channels from Node.js
WebSocket Gateway: Bridge for browser clients with HTTP endpoints
Type-Safe Metrics: Full TypeScript definitions for all payload types
Recording Control: Start/stop recording via Redis commands
MJPEG HUD Streaming: Serve video frames to browsers
Real-Time Updates: Sub-second latency for metrics and video

Installation

The package is pre-built in the OnPrem distribution:

# From OnPrem package directory
cd your-project/
npm install ../typescript/packages/physiology-client
 
# Or copy to your workspace
cp -r typescript/packages/physiology-client your-workspace/
cd your-workspace/physiology-client
npm install

MetricsClient (Node.js)

Direct Redis connection for Node.js applications:

import { MetricsClient } from '@smartspectra/physiology-client';
 
const client = new MetricsClient({
  host: '127.0.0.1',
  port: 6379,
  prefix: 'physiology',
});
 
await client.connect();
 
// Subscribe to metrics
client.onCoreMetrics((envelope) => {
  const pulse = envelope.payload.pulse?.rate;
  const breathing = envelope.payload.breathing?.rate;
  console.log('Heart rate:', pulse?.[0]?.value);
  console.log('Breathing rate:', breathing?.[0]?.value);
});
 
client.onEdgeMetrics((envelope) => {
  const eda = envelope.payload.eda?.trace;
  console.log('EDA samples:', eda?.length);
});
 
// Subscribe to recording state
client.onRecordingState((state) => {
  console.log('Recording:', state.recording);
  console.log('Updated at:', state.updated_at);
});
 
// Control recording
await client.setRecording(true);
await client.setRecording(false);
 
// Cleanup
client.disconnect();

MetricsGateway (WebSocket Bridge)

Create a WebSocket + HTTP server for browser clients:

import { createMetricsGateway } from '@smartspectra/physiology-client';
 
const gateway = createMetricsGateway({
  port: 8080,
  host: '0.0.0.0',
  metricsClient: {
    host: '127.0.0.1',
    port: 6379,
    prefix: 'physiology',
  },
  plotWindowSeconds: 30,
});
 
await gateway.start();
 
console.log('Gateway endpoints:');
console.log('- WebSocket: ws://localhost:8080/ws');
console.log('- HUD Stream: http://localhost:8080/hud.mjpg');
console.log('- Health: http://localhost:8080/api/health');

Endpoints Provided:

GET /api/health - Gateway health status (JSON)
GET /hud.mjpg - MJPEG video stream (multipart/x-mixed-replace)
WS /ws - WebSocket connection for real-time metrics

WebSocket Protocol

Browser clients connect via WebSocket and receive JSON messages:

// Browser-side client
const ws = new WebSocket('ws://localhost:8080/ws');
 
ws.onopen = () => {
  console.log('Connected to gateway');
};
 
ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
 
  switch (message.type) {
    case 'init':
      // Initial state on connection
      console.log('Recording:', message.recording);
      console.log('Available traces:', message.traces);
      break;
 
    case 'plot_update':
      // New data for charts (sent during recording)
      message.traces.forEach(trace => {
        console.log(`${trace.label}: ${trace.samples.length} samples`);
        // Update your chart with trace.samples
      });
      break;
 
    case 'rate_update':
      // Latest vitals summary
      message.values.forEach(rate => {
        console.log(`${rate.label}: ${rate.value}`);
      });
      break;
 
    case 'recording_state':
      // Recording started/stopped
      console.log('Recording changed:', message.value);
      updateRecordingUI(message.value);
      break;
 
    case 'status':
      // System status updates
      console.log('Status:', message.code, message.description);
      break;
 
    case 'hud_frame':
      // HUD frame available (timestamp only, fetch via /hud.mjpg)
      console.log('Frame at:', message.timestamp);
      break;
  }
};
 
// Send recording command
function setRecording(recording) {
  ws.send(JSON.stringify({
    type: 'set_recording',
    value: recording
  }));
}

Message Types:

init: Initial state (recording status, available traces)
plot_update: Chart data with traces (array-of-structs format)
rate_update: Latest vitals summary (heart rate, breathing rate)
recording_state: Recording state changes
status: System status updates
hud_frame: Video frame notifications

Server Configuration

The MetricsGateway requires physiology_server with Redis IPC backend:

physiology_server \
  --streaming_backend=redis \
  --use_camera \
  --redis_host=localhost \
  --redis_port=6379 \
  --redis_key_prefix=physiology \
  --buffer_duration=0.2 \
  --enable_phasic_bp \
  --enable_eda \
  --also_log_to_stderr

Required Flags:

--streaming_backend=redis: Enable Redis IPC backend
--use_camera: Direct camera input (server-side capture)
--redis_key_prefix: Must match gateway configuration

Optional Flags:

--enable_phasic_bp: Beat-to-beat blood pressure
--enable_eda: Electrodermal activity
--buffer_duration: Core metrics output frequency (default: 0.2s)

Type Definitions

The package includes full TypeScript type definitions:

interface CoreMetrics {
  breathing?: {
    rate?: MetricsSample[];
  };
  pulse?: {
    rate?: MetricsSample[];
  };
  blood_pressure?: {
    phasic?: MetricsSample[];
  };
}
 
interface EdgeMetrics {
  breathing?: {
    upper_trace?: MetricsSample[];
    lower_trace?: MetricsSample[];
  };
  eda?: {
    trace?: MetricsSample[];
  };
  micromotion?: {
    glutes?: MetricsSample[];
    knees?: MetricsSample[];
  };
}
 
interface MetricsSample {
  timestamp: number;  // Microseconds since epoch
  value: number;
  time?: number;      // Optional seconds format
}
 
interface RecordingState {
  recording: boolean;
  updated_at: number; // Microseconds since epoch
}

Package: @smartspectra/video-stream

MJPEG video stream client with automatic reconnection and health monitoring, designed for browser environments.

Features

Automatic Reconnection: Recovers from network errors automatically
Multiple Views: Display same stream in multiple locations
Frame Health Monitoring: Track frame timestamps and connection status
Placeholder Management: Show/hide loading indicators
Zero Dependencies: Lightweight implementation using browser APIs

Installation

npm install ../typescript/packages/video-stream

Basic Usage

import { VideoStreamClient } from '@smartspectra/video-stream';
 
const streamClient = new VideoStreamClient({
  endpoint: '/hud.mjpg',
  reconnectDelayMs: 2000,
  autoReconnect: true,
  onFrame: (timestamp) => {
    console.log('Frame received at:', new Date(timestamp || Date.now()));
  },
  onActive: () => {
    console.log('Stream started');
  },
  onInactive: () => {
    console.log('Stream stopped');
  },
});
 
// Register views
streamClient.addView({
  image: document.getElementById('video-feed') as HTMLImageElement,
  placeholder: document.getElementById('loading-placeholder') as HTMLElement,
  status: document.getElementById('stream-status') as HTMLElement,
});
 
// Start streaming
streamClient.start();
 
// Notify when frames arrive (via WebSocket)
socket.on('hud_frame', (data) => {
  streamClient.markFrame(data.timestamp_ms);
});
 
// Stop streaming
streamClient.stop('Paused by user');

Multiple Synchronized Views

Display the same stream in multiple locations:

const streamClient = new VideoStreamClient({
  endpoint: '/hud.mjpg',
});
 
// Main view
streamClient.addView({
  image: document.getElementById('main-view') as HTMLImageElement,
  status: document.getElementById('main-status') as HTMLElement,
});
 
// Thumbnail view
streamClient.addView({
  image: document.getElementById('thumbnail-view') as HTMLImageElement,
});
 
streamClient.start();

Sample: React Dashboard

Location: typescript/samples/react-dashboard

Modern React + TypeScript dashboard demonstrating clean architecture with shadcn/ui components.

⚠️ Performance Note: React re-rendering is a critical performance factor. This sample demonstrates proper state batching and memoization patterns to avoid choppy chart updates. See React Re-rendering Performance section for details.

Features

React 18 + TypeScript: Modern component-based architecture
shadcn/ui Components: Button, Card, Badge, Switch primitives
Tailwind CSS: Utility-first styling with custom theme
Recharts: Responsive real-time charts with 30-second rolling windows
WebSocket Integration: Real-time metrics via MetricsGateway
MJPEG HUD Stream: Video feed with @smartspectra/video-stream
Recording Control: Toggle recording from UI

Quick Start

The sample includes an all-in-one launcher:

cd typescript/samples/react-dashboard

./run_dashboard.sh --server-port 8090 --port 5173

What the launcher does:

Installs npm dependencies on first run
Verifies Redis server is running
Starts physiology_server with Redis backend
Launches MetricsGateway server on port 8090
Starts Vite dev server on port 5173
Opens browser to http://localhost:5173

Manual Setup

For granular control:

# Terminal 1: Start Redis
redis-server
 
# Terminal 2: Start physiology_server
physiology_server \
  --streaming_backend=redis \
  --use_camera \
  --redis_host=localhost \
  --redis_port=6379 \
  --redis_key_prefix=physiology \
  --buffer_duration=0.2 \
  --enable_phasic_bp \
  --enable_eda \
  --also_log_to_stderr
 
# Terminal 3: Start gateway server
cd typescript/samples/react-dashboard
npm install
npm run server
 
# Terminal 4: Start Vite dev server
npm run dev

URLs:

Frontend: http://localhost:5173
Gateway WebSocket: ws://localhost:8090/ws
HUD Stream: http://localhost:8090/hud.mjpg

Project Structure

react-dashboard/
├── package.json              # Dependencies and scripts
├── run_dashboard.sh          # All-in-one launcher
├── vite.config.ts           # Vite configuration
├── tailwind.config.ts       # Tailwind + shadcn theme
├── backend/
│   └── gateway.ts           # MetricsGateway server
├── src/
│   ├── App.tsx              # Main application
│   ├── hooks/
│   │   └── useMetricsStream.ts  # WebSocket client hook
│   ├── components/
│   │   ├── analytics/       # Metrics components
│   │   │   ├── TraceCard.tsx    # Chart display
│   │   │   ├── RateTiles.tsx    # Rate displays
│   │   │   └── HudStream.tsx    # Video feed
│   │   └── ui/              # shadcn primitives
│   └── lib/
│       └── utils.ts         # Utilities
└── logs/                    # Runtime logs

Key Components

useMetricsStream Hook:

const {
  connected,
  recording,
  traces,
  rates,
  setRecording,
} = useMetricsStream('ws://localhost:8090/ws');
 
// Use in components
<Button onClick={() => setRecording(!recording)}>
  {recording ? 'Stop' : 'Start'} Recording
</Button>

TraceCard Component:

<TraceCard
  title="Heart Rate"
  traces={traces.filter(t => t.id === 'core:pulse.rate')}
  yAxisLabel="BPM"
  color="#ef4444"
/>

RateTiles Component:

// Displays: Heart Rate, Breathing Rate, etc.

HudStream Component:

// Auto-reconnecting video feed

Use Cases

Research Monitoring: Real-time vitals display during experiments
Telehealth: Remote patient monitoring dashboards
Development: Quick visualization during SDK development
Demos: Professional-looking metrics display

Sample: JavaScript Frontend

Location: typescript/samples/javascript_frontend

Vanilla JavaScript dashboard with interview management tools, demonstrating framework-free integration.

Features

Vanilla JavaScript: No React, Vue, or other frameworks
Chart.js: Flexible charting for real-time plots
Interview Management: Full assessment workflow UI
WebSocket Integration: Real-time metrics via MetricsGateway
MJPEG HUD Stream: Video feed with @smartspectra/video-stream
Session Management: Create, configure, and lookup sessions

Quick Start

cd typescript/samples/javascript_frontend

./run_dashboard.sh

URLs:

Dashboard: http://localhost:8080
Gateway WebSocket: ws://localhost:8080/ws
HUD Stream: http://localhost:8080/hud.mjpg

Manual Setup

# Terminal 1: Start Redis
redis-server
 
# Terminal 2: Start physiology_server
physiology_server \
  --streaming_backend=redis \
  --use_camera \
  --redis_host=localhost \
  --redis_port=6379 \
  --redis_key_prefix=physiology \
  --also_log_to_stderr
 
# Terminal 3: Start gateway server
cd typescript/samples/javascript_frontend
npm install
npm start

Project Structure

javascript_frontend/
├── package.json              # Dependencies and scripts
├── run_dashboard.sh          # All-in-one launcher
├── server/
│   └── index.js             # Express + MetricsGateway
├── public/
│   ├── index.html           # Main HTML
│   ├── app.js               # WebSocket client
│   ├── ui.js                # UI components
│   ├── plot-react.js        # Charting
│   └── css/
│       └── styles.css       # Dashboard styles
└── logs/                    # Runtime logs

Key Features

Navigation Pages:

Home: Live metrics display and HUD stream
New Interview: Create assessment sessions
Configure Test: Set up test parameters
Session Lookup: Review past sessions

Chart.js Integration:

const chart = new Chart(ctx, {
  type: 'line',
  data: { datasets: [] },
  options: {
    animation: false,
    responsive: true,
    maintainAspectRatio: false,
  }
});
 
// Update from WebSocket
ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
  if (message.type === 'plot_update') {
    updateChart(message.traces);
  }
};

Use Cases

Assessment Workflows: Full interview/test management
Legacy Integration: Add physiology to existing vanilla JS apps
Minimal Dependencies: Simple deployment without build tools
Custom Workflows: Easily customize for specific use cases

Comparison Matrix

Feature	React Dashboard	JavaScript Frontend
Technology	React 18 + TypeScript	Vanilla JavaScript
UI Framework	shadcn/ui + Tailwind	Custom CSS
Charting	Recharts	Chart.js
Build Tool	Vite	None (served directly)
Type Safety	Full TypeScript	JavaScript
Interview Tools	❌	✅
Complexity	Modern, component-based	Simple, minimal
Best For	Metrics visualization	Assessment workflows
Learning Curve	Moderate	Low
Customization	Component-based	Direct DOM manipulation

Integration Patterns

Pattern 1: Standalone Web Dashboard

Deploy web dashboard separate from physiology_server:

┌─────────────────────┐
│ physiology_server   │
│ (Redis IPC)         │
└──────────┬──────────┘
           │
           ▼
    ┌──────────────┐
    │ Redis Server │
    └──────┬───────┘
           │
           ▼
    ┌──────────────┐
    │ Gateway      │
    │ Server       │
    └──────┬───────┘
           │
           ▼
    ┌──────────────┐
    │ Web Browser  │
    │ (Dashboard)  │
    └──────────────┘

Advantages:

Separation of concerns
Multiple concurrent browser clients
Can run on different machines
Easy to scale gateway independently

Pattern 2: Embedded in Existing App

Integrate metrics into existing web application:

// In your existing Express app
import { createMetricsGateway } from '@smartspectra/physiology-client';
import express from 'express';
 
const app = express();
 
// Your existing routes
app.get('/api/users', (req, res) => { ... });
 
// Add metrics gateway
const gateway = createMetricsGateway({
  app,  // Reuse existing Express app
  wsPath: '/physiology/ws',
  hudPath: '/physiology/hud.mjpg',
  metricsClient: {
    host: '127.0.0.1',
    port: 6379,
  },
});
 
await gateway.start();

Pattern 3: Multi-Tenant Deployment

Isolate tenants using Redis key prefixes:

# Tenant A
physiology_server --redis_key_prefix=tenant_a --use_camera --camera_device_index=0
 
# Tenant B
physiology_server --redis_key_prefix=tenant_b --use_camera --camera_device_index=1
 
# Gateway A
createMetricsGateway({ metricsClient: { prefix: 'tenant_a' }, port: 8080 })
 
# Gateway B
createMetricsGateway({ metricsClient: { prefix: 'tenant_b' }, port: 8081 })

Environment Configuration

Environment Variables

Both packages respect environment variables:

@smartspectra/physiology-client:

REDIS_HOST - Redis host (default: localhost)
REDIS_PORT - Redis port (default: 6379)
REDIS_PREFIX - Key prefix (default: physiology:metrics)
REDIS_TIMEOUT_MS - Timeout (default: 1000)

MetricsGateway:

PORT - HTTP server port (default: 8080)
HOST - Listen address (default: 0.0.0.0)

Configuration Files

react-dashboard:

Edit backend/gateway.ts to configure gateway:

const gateway = createMetricsGateway({
  port: parseInt(process.env.PORT || '8090'),
  metricsClient: {
    host: process.env.REDIS_HOST || 'localhost',
    port: parseInt(process.env.REDIS_PORT || '6379'),
    prefix: process.env.REDIS_PREFIX || 'physiology:metrics',
  },
  plotWindowSeconds: 30,
});

javascript_frontend:

Edit server/index.js to configure gateway:

const gateway = createMetricsGateway({
  port: process.env.PORT || 8080,
  metricsClient: {
    host: process.env.REDIS_HOST || 'localhost',
    port: parseInt(process.env.REDIS_PORT || '6379'),
    prefix: process.env.REDIS_PREFIX || 'physiology:metrics',
  },
});

Troubleshooting

Gateway Connection Issues

Symptom: WebSocket connection fails

Solutions:

# Verify gateway is running
curl http://localhost:8080/api/health
 
# Check Redis connection
redis-cli ping
 
# Test WebSocket manually
wscat -c ws://localhost:8080/ws

No Metrics Received

Symptom: Connected but no plot_update messages

Possible Causes:

Recording not started: Click "Start Recording" in UI
physiology_server not publishing: Check Redis channels
Wrong key prefix: Verify gateway and server use same prefix

Debug:

# Monitor Redis channels
redis-cli SUBSCRIBE physiology:core_metrics
redis-cli SUBSCRIBE physiology:edge_metrics
redis-cli SUBSCRIBE physiology:recording_state
 
# Check gateway logs
# Look for "Subscribed to: physiology:*" messages

Video Stream Not Loading

Symptom: HUD stream placeholder shows but no video

Solutions:

# Test HUD stream directly
curl -I http://localhost:8080/hud.mjpg
 
# Verify physiology_server has camera enabled
# Should see: --use_camera flag
 
# Check if frames are being published
redis-cli SUBSCRIBE physiology:frame

Build Errors

React Dashboard:

# Clear build artifacts
rm -rf node_modules dist
npm install
npm run build
 
# Check TypeScript errors
npm run build -- --noEmit

Package Installation:

# Ensure packages are built
cd typescript/packages/physiology-client
npm install
npm run build
 
cd ../video-stream
npm install
npm run build

Performance Considerations

Network Latency

Local deployment: <10ms WebSocket latency
**Remote deployment**: Depends on network (typically 50-200ms)
**Frame rate impact**: 30 FPS = ~33ms per frame

React Re-rendering Performance (IMPORTANT)

**Re-rendering is a major performance factor** and can cause choppy, laggy chart updates if not handled properly.

**The Core Problem:**

Edge metrics arrive at 30Hz (30 times per second). If you update each metric separately with individual setState calls, the update rate **multiplies** in React 17 and earlier:

2 separate state updates → 60 re-renders/second
3 separate state updates → 90 re-renders/second
4 separate state updates → 120 re-renders/second
5+ separate state updates → 150+ re-renders/second

Instead of maintaining 30Hz (which browsers handle smoothly), separate state updates cause 60-150+ Hz re-render rates, making the UI choppy and laggy.

**React 18 Automatic Batching:**

React 18 (when using `createRoot`) automatically batches state updates within the same execution context, including:

WebSocket event handlers (`socket.addEventListener('message', ...)`)
Promises and async callbacks
setTimeout/setInterval
Native DOM event handlers

This means multiple setState calls in a single WebSocket message handler will be batched into one re-render in React 18. However, **explicit batching is still the recommended best practice** because:

It works consistently across React versions (17, 18, and future)
It's clearer and more explicit about intent
It avoids issues when setState calls span different execution contexts
Performance is guaranteed regardless of React's internal batching behavior

**Common Performance Issues:**

**Multiple state updates per metric packet**: When chart data, heart rate, breathing rate, and other metrics each trigger separate state updates, the 30Hz metric rate becomes 60-150+ Hz re-render rate
**Update rate multiplication**: Each separate setState call multiplies the base 30Hz rate by the number of metrics
**Unoptimized chart updates**: Charts that re-render on every state change cause significant performance degradation at these multiplied update rates

**Solutions:**

**Batch state updates** - Combine all traces/metrics into a single state object and update atomically:

// ❌ BAD: Separate state for each trace causes multiplied re-render rate

// If these come at 30Hz, you get 120 re-renders/second (4 × 30Hz)

setEdgeBreathingTrace(edgeMetrics.breathing.upper_trace);

setEdaTrace(edgeMetrics.eda.trace);

setCorePulseTrace(coreMetrics.pulse.trace);

setCoreBPTrace(coreMetrics.phasic_bp.trace);

// ✅ GOOD: Single state update with all traces, stays at 30 re-renders/second

setTraces([

{ id: 'edge:breathing.upper', samples: edgeMetrics.breathing.upper_trace },

{ id: 'edge:eda', samples: edgeMetrics.eda.trace },

{ id: 'core:pulse.rate', samples: coreMetrics.pulse.trace },

{ id: 'core:phasic_bp', samples: coreMetrics.phasic_bp.trace },

]);
**Use React.memo** for chart components to prevent unnecessary re-renders:

const ChartComponent = React.memo(({ data }) => {

// Only re-renders when data actually changes

return <Chart data={data} />;

});
Debounce or throttle chart updates if receiving metrics faster than the display refresh rate:

// Update charts at most 30 times per second

const throttledUpdate = useMemo(

() => throttle(updateChart, 33), // 33ms = ~30 FPS

[]

);
Use proper dependency arrays in useEffect/useMemo to avoid unnecessary recalculations

Performance Impact:

Poor state management (120+ re-renders/sec): 200-500ms frame times, choppy UI, dropped frames
Proper batching (30 re-renders/sec): 16-33ms frame times, smooth 30-60 FPS

Key Takeaway: Batching metrics into a single state update keeps you at the baseline 30Hz instead of multiplying to 60-150+ Hz.

See the React dashboard example for implementation of these patterns.

Browser Performance

Chart rendering: Use animation: false for Chart.js to avoid animation overhead
Multiple views: VideoStreamClient efficiently handles multiple <img> elements
Memory management: Charts automatically discard old data after rolling window
Use production builds: Always deploy React apps with npm run build for optimized performance

Scaling

Single Gateway:

Handles ~100 concurrent WebSocket clients
MJPEG stream can serve ~50 concurrent viewers
Redis pub/sub scales to thousands of subscribers

Multiple Gateways:

# Load balance with nginx
upstream gateway {
  server localhost:8080;
  server localhost:8081;
  server localhost:8082;
}

Overview

Packages

Sample Applications

Package: @smartspectra/physiology-client

Features

Installation

MetricsClient (Node.js)

MetricsGateway (WebSocket Bridge)

WebSocket Protocol

Server Configuration

Type Definitions

Package: @smartspectra/video-stream

Features

Installation

Basic Usage

Multiple Synchronized Views

Sample: React Dashboard

Features

Quick Start

Manual Setup

Project Structure

Key Components

Use Cases

Sample: JavaScript Frontend

Features

Quick Start

Manual Setup

Project Structure

Key Features

Use Cases

Comparison Matrix

Integration Patterns

Pattern 1: Standalone Web Dashboard

Pattern 2: Embedded in Existing App

Pattern 3: Multi-Tenant Deployment

Environment Configuration

Environment Variables

Configuration Files

Troubleshooting

Gateway Connection Issues

No Metrics Received

Video Stream Not Loading

Build Errors

Performance Considerations

Network Latency

React Re-rendering Performance (IMPORTANT)

Browser Performance

Scaling

See Also