Data Types

InsightType

Origin of an Insight, mirroring the request that produced it. Wire values match the request type field accepted by the Presage 3.0 Analytics Gateway: "vitals", "speech", and "combined".

enum InsightType {
  INSIGHT_TYPE_VITALS = 0;
  INSIGHT_TYPE_SPEECH = 1;
  INSIGHT_TYPE_COMBINED = 2;
}

• INSIGHT_TYPE_VITALS -- Auto-fired vitals snapshot dispatched periodically by InsightSession with the accumulated metrics buffer and no caller prompt.
• INSIGHT_TYPE_SPEECH -- On-demand request from RequestInsight with a prompt but no buffered metrics at dispatch time (prompt-only nudge).
• INSIGHT_TYPE_COMBINED -- On-demand request from RequestInsight carrying both the caller's prompt and a non-empty metrics buffer.

Insight

LLM-generated analysis or error returned by the insights endpoint.

Properties

message Insight {
  int32 request_id = 1;
  string processed_at = 2;
  oneof result {
    string analysis = 3;
    string error = 4;
  }
  InsightType type = 5;
}

• int32 request_id -- Client-generated request ID. Populated client-side by the async callback closure (not returned by the server). Lets callers correlate responses to requests.
• string processed_at -- ISO timestamp of when the analysis was produced
• string analysis -- The LLM analysis text, set on success.
• string error -- Error message, set on failure.
• InsightType type -- Origin of this insight. Mirrors the request type that triggered it so callers can route auto-fired vitals updates separately from on-demand speech/combined replies.

FeatureType

FeatureType defines high-level physiological measurement categories. Each feature represents a super-metric that may encompass multiple individual metrics.

enum FeatureType {
  BREATHING = 0;
  MICROMOTION = 1;
  EDA = 2;
  FACE = 3;
  CARDIO = 4;
}

• BREATHING -- Breathing measurements (chest and abdomen)
• MICROMOTION -- Micromotion detection (glutes, knees)
• EDA -- Electrodermal activity (EDA) measurement
• FACE -- Facial tracking and analysis
• CARDIO -- Cardiovascular metrics (pulse, HRV, blood pressure)

MetricType

MetricType defines individual physiological measurements that can be requested and produced during video-based vital signs analysis.

enum MetricType {
  CHEST_BREATHING = 0;
  ABDOMEN_BREATHING = 1;
  BREATHING_RATE = 2;
  BREATHING_AMPLITUDE = 3;
  APNEA = 4;
  RESPIRATORY_LINE_LENGTH = 5;
  BASELINE = 6;
  INHALE_EXHALE_RATIO = 7;
  GLUTES_MICROMOTION = 8;
  KNEES_MICROMOTION = 9;
  EDA_TRACE = 10;
  FACE_LANDMARKS = 11;
  BLINKING = 12;
  TALKING = 13;
  EXPRESSIONS = 14;
  PULSE_RATE = 15;
  ARTERIAL_PRESSURE_TRACE = 16;
  HRV = 17;
}

• CHEST_BREATHING -- Breathing upper (chest) metrics
• ABDOMEN_BREATHING -- Breathing lower (abdomen) metrics
• BREATHING_RATE -- Breathing aggregate metrics
• GLUTES_MICROMOTION -- Micromotion metrics
• EDA_TRACE -- EDA metrics
• FACE_LANDMARKS -- Face metrics
• PULSE_RATE -- Cardio metrics

RequestedMetrics

Wrapper message for passing a list of requested metrics through MediaPipe packets

Properties

message RequestedMetrics {
  repeated MetricType metrics = 1;
}

Measurement

Represents a single measurement with timestamp and stability information. Used for various physiological measurements throughout the system.

Properties

message Measurement {
  float value = 1;
  bool stable = 2;
  int64 timestamp = 3;
}

• float value -- The measured or estimated value
• bool stable -- Whether the measurement is considered stable/reliable
• int64 timestamp -- Absolute timestamp at which the measurement was taken, in microseconds, since Linux epoch

DetectionStatus

Represents detection status with timestamp information. Used to track whether a particular physiological feature or state is detected.

Properties

message DetectionStatus {
  bool detected = 1;
  bool stable = 2;
  int64 timestamp = 3;
}

• bool detected -- Whether the feature/state was detected
• bool stable -- Whether the detection is considered stable/reliable
• int64 timestamp -- Absolute timestamp at which the detection status was updated, in microseconds, since Linux epoch

MeasurementWithConfidence

Represents a measurement with an associated confidence score. Extends basic measurement with confidence information for quality assessment.

Properties

message MeasurementWithConfidence {
  float value = 1;
  bool stable = 2;
  float confidence = 3;
  int64 timestamp = 4;
}

• float value -- The measured value
• bool stable -- Whether the measurement is considered stable/reliable
• float confidence -- Confidence score for the measurement (0.0 to 1.0)
• int64 timestamp -- Absolute timestamp at which the measurement was taken, in microseconds, since Linux epoch

ExpressionType

Enumerates the supported facial expression types output by the model.

enum ExpressionType {
  UNSPECIFIED = 0;
  ANGRY = 1;
  CONTEMPT = 2;
  DISGUST = 3;
  FEAR = 4;
  HAPPY = 5;
  NEUTRAL = 6;
  SAD = 7;
  SURPRISE = 8;
}

• UNSPECIFIED -- Expression is unspecified or unknown
• ANGRY -- Angry expression
• CONTEMPT -- Contempt expression
• DISGUST -- Disgust expression
• FEAR -- Fear expression
• HAPPY -- Happy expression
• NEUTRAL -- Neutral expression
• SAD -- Sad expression
• SURPRISE -- Surprise expression

ExpressionScore

Associates an expression type with its confidence score.

Properties

message ExpressionScore {
  ExpressionType type = 1;
  float confidence = 2;
}

• ExpressionType type -- Expression type identifier
• float confidence -- Confidence score for the expression (0.0 to 1.0)

Expression

Represents a detected expression with metadata. Used for facial expression analysis and emotion detection.

Properties

message Expression {
  bool stable = 1;
  int64 timestamp = 2;
  repeated ExpressionScore scores = 3;
}

• bool stable -- Whether the detection is considered stable/reliable
• int64 timestamp -- Absolute timestamp at which the expression was detected, in microseconds, since Linux epoch
• repeated ExpressionScore scores -- Confidence distribution across all expression types

Hrv

Represents a single HRV measurement computed over some period of pulse data.

Properties

message Hrv {
  double rmssd = 1;
  double mean_nn = 2;
  double sdnn = 3;
  double baevsky = 4;
  int64 timestamp = 5;
  float confidence = 6;
}

• double rmssd -- root mean square of successive differences between normal heartbeats
• double mean_nn -- mean normal-to-normal (NN) interval length
• double sdnn -- Standard Deviation of normal-to-normal (NN) Intervals
• double baevsky -- Baevsky's Stress Index, in 1/ms^2 (inverse-square milliseconds)

Strict

Container for strict/exact values that require high precision. Used when measurements need to be treated with special precision requirements.

Properties

message Strict {
  float value = 1;
}

• float value -- The strict value requiring high precision

Pulse

Comprehensive pulse-related measurements and derived metrics. Contains heart rate, pulse trace, and respiratory coupling information.

Properties

message Pulse {
  repeated MeasurementWithConfidence rate = 1;
  repeated Measurement trace = 2;
  repeated Measurement pulse_respiration_quotient = 3;
  Strict strict = 4;
}

• repeated MeasurementWithConfidence rate -- Heart rate measurements with confidence scores
• repeated Measurement trace -- Raw pulse trace measurements
• repeated Measurement pulse_respiration_quotient -- Pulse-respiration quotient measurements indicating cardio-respiratory coupling
• Strict strict -- Strict/high-precision pulse measurements over a fixed time interval. Populated when strict mode analysis is enabled.

Breathing

Comprehensive breathing/respiratory measurements and derived metrics. Contains respiratory rate, traces, and various breathing pattern indicators.

Properties

message Breathing {
  repeated MeasurementWithConfidence rate = 1;
  repeated Measurement upper_trace = 2;
  repeated Measurement lower_trace = 3;
  repeated Measurement amplitude = 4;
  repeated DetectionStatus apnea = 5;
  repeated Measurement respiratory_line_length = 6;
  repeated Measurement baseline = 7;
  repeated Measurement inhale_exhale_ratio = 8;
  Strict strict = 9;
}

• repeated MeasurementWithConfidence rate -- Respiratory rate measurements with confidence scores
• repeated Measurement upper_trace -- Chest breathing movement trace measurements
• repeated Measurement lower_trace -- Abdominal breathing movement trace measurements
• repeated Measurement amplitude -- Breathing amplitude measurements
• repeated DetectionStatus apnea -- Apnea (breathing cessation) detection status
• repeated Measurement respiratory_line_length -- Respiratory line length measurements for breathing pattern analysis
• repeated Measurement baseline -- Baseline breathing measurements
• repeated Measurement inhale_exhale_ratio -- Inhale to exhale duration ratio measurements
• Strict strict -- Strict/high-precision breathing measurements over a fixed time interval. Populated when strict mode analysis is enabled.

Landmarks

Facial landmark coordinates with temporal and stability information. Used for face tracking and facial feature analysis.

Properties

message Landmarks {
  repeated Point2dFloat value = 1;
  bool stable = 2;
  bool reset = 3;
  int64 timestamp = 4;
}

• repeated Point2dFloat value -- Array of 2D coordinate points representing facial landmarks
• bool stable -- Whether the landmark detection is considered stable/reliable
• bool reset -- Indicates whether the landmark set was reset (cannot be directly associated with previous set)
• int64 timestamp -- Absolute timestamp at which the landmarks were detected, in microseconds, since Linux epoch

Face

Comprehensive facial analysis measurements and detections. Contains blinking, talking detection, landmarks, and expressions.

Properties

message Face {
  repeated DetectionStatus blinking = 1;
  repeated DetectionStatus talking = 2;
  repeated Landmarks landmarks = 3;
  repeated Expression expression = 4;
}

• repeated DetectionStatus blinking -- Blinking detection status over time
• repeated DetectionStatus talking -- Talking/speech detection status over time
• repeated Landmarks landmarks -- Facial landmark coordinates over time
• repeated Expression expression -- Detected expressions over time

MicroMotion

Micro-motion measurements for body movement analysis. Tracks small movements in specific body regions.

Properties

message MicroMotion {
  repeated Measurement glutes = 1;
  repeated Measurement knees = 2;
}

• repeated Measurement glutes -- Micro-motion measurements in the glutes region
• repeated Measurement knees -- Micro-motion measurements in the knees region

Eda

Electrodermal Activity (EDA) measurements. Tracks skin conductance changes related to autonomic nervous system activity.

Properties

message Eda {
  repeated Measurement trace = 1;
}

• repeated Measurement trace -- EDA trace measurements over time

Cardio

Properties

message Cardio {
  repeated MeasurementWithConfidence pulse_rate = 1;
  repeated MeasurementWithConfidence arterial_pressure_trace = 2;
  repeated Hrv hrv = 3;
}

• repeated MeasurementWithConfidence pulse_rate -- Heart rate measurements with confidence scores
• repeated MeasurementWithConfidence arterial_pressure_trace -- Arterial pressure trace (uncalibrated, unitless) measurements with confidence scores
• repeated Hrv hrv -- Heart rate variability measurements with confidence scores

Metrics

Comprehensive physiological metrics container. Contains all available physiological measurements and analysis results.

Properties

message Metrics {
  Breathing breathing = 1;
  MicroMotion micromotion = 2;
  Eda eda = 3;
  Face face = 4;
  Cardio cardio = 5;
}

• Breathing breathing -- Breathing and respiratory analysis results
• MicroMotion micromotion -- Micro-motion analysis results
• Eda eda -- Electrodermal activity measurements. Note: processing needs to run for over 35 seconds to generate the first EDA result.
• Face face -- Facial analysis results
• Cardio cardio -- Cardiovascular measurements (pulse rate, arterial pressure, HRV)

Point2dInt32

Represents a 2D point with integer coordinates. Used for pixel-based coordinate systems and discrete positioning.

Properties

message Point2dInt32 {
  int32 x = 1;
  int32 y = 2;
}

• int32 x -- X coordinate as 32-bit signed integer
• int32 y -- Y coordinate as 32-bit signed integer

Point2dFloat

Represents a 2D point with floating-point coordinates. Used for precise positioning, normalized coordinates, and sub-pixel accuracy.

Properties

message Point2dFloat {
  float x = 1;
  float y = 2;
}

• float x -- X coordinate as floating-point value
• float y -- Y coordinate as floating-point value

Point3dFloat

Represents a 3D point with floating-point coordinates. Used for spatial positioning, depth information, and 3D landmark representation.

Properties

message Point3dFloat {
  float x = 1;
  float y = 2;
  float z = 3;
}

• float x -- X coordinate as floating-point value
• float y -- Y coordinate as floating-point value
• float z -- Z coordinate (depth) as floating-point value

Trace

Internal-only message — not part of the public SDK surface. Used by edge graph calculators to pass time-series samples between processing stages. Public outputs use repeated Measurement directly on the Pulse / Breathing / Eda / Cardio messages, so this type is never serialized to SDK consumers.

Properties

message Trace {
  repeated Measurement sample = 1;
}