 |
SmartSpectra C++ SDK
Measure human vitals from video with SmartSpectra C++ SDK.
|
|
SmartSpectra C++ SDK
Measure human vitals from video with SmartSpectra C++ SDK.
|
Logs from physiology_server can be mirrored to a file. For example, the tee terminal command can be used: physiology_server <... other arguments> 2>&1 | tee path/to/physiology_server.log.
However, if you need to consolidate all the logs on your client application, you can also use the StreamLogs endpoint. Here are some excerpts from the Example Physiology gRPC Client App that demonstrates how to use it:
Here, in handle_logs, we stream logs continuously by iterating over results from the StreamLogs endpoint, and simply print them out with a description prefix and trailing timestamp. To do this, we pass our asynchronous function handle_logs to asyncio.gather along with other awaitables.
Errors are typically unrecoverable and require a full system restart. The error meta-information be provided with every failed gRPC call from your client app and are expected to be handled there, if at all.
The rest of this document takes apart the existing Example Physiology gRPC Client App to help you handle errors in your own Python OnPrem client app.
The following Python example excerpt demonstrates how to gracefully handle connection exceeding a predefined timeout.
Here, asincio.wait_for is used to wait for the channel to be ready within the timeout specified by connection_timeout_s. If the channel is not ready within the timeout, the program exits with an error message.
Note: for more information on Core/Preprocessing breakdown, please take a look at OnPrem Architecture.
Here, we attempt to start Physiology Core, exiting with an error message if it fails:
The grpc.StatusCode.ALREADY_EXISTS code indicates that an instance of Physiology Core is already running in the background, i.e. it was either started manually or the previous shutdown of the program failed to gracefully close the process.
Any other error will manifest something like this in the standard output:
If Core startup succeeds, we attempt to start the preprocessing routine, which is necessary for inputs to Core:
If any error occurs during the Edge/preprocessing startup, you'll see an error message resembling the following:
In this particular instance, we are notified within the grpc_message that we haven't yet started Physiology Core, which Preprocessing/Edge relies on. Hence, for this particular case, the error can easily be fixed by a call to the StartCore endpoint, like we do in the above example. Other errors may not be as easy to handle and may call for submitting a bug report to Presage Technologies.
Finally, the ResetProcessing combines both of the above and also cleans out internal state. Like StartCore and StartProcessing, it is a unary gRPC call, i.e. it returns a single result after it finishes processing. However, note that, in the example Python app, the ResetProcessing call is made from within the handle_keyboard_input function, which is called from frame_capture_loop. frame_capture_loop, in turn, is used as a callback argument to self.physiology_client.AddFrameWithTimestamp inside the asincio.gather call:
Even though the above is itself wrapped in a try-except block, we cannot gather the exceptions from ResetProcessing or other gRPC asynchronous unary endpoint calls made from within our awaitables to asyncio.gather without special adjustments to asyncio.gather arguments, e.g., return_exceptions=True (which would require us to handle the values returned from asyncio.gather as potential exceptions when it stops running, see full documentation here). Instead, we opt for a more straightforward approach and simply wrap each such unary endpoint call with a try-except block, printing any error message and propagating the exception up the stack. For ResetProcessing, this looks as follows:
With this approach, if an error occurs in the call to physiology_server's ResetProcessing, you should see something like:
Note that the "RPC cancelled, exiting." gets printed from the outer try-except block that surround asynchio.gather, which also gracefully handles the shutdown after the exception is raised.
Luckily, errors from streaming gRPC endpoints we from the awaitables to asyncio.gather are much more straightforward to handle. As it stands, if, say, there was an error in the AddFrameWithTimestamp call, we would see something like this get printed:
One caveat is that this error may come combined with some other gRPC error in the output, the latter being caused by inconsistent state from the former. E.g. in the above case, we might also see something like Failed to connect to remote host: Connection refused"</tt>, because the system is retrying the connection while the Physiology Server had already been shut down due to the error. Here is another example error message which shows what an error occurring in Core or Edge could look like: @icode Traceback (most recent call last): File "/home/greg/Builds/physiology/smartspectra/cpp/on_prem/samples/full_example_on_prem_client.py", line 281, in run await asyncio.gather( File "/usr/lib/python3.11/asyncio/tasks.py", line 694, in _wrap_awaitable return (yield from awaitable.__await__()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/greg/Builds/physiology/.venv/lib/python3.11/site-packages/grpc/aio/_call.py", line 318, in __await__ raise _create_rpc_error( grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with: status = StatusCode.INTERNAL details = "Graph has errors: Calculator::Process() for node "MetricsGrpcCalculator" failed: MetricsGrpcCalculator::Process intentional / demo error" debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:50051 {grpc_message:"Graph has errors: \nCalculator::Process() for node \"MetricsGrpcCalculator\" failed: MetricsGrpcCalculator::Process intentional / demo error", grpc_status:13, created_time:"2025-07-14T15:39:07.784646959-04:00"}" > @endicode Here, the statement starting with "Graph has errors:" is the key piece of information. @subsubsection autotoc_md146 Combining Error and Log Information You may have noted that all gRPC errors typically come with very precise timestamps. You can cross-correlate the error information with the logs you get from physiology_server. You can also employ the available logging options (run <tt>physiology_server --help=main</tt> and check options starting with "log") when it makes sense in the context of the issue you're facing.
Unfortunately, if warnings are printed by Core, they don't always include the exact timestamp. This will be addressed in the near future. Edge log statements will always have timestamps.