2.2.8. Media Server Services

In addition to the core feed management features of a media server session per Media Server Session, the media server session also provides supplementary “services”, e.g. managing remote recordings for the media session.

2.2.8.1. Media server services connection

Media server services are owned and managed by the main MediaServerSession object. Services are automatically connected when the main session object itself conncts to the media session.

Note that the services connection is independent of the main media session connection. For this reason, services connect and activate shortly after the main session connection is established. Also, services may disconnect independently of the main session connection (e.g. due to poor network connectivity).

There are two events emitted by the media server session to notify when services are connected or disconnected:

C++Python

// Notify when the session services are connected
mediaSession->onSessionServicesConnected(
    [](const auto& session)
    {
        // Session services are now connected and available...
    });

// Notify when the session services disconnect, possibly in error
mediaSession->onSessionServicesDisconnected(
    [](const auto& session, error_code error)
    {
        if (error) {
            // Handle error disconnecting...
        } else {
            // Session services have disconnected gracefully...
        }
    });

Because services are supplementary to the core media session functionality, service disconnection is not considered a critical error by the SDK, and it will continue to attempt reconnection automatically. Your application may consider service disconnection as an error condition that needs to be handled according to your needs (e.g. closing the session if services are disconnected for too long).

Finally, you can also check if services are currently connected directly:

C++Python

if (mediaSession->areSessionServicesConnected()) {
    // Session services are currently connected...
}

2.2.8.2. Remote recording service

The remote recording feature offers the media session recording functionality available in the main Proximie Live App web application’s session interface:

• Monitor and query the status of the session recording

• Start/stop remote recordings of the media session on the media server

• Report on recording duration, including calculating live time when actively recording

If your application needs to support remote recording, there are a few steps to follow.

Firstly, you should set up a onRecordingStateChanged() callback for when the recording state changes. This callback will be invoked whenever the recording status changes, and should be considered the “source of truth” for the recording status. Even when the application requests a recording status change, the actual status could be overwritten by other participants in the session, or poor network conditions could cause delays or failures in processing the request.

C++Python

// Notify when the remote recording state changes
mediaSession->onRecordingStateChanged(
    [](const auto& session, const auto& state)
    {
        // Handle recording state change in state...
        // e.g. state.on indicates if recording is active
    });

The state passed to the callback is a MediaServerSession::SessionRecordingState value object, which contains information about the current recording status.

The on member value indicates if recording is currently active or not. You can call the state’s totalRunningDurationMs() method to find the total recorded duration in milliseconds.

To request a change in the recording status, use the requestSessionRecording() method. This method sends a request to the media server to start or stop recording the session. You should check the result returned from the call to confirm the request was accepted.

C++Python

// Make a request to activate recording
auto requested = mediaSession->requestSessionRecording(true);
if (!requested) {
    // Request failed...
    auto error = requested.error();
}

As noted above, the actual recording status will be reported via the recording state callback and you should not assume that the recording reflects your requested state even if the request was accepted. You can query the current recording state at any time using the sessionRecordingState() method.

C++Python

auto recordingState = mediaSession->sessionRecordingState();
if (recordingState.updatePending) {
    // A recording state update is currently pending...
}

If you have made a request but not received a confirmed state via the callback, the state returned by sessionRecordingState() will have its updatePending member set to true. This indicates that the state is still pending confirmation from the media server. Importantly, the state’s on flag only ever reflects the last confirmed recording status, and is not changed when the request is sent. Therefore, your application should take care in reporting recording status to users until the pending state is resolved.

If updatePending is true

• A request has been made but is not yet confirmed,

• The current on value does not reflect the request, just the last confirmed status.

If updatePending is false:

• No requests are pending, (though of course other participants may still change the status),

• The current on value reflects the confirmed recording status.

Applications that want to show a “live” recording duration can do so by periodically checking the recording state and calling the totalRunningDurationMs() method. When recording is active, this method will return the total recorded duration including the current live time.

Important

In this version of the SDK, recording sessions must have an audio feed present in order for recordings to be correctly processed. This is a requirement of the underlying recording service, but this limitation will be removed in a forthcoming SDK release.

If your application has no requirement for audio but still needs to record video, you can use special audio components AudioInputComponentSilence (for a silent input) and AudioOutputComponentNull (for a null output) to satisfy this requirement. The recording sample demonstrates this approach - see Running samples for more information.

2.2.8.3. Annotation/Telestration service (Experimental)

Warning

Experimental feature — the annotation/telestration API is experimental in this release. It may change, have known issues, or be removed in a future release without notice. Feedback is welcome.

The annotation service allows your application to receive real-time annotation (telestration) events for streams in a media session. Annotations include drawing commands such as drawPath, erasePath, clearAll, and cursor movement events.

Event types

Command	Description	Persistent	Frequency
drawPath	A completed stroke (freehand line). Contains full path coordinates in params.	Yes	Once per stroke
erasePath	Removes a previously drawn stroke identified by strokeId.	Yes	Once per erase
clearAll	Clears all annotations on the stream. Destructively removes all history for the stream.	No	Manual trigger
moveCursor	Cursor position update (no drawing). High-frequency while the user moves the pointer.	No	Many per second
drawSegment	Intermediate drawing segment while a stroke is in progress (before drawPath).	No	Many per second

Note

Persistent events are saved to the annotation history and can be retrieved via experimental().fetchAnnotationHistory. Non-persistent events are only delivered live. clearAll destructively deletes all prior history for the stream rather than being stored as a history entry itself.

Event tuple format

Each annotation event is delivered as a JSON array (tuple) with the following structure:

[seqNum, profileId, details, timestamp, userId]

Index	Field	Description
0	seqNum	Integer sequence number, unique per event within a stream. Used for deduplication.
1	profileId	Identifier of the media server profile that generated the event (treat as an opaque JSON value).
2	details	JSON object containing command, params, and streamId (see below).
3	timestamp	Unix timestamp in milliseconds when the event was created.
4	userId	UUID string identifying the user who performed the annotation.

The details object at index 2 contains:

Field	Description
command	The event type string (drawPath, erasePath, clearAll, moveCursor, drawSegment).
streamId	The stream (feed) this annotation applies to.
params	Command-specific parameters (e.g. strokeId, path coordinates, cursor position).

The params object varies by command:

• drawPath / drawSegment: Contains strokeId (string), color (hex string), size (line width), and points (array of {x, y} coordinates in 1280x720 space).

• erasePath: Contains strokeId (string) identifying the stroke to remove.

• moveCursor: Contains x and y coordinates of the cursor position.

• clearAll: No params (or empty object).

Note

Coordinates are relative to a 1280x720 reference resolution. Applications must scale to their displayed video dimensions.

There are two complementary ways to receive annotation data:

• Live events — subscribe to real-time annotation events delivered via the session’s Socket.IO connection

• History fetch — retrieve persisted annotation events via REST for events that occurred before subscribing

Both methods deliver events in the same tuple format, making it straightforward to merge live and historical data in your application.

Subscribing to live events

Because this feature is experimental, all annotation methods are accessed via mediaSession->experimental() rather than directly on the session object.

To receive live annotation events, subscribe to the stream(s) you are interested in. Subscription is local state and can be called at any time, regardless of whether services are currently connected:

C++Python

// Subscribe to live annotation events for a specific stream
auto subscribed = mediaSession->experimental().subscribeToAnnotations("my-stream-id");
if (!subscribed) {
    // Subscription failed...
    auto error = subscribed.error();
}

To stop receiving events for a stream:

C++Python

// Unsubscribe from annotation events for a stream
auto unsubscribed = mediaSession->experimental().unsubscribeFromAnnotations("my-stream-id");
if (!unsubscribed) {
    // Unsubscription failed...
    auto error = unsubscribed.error();
}

To unsubscribe from all streams at once:

C++Python

// Unsubscribe from all annotation streams at once
mediaSession->experimental().unsubscribeFromAllAnnotations();

Note

All subscriptions are automatically cleared when the session is disconnected. You do not need to explicitly unsubscribe before disconnecting.

Set up a callback to handle incoming annotation events. Each event is delivered as a JSON array tuple containing the sequence number, profile ID, details object, timestamp, and user ID:

C++Python

// Set up a callback for live annotation events
mediaSession->experimental().onAnnotationReceived(
    [](const auto& session, string_view streamId, const json_value& event)
    {
        // event is a JSON array: [seqNum, profileId, details, timestamp, userId]
        const auto* tuple = event.if_array();
        if (tuple == nullptr || tuple->size() < 3) {
            return;
        }
        const auto* details = tuple->at(2).if_object();
        if (details == nullptr) {
            return;
        }
        // Access annotation fields from details, e.g. "command", "params"...
    });

Fetching annotation history

To retrieve persisted annotation events (e.g. events that occurred before your application joined the session), use the history fetch method:

C++Python

// Fetch persisted annotation history for a stream
auto fetched = mediaSession->experimental().fetchAnnotationHistory(
    "my-stream-id",
    [](const auto& result)
    {
        if (result.error) {
            // Handle async fetch error...
            return;
        }
        if (!result.response || !result.response->valueJson) {
            return;
        }
        const auto* historyArray = result.response->valueJson->if_array();
        if (historyArray == nullptr) {
            return;
        }
        for (const auto& entry : *historyArray) {
            // Process each event tuple...
        }
    });
if (!fetched) {
    // Synchronous error (e.g. not connected)
    auto error = fetched.error();
}

The history response is a JSON array where each element has the same tuple format as live events. This allows applications to use the same parsing logic for both live and historical data.

Deduplicating live and historical events

When an application subscribes to live events and also fetches history, there may be an overlap period where the same event appears in both sources. Each event tuple contains a sequence number (seqNum at index 0) that is unique per event within a stream. Use this value to deduplicate:

• Track seen sequence numbers in a set shared between the live callback and the history handler.

• Before processing any event, check whether its seqNum has already been seen.

• For multiple streams, use a composite key (streamId, seqNum) since sequence numbers are scoped per stream.

The annotation-reception sample demonstrates a complete implementation of deduplication between live events and history, including validation helpers and thread-safety considerations.

Note

Event ordering. When subscribing to live events and fetching history concurrently, live events may arrive before the history response. For example, if historical events A, B, C exist and a new event D arrives live before the history fetch completes, your application may process them as D, A, B, C rather than the temporal order A, B, C, D.

If strict ordering matters for your use case, consider buffering live events until the history response has been received and processed, then replaying the buffered events (filtering any duplicates by seqNum).