5.1.1. Release notes for v0.14

Queue diagnostics (experimental feature)

To help diagnose latency issues, we have an added an experimental feature to trace queue overruns. Queues help manage media flow where the input rate cannot be consistently consumed by further processing (e.g. an input video is encoded before sending to a remote session). When the input cannot be processed in time, we can either hold on to frames to keep quality/continuity of the media, or drop frames to avoid a (potentially increasing) backlog. SDK uses queues with a finite buffer for incoming media, which will drop older buffers when full. When queuing frames, we are of course introducing latency (an incoming media frame is held onto and processed later). Queues should be tuned to balance excessive latency (buffer too large) whilst maintaining quality by avoiding dropping buffers if possible (enough buffer to smooth occasional slow-processing).

The experimental queue diagnostics feature allows the application to enable tracing of queue overruns. The log output can be “spammy”, so we advise switching on for one feed at a time to get a clear picture rather than interleaving output in the log.

Enabling diagnostics only requires you to make a request on the feed, but importantly the feed must have started before diagnostics can be enabled, to set up the appropriate pipeline hooks.

// Include the FeedDiagnosticsRequest header
#include <PxMedia/FeedDiagnosticsRequest.h>

// When the feed starts, enable the diagnostics using a given log level for output
someFeed->onFeedStarted(
    [](auto& feed)
    {
        std::cout << "Feed " << feed->streamId() << " started playing" << std::endl;
        auto diagnostics = feed->feedRequest(experimental::FeedDiagnosticsRequest(LOG_LEVEL));
        if (!diagnostics) {
            // Failed to enable diagnostics, check diagnostics.error() for cause
        }
    });

When enabled, the log will output when the pipeline’s queue is overrunning, including the buffer size which indicates the potential latency introduced by the queue’s buffering.

` [INFO] MediaPipeline [webcam] queue [queue6]: OVERRUN (100ms) `

Local UDP latency improvement in CPU-constrained scenarios

An queue has been added between the WebRTC receiver and the local UDP output. This is intended to address a growing-latency issue that could occur in CPU-constrained scenarios, when the local UDP output encoder was not able to keep up with the incoming video feed. The new queue will buffer frames up to a limit (100ms), and discard older frames if the queue fills up. This was the original intended behaviour of the pipeline.

Local recording audio mixing changes

In previous SDK releases, PeerSession::createRecordingFeed would record given audio feeds into separate audio tracks in the output files. In this release, the method has been updated to mix the specified audio feeds into a single audio track in the output files by default.

The old default behaviour can be achieved by using the new overload of PeerSession::createRecordingFeed, which allows an application to specify which audio feeds to mix into which audio tracks in the output files, using the audioTrackFeedIds parameter.

Bug fixes

• Fixed an issue where attempting to create a feed when not connected to a media server session would result in a crash. Now, a meaningful error is returned instead.

Fixes to previous known issues with VideoStagePrivacy

Now when using VideoStagePrivacy:

• When the privacy state is toggled, the video feed resolution matches the incoming feed resolution (previously it was set to a fixed output resolution).

• When using the privacy feature with AVOutputFeedFile, the output file will be continuous (previously, it would split the file when privacy was toggled).

• Visual artefacts and conflicts between privacy and other video stages such as rotate and crop, or adaptive streaming, have now been resolved.