2.2.17. Video Inputs

Outgoing video feed types provided by the SDK (MediaServerOutgoingVideoFeed, PeerLocalVideoFeed) accept video input from a variety of sources, such as:

providing the ability to capture and transmit video from, respectively:

a V4L2 device (e.g. a webcam)
an X11 desktop or window
a test pattern generator
a file
the output of another feed via inter (a cross-feed communication pipe)
a shared memory area written by another process

For other video input components, see subclasses of the VideoInputFeedComponent and VideoFormattedInputFeedComponent classes.

For example, to transmit a webcam to a media server:

    PxMedia::VideoInputFeedV4Linux2 inputVideo;
    inputVideo.deviceProperties().device("/dev/video0");
    inputVideo.videoCapabilities().frameRate(30).mediaType(
        PxMedia::VideoCapabilities::MediaType::IMAGE_JPEG);
    auto outgoingFeed =
        PxMedia::MediaServerOutgoingVideoFeed::create(mediaSession, props, inputVideo, localOut);

    video = media.VideoInputFeedV4Linux2()
    video.device_properties().device("/dev/video0")
    vc = video.video_capabilities()
    frame_rate = media.FrameRate(30)
    vc.frame_rate = frame_rate
    vc.media_type = media.MediaType.IMAGE_JPEG

    feed = media.MediaServerOutgoingVideoFeed.create(media_session, props, video, output)

To transmit a screen or window capture to a peer:

    PxMedia::VideoInputFeedXImage inputVideo;
    inputVideo.videoCapabilities().frameRate(30);
    PxUtility::Rectangle<int> captureArea;
    captureArea.right(1279).bottom(719);
    inputVideo.captureArea(captureArea);
    auto outgoingFeed =
        PxMedia::PeerLocalVideoFeed::Builder(peerSession, feedProperties, inputVideo)
            .localOutput(localOut)
            .encoder(encoderProperties)
            .create();

    screen = media.VideoInputFeedXImage()
    vc = screen.video_capabilities()
    frame_rate = media.FrameRate(30)
    vc.frame_rate = frame_rate
    capture_area = utility.RectangleInt()
    capture_area.right = 1279
    capture_area.bottom = 719
    screen.capture_area(capture_area)

    feed = (
        media.PeerLocalVideoFeed.Builder(peer_session, props, screen)
        .local_output(local_out)
        .encoder(encoder)
        .create()
    )

To transmit a test video stream to a media server:

    PxMedia::VideoInputFeedTestPattern inputVideo;
    inputVideo.videoCapabilities().frameRate(30).width(1280).height(720);
    auto outgoingFeed =
        PxMedia::MediaServerOutgoingVideoFeed::create(mediaSession, props, inputVideo, localOut);

    test = media.VideoInputFeedTestPattern()
    vc = test.video_capabilities()
    frame_rate = media.FrameRate(30)
    vc.frame_rate = frame_rate
    vc.width = 1280
    vc.height = 720
    feed = media.MediaServerOutgoingVideoFeed.create(media_session, props, test, output)

To transmit a test video file to a media server:

    PxMedia::VideoInputFeedFile inputVideo;
    inputVideo.filename(videoFilename);
    inputVideo.baseFormat(PxMedia::VideoMediaFormat::BaseMedia::VIDEO_ENCODED);
    inputVideo.videoCapabilities().frameRate(30).mediaType(
        Proximie::PxMedia::VideoCapabilities::MediaType::VIDEO_XRAW);
    auto outgoingFeed =
        PxMedia::MediaServerOutgoingVideoFeed::create(mediaSession, props, inputVideo, localOut);

    file_src = media.VideoInputFeedFile()
    file_src.filename("/path/to/video.mp4")
    file_src.base_format(media.BaseMedia.VIDEO_ENCODED)
    vc = file_src.video_capabilities()
    frame_rate = media.FrameRate(30)
    vc.frame_rate = frame_rate
    vc.media_type = media.MediaType.VIDEO_XRAW
    feed = media.MediaServerOutgoingVideoFeed.create(media_session, props, file_src, output)

Video file sources have a natural end while single-image files repeat the single image frame indefinitely. If the feed internally consumes the video data without transmitting to another feed then the feed will stop automatically when the end of the video is reached and the onFeedStopped callback will be executed if present. This is the case for UtilityVideoLocalFeed and AVOutputFeedFile. Otherwise the feed will continue in the playing state when the video ends. The output component will receive and retain the final image from the video.

To transmit video sourced from the output of another feed:

    PxMedia::VideoInputFeedInter inputVideo = outInter.makeInput();
    auto                         outgoingFeed =
        PxMedia::MediaServerOutgoingVideoFeed::create(mediaSession, props, inputVideo, localOut);

    input_video = output_inter.make_input()
    feed = media.MediaServerOutgoingVideoFeed.create(media_session, props, input_video, output)

VideoInputFeedInter instances must be obtained via makeInput() before the associated output is used to create the feed. Each instance can only be used in a single feed — violating either constraint is detected and surfaces as a PipelineElementBadProps error when the affected feed is created.

2.2.17.1. Shared Memory Input

VideoInputFeedSharedMemory reads raw video from a shared memory area written by another process (typically a matching VideoOutputFeedSharedMemory, see Video Outputs). The shared memory transport enables efficient, zero-copy video transfer between processes on the same machine. The two endpoints are paired by a shared control socket path:

    PxMedia::VideoInputFeedSharedMemory inputVideo;
    inputVideo.socketPath("/tmp/px-shm-video");
    // Shared memory carries raw video with no caps negotiation, so the capabilities
    // must be fully specified - including the pixel format - and must match the
    // format produced by the writing endpoint.
    inputVideo.videoCapabilities()
        .mediaType(PxMedia::VideoCapabilities::MediaType::VIDEO_XRAW_I420)
        .frameRate(30)
        .width(1280)
        .height(720);

    PxMedia::VideoOutputFeedAuto                   localOut;
    PxMedia::UtilityVideoLocalFeed::FeedProperties utilProps{"shared-memory-reader"};
    auto result = PxMedia::UtilityVideoLocalFeed::create(context, utilProps, inputVideo, localOut);

Warning

The shared memory transport carries raw bytes with no caps negotiation. The capabilities set on the input (resolution, frame rate and pixel format) must exactly match the published frame produced by the writing endpoint. No dimensions travel over the transport and no validation is performed, so a mismatch is not reported as an error - the reader silently misinterprets the bytes and renders grey bars, garbled/scrambled video, or a black frame.

Pay particular attention when the writer applies a resolution-changing stage, because the reader must be configured with the writer’s output dimensions, not its source dimensions:

A 90° rotate (VideoStageRotateFlip) swaps width and height (for example 1280x720 becomes 720x1280).
A crop (VideoStageCrop) reduces the frame dimensions to the cropped size.

Format-preserving stages leave the dimensions unchanged and so need no adjustment on the reader.

For the same reason, adaptive streaming is incompatible with a shared-memory reader. The reader’s caps are fixed when its feed is created and the transport cannot renegotiate, so if the writer adapts its resolution, frame rate or encoded format at runtime the published frame changes while the reader keeps interpreting the bytes against its original caps - misreading every frame after the change. Keep the published format fixed over a shared-memory transport.

An empty socket path surfaces as a PipelineElementBadProps error when the feed is created.

Note

The writing endpoint must be started first and must outlive the reader. The shared memory transport does not reconnect: when the writer exits, its control socket disappears and the reader’s underlying shmsrc reports a stream error. The SDK surfaces this as a feed-stopped notification carrying an error (register a handler with onFeedStopped()), so a reader can observe the disconnect and shut down cleanly rather than treating it as an unexpected failure.