Lower CPU usage by avoiding pixel format transformation

[jkuepker]

With the virtual cam disabled, my CPU usage in OBS hovers around 2.9%-3.3%. After starting virtual cam it jumps and hovers between 5.5%-6.0%.

Does this plugin respect the encoder settings for either Streaming/Recording?

Other items changed from default.

OUTPUT:
Encoder Apple VT H264 Hardware Encoder

VIDEO:
Base and Output: 1920x1080
Downscale Filter: Bilinear

[johnboiles]

Yah this plugin does a CPU transformation of the pixel format to UYVY (422) since that's what we advertise to the host software (see also apple docs).

It's been in the back of my mind to investigate this more -- it may be possible that one of the supported native pixel formats in OBS may also work as a webcam pixel format in the DAL plugin. :

Here is the list of constants I can use for that value in the code. Maybe kCMVideoCodecType_AppleProRes4444 would work with OBS's native I444 format.

I'd love help trying this out. Originally it was just a trial and error process until I figured out that VIDEO_FORMAT_UYVY in OBS worked as kCMVideoCodecType_422YpCbCr8 in the DAL plugin.

[jkuepker]

I can definitely test this for you. Let me know how. Thank you.

[johnboiles]

If you can build this from source (see DEVELOPING.md) then I would remove the lines in plugin-main.mm that do the output conversion and try different codec constants in the DAL plugin to see if they work.

[jkuepker]

Working on this but I seem to be missing something.
The OBS plugin has a reference to VIDEO_FORMAT_UYVY
The DAL plugin has a reference to kCMVideoCodecType_422YpCbCr8

But I cannot figure out the reference to that in OBS.
I see this VIDEO_FORMAT_UYVY and it is related to kCVPixelFormatType_422YpCbCr8

Do I also need to change the line above in the DAL plugin as well?

I have tested a few different VIDEO_FORMATS_* and they are below.
VIDEO_FORMAT_I444:

VIDEO_FORMAT_YUY2:

VIDEO_FORMAT_NV12:

[johnboiles]

Yeah you probably would need to change the kCVPixelFormatType_422YpCbCr8 reference in the DAL plugin as well for this to work.

If you push your changes to a branch on your fork and make a draft PR, happy to follow along with your code changes and chime in if I have ideas.

[johnboiles]

I tried this out (in this branch) with also changing the references to 422 in the DAL plugin to 444 and it crashed 🤷

Next maybe I'd try to match RGB formats maybe.

diff --git a/src/dal-plugin/CMSampleBufferUtils.mm b/src/dal-plugin/CMSampleBufferUtils.mm
index b13abc4d..cccc4795 100644
--- a/src/dal-plugin/CMSampleBufferUtils.mm
+++ b/src/dal-plugin/CMSampleBufferUtils.mm
@@ -19,7 +19,7 @@ OSStatus CMSampleBufferCreateFromData(NSSize size, CMSampleTimingInfo timingInfo

     // Create an empty pixel buffer
     CVPixelBufferRef pixelBuffer;
-    err = CVPixelBufferCreate(kCFAllocatorDefault, size.width, size.height, kCVPixelFormatType_422YpCbCr8, nil, &pixelBuffer);
+    err = CVPixelBufferCreate(kCFAllocatorDefault, size.width, size.height, kCMVideoCodecType_AppleProRes4444, nil, &pixelBuffer);
     if (err != noErr) {
         DLog(@"CVPixelBufferCreate err %d", err);
         return err;
@@ -118,7 +118,7 @@ OSStatus CMSampleBufferCreateFromDataNoCopy(NSSize size, CMSampleTimingInfo timi
     };

     CMFormatDescriptionRef format;
-    err = CMVideoFormatDescriptionCreate(NULL, kCMVideoCodecType_422YpCbCr8, size.width, size.height, (__bridge CFDictionaryRef)extensions, &format);
+    err = CMVideoFormatDescriptionCreate(NULL, kCMVideoCodecType_AppleProRes4444, size.width, size.height, (__bridge CFDictionaryRef)extensions, &format);
     if (err != noErr) {
         DLog(@"CMVideoFormatDescriptionCreate err %d", err);
         return err;
diff --git a/src/dal-plugin/Stream.mm b/src/dal-plugin/Stream.mm
index 7f1c5504..2ce13cee 100644
--- a/src/dal-plugin/Stream.mm
+++ b/src/dal-plugin/Stream.mm
@@ -268,7 +268,7 @@

 - (CMVideoFormatDescriptionRef)getFormatDescription {
     CMVideoFormatDescriptionRef formatDescription;
-    OSStatus err = CMVideoFormatDescriptionCreate(kCFAllocatorDefault, kCMVideoCodecType_422YpCbCr8, self.testCardSize.width, self.testCardSize.height, NULL, &formatDescription);
+    OSStatus err = CMVideoFormatDescriptionCreate(kCFAllocatorDefault, kCMVideoCodecType_AppleProRes4444, self.testCardSize.width, self.testCardSize.height, NULL, &formatDescription);
     if (err != noErr) {
         DLog(@"Error %d from CMVideoFormatDescriptionCreate", err);
     }
diff --git a/src/obs-plugin/plugin-main.mm b/src/obs-plugin/plugin-main.mm
index 7e211adc..0158f9e0 100644
--- a/src/obs-plugin/plugin-main.mm
+++ b/src/obs-plugin/plugin-main.mm
@@ -48,11 +48,11 @@ static bool virtualcam_output_start(void *data)

     obs_get_video_info(&videoInfo);

-    struct video_scale_info conversion = {};
-    conversion.format = VIDEO_FORMAT_UYVY;
-    conversion.width = videoInfo.output_width;
-    conversion.height = videoInfo.output_height;
-    obs_output_set_video_conversion(output, &conversion);
+//    struct video_scale_info conversion = {};
+//    conversion.format = VIDEO_FORMAT_UYVY;
+//    conversion.width = videoInfo.output_width;
+//    conversion.height = videoInfo.output_height;
+//    obs_output_set_video_conversion(output, &conversion);
     if (!obs_output_begin_data_capture(output, 0)) {
         return false;
     }

[jh97uk]

I'm also getting extremely high CPU usage with this, (in the neighbourhoods of 80%) I'll give these settings a go.