1 Topic by jontyc (edited by jontyc 2014-11-19 10:31:36)

  • jontyc
  • Member
  • Offline

I'll probably have to download the source and try to debug this one, but I thought I'd just run it by you first.

So PS3 Eye camera, latest CodeLabs driver (the single cam one), Kinovea 0.8.23, Lenovo laptop (Win7-64)

This is a screenshot of a live stream:

http://i.imgur.com/QAdLKf3.jpg

Now:

1. The test program that comes with the CodeLabs driver shows the video stream correctly.

2. On a desktop PC (also Win7-64), the same camera, same Kinovea and PS3 driver version shows the stream correctly.

This screenshot was 640x480@75fps but all other modes show similar issues.

2 Reply by joan

  • joan
  • joan
  • Admin
  • Offline

Ooh, very interesting :-)

I am deep into the capture code these days so this case is of utmost interest.

Basically it should work like this :
1. The camera driver exposes a number of formats that it supports, usually what the camera can source.
2. The capture code creates a "Sample Grabber" specifying format RGB24.
3. The capture code calls DirectShow to connect the camera driver to the sample grabber, and DirectShow adds the necessary converters inbetween (the path to go from source to destination might have several steps).
4. During capture, samples are grabbed in RGB24, a Bitmap object is created with the sample inside, and the bitmap is sent downstream to the rest of the code.

In 1, the source format from the driver can be RGB24 already, but usually it's in some form of YUV format like YUY2, I420 (a list here). It may also be in a compressed format if the camera has on-board compression (MJPEG or H.264).

What might happen is that DirectShow cannot find the proper color space converter to go from the YUV format to RGB24, but normally if there is no match the connection fails entirely.
Another possibility is that it can find the proper color conversion chain but somewhere something is interpreting the source wrongly.

I think I still have a working PS3Eye, I'll also see if I can find something.
I don't know if the CodeLabs driver is based on the Microsoft UVC driver like for most cameras, I'll check that later.

If you want to start to investigate, you can download GraphStudioNext. Add your camera and look at the properties of the output pin.

3 Reply by jontyc

  • jontyc
  • Member
  • Offline

It's sounding very feasible what you say as I've also noticed when viewing the stream at anything but 100% scaling the striations start showing CMY colors:

http://i.imgur.com/nUSZKOu.png

The above image was screenshot using Windows snipping tool. Taking a snapshot of the stream in Kinovea doesn't show the colors:



http://i.imgur.com/qK213Uq.jpg

So GraphStudioNext - I can see what you're asking as I have no problem seeing the properties of the output pin of the laptop's inbuilt webcam. Strangely the PS3 Eye camera isn't appearing, despite its driver supporting DirectShow. Only the PS3 Eye's microphone appears. This was on the same on the PC that the camera and Kinovea are working ok.

4 Reply by joan

  • joan
  • joan
  • Admin
  • Offline

OK, I still have a PS3Eye around. Although the lens is dead it's still good for testing.

I'm using CL-Eye-Driver-5.3.0.0341. In Graph studio, Graph > Insert filter, I have it listed under "PS3Eye Camera" in the DirectShow filters category and in the Video Capture Sources category. You can change the category in the top left combo box.

I also have it listed as "USB Camera-B4.04.27.1" in the WDM Streaming Capture devices. This is the actual name of the device at the USB level. It's not UVC compliant though, so Windows cannot automatically wrap it in a DirectShow capture source filter (hence the need for a third party driver). This filter only has the audio source.

Anyway, the list of supported media types is presented strangely, each framerate is a different media type. Normally there would be a list of media types for the various frame sizes, and a list of framerates supported at each frame size.

I can see that the main media types supported are RGB32 and RGB24, with two resolutions each.
We can also see this in the demo application, under Options > Video capture pin. Half of the entries are listed as "(32 bits)" and the lower half as "(24 bits)".

One way to get the problem you are having would be if some component is interpreting an RGB32 sample as an RGB24 image. Unfortunately due to a limitation in the current version, the RGB32 is automatically chosen in Kinovea. I'm experimenting with a version that lifts this limitation so I might contact you off-list for testing if it's ok with you.

5 Reply by jontyc (edited by jontyc 2014-11-20 00:27:37)

  • jontyc
  • Member
  • Offline

Yes, I also have the "USB Camera B4...." entry but assumed it was only the microphone on the camera as it had no video source.  The video properties on the output pin property sheet are all disabled, so unlike you I can't see that it supports RGB32 and RGB24 inside GraphStudioNext.

The CL_Test app - I can see both are supported.

A strange observation: GraphStudioNext appears to get stuck in an infinite loop polling the PS3 Eye camera when I start it up with the camera attached. I have to unplug the camera for the GraphStudioNext to become responsive again. If I then plug it back in, I can add it as a filter.

Now what you are saying with the RGB32/RGB24 stuff sounds very feasible for the banding, but the stream is also coming in vertically flipped. It is like there are problems enumerating this camera for its properties, of which I'm guessing there's some vertical orientation flag.

Sure, please contact me for testing. As I don't see any problems on my desktop PC, there's a chance you won't either. The only thing I can think of any significant difference between my laptop and desktop PC is that the laptop has an integrated webcam as well, but uninstalling that didn't fix anything.

6 Reply by jontyc (edited by jontyc 2014-11-20 01:14:32)

  • jontyc
  • Member
  • Offline

A further observation: the video stream is working fine in Skype, suggest CL_Test wasn't doing anything sneaky to get the camera working.

7 Reply by joan

  • joan
  • joan
  • Admin
  • Offline

With regards to flipping, it's a known difficulty with DirectShow. Some formats needs to be flipped, others not.

Currently the code assumes the image is flipped (as it should for RGB24) and during the creation of the bitmap it inverts it. As it works on other machines I believe that assumption is correct but maybe there is another filter upstream that makes a faulty conversion…

8 Reply by jontyc (edited by jontyc 2014-11-21 12:39:13)

  • jontyc
  • Member
  • Offline

I suspect the problem's in AForge, rather than Kinovea or the driver.

Why I'm thinking this:

  • Saving out the image as soon as it comes from AForge shows it's bad.

  • I have an ActionScript 3 sample app from CodeLabs interfacing with the camera via DirectShow and it works flawlessly.

9 Reply by joan

  • joan
  • joan
  • Admin
  • Offline

Interesting.
In AForge the filter graph is built using Intelligent Connect, the capture source and sample grabber are added to the graph, then a call to Render is made which asks the filter graph manager to do the connection. Can you tell if the CodeLabs sample use DirectConnect instead by any chance ?
I've seen that even when doing RGB24 to RGB24 there are extra filters added by the manager for some reason, including a color space converter…
I'm thinking of bypassing the Intelligent Connect for media types supported natively by the application, it would avoid extra copies and allocations along the way, in addition to avoid this type of surprises.

10 Reply by jontyc (edited by jontyc 2014-11-22 10:48:40)

  • jontyc
  • Member
  • Offline
joan wrote:

Interesting.
Can you tell if the CodeLabs sample use DirectConnect instead by any chance ?

The ActionScript3 sample gives no hint, nor could I find any documentation specifying the inner workings of AS3's Camera.getCamera() method. The sample just uses AS3's Camera and Video classes:

camera = Camera.getCamera( );
camera.setMode( 320, 240, 60 );
video = new Video( camera.width, camera.height );
video.attachCamera( camera );
addChild( video );  // Add to display

Yes, bypassing IntelligentConnect would be good, especially if you've had little quirky things like this in the past.

Rather than bringing over the AForge code to find the issue, as the Kinovea code is clean and well-structured (a welcome change and very much appreciated!), I've started implementing a PS3Eye CameraManager class to interface directly with its SDK rather than going through DirectShow.

I do find even on my machine where the PS3Eye works well with Kinovea that I could only alter the frame rates and resolution--the exposure, gain,... fields were disabled for modifications--so this will fix that too.  Also the PS3Eye SDK allows for lens distortion correction - not sure of the merits of doing it in camera driver versus Kinovea (speed I guess) but it's one thing I'll keep in mind for future.

BTW - the name "Kinovea" - related to 'kino' - the word for 'movie cinema' in many Slavic languages?