Enable windowed and pixel binning modes - potential for high speed filming and increased DR
Open, LowPublic

Description

Reading through the data sheet for the Cmosis sensor there are a couple of exciting features of the chip.

Firstly it says that he max frame rate at 4K is 140fps in 12 bit (Whoa!), 300 fps at 4K in 10 bit (Whoa), but 528 fps (Whoa!) in 12 bit via X/Y Subsampling, which seems akin to the Half-4K Raw you get in the C500, and 1048 fps in 10 bit via X/Y Subsampling (WHOA!!!! That's phantom territory!) The possibilities here are very very exciting - at least for the Gamma if not the Beta.

The other interesting bit was this: "To maintain the same field of view but reduce the noise coming
out of the sensor, a binning mode is implemented on the chip. This mode will average a number of pixels (at pixel level) to reduce the noise and data coming from the chip."

Reducing the noise coming off the chip should allow us to increase the dynamic range (by cleaning up shadow stops). Obviously effective resolution is lost - but since the chip is 4K that still opens up the possibility for increased DR at HD resolutions. Although it is worth noting that this operation would require a different OLPF to be used in front of the sensor to avoid moire.

I'm also wondering if we can steal a few magic tricks from other camera manufacturers - specifically Arri who have engineered an incredible camera mostly through their ability to deal with sensor noise. As I understand it they do this in part by black shading the chip in between every exposure. Given the speed of the chip I'm wondering if something like that could be programmed into the FGPA. Secondly they talk about their 'dual gain architecture' which I'm guessing involves reading each pixel twice - once at a low gain and once again at a high gain and combining the data to increase DR. Again, I'm wondering if something similar could be programmed with this chip (although it does sound similar to the multiple slope HDR mode already engineered in the chip).

colinelves updated the task description. (Show Details)
colinelves raised the priority of this task from to Low.
colinelves added a subscriber: colinelves.
Bertl added a subscriber: Bertl.Jan 7 2015, 4:12 AM

"black shading" sounds interesting, please elaborate what it is and how it is done.

Also: http://en.m.wikipedia.org/wiki/Fixed-pattern_noise

http://www.automaatioseura.fi/jaostot/mvn/mvn2007/parameter.html

I think both Cmosis and Alex from Magic lantern should be able to help with this.

The black shading between each frame is indeed interesting, but it would mean the sensor can capture data without "opening" the (electronic) shutter.
What does it even mean in the context of an electronic shutter?

What can be captured in this state? Fixed noise from AD converters and electronics?

Didn't see anything like this in the cmosis sensor datasheet.

Yes, I'm not sure how it works. There are some pixels on the Cmosis chip set aside as black reference I believe but I'd need to look through the data sheet to find them.

It should, however, be possible to run a black shading/black balance operation in the usual way as Red and pretty much ever pro and prosumer camera does: you get the user to cap the lens then run an image capture sequence at various gain levels and separated RGB to produce a reference of the fixed pattern noise (I guess you average a series of frames to differentiate out fixed pattern and random noise) then save this as a reference image to subtract it from each frame (raw and non-raw - although you could, I suppose avoid this in Raw assuming you allow the reference frames to be saved for later use in post processing).

One thing worth having, since the sensor does have an in built temperature sensor, would be to have a warning come up whenever the sensor temperature changes significantly from the temperature that was present when the current black shading operation was carried out (reds do this) - as sensor temp has a big effect on FPN.

Here's the bit I was thinking about:

5.15 BLACK REFERENCE COLUMNS
When the appropriate SPI register is set, the 8 first and 8 last columns will be put to an electrical black reference. This electrical black reference can be used to correct row noise.

But there's also this:

5.16 TEST PATTERN
The CMV12000 has a built-in fixed test pattern. The pattern consists of increasing pixel values per column per channel. Per (top and bottom) channel the values of the column increase with 1. The value of the first column of a channel increases with 1 per channel. So channels 1/33 will contain 0, 1, 2 … 126, 127, channels 2/34 contain 1, 2, 3 … 127, 128, channels 32/64 contain 31, 32, 33 … 157, 158.

I'm wondering if there's any difference (caused by noise) between the test pattern read out and it's theoretical values - in which case maybe this may be used to reduce noise as well.

Although Alex at Magic Lantenr seemed to indicate that the black reference collums are too small to do much to reduce the noise: http://www.magiclantern.fm/forum/index.php?topic=11787.0

He suggested contacting Cmosis to see if they could be persuaded to increase the number of Optical Black pixel dedicated to this (from 8x8 to 200x200) - I'm not sure if any of you guys have done this.

Bertl added a comment.Jan 19 2015, 7:00 PM

The Test Pattern is precisely that, a pattern which is sent from the sensor _instead_ of the actual image.

It can be used to verify the sensor-FPGA connection and test the bit-error rate between sensor and FPGA.
No ADC, no conversion, no noise.

@black columns: I doubt that they are willing to change the sensor design, but feel free to try to convince them.

Best,
Herbert

I'm willing to give it a go. Who do I contact?

Response from Cmosis:

Hello Colin,

Although adding extra dark reference columns to the existing CMV12000 design is indeed not a big change, the vost involved is very high. To be able to produce sensors with such a change the masks to produce the sensors must be remade and ordered. This is very expensive so a high NRE cost will be subject to such a change (a few 100k). The cost of the sensor itself will not increase lot compared to the current CMV12000 pricing.

Regards,

Pieter Willems
Manager standard products
www.cmosis.com

I think it's probably not worth it, no? Maybe just work on better processing ;-)