I just had an idea: in order to reduce noise (which also has the added affect of increasing DR) I was thinking it would be possible to utilise the high frame rate of the chip to average a series of frames (this should eradicate a lot of temporal noise) - and express this as a reduced ISO.
Alex at ML seemed to suggest this at one time - but I thought it was worh bringing up again, in terms of expressing it formally as reduced ISO in the camera.
Ah thanks for bringing this a task to the lab, yes we discussed it with Alex and it seems to have a very high potential for increasing dynamic range and reducing noise.
In theory (as far as I understood it) we could shoot 200 FPS for an output format of 25FPS. Then we have 8 frames for what is meant to be one frame. factor 8 means 3 additional F-stops dynamic range. In addition we could alter the exposure time of these 8 frames and that way further increase DN - this would need some experiments.
Another thing we should investigate is that the image sensor datasheet hints at the possibility to have overlapping windows of interest so this could mean we get two read outs before one reset which could again be a way to get multiple samples within one exposure time to increase DN and reduce noise.
Lots of potential but also lots of work ahead :)
I believe frame averaging doesn't have any motion artefact problems - it's effectively the same as taking one frame of 1/50th second exposure - but temporal noise is effectively lost in the averaging. Fixed pattern noise remains (although a black balance can deal with a lot of that. The main drawback is a significantly reduced ISO.
Original post is here: http://www.magiclantern.fm/forum/index.php?topic=11787.0
What about long exposures?
Here we have a huge advantage: on this camera, we have access to the FPGA, which is much faster at image processing than the general-purpose ARM processor. One could capture short exposures and average them on the fly, which has a very nice impact on the dynamic range.
How much? Stacking N images will improve shadow noise by log2(sqrt(N)) stops - so averaging 64 images will give 3 more stops of DR, just like that. Assuming the hardware is fast enough to average 4K at say 100fps, a 10-second exposure could have a 5-stop DR boost. Without motion artifacts or resolution loss.
Where's the catch?
Read noise (in electrons) does not depend much on exposure time (on 5D3, the noise difference between 1/8000 and 15" is minimal). Therefore:
So, a stacked exposure, compared to a long exposure, would give:
At very long exposures (hours), things may be different - such a long exposure may no longer be as clean as a short one, or it might simply clip too many highlights. I don't have any experience with astro, so I'm just guessing.
Yes, I'm going to implement this in Magic Lantern as well - we have just found a routine that adds or subtracts two RAW buffers on the dedicated image processor (without using the main CPU).
This sensor can do 300 fps. Can you do the above trick for video?
(credits @anton__ for this idea)
The sensor does not do 300fps at maximum resolution, but I guess it can do this at 1080p (even with hardware binning). Note, this is pure speculation. If the Axiom hardware is fast enough to add 1080p frames at 300fps (I have no idea if it is), you could create a 1/50 exposure (180-degree shutter at 25fps) out of 6 frames captured at 1/300. This means 1.3 EV boost in DR, at the cost of 1.3 EV of shadow noise (it will require more light). Good for emulating a low ISO, without motion artifacts like the other HDR modes.
Again - I don't know if the hardware is fast enough for this.
On the ISO/DR graph (scroll up) I've plotted what would happen if you would average 4 frames.
Can you expand upon that as I'm not too sure what you're talking about and Google doesn't help.
I did find this, however: http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm
The sensor does not do 300fps at maximum resolution.
In 10 bit mode it does: http://www.cmosis.com/products/standard_products/cmv12000
I believe Alex was talking about the mk1 sensor. The latest version is faster I believe.
Also there is a marked difference between 10bit and 12bit - which is worth bearing in mind.
All of these dynamic range hacks fall to the dreaded issue of motion; what can work theoretically for still images, works appallingly for motion based work. Hence the vast majority of these cheats are relegated to the bin as marketing hacks, with the exception being an exceptionally constrained and limited shooting context.
Given 1/48th or 1/50th motion blur, the object, or rapid camera motion, will induce motion blur in the scene.
No frame averaging, however clever, will solve this issue as it is literally a mixture of two objects. Worse, it is temporal and the contexts change.
So frame averaging must either remove the aesthetic of motion blur, or suffer very inconsistent issues where the blur is present. Hence motion blur artifacting is often one of the most problematic factors facing these dynamic range cheats. This is no small order as, when shooting at the cinema standard 24/25 FPS, there is a vast amount of motion blur present.
I imagine it would depend a lot on how the averaging was done. Anyway - can you post some examples of the sort of artefacts you're talking about? Only I had a look and all I could find were examples of stills built from multiple video frames - and these had quote nice motion blur.
I'm not disagreeing with you - like I said I've no experience of it, but I'm interested in trying to understand what you are saying.
Note about astronomy imaging (for space telescope):
THe challenge there is capturing very faint object without having the image "damaged" by highlights. Very long exposure time are used.
caveats : On CCD sensor, you have a white line when there is a pixel that oversaturated.
caveats : On CMOS sensor, your bright spot "bleeds" on neighbouring pixels.
Examples can be found if you search for SOHO solar images or STEREO solar images (both of them are satellites looking at the Sun)
If we say the sensor is running at 250fps and each frame records 1/250s of light coming onto the sensor.
For a 25fps Video at 1/50s you would use the first 5 Frames of the 250fps Stream - average them to calculate the first Image of the 25fps Stream.
Wait 25/s and then take the next 5 Frames and so on.
Think about a photographer using a flash. The flash would only destroys one image of you're 250 fps stream - the rest would be great. After averaging with the other 4 Frames a lot of details in this frame would be restored. So for high and short light impulses or higher dynamic range this technique could be really useful.
Motion artifacts are created when some time is left between those 250fps Stream Images e.g. for the readouttime and refresh of the Sensor.
Any Idea how much time the sensor needs between two frames?
It should be possible to examine these kind of motion artefacts if we take some highspeed Material with different exposure speeds and process it back in realtime with frameblending. I'm currently thinking about a movement that would be suited for this and is reproduceable any ideas?