Live histogram, waveform, vectorscope
Open, NormalPublic

Description

To apply for this task please complete one of those mandatory coding challenge together with your application: T871 or T872
Applications without matching coding challenge completed will not be considered.


The AXIOM Beta features a small program called cmv_hist3 that calculated raw histogram values (https://en.wikipedia.org/wiki/Image_histogram) from a current image in the real time processing pipeline. cmv_hist3 is not meant to be fully real time capable as it is enough in most cases to calculate a new histogram with every couple of frames captured (image sensor can also run up to 300 fps).

cmv_hist3 source code: https://github.com/apertus-open-source-cinema/axiom-beta-firmware/tree/master/software/sensor_tools/hist

First steps are understanding and improving this tool with small new features (crop image and extract histogram only from selection), etc.

The next steps are creating similar tools for Waveform (https://en.wikipedia.org/wiki/Waveform_monitor) and Vectorscope (https://en.wikipedia.org/wiki/Vectorscope) display as typically used with cinematography/color analysis.

A good introduction to these tools can be found here:

While these tools can be fully CPU-based, using subsampling to achieve a decent frame rate, it's best to accelerate them using FPGA. Of course, this will increase the task difficulty and is therefore optional.

Prerequisites:

  • Basic math knowledge

Language Skills:

  • C
  • optional: VHDL

Difficulty: Medium

Mentors: Alex, Bertl

To get in touch with any mentor check the Mentor Contact List.

Notes: If you do not have access to the required hardware platforms/dev kits we will provide them (or remote access) to you for the duration of the project.

sebastian created this task.Feb 9 2017, 4:52 PM
sebastian updated the task description. (Show Details)
sebastian raised the priority of this task from to Normal.
sebastian moved this task to FPGA / HDL on the Google Summer of Code 2019 board.
sebastian moved this task from FPGA / HDL to Misc on the Google Summer of Code 2019 board.
sebastian added a subscriber: sebastian.
Bertl updated the task description. (Show Details)Feb 9 2017, 5:15 PM
alex updated the task description. (Show Details)Feb 9 2017, 5:29 PM
alex updated the task description. (Show Details)
alex updated the task description. (Show Details)
alex updated the task description. (Show Details)Feb 9 2017, 5:32 PM
alex renamed this task from camera internal histogram, waveform, vectorscopes to Live histogram, waveform, vectorscope.
GrahamM claimed this task.Feb 17 2017, 6:26 PM

A good example ist the waveform of Final Cut Pro X, it has color information merged into the luma based waveform. You can watch it here at minute 3:10 youtube

GrahamM added a comment.EditedFeb 20 2017, 11:51 PM

Above (top right) is an early crude waveform produced from data from the Axiom Beta. The larger waveform is from a recording at 1080p as viewed on DaVinci Resolve, which includes the Axiom Beta overlay.

I will have a look at that christophvarga.

Here is the final cut version if you are interested

could you post a screenshot of an example where whitebalance is almost correct? i would be interested how that looks like.

christophvarga added a comment.EditedMar 1 2017, 11:22 PM

someone at natron.fr is probably also working on an waveform monitor https://forum.natron.fr/t/luma-waveform-display-using-shadertoy/985

hello do we have IRC for apertus for gsoc's 2017 projects?

Bertl added a subscriber: Bertl.Mar 23 2017, 9:10 AM
Bertl added a comment.Mar 23 2017, 9:12 AM

Yes, we have IRC, it is on #apertus @ irc.freenode.
Just join there and ask if you need anything.

Best,
Herbert

mash added a subscriber: mash.Mar 28 2017, 6:13 PM

the waveform and vectorscope feature requests are long outstanding wishes.

recently natron saw some important improvements concerning the viewer and more complex groups handling different color spaces / RRTs / ODTs in viewers and scopes. that was a required prerequisite fore more useful scopes, because this kind of metering tools do not make much sense without an adequate translation from the internal linear data representation to more common display referred standards.

i would suggest to utilize arryfire and it's very fast GPU accelerated histogram routines in the actual implementation.

Bertl added a comment.Mar 29 2017, 2:49 PM

@mash: while that is a good idea for cutting/post software, this is not an option for the AXIOM Beta as there is no GPU to accelerate anything.

Nevertheless, part of the image analysis and probably most of the result display can be done in the FPGA directly on the image data-streams.

Best,
Herbert

mash added a comment.EditedMar 29 2017, 3:54 PM
In T734#11374, @Bertl wrote:

@mash: while that is a good idea for cutting/post software, this is not an option for the AXIOM Beta as there is no GPU to accelerate anything.

yes -- you are right!
i didn't think about this constraints of an implementation on the camera -- sorry!

but the arrayfire hint could be still useful for colleagues interested in this topic, because it's using a platform neutral modularization for this kind of performance critical tasks. the same code will use available acceleration by CUDA or OpenCL, but will also run in multiple threads or single threaded just on a CPU. and it's IMHO really recommendable to utilize well optimized libraries like this for histogram calculation, because it's not trivial to do it better by writing it from scratch.

in case of an in camera implementation, i would also abstain from more precise color transformations or messurement values translated to e.g. rec709 levels / IRE etc. this doesn't make much sense on a RAW camera and would only degrade the performance. simple histograms and waveform representations of the RAW sensor values or their logarithmic equivalent are more useful in this case to control exposure and the clipping of channels. a vectorscope doesn't make much sense under this circumstances neither.

sebastian removed GrahamM as the assignee of this task.Jan 19 2018, 9:07 PM
sebastian added a subscriber: GrahamM.
sebastian updated the task description. (Show Details)Jan 20 2018, 8:16 AM
sebastian moved this task from Misc to Misc on the Google Summer of Code 2019 board.
Bertl updated the task description. (Show Details)Jan 23 2018, 1:01 PM
Bertl changed the edit policy from "All Users" to "Administrators".
Bertl updated the task description. (Show Details)Mar 4 2018, 1:23 PM
sebastian updated the task description. (Show Details)Mar 8 2018, 8:27 PM
sebastian changed the edit policy from "Administrators" to "All Users".Tue, Feb 5, 9:54 AM
Bertl updated the task description. (Show Details)Wed, Feb 6, 12:27 PM
Bertl updated the task description. (Show Details)Wed, Feb 6, 1:17 PM
sebastian updated the task description. (Show Details)Thu, Feb 14, 11:25 AM