Hi,

I am a final year undergraduate student at National University of Sciences and Technology, with a major in Electrical Engineering. I participated in GSOC last year with TimeLab organisation and developed a time series simulator in Python.
I have a particular interest in parallel hardware programming. Recently, as a part of my semester project, I implemented a pipelined version of Convolutional Neural Network on FPGA using Verilog.

I am really interested in this project but I have a couple of questions that I would like clarified:

• First, I am wondering if the mentors would be open to the implementation of other techniques (in addition to Sobel Filters) that are useful in film making?
• Are we expected to port the algorithms on FPGAs to analyse performance?

First, I am wondering if the mentors would be open to the implementation of other techniques (in addition to Sobel Filters) that are useful in film making?

Definitely, what did you have in mind?

Are we expected to port the algorithms on FPGAs to analyse performance?

You mean try performance on different FPGAs?

• I had in mind techniques that could be used for blurring, sharpening, un-sharpening, embossing, etc. I am not sure if any of these are already implemented, but my idea is to develop a common module in Verilog that can accept kernel of any size (e.g, 3*3 or 5*5) and produce the desired effect on image. For example, the user could input a kernel of size 5*5 to produce Gaussian blur.

• Yes, I meant to quantify performance on different FPGAs.

I had in mind techniques that could be used for blurring, sharpening, un-sharpening, embossing, etc. I am not sure if any of these are already implemented, but my idea is to develop a common module in Verilog that can accept kernel of any size (e.g, 3*3 or 5*5) and produce the desired effect on image. For example, the user could input a kernel of size 5*5 to produce Gaussian blur.

A general kernel filter would make sense I guess, though I see no real use to blurring, sharpening or embossing effects in a cinema camera yet. But I am open to be convinced otherwise.

Yes, I meant to quantify performance on different FPGAs.

Our hardware designs are finished and the FPGA we use is decided so while its of course interesting to see differences the camera will use only the http://zedboard.org/product/microzed with Xilinx Zynq 7020.

Hi,
Here some of my queries regarding this project.
->What kind of edges does the camera require? What if the algorithm is developed if it only shows the edges when the camera is in motion? When the user is stationary it will store the edge information in memory, so that it will not have to do any processing.I think this may save computation time.

->Is the micro-zed connected to the CCD sensors() through its PMOD header?

->I was studying the " image processing pipeline paragraph" given in https://wiki.apertus.org/index.php/AXIOM_Beta/AXIOM_Beta_Software. I assume that the Sobel Filter to be designed will be in Image Overlay block of the pipeline?

Also, in addition to the above post, I would like to know about the various signals which the camera sensor (Truesense KAC12040 or Cmosis CMV!2000) data bus will provide?

Here is a link to a simple flowchart: https://github.com/sagniknitr/Real-time-sobel-filter-in-FPGA/blob/master/gsoc2.jpg

I would like to know the mentor's comment on this.

A useful resource for pipelined convolution implementation: https://daim.idi.ntnu.no/masteroppgaver/013/13656/masteroppgave.pdf

See section 4.4.4 (page 37) for the specific implementation.

@sagnikbasu: Sorry for the delay, I obviously missed your questions.

Regardless of the source of the sensel data (Sensor, Memory, etc) it can be assumed that a number of consecutive sensel will be available for processing and the blocks will be fetched in a left-to-right and top-to-bottom order. The sensel need to be buffered for processing (if the algorithm requires to access previous data) and the output needs to happen in a similar way (i.e. pipelined and in some left-to-right, top-to-bottom order).

The amount of data which needs to be processed is quite large (4K @ 60+ FPS) so optimiziation and pipelining are key, decimation might also be an option.

The MicroZed is connected via up to 36 LVDS pairs with the CMOS Sensor. The Data is transferred in several serial streams.

Hope that clarifies,
Herbert

This is the paper that demosaics/ interpolates the pixel as well as preserve the edge information.:
http://ieeexplore.ieee.org/abstract/document/1703585

Cannot open it without logging in or subscription.

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=9F84182448B48EEFAAC44D32A2770BF0?doi=10.1.1.78.5036&rep=rep1&type=pdf

This link is open for all