Create Task
Maniphest T934

OpenCV.js based automated inspection of PCBs
Open, NormalPublic
Actions

Description

The goal is to create a solution for automatically analyzing image of the AXIOM Beta PCBs we produce in various stages of assembly/production.

Very often the method is to compare areas of a known good reference image to an unknown PCB and report areas with differences (where match is below threshold).

Images are typically scanned with a flatbed scanner where physical DPI is known or photographed where exact camera distance is unknown but PCB dimensions are known.

A first step to preprocess an image for further matching analysis is properly aligning the PCB (rotation/scaling).
PCBs typically have so called fiducials (optical markers) with known location on the PCBs (these coordinates are supplied):

OpenCV.js: https://docs.opencv.org/3.3.1/d5/d10/tutorial_js_root.html

sebastian created this task.Feb 8 2018, 11:42 AM
sebastian updated the task description. (Show Details)
sebastian raised the priority of this task from to Normal.
sebastian added a project: AXIOM Beta Software.
sebastian moved this task to Backlog on the AXIOM Beta Software board.
sebastian added a subscriber: sebastian.
sebastian updated the task description. (Show Details)Feb 8 2018, 1:46 PM
sebastian updated the task description. (Show Details)
sebastian added subscribers: BAndiT1983, Bertl.
BAndiT1983 added a comment.Feb 8 2018, 3:10 PM

I would suggest to avoid JS, performance could possibly be low, but OpenCV sounded great. Just had no time to look into that lib. Also TensorFlow could be interesting, as it uses AI.

sebastian added a comment.Feb 8 2018, 4:33 PM

I don't think this is application is performance critical, if processing the image take 1 second or 10 seconds doesn't matter much and still saves us a lot of time compared to inspecting each component individually under the microscope.
Using JS would have the advantage that we need to deal with almost no overhead for creating GUI elements and dealing with image rendering compared to qt, etc.

Thoughts?

BAndiT1983 added a comment.Feb 8 2018, 4:34 PM

Then try it this way first, can optimize later.

BAndiT1983 added a comment.Feb 19 2018, 7:38 PM

Current state of testing PCB inspection:

  • Done several tests, especially search fo fiducials
  • Seems rather promising, but to eliminate false positives some tricks are required
  • Screenshot shows several things which were tested: smal fiducial image is used as template for template matching, found ones are marked with black rectangle (grayscale image) to prevent repeated matching, after 4 fiducials are found the image is cropped down and straightened out
  • Found fiducials are marked by yellow rectangles at the moment (hard to spot, because of display scaling of OpenCV)
  • Straightened PCB image has not the right resolution yet
  • Also it's flipped, as order of fiducials wasn't respected yet
  • To get the right order, e.g. top-left, top-right, bottom-left, bottom-right, the image could be split horizontally and vertically in the middle and then matching applied
  • Another process step would be to find the contour, straighten the image by warping and crop it down to the contour first (see the screenshot below)

BAndiT1983 added a comment.Apr 10 2018, 10:51 PM

Finally restored the page again, not pretty, but fiducial search is working and the image is unwarped.

Next todos:

  • Write README.md about setup and usage (Linux)
  • Progress bar for upload will be added, so there is a feedback
  • Maybe some simple animation, while processing is ongoing

sebastian added a comment.Apr 11 2018, 3:19 PM

Looks good, I just added the https://github.com/apertus-open-source-cinema/pcb-aoi repo to phabricator