This Week
Team Goal: Implement algorithms for whole design
Personal Goal: Implement Bradley's classification code on the OpenMV board using OpenMV image library functions
While other weeks have seen rapid progress, unfortunately, this week I hit several roadblocks.
My goals were to implement algorithms for number, shape, fill, and color detection on the OpenMV board. Number of shapes seemed like the natural first classification to tackle, as it would involve getting bounding boxes on each shape, which would be helpful for the other classifications.
At first, I tried using the same preprocessing steps as I did for initial card localization (i.e. gaussian blur, adaptive threshold, erode and dilate, find blobs). Even after much trial and error tweaking the various parameters for each step, I was getting inconsistent results.
A friend recommended I try using GRIP, a lightweight software that allows one to visually chain together OpenCV operations and easily adjust parameters while seeing the results rendered in real time:
The results seemed promising, and the input images I used were directly saved off of the OpenMV board, so in theory it should work the same when performed on the board. To my dismay, after running blur and adaptive threshold with the same parameters, I got these ugly results:
I tried an alternate approach where I used OpenCV's laplacian() function to highlight edges on which I could threshold. Once again, in GRIP the results looked promising:
This gave me the best results so far, and I started trying to detect blobs from within the processed image. However, blob detection was not very robust:
In some cases, it would work perfectly (as in the leftmost picture above), but oftentimes it would either detect blobs in random parts of the card or not detect an outline as a valid shape since it wasn't continuous all the way around its perimeter (fourth image). No amount of tweaking parameters seemed to meaningfully fix the result: there were too many random visual artifacts that made isolating what was a shape from not very difficult.
Determining which parts of the preprocessed images using this set of processing steps seemed to be beyond the capabilities of simple filtering by geometry, and may require some form of CNN or Haar Cascade.
I tried another result in which I used OpenMV's automatic thresholding function, which according to the docs uses Otsu's method to determine the most optimal thresholds for each channel within an image. I printed out the calculated thresholds and tried them out using OpenMV IDE's built-in threshold editor and it seemed promising:
After implementing the automatic thresholding, I obtained the following result:
It appears to work extremely well for solid shapes, and gets rid of visual artifacts better than some of the other techniques. However, some of the shapes were obscured completely. There is relatively low contrast between the shape outlines (especially non-solid filled ones) and the card background, and inconsistent lighting over the entire playing area leads to inconsistent results. The steps that worked for card localization were successful because of the high contrast between the black background and white card face, but for the colored shapes themselves, the contrast is significantly less.
I am not completely sure how to proceed from here. Some more things to try might be finding a way to increase the contrast of each image before computing thresholds, or finding a way to adjust camera settings to make the shape boundaries stand out more. If all else fails, I may work with Bradley to bypass preprocessing altogether and use a CNN for every card parameter.
Next Week
Team Goal: Finish implementing classification functions in OpenMV
Personal Goal: Continue experimenting with preprocessing steps to isolate shape bounding boxes
I was hoping to be done with implementing all the classification functions this week, but counting the shapes on the card ended up taking significantly more time than expected, and is still not finished. My goal for next week will be to work with Bradley to get shape localization working to unblock progress on the rest of the classification algorithms.
Commentaires