Hello, readers! I am very pleased to receive some good response to my last post on OpenCV edge detection. Thank you, guys. Shahsparx being a very young blog it means a lot. So I thought it would be a good idea to share some more OpenCV feature detection methods. In this post, I am aiming to give you some idea about OpenCV corner detection methods.
Considering that there are a good number of functions that deal with finding features, I am going to discuss more of them in my next post. Here, I am going to show you two OpenCV corner detection methods that look for traditional ‘corners.’
So are you ready?
Let’s get started.
First of all, let’s dive right into the basics.
OpenCV Corner Detection Basics
So what are ‘features’ in the context of image processing? In most simple words, a feature is an ‘interesting ‘ part of the picture.
An image’s feature could be anything. It depends on the problem or the nature of the application. We can go through every pixel of the picture and decide if the feature we are interested in is present there or not.
So what features should we be looking for in a picture? Well, some typical features that people look for depending on the problem they are trying to solve are the following,
So now we have an idea about the different features to look in an image. If you have missed the previous post on Edge detection in OpenCV, you should check it out. So let us see what has OpenCV got to offer for corner detection.
Corner detection in OpenCV
Just like the edges, corners are an important feature in an image. Traditionally, we define corners as the point where two edges intersect. In computer vision, the terms ‘corner’ and interest point are commonly used interchangeably.
So what exactly is an ‘interest point’? An ‘interest point’ is simply a point in an image that is well defined. They can be robustly detected with high degree of repeatability. Therefore we can say that a corner is also an ‘interest point.’
But an ‘interest point’ may not necessarily be a corner. For example, interest points may be
- A point that is a local intensity maximum or minimum could be an ‘interest point.”
- The end point of a line may also be an ‘interest point.’
- A point located on a curve which has a local maximum curvature.
Let’s us now talk about some properties of corners. They have a high curvature among neighboring local points. Look at any red corner in the above chess board image. You will notice that those red points lie in the juncture of regions having different intensities.
Rotational invariance is another property of corners. In simple words, you can rotate an image at any angle. But the corners will still stay the same. Also, the number of detected corners will also remain the same after rotation.
Finally, why do people need to identify corners in the image? Well, it has a lot of application in computer vision. I have listed the important ones below.
- Motion detection
- object recognition
- Image registration
- Panorama stitching
- Video tracking
- 3D modelling
- Image mosaicing
So now I think we have enough idea about what corners are and why they are important in Computer Vision. Let us now jump into some action. I am going to show you some OpenCV corner detection methods.
Here we go.
Harris Corner Detection
Harris Corner detection method is based on calculating each pixel’s gradient. It is relatively simple in principle. If the absolute gradient around the pixel in two directions are both great, then it judges that pixel as a corner.
If you are interested in the mathematical principle and derivation of Harris Corner Detection algorithm, you can follow this link.
I am using python to show you the code. See explanation below the code.
import cv2 import numpy as np import copy img = cv2.imread('maze.png') img_orig = copy.copy(img) gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) #must give a float32 data type input gray = np.float32(gray) dst = cv2.cornerHarris(gray,2,3,0.04) #result is dilated for marking the corners, not important dst = cv2.dilate(dst,None) # Threshold for an optimal value, it may vary depending on the image. img[dst>0.001*dst.max()]=[0,0,255] cv2.imshow('Original',img_orig) cv2.imshow('Harris Corner Detector',img) if cv2.waitKey(0) & 0xff == 27: cv2.destroyAllWindows()
The code is pretty straight forward. We are taking an input image and converting it to grayscale. You must note that we convert our image to float32 type as it is a prerequisite for the algorithm.
We are calling the cornerHarris(image,blocksize,K size,k) method with 4 parameters. Let’s see why.
- The image parameter is the image in which we want to find the corners.
- Block size is simply the size of the neighborhood in which we want to compare the gradient. See the basics part above for clarification.
- The ‘ksize’ is the aperture parameter of the Sobel operator. The Sobel operator here is calculating the gradient in two directions. Also, note that the kernel size must be odd and smaller than 31.
- Finally, the k parameter of the function is the Harris detector free parameter in the equation. I have not discussed the mathematical part here. You can visit here to know more about it.
As you can see it works pretty well for the parameters, I have passed to the function. Note that you could change the arguments in the cornerHarris() function. The parameters largely depend upon the nature of the image and the desired output.
Let us now see the implementation of another popular OpenCV corner detection method.
Shi-Tomasi Corner Detector
In theory, Shi-Tomasi Corner detector algorithm is not much different than Harris Corner Detector. I will not go much into the mathematical part, but let me give you a slight idea.
There is a scoring function that checks if a selected window contains a corner or not. So both the algorithms differ here on the scoring function. Shi-Tomasi proposed a different scoring function, improving upon a few cases where Harris corner detector misses.
Shi-Tomasi detector can find the ‘N’ strongest corners in the picture. This selection facility is handy in many cases.
Here is the original paper describing the detector in full detail.
Enough for the theory? Let us see Shi-Tomasi corner detector in action.
import cv2 import numpy as np import copy img = cv2.imread('box.jpg') img_orig = copy.copy(img) grayimg = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) corners = cv2.goodFeaturesToTrack(grayimg,10,0.05,25) corners = np.float32(corners) for item in corners: x,y = item cv2.circle(img,(x,y),5,255,-1) cv2.imshow("Original",img_orig) cv2.imshow("Top 10 corners",img) if cv2.waitKey(0) & 0xff == 27: cv2.destroyAllWindows()
We have an image of a box here. Let us see the results.
I have shown how we can select out the strongest ‘n’ corners using the goodFeaturesToTrack() function. You can see the four parameters I have passed. The second parameter specifies the number of corners I want to find. The last two arguments denote the quality level and minimum distance between which the corners are detected.
Finally, we know what corners are and why they are an important feature of an image. Also, I showed you how to implement Harris Corner Detection method and Shi-Tomasi Corner detection.
We also know that the above-shown methods are rotation invariant. But they are still not that good for some cases. What if we zoom in and enlarge our image? You will notice that the sharpness of many corners will fade away. So those corners will disappear upon scaling. However, it is still the same image.
So we can say that a feature detection method should be scale invariant. This problem is solved by some better methods of feature detection.
I will show you how to implement the scale invariant algorithms such as SIFT and SURF in the next post in this series. We will also see the FAST algorithm’s implementation in OpenCV. I will also show you how to detect corners in videos.
If you liked this tutorial, then please give a moment to leave a comment below. It means a lot. Also, if you have any questions about OpenCV corner detection methods, tell me below. I have also posted an Oracle tutorial series; please check it out if you want.
Download Code for OpenCV corner detection methods
If you need the code shown above, you can download it from the Github Repository.