Recently, we contributed a barcode recognition algorithm to opencv_contrib. In this blog post, we will introduce the algorithm and how to use it.
About the authors:
Junhao Liang and Tianqi Wang are undergraduates at Southern University of Science and Technology in Shenzhen. Their main research interest is computer vision
Introduction
Barcodes are a major technique to identify commodities in real life. A common barcode is a pattern of parallel lines arranged by black bars and white bars with vastly different reflectivity and contrast. Barcode recognition is the process of scanning the barcode in the horizontal direction to get a string of binary codes composed of bars of different widths and colors. This data is the code information of the barcode. The content of the barcode can be decoded by matching with various barcode encoding methods. Currently our code supports EAN-13, EAN-8 and UPC-A encoding methods.
EAN-13
The EAN-13 barcode is based on the UPC-A standard, which was first implemented in Europe by the International Item Coding Association and later gradually spread worldwide. Most of the common goods in life use the EAN-13 barcode. For more detail see EAN – Wikipedia. For convenience, we’re going to use EAN-13 as an example in the following.
As is shown in Figure 1, the 1-D barcode is a graphic identifier that expresses information by arranging multiple black bars and blank spaces with different widths according to certain coding rules. Compared with other images, the barcode area has the following two important characteristics: First, the bars and spaces in the barcode area are arranged in parallel, and the direction tends to be consistent; Second, for the legibility of the barcode, there is a large reflectivity difference between the bar and space, so that the gray contrast in the barcode area is large, and the edge information is rich.
Detail of Algorithm
Shown in Figure 2 is our algorithm, organized in discrete three steps. First, we locate the barcode in the image, then crop and binarize the ROI, and finally decode the ROI. In the following, we mainly introduce the principle of localization and decoding algorithm.
Barcode Localization
Directional coherence of the barcode is the most distinctive feature in a barcode image under a complex background. It can be used to detect areas that may contain a barcode and eliminate most background. Then the precise barcode area can be found according to other barcode features. We will describe the details of this algorithm.
Pre-processing
First, we need to convert the image to gray-scale and scale it to a preset size. Then we use the Scharr operator to calculate the gradients in the x and y directions respectively.
Directional Coherence Computing
We divide the image into patches by preset scale and then compute the directional coherence one by one. We can filter out those patches with low gradient coherence, the white boxes mean patches with high gradient coherence in Figure 3.
Erosion
Erosion operation is defined as follows. For each patch, we divide eight neighborhoods into 4 groups as figure 4 showed. We preserve this patch if there are more than half groups that have at least one neighborhood.
By erosion, we can filter out the isolated patches.
Patches Connection
This is the most important step. We need to connect the adjacent patches according to their gradient orientation, which is to form regions of patches with numerically similar averaging gradient orientation. Then, we use a rotated rectangle to fit the connected area.
Barcode Decoding
We refer to the ZXing barcode decoding algorithm and currently support three types of bar code decoding: EAN-13, EAN-8, and UPC-A.
After locating the barcode area, we crop the ROI and complete the following process.
Super-resolution
To improve the decoding of low-resolution barcodes, we use a super-resolution model which is used in WeChat’s QR-code recognition. Different super resolution scales are used for different sizes of barcode images. Small barcode image will be enlarged more.
Binarization
In this step, we use the Otsu’s method to binarize barcode images and complete the decoding process firstly. If it fails to decode, ZXing’s Hybrid-binarization will be applied and the decoding process will be tried again. Using these two binarizations together can improve the decoding success rate.
Decoding
In this step, we iteratively choose different decoders (EAN8, EAN13…) to scan the barcode multiple times to decode each digit and vote on the results. Once one of the decoders returns a result with high confidence, this step will return the result.
For example, in one iteration the decoder is EAN-13 format. The figure 9 shows the scanning lines, green lines represent scanning with successful decoding (correct in format but correctness of content not guaranteed) and red lines represent scanning with unsuccessful decoding. Only successful scanning participates in voting. Assume in these successful scanning, result “6922255451427” occurs 10 times and result “6922255412374” occurs 3 times. The EAN-13 decoder gives the result “6922255451427” with confidence 10/13 = 0.769. Due to its confidence being higher than 0.5, we will return it immediately. Otherwise, it will be the candidate and step into the next format decoder to get the result with the highest confidence.
Using Barcode Recognition in OpenCV
C++
#include "opencv2/barcode.hpp"
#include "opencv2/imgproc.hpp"
using namespace cv;
Ptr<barcode::BarcodeDetector> bardet = makePtr<barcode::BarcodeDetector>();
Mat input = imread("your file path");
Mat corners; //the corners of detected barcodes，if N barcodes detected, then the shape is [N][4][2]
std::vector<std::string> decoded_info; //the decoded infos, if fail to decode, then the string is empty
std::vector<barcode::BarcodeType> decoded_format; //the decoded types, if fail to decode, then the type is BarcodeType::NONE
bool ok = bardet->detectAndDecode(input, decoded_info, decoded_format, corners);
Python
import cv2
bardet = cv2.barcode_BarcodeDetector()
img = cv2.imread("your file path")
ok, decoded_info, decoded_type, corners = bardet.detectAndDecode(img)
See more details in sample code.
Performance
We compared the performance of our algorithm with that of ZXing. The test data is in BarcodeTestDataset. Our test data contains many small codes and skewed codes, that ZXing can’t handle. The result shows our algorithm achieves much higher accuracy than ZXing, and it looks like our algorithm is faster. However, ZXing spends most of its time doing the grayscale conversion, and it just costs 1ms for decoding, which means ZXing is much faster. It is still easy to understand because our algorithm does barcode localization and super-resolution.
(The test time excludes initialization and image reading, and our algorithm is based on C ++, and ZXing is based on Java)
Related Resources
This article refers to several online resources, this section lists those for convenience, as well as some additional materials:
