CN111767750A - Image processing method and device - Google Patents

Image processing method and device Download PDF

Info

Publication number
CN111767750A
CN111767750A CN201910447841.XA CN201910447841A CN111767750A CN 111767750 A CN111767750 A CN 111767750A CN 201910447841 A CN201910447841 A CN 201910447841A CN 111767750 A CN111767750 A CN 111767750A
Authority
CN
China
Prior art keywords
bar code
barcode
target image
angle
deflection angle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910447841.XA
Other languages
Chinese (zh)
Inventor
孙旭
董玉新
刘巍
陈宇
翁志
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Wodong Tianjun Information Technology Co Ltd
Priority to CN201910447841.XA priority Critical patent/CN111767750A/en
Publication of CN111767750A publication Critical patent/CN111767750A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06KGRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/14Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
    • G06K7/1404Methods for optical code recognition
    • G06K7/1439Methods for optical code recognition including a method step for retrieval of the optical code
    • G06K7/1456Methods for optical code recognition including a method step for retrieval of the optical code determining the orientation of the optical code with respect to the reader and correcting therefore

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Electromagnetism (AREA)
  • General Health & Medical Sciences (AREA)
  • Toxicology (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the application discloses an image processing method and device. One embodiment of the method comprises: acquiring a target image presenting a bar code; determining a barcode region in a target image; inputting the bar code area into a pre-trained angle prediction model to obtain a deflection angle of the bar code in the bar code area, wherein the angle prediction model is used for identifying the deflection angle of the bar code in the bar code area, and the deflection angle is an included angle between a bar in the bar code and a vertical coordinate axis in a preset first coordinate system; and rotating the bar code in the target image based on the deflection angle of the obtained bar code to obtain the bar code after angle correction. This embodiment improves the accuracy of barcode recognition.

Description

Image processing method and device
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to an image processing method and device.
Background
In recent years, with the spread of computer applications, the use of bar codes has been greatly developed. The bar code can mark information such as the country of manufacture, manufacturer, name of goods, date of manufacture, book classification number, start and stop location of mail, category, date, etc., so that the bar code is widely applied to many fields such as commodity circulation, book management, post and telecommunications management, bank systems, etc.
Disclosure of Invention
The embodiment of the application provides an image processing method and device.
In a first aspect, an embodiment of the present application provides an image processing method, including: acquiring a target image presenting a bar code; determining a barcode region in a target image; inputting the bar code area into a pre-trained angle prediction model to obtain a deflection angle of the bar code in the bar code area, wherein the angle prediction model is used for identifying the deflection angle of the bar code in the bar code area, and the deflection angle is an included angle between a bar in the bar code and a vertical coordinate axis in a preset first coordinate system; and rotating the bar code in the target image based on the deflection angle of the obtained bar code to obtain the bar code after angle correction.
In some embodiments, determining a barcode region in the target image comprises: the barcode region in the target image is determined based on a pre-trained barcode detection model.
In some embodiments, the barcode detection model comprises a lightweight neural network comprising at least two cascaded feature extraction layers and a single multi-bounding box detection model comprising a fully connected layer and a prediction layer; and determining a barcode region in the target image based on a pre-trained barcode detection model, comprising: inputting a target image into a lightweight neural network, and fusing feature vectors output by the last two feature extraction layers of the lightweight neural network; inputting the fused feature vector into a full connection layer to obtain position information of a bar code region in the target image under a preset second coordinate system; and inputting the position information into the prediction layer to obtain a bar code area in the target image.
In some embodiments, rotating the barcode in the target image based on the obtained deflection angle of the barcode results in an angle corrected barcode, comprising: and rotating the bar code in the target image by a deflection angle to enable the bars in the bar code in the target image to be parallel to the ordinate axis in the preset first coordinate system, so as to obtain the bar code after angle correction.
In some embodiments, the method further comprises: and identifying the bar code after the angle correction to obtain an identification result.
In some embodiments, identifying the angle-corrected barcode results in an identification result, including: carrying out fuzzy detection on the angle-corrected bar code through a Laplace operator to obtain the fuzzy degree of the angle-corrected bar code; performing contrast enhancement processing on the barcode after the angle correction in response to the fact that the ambiguity is larger than a preset ambiguity threshold; and identifying the processed bar code to obtain an identification result.
In a second aspect, an embodiment of the present application provides an image processing apparatus, including: an acquisition unit configured to acquire a target image presented with a barcode; a determination unit configured to determine a barcode region in a target image; the input unit is configured to input the bar code area into a pre-trained angle prediction model to obtain a deflection angle of the bar code in the bar code area, wherein the angle prediction model is used for identifying the deflection angle of the bar code in the bar code area, and the deflection angle is an included angle between a bar in the bar code and a vertical coordinate axis in a preset first coordinate system; and the rotating unit is configured to rotate the bar code in the target image to obtain the bar code after angle correction based on the deflection angle of the obtained bar code.
In some embodiments, the determination unit is further configured to determine the barcode region in the target image as follows: the barcode region in the target image is determined based on a pre-trained barcode detection model.
In some embodiments, the barcode detection model comprises a lightweight neural network comprising at least two cascaded feature extraction layers and a single multi-bounding box detection model comprising a fully connected layer and a prediction layer; and the determination unit is further configured to determine the barcode region in the target image based on the pre-trained barcode detection model as follows: inputting a target image into a lightweight neural network, and fusing feature vectors output by the last two feature extraction layers of the lightweight neural network; inputting the fused feature vector into a full connection layer to obtain position information of a bar code region in the target image under a preset second coordinate system; and inputting the position information into the prediction layer to obtain a bar code area in the target image.
In some embodiments, the rotation unit is further configured to rotate the barcode in the target image based on the obtained deflection angle of the barcode to obtain an angle-corrected barcode as follows: and rotating the bar code in the target image by a deflection angle to enable the bars in the bar code in the target image to be parallel to the ordinate axis in the preset first coordinate system, so as to obtain the bar code after angle correction.
In some embodiments, the apparatus further comprises: and the identification unit is configured to identify the barcode after the angle correction to obtain an identification result.
In some embodiments, the identification unit is further configured to identify the angle-corrected barcode as follows: carrying out fuzzy detection on the angle-corrected bar code through a Laplace operator to obtain the fuzzy degree of the angle-corrected bar code; performing contrast enhancement processing on the barcode after the angle correction in response to the fact that the ambiguity is larger than a preset ambiguity threshold; and identifying the processed bar code to obtain an identification result.
In a third aspect, an embodiment of the present application provides an electronic device, including: one or more processors; a storage device, on which one or more programs are stored, which, when executed by the one or more processors, cause the one or more processors to implement the method as described in any implementation manner of the first aspect.
In a fourth aspect, the present application provides a computer-readable medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method as described in any implementation manner of the first aspect.
According to the image processing method and device provided by the embodiment of the application, the target image presented with the bar code is obtained; then, determining a bar code area in the target image; then, inputting the bar code area into a pre-trained angle prediction model to obtain the deflection angle of the bar code in the bar code area; and finally, rotating the bar code in the target image based on the deflection angle of the obtained bar code to obtain the bar code after angle correction. In this way, the bar code can be subjected to angle correction, so that the accuracy of bar code identification can be improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which various embodiments of the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of an image processing method according to the present application;
FIG. 3 is a schematic diagram of an application scenario of an image processing method according to the present application;
FIG. 4 is a flow diagram of yet another embodiment of an image processing method according to the present application;
FIG. 5 is a flow diagram of another embodiment of an image processing method according to the present application;
FIG. 6 is a schematic block diagram of one embodiment of an image processing apparatus according to the present application;
FIG. 7 is a block diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows an exemplary system architecture 100 to which the image processing method or the image processing apparatus of the present application can be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 over the network 104 to receive or send messages (e.g., the server 105 receives a target image presented with a barcode captured by the terminal devices 101, 102, 103), and so on. Various communication client applications, such as an image acquisition application, a payment application, a face recognition application, etc., may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, and 103 may capture an object presenting a barcode to acquire a target image presenting the barcode; then, the barcode region in the acquired target image can be determined; then, the bar code area can be input into a pre-trained angle prediction model to obtain the deflection angle of the bar code in the bar code area; finally, the barcode in the target image may be rotated based on the obtained deflection angle of the barcode to obtain an angle-corrected barcode.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a camera and supporting information interaction, including but not limited to a smart cooler, a smart phone, a tablet computer, a laptop computer, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The server 105 may be a server that provides various services. For example, a target image in which a barcode is presented may be analyzed to determine a deflection angle of the barcode in a barcode region, so that the barcode in the target image may be rotated to obtain an angle-corrected barcode. The server 105 may first acquire a target image presented with a barcode; then, the bar code area in the target image can be determined; then, the bar code area can be input into a pre-trained angle prediction model to obtain the deflection angle of the bar code in the bar code area; finally, the barcode in the target image may be rotated based on the obtained deflection angle of the barcode to obtain an angle-corrected barcode.
The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
It should be noted that the image processing method provided in the embodiment of the present application may be executed by the terminal devices 101, 102, and 103, or may be executed by the server 105.
It should be noted that the local areas of the terminal devices 101, 102, and 103 may store a pre-trained angle prediction model, and the terminal devices 101, 102, and 103 may determine a barcode area in the target image and rotate the barcode in the target image to obtain the barcode after angle correction. The exemplary system architecture 100 may not have a network 104 and server 105 at this point.
It should be noted that the local area of the server 105 may also store the target image presented with the barcode, and the server 105 may locally acquire the target image presented with the barcode. The exemplary system architecture 100 may not have terminal devices 101, 102, 103 and network 104 present at this time.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of an image processing method according to the present application is shown. The image processing method comprises the following steps:
step 201, a target image presenting a bar code is acquired.
In the present embodiment, an execution subject of the image processing method (e.g., a server shown in fig. 1) may acquire a target image presented with a barcode. The bar code is a graphic identifier in which a plurality of black bars and spaces having different widths are arranged according to a certain encoding rule to express a set of information. Common bar codes are a pattern of parallel lines of dark bars (simply bars) and white bars (simply spaces) of widely differing reflectivity. The target image may be an image to be processed. If the execution main body is a server, the execution main body can acquire a target image containing a bar code shot by the terminal equipment from the terminal equipment.
As an example, if the terminal device is an intelligent refrigerator device for storing medicines (e.g., vaccines), when the medicines are taken out from the intelligent refrigerator device, the camera of the intelligent refrigerator device may capture a barcode on a medicine package, and then the captured image including the barcode may be sent to the execution main body.
At step 202, a barcode region in the target image is determined.
In this embodiment, the executing entity may determine the barcode region in the target image acquired in step 201. The execution body may determine the barcode region in the target image using a barcode detection method based on morphology, a barcode detection method based on a window histogram, and a barcode detection method based on grayscale and stripe characteristics of a barcode.
Step 203, inputting the barcode region into a pre-trained angle prediction model to obtain the deflection angle of the barcode in the barcode region.
In this embodiment, the execution subject may input the barcode region into a pre-trained angle prediction model, and obtain a deflection angle of the barcode in the barcode region. The angle prediction model described above may be used to identify the deflection angle of the barcode in the barcode region. The angle prediction model can be a lightweight neural network (MobileNet), and the lightweight neural network is an efficient model provided for mobile and embedded devices. The lightweight neural network is based on a streamlined architecture (streamlined), and a lightweight deep neural network is constructed using a deep Separable convolution (Depthwise Separable Convolations). Here, the lightweight neural network may include five cascaded convolutional layers, where the first four convolutional layers are feature extraction layers and the last convolutional layer is a prediction layer.
In this embodiment, the deflection angle may be an angle between a bar in the barcode and an ordinate axis in the preset first coordinate system. In general, the barcode region may be a rectangular region. The preset first coordinate system may be an abscissa axis and an ordinate axis of two adjacent sides of the barcode region. Here, the ordinate axis of the preset first coordinate system is generally a side where an included angle between two adjacent sides of the barcode region and the bar in the barcode is smaller.
And 204, based on the deflection angle of the obtained bar code, rotating the bar code in the target image to obtain the bar code after angle correction.
In this embodiment, the executing body may rotate the barcode in the target image to obtain the barcode with the corrected angle based on the deflection angle of the obtained barcode. The execution body may determine a complementary angle of the deflection angle as a target angle, determine a direction in which an included angle between a bar in the barcode and a ordinate axis of the preset first coordinate system becomes larger after the barcode in the target image is rotated as a rotation direction of the barcode, and rotate the barcode in the target image by the target angle according to the determined rotation direction to obtain the barcode after the angle correction.
In some optional implementations of the embodiment, the executing subject may determine the barcode region in the target image by: determining a barcode region in the target image based on a pre-trained barcode detection model. Specifically, the barcode detection model may include a model based on a two-step detection algorithm (two-stage detection). The above-described model based on the two-step detection algorithm may include R-CNN (Region-Convolutional neural Network), Faster-RCNN (fast-Region Convolutional neural Network), Light Head RCNN (Light Head-Convolutional neural Network), and the like. Such models can generate candidate regions and classify and regress the candidate regions by RPN (Region pro-polysalnetwork). The barcode detection model may also be based on an end-to-end detection algorithm (one-stage detection). The end-to-end detection algorithm may include regression-based detection algorithms yolo (young Only Look one) and SSD (Single Shot multi box Detector, Single multi bounding box detection model). And the YOLO treats the object detection task as a regression problem, and directly obtains the coordinates of the boundary box, the confidence coefficient and the class probability of the object contained in the box through all pixels of the whole picture.
In some optional implementations of this embodiment, the executing body may rotate the barcode in the target image to obtain the barcode with an angle correction based on a deflection angle of the obtained barcode by: the execution body may rotate the barcode in the target image by the deflection angle, so that the bars in the barcode in the target image are parallel to the ordinate axis in the preset first coordinate system, thereby obtaining the barcode after angle correction. As an example, the executing body may determine, as a rotation direction of the barcode, a direction in which an angle between a bar in the barcode and an ordinate axis in a preset first coordinate system becomes smaller after the barcode in the target image is rotated, and rotate the barcode in the target image by the deflection angle according to the determined rotation direction to obtain the angle-corrected barcode.
In some optional implementations of the embodiment, the executing body may recognize the barcode after the angle correction to obtain a recognition result. As an example, the angle-corrected barcode may be recognized using an open-source ZBar code recognition library. ZBar is an open source library for scanning, reading two-dimensional codes and bar codes.
With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the image processing method according to the present embodiment. In the application scenario of fig. 3, the executing agent of the image processing method (e.g., server or terminal device) may first acquire a target image 301 presented with a barcode. Thereafter, the executing subject may determine the barcode region 302 in the target image 301. For example, the execution subject described above may determine the barcode region 302 in the target image 301 based on a barcode detection method of the gradation and streak features of the barcode. Then, the executing body may input the barcode region 302 into the angle prediction model 303 trained in advance, and obtain the deflection angle 304 of the barcode in the barcode region 302. Here, the deflection angle 304 of the bar code is 15 degrees. Here, the rotation angle of the barcode of 15 degrees is an angle formed by a bar in the barcode and a shorter side in the barcode region. Finally, the executing body may rotate the barcode in the target image 301 to obtain the angle-corrected barcode 305 based on the deflection angle 304 of the obtained barcode. The execution body may determine that a complementary angle of the deflection angle 15 degrees is 165 degrees, set 165 degrees as a target angle, determine a direction in which an included angle between a bar in the barcode and a shorter side in the barcode region becomes larger after the barcode in the target image 301 is rotated as a rotation direction of the barcode, and rotate the barcode in the target image 301 by 165 degrees according to the determined rotation direction to obtain the angle-corrected barcode 305.
In the method provided by the above embodiment of the present application, the barcode region in the target image in which the barcode is presented is determined, and the barcode region is input into the pre-trained angle prediction model, so as to obtain the deflection angle of the barcode in the barcode region, and rotate the barcode in the target image to obtain the barcode after the angle correction. In this way, the bar code can be subjected to angle correction, so that the accuracy of bar code identification can be improved.
With further reference to fig. 4, a flow 400 of yet another embodiment of an image processing method is shown. The flow 400 of the image processing method comprises the following steps:
step 401, a target image presenting a barcode is acquired.
In this embodiment, step 401 may be performed in a similar manner to step 201, and is not described herein again.
And step 402, inputting the target image into the lightweight neural network, and fusing the feature vectors output by the last two feature extraction layers of the lightweight neural network.
In this embodiment, the execution subject may input the target image into a lightweight neural network, and fuse feature vectors output by the last two feature extraction layers of the lightweight neural network. Lightweight neural networks are an efficient model proposed for mobile and embedded devices. The lightweight neural network is based on a streamline architecture, and the lightweight deep neural network is constructed by using depth Separable Convolutions (Depthwise Separable Convolentions). The lightweight neural network can extract image features from a target image. The lightweight neural network may include at least two cascaded feature extraction layers. Fusing the feature vectors may also be understood as merging the feature vectors. As an example, if the lightweight neural network includes four cascaded feature extraction layers, and the target image is input to the lightweight neural network, the third feature extraction layer of the lightweight neural network outputs 16 feature vectors, the fourth feature extraction layer of the lightweight neural network also outputs 16 feature vectors, and the 16 feature vectors output by the third feature extraction layer and the 16 feature vectors output by the fourth feature extraction layer are fused to obtain 32 feature vectors.
And 403, inputting the fused feature vector into the full connection layer to obtain the position information of the barcode region in the target image in a preset second coordinate system.
In this embodiment, the executing entity may input the fused feature vector into a full connection layer of a single multi-bounding box detection model, so as to obtain position information of the barcode region in the target image in a preset second coordinate system. The single-time multi-bounding box detection model is a multi-target detection model for directly predicting target classes and bounding boxes (bounding boxes). Based on a forward-propagating Convolutional Neural Network (CNN), a series of bounding boxes with fixed sizes and the possibility that each box contains an object instance are generated, and then Non-Maximum Suppression (Non-Maximum Suppression) is performed to obtain a final prediction. The above full-link layer can be used to perform regression analysis on the fused feature vectors.
In this embodiment, the preset second coordinate system may be a coordinate system in which two adjacent sides of the target image are respectively used as an abscissa and an ordinate. The position information may include coordinate information, for example, coordinates (x) of the upper left vertex of the barcode region1,y1) And coordinates (x) of lower right vertex2,y2) Or the coordinates (x) of the lower left vertex of the bar code region3,y3) And coordinates (x) of the top right vertex4,y4)。
Step 404, inputting the position information into the prediction layer to obtain the barcode region in the target image.
In this embodiment, the execution body may input the position information to a prediction layer in the single-time multi-bounding box detection model to obtain a barcode region in the target image. The prediction layer may predict the barcode region based on the position information.
As an example, the execution body may set coordinates (x) of an upper left vertex of the barcode region1,y1) And coordinates (x) of lower right vertex2,y2) And inputting the image data into a prediction layer in the single-time multi-bounding box detection model, wherein the prediction layer can output the barcode region in the target image.
Step 405, inputting the barcode region into a pre-trained angle prediction model to obtain the deflection angle of the barcode in the barcode region.
And step 406, based on the deflection angle of the obtained bar code, rotating the bar code in the target image to obtain the bar code after angle correction.
In the present embodiment, the steps 405-406 can be performed in a similar manner as the steps 203-204, and will not be described herein again
As can be seen from fig. 4, compared with the embodiment corresponding to fig. 2, the flow 400 of the image processing method in this embodiment embodies a step of fusing the feature vectors output by the last two feature extraction layers of the lightweight neural network, and determining the barcode region in the target image by using the fused feature vectors. Therefore, the scheme described in the embodiment can increase the characteristic information quantity of the bar code, and better solves the detection of small target objects, thereby further improving the accuracy of bar code identification.
With further reference to fig. 5, a flow 500 of another embodiment of an image processing method is shown. The flow 500 of the image processing method includes the following steps:
step 501, a target image presenting a bar code is acquired.
Step 502, a barcode region in a target image is determined.
Step 503, inputting the barcode region into a pre-trained angle prediction model to obtain the deflection angle of the barcode in the barcode region.
And 504, rotating the bar code in the target image based on the deflection angle of the obtained bar code to obtain the bar code after angle correction.
In the present embodiment, the steps 501-504 can be performed in a similar manner to the steps 201-204, and are not described herein again.
And 505, performing fuzzy detection on the angle-corrected bar code through a Laplace operator to obtain the fuzzy degree of the angle-corrected bar code.
In this embodiment, the execution body may perform blur detection on the angle-corrected barcode by using a laplacian Operator (Laplace Operator), so as to obtain a blur degree of the angle-corrected barcode. Specifically, the execution body may perform a convolution operation with the image of the barcode after the angle correction by using a laplacian operator, and then calculate a variance to obtain a blur degree of the barcode after the angle correction. The laplacian is a second order differential operator in n-dimensional euclidean space, defined as the divergence of the gradient.
The essence of the image blur is that the image is subjected to an averaging operation or an integrating operation, so that the image can be subjected to an inverse operation, for example, a differential operation, to highlight the image details and make the image clearer. Since laplacian is a differential operator, its application can enhance the abrupt gray level change region in the image and weaken the slow change region of gray level.
Step 506, determining whether the ambiguity is greater than a preset ambiguity threshold.
In this embodiment, the executing entity may determine whether the ambiguity obtained in step 505 is greater than a preset ambiguity threshold. If the ambiguity is greater than the preset ambiguity threshold, step 507 may be executed.
In step 507, in response to determining that the ambiguity is greater than the preset ambiguity threshold, contrast enhancement processing is performed on the barcode after the angle correction.
In this embodiment, if it is determined in step 507 that the blur degree is greater than the preset blur degree threshold, the executing body may perform contrast enhancement processing on the barcode after the angle correction. Here, Adaptive Contrast Enhancement (ACE) processing may be performed on the image of the barcode after the angle correction.
Here, the execution subject may perform adaptive contrast enhancement on the image of the barcode after the angle correction by: the image of the angle-corrected bar code is divided into two parts: one part is a low frequency part, which can be obtained by low pass filtering (smooth blurring) of the image; the other part is a high frequency part, which can be subtracted from the original. The goal of adaptive contrast enhancement is to enhance the high frequency portions representing the details, i.e., multiply the high frequency portions by a certain gain value, and then recombine to obtain an enhanced image. The heart of the ACE algorithm is therefore the calculation of the gain factor of the high frequency part, one solution is to set the gain to a fixed value, and the other solution is to express the gain value as a variance related quantity.
And step 508, identifying the processed bar code to obtain an identification result.
In this embodiment, the executing entity may identify the barcode processed in step 507 to obtain an identification result. As an example, the processed barcode may be identified using an open source ZBar barcode recognition library. ZBar is an open source library for scanning, reading two-dimensional codes and bar codes.
As can be seen from fig. 5, compared with the embodiment corresponding to fig. 2, the flow 500 of the image processing method in this embodiment represents steps of performing blur detection on the barcode, performing contrast enhancement processing on the barcode if the blur is greater than the preset blur threshold, and identifying the processed barcode. Therefore, the scheme described in the embodiment can process the blurred bar code, and the blurring degree of the bar code is reduced, so that the accuracy of bar code identification is further improved.
With further reference to fig. 6, as an implementation of the methods shown in the above figures, the present application provides an embodiment of an image processing apparatus, which corresponds to the embodiment of the method shown in fig. 2, and which is particularly applicable in various electronic devices.
As shown in fig. 6, the image processing apparatus 600 of the present embodiment includes: an acquisition unit 601, a determination unit 602, an input unit 603, and a rotation unit 604. Wherein the acquisition unit 601 is configured to acquire a target image presented with a barcode; the determination unit 602 is configured to determine a barcode region in the target image; the input unit 603 is configured to input the barcode region into a pre-trained angle prediction model, and obtain a deflection angle of the barcode in the barcode region, where the angle prediction model is used to identify the deflection angle of the barcode in the barcode region, and the deflection angle is an included angle between a bar in the barcode and a vertical axis in a preset first coordinate system; the rotating unit 604 is configured to rotate the barcode in the target image to obtain an angle-corrected barcode based on the obtained deflection angle of the barcode.
In the present embodiment, specific processing of the acquisition unit 601, the determination unit 602, the input unit 603, and the rotation unit 604 of the image processing apparatus 600 may refer to step 201, step 202, step 203, and step 204 in the corresponding embodiment of fig. 2.
In some optional implementations of the present embodiment, the determining unit 602 may determine the barcode region in the target image by: determining a barcode region in the target image based on a pre-trained barcode detection model. Specifically, the barcode detection model described above may include a model based on a two-step detection algorithm. The above-described models based on the two-step detection algorithm may include R-CNN, Faster-RCNN, Light Head RCNN, and the like. Such models can generate candidate regions by RPN and classify and regress the candidate regions. The barcode detection model described above may also be based on an end-to-end detection algorithm. The end-to-end detection algorithms described above may include regression-based detection algorithms YOLO and SSD. And the YOLO treats the object detection task as a regression problem, and directly obtains the coordinates of the boundary box, the confidence coefficient and the class probability of the object contained in the box through all pixels of the whole picture.
In some optional implementations of the present embodiment, the barcode detection model may include a lightweight neural network and a single-shot multi-bounding box detection model. Lightweight neural networks are an efficient model proposed for mobile and embedded devices. The lightweight neural network is based on a streamlined architecture, and a lightweight deep neural network is constructed using deep separable convolution. The lightweight neural network can extract image features from a target image. The lightweight neural network may include at least two cascaded feature extraction layers. The single-time multi-boundary-box detection model is a multi-target detection model for directly predicting target types and boundary boxes. It is based on a forward-propagating convolutional neural network, which generates a series of fixed-size bounding boxes and the likelihood that each box contains an object instance, and then performs a non-maximum suppression to obtain the final prediction. The single-pass multi-bounding box detection model described above may include a fully-connected layer and a predicted layer. The above full-link layer can be used to perform regression analysis on the fused feature vectors.
In some optional implementation manners of this embodiment, the determining unit 602 may input the target image into a lightweight neural network, and fuse feature vectors output by the last two feature extraction layers of the lightweight neural network. Fusing the feature vectors may also be understood as merging the feature vectors. Then, the determining unit 602 may input the fused feature vector into a full connection layer of a single multi-bounding box detection model, so as to obtain position information of the barcode region in the target image in a preset second coordinate system. Finally, the determining unit 602 may input the position information to a prediction layer in the single-time multi-bounding box detection model to obtain a barcode region in the target image. The prediction layer may predict the barcode region based on the position information.
In some optional implementations of the embodiment, the rotating unit 604 may rotate the barcode in the target image to obtain the barcode with an angle correction based on a deflection angle of the obtained barcode by: the rotating unit 604 may rotate the barcode in the target image by the deflection angle, so that the bars in the barcode in the target image are parallel to the ordinate axis in the preset first coordinate system, and the barcode after angle correction is obtained. For example, the rotating unit 604 may determine, as a rotation direction of the barcode, a direction in which an angle between a bar in the barcode and an ordinate axis in a preset first coordinate system becomes smaller after the barcode in the target image is rotated, and rotate the barcode in the target image by the deflection angle according to the determined rotation direction to obtain the barcode after the angle correction.
In some optional implementations of the present embodiment, the image processing apparatus 600 may further include a recognition unit (not shown in the figure). The identification unit can identify the bar code after the angle correction to obtain an identification result. As an example, the angle-corrected barcode may be recognized using an open-source ZBar code recognition library. ZBar is an open source library for scanning, reading two-dimensional codes and bar codes.
In some optional implementation manners of this embodiment, the identification unit may perform blur detection on the angle-corrected barcode by using a laplacian operator to obtain a blur degree of the angle-corrected barcode. Specifically, the identification means may perform a convolution operation with the image of the barcode after the angle correction using a laplacian operator, and then calculate a variance to obtain the blur degree of the barcode after the angle correction. The identification unit may determine whether the ambiguity is greater than a preset ambiguity threshold. If the ambiguity is greater than a preset ambiguity threshold, contrast enhancement processing may be performed on the barcode after the angle correction. Here, the adaptive contrast enhancement processing may be performed on the image of the barcode after the angle correction. The identification unit can identify the processed bar code to obtain an identification result. As an example, the processed barcode may be identified using an open source ZBar barcode recognition library. ZBar is an open source library for scanning, reading two-dimensional codes and bar codes.
Referring now to fig. 7, a schematic diagram of an electronic device (e.g., the server or terminal device of fig. 1) 700 suitable for use in implementing embodiments of the present disclosure is shown. When the electronic device in the embodiments of the present disclosure is a terminal device, the terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle-mounted terminal (e.g., a car navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 7, electronic device 700 may include a processing means (e.g., central processing unit, graphics processor, etc.) 701 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from storage 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for the operation of the electronic apparatus 700 are also stored. The processing device 701, the ROM 702, and the RAM703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
Generally, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 illustrates an electronic device 700 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 7 may represent one device or may represent multiple devices as desired.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communication means 709, or may be installed from the storage means 708, or may be installed from the ROM 702. The computer program, when executed by the processing device 701, performs the above-described functions defined in the methods of embodiments of the present disclosure. It should be noted that the computer readable medium described in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a target image presenting a bar code; determining a barcode region in a target image; inputting the bar code area into a pre-trained angle prediction model to obtain a deflection angle of the bar code in the bar code area, wherein the angle prediction model is used for identifying the deflection angle of the bar code in the bar code area, and the deflection angle is an included angle between a bar in the bar code and a vertical coordinate axis in a preset first coordinate system; and rotating the bar code in the target image based on the deflection angle of the obtained bar code to obtain the bar code after angle correction.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an acquisition unit, a determination unit, an input unit, and a rotation unit. The names of these units do not in some cases constitute a limitation on the unit itself, and for example, the acquisition unit may also be described as "a unit that acquires a target image presented with a barcode".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.

Claims (14)

1. An image processing method comprising:
acquiring a target image presenting a bar code;
determining a barcode region in the target image;
inputting the bar code area into a pre-trained angle prediction model to obtain a deflection angle of the bar code in the bar code area, wherein the angle prediction model is used for identifying the deflection angle of the bar code in the bar code area, and the deflection angle is an included angle between a bar in the bar code and a vertical coordinate axis in a preset first coordinate system;
and rotating the bar code in the target image based on the deflection angle of the obtained bar code to obtain the bar code after angle correction.
2. The method of claim 1, wherein the determining a barcode region in the target image comprises:
determining a barcode region in the target image based on a pre-trained barcode detection model.
3. The method of claim 2, wherein the barcode detection model comprises a lightweight neural network comprising at least two cascaded feature extraction layers and a single multi-bounding box detection model comprising a fully connected layer and a prediction layer; and
the determining a barcode region in the target image based on a pre-trained barcode detection model comprises:
inputting the target image into the lightweight neural network, and fusing the feature vectors output by the last two feature extraction layers of the lightweight neural network;
inputting the fused feature vector into the full-connection layer to obtain position information of a bar code area in the target image under a preset second coordinate system;
and inputting the position information into the prediction layer to obtain a bar code area in the target image.
4. The method of claim 1, wherein the rotating the barcode in the target image based on the derived deflection angle of the barcode to obtain an angle-corrected barcode comprises:
and rotating the bar code in the target image by the deflection angle to enable the bars in the bar code in the target image to be parallel to the ordinate axis in the preset first coordinate system, so as to obtain the bar code after angle correction.
5. The method according to one of claims 1-4, wherein the method further comprises:
and identifying the bar code after the angle correction to obtain an identification result.
6. The method of claim 5, wherein said identifying the angle corrected barcode results in an identification result comprising:
carrying out fuzzy detection on the bar code after the angle correction through a Laplace operator to obtain the fuzzy degree of the bar code after the angle correction;
performing contrast enhancement processing on the angle-corrected bar code in response to determining that the ambiguity is greater than a preset ambiguity threshold;
and identifying the processed bar code to obtain an identification result.
7. An image processing apparatus comprising:
an acquisition unit configured to acquire a target image presented with a barcode;
a determination unit configured to determine a barcode region in the target image;
the input unit is configured to input the barcode region into a pre-trained angle prediction model to obtain a deflection angle of a barcode in the barcode region, wherein the angle prediction model is used for identifying the deflection angle of the barcode in the barcode region, and the deflection angle is an included angle between a bar in the barcode and a vertical coordinate axis in a preset first coordinate system;
and the rotating unit is configured to rotate the bar code in the target image to obtain the bar code after angle correction based on the deflection angle of the obtained bar code.
8. The apparatus of claim 7, wherein the determination unit is further configured to determine the barcode region in the target image as follows:
determining a barcode region in the target image based on a pre-trained barcode detection model.
9. The apparatus of claim 8, wherein the barcode detection model comprises a lightweight neural network comprising at least two cascaded feature extraction layers and a single multi-bounding box detection model comprising a fully connected layer and a prediction layer; and
the determination unit is further configured to determine a barcode region in the target image based on a pre-trained barcode detection model as follows:
inputting the target image into the lightweight neural network, and fusing the feature vectors output by the last two feature extraction layers of the lightweight neural network;
inputting the fused feature vector into the full-connection layer to obtain position information of a bar code area in the target image under a preset second coordinate system;
and inputting the position information into the prediction layer to obtain a bar code area in the target image.
10. The apparatus of claim 7, wherein the rotation unit is further configured to rotate the barcode in the target image based on a deflection angle of the obtained barcode to obtain an angle-corrected barcode as follows:
and rotating the bar code in the target image by the deflection angle to enable the bars in the bar code in the target image to be parallel to the ordinate axis in the preset first coordinate system, so as to obtain the bar code after angle correction.
11. The apparatus according to one of claims 7-10, wherein the apparatus further comprises:
and the identification unit is configured to identify the angle-corrected bar code to obtain an identification result.
12. The apparatus according to claim 11, wherein the identification unit is further configured to identify the angle-corrected barcode as a result of the identification as follows:
carrying out fuzzy detection on the bar code after the angle correction through a Laplace operator to obtain the fuzzy degree of the bar code after the angle correction;
performing contrast enhancement processing on the angle-corrected bar code in response to determining that the ambiguity is greater than a preset ambiguity threshold;
and identifying the processed bar code to obtain an identification result.
13. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.
14. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-6.
CN201910447841.XA 2019-05-27 2019-05-27 Image processing method and device Pending CN111767750A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910447841.XA CN111767750A (en) 2019-05-27 2019-05-27 Image processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910447841.XA CN111767750A (en) 2019-05-27 2019-05-27 Image processing method and device

Publications (1)

Publication Number Publication Date
CN111767750A true CN111767750A (en) 2020-10-13

Family

ID=72718908

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910447841.XA Pending CN111767750A (en) 2019-05-27 2019-05-27 Image processing method and device

Country Status (1)

Country Link
CN (1) CN111767750A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113553876A (en) * 2021-09-22 2021-10-26 长沙海信智能系统研究院有限公司 Bar code identification method, device, equipment and storage medium
CN113627210A (en) * 2021-08-19 2021-11-09 南京华捷艾米软件科技有限公司 Method and device for generating bar code image, electronic equipment and storage medium
WO2023022404A1 (en) * 2021-08-18 2023-02-23 주식회사 에너자이 Barcode image identification method and device therefor
CN116451720A (en) * 2023-06-09 2023-07-18 陕西西煤云商信息科技有限公司 Warehouse material scanning and identifying method and identifying system thereof

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105809094A (en) * 2014-12-31 2016-07-27 研祥智能科技股份有限公司 Bar code identification method based on machine vision
CN107633192A (en) * 2017-08-22 2018-01-26 电子科技大学 Bar code segmentation and reading method under a kind of complex background based on machine vision
CN109165538A (en) * 2018-07-18 2019-01-08 北京飞搜科技有限公司 Bar code detection method and device based on deep neural network
CN109543486A (en) * 2018-10-29 2019-03-29 华南理工大学 Bar code localization method neural network based and system
CN109711508A (en) * 2017-10-25 2019-05-03 北京京东尚科信息技术有限公司 Image processing method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105809094A (en) * 2014-12-31 2016-07-27 研祥智能科技股份有限公司 Bar code identification method based on machine vision
CN107633192A (en) * 2017-08-22 2018-01-26 电子科技大学 Bar code segmentation and reading method under a kind of complex background based on machine vision
CN109711508A (en) * 2017-10-25 2019-05-03 北京京东尚科信息技术有限公司 Image processing method and device
CN109165538A (en) * 2018-07-18 2019-01-08 北京飞搜科技有限公司 Bar code detection method and device based on deep neural network
CN109543486A (en) * 2018-10-29 2019-03-29 华南理工大学 Bar code localization method neural network based and system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023022404A1 (en) * 2021-08-18 2023-02-23 주식회사 에너자이 Barcode image identification method and device therefor
CN113627210A (en) * 2021-08-19 2021-11-09 南京华捷艾米软件科技有限公司 Method and device for generating bar code image, electronic equipment and storage medium
CN113553876A (en) * 2021-09-22 2021-10-26 长沙海信智能系统研究院有限公司 Bar code identification method, device, equipment and storage medium
CN116451720A (en) * 2023-06-09 2023-07-18 陕西西煤云商信息科技有限公司 Warehouse material scanning and identifying method and identifying system thereof

Similar Documents

Publication Publication Date Title
CN108710885B (en) Target object detection method and device
CN109410218B (en) Method and apparatus for generating vehicle damage information
CN109816589B (en) Method and apparatus for generating cartoon style conversion model
CN108229419B (en) Method and apparatus for clustering images
CN108256479B (en) Face tracking method and device
CN108427927B (en) Object re-recognition method and apparatus, electronic device, program, and storage medium
CN111767750A (en) Image processing method and device
WO2016054779A1 (en) Spatial pyramid pooling networks for image processing
CN108765340B (en) Blurred image processing method and device and terminal equipment
JP2021508123A (en) Remote sensing Image recognition methods, devices, storage media and electronic devices
CN109711508B (en) Image processing method and device
CN109377508B (en) Image processing method and device
KR102606734B1 (en) Method and apparatus for spoof detection
CN110852349A (en) Image processing method, detection method, related equipment and storage medium
CN109118456B (en) Image processing method and device
CN110288625B (en) Method and apparatus for processing image
CN108470179B (en) Method and apparatus for detecting an object
CN108241855B (en) Image generation method and device
CN111209856B (en) Invoice information identification method and device, electronic equipment and storage medium
CN110490058B (en) Training method, device and system of pedestrian detection model and computer readable medium
CN114898177B (en) Defect image generation method, model training method, device, medium and product
CN112712036A (en) Traffic sign recognition method and device, electronic equipment and computer storage medium
CN115346278A (en) Image detection method, device, readable medium and electronic equipment
CN111783777B (en) Image processing method, apparatus, electronic device, and computer readable medium
CN110852250B (en) Vehicle weight removing method and device based on maximum area method and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination