CN112215245A

CN112215245A - Image identification method and device

Info

Publication number: CN112215245A
Application number: CN202011223207.7A
Authority: CN
Inventors: 刘长伟; 薛媚方; 庾豪威; 王志丹; 冯婷婷; 闫石
Original assignee: China United Network Communications Group Co Ltd
Current assignee: China United Network Communications Group Co Ltd
Priority date: 2020-11-05
Filing date: 2020-11-05
Publication date: 2021-01-12

Abstract

The embodiment of the application provides an image recognition method and device, relates to the technical field of communication, and solves the technical problems of large error and low accuracy of the existing method for recognizing the color of a license plate. An image recognition method, comprising: the image recognition device firstly obtains a license plate image of a vehicle to be processed, and then inputs the license plate image into a multitask convolution neural network MTCNN model to obtain the shape of the license plate image. Subsequently, if the shape of the license plate image meets the preset shape, the image recognition device recognizes the color of the license plate image according to a clustering algorithm, and determines the recognized color as the color of the license plate of the vehicle to be processed.

Description

Image identification method and device

Technical Field

The present invention relates to the field of communications technologies, and in particular, to an image recognition method and apparatus.

Background

The license plate is an important mark of the vehicle, and the color of the license plate is one of important characteristics of the license plate. How to quickly and accurately identify the color of the license plate is a technical problem that needs to be solved urgently by a license plate identification device.

At present, a license plate recognition device usually recognizes the color of a license plate according to methods such as a pixel color statistical method, a machine learning algorithm or an infrared camera. However, the existing method for recognizing the color of the license plate cannot be adapted to the severe environment (such as heavy rain weather, sand storm weather, etc.), and thus the existing method for recognizing the color of the license plate has large error and low accuracy.

Disclosure of Invention

The application provides an image recognition method and device, and solves the technical problems of large error and low accuracy of the existing method for recognizing the color of a license plate.

In order to achieve the purpose, the technical scheme is as follows:

in a first aspect, an image recognition method is provided, including: the image recognition device firstly obtains a license plate image of a vehicle to be processed, and then inputs the license plate image into a multitask convolution neural network MTCNN model to obtain the shape of the license plate image. Subsequently, if the shape of the license plate image meets the preset shape, the image recognition device recognizes the color of the license plate image according to a clustering algorithm, and determines the recognized color as the color of the license plate of the vehicle to be processed.

As can be seen from the above, after the image recognition device obtains the license plate image of the vehicle to be processed, the image recognition device may input the license plate image into the MTCNN model to obtain the shape of the license plate image. The MTCNN model can determine the shape of the license plate through a multi-layer network structure, so that the accuracy of license plate image recognition is improved. Secondly, if the shape of the license plate image meets the preset shape, the image recognition device recognizes the color of the license plate image according to a clustering algorithm, and determines the recognized color as the color of the license plate of the vehicle to be processed. The color of the license plate can be determined by the clustering algorithm through a complex mathematical algorithm, so that the accuracy of license plate image recognition is further improved.

In a second aspect, an image recognition apparatus is provided, including: the device comprises an acquisition unit, a processing unit and an identification unit. The acquisition unit is used for acquiring the license plate image of the vehicle to be processed. And the processing unit is used for inputting the license plate image acquired by the acquisition unit into the multitask convolutional neural network MTCNN model so as to obtain the shape of the license plate image. And the recognition unit is used for recognizing the color of the license plate image according to a clustering algorithm if the shape of the license plate image obtained by the processing unit meets the preset shape, and determining the recognized color as the color of the license plate of the vehicle to be processed.

In a third aspect, an image recognition apparatus is provided that includes a memory and a processor. The memory is used for storing computer execution instructions, and the processor is connected with the memory through a bus. When the image recognition apparatus is running, the processor executes computer-executable instructions stored in the memory to cause the image recognition apparatus to perform the image recognition method of the first aspect.

The image recognition device may be a network device, or may be a part of a device in the network device, such as a system on chip in the network device. The system on chip is configured to support the network device to implement the functions related to the first aspect and any one of the possible implementations thereof, for example, to receive, determine, and shunt data and/or information related to the image recognition method. The chip system includes a chip and may also include other discrete devices or circuit structures.

In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising computer-executable instructions that, when executed on a computer, cause the computer to perform the image recognition method of the first aspect.

In a fifth aspect, there is provided a computer program product comprising computer instructions which, when run on a computer, cause the computer to perform the image recognition method as described in the first aspect above and its various possible implementations.

It should be noted that all or part of the above computer instructions may be stored on the first computer readable storage medium. The first computer readable storage medium may be packaged with or separately from a processor of the image recognition apparatus, which is not limited in this application.

For the description of the second, third, fourth and fifth aspects of the present invention, reference may be made to the detailed description of the first aspect; in addition, for the beneficial effects described in the second aspect, the third aspect, the fourth aspect and the fifth aspect, reference may be made to beneficial effect analysis of the first aspect, and details are not repeated here.

In the present application, the names of the above-mentioned image recognition apparatuses do not limit the devices or functional modules themselves, and in actual implementation, the devices or functional modules may appear by other names. Insofar as the functions of the respective devices or functional blocks are similar to those of the present invention, they are within the scope of the claims of the present invention and their equivalents.

These and other aspects of the invention will be more readily apparent from the following description.

Drawings

Fig. 1 is a schematic structural diagram of an image recognition system according to an embodiment of the present disclosure;

fig. 2 is a schematic diagram of a hardware structure of an image recognition apparatus according to an embodiment of the present disclosure;

fig. 3 is a schematic hardware structure diagram of another image recognition apparatus according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of an image recognition method according to an embodiment of the present application;

fig. 5 is a schematic flowchart of another image recognition method according to an embodiment of the present application;

fig. 6 is a schematic flowchart of another image recognition method according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of an image recognition apparatus according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

For the convenience of clearly describing the technical solutions of the embodiments of the present application, in the embodiments of the present application, the terms "first" and "second" are used to distinguish the same items or similar items with basically the same functions and actions, and those skilled in the art can understand that the terms "first" and "second" are not used to limit the quantity and execution order.

As described in the background, the license plate recognition apparatus usually recognizes the color of the license plate according to a pixel color statistical method, a machine learning algorithm, or an infrared camera, etc. However, the existing method for recognizing the color of the license plate cannot be adapted to the severe environment (such as heavy rain weather, sand storm weather, etc.), and thus the existing method for recognizing the color of the license plate has large error and low accuracy.

In order to solve the above problem, an embodiment of the present application provides an image recognition method, where after obtaining a license plate image of a vehicle to be processed, an image recognition device may input the license plate image into an MTCNN model to obtain a shape of the license plate image. The MTCNN model can determine the shape of the license plate through a multi-layer network structure, so that the accuracy of license plate image recognition is improved. Secondly, if the shape of the license plate image meets the preset shape, the image recognition device recognizes the color of the license plate image according to a clustering algorithm, and determines the recognized color as the color of the license plate of the vehicle to be processed. The color of the license plate can be determined by the clustering algorithm through a complex mathematical algorithm, so that the accuracy of license plate image recognition is further improved.

The image recognition method provided by the embodiment of the application is suitable for the image recognition system 10. Fig. 1 shows a structure of the image recognition system 10. As shown in fig. 1, the image recognition system 10 includes: a vehicle 11 and an image recognition device 12. The image recognition device 12 may acquire an image of the vehicle 11 by a camera or the like.

The vehicle 11 and the image recognition apparatus 12 in fig. 1 are similar in basic hardware configuration, and each include elements included in the image recognition apparatus shown in fig. 2. The following describes the hardware configuration of the vehicle 11 and the image recognition apparatus 12 in fig. 1, taking the image recognition apparatus shown in fig. 2 as an example.

Fig. 2 shows a hardware structure diagram of an image recognition apparatus provided in an embodiment of the present application. As shown in fig. 2, the image recognition apparatus includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, the memory 22 and the communication interface 23 may be connected by a bus 24.

The processor 21 is a control center of the image recognition apparatus, and may be a single processor or a collective term for a plurality of processing elements. For example, the processor 21 may be a Central Processing Unit (CPU), other general-purpose processors, or the like. Wherein a general purpose processor may be a microprocessor or any conventional processor or the like.

For one embodiment, processor 21 may include one or more CPUs, such as CPU 0 and CPU 1 shown in FIG. 2.

The memory 22 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that may store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that may store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

In a possible implementation, the memory 22 may exist separately from the processor 21, and the memory 22 may be connected to the processor 21 via a bus 24 for storing instructions or program codes. The processor 21, when calling and executing the instructions or program codes stored in the memory 22, can implement the image recognition method provided by the embodiment of the present invention.

In another possible implementation, the memory 22 may also be integrated with the processor 21.

And a communication interface 23 for connecting with other devices through a communication network. The communication network may be an ethernet network, a radio access network, a Wireless Local Area Network (WLAN), or the like. The communication interface 23 may include a receiving unit for receiving data, and a transmitting unit for transmitting data.

The bus 24 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 2, but it is not intended that there be only one bus or one type of bus.

It is to be noted that the configuration shown in fig. 2 does not constitute a limitation of the image recognition apparatus. In addition to the components shown in fig. 2, the image recognition apparatus may include more or fewer components than shown, or some components may be combined, or a different arrangement of components.

Fig. 3 shows another hardware configuration of the image recognition apparatus in the embodiment of the present application. As shown in fig. 3, the image recognition device may include a processor 31 and a communication interface 32. The processor 31 is coupled to a communication interface 32.

The function of the processor 31 may refer to the description of the processor 21 above. The processor 31 also has a memory function, and the function of the memory 22 can be referred to.

The communication interface 32 is used to provide data to the processor 31. The communication interface 32 may be an internal interface of the image recognition apparatus, or may be an external interface (corresponding to the communication interface 23) of the image recognition apparatus.

It is to be noted that the configuration shown in fig. 2 (or fig. 3) does not constitute a limitation of the image recognition apparatus, and the image recognition apparatus may include more or less components than those shown in fig. 2 (or fig. 3), or combine some components, or a different arrangement of components, in addition to the components shown in fig. 2 (or fig. 3).

The image recognition method provided by the embodiment of the present application will be described in detail below with reference to the image recognition system shown in fig. 1 and the image recognition apparatus shown in fig. 2 (or fig. 3).

Fig. 4 is a schematic flowchart of an image recognition method according to an embodiment of the present application. As shown in fig. 4, the image recognition method includes the following S401-S404.

S401, the image recognition device obtains a license plate image of the vehicle to be processed.

The image recognition device can firstly acquire the image of the vehicle to be processed, and then determine the license plate image corresponding to the vehicle to be processed in the image of the vehicle to be processed according to the target detection algorithm.

The target detection algorithm is a third version of target detection algorithm (you only look once, YOLO V3). The output matrices of YOLO V3 are 13 × 18, 26 × 18, and 52 × 18, respectively.

Specifically, the image recognition device may obtain a picture or a video including a vehicle through a camera or a camera, and then perform vehicle detection through YOLO V3 to extract a license plate image of the vehicle to be detected. The YOLO V3 body uses a dark net-53 neural network structure, and its output is the result of three different scales, namely, the output matrix is 13 × 18, 26 × 18, 52 × 18 corresponding to the three scales.

The output matrices of the available YOLO V3 are 13 × 255, 26 × 255, and 52 × 255, respectively, and 80 kinds of objects can be detected. In the embodiment of the present application, the output matrix of the YOLO V3 is changed to 13 × 18, 26 × 18, and 52 × 18, and only the license plate image of the vehicle needs to be detected. Thus, the YOLO V3 in the embodiment of the present application is lighter and operates faster.

S402, inputting the license plate image into a multitask convolutional neural network (MTCNN) model by the image recognition device to obtain the shape of the license plate image.

After obtaining a license plate image of a vehicle to be processed, an image recognition device inputs the license plate image into a multi-task convolutional neural network (MTCNN) model to obtain a frame angle of the license plate image, and then determines an original shape of the license plate image according to the frame angle of the license plate image.

The MTCNN model comprises a three-layer network structure, and the classification output of the three-layer network structure is F x 2; the regression outputs of the three-layer network structure are all F × 4; the detection output of the key points of the three-layer network structure is F × 8; the classified output is used for judging whether the license plate image comprises the license plate or not; the regression output is used for judging the relative offset of the abscissa of the upper left corner of the frame corner of the license plate image, the relative offset of the ordinate of the frame corner of the license plate image, the error of the width of the frame corner of the license plate image and the error of the height of the frame corner of the license plate image.

Specifically, the image recognition device firstly sends the license plate image into a first-layer network structure P-Net of the MTCNN model to obtain some candidate frames possibly containing the license plate. Then, improving the P-Net to output classification as M × 2, and judging whether the license plate is included; the regression output is M4, namely representing the relative offset of the abscissa of the upper left corner of the frame, the relative offset of the ordinate of the upper left corner of the frame, the error of the width of the frame and the error of the height of the frame; the detection output of the key points is M x 8, and the coordinate values of the upper left corner, the lower left corner, the upper right corner and the lower right corner of the corresponding license plate are obtained.

And then, intercepting and integrating the frames obtained by the P-Net into a matrix from the original image to be used as the input of a second-layer network structure R-Net of the MTCNN model, and further screening the R-Net to obtain more accurate candidate frames containing the license plate. Improving R-Net to make the classified output of R-Net be N x 2; the regression output is N × 4; the keypoint detection output is N x 8.

Then, taking a more accurate frame obtained by R-Net as the input of a third-layer network structure O-Net of the MTCNN model, and improving the O-Net to enable the O-Net to be classified and output as V x 2; the regression output is V4; the keypoint detection output is V x 8.

And finally, training the improved model, and obtaining A, B, C, D coordinate values of four corners of the license plate image by O-Net. The coordinate values A, B, C, D of the four corners of the license plate image are the frame corners of the license plate image. Subsequently, the image recognition device determines the original shape of the license plate image according to the frame angle of the license plate image.

And if the original shape of the license plate image is an inclined shape, correcting the license plate image in the inclined shape according to an affine transformation method to obtain a license plate image in a non-inclined shape, and determining the shape of the corrected license plate image as the shape of the license plate image.

Specifically, the affine transformation can be described by the following transformation matrix:

wherein, a, b, c and d are transformation parameters, and different parameters can transform the existing x and y coordinate axes into different forms, namely the inclination can be corrected. The a, b, c and d corresponding to different transformations are different in constraint, all affine transformations excluding translation transformation are linearly transformed, and the method is characterized in that the position of an origin is unchanged, and the result of multiple linear transformations is still linearly transformed. To cover translation, homogeneous coordinates are introduced, and 1 dimension is increased on the basis of the original 2-dimensional coordinates, as follows:

therefore, the transformation matrix of affine transformation is unified

To describe that the a, b, c, d, e, f constraints of different basis transforms are different.

And if the original shape of the license plate image is a non-inclined shape, determining the original shape of the license plate image as the shape of the license plate image.

And S403, if the shape of the license plate image does not meet the preset shape, deleting the license plate image by the image recognition device.

Specifically, after determining the shape of the license plate image, the image recognition device deletes the license plate image if the area of the license plate image is smaller than a manually set threshold, that is, removes the license plate image with low resolution. This step prevents the recognition of remote license plates and focuses on detecting close-range license plates.

S404, if the shape of the license plate image meets the preset shape, the image recognition device recognizes the color of the license plate image according to a clustering algorithm, and determines the recognized color as the color of the license plate of the vehicle to be processed.

And if the shape of the license plate image meets the preset shape, the image recognition device firstly performs saturation enhancement processing on the corrected license plate image.

Optionally, when the image recognition device performs saturation enhancement processing on the corrected license plate image, the RGB image values may be normalized to [0, 1 ]. The color space conversion is then performed using the color space conversion function in the image processing framework openCV.

The color space conversion function is used for converting an image from one color space to another color space (which is supported by the color space commonly used at present), and the type of data can be ensured to be unchanged in the conversion process, namely the data type and the bit depth of the converted image are consistent with those of the source image. The saturation and brightness components may then be adjusted according to a process gray scale image contrast enhancement gamma transform or linear transform, and finally converted to the RGB color space.

Then, the image recognition device clusters the RGB values of the converted license plate image through a K-means clustering algorithm, outputs a clustering result of the RGB values of each point of the image, counts the results after the K-means clustering, and outputs the result with the largest statistical value, namely the license plate color.

The embodiment of the application provides an image identification method, which comprises the following steps: the image recognition device firstly obtains a license plate image of a vehicle to be processed, and then inputs the license plate image into a multitask convolution neural network MTCNN model to obtain the shape of the license plate image. Subsequently, if the shape of the license plate image meets the preset shape, the image recognition device recognizes the color of the license plate image according to a clustering algorithm, and determines the recognized color as the color of the license plate of the vehicle to be processed.

Optionally, in combination with fig. 4, as shown in fig. 5, S401 may be replaced by S501-S502.

S501, an image recognition device acquires an image of a vehicle to be processed;

s502, the image recognition device determines a license plate image corresponding to the vehicle to be processed in the image of the vehicle to be processed according to a target detection algorithm.

Optionally, in conjunction with fig. 5, as shown in fig. 6, S402 may be replaced with S601-S604.

S601, the image recognition device inputs the license plate image into the MTCNN model to obtain a frame corner of the license plate image.

S602, the image recognition device determines the original shape of the license plate image according to the frame angle of the license plate image.

S603, if the original shape of the license plate image is the inclined shape, the image recognition device corrects the license plate image in the inclined shape according to an affine transformation method to obtain a license plate image in a non-inclined shape, and the shape of the corrected license plate image is determined as the shape of the license plate image.

S604, if the original shape of the license plate image is a non-inclined shape, the image recognition device determines the original shape of the license plate image as the shape of the license plate image.

The scheme provided by the embodiment of the application is mainly introduced from the perspective of a method. To implement the above functions, it includes hardware structures and/or software modules for performing the respective functions. Those of skill in the art will readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiment of the present application, the image recognition apparatus may be divided into the functional modules according to the method example, for example, each functional module may be divided according to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. Optionally, the division of the modules in the embodiment of the present application is schematic, and is only a logic function division, and there may be another division manner in actual implementation.

Fig. 7 is a schematic structural diagram of an image recognition apparatus 70 according to an embodiment of the present disclosure. The image recognition device 70 is used to solve the technical problems of large error and low accuracy of the existing method for recognizing the color of the license plate, for example, to execute the image recognition method shown in fig. 4, 5 or 6. The image recognition apparatus 70 includes: an acquisition unit 701, a processing unit 702 and a recognition unit 703.

The acquiring unit 701 is configured to acquire a license plate image of a vehicle to be processed. For example, in conjunction with fig. 4, the acquisition unit 701 is configured to perform S401.

And the processing unit 702 is configured to input the license plate image acquired by the acquisition unit 701 into a multitask convolutional neural network MTCNN model to obtain a shape of the license plate image. For example, in conjunction with fig. 4, the processing unit 702 is configured to execute S402.

The recognition unit 703 is configured to, if the shape of the license plate image obtained by the processing unit 702 meets a preset shape, recognize a color of the license plate image according to a clustering algorithm, and determine the recognized color as the color of the license plate of the vehicle to be processed. For example, in conjunction with fig. 4, the recognition unit 703 is configured to perform S404.

Optionally, the obtaining unit 701 is specifically configured to:

an image of a vehicle to be processed is acquired. For example, in conjunction with fig. 5 or fig. 6, the acquisition unit 701 is configured to perform S501.

And determining a license plate image corresponding to the vehicle to be processed in the image of the vehicle to be processed according to a target detection algorithm. For example, in conjunction with fig. 5 or fig. 6, the acquisition unit 701 is configured to execute S502.

Optionally, the target detection algorithm is a third version of target detection algorithm YOLO V3. The output matrices of YOLO V3 are 13 × 18, 26 × 18, and 52 × 18, respectively.

Optionally, the processing unit 702 is specifically configured to:

and inputting the license plate image into the MTCNN model to obtain a frame corner of the license plate image. For example, in connection with fig. 6, the processing unit 702 is configured to execute S601.

And determining the original shape of the license plate image according to the frame angle of the license plate image. For example, in conjunction with fig. 6, the processing unit 702 is configured to execute S602.

And if the original shape of the license plate image is an inclined shape, correcting the license plate image in the inclined shape according to an affine transformation method to obtain a license plate image in a non-inclined shape, and determining the shape of the corrected license plate image as the shape of the license plate image. For example, in conjunction with fig. 6, the processing unit 702 is configured to execute S603.

And if the original shape of the license plate image is a non-inclined shape, determining the original shape of the license plate image as the shape of the license plate image. For example, in conjunction with fig. 6, the processing unit 702 is configured to execute S604.

Optionally, the MTCNN model includes a three-layer network structure, and the classification outputs of the three-layer network structure are F × 2. The regression outputs of the three-layer network structure are all F4. The detection output of the key points of the three-layer network structure is F x 8. And the classified output is used for judging whether the license plate image comprises the license plate. The regression output is used for judging the relative offset of the abscissa of the upper left corner of the frame corner of the license plate image, the relative offset of the ordinate of the frame corner of the license plate image, the error of the width of the frame corner of the license plate image and the error of the height of the frame corner of the license plate image.

Embodiments of the present application also provide a computer-readable storage medium, which includes computer-executable instructions. When the computer executes the instructions to run on the computer, the computer is caused to execute the steps executed by the image recognition device in the image recognition method provided by the above embodiment.

The embodiment of the present application further provides a computer program product, where the computer program product is directly loadable into a memory and contains a software code, and the computer program product is loaded and executed by a computer, so as to implement the steps executed by the image recognition device in the image recognition method provided by the foregoing embodiment.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented using a software program, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The processes or functions according to the embodiments of the present application are generated in whole or in part when the computer-executable instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). Computer-readable storage media can be any available media that can be accessed by a computer or can comprise one or more data storage devices, such as servers, data centers, and the like, that can be integrated with the media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

Through the above description of the embodiments, it is clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules or units is only one logical function division, and there may be other division ways in actual implementation. For example, various elements or components may be combined or may be integrated into another device, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form. Units described as separate parts may or may not be physically separate, and parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed to a plurality of different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially or partially contributed to by the prior art, or all or part of the technical solutions may be embodied in the form of a software product, where the software product is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An image recognition method, comprising:

acquiring a license plate image of a vehicle to be processed;

inputting the license plate image into a multitask convolutional neural network (MTCNN) model to obtain the shape of the license plate image;

and if the shape of the license plate image meets the preset shape, identifying the color of the license plate image according to a clustering algorithm, and determining the identified color as the color of the license plate of the vehicle to be processed.

2. The image recognition method of claim 1, wherein the obtaining of the license plate image of the vehicle to be processed comprises:

acquiring an image of a vehicle to be processed;

and determining a license plate image corresponding to the vehicle to be processed in the image of the vehicle to be processed according to a target detection algorithm.

3. The image recognition method of claim 2, wherein the target detection algorithm is a third version target detection algorithm YOLO V3; the output matrices of YOLO V3 are 13 × 18, 26 × 18, and 52 × 18, respectively.

4. The image recognition method of claim 1, wherein the inputting the license plate image into a multitask convolutional neural network (MTCNN) model to obtain a shape of the license plate image comprises:

inputting the license plate image into the MTCNN model to obtain a frame corner of the license plate image;

determining the original shape of the license plate image according to the frame angle of the license plate image;

if the original shape of the license plate image is an inclined shape, correcting the license plate image in the inclined shape according to an affine transformation method to obtain a license plate image in a non-inclined shape, and determining the shape of the corrected license plate image as the shape of the license plate image;

5. The image recognition method of claim 4, wherein the MTCNN model comprises a three-tier network structure, the classification outputs of which are each F x 2; the regression output of the three-layer network structure is F4; the detection output of the key points of the three-layer network structure is F × 8; the classified output is used for judging whether the license plate image comprises the license plate; the regression output is used for judging the relative offset of the abscissa of the upper left corner of the frame corner of the license plate image, the relative offset of the ordinate of the frame corner of the license plate image, the error of the width of the frame corner of the license plate image and the error of the height of the frame corner of the license plate image.

6. An image recognition apparatus, comprising: the device comprises an acquisition unit, a processing unit and an identification unit;

the acquisition unit is used for acquiring a license plate image of a vehicle to be processed;

the processing unit is used for inputting the license plate image acquired by the acquiring unit into a multitask convolutional neural network (MTCNN) model to obtain the shape of the license plate image;

and the recognition unit is used for recognizing the color of the license plate image according to a clustering algorithm if the shape of the license plate image obtained by the processing unit meets a preset shape, and determining the recognized color as the color of the license plate of the vehicle to be processed.

7. The image recognition device according to claim 6, wherein the obtaining unit is specifically configured to:

acquiring an image of a vehicle to be processed;

8. The image recognition device of claim 7, wherein the target detection algorithm is a third version target detection algorithm YOLO V3; the output matrices of YOLO V3 are 13 × 18, 26 × 18, and 52 × 18, respectively.

9. The image recognition device according to claim 6, wherein the processing unit is specifically configured to:

10. The image recognition apparatus of claim 9, wherein the MTCNN model comprises a three-tier network structure, the classification outputs of which are each F x 2; the regression output of the three-layer network structure is F4; the detection output of the key points of the three-layer network structure is F × 8; the classified output is used for judging whether the license plate image comprises the license plate; the regression output is used for judging the relative offset of the abscissa of the upper left corner of the frame corner of the license plate image, the relative offset of the ordinate of the frame corner of the license plate image, the error of the width of the frame corner of the license plate image and the error of the height of the frame corner of the license plate image.

11. An image recognition apparatus comprising a memory and a processor; the memory is used for storing computer execution instructions, and the processor is connected with the memory through a bus;

when the image recognition apparatus is running, the processor executes the computer-executable instructions stored by the memory to cause the image recognition apparatus to perform the image recognition method of any one of claims 1-5.

12. A computer-readable storage medium comprising computer-executable instructions that, when executed on a computer, cause the computer to perform the image recognition method of any one of claims 1-5.