WO2021175006A1

WO2021175006A1 - Vehicle image detection method and apparatus, and computer device and storage medium

Info

Publication number: WO2021175006A1
Application number: PCT/CN2021/070733
Authority: WO
Inventors: 丁晶晶
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2020-03-04
Filing date: 2021-01-08
Publication date: 2021-09-10
Also published as: CN111461170A

Abstract

Disclosed are a vehicle image detection method and apparatus, and a computer device and a storage medium. By improving the convolution kernel and pooling layers of a convolutional neural network, the convolutional neural network is expanded to support the convolution of 3D images. A box network is constructed, and the output features of each of 3D pooling layers are identified by using a three-dimensional sliding box, so that in a detection process, the position information of each component can be output, and by means of further convolution and pooling operations in a classification network, it can be determined whether the position information of each component is the position information of a damaged part, and the position information of the damaged part can be located more accurately while ensuring the efficiency of identifying each component.

Description

Vehicle image detection method, device, computer equipment and storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on March 4, 2020, the application number is 202010142987.6, and the invention title is "Vehicle image detection method, device, computer equipment and storage medium", the entire content of which is incorporated by reference Incorporated in this application.

Technical field

This application relates to the field of computer vision, and in particular to a vehicle image detection method, device, computer equipment and storage medium.

To

Background technique

With the development of science and technology, vehicles occupy an increasingly important position in people's daily lives. When the vehicle is damaged due to a collision or other conditions, it is necessary to determine the damage of the vehicle. At present, there is a technology to detect and identify the damaged part in the vehicle image through the image recognition algorithm. However, the inventor realizes that the current The recognition accuracy and detection accuracy of the adopted vehicle image recognition and detection technology are relatively low, so that it is impossible to accurately determine the degree of damage to vehicle components.

To

Application content

The embodiments of the present application provide a vehicle image detection method, device, computer equipment, and storage medium to solve the problem of low vehicle image detection and positioning accuracy.

A vehicle image detection method, including:

Acquiring an image to be identified, where the image to be identified is obtained after image processing of a vehicle image collected by a three-dimensional image acquisition device;

Acquiring a vehicle image recognition model, the vehicle image recognition model including a convolutional neural network and a frame network;

Input the image to be recognized into the convolutional neural network to obtain the output characteristics of the pooling layer of the convolutional neural network. The convolutional layer of the convolutional neural network adopts a 3D convolution kernel, and the pooling layer adopts 3D pooling layer;

Input the output features of the pooling layer into the frame network, and use a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified.

A vehicle image detection device, including:

The first acquisition module is configured to acquire an image to be identified, the image to be identified is obtained after image processing of a vehicle image acquired by a three-dimensional image acquisition device;

The second acquisition module is used to acquire a vehicle image recognition model, the vehicle image recognition model including a convolutional neural network and a frame network;

The first input module is used to input the image to be recognized into the convolutional neural network to obtain the output characteristics of the pooling layer of the convolutional neural network, and the convolutional layer of the convolutional neural network adopts a 3D volume Accumulate core, pooling layer adopts 3D pooling layer;

The second input module is used to input the output features of the pooling layer into the box network, and use a three-dimensional sliding box to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified .

A computer device includes a memory, a processor, and computer-readable instructions that are stored in the memory and can run on the processor, wherein the processor implements the following steps when the processor executes the computer-readable instructions:

One or more readable storage media storing computer readable instructions, where when the computer readable instructions are executed by one or more processors, the one or more processors execute the following steps:

The aforementioned vehicle image detection method, device, computer equipment and storage medium improve the convolution kernel and pooling layer of the convolutional neural network to expand the convolutional neural network to support the convolution of 3D images. Construct a frame network, and use a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer, so that in the detection process, the position information of each component can be output, and through further convolution and pooling in the classification network The operation can determine whether the position information of each component is the position information of the damaged part. Under the premise of ensuring the efficiency of identifying each component, the position information of the damaged part can be located more accurately.

The details of one or more embodiments of the present application are presented in the following drawings and description, and other features and advantages of the present application will become apparent from the description, drawings and claims.

To

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a schematic diagram of an application environment of a vehicle image detection method in an embodiment of the present application;

FIG. 2 is a flowchart of a vehicle image detection method in an embodiment of the present application;

FIG. 3 is another flowchart of the vehicle image detection method in an embodiment of the present application;

FIG. 4 is another flowchart of the vehicle image detection method in an embodiment of the present application;

FIG. 5 is another flowchart of the vehicle image detection method in an embodiment of the present application;

FIG. 6 is another flowchart of the vehicle image detection method in an embodiment of the present application;

Fig. 7 is a principle block diagram of the license plate verification device in an embodiment of the present application;

FIG. 8 is another principle block diagram of the license plate verification device in an embodiment of the present application;

FIG. 9 is another principle block diagram of the license plate verification device in an embodiment of the present application;

FIG. 10 is another principle block diagram of the license plate verification device in an embodiment of the present application;

11 is another principle block diagram of the license plate verification device in an embodiment of the present application;

Fig. 12 is a schematic diagram of a computer device in an embodiment of the present application.

To

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The embodiment of the present application provides a vehicle image detection method, and the image detection method can be applied in the application environment shown in FIG. 1. Specifically, the image detection method is applied in an image detection system. The image detection system includes a client and a server as shown in Figure 1. The client and the server communicate through the network to solve the problem of low vehicle image detection and positioning accuracy. . Among them, the client is also called the client, which refers to the program that corresponds to the server and provides local services to the client. The client can be installed on, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server can be implemented as an independent server or a server cluster composed of multiple servers.

In an embodiment, as shown in FIG. 2, a vehicle image detection method is provided. The method is applied to the server in FIG. 1 as an example for description, and includes the following steps:

S11: Acquire an image to be identified, where the image to be identified is obtained after image processing is performed on a vehicle image collected by a three-dimensional image acquisition device.

Among them, the image to be recognized is a vehicle image that needs to be recognized, and the image to be recognized is obtained after image processing is performed on a vehicle image collected by a three-dimensional image acquisition device. Specifically, the data structure of the vehicle image collected by the three-dimensional image acquisition device may be: 1*H*W*D, where 1 represents that the vehicle image is a single-channel image, and H is the length of the vehicle image. Data, W is the data of the width of the vehicle image, and D is the data of the depth of field of the vehicle image. Exemplarily, the data of the depth of field is set to 512, and the specific data of H and W depends on the pixel situation of the three-dimensional image acquisition device. Optionally, the three-dimensional image acquisition device may be a tof3d camera, a binocular camera, or other image acquisition devices that can acquire a depth channel.

The image processing steps may include image enhancement, normalization and other processing procedures. Preferably, the image processing process may further include: performing data reconstruction (resize) on H and W in the data structure of the image to be recognized, so as to set the values of H and W to 512, respectively. Optionally, data reconstruction may include the following parameter control methods: nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. Preferably, the data is reconstructed in the two directions of H and W. Therefore, the bilinear interpolation method can be used to reconstruct the data in the two directions of H and W.

Exemplarily, data reconstruction is performed on H and W in the vehicle image data structure, and the data of H and W are both 512, and the initial setting of D is 512. Therefore, after data reconstruction, the vehicle image data structure is: 1*512*512*512.

S12: Obtain a vehicle image recognition model, where the vehicle image recognition model includes a convolutional neural network and a frame network.

Wherein, the vehicle image recognition model is obtained by training using a convolutional neural network and a frame network, and is used to recognize the image to be recognized.

Specifically, the convolutional neural network includes five major layers, the first layer is the input layer, the second to fourth layers are intermediate layers, and the second to fourth layers each contain three small layers, one of which is The small layer contains a pooling layer, and the fifth layer is the output layer, which contains only one pooling layer.

The box network is constructed by combining the pooling layers of the second to fifth layers of the convolutional neural network, and the result of the pooling layers of the second to fifth layers of the convolutional neural network is used as the box The input of the network, the essence of the box network is a neural network classifier with sliding window, that is, on the second to fifth layers, 3D stereo boxes are used for sliding, and each stereo box is connected to the classifier to distinguish the boxes. Whether there is an object in and the type of the object in order to determine whether it is the location information of the target type.

S13: Input the image to be recognized into the convolutional neural network to obtain the output characteristics of the pooling layer of the convolutional neural network, and the convolutional layer of the convolutional neural network adopts a 3D convolution kernel to pool The layer adopts 3D pooling layer.

Wherein, the output feature of the pooling layer refers to the result of all pooling layers from the second layer to the fifth layer of the convolutional neural network. Specifically, the image to be recognized is input into the convolutional neural network, and the image to be recognized is subjected to a convolution operation from the input layer to the convolution layer, and the convolution of the image to be recognized can be obtained through the convolution operation. Layer output features, input the output features of the convolutional layer to the pooling layer, and perform a pooling operation from the convolutional layer to the pooling layer to obtain the output features of the pooling layer. The purpose of the pooling operation from the convolutional layer to the pooling layer is to reduce the number of output features of the convolutional layer obtained from the previous layer, and to further reduce the range of output features.

Optionally, the size of the 3D convolution kernel is: 3*3*3, and the size of the 3D pooling layer is: 2*2*2.

S14: Input the output features of the pooling layer into the frame network, and use a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified.

The three-dimensional sliding frame refers to a three-dimensional frame that slides on each pooling layer, and the size of the three-dimensional frame used by the three-dimensional sliding frame in pooling layers of different levels is different. The identification information of the image to be identified refers to the position information of each automobile component obtained when the three-dimensional sliding frame recognizes the output feature of each pooling layer in the 3D pooling layer.

Input the output features of the pooling layer into the box network, as the input of the box network, control the three-dimensional sliding box of each pooling layer to start sliding recognition at the starting point of the corresponding pooling layer, and perform two pixel intervals Sliding, as the three-dimensional sliding frame of each layer slides on the corresponding pooling layer, the output characteristics of the pooling layer are identified, and the identification information of the image to be identified, that is, the position information of each automobile component can be identified. Further, the size of the three-dimensional sliding box that recognizes the identification information of the image to be recognized in the box network is input to the classification network, and the middle layer of the classification network is a multi-layer convolutional network, and the size of the three-dimensional sliding box in the input layer Perform convolution and pooling operations. After the output layer is fully connected, softmax classification is performed to determine whether the identification information of the image to be recognized is the location information of the damaged part.

In this embodiment, by improving the convolution kernel and pooling layer of the convolutional neural network, the convolutional neural network is expanded to support the convolution of 3D images. Construct a frame network, and use a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer, so that in the detection process, the position information of each component can be output, and through further convolution and pooling in the classification network The operation can determine whether the position information of each component is the position information of the damaged part. Under the premise of ensuring the efficiency of identifying each component, the position information of the damaged part can be located more accurately.

In one embodiment, the vehicle image recognition model further includes a hotspot network, and after the vehicle image identification information is obtained, the vehicle image detection method further includes:

S15: If the identification information of the image to be identified is the first type of information, input the output feature of the pooling layer into a hotspot network, where the first type of information indicates that there is a damaged part in the image to be identified.

Specifically, after the three-dimensional sliding frame is used to identify the output features of each pooling layer in the 3D pooling layer, the position information of each car component in the image to be identified can be obtained, and after classification by the classification network, the position information can be obtained. Identify whether there is a damaged part in the image to be identified. If there is a damaged part in the image to be identified, the identification information of the image to be identified is the first type of information. Further, if the identification information of the image to be identified is the first type of information, the hotspot network will be activated, and the output characteristics of the pooling layer will be input into the hotspot network.

In another specific embodiment, if the identification information of the image to be identified is not the first type of information, that is, when there is no damaged part in the image to be identified, the hotspot network will not be activated, that is, the The output features of the pooling layer will not be input to the hotspot network.

S16: In the hotspot network, perform volume integration type recognition on the output features of the pooling layer to obtain hotspot area information.

Specifically, after the hotspot network is activated, the output characteristics of the pooling layer are input to the hotspot network, a convolutional network is connected between the input layer and the output layer of the hotspot network, and a convolutional network is adopted. Performing volume integral recognition on the output features of the pooling layer, identifying the key points of the key parts of the first type of information, and extracting the area information with the most concentrated number of lattice values of the key points as the hotspot area information, The output layer of the hotspot network outputs a three-dimensional one-hot bitmap.

Wherein, the key part is the location information of the damaged part. Since in each pooling layer from the third layer to the fifth layer, there will be multiple key points similar to the key parts, the number of lattice values of all the key points similar to the key parts is compared to select the key The area information where the number of dot matrix values is the most concentrated is used as the hot spot area information. Since the identification information of the image to be identified is identified by the classification network, multiple first-type information will be obtained, and there will be multiple key parts identified by the volume integral network in the hotspot network, so the hotspot area information There will be more than one.

S17: Perform median filtering on the hot spot area information to obtain filtered hot spot area information.

Specifically, performing median filtering on the hotspot area information refers to a process of setting the gray value of each pixel in the hotspot area information to the median of the gray values of all pixels in a certain neighborhood window of the point. Since a certain loss will be caused to the edge in the process of convolution and integration of the output features of the pooling layer, in order to protect the edge signal of the image, the median filter can effectively protect the edge signal.

Wherein, the size of the filter core of the median filter is 1/8 of the input size of the hotspot network, and is rounded up. Since the input size of the hotspot network is determined by the output characteristics of the pooling layer, the size of the filter kernels on the pooling layer of different levels is also different. Exemplarily, if the size of the second layer pooling layer is 256*256*256, the filter core size used on the second layer is: 32*32*32.

S18: Perform area maximum extraction on the filtered hotspot area information to obtain key position information of the image to be recognized.

Since there will be more than one hotspot area information, there will be more than one hotspot area information after filtering. Therefore, it is impossible to determine which one is the key point three-dimensional coordinate of the key part, that is, the key position information. The dot matrix values of the hot spot area information dot map are compared, and the area with the maximum dot matrix value of the filtered hot spot area information dot map is used as the key position information area of the image to be recognized.

Wherein, the key position information of the image to be recognized refers to the three-dimensional coordinates of the key point of the key part.

In this embodiment, if the frame network identifies the damaged part of the car in the output features of the pooling layer, the hot spot network is activated. The hot spot network can extract the key positions of the key parts identified in the frame network, and output the key The three-dimensional coordinates of the location, on the premise of ensuring that the damaged part can be accurately identified, output the three-dimensional coordinates of the damaged part, which improves the recognition accuracy and accuracy.

In an embodiment, extracting the maximum value of the area from the filtered hotspot area information to obtain the key position information of the image to be recognized includes:

S181: Obtain all the dot matrix values of the filtered hot spot area information dot matrix image.

Wherein, since the identification information of the image to be recognized is recognized by the classification network, multiple first-type information will be obtained, so that there will also be multiple key parts recognized by the volume integral network in the hotspot network, so the hotspot There will also be multiple area information, and there will also be multiple hot spot area information after filtering, and the dot matrix values of all the filtered hot spot area information bitmaps are extracted.

S182: Compare the dot matrix values of all filtered hot spot area information bitmaps, and extract the maximum value of the filtered hot spot area information.

Among them, after obtaining the dot matrix values of all the filtered hot spot area information bitmaps, the dot matrix values of all the filtered hot spot area information bitmaps are compared, and the points of the filtered hot spot area information bitmap are compared. The area with the maximum value is taken as the area of the key position information of the image to be recognized, and the maximum value in the area with the maximum value of the filtered hot spot area information dot matrix is extracted as the area to be recognized. The key position information of the recognition image, that is, the three-dimensional coordinates of the damaged part of the image to be recognized.

In this embodiment, since it is recognized that there can be many key parts including the damaged part, in order to further determine the key position of the damaged part, the dot matrix values of all the filtered hot spot area information bitmaps are compared, and the extracted area is the largest It can output the three-dimensional coordinates of the key position of the damaged part, which improves the recognition rate of the damaged part and the accuracy of positioning.

In one embodiment, the use of a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified includes:

S141: Determine the size and number of three-dimensional sliding boxes on each pooling layer according to the size of each pooling layer.

Among them, since in the convolutional neural network, the dimensionality of the convolutional neural network will be reduced by 2*2*2 every time it passes through a pooling layer, so the size and number of three-dimensional sliding boxes on each pooling layer The difference is that the size of the first layer of the convolutional neural network, that is, the input layer, is 512*512*512.

Specifically, the method for determining the size and number of the three-dimensional sliding boxes of each pooling layer is as follows:

The second layer: After the second layer of pooling layer, the dimension is reduced to 256*256*256, the second layer is divided into 16 equal parts, and the size of each three-dimensional sliding frame of the second layer is 16*16*16.

The third layer: After the third layer of pooling layer, the dimension is reduced to 128*128*128, the third layer is divided into 8 equal parts, and the equivalent size of each three-dimensional sliding frame of the third layer is 16*16*16 , But from the view point of view, the size of each three-dimensional sliding frame on the third layer is 32*32*32.

The fourth layer: After the fourth layer of pooling layer, the dimension is reduced to 64*64*64, the fourth layer is divided into 8 equal parts, and the equivalent size of each three-dimensional sliding frame of the fourth layer is 16*16*16 , But from the view point of view, the size of each three-dimensional sliding frame on the fourth layer is 64*64*64.

Fifth layer: After the fifth layer of pooling layer, the dimension is reduced to 32*32*32, the fifth layer is divided into 8 equal parts, and the equivalent size of each three-dimensional sliding frame of the fifth layer is 16*16*16 , But from the view point of view, the size of each three-dimensional sliding frame on the fifth layer is 128*128*128.

S142: Control the three-dimensional sliding frame to slide on each pooling layer, and identify the output characteristics of the pooling layer to obtain identification information.

The identification information refers to the frame selection information recognized during the sliding process of the three-dimensional sliding frame on each pooling layer. Specifically, input the output characteristics of the pooling layer into the box network, as the input of the box network, control the three-dimensional sliding box of each pooling layer to start sliding recognition at the starting point of the corresponding pooling layer, and use two pixels To slide at intervals, as the three-dimensional sliding frame of each layer slides on the corresponding pooling layer, the output characteristics of the pooling layer are identified, and the identification information can be identified.

S143: If the identification information is the location information of the target category, input the frame selection size of the three-dimensional sliding frame into the classification network to perform volume integration.

Wherein, the position information of the target type is the position information of each component identified by the three-dimensional sliding frame in the output feature of the pooling layer. In the process of identifying the output feature of the pooling layer, the three-dimensional sliding frame will determine whether the identification information is the position information of the detection component, and if it is the position information of the detection component, it will also determine the type of the detection component. If the position information of each component is included in the identification information, input the frame selection size of the three-dimensional sliding box into the classification network, train the convolutional neural network and the frame network, and further determine whether the position information of the target type is the damaged part Location information.

Specifically, the structure of the classification network includes:

The input layer is the selection size of the three-dimensional sliding box;

The middle layer, the middle layer is a multi-layer convolutional network, each convolutional network layer has a pooling layer, the step size of each pooling layer is 2, and the size of the last layer after convolution is 2*2* 2;

The output layer is fully connected on the output layer, and softmax classification is performed after the full connection.

In this embodiment, a three-dimensional sliding frame is used on the pooling layer through the frame network. In the process of recognizing the output characteristics of the pooling layer, it is possible to determine whether the recognition is a detection object, and to determine the type of the detection object. After detecting the type of the object and the object, the selection size of the three-dimensional sliding box is input to the classification network, and the volume integration class is performed on it, which improves the accuracy of recognition.

In an embodiment, before the obtaining the vehicle image recognition model, the vehicle image detection method further includes:

S19: Obtain a training sample set, where the training sample set includes a vehicle sample image and corresponding annotation data, and the vehicle sample image is obtained after image processing is performed on an image collected by a three-dimensional image collection device.

Wherein, the vehicle sample image is a vehicle sample image that needs to be trained, the annotation data is used to classify the vehicle sample image, and the name of the vehicle sample image classification is marked with a number. Specifically, the data structure of the vehicle sample image collected by the three-dimensional image acquisition device may be: 1*H*W*D, where 1 represents that the vehicle image is a single-channel image, and H is the length of the vehicle image W is the data of the width of the vehicle image, and D is the data of the depth of field of the vehicle image. Exemplarily, the data of the depth of field is set to 512, and the specific data of H and W depends on the pixel situation of the three-dimensional image acquisition device. Optionally, the three-dimensional image acquisition device may be a tof3d camera, a binocular camera, or other image acquisition devices that can acquire a depth channel.

Wherein, the image processing steps may include processing procedures such as image enhancement and normalization. Preferably, the method includes: adopting a resizing method in each depth channel. Optionally, the resizing method may include the following parameter control methods: nearest neighbor interpolation method, bilinear interpolation method, and bicubic interpolation method. Preferably, the bilinear interpolation method is adopted in this application.

The data of H and W in the vehicle image data structure are reconstructed, and the data of H and W are both 512, and the initial setting of D is 512. Therefore, after data reconstruction, the vehicle image data structure is: 1*512*512*512.

S20: Obtain a preset neural network model, where the preset neural network model includes a convolutional neural network, a box network, and a hotspot network.

Wherein, the preset neural network model is composed of a convolutional neural network, a frame network, and a hotspot network, and is used to perform recognition training on the training sample set.

Wherein, the structure of the convolutional neural network has 5 layers, the first layer is the input layer, the data structure of the input layer is: 512*512*512, the size of the convolution kernel is: 3*3*3, the first A total of 64 convolution kernels in the layer;

The second to fourth layers are intermediate layers. The size of the convolution kernel of the intermediate layer is: 3*3*3. When the longitudinal dimension passes through a large layer, the number of convolution kernels doubles, and the intermediate layer There are three small layers, one of which contains a pooling layer, and the step size of the pooling layer is 2, that is, after each pooling layer, the dimensionality of the data structure is reduced by 2*2*2;

The fifth layer is the output layer, the output layer contains only one pooling layer, and the step size of the pooling layer is 2.

The hotspot network is constructed by combining the pooling layers of the third to fifth layers of the convolutional neural network, and is constructed at the equivalent position of the frame network of the third to fifth layers, and the second of the convolutional neural network The result of the pooling layer from layer to the fifth layer is used as the input of the hotspot network. The output of the hotspot network is a three-dimensional one-hot bitmap. Consistent 0,x dot pattern, where x is a key part and 0 is a non-key part. A convolutional network is connected between the input layer and the output layer of the hotspot network.

S21: Use the training sample set to train the preset neural network model to obtain a vehicle image recognition model.

Wherein, the vehicle image recognition model is based on a preset neural network model and is a model obtained after training according to the training sample set. The process of establishing a vehicle image recognition model includes: inputting the training sample set into the preset neural network model, and defining the network of the convolutional neural network, the frame network and the hotspot network in the preset neural network model Structure, initialize the network weights, define the forward propagation process, use the defined forward propagation process to iteratively train the preset neural network model to obtain the trained model, and test and verify the trained model to obtain Vehicle image recognition model.

In this embodiment, the vehicle sample image is collected by the three-dimensional image acquisition device, and the data structure of the vehicle sample image is readjusted, and the length and width pixels are the same as the pixels in the set depth direction, which avoids different three-dimensional image acquisition The problem of the different pixels of the vehicle sample images collected by the equipment; the vehicle image recognition model obtained through the preset neural network model through the training sample set can be more accurate and quickly identify and determine the subsequent vehicle images that need to be recognized.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

In one embodiment, a vehicle image detection device is provided, and the vehicle image detection device corresponds to the vehicle image detection method in the above-mentioned embodiment in a one-to-one correspondence. As shown in FIG. 7, the vehicle image detection device includes a first acquisition module 11, a second acquisition module 12, a first input module 13 and a second input module 14. The detailed description of each functional module is as follows:

The first acquisition module 11 is configured to acquire an image to be identified, which is obtained after image processing of a vehicle image acquired by a three-dimensional image acquisition device;

The second acquisition module 12 is configured to acquire a vehicle image recognition model, the vehicle image recognition model including a convolutional neural network and a frame network;

The first input module 13 is configured to input the image to be recognized into the convolutional neural network to obtain the output characteristics of the pooling layer of the convolutional neural network, and the convolutional layer of the convolutional neural network adopts 3D Convolution kernel, pooling layer adopts 3D pooling layer;

The second input module 14 is used to input the output features of the pooling layer into the box network, and use a three-dimensional sliding box to identify the output features of each pooling layer in the 3D pooling layer to obtain the recognition of the image to be identified information.

Preferably, as shown in FIG. 8, the vehicle image detection device further includes:

The third input module 15 is configured to input the output characteristics of the pooling layer into the hotspot network when the identification information of the image to be recognized is the first type of information, and the first type of information indicates the to be recognized There are damaged parts in the image;

The first recognition module 16 is configured to perform volume integral recognition on the output features of the pooling layer in the hotspot network to obtain hotspot area information;

The median filtering module 17 is configured to perform median filtering on the hot spot area information to obtain filtered hot spot area information;

The information extraction module 18 is used for extracting the maximum value of the area from the filtered hot area information to obtain the key position information of the image to be recognized.

Preferably, as shown in FIG. 9, the information extraction module 18 further includes:

The third obtaining module 181 is configured to obtain the bitmap value of the filtered hot spot area information bitmap;

The first comparison module 182 is configured to compare the dot matrix values of the filtered hot spot area information dot map, and extract the maximum value of the filtered hot spot area information.

Preferably, as shown in FIG. 10, the second input module 14 further includes:

The first calculation module 141 is configured to determine the size and number of three-dimensional sliding frames on each pooling layer according to the size of each pooling layer;

The second recognition module 142 is configured to control the three-dimensional sliding frame to slide on each pooling layer, to identify the output characteristics of the pooling layer, and to obtain identification information;

The fourth input module 143 is configured to input the frame selection size of the three-dimensional sliding frame into the classification network to perform volume integration when the identification information is the location information of the target category.

Preferably, as shown in FIG. 11, the vehicle image detection device further includes:

The third acquisition module 19 is configured to acquire a training sample set, the training sample set includes a vehicle sample image and corresponding annotation data, the vehicle sample image is obtained after image processing is performed on an image collected by a three-dimensional image acquisition device;

The fourth acquisition module 20 is configured to acquire a preset neural network model, where the preset neural network model includes a convolutional neural network, a box network, and a hotspot network;

The first training module 21 is configured to use the training sample set to train the preset neural network model to obtain a vehicle image recognition model.

For the specific definition of the vehicle image detection device, please refer to the above definition of the vehicle image detection method, which will not be repeated here. Each module in the above-mentioned vehicle image detection device may be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 12. The computer equipment includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer readable instructions in the readable storage medium. The database of the computer equipment is used to store the data used in the above-mentioned vehicle image detection method. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer readable instruction is executed by the processor to realize a vehicle image detection method.

In one embodiment, a computer device is provided, including a memory, a processor, and computer readable instructions stored in the memory and capable of running on the processor, and the processor implements the following steps when the processor executes the computer readable instructions:

In one embodiment, one or more readable storage media storing computer readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage. Medium; the readable storage medium stores computer readable instructions, and when the computer readable instructions are executed by one or more processors, the one or more processors implement the following steps:

A person of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile computer. In a readable storage medium or a volatile computer readable storage medium, when the computer readable instruction is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as needed. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

To

Claims

A vehicle image detection method, wherein the vehicle image detection method includes:

Acquiring an image to be identified, where the image to be identified is obtained after image processing of a vehicle image collected by a three-dimensional image acquisition device;

Acquiring a vehicle image recognition model, the vehicle image recognition model including a convolutional neural network and a frame network;

Input the image to be recognized into the convolutional neural network to obtain the output characteristics of the pooling layer of the convolutional neural network. The convolutional layer of the convolutional neural network adopts a 3D convolution kernel, and the pooling layer adopts 3D pooling layer;

Input the output features of the pooling layer into the frame network, and use a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified.
The vehicle image detection method according to claim 1, wherein the vehicle image recognition model further comprises a hotspot network, and after the identification information of the vehicle image is obtained, the vehicle image detection method further comprises:

If the identification information of the image to be identified is the first type of information, input the output feature of the pooling layer into the hotspot network, where the first type of information indicates that there is a damaged part in the image to be identified;

In the hotspot network, perform volume integral type recognition on the output features of the pooling layer to obtain hotspot area information;

Performing median filtering on the hot spot area information to obtain filtered hot spot area information;

The area maximum value extraction is performed on the filtered hot area information to obtain the key position information of the image to be recognized.
The vehicle image detection method according to claim 2, wherein said extracting the maximum value of the area from the filtered hotspot area information to obtain the key position information of the image to be recognized comprises:

Acquiring the bitmap value of the filtered hot spot area information bitmap;

Comparing the dot matrix values of the filtered hot spot area information dot map, and extracting the maximum value of the filtered hot spot area information.
5. The vehicle image detection method according to claim 1, wherein said using a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified comprises:

According to the size of each pooling layer, determine the size and number of three-dimensional sliding boxes on each pooling layer;

Controlling the three-dimensional sliding frame to slide on each pooling layer, and identifying output features of the pooling layer to obtain identification information;

If the identification information is the location information of the target category, the frame selection size of the three-dimensional sliding frame is input into the classification network to perform volume integration.
5. The vehicle image detection method according to claim 1, wherein, before said obtaining the vehicle image recognition model, the vehicle image detection method further comprises:

Acquiring a training sample set, the training sample set including a vehicle sample image and corresponding annotation data, the vehicle sample image is obtained after image processing is performed on an image collected by a three-dimensional image acquisition device;

Acquiring a preset neural network model, where the preset neural network model includes a convolutional neural network, a box network, and a hotspot network;

The training sample set is used to train the preset neural network model to obtain a vehicle image recognition model.
A vehicle image detection device, which includes:

The first acquisition module is configured to acquire an image to be identified, the image to be identified is obtained after image processing of a vehicle image acquired by a three-dimensional image acquisition device;

The second acquisition module is used to acquire a vehicle image recognition model, the vehicle image recognition model including a convolutional neural network and a frame network;

The first input module is used to input the image to be recognized into the convolutional neural network to obtain the output characteristics of the pooling layer of the convolutional neural network, and the convolutional layer of the convolutional neural network adopts a 3D volume Accumulate core, pooling layer adopts 3D pooling layer;

The second input module is used to input the output features of the pooling layer into the box network, and use a three-dimensional sliding box to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified .
7. The vehicle image detection device of claim 6, wherein the vehicle image detection device further comprises:

The third input module is configured to input the output characteristics of the pooling layer into the hotspot network when the identification information of the image to be recognized is the first type of information, where the first type information indicates the image to be recognized There is an injury site in;

The first recognition module is configured to perform volume integration type recognition on the output features of the pooling layer in the hotspot network to obtain hotspot area information;

The median filtering module is configured to perform median filtering on the hot spot area information to obtain filtered hot spot area information;

The information extraction module is used for extracting the area maximum value from the filtered hot area information to obtain the key position information of the image to be recognized.
The vehicle image detection device according to claim 6, wherein the information extraction module further comprises:

The third acquiring module is used to acquire the dot matrix value of the filtered hot spot area information dot matrix image;

The first comparison module is configured to compare the dot matrix values of the filtered hot spot area information dot map, and extract the maximum value of the filtered hot spot area information.
The vehicle image detection device according to claim 6, wherein the second input module further comprises:

The first calculation module is used to determine the size and number of three-dimensional sliding boxes on each pooling layer according to the size of each pooling layer;

The second recognition module is used to control the three-dimensional sliding frame to slide on each pooling layer, to identify the output characteristics of the pooling layer, and to obtain identification information;

The fourth input module is used for inputting the frame selection size of the three-dimensional sliding frame into the classification network to perform volume and integral classification when the identification information is the position information of the target category.
7. The vehicle image detection device of claim 6, wherein the vehicle image detection device further comprises:

The third acquisition module is configured to acquire a training sample set, the training sample set includes a vehicle sample image and corresponding annotation data, the vehicle sample image is obtained after image processing is performed on an image collected by a three-dimensional image acquisition device;

The fourth acquisition module is used to acquire a preset neural network model, where the preset neural network model includes a convolutional neural network, a box network, and a hotspot network;

The first training module is configured to use the training sample set to train the preset neural network model to obtain a vehicle image recognition model.
A computer device includes a memory, a processor, and computer-readable instructions that are stored in the memory and can run on the processor, wherein the processor implements the following steps when the processor executes the computer-readable instructions:

Acquiring an image to be identified, where the image to be identified is obtained after image processing of a vehicle image collected by a three-dimensional image acquisition device;

Acquiring a vehicle image recognition model, the vehicle image recognition model including a convolutional neural network and a frame network;

Input the image to be recognized into the convolutional neural network to obtain the output characteristics of the pooling layer of the convolutional neural network. The convolutional layer of the convolutional neural network adopts a 3D convolution kernel, and the pooling layer adopts 3D pooling layer;

Input the output features of the pooling layer into the frame network, and use a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified.
The computer device of claim 11, wherein the vehicle image recognition model further includes a hotspot network, and after the vehicle image identification information is obtained, the processor also implements the computer readable instruction The following steps:

If the identification information of the image to be identified is the first type of information, input the output feature of the pooling layer into the hotspot network, where the first type of information indicates that there is a damaged part in the image to be identified;

In the hotspot network, perform volume integral type recognition on the output features of the pooling layer to obtain hotspot area information;

Performing median filtering on the hot spot area information to obtain filtered hot spot area information;

The area maximum value extraction is performed on the filtered hot area information to obtain the key position information of the image to be recognized.
The computer device according to claim 12, wherein said extracting the maximum value of the area from the filtered hotspot area information to obtain the key position information of the image to be recognized comprises:

Acquiring the bitmap value of the filtered hot spot area information bitmap;

Comparing the dot matrix values of the filtered hot spot area information dot map, and extracting the maximum value of the filtered hot spot area information.
11. The computer device according to claim 11, wherein said using a three-dimensional sliding frame to identify the output feature of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified comprises:

According to the size of each pooling layer, determine the size and number of three-dimensional sliding boxes on each pooling layer;

Controlling the three-dimensional sliding frame to slide on each pooling layer, and identifying output features of the pooling layer to obtain identification information;

If the identification information is the location information of the target category, the frame selection size of the three-dimensional sliding frame is input into the classification network to perform volume integration.
The computer device according to claim 11, wherein, before said acquiring the vehicle image recognition model, the processor further implements the following steps when executing the computer-readable instruction:

Acquiring a training sample set, the training sample set including a vehicle sample image and corresponding annotation data, the vehicle sample image is obtained after image processing is performed on an image collected by a three-dimensional image acquisition device;

Acquiring a preset neural network model, where the preset neural network model includes a convolutional neural network, a box network, and a hotspot network;

The training sample set is used to train the preset neural network model to obtain a vehicle image recognition model.
One or more readable storage media storing computer readable instructions, where when the computer readable instructions are executed by one or more processors, the one or more processors execute the following steps:

Acquiring an image to be identified, where the image to be identified is obtained after image processing of a vehicle image collected by a three-dimensional image acquisition device;

Acquiring a vehicle image recognition model, the vehicle image recognition model including a convolutional neural network and a frame network;

Input the image to be recognized into the convolutional neural network to obtain the output characteristics of the pooling layer of the convolutional neural network. The convolutional layer of the convolutional neural network adopts a 3D convolution kernel, and the pooling layer adopts 3D pooling layer;

Input the output features of the pooling layer into the frame network, and use a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified.
The readable storage medium of claim 16, wherein the vehicle image recognition model further comprises a hotspot network, and after the vehicle image identification information is obtained, the computer readable instructions are processed by one or more When the processor executes, the one or more processors further execute the following steps:

If the identification information of the image to be identified is the first type of information, input the output feature of the pooling layer into the hotspot network, where the first type of information indicates that there is a damaged part in the image to be identified;

In the hotspot network, perform volume integral type recognition on the output features of the pooling layer to obtain hotspot area information;

Performing median filtering on the hot spot area information to obtain filtered hot spot area information;

The area maximum value extraction is performed on the filtered hot area information to obtain the key position information of the image to be recognized.
17. The readable storage medium of claim 17, wherein said extracting the maximum value of the area from the filtered hotspot area information to obtain the key position information of the image to be recognized comprises:

Acquiring the bitmap value of the filtered hot spot area information bitmap;

Comparing the dot matrix values of the filtered hot spot area information dot map, and extracting the maximum value of the filtered hot spot area information.
15. The readable storage medium according to claim 16, wherein said using a three-dimensional sliding frame to identify the output features of each pooling layer in the 3D pooling layer to obtain the identification information of the image to be identified comprises:

According to the size of each pooling layer, determine the size and number of three-dimensional sliding boxes on each pooling layer;

Control the three-dimensional sliding frame to slide on each pooling layer, and identify the output characteristics of the pooling layer to obtain identification information;

If the identification information is the location information of the target category, the frame selection size of the three-dimensional sliding frame is input into the classification network to perform volume integration.
The readable storage medium according to claim 16, wherein, before the acquisition of the vehicle image recognition model, when the computer-readable instructions are executed by one or more processors, the one or more processors also Perform the following steps:

Acquiring a training sample set, the training sample set including a vehicle sample image and corresponding annotation data, the vehicle sample image is obtained after image processing is performed on an image collected by a three-dimensional image acquisition device;

Acquiring a preset neural network model, where the preset neural network model includes a convolutional neural network, a box network, and a hotspot network;

The training sample set is used to train the preset neural network model to obtain a vehicle image recognition model.

To