CN110889858A - Automobile part segmentation method and device based on point regression - Google Patents
Automobile part segmentation method and device based on point regression Download PDFInfo
- Publication number
- CN110889858A CN110889858A CN201911221210.2A CN201911221210A CN110889858A CN 110889858 A CN110889858 A CN 110889858A CN 201911221210 A CN201911221210 A CN 201911221210A CN 110889858 A CN110889858 A CN 110889858A
- Authority
- CN
- China
- Prior art keywords
- automobile
- data
- heat map
- batch
- point detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000011218 segmentation Effects 0.000 title claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 45
- 238000001514 detection method Methods 0.000 claims abstract description 36
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims abstract description 17
- 230000008569 process Effects 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 3
- 239000000306 component Substances 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 238000004091 panning Methods 0.000 description 3
- 238000013515 script Methods 0.000 description 3
- 238000010923 batch production Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 241000894007 species Species 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000037081 physical activity Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a point regression-based automobile part segmentation method, which comprises the following steps: a. training a key point detection model of the automobile; b. extracting regional picture data from a picture to be detected based on an automobile key point detection model; c. processing the regional picture data based on an automobile key point detection engine, and obtaining heat map data; d. calculating corresponding coordinates of key points of the automobile according to the heat map data; e. connecting the associated key points of the automobile, thereby segmenting specific parts of the automobile; in step a, the method comprises the following steps: i. dividing all training data into a plurality of batchs, wherein the size of each batch is 16; performing data enhancement operation on the data in each batch, inputting the data after the enhancement operation into an artificial intelligence network model, and calculating an output heat map; calculating a loss value for each batch from the output heatmap and the real tag; and iv, performing back propagation according to the loss value, and updating the network weight of the artificial intelligence network model. The invention has simple operation, convenient use and extremely high commercial value.
Description
Technical Field
The invention belongs to the field of computer application, and particularly relates to an automobile part segmentation method and device based on point regression.
Background
SPPE is a single attitude estimation English acronym, a specific network structure name is pyraNet, and human body attitude estimation is a key problem in the field of computer vision and can be applied to the aspects of human body activity analysis, human-computer interaction, video monitoring and the like. The human body posture estimation mainly refers to the detection of the position, direction and scale information of each part of the human body from the image. Human body posture estimation is often promoted by people in a video tracking environment, and due to the foundation and convenience of human body posture estimation based on monocular images, the research focus of human body posture estimation is turned to static picture human body posture estimation in recent years by foreign and domestic schools.
Flow convnet in 2015 regarded attitude estimation as a detection problem and the output was heatmap. The innovation point is that the internal relation between the joint points is extracted from 3 and 7 layers of a convolution neural network and then is called as a space fusion model through convolution operation; while using the optical flow information to align the heatmap prediction of adjacent frames. Finally, the heatmaps of the parameters are merged into a scoremap by using a parameter pooling method. Original text linking: https:// blog.csdn.net/qq _ 36165459/article/details/78320535.
How to apply the single posture estimation applied in the human body to the automobile, namely, by performing SPPE analysis on the whole automobile, parts of each part of the automobile are segmented, which can make great contribution to the corresponding field development of the computer, and how to analyze and segment the automobile, thereby forming each part of the automobile becomes a technical problem to be solved urgently at present, but at present, there is no technical scheme capable of solving the technical problem, and specifically, there is a method and a device for segmenting automobile parts based on point regression.
Disclosure of Invention
Aiming at the technical defects in the prior art, the invention aims to provide an automobile part segmentation method based on point regression, which comprises the following steps:
a. training a key point detection model of the automobile;
b. extracting regional picture data from a picture to be detected based on the automobile key point detection model, wherein the content of the picture to be detected comprises an automobile;
c. processing the region picture data based on an automobile key point detection engine, and obtaining heat map data;
d. calculating corresponding coordinates of key points of the automobile according to the heat map data;
e. connecting the associated key points of the automobile, thereby segmenting specific parts of the automobile;
in the step a, the method comprises the following steps:
v. dividing all training data into a plurality of batchs, wherein each batch is 16 in size;
vi, performing data enhancement operation on the data in each batch, inputting the data subjected to the enhancement operation into an artificial intelligence network model, and calculating an output heat map;
calculating a loss value for each batch from the output heatmap and the real tag;
and viii, performing back propagation according to the loss value, and updating the network weight of the artificial intelligence network model.
Preferably, in the step iv, the step is realized by:
iv1, counting losses of all pixel points according to the predicted heat map;
and iv2, arranging loss values of all the pixel points in a descending order, and according to the ratio of 1: 4, screening positive and negative samples according to the proportion;
and iv3, summing the losses of the screened positive and negative samples.
Preferably, in the step i, the stacking process is performed using less than 8 hourglass.
Preferably, in the step i, the stacking process is performed using not more than 6 hourglass.
Preferably, in the step i, a stacking process is performed using 5 ourglass.
Preferably, in step i, the number of convolution kernels of the first convolution layer conv1 is 3 × 3 convolution kernels.
Preferably, the artificial intelligence network model is a PyraNet based on human body key point detection network structure.
According to another aspect of the present invention, there is provided an automobile part segmentation apparatus based on point regression, including:
the first processing device trains the automobile key point detection model;
the first acquisition device extracts area picture data from a picture to be detected based on the automobile key point detection model;
the second acquisition device processes the regional picture data based on the automobile key point detection engine and obtains heat map data;
the first computing device: calculating corresponding coordinates of key points of the automobile according to the heat map data;
a second processing device: connecting the associated key points of the automobile, thereby segmenting specific parts of the automobile;
the first processing device includes:
a third processing device: dividing all training data into a plurality of batchs, wherein the size of each batch is 16;
the second computing device: performing data enhancement operation on the data in each batch, inputting the data subjected to the enhancement operation into an artificial intelligence network model, and calculating an output heat map;
the third calculating means: calculating the loss value of each batch according to the output heat map and the real label;
a fourth processing device: and performing back propagation according to the loss value, and updating the network weight of the artificial intelligent network model.
Preferably, the fourth processing device includes:
a fifth processing device: counting losses of all pixel points according to the predicted heat map;
a sixth processing device: and (3) arranging loss values of all the pixel points in a descending order, and according to the ratio of 1: 4, screening positive and negative samples according to the proportion;
the fourth calculating means: and summing the losses of the screened positive and negative samples.
The invention discloses a point regression-based automobile part segmentation method, which comprises the following steps of: a. training a key point detection model of the automobile; b. extracting regional picture data from a picture to be detected based on the automobile key point detection model, wherein the content of the picture to be detected comprises an automobile; c. processing the region picture data based on an automobile key point detection engine, and obtaining heat map data; d. calculating corresponding coordinates of key points of the automobile according to the heat map data; e. connecting the associated key points of the automobile, thereby segmenting specific parts of the automobile; in the step a, the method comprises the following steps: i. dividing all training data into a plurality of batchs, wherein the size of each batch is 16; performing data enhancement operation on the data in each batch, inputting the data after the enhancement operation into an artificial intelligence network model, and calculating an output heat map; calculating a loss value for each batch from the output heatmap and the real tag; and iv, performing back propagation according to the loss value, and updating the network weight of the artificial intelligence network model. The method can realize the analysis of the automobile picture through single posture estimation, realize the segmentation of each part of the automobile through establishing model analysis, and has the advantages of simple operation, convenient use and extremely high commercial value.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a schematic flow chart diagram illustrating a method for segmenting automobile parts based on point regression according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a specific process for training a key point detection model of a vehicle according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a specific process of updating the network weights of the artificial intelligence network model according to the back propagation of the loss values in the second embodiment of the present invention; and
fig. 4 shows an automobile component segmentation apparatus based on point regression according to another embodiment of the present invention.
Detailed Description
In order to better and clearly show the technical scheme of the invention, the invention is further described with reference to the attached drawings.
Fig. 1 shows a detailed flowchart of an automobile part segmentation method based on point regression according to an embodiment of the present invention, that is, the present invention completes the segmentation of an automobile part by SPPE, that is, single posture estimation, and reduces the scale of a model and the amount of calculation by innovating on the model of single posture estimation, thereby improving the operation efficiency, and specifically, as a main step of the present invention, provides an automobile part segmentation method based on point regression, which mainly realizes the segmentation of a specific part of an automobile based on a trained automobile model, and includes the following steps:
firstly, step S101 is performed to train a car key point detection model, and those skilled in the art understand that the invention may train the car key point detection model by using an open source data set, in a preferred embodiment, a hundred-degree Apollo open source data set may be used, further, the car key points are connection parts connected among various parts of the car, similar to single-person posture estimation, the car key points may be understood as an integral framework for composing, analyzing and constructing structures of various parts of the car, further, the car key point detection model may be obtained by analyzing, enhancing and calculating a plurality of pictures containing the car in big data, which is a technical core of the invention, and we will make further description in the specific embodiments described later, and will not be repeated herein.
Then, step S102 is performed, area picture data is extracted from a picture to be detected based on the car key point detection model, where the content of the picture to be detected includes cars, the picture to be detected is a picture to be divided, the picture to be detected includes at least one car, which may be a car, an SUV, or the like, and in other embodiments, the picture to be detected may further include a plurality of cars, further, the width and height of the picture are at least greater than 512, the picture to be detected preferably needs to be normalized before the car key point detection model is input, the area picture data is a part of the picture to be detected, which is preferably an area where the cars exist in the picture to be detected, which can obtain rectangular area picture data by cropping the picture to be detected, further, the position coordinates (xmin, ymin, width, height) of all cars in the picture are detected through a car detection model, and relevant area picture data are extracted.
Then, step S103 is performed, the image data of the area is processed based on the automobile key point detection engine, and heat map data is obtained, where the heat map data is heat map, the heat map can reflect data information in a two-dimensional matrix or a table by using color change, and it can visually represent the size of the data value by using defined color shades. The data are clustered according to abundance similarity among species or samples according to needs, the clustered data are represented on a heatmap, the high-abundance and low-abundance species can be clustered in blocks, and the similarity and difference of community compositions of a plurality of samples on each classification level are reflected through color gradient and similarity degree. And further, in a preferred embodiment, the picture to be detected includes a plurality of area picture data, that is, includes a plurality of car data, and all the car data are respectively input into the car key point detection model to respectively obtain the heatmaps corresponding to the cars.
After step S103, step S104 is executed, and corresponding coordinates of the key points of the automobile are calculated according to the heat map data, in such an embodiment, the automobile is preferably subjected to a coordinate processing, the positions of the key points of the automobile in the coordinate axes are determined by the key points of the automobile corresponding to the heat map in the heat map data, and then the corresponding coordinates of the key points of the automobile are calculated according to the heat map data.
Finally, step S105 is executed to connect the associated key points of the vehicle, so as to segment the specific components of the vehicle, in such an embodiment, the key points of the vehicle are the point data of the connection position of each specific component of the vehicle, after all the key points of the vehicle are determined, the key points of the vehicle only need to be connected in a certain manner, that is, a segmentation line is formed, and then the specific components in the vehicle are determined based on the segmentation line.
Fig. 2 shows a schematic flowchart of a specific process for training a vehicle key point detection model according to a first embodiment of the present invention, and those skilled in the art understand that fig. 2 is an important technical solution of the present invention, and is a more central part of the present invention, namely how to implement establishment of a vehicle key point model, specifically, the following steps are included:
first, in step S1011, all the training data is divided into a plurality of batches, each having a size of 16, and a Batch process (Batch) is also called a Batch script. As the name implies, batch processing is the processing of a batch of objects, generally known as a simplified scripting language, which is used in DOS and Windows systems. The extension of the batch file is bat. The currently more common batch processes include two types: DOS batch and PS batch. PS batch processing is based on Photoshop which is a powerful picture editing software and is used for batch processing scripts of pictures; DOS batch processing is based on DOS commands, and scripts for automatically executing DOS commands in batches to implement particular operations. More complicated cases require the use of if, for, goto, etc. commands to control the program's operation, as in high level languages such as C, Basic. If more complex applications are to be implemented, it is necessary to use external programs, including external commands provided by the system itself and tools or software provided by third parties. Batch programs, although running in a command line environment, can not only use command line software, any program currently running under the system can be put into a batch file to run.
Further, the training data is data used for establishing a model, and may be a plurality of automobiles, further, all the training data is divided into a plurality of batchs, and the size of each batch is 16, while in other embodiments, the number of the batchs may be 4, 8, 16 or more, further, the size of each batch may be 16, 32, 64 or even higher, which is not described herein again.
Then, step S1012 is performed, data enhancement is performed on the data in each batch, and the enhanced data is input into an artificial intelligence network model to calculate an output thermal map. How to obtain a large amount of data: one method is to obtain new data, which is cumbersome and requires a lot of cost, and the second method is to enhance the data, i.e. to create more data by using the existing data, such as flipping, translation or rotation, to make the neural network have better generalization effect.
Further, data-enhanced classification: data enhancement can be divided into two categories, one is offline enhancement and one is online enhancement. And (3) offline enhancement: the data set is directly processed, the number of data becomes the number of enhancement factors x original data sets, and the method is often used for online enhancement when the data set is very small: the enhanced method is used for enhancing the data of the batch after obtaining the data of the batch, and corresponding changes such as rotation, translation, turnover and the like are carried out on the data of the batch, the method is long for large data sets due to the fact that some data sets cannot accept the increase of linear level, and many machine learning frameworks support the data enhancement mode and can use GPU optimization calculation.
Further, commonly used data enhancement techniques: first, the enhancement factor is defined: refers to the multiple of the increase in data after offline enhancement. 1. Turning: enhancement factor 2 or 3, data flipping is a common data enhancement method, which differs from rotation 180 by making a mirror-like flip. 2. Rotating: the enhancement factor 2 to 4 rotation is clockwise or counterclockwise rotation, and it is noted that when rotating, it is better to rotate 90-180 degrees or dimension problem occurs, 3, zoom: enhancement factor arbitrary images can be enlarged or reduced. When enlarged, the size of the enlarged image will be larger than the original size. Most image processing architectures crop the enlarged image to its original size. 4. Cutting: enhancement factor the more popular name of any such method is random cropping, which is to select a part from an image at random, then crop the part, and then adjust the part to the size 5 and translation of the original image: enhancement factor arbitrary translation is moving the image in either the x or y direction (or both directions). We need to make assumptions about the background during panning, such as black, etc., because some images are empty during panning, and because objects in the images may appear at arbitrary positions, the panning enhancement method is very useful, 6, add noise: enhancement factors looking at the noise type overfitting at will usually occur when the neural network learns the high frequency features, which may not help the task of the neural network and may affect the low frequency features, and in order to eliminate the high frequency features, we add noise data randomly to eliminate the features, the above description of data enhancement belongs to the existing technology, and is not repeated herein.
And then, step S1013 is carried out, the loss value of each batch is calculated according to the output heat map and a real tag, the real tag is previous data which is not calculated by an artificial intelligence network model, and further, the loss value of each batch is obtained by comparing the output heat map with the real tag.
Finally, step S1014 is executed, back propagation is performed according to the loss value, and the network weight of the artificial intelligence network model is updated, and those skilled in the art understand that in this step, the loss of all pixel points is counted according to the predicted heat map; and (3) arranging loss values of all the pixel points in a descending order, and according to the ratio of 1: 4, screening positive and negative samples according to the proportion; the loss sums of the positive and negative samples are selected, and the network weight of the artificial intelligence network model is updated, which will be further described in the following detailed description, and will not be described herein.
Further, in the step S1011, the stacking process is performed using less than 8 hourglasss, the Hourglass is a network structure of the convolutional neural network, the Hourglass is a core component of the present application, and is composed of Residual modules, and has different complexity levels according to orders, while in another embodiment, the stacking process is performed using no more than 6 hourglasss, and in other embodiments, the stacking process may be performed using 5 hourglasss. Those skilled in the art understand that the original network structure, i.e. stacking 8 hourglasss in single-person attitude estimation, has found through experiments that in automobile attitude estimation, 6 units can be used to obtain closer accuracy, and the calculation amount can be reduced by 25%, thereby improving the calculation efficiency.
Further, in step S1011, the number of convolution kernels of the first convolution layer conv1 is 3 by 3 convolution kernels. Those skilled in the art will appreciate that in the original mesh structure, the number of convolution kernels of the first convolution layer conv1 is 7 × 7, and the 7 × 7 convolution kernels of the first convolution layer conv1 are replaced by 3 × 3 convolution kernels, having the same receptive field, while both the parameters and the computational load are reduced by nearly half.
Further, the artificial intelligence network model is based on a human body key point detection network structure PyraNet. The invention discloses a full-automatic automobile part intelligent segmentation method based on deep learning, which realizes accurate automobile part segmentation by estimating automobile part key points and connecting associated key points, SPPE estimates English acronyms for single posture, the specific network structure name is pyraNet, and human posture estimation is a key problem in the field of computer vision.
Fig. 3 is a schematic diagram illustrating a specific process of updating the network weights of the artificial intelligence network model according to the back propagation of the loss values in the second embodiment of the present invention, wherein the step S1014 includes the following steps:
firstly, step S10141 is performed, loss of all pixel points is counted according to the predicted heat map, and then in step S10142, loss values of all pixel points are arranged in a descending order according to the ratio of 1: 4, and finally summing the losses of the screened positive and negative samples in step S10413. Calculating loss values corresponding to all pixel points according to the predicted heat map, and arranging all loss values in descending order according to the ratio of 1: and 4, screening positive and negative samples according to the proportion, and summing the loss values of the screened positive and negative samples. The purpose is as follows: the original algorithm directly accumulates the loss of all pixel points, but the problem of imbalance between positive and negative samples (the proportion is less than 1: 100) exists, most of energy is spent on fitting negative sample information by the network, and after improvement, the loss value of the positive samples and the loss value of some difficult samples are reduced by the network more.
Fig. 4 shows an automobile component segmentation apparatus based on point regression according to another embodiment of the present invention. According to another aspect of the present invention, there is provided a point regression-based automobile component segmentation apparatus, including a first processing device for training an automobile key point detection model, and the basic working principle of the first processing device may refer to the foregoing step S101, which is not described herein again.
Further, the segmentation device further includes a first obtaining device, which extracts the region picture data from the picture to be detected based on the automobile key point detection model, and the basic working principle of the first obtaining device may refer to the step S102, which is not described herein again.
Further, the segmentation apparatus further includes a second obtaining apparatus, wherein the image data of the region is processed based on the automobile key point detection engine, and the heatmap data is obtained, and the basic working principle of the second obtaining apparatus may refer to the step S103, which is not described herein again.
Further, the segmentation means further comprises first computing means for: the corresponding coordinates of the key points of the vehicle are calculated according to the heat map data, and the basic working principle of the first calculation apparatus may refer to the step S104, which is not described herein again.
Further, the dividing device further includes a second processing device: the associated key points of the vehicle are connected to segment specific parts of the vehicle, and the basic working principle of the second processing device may refer to the foregoing step S105, which is not described herein again.
Further, the first processing means includes third processing means: all the training data are divided into a plurality of batchs, each of the batchs has a size of 16, and the basic working principle of the third processing device may refer to the step S1011, which is not described herein again.
Further, the first processing device further comprises a second computing device: the data in each batch is subjected to data enhancement operation, the enhanced data is input into the artificial intelligence network model, and the output heatmap is calculated, and the basic working principle of the second computing device may refer to the step S1012, which is not described herein again.
Further, the first processing means further comprises third computing means for: the loss value of each batch is calculated according to the output heatmap and the real tag, and the basic operation principle of the third calculating device may refer to the foregoing step S1013, which is not described herein again.
Further, the first processing device further includes a fourth processing device: and performing back propagation according to the loss value, and updating the network weight of the artificial intelligence network model, where the basic working principle of the fourth processing device may refer to the foregoing step S1014, which is not described herein again.
Further, the fourth processing means includes fifth processing means: the loss of all the pixels is counted according to the predicted heat map, and the basic working principle of the fifth processing device may refer to the step S10141, which is not repeated herein.
Further, the fourth processing device further includes a sixth processing device: and (3) arranging loss values of all the pixel points in a descending order, and according to the ratio of 1: 4, the basic working principle of the sixth processing device can refer to the foregoing step S10142, which is not described herein again.
Further, the fourth processing device further includes a fourth computing device: the basic operation principle of the fourth calculating means may refer to the foregoing step S10143, and details thereof are omitted here.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.
Claims (9)
1. A point regression-based automobile part segmentation method comprises the following steps:
a. training a key point detection model of the automobile;
b. extracting regional picture data from a picture to be detected based on the automobile key point detection model, wherein the content of the picture to be detected comprises an automobile;
c. processing the region picture data based on an automobile key point detection engine, and obtaining heat map data;
d. calculating corresponding coordinates of key points of the automobile according to the heat map data;
e. connecting the associated key points of the automobile, thereby segmenting specific parts of the automobile;
the method is characterized in that the step a comprises the following steps:
i. dividing all training data into a plurality of batchs, wherein the size of each batch is 16;
performing data enhancement operation on the data in each batch, inputting the data after the enhancement operation into an artificial intelligence network model, and calculating an output heat map;
calculating a loss value for each batch from the output heatmap and the real tag;
and iv, performing back propagation according to the loss value, and updating the network weight of the artificial intelligence network model.
2. The method according to claim 1, wherein in step iv, it is achieved by:
iv1, counting losses of all pixel points according to the predicted heat map;
and iv2, arranging loss values of all the pixel points in a descending order, and according to the ratio of 1: 4, screening positive and negative samples according to the proportion;
and iv3, summing the losses of the screened positive and negative samples.
3. Method according to claim 1 or 2, characterized in that in step i, a stacking process is performed using less than 8 hourglass.
4. A method according to claim 3, wherein in step i, no more than 6 hourglasss are used for the stacking process.
5. The method according to claim 4, wherein in the step i, a stacking process is performed using 5 hourglasss.
6. The method according to any one of claims 1 to 4, wherein in step i, the number of convolution kernels of the first convolution layer conv1 is 3 by 3 convolution kernels.
7. The method according to any one of claims 1 to 6, wherein the artificial intelligence network model is a human-based keypoint detection network architecture PyraNet.
8. An automobile part segmentation device based on point regression, comprising:
the first processing device (1) trains the automobile key point detection model;
the first acquisition device (2) extracts area picture data from a picture to be detected based on the automobile key point detection model;
the second acquisition device (3) processes the area picture data based on the automobile key point detection engine and obtains heat map data;
first computing means (4): calculating corresponding coordinates of key points of the automobile according to the heat map data;
second treatment device (5): connecting the associated key points of the automobile, thereby segmenting specific parts of the automobile;
the first processing device (1) comprises:
third processing device (11): dividing all training data into a plurality of batchs, wherein the size of each batch is 16;
second computing means (12): performing data enhancement operation on the data in each batch, inputting the data subjected to the enhancement operation into an artificial intelligence network model, and calculating an output heat map;
third calculation means (13): calculating the loss value of each batch according to the output heat map and the real label;
fourth processing device (14): and performing back propagation according to the loss value, and updating the network weight of the artificial intelligent network model.
9. The apparatus of claim 8, wherein the fourth processing means comprises:
fifth processing device (141): counting losses of all pixel points according to the predicted heat map;
sixth processing means (142): and (3) arranging loss values of all the pixel points in a descending order, and according to the ratio of 1: 4, screening positive and negative samples according to the proportion;
fourth calculation means (143): and summing the losses of the screened positive and negative samples.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911221210.2A CN110889858A (en) | 2019-12-03 | 2019-12-03 | Automobile part segmentation method and device based on point regression |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911221210.2A CN110889858A (en) | 2019-12-03 | 2019-12-03 | Automobile part segmentation method and device based on point regression |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110889858A true CN110889858A (en) | 2020-03-17 |
Family
ID=69750150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911221210.2A Pending CN110889858A (en) | 2019-12-03 | 2019-12-03 | Automobile part segmentation method and device based on point regression |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110889858A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111709884A (en) * | 2020-04-29 | 2020-09-25 | 高新兴科技集团股份有限公司 | License plate key point correction method, system, equipment and storage medium |
| CN116894844A (en) * | 2023-07-06 | 2023-10-17 | 北京长木谷医疗科技股份有限公司 | Hip joint image segmentation and key point linkage identification method and device |
Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105787439A (en) * | 2016-02-04 | 2016-07-20 | 广州新节奏智能科技有限公司 | Depth image human body joint positioning method based on convolution nerve network |
| US20180130229A1 (en) * | 2016-11-08 | 2018-05-10 | Nec Laboratories America, Inc. | Surveillance system with landmark localization on objects in images using convolutional neural networks |
| CN108229445A (en) * | 2018-02-09 | 2018-06-29 | 深圳市唯特视科技有限公司 | A kind of more people's Attitude estimation methods based on cascade pyramid network |
| CN108256431A (en) * | 2017-12-20 | 2018-07-06 | 中车工业研究院有限公司 | A kind of hand position identification method and device |
| CN108764048A (en) * | 2018-04-28 | 2018-11-06 | 中国科学院自动化研究所 | Face critical point detection method and device |
| CN109214282A (en) * | 2018-08-01 | 2019-01-15 | 中南民族大学 | A kind of three-dimension gesture critical point detection method and system neural network based |
| CN109376659A (en) * | 2018-10-26 | 2019-02-22 | 北京陌上花科技有限公司 | Training method, face critical point detection method, apparatus for face key spot net detection model |
| CN109657595A (en) * | 2018-12-12 | 2019-04-19 | 中山大学 | Based on the key feature Region Matching face identification method for stacking hourglass network |
| CN109711320A (en) * | 2018-12-24 | 2019-05-03 | 兴唐通信科技有限公司 | A kind of operator on duty's unlawful practice detection method and system |
| CN109766887A (en) * | 2019-01-16 | 2019-05-17 | 中国科学院光电技术研究所 | A kind of multi-target detection method based on cascade hourglass neural network |
| CN109919097A (en) * | 2019-03-08 | 2019-06-21 | 中国科学院自动化研究所 | Face and key point combined detection system, method based on multi-task learning |
| CN110084221A (en) * | 2019-05-08 | 2019-08-02 | 南京云智控产业技术研究院有限公司 | A kind of serializing face critical point detection method of the tape relay supervision based on deep learning |
| CN110175575A (en) * | 2019-05-29 | 2019-08-27 | 南京邮电大学 | A kind of single Attitude estimation method based on novel high-resolution network model |
| CN110175544A (en) * | 2019-05-14 | 2019-08-27 | 广州虎牙信息科技有限公司 | Construction method, device, electronic equipment and the storage medium of object module |
| CN110232696A (en) * | 2019-06-20 | 2019-09-13 | 腾讯科技（深圳）有限公司 | A kind of method of image region segmentation, the method and device of model training |
| CN110276316A (en) * | 2019-06-26 | 2019-09-24 | 电子科技大学 | A kind of human body critical point detection method based on deep learning |
| US10475182B1 (en) * | 2018-11-14 | 2019-11-12 | Qure.Ai Technologies Private Limited | Application of deep learning for medical imaging evaluation |
| CN110443222A (en) * | 2019-08-14 | 2019-11-12 | 北京百度网讯科技有限公司 | Method and apparatus for training face's critical point detection model |
| WO2019222383A1 (en) * | 2018-05-15 | 2019-11-21 | Northeastern University | Multi-person pose estimation using skeleton prediction |
| CN110490256A (en) * | 2019-08-20 | 2019-11-22 | 中国计量大学 | A kind of vehicle checking method based on key point thermal map |
| CN110503083A (en) * | 2019-08-30 | 2019-11-26 | 北京妙医佳健康科技集团有限公司 | A kind of critical point detection method, apparatus and electronic equipment |
-
2019
- 2019-12-03 CN CN201911221210.2A patent/CN110889858A/en active Pending
Patent Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105787439A (en) * | 2016-02-04 | 2016-07-20 | 广州新节奏智能科技有限公司 | Depth image human body joint positioning method based on convolution nerve network |
| US20180130229A1 (en) * | 2016-11-08 | 2018-05-10 | Nec Laboratories America, Inc. | Surveillance system with landmark localization on objects in images using convolutional neural networks |
| CN108256431A (en) * | 2017-12-20 | 2018-07-06 | 中车工业研究院有限公司 | A kind of hand position identification method and device |
| CN108229445A (en) * | 2018-02-09 | 2018-06-29 | 深圳市唯特视科技有限公司 | A kind of more people's Attitude estimation methods based on cascade pyramid network |
| CN108764048A (en) * | 2018-04-28 | 2018-11-06 | 中国科学院自动化研究所 | Face critical point detection method and device |
| WO2019222383A1 (en) * | 2018-05-15 | 2019-11-21 | Northeastern University | Multi-person pose estimation using skeleton prediction |
| CN109214282A (en) * | 2018-08-01 | 2019-01-15 | 中南民族大学 | A kind of three-dimension gesture critical point detection method and system neural network based |
| CN109376659A (en) * | 2018-10-26 | 2019-02-22 | 北京陌上花科技有限公司 | Training method, face critical point detection method, apparatus for face key spot net detection model |
| US10475182B1 (en) * | 2018-11-14 | 2019-11-12 | Qure.Ai Technologies Private Limited | Application of deep learning for medical imaging evaluation |
| CN109657595A (en) * | 2018-12-12 | 2019-04-19 | 中山大学 | Based on the key feature Region Matching face identification method for stacking hourglass network |
| CN109711320A (en) * | 2018-12-24 | 2019-05-03 | 兴唐通信科技有限公司 | A kind of operator on duty's unlawful practice detection method and system |
| CN109766887A (en) * | 2019-01-16 | 2019-05-17 | 中国科学院光电技术研究所 | A kind of multi-target detection method based on cascade hourglass neural network |
| CN109919097A (en) * | 2019-03-08 | 2019-06-21 | 中国科学院自动化研究所 | Face and key point combined detection system, method based on multi-task learning |
| CN110084221A (en) * | 2019-05-08 | 2019-08-02 | 南京云智控产业技术研究院有限公司 | A kind of serializing face critical point detection method of the tape relay supervision based on deep learning |
| CN110175544A (en) * | 2019-05-14 | 2019-08-27 | 广州虎牙信息科技有限公司 | Construction method, device, electronic equipment and the storage medium of object module |
| CN110175575A (en) * | 2019-05-29 | 2019-08-27 | 南京邮电大学 | A kind of single Attitude estimation method based on novel high-resolution network model |
| CN110232696A (en) * | 2019-06-20 | 2019-09-13 | 腾讯科技（深圳）有限公司 | A kind of method of image region segmentation, the method and device of model training |
| CN110276316A (en) * | 2019-06-26 | 2019-09-24 | 电子科技大学 | A kind of human body critical point detection method based on deep learning |
| CN110443222A (en) * | 2019-08-14 | 2019-11-12 | 北京百度网讯科技有限公司 | Method and apparatus for training face's critical point detection model |
| CN110490256A (en) * | 2019-08-20 | 2019-11-22 | 中国计量大学 | A kind of vehicle checking method based on key point thermal map |
| CN110503083A (en) * | 2019-08-30 | 2019-11-26 | 北京妙医佳健康科技集团有限公司 | A kind of critical point detection method, apparatus and electronic equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111709884A (en) * | 2020-04-29 | 2020-09-25 | 高新兴科技集团股份有限公司 | License plate key point correction method, system, equipment and storage medium |
| CN116894844A (en) * | 2023-07-06 | 2023-10-17 | 北京长木谷医疗科技股份有限公司 | Hip joint image segmentation and key point linkage identification method and device |
| CN116894844B (en) * | 2023-07-06 | 2024-04-02 | 北京长木谷医疗科技股份有限公司 | Hip joint image segmentation and key point linkage identification method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | Fully convolutional networks for surface defect inspection in industrial environment | |
| CN111914698B (en) | Human body segmentation method, segmentation system, electronic equipment and storage medium in image | |
| CN111696110B (en) | Scene segmentation method and system | |
| CN111950453A (en) | Optional-shape text recognition method based on selective attention mechanism | |
| CN113807355A (en) | Image semantic segmentation method based on coding and decoding structure | |
| CN110956126A (en) | Small target detection method combined with super-resolution reconstruction | |
| CN111401293B (en) | Gesture recognition method based on Head lightweight Mask scanning R-CNN | |
| JP2022510622A (en) | Image processing model training methods, image processing methods, network equipment, and storage media | |
| CN109886159B (en) | Face detection method under non-limited condition | |
| CN112001399B (en) | Image scene classification method and device based on local feature saliency | |
| CN109299305A (en) | A kind of spatial image searching system based on multi-feature fusion and search method | |
| CN113052755A (en) | High-resolution image intelligent matting method based on deep learning | |
| CN116229192B (en) | ODConvBS-YOLOv s-based flame smoke detection method | |
| CN113487610B (en) | Herpes image recognition method and device, computer equipment and storage medium | |
| CN110930378A (en) | Emphysema image processing method and system based on low data demand | |
| CN110852199A (en) | Foreground extraction method based on double-frame coding and decoding model | |
| CN114972312A (en) | Improved insulator defect detection method based on YOLOv4-Tiny | |
| Geng et al. | An improved helmet detection method for YOLOv3 on an unbalanced dataset | |
| CN111462184B (en) | Online sparse prototype tracking method based on twin neural network linear representation model | |
| CN110889858A (en) | Automobile part segmentation method and device based on point regression | |
| CN113496148A (en) | Multi-source data fusion method and system | |
| Shit et al. | An encoder‐decoder based CNN architecture using end to end dehaze and detection network for proper image visualization and detection | |
| CN111612803B (en) | Vehicle image semantic segmentation method based on image definition | |
| CN113177956A (en) | Semantic segmentation method for unmanned aerial vehicle remote sensing image | |
| CN116778164A (en) | Semantic segmentation method for improving deep V < 3+ > network based on multi-scale structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200317 |