CN114663882B - Electric automobile chassis scratch three-dimensional detection method based on deep learning - Google Patents
Electric automobile chassis scratch three-dimensional detection method based on deep learning Download PDFInfo
- Publication number
- CN114663882B CN114663882B CN202210568310.8A CN202210568310A CN114663882B CN 114663882 B CN114663882 B CN 114663882B CN 202210568310 A CN202210568310 A CN 202210568310A CN 114663882 B CN114663882 B CN 114663882B
- Authority
- CN
- China
- Prior art keywords
- scratch
- camera
- point cloud
- deep learning
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/60—Other road transportation technologies with climate change mitigation effect
- Y02T10/70—Energy storage systems for electromobility, e.g. batteries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a deep learning-based three-dimensional detection method for scratches of an electric automobile chassis, which relates to the technical field of electric automobile chassis detection and comprises the following steps: s1, collecting pictures and making a data set; s2, performing data set training by using a deep learning method, extracting scratch characteristics and detecting a scratch area; s3, detecting the automobile chassis picture according to the obtained data model, and selecting the position where the scratch possibly occurs; and S4, obtaining a matrix from the RGB picture to the point cloud, intercepting the local point cloud picture, S5, processing the local point cloud picture, and acquiring the depth information of the scratch, thereby qualitatively judging the scratch. According to the invention, the plane graph is screened firstly, and then the deleted area is accurately detected, so that the detection efficiency is improved.
Description
Technical Field
The invention relates to the technical field of electric automobile chassis detection, in particular to a three-dimensional electric automobile chassis scratch detection method based on deep learning.
Background
Electric automobile chassis mar detection is crucial, and chassis mar degree of depth is too big, can lead to battery chassis assembly inner structure impaired, influences driving safety.
According to a traditional automobile chassis detection scheme, a line scan camera is adopted to obtain a chassis 2D image, then the image is processed through a deep learning method such as a faster-rcnn model, a mask-rcnn model and the like, and scratch coordinates are obtained; or some conventional image detection methods such as frequency domain processing, blob analysis, threshold segmentation, edge detection, etc. identify the defect location. The method is widely used for detecting the chassis assembly of the electric automobile before delivery, but is also easy to generate misjudgment phenomenon caused by surface characteristic change caused by environmental light, dirt and change of processing technological parameters. In the use process of the finished product electric vehicle, chassis scratch or crack detection is carried out, and the 2D image scheme can also be interfered, for example, the chassis is adhered with some dust, dirt and the like, so that the over-detection rate and the false detection rate are high, and the product detection effect is influenced.
When a 3D solution is adopted, a line structured light camera is usually adopted to obtain chassis 3D point cloud at present, key points are searched, ROI is segmented out, a plane is locally fitted, the distance from the point to the plane is obtained, and the depth of a local point cloud defect can be judged, so that whether scratches exist in the point cloud. The pure point cloud scheme can eliminate the influence of surface difference caused by dirt and processes, but the global search for scratches is easily influenced by the height unevenness of the automobile chassis, and the point cloud processing speed is slow, so that real-time detection cannot be realized.
In order to overcome the defects, technical personnel in the field actively innovate and research to create a three-dimensional detection method for scratches of the chassis of the electric automobile based on deep learning.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a deep learning-based three-dimensional detection method for scratches of an electric vehicle chassis.
In order to solve the technical problems, the invention adopts a technical scheme that: the three-dimensional detection method for the scratches of the electric automobile chassis based on deep learning is characterized by comprising the following steps: the method comprises the following steps: s1, collecting a picture of the automobile chassis by adopting an RGB-D camera, marking the position of the scratch by utilizing data set marking software, making the picture into a data set, and expanding the data set by adopting a data enhancement means;
s2, performing data set training by using a deep learning method, extracting scratch characteristics and detecting a scratch area;
s3, detecting the automobile chassis picture according to the obtained data model, selecting the position where the scratch possibly occurs, and forming a local point cloud picture;
s4, transforming the coordinates on the automobile chassis picture to a point cloud coordinate system according to the matrix change relation when the RGB-D camera shoots, intercepting the local point cloud picture of the position selected in the S3 step, and carrying out noise reduction treatment on the local point cloud picture;
and S5, processing the local point cloud picture, and acquiring the depth information of the scratch so as to qualitatively judge the scratch.
After a photo of an automobile chassis is shot and a data set is manufactured, deep learning is carried out on a screen image shot by an RGB camera, scratch information on a plane image is extracted, an area possibly with scratches is selected, processing software only needs to combine the selected area on the plane image, the depth information of the scratches is obtained after the local point cloud image is processed, the plane image is combined with the point cloud image, and the processing software does not need to carry out data processing on the whole point cloud image, so that the point cloud processing speed is improved, and the purpose of real-time detection is achieved.
Further, the RGB-D camera includes a housing, a left IR camera, a right IR camera, and an RGB camera, the RGB camera is located at the center of the housing, the left IR camera and the right IR camera are symmetrical with respect to the RGB camera, a MEMS micro-galvanometer projector (MEMS optical machine) is fixedly installed above the RGB camera, the RGB-D camera includes the RGB camera and the IR camera, and when in use, a plan view taken by the RGB camera can be directly used, and at the same time: the three cameras respectively shoot pictures of the checkerboard calibration plates at different positions, internal and external parameters of the cameras are calibrated by a Zhang Yongyou method, and then the rotation transformation relation of the left camera and the right camera is calculated to obtain a three-dimensional point cloud picture of a shot object.
Further, the step S5 includes the step S501 of solving a Hessian matrix for the picture of the automobile chassis, and determining the direction of each normal; s502, sectioning the point cloud according to the obtained normal direction to obtain the deepest point of the scratch section; s503, continuously repeating the S502 along the scratch direction to form a scratch bottom connecting line; s504, fitting a local point cloud plane where the scratch is located, obtaining a plane unit normal vector, and then performing rotation transformation on the plane unit normal vector to be parallel to an XOY plane. Then, the barycentric coordinate of the local point cloud is solved, the barycentric coordinate is translated and converted to the original point cloud origin, the scratch depth is obtained according to the Z coordinate of the scratch connecting line, and S505, the damage degree of the scratch is qualitatively judged according to different scratch depths;
and acquiring an extreme point of a scratch profile by using a fringe center extraction method based on a Hessian matrix, integrating to obtain a bottom connecting line of the scratch, and then obtaining a deepest point of the scratch by using matrix transformation so as to judge the scratch.
Further, the specific process of step 502 includes S50201, removing noise by using bilateral filtering; s50202, iterative fitting of a Gaussian curve is carried out by using a random sampling consistency method, the extreme point of the Gaussian curve is the deepest point of the scratch section, noise exists when an RGB-D camera is used for shooting pictures, the noise is removed, and the accuracy of results is improved.
The data set labeling software is labelme software or labelimg software, and labelme or labelimg data set labeling software is used for labeling the position of the scratch to manufacture the data set.
Further, the data enhancement means includes random cropping, warping, augmentation, mirroring.
Furthermore, the deep learning method is RCNN series network, YOLO series network, SSD series network and MTCNN, the pictures with the scratches and the marking information of the scratch positions are loaded into the target detection network for training through the deep learning method, the target detection network is used in a training and predicting stage, and the training stage aims at training weight parameters for prediction, so that the result in the predicting stage is more accurate.
The invention has the beneficial effects that:
1. the method adopts a self-research camera, combines the advantages of 2D and 3D, detects the possible areas of the scratches in the plane map through a deep learning method, makes a point cloud picture according to the 3D picture, then intercepts a local point cloud picture on the point cloud picture according to the intercepted plane picture, processes the local point cloud picture, and obtains the depth information of the scratches so as to qualitatively judge the scratches;
2. according to the characteristic that the normal direction of the scratch is in a Gaussian distribution characteristic, the center line of the scratch is extracted from the scratch depth map by using a stripe center extraction method based on a Hessian matrix, the normal direction of the scratch in the depth map is obtained, and a scratch section line is constructed. And (3) utilizing bilateral filtering and combining a data statistics method, iteratively removing noise points, fitting a Gaussian curve and obtaining extreme points. And connecting the extreme points of each scratch section line, thereby obtaining the point cloud coordinates of the scratch in the local coordinate system.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood and to implement them in accordance with the contents of the description, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings.
Drawings
FIG. 1 is a schematic diagram of the RGB-D camera of the present invention;
FIG. 2 is a schematic diagram of deep learning software detecting scratches according to the present invention;
FIG. 3 is a partial cloud of points where scratches according to the present invention are located;
FIG. 4 is a schematic view of the normal to the scratch of the present invention;
FIG. 5 is a cross-sectional view of a scratch of the present invention;
FIG. 6 is a line drawing of the bottom of a scratch according to the present invention;
the parts in the drawings are marked as follows:
1. an IR camera; 2. an RGB camera; 4. A housing; 5. areas where scratches may occur; 51. scratching; 6. cross hatching.
Detailed Description
The following description is given for illustrative embodiments of the invention with reference to specific embodiments, and the advantages and effects of the invention will be apparent to those skilled in the art from the disclosure of the present invention. The invention may be embodied in other different forms, i.e. it is capable of various modifications and changes without departing from the scope of the invention as disclosed.
Example (b): the three-dimensional detection method for the scratches of the electric automobile chassis based on the deep learning comprises the following steps of S1, collecting images of the automobile chassis by using an RGB-D camera.
As shown in fig. 1, the RGB-D camera used in step S1 includes a housing 4, a left IR camera 1, a right IR camera and an RGB camera 2, the RGB camera is located at the center of the housing, the left IR camera and the right IR camera are symmetrical with respect to the RGB camera, a MEMS micro-galvanometer projector (MEMS optical machine) is fixedly mounted above the RGB camera, the RGB-D camera includes the RGB camera and the IR camera, and when in use, a plan view taken by the RGB camera can be directly used, and at the same time: the three cameras respectively shoot pictures of the checkerboard calibration plates at different positions, internal and external parameters of the cameras are calibrated by a Zhang Yongyou method, and then the rotation transformation relation of the left camera and the right camera is calculated to obtain a three-dimensional point cloud picture of a shot object.
Then, marking the position of the scratch by using data set marking software which is labelme software or labelimg software, marking the position of the scratch by using labelme or labelimg data set marking software, making the picture into a data set, and then expanding the data set by using data enhancement means such as random cutting, distortion, amplification, mirror image and other methods;
step S2, extracting the minimum bounding box of the position of the scratch by using a target detection method in the deep learning method, then performing data set training, extracting scratch characteristics and detecting a scratch area; the deep learning method comprises an RCNN series network, a YOLO series network, an SSD series network and an MTCNN, wherein the pictures with the scratches and the marking information of the scratch positions are loaded into a target detection network for training through the deep learning method, the target detection network is used in a training and prediction stage, and the training stage aims at training weight parameters for prediction, so that the result in the prediction stage is more accurate.
As shown in fig. 2, in step S3, the plan view of the chassis of the automobile photographed by the RGB camera is detected according to the obtained data model, and the position of the scratch 51 is selected, and this step is only performed on the plan view photographed by the RGB camera, so that the speed of system detection can be increased, and the region 5 where the scratch may occur can be conveniently detected with emphasis.
As shown in fig. 3, in step S4, according to the matrix change relationship of the RGB camera relative to the left IR camera calibrated in advance when the RGB-D camera is used for shooting, the coordinates on the automobile chassis picture are transformed to the point cloud coordinate system, the local point cloud picture at the position selected in step S3 is captured to form a local point cloud picture, and the local point cloud picture is subjected to noise reduction.
And S5, processing the local point cloud picture, and acquiring the depth information of the scratch so as to qualitatively judge the scratch.
The step S5 further includes the following steps, as shown in fig. 4, step S501, solving a Hessian matrix for the picture of the automobile chassis, and determining the direction of each normal; as shown in fig. 5, step S502, a section line 6 of the point cloud is drawn according to the obtained normal direction, and a deepest point of the scratch section is obtained; the specific process of the step 502 comprises S50201, and noise is removed by adopting bilateral filtering; s50202, iterative fitting of a Gaussian curve is carried out by using a random sampling consistency method, the extreme point of the Gaussian curve is the deepest point of the scratch section, noise exists when an RGB-D camera is used for shooting pictures, the noise is removed, and the accuracy of results is improved. As shown in fig. 6, step S503 is repeated S502 along the scratch direction to form a scratch bottom connection line; s504, fitting a local point cloud plane where the scratch is located, obtaining a plane unit normal vector, and then performing rotation transformation on the plane unit normal vector to be parallel to an XOY plane. Then, the barycentric coordinate of the local point cloud is solved, the barycentric coordinate is translated and converted to the original point cloud origin, the scratch depth is obtained according to the Z coordinate of the scratch connecting line, and S505, the damage degree of the scratch is qualitatively judged according to different scratch depths;
and acquiring an extreme point of a scratch profile by using a fringe center extraction method based on a Hessian matrix, integrating to obtain a bottom connecting line of the scratch, and then obtaining a deepest point of the scratch by using matrix transformation so as to judge the scratch.
The working process and working principle of the invention are as follows:
after an automobile chassis photo is shot by the RGB-D camera and made into a data set, scratch information on a plane graph is extracted by deep learning of a screen image shot by the RGB camera, so that an area where scratches possibly exist is selected, processing software only needs to combine the selected area on the plane graph, the selected area where scratches possibly exist is compared with a point cloud image, a local point cloud image where scratches possibly appear is obtained, and the depth information of the scratches is obtained after the local point cloud image is processed.
The above description is only an embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures made by using the contents of the specification and the drawings, or other related technical fields, are all included in the scope of the present invention.
Claims (6)
1. The electric automobile chassis scratch three-dimensional detection method based on deep learning is characterized by comprising the following steps:
s1, collecting a picture of the automobile chassis by adopting an RGB-D camera, marking the position of the scratch by utilizing data set marking software, making the picture into a data set, and expanding the data set by adopting a data enhancement means;
s2, performing data set training by using a deep learning method, extracting scratch characteristics and detecting a scratch area;
s3, detecting the automobile chassis picture according to the obtained data model, and selecting the position where the scratch possibly occurs;
s4, transforming the coordinates on the automobile chassis picture to a point cloud coordinate system according to the matrix change relation when the RGB-D camera shoots, intercepting the local point cloud picture of the position selected in the S3 step, and carrying out noise reduction treatment on the local point cloud picture;
s5, processing the local point cloud picture to obtain the depth information of the scratch, thereby carrying out qualitative judgment on the scratch; the step S5 also comprises the following steps of S501, solving a Hessian matrix for the automobile chassis picture, and determining the direction of each normal; s502, a sectional line of the point cloud is made according to the direction of the obtained normal line, and the deepest point of the scratch section is obtained; s503, continuously repeating the S502 along the scratch direction to form a scratch bottom connecting line; s504, fitting a local point cloud plane where the scratch is located, obtaining a plane unit normal vector, and then performing rotation transformation on the plane unit normal vector to be parallel to an XOY plane; and then, calculating barycentric coordinates of the local point cloud, translating the barycentric coordinates to the original point cloud origin, obtaining scratch depths according to Z coordinates of scratch connecting lines, and S505, qualitatively judging the damage degrees of scratches according to different scratch depths.
2. The deep learning-based three-dimensional detection method for scratches on the chassis of an electric vehicle according to claim 1, wherein: the RGB-D camera comprises a shell, a left IR camera, a right IR camera and an RGB camera, wherein the RGB camera is located in the center of the shell, the left IR camera and the right IR camera are symmetrical relative to the RGB camera, and a MEMS micro-vibrating mirror projector is fixedly mounted above the RGB camera.
3. The deep learning-based three-dimensional detection method for scratches on the chassis of an electric vehicle according to claim 1, wherein: the specific process of the step S502 comprises S50201, and noise is removed by adopting bilateral filtering; s50202, iterative fitting of a Gaussian curve is carried out by using a random sampling consistency method, and the extreme point of the Gaussian curve is the deepest point of the scratch section.
4. The deep learning-based three-dimensional detection method for the scratches of the electric vehicle chassis according to claim 1, wherein: the data set labeling software is labelme software or labelimg software.
5. The deep learning-based three-dimensional detection method for the scratches of the electric vehicle chassis according to claim 1, wherein: the data enhancement means comprises random cutting, distortion, amplification and mirror image.
6. The deep learning-based three-dimensional detection method for the scratches of the electric vehicle chassis according to claim 1, wherein: the deep learning method comprises an RCNN series network, a YOLO series network, an SSD series network and an MTCNN.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210568310.8A CN114663882B (en) | 2022-05-24 | 2022-05-24 | Electric automobile chassis scratch three-dimensional detection method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210568310.8A CN114663882B (en) | 2022-05-24 | 2022-05-24 | Electric automobile chassis scratch three-dimensional detection method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114663882A CN114663882A (en) | 2022-06-24 |
| CN114663882B true CN114663882B (en) | 2022-09-06 |
Family
ID=82037310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210568310.8A Active CN114663882B (en) | 2022-05-24 | 2022-05-24 | Electric automobile chassis scratch three-dimensional detection method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114663882B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115205278B (en) * | 2022-08-02 | 2023-05-02 | 昆山斯沃普智能装备有限公司 | Electric automobile chassis scratch detection method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378900A (en) * | 2019-08-01 | 2019-10-25 | 北京迈格威科技有限公司 | The detection method of product defects, apparatus and system |
| CN112070751A (en) * | 2020-09-10 | 2020-12-11 | 深兰人工智能芯片研究院（江苏）有限公司 | Wood floor defect detection method and device |
| CN113689392A (en) * | 2021-08-18 | 2021-11-23 | 北京理工大学 | Railway fastener defect detection method and device |
-
2022
- 2022-05-24 CN CN202210568310.8A patent/CN114663882B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378900A (en) * | 2019-08-01 | 2019-10-25 | 北京迈格威科技有限公司 | The detection method of product defects, apparatus and system |
| CN112070751A (en) * | 2020-09-10 | 2020-12-11 | 深兰人工智能芯片研究院（江苏）有限公司 | Wood floor defect detection method and device |
| CN113689392A (en) * | 2021-08-18 | 2021-11-23 | 北京理工大学 | Railway fastener defect detection method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114663882A (en) | 2022-06-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Shan et al. | A stereovision-based crack width detection approach for concrete surface assessment | |
| CN109215063B (en) | Registration method of event trigger camera and three-dimensional laser radar | |
| CN104574393B (en) | A kind of three-dimensional pavement crack pattern picture generates system and method | |
| CN109632808B (en) | Edge defect detection method and device, electronic equipment and storage medium | |
| CN111652085B (en) | Object identification method based on combination of 2D and 3D features | |
| CN107767456A (en) | A kind of object dimensional method for reconstructing based on RGB D cameras | |
| CN110738642A (en) | Mask R-CNN-based reinforced concrete crack identification and measurement method and storage medium | |
| CN113744351B (en) | Underwater structure light measurement calibration method and system based on multi-medium refraction imaging | |
| CN112037203A (en) | Side surface defect detection method and system based on complex workpiece outer contour registration | |
| CN109483887B (en) | Online detection method for contour accuracy of forming layer in selective laser melting process | |
| CN109166125A (en) | A kind of three dimensional depth image partitioning algorithm based on multiple edge syncretizing mechanism | |
| CN112067233A (en) | Six-degree-of-freedom motion capture method for wind tunnel model | |
| CN111489337A (en) | Method and system for removing false defects through automatic optical detection | |
| CN115830018B (en) | Carbon block detection method and system based on deep learning and binocular vision | |
| CN114663882B (en) | Electric automobile chassis scratch three-dimensional detection method based on deep learning | |
| CN108510544B (en) | Light strip positioning method based on feature clustering | |
| CN114396875B (en) | Rectangular package volume measurement method based on vertical shooting of depth camera | |
| CN115546153A (en) | Product surface defect detection method based on three-dimensional point cloud and two-dimensional image fusion perception | |
| CN115222884A (en) | Space object analysis and modeling optimization method based on artificial intelligence | |
| CN113705564B (en) | Pointer type instrument identification reading method | |
| CN114549549A (en) | Dynamic target modeling tracking method based on instance segmentation in dynamic environment | |
| CN112330667B (en) | Morphology-based laser stripe center line extraction method | |
| KR101733028B1 (en) | Method For Estimating Edge Displacement Againt Brightness | |
| CN116596987A (en) | Workpiece three-dimensional size high-precision measurement method based on binocular vision | |
| CN116958837A (en) | Municipal facilities fault detection system based on unmanned aerial vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |