CN117764897A

CN117764897A - Target object crack detection method and device, electronic equipment and storage medium

Info

Publication number: CN117764897A
Application number: CN202211122511.1A
Authority: CN
Inventors: 伍广明
Original assignee: Fengtu Technology Shenzhen Co Ltd
Current assignee: Fengtu Technology Shenzhen Co Ltd
Priority date: 2022-09-15
Filing date: 2022-09-15
Publication date: 2024-03-26

Abstract

The application provides a target object crack detection method, a target object crack detection device, electronic equipment and a storage medium, relates to the technical field of image recognition, solves the problems of high maintenance cost and poor timeliness of manually and regularly checking target object cracks, and comprises the following steps: acquiring a target object video containing a target object to be detected; identifying the outer surface of a target object to be detected in a target object video to obtain a target object outer surface image of the target object to be detected; performing contrast enhancement treatment and filtering treatment on the target object outer surface image to obtain a target object outer surface pretreatment image; the method and the device have the advantages that whether the object to be detected in the object outer surface pretreatment image has the crack or not is determined by carrying out crack detection on the object outer surface pretreatment image, the hidden danger of the crack of the object to be detected is conveniently and real-timely checked, the problem that personnel are required to periodically observe the object to be detected on site is solved, the crack detection cost of the object to be detected is reduced, and the crack detection efficiency is improved.

Description

Target object crack detection method and device, electronic equipment and storage medium

Technical Field

The application relates to the technical field of image recognition, in particular to a target object crack detection method and device, electronic equipment and a storage medium.

Background

With the development of cities, due to the aging and insufficient maintenance of old urban buildings, the damage of the buildings is increasingly serious, so that the hidden danger of cracks exists on the walls of the buildings, and in order to discover the risks of the cracks of the old buildings in time and maintain and rebuild the old buildings in time, serious damage to property and life of residents is avoided.

However, the solution is usually manual regular investigation, consumes large manpower, has high maintenance cost, still has the problem of untimely discovery, has poor aging, and can not accurately analyze the crack problem of the wall body by checking the wall body by human eyes.

Disclosure of Invention

The application provides a target object crack detection method, device, electronic equipment and storage medium, which can realize automatic detection of a target object crack, facilitate real-time detection of the hidden danger of the target object crack, reduce detection cost and improve detection efficiency.

In one aspect, the present application provides a target crack detection method, including:

Acquiring a target object video containing a target object to be detected;

identifying the outer surface of the object to be detected in the object video to obtain an object outer surface image of the object to be detected;

performing contrast enhancement processing and filtering processing on the target object outer surface image to obtain a target object outer surface pretreatment image;

and detecting cracks on the target object outer surface pretreatment image, and determining whether the target object to be detected in the target object outer surface pretreatment image has cracks or not.

In one possible implementation manner of the present application, the performing contrast enhancement processing and filtering processing on the image of the outer surface of the object to obtain a preprocessed image of the outer surface of the object includes:

and carrying out nonlinear stretching on the object outer surface image according to the histogram of the object outer surface image, enhancing the contrast of the object outer surface image, and obtaining the object outer surface image with enhanced contrast.

Edge enhancement is carried out on the surface image of the object after contrast enhancement by adopting an edge detection method;

performing filtering treatment on the edge-enhanced object outer surface image by adopting a morphological opening and closing operation method to obtain a filtered object outer surface image;

and taking the filtered image of the outer surface of the target object as a preprocessing image of the outer surface of the target object.

In one possible implementation manner of the present application, the detecting the crack in the target object outer surface pretreatment image, determining whether the target object to be detected in the target object outer surface pretreatment image has a crack, includes:

detecting all noise areas in the target object outer surface pretreatment image;

judging whether the area size of all the noise areas is larger than a preset noise area size threshold value or not, and

when the area size of at least one noise area in all the noise areas is larger than or equal to the noise area size threshold, determining that a crack exists in the object to be detected in the object outer surface pretreatment image;

and when the area sizes of all the noise areas are smaller than the noise area size threshold, determining that no crack exists in the object to be detected in the object outer surface pretreatment image.

In one possible implementation manner of the present application, after the performing crack detection on the target object external surface pretreatment image, determining whether the target object to be detected in the target object external surface pretreatment image has a crack, the method further includes:

when the target object to be detected in the target object outer surface pretreatment image has no crack, storing the target object outer surface image into a preset target object sample image library;

when the object to be detected in the object surface pretreatment image has a crack, generating warning information of the crack of the object to be detected to a target receiving terminal.

In one possible implementation manner of the present application, the identifying the outer surface of the object to be detected in the object video, to obtain an object outer surface image of the object to be detected, includes:

detecting the target object to be detected on the target object video to generate a target object detection frame;

intercepting an initial image of the target object in the corresponding area according to the target object detection frame;

determining a plurality of candidate corner points corresponding to the target object to be detected in the initial image of the target object;

And generating the external surface image of the target object according to the candidate corner points.

In one possible implementation manner of the present application, the determining a plurality of candidate corner points corresponding to the object to be detected in the initial image of the object includes:

detecting a plurality of initial corner points in the initial image of the target object by using a Harris corner point detector;

and filtering a plurality of initial corner points which are not near the plurality of vertexes of the target object detection frame in the initial image of the target object according to Euler distances between the plurality of initial corner points and the plurality of vertexes of the target object detection frame, and taking the rest of the initial corner points in the initial image of the target object as a plurality of candidate corner points.

In one possible implementation manner of the present application, the generating the object external surface image according to the plurality of candidate corner points includes:

affine transformation is carried out on an irregular image area surrounded by the candidate angular points, so as to obtain a regular image area;

and taking the regular image as the image of the outer surface of the target object.

In another aspect, the present application provides a target crack detection system, the system comprising:

The acquisition module is used for acquiring a target object video containing a target object to be detected;

the identification module is used for obtaining an object outer surface image of the object to be detected for identifying the outer surface of the object to be detected in the object video;

the preprocessing module is used for carrying out contrast enhancement processing and filtering processing on the target object outer surface image to obtain a target object outer surface preprocessing image;

and the detection module is used for detecting cracks of the pretreatment image of the outer surface of the target object and determining whether the target object to be detected in the pretreatment image of the outer surface of the target object has cracks or not.

In another aspect, the present application also provides a computer readable storage medium having stored thereon a computer program to be loaded by a processor to perform the steps of the target crack detection method.

According to the method, the outer surface of the object to be detected in the object video is identified by acquiring the object video containing the object to be detected, the object outer surface image of the object to be detected is obtained, the contrast enhancement treatment and the filtering treatment are carried out on the object outer surface image, the object outer surface pretreatment image is obtained, the crack detection is carried out on the object outer surface pretreatment image, and whether the object to be detected in the object outer surface pretreatment image has a crack or not is determined, so that the method can be used for detecting the crack condition of the object to be detected based on the object video of the object to be detected in real time conveniently and automatically, the hidden danger of the crack of the object to be detected in real time is eliminated, the problem that a part of the objects to be detected at present need personnel to periodically observe the object to be detected on site is solved, the crack detection cost of the object to be detected is reduced, and the crack detection efficiency is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic view of a scene of a target crack detection system provided in an embodiment of the present application;

FIG. 2 is a schematic flow chart of an embodiment of a method for detecting a target crack according to the present disclosure;

FIG. 3 is a schematic structural diagram of an embodiment of a target crack detection device provided in an embodiment of the present application;

fig. 4 is a schematic structural diagram of an embodiment of an electronic device provided in an embodiment of the present application.

Detailed Description

The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.

In the description of the present invention, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.

In this application, the term "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the invention. In the following description, details are set forth for purposes of explanation. It will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and processes have not been described in detail so as not to obscure the description of the invention with unnecessary detail. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

The embodiment of the application provides a target object crack detection method, a target object crack detection device, an electronic device and a storage medium, and the target object crack detection method, the target object crack detection device, the electronic device and the storage medium are respectively described in detail below.

The execution body of the target object crack detection method in the embodiment of the present application may be a target object crack detection device provided in the embodiment of the present application, or different types of electronic devices such as a server device, a physical host, or a User Equipment (UE) integrated with the target object crack detection device, where the target object crack detection device may be implemented in a hardware or software manner, and the UE may specifically be a terminal device such as a smart phone, a tablet computer, a notebook computer, a palm computer, a desktop computer, or a personal digital assistant (Personal Digital Assistant, PDA).

The electronic device may be operated in a single operation mode, or may also be operated in a device cluster mode.

As shown in fig. 1, fig. 1 is a schematic view of a target object crack detection system according to an embodiment of the present application. The target object crack detection system may include a video acquisition device for capturing a target object to be detected and an electronic device 100 for completing a target object crack detection method, where a target object crack detection device is integrated in the electronic device 100. For example, the electronic device may acquire a target object video including a target object to be detected, identify an outer surface of the target object to be detected in the target object video, obtain a target object outer surface image of the target object to be detected, perform contrast enhancement processing and filtering processing on the target object outer surface image, obtain a target object outer surface pretreatment image, perform crack detection on the target object outer surface pretreatment image, and determine whether the target object to be detected in the target object outer surface pretreatment image has a crack.

In addition, as shown in fig. 1, the object crack detection system may further include a memory 200 for storing data such as video data, image data of an object to be detected, device data of a video capture device for capturing video of the object to be detected, and the like.

It should be noted that, the schematic view of the scenario of the target object crack detection system shown in fig. 1 is only an example, and the target object crack detection system and scenario described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application, and as one of ordinary skill in the art can know, along with the evolution of the target object crack detection system and the appearance of a new service scenario, the technical solutions provided in the embodiments of the present invention are applicable to similar technical problems.

Next, an object crack detection method provided in the embodiment of the present application will be described, in the embodiment of the present application, an electronic device is used as an execution body, and for simplicity and convenience of description, in a subsequent method embodiment, the execution body will be omitted, where the object crack detection method includes:

obtaining a target object video containing a target object to be detected, identifying the outer surface of the target object to be detected in the target object video, obtaining a target object outer surface image of the target object to be detected, performing contrast enhancement processing and filtering processing on the target object outer surface image to obtain a target object outer surface pretreatment image, performing crack detection on the target object outer surface pretreatment image, and determining whether the target object to be detected in the target object outer surface pretreatment image has cracks or not.

Therefore, the method can detect the crack condition of the target object to be detected based on the target object video of the target object to be detected in real time and automatically, so that the hidden danger of the crack of the target object to be detected is detected in real time, the problem that partial target objects at present need personnel to periodically observe the target object to be detected on site is solved, the crack detection cost of the target object to be detected is reduced, and the crack detection efficiency is improved.

Fig. 2 is a schematic flow chart of an embodiment of a method for detecting a target object crack according to the embodiment of the present application, and fig. 2 is a schematic flow chart of a method for detecting a target object crack according to the embodiment of the present application. It should be noted that although a logical order is depicted in the flowchart, in some cases the steps depicted or described may be performed in a different order than presented herein. The target object crack detection method specifically comprises the following steps 201 to 204:

201. and acquiring a target object video containing the target object to be detected.

The object to be detected is any object in nature, for example, a mountain, a resident building wall, a bridge, a road, an electric tower, a tunnel, and the like, and may be various objects in daily life, such as a vase, a cabinet, a display, and the like, which is not particularly limited in this embodiment.

The target object video is a video obtained by shooting the video acquisition device from any angle outside the target object to be detected, in the application process, for example, the target object to be detected with larger size such as a resident building, a bridge, a road, an electric tower, a tunnel and the like, the video acquisition device can be a fixed shooting device such as a thunder integrated machine, a camera and the like which is arranged near the target object to be detected, and the video acquisition device can also be a mobile shooting device such as a vehicle-mounted shooting device, an unmanned aerial vehicle and the like, and the embodiment is not particularly limited to this.

In this embodiment, after the video acquisition device acquires the target object video of the target object to be detected, a connection channel is established between the video acquisition device and the electronic device for executing the target object crack detection method through the network transmission module, and the target object video or the image acquired by the video acquisition device is sent to the electronic device in a message form, so as to achieve acquisition of the target object video of the target object to be detected.

In this embodiment, when the geographic information and the shooting time information of the shot target object to be detected are additionally required to be recorded, the temporal and spatial geographic information of the video acquisition device may be simultaneously transmitted when the video acquisition device transmits the target object video to the electronic device, where the temporal and spatial geographic information includes information such as coordinate position information, attitude angle information, and timestamp information when the video acquisition device acquires the target object video of the target object to be detected, and the data content transmitted by the video acquisition device is not specifically limited in this embodiment.

202. And identifying the outer surface of the object to be detected in the object video to obtain an object outer surface image of the object to be detected.

Because the video acquisition device acquires the pictures of other elements except the target object to be detected when acquiring the target object video, in order to avoid interference of the pictures of other elements in the target object video to crack detection of the target object to be detected, the area of the target object to be detected, namely the target object outer surface image of the target object to be detected, needs to be determined from the target object video before the crack detection is carried out on the target object to be detected in the target object video.

In this embodiment, identifying the outer surface of the object to be detected in the object video to obtain the object outer surface image of the object to be detected, specifically including steps 301 to 304:

301. and detecting the target object to be detected on the target object video to generate a target object detection frame.

In this embodiment, an image frame set of a target object video is taken as input, target object detection to be detected is performed on the target object video through a trained target object detection model, each image frame is output to have an image frame set of a target object to be detected and a target object detection frame, and then crack detection of the target object to be detected is performed on the basis of one of the image frames in the image frame set; in the application process, the crack detection analysis of the object to be detected may be based on any frame in the image frame set, which has the object to be detected and the object detection frame, so as to detect the crack of the object to be detected more comprehensively and completely, or may be based on multiple frames in the image frame set or each frame of image frame, which is not particularly limited in this embodiment.

302. And intercepting an initial image of the target object in the corresponding area according to the target object detection frame.

Because each image frame includes the object to be detected and other irrelevant areas, in order to avoid interference of crack detection of the object to be detected in other irrelevant areas in the image frame, in this embodiment, based on the area where the object to be detected is marked by the object detection frame, an image of the area where the object to be detected is located is intercepted, an initial image of the object is obtained, and then crack detection of the object to be detected is performed based on the initial image of the object.

303. And determining a plurality of candidate corner points corresponding to the object to be detected in the initial image of the object.

In this embodiment, after the initial image of the target object is obtained, in order to restore the initial image of the target object as clearly as possible, so as to extract a plurality of candidate corner points corresponding to the target object to be detected in the initial image of the target object, firstly, a cvtColor function in OpenCv is adopted to convert the initial image of the target object from an RGB color space to a Lab color space, extract L texture information of the initial image of the target object, and then determine a plurality of candidate corner points corresponding to the target object to be detected in the initial image of the target object based on the L texture information of the initial image of the target object. In this embodiment, the L texture information of the initial image of the target object may be extracted by using a gray level co-occurrence matrix, gray level stroke statistics, gray level difference statistics, local gray level statistics, half-square difference map, autocorrelation function, and the like, which is not limited in this embodiment.

In this embodiment, determining a plurality of candidate corner points corresponding to a target object to be detected in an initial image of the target object specifically includes 3031 to 3032:

3031. a Harris corner detector is used for detecting a plurality of initial corner points in an initial image of the target object.

Specifically, a Harris angle point detector is adopted to slide in any direction on an initial image of a target object, the gray level change degree of pixels in a sliding window is compared under the two conditions before sliding and after sliding, if the sliding in any direction has larger gray level change, the sliding window can be considered to have interest points, and the interest points are also called corner points;

since the contour edge of the object to be detected in the initial image of the object is required to be determined according to the corner points, the outer surface image of the object is determined according to the contour edge of the object to be detected, the Harris corner point detector is adopted for corner point detection in the initial image of the object for the first time, a plurality of initial corner points in the initial image of the object can be obtained, but not all the initial corner points in the initial image of the object can help to determine the contour edge of the object to be detected in the initial image of the object, and therefore, the required candidate corner points are required to be determined from the initial corner points.

3032. And filtering a plurality of initial corner points which are not near the plurality of vertexes of the target object detection frame in the initial image of the target object according to Euler distances between the plurality of initial corner points and the plurality of vertexes of the target object detection frame, and taking the rest of the plurality of initial corner points in the initial image of the target object as a plurality of candidate corner points.

According to Euler distances between a plurality of initial corner points and a plurality of vertexes of the target object detection frame, the method specifically comprises the following steps:

calculating Euler distances between each initial angular point in the plurality of initial angular points and each vertex of the target object detection frame, namely calculating the distances between the initial angular points and the vertices of the target object detection frame in a specific plane rectangular coordinate system to obtain a plurality of groups of distance calculation value groups corresponding to the plurality of initial angular points respectively, wherein each group of distance calculation value groups comprises a plurality of distance calculation values, and the number of the distance calculation values in each group of distance calculation value groups is the same as that of the vertices of the target object detection frame;

filtering a plurality of initial corner points which are not near a plurality of top points of the target object detection frame in the target object initial image, taking the rest of the plurality of initial corner points in the target object initial image as a plurality of candidate corner points, wherein the method specifically comprises the following steps of:

All the calculated distance calculation values are ordered, whether the order is ascending or descending, initial corner points exceeding a preset distance threshold value in the distance calculation values (namely, a plurality of initial corner points which are not near a plurality of vertexes of the target object detection frame) are filtered, initial corner points smaller than the preset distance threshold value in the distance calculation values (namely, a plurality of initial corner points near a plurality of vertexes of the target object detection frame) are reserved, and the reserved initial corner points are used as a plurality of candidate corner points.

304. And generating an external surface image of the target object according to the plurality of candidate corner points.

In this embodiment, generating, according to a plurality of candidate corner points, an object external surface image includes:

carrying out affine transformation on an irregular image area surrounded by a plurality of candidate corner points to obtain a regular image area; and taking the regular image as an external surface image of the target object.

And carrying out translation, rotation, scale transformation and the like on an irregular image area surrounded by a plurality of candidate corner points so as to carry out image correction and texture correction on the irregular image area and obtain a regular image area.

In this embodiment, the object outer surface in the object video may be identified by other methods, so as to obtain an object outer surface image of the object to be detected, and the method for identifying the object outer surface image of the object to be detected is not specifically limited in this embodiment.

203. And performing contrast enhancement processing and filtering processing on the target object outer surface image to obtain a target object outer surface pretreatment image.

After the object outer surface image is obtained, in order to facilitate detection of whether the object to be detected has a crack in the object outer surface image, the object outer surface image needs to be preprocessed to highlight image features related to the crack in the object outer surface image; in this embodiment, the preprocessing of the object external surface image may include performing contrast enhancement processing and filtering processing on the object external surface image, or may use other preprocessing methods to highlight the image features related to the crack in the object external surface image, which is not specifically limited in this embodiment.

In this embodiment, contrast enhancement processing and filtering processing are performed on the image of the outer surface of the object to obtain a preprocessed image of the outer surface of the object, including steps 401 to 403:

401. and carrying out nonlinear stretching on the object outer surface image according to the histogram of the object outer surface image, enhancing the contrast of the object outer surface image, and obtaining the object outer surface image with enhanced contrast.

In order to make the image contrast of the initial image of the object higher, the texture and the edge are clearer, so that the crack of the object to be detected in the outer surface image of the object can be identified later, in the embodiment, the outer surface image of the object is converted into a gray level image, namely, the gray level image of the outer surface of the object is converted, then a gray level stretching function is adopted to stretch the specific image gray level in the histogram of the gray level image of the outer surface of the object in a nonlinear mode, so that the probability density of the image gray level of the gray level image of the outer surface of the object is uniformly distributed, thereby increasing the dynamic range of the image gray level and enhancing the contrast of the gray level image of the outer surface of the object.

In addition, since the target initial image has been converted from the RGB color space to the Lab color space in step 303 and the L texture information of the target initial image is extracted, the L texture information of the target initial image is gray, that is, the target initial image has been converted into a gray image in step 303, it is possible to select a step of converting the target outer surface image into a gray image with or without being added in this step.

402. And carrying out edge enhancement on the object external surface image with enhanced contrast by adopting an edge detection method.

The edge is a place where the pixel value in the image changes rapidly, the gray value change is larger for the edge part of the image, the gradient value is larger, and the gray value change is smaller for the smoother part of the image; if the object to be detected has a crack in the object outer surface image, edge enhancement can be performed on the object outer surface image to more highlight the crack image in the object outer surface image, so that the crack of the object to be detected in the object outer surface image can be detected more accurately.

In this embodiment, a Canny edge detection method is used to perform edge enhancement on the object external surface image after contrast enhancement, and the Canny edge detection method specifically includes the following steps:

4021. and carrying out noise reduction treatment on the object outer surface image to obtain a smooth object outer surface image.

The Canny edge detection method is essentially that the edge contour of the image of the outer surface of the object is enhanced by a gradient operator, but the gradient operator is easily affected by noise, so that the effect of enhancing the edge contour of the image is poor, and therefore noise in the image of the outer surface of the object needs to be removed first.

4022. And calculating the gradient amplitude and the gradient direction of the smooth object outer surface image.

The gradient of the image represents the changing speed of the image, reflects the edge information of the image, and obtains possible edges in the image of the outer surface of the object by calculating the gradient amplitude and the gradient direction of the image of the outer surface of the smooth object, wherein in the embodiment, the "possible edges" refer to the places where the gray scale changes and may be edges or not edges, and refer to the collection of all the possible edges.

4023. And performing non-maximum suppression on possible edges in the object outer surface image to obtain an initial edge image corresponding to the object outer surface image.

Since the gray level variation in the object's outer surface image based on step 4022 is concentrated, the possible edges may still be very blurred, so all gradient values except the local maximum in the object's outer surface image are suppressed by non-maximum suppression, and by setting them to 0, i.e. the gradient direction in the local range, the gray level variation is retained maximally, and the other ones are not retained, so as to highlight the position with the strongest gray level variation in the object's outer surface image.

4024. And carrying out double-threshold screening on the initial edge image corresponding to the object outer surface image to obtain a final edge image corresponding to the object outer surface image.

After non-maximum suppression of possible edges in the object outer surface image, there may still be many possible edge points in the initial edge image;

setting a pixel corresponding to the gray level change as a strong edge pixel when the gray level change in the initial edge image is larger than the high threshold value, deleting the pixel corresponding to the gray level change when the gray level change in the initial edge image is smaller than the low threshold value, namely setting the pixel as 0, and setting the pixel corresponding to the gray level change as a weak edge when the gray level change in the initial edge image is between the low threshold value and the high threshold value;

further, if a strong edge pixel exists in a certain area of the initial edge image, the strong edge pixel is reserved, otherwise, the pixel in the area is deleted, namely, the dual-threshold screening of the initial edge image corresponding to the external surface image of the target object is realized, and finally, the final edge image corresponding to the external surface image of the target object is obtained.

403. And filtering the object outer surface image after edge enhancement by adopting a morphological opening and closing operation method to obtain a filtered object outer surface gray level image.

Filtering the object outer surface image with the enhanced edges by a morphological opening and closing operation method to remove isolated noise points, namely filtering too small image points in the object outer surface image, and taking the object outer surface image after the filtering treatment as an object outer surface pretreatment image.

204. And detecting cracks on the target object outer surface pretreatment image, and determining whether the target object to be detected in the target object outer surface pretreatment image has cracks or not.

In this embodiment, the target object outer surface pretreatment image may be used as input, and the trained target object crack detection model is used to detect the crack in the target object outer surface pretreatment image, and output the result of whether the target object has a crack in the target object outer surface image.

In another embodiment of the present application, a manual identification manner may also be used to detect the crack in the pretreatment image of the outer surface of the target object, so as to determine whether the target object to be detected in the pretreatment image of the outer surface of the target object has a crack, and although the method still needs to manually detect the crack, compared with a manner in which the integrity of the target object to be detected is checked by a gridding member periodically to the site, the solution provided in the embodiment still has higher efficiency.

In another embodiment of the present application, performing crack detection on the target object external surface pretreatment image to determine whether the target object to be detected in the target object external surface pretreatment image has a crack, and may further specifically include:

preprocessing all noise areas in the image on the outer surface of the object to be detected; judging whether the area size of all the noise areas is larger than a preset noise area size threshold, and determining that a crack exists in the object to be detected in the object outer surface pretreatment image when the area size of at least one noise area in all the noise areas is larger than or equal to the noise area size threshold; and when the area sizes of all the noise areas are smaller than the noise area size threshold, determining that no crack exists in the object to be detected in the object surface pretreatment image.

In this embodiment, after the contrast enhancement processing and the filtering processing are performed on the external surface image of the target object, a relatively large and clear noise area is reserved in the obtained external surface pretreatment image of the target object, where the noise area may be a crack of the target object to be detected or may not, so that whether the crack exists on the external surface of the target object can be determined by determining the area of the noise area in the external surface pretreatment image of the target object;

Exemplary, noise region size threshold is setIs 3 square centimeters (cm) ² ) When the area of a certain noise point area in the target object outer surface pretreatment image is 5 square centimeters, determining that a crack exists in the target object to be detected in the target object outer surface pretreatment image; and when the area of a certain noise point area in the target object outer surface pretreatment image is 2 square centimeters, determining that the target object to be detected in the target object outer surface pretreatment image has no cracks.

In this embodiment, the noise area size threshold may be manually preset as required, or adaptively set in an electronic device that may implement the crack detection method, and a specific value of the noise area size threshold may be set as required, which is not specifically limited in this embodiment.

In one possible implementation manner of the present application, after performing crack detection on the target object outer surface pretreatment image, determining whether the target object to be detected in the target object outer surface pretreatment image has a crack, the method further includes:

when the object to be detected in the object external surface pretreatment image has no crack, storing the object external surface image into a preset object sample image library;

The target object sample image library is used for storing a sample image frame set of a target object sample video and a set of different target object outer surface pretreatment sample images, image data in the target object sample image library can be used for training a target object detection model and a target object crack detection model, when a target object to be detected in the target object outer surface pretreatment image does not have a crack, the target object outer surface pretreatment image is used as the target object outer surface pretreatment sample image to be stored in a preset target object sample image library, continuous accumulation of model training data is realized, and continuous iteration of the target object detection model and the target object crack detection model is facilitated;

when the object to be detected in the object surface pretreatment image has a crack, generating warning information of the crack of the object to be detected to the object receiving terminal.

When the fact that the object to be detected has a crack is determined, warning information of potential safety hazards is sent to the object receiving terminal, so that potential safety hazards of the object to be detected of the opposite side can be notified in time, and potential safety hazard checking efficiency of the object to be detected is improved; in this embodiment, the target receiving terminal may be another electronic device of a different type, such as a server device, a physical host, or a user device, for receiving the warning information, where information transmission may be performed between the electronic device for receiving the warning information and the electronic device for implementing the target object crack detection method according to any existing information transmission manner, which is not limited in this embodiment.

In another embodiment of the present application, the object detection model may be trained by the following steps:

training by taking an efficientnet as a backbone network and taking a yolox or anchor free target detection model as a model to be trained, taking a sample image frame set of target sample videos stored in a target sample image library as input of the model to be trained, and taking an image frame containing a target to be detected and a target detection frame as output to perform model training to obtain a target detection model;

in the training process, the modeling capability of the model to be trained can be improved by adopting a data enhancement mode, wherein the data enhancement mode specifically comprises the following modes but not limited to the following modes:

(1) Random cropping (Random Crop) is performed on the sample image frames in the sample image frame set, specifically, cropping is performed on the sample image frames in a region with a Random ratio of 0.6-1.0, and the cropped sample image frames are used as input of a model to be trained;

(2) Embedding a Dropblock layer in a network, namely, a backbone network comprises a plurality of inner winding layers, a pooling layer and a Dropblock layer, wherein in the Dropblock layer, the discarding probability of a neighborhood space pixel point with the area size of K multiplied by R in a discarding feature map is p, and the Dropblock layer is used for setting the discarding probability of a neighborhood space pixel point with the area size of 3 multiplied by 3 in the discarding feature map to be 0.1;

(3) The Mosaic mosaics are adopted to realize data enhancement, specifically, four sample image frames in a sample image frame set are spliced into one Mosaic image randomly, and a new sample image obtained by splicing is used as training data to be used as input for model training.

In this embodiment, multi-scale training (Multi Scale Training, MST) is used to reduce the risk of model overfitting and enhance the robustness of the target detection model.

In another embodiment of the present application, the target crack detection model may be trained by:

model training is carried out by adopting efficientnet as a backbone network and adopting a 2-layer 3*3 network structure, before training, data marking is carried out on the target object outer surface pretreatment sample images stored in a target object sample image library, and an exemplary method is that the target object outer surface pretreatment sample images with cracks are marked with 'target object outer surface cracks', the target object outer surface pretreatment sample images without cracks are marked with 'target object outer surface complete', a set of marked target object outer surface pretreatment sample images is used as input of a model to be trained, and a target object outer surface pretreatment sample image containing target objects to be detected and a target object crack detection label are used as output to carry out model training, so that a target object crack detection model is obtained;

(1) Random clipping (Random loop) is performed on the target object outer surface pretreatment sample images in the target object outer surface pretreatment sample image collection, specifically, clipping is performed on a region with a Random ratio of 0.6-1.0 on the target object outer surface pretreatment sample images, and the clipped target object outer surface pretreatment sample images are used as input of a model to be trained;

(3) The Mosaic mosaics are adopted to realize data enhancement, specifically, four target object outer surface pretreatment sample images in a target object outer surface pretreatment sample image collection set are randomly spliced into one Mosaic image, and new spliced sample images are used as training data to be input for model training.

In the embodiment, multi-scale training is adopted, so that the risk of model overfitting is reduced, and the robustness of the target object crack detection model is enhanced.

In order to better implement the target object crack detection method in the embodiment of the present application, on the basis of the target object crack detection method, the embodiment of the present application further provides a target object crack detection device, as shown in fig. 3, where the target object crack detection device 500 includes:

an obtaining module 501, configured to obtain a target object video including a target object to be detected;

the identifying module 502 is configured to obtain an object external surface image of the object to be detected, for identifying an external surface of the object to be detected in the object video;

a preprocessing module 503, configured to perform contrast enhancement processing and filtering processing on the target object outer surface image, so as to obtain a target object outer surface preprocessed image;

the detection module 504 is configured to perform crack detection on the target object external surface pretreatment image, and determine whether the target object to be detected in the target object external surface pretreatment image has a crack.

The preprocessing module 503 is also specifically:

and the method is used for carrying out nonlinear stretching on the object outer surface image according to the histogram of the object outer surface image, enhancing the contrast of the object outer surface image and obtaining the object outer surface image with enhanced contrast.

The preprocessing module 503 is also specifically:

the method is used for carrying out edge enhancement on the surface image of the object after the contrast enhancement by adopting an edge detection method;

the method is used for carrying out filtering treatment on the object outer surface image after edge enhancement by adopting a morphological opening and closing operation method to obtain the object outer surface image after filtering treatment;

and the target object outer surface image after the filtering processing is used as the target object outer surface pretreatment image.

The detection module 504 is also specifically:

all noise areas in the pretreatment image of the outer surface of the object to be detected;

for determining whether the area size of all noise areas is greater than a preset noise area size threshold, and

and determining that the object to be detected in the object outer surface pretreatment image has no crack when the area sizes of all the noise areas are smaller than the noise area size threshold.

The target object crack detection device also comprises a storage module and a notification module,

The storage module is used for storing the object outer surface image to a preset object sample image library when the object to be detected in the object outer surface pretreatment image does not have cracks;

and the notification module is used for generating warning information of the crack of the object to be detected to the object receiving terminal when the crack of the object to be detected exists in the object surface pretreatment image.

The identification module 502 is also specifically:

the method comprises the steps of detecting a target object to be detected on a target object video to generate a target object detection frame;

the target object detection frame is used for intercepting an initial image of a target object in a corresponding area;

the method comprises the steps of determining a plurality of candidate corner points corresponding to a target object to be detected in an initial image of the target object;

and the method is used for generating an external surface image of the object according to the plurality of candidate corner points.

The identification module 502 is also specifically:

the method comprises the steps of detecting a plurality of initial corner points in an initial image of a target object by using a Harris corner point detector;

and the method is used for filtering a plurality of initial corner points which are not near the plurality of vertexes of the target object detection frame in the initial image of the target object according to Euler distances between the plurality of initial corner points and the plurality of vertexes of the target object detection frame, and taking the rest of the plurality of initial corner points in the initial image of the target object as a plurality of candidate corner points.

The identification module 502 is also specifically:

affine transformation is carried out on an irregular image area surrounded by a plurality of candidate angular points to obtain a regular image area;

for taking the regular image as the object outer surface image.

In another embodiment of the present application, as shown in fig. 4, the present application further provides an electronic device, which shows a schematic structural diagram of the electronic device according to the embodiment of the present application, specifically:

the electronic device may include one or more processing cores 'processors 601, one or more computer-readable storage media's memory 602, power supply 603, and input unit 604, among other components. Those skilled in the art will appreciate that the electronic device structure shown in fig. 4 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:

the processor 601 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 602, and calling data stored in the memory 602, thereby performing overall monitoring of the electronic device. Optionally, the processor 601 may include one or more processing cores; the processor 601 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and preferably, the processor 601 may integrate an application processor primarily handling operating systems, user interfaces, application programs, and the like, with a modem processor primarily handling wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 601.

The memory 602 may be used to store software programs and modules, and the processor 601 may execute various functional applications and data processing by executing the software programs and modules stored in the memory 602. The memory 602 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide access to the memory 602 by the processor 601.

The electronic device further comprises a power supply 603 for supplying power to the various components, preferably the power supply 603 may be logically connected to the processor 601 by a power management system, so that functions of charge, discharge, power consumption management and the like are managed by the power management system. The power supply 603 may also include one or more of any components, such as a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The electronic device may further comprise an input unit 604, which input unit 604 may be used for receiving input digital or character information and for generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.

Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 601 in the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 executes the application programs stored in the memory 602, so as to implement various functions as follows:

acquiring a target object video containing a target object to be detected;

identifying the outer surface of a target object to be detected in a target object video to obtain a target object outer surface image of the target object to be detected;

performing contrast enhancement treatment and filtering treatment on the target object outer surface image to obtain a target object outer surface pretreatment image;

Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.

In some embodiments of the present application, the present application also provides a computer-readable storage medium, which may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like. The method for detecting the target object crack comprises the steps of storing a computer program, wherein the computer program is loaded by a processor to execute the steps in the method for detecting the target object crack provided by the embodiment of the application. For example, the loading of the computer program by the processor may perform the steps of:

acquiring a target object video containing a target object to be detected;

In the foregoing embodiments, the descriptions of the embodiments are focused on, and the portions of one embodiment that are not described in detail in the foregoing embodiments may be referred to in the foregoing detailed description of other embodiments, which are not described herein again.

The foregoing describes in detail a target crack detection method, apparatus, electronic device and storage medium provided in the embodiments of the present application, and specific examples are applied to illustrate the principles and embodiments of the present invention, where the foregoing examples are only used to help understand the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present invention, the present description should not be construed as limiting the present invention.

Claims

1. A target crack detection method, comprising:

acquiring a target object video containing a target object to be detected;

2. The method for detecting a crack in an object according to claim 1, wherein the performing contrast enhancement processing and filtering processing on the image of the outer surface of the object to obtain a preprocessed image of the outer surface of the object comprises:

3. The method for detecting a crack in an object according to claim 2, wherein the performing contrast enhancement processing and filtering processing on the image of the outer surface of the object to obtain a preprocessed image of the outer surface of the object comprises:

4. The method for detecting a crack in a target object according to claim 1, wherein the step of performing crack detection on the target object external surface pretreatment image to determine whether the target object to be detected in the target object external surface pretreatment image has a crack, comprises:

5. The method for detecting a crack in a target object according to claim 4, wherein after the crack detection is performed on the target object outer surface pretreatment image to determine whether the target object to be detected in the target object outer surface pretreatment image has a crack, the method further comprises:

6. The method for detecting a crack in a target according to claim 1, wherein the identifying the outer surface of the target to be detected in the target video to obtain the target outer surface image of the target to be detected includes:

7. The method for detecting a crack in a target object according to claim 6, wherein determining a plurality of candidate corner points corresponding to the target object to be detected in the initial image of the target object includes:

8. The method for detecting a crack in a target object according to claim 7, wherein generating the image of the outer surface of the target object according to the plurality of candidate corner points comprises:

9. A target crack detection system, the system comprising:

10. A computer-readable storage medium, having stored thereon a computer program, the computer program being loaded by a processor to perform the steps of the object crack detection method according to any one of claims 1 to 8.