CN113034456A

CN113034456A - Bolt looseness detection method, device, equipment and storage medium

Info

Publication number: CN113034456A
Application number: CN202110288604.0A
Authority: CN
Inventors: 陈路燕; 邹建法; 聂磊
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-03-18
Filing date: 2021-03-18
Publication date: 2021-06-25
Anticipated expiration: 2041-03-18
Also published as: CN113034456B

Abstract

The application discloses a bolt looseness detection method, a bolt looseness detection device, bolt looseness detection equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the technical field of deep learning and image processing. One embodiment of the method comprises: acquiring an image to be detected containing a bolt and a depth information map corresponding to the image to be detected; inputting an image to be detected into a pre-trained target detector to obtain the position information of a bolt in the image to be detected; and determining whether the bolt is loosened or not by utilizing the position information, the depth information graph and the pre-trained classification model of the bolt. According to the embodiment, the position of the bolt is accurately positioned by means of target detection, so that whether the bolt is loosened or not is determined, the accuracy of a detection result is improved, and the detection method is simple and easy to implement and high in robustness.

Description

Bolt looseness detection method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the field of computers, in particular to the field of artificial intelligence such as deep learning and image processing, and particularly relates to a bolt looseness detection method, device, equipment and storage medium.

Background

A bolt is a very important component in the field of industrial production. The equipment such as railways, windmills, factory machinery and the like has a large number of bolt application scenes. In these application scenarios, once the bolt is loosened, the equipment may not operate normally, and serious consequences which are difficult to recover may be caused. Therefore, bolt looseness detection is one of the very important requirements in the field of industrial production.

Taking a train of a railway as an example, the safety of the train as an important transportation tool in daily travel of the national is not neglected. In order to ensure the safe operation of the locomotive, the railway staff needs to perform a state check on the locomotive every day, and one of the check items is to check whether the bolt is loose. The number of trains traveling nationwide in one day is extremely large, each train has a plurality of carriages, each carriage has a large number of bolts, and the manpower demand for maintenance is very large.

Disclosure of Invention

The embodiment of the application provides detection, a device, equipment and a storage medium for bolt looseness.

In a first aspect, an embodiment of the present application provides a method for detecting bolt looseness, including: acquiring an image to be detected containing a bolt and a depth information map corresponding to the image to be detected; inputting an image to be detected into a pre-trained target detector to obtain the position information of a bolt in the image to be detected; and determining whether the bolt is loosened or not by utilizing the position information, the depth information graph and the pre-trained classification model of the bolt.

In a second aspect, an embodiment of the present application provides a bolt looseness detection device, including: the acquisition module is configured to acquire an image to be detected containing a bolt and a depth information map corresponding to the image to be detected; the input module is configured to input the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected; and the determining module is configured to determine whether the bolt is loosened by using the position information, the depth information map and a pre-trained classification model of the bolt.

In a third aspect, an embodiment of the present application provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of the implementations of the first aspect.

In a fourth aspect, embodiments of the present application propose a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method as described in any one of the implementations of the first aspect.

In a fifth aspect, the present application provides a computer program product, which includes a computer program that, when executed by a processor, implements the method as described in any implementation manner of the first aspect.

According to the method, the device, the equipment and the storage medium for detecting the bolt looseness, firstly, an image to be detected containing the bolt and a depth information map corresponding to the image to be detected are obtained; inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected; and finally, determining whether the bolt is loosened or not by utilizing the position information, the depth information graph and the pre-trained classification model of the bolt. The application provides a bolt looseness detection method, the method accurately positions the position of a bolt by means of target detection, a classification model is designed according to depth difference information between the surface of the bolt and a bottom plate, whether the bolt is loosened or not is further determined, accuracy of a detection result is improved, the detection method is simple and easy to implement, and robustness is high.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings. The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:

FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;

FIG. 2 is a flow chart of one embodiment of a method of detecting bolt loosening according to the present application;

FIG. 3 is a flow chart of another embodiment of a method of detecting bolt loosening according to the present application;

FIG. 4 is an effect diagram after normalization of depth information;

FIG. 5 is a view of an application scenario of the bolt looseness detection method;

FIG. 6 is a schematic structural view of one embodiment of a bolt loosening detection arrangement according to the present application;

fig. 7 is a block diagram of an electronic device for implementing the bolt loosening detection method according to the embodiment of the present application.

Detailed Description

The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of the bolt loosening detection method or bolt loosening detection apparatus of the present application may be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user can use the

terminal devices

101, 102, 103 to interact with the server 105 via the network 104 to receive or transmit images to be detected or the like. Various client applications, such as photographing software and the like, may be installed on the

terminal devices

101, 102, 103.

The

terminal apparatuses

101, 102, and 103 may be hardware or software. When the

terminal devices

101, 102, 103 are hardware, they may be various electronic devices including, but not limited to, smart phones, tablet computers, laptop portable computers, desktop computers, and the like. When the

terminal apparatuses

101, 102, 103 are software, they can be installed in the above-described electronic apparatuses. It may be implemented as multiple pieces of software or software modules, or as a single piece of software or software module. And is not particularly limited herein.

The server 105 may provide various services. For example, the server 105 may analyze and process the images to be detected acquired from the

terminal apparatuses

101, 102, 103, and generate a processing result (e.g., a determination result of whether or not the bolt is loose).

The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.

It should be noted that the bolt looseness detection method provided in the embodiment of the present application is generally executed by the server 105, and accordingly, a bolt looseness detection device is generally disposed in the server 105.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow 200 of one embodiment of a method of detecting bolt loosening according to the present application is shown. The bolt looseness detection method comprises the following steps:

step 201, acquiring an image to be detected containing a bolt and a depth information map corresponding to the image to be detected.

In the present embodiment, an execution subject (for example, the server 105 shown in fig. 1) of the bolt looseness detection method may acquire an image to be detected containing a bolt and a depth information map corresponding to the image to be detected. The image to be detected can be an image which is acquired by a camera and contains a bolt, the image format is not particularly limited in the application, and for example, the image to be detected can be a color image which is acquired by a common camera and has an RGB format. The corresponding depth information map may be a depth information map corresponding to an image to be detected, which is acquired using a depth camera.

Step 202, inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected.

In this embodiment, the executing body may input the acquired image to be detected to a pre-trained target detector to obtain the position information of the bolt in the image to be detected. As an example, the target detector may be obtained by: acquiring a training sample set, wherein training samples in the training sample set comprise RGB images containing bolts and manual labeling results of bolt positions in the images; and (3) taking the RGB image containing the bolt as input, taking the marking result of the position of the bolt in the image as output, and training to obtain the target detector. The selection of the target detector is more, the target detector based on deep learning is used in the application, and other target detectors can be adopted according to the actual situation, which is not specifically limited in the application. Inputting the image to be detected acquired in step 201 into a pre-trained target detector, and outputting the position information of the bolt in the image to be detected, wherein the position information of the bolt can be the coordinate information of the bolt in the image to be detected.

And step 203, determining whether the bolt is loosened by using the position information, the depth information map and the pre-trained classification model of the bolt.

In this embodiment, the execution subject may determine whether the bolt is loose by using the position information of the bolt obtained in step 202, the depth information map, and a classification model trained in advance. Alternatively, according to the position information of the bolt obtained in step 202, a bolt region may be found in the depth information map corresponding to the image to be detected, and a corresponding bolt local depth information map may be cut out. As an example, the classification model may be obtained by: acquiring a training sample set, wherein the training sample in the training sample set comprises a local depth information map of a sample bolt and a state label of the bolt, and the state label of the bolt is a label for indicating whether the bolt is loosened; and taking the local depth information map of the sample bolt as input, taking the state label of the bolt as output, and training to obtain the classification model. Therefore, the judgment result of whether the bolt is loosened or not is obtained by using the local depth information map and the trained classification model.

According to the bolt looseness detection method provided by the embodiment of the application, firstly, an image to be detected containing a bolt and a depth information map corresponding to the image to be detected are obtained; inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected; and finally, determining whether the bolt is loosened or not by utilizing the position information, the depth information graph and the pre-trained classification model of the bolt. According to the bolt looseness detection method, the position of the bolt is accurately positioned by means of target detection, the classification model is designed according to the depth difference information between the surface of the bolt and the bottom plate, whether the bolt is loosened or not is further determined, the accuracy of a detection result is improved, the detection method is simple and easy to achieve, and the robustness is high.

With continued reference to fig. 3, fig. 3 illustrates a flow 300 of another embodiment of a method of detecting bolt loosening according to the present application. The bolt looseness detection method comprises the following steps:

step 301, acquiring an image to be detected containing a bolt and a depth information map corresponding to the image to be detected.

In this embodiment, the execution subject of the bolt looseness detection method may acquire an image to be detected including the bolt and a depth information map corresponding to the image to be detected. Step 301 corresponds to step 201 of the foregoing embodiment, and the specific implementation manner may refer to the foregoing description of step 201, which is not described herein again.

Step 302, inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected.

In this embodiment, the executing body may input the acquired image to be detected to a pre-trained target detector to obtain the position information of the bolt in the image to be detected. Step 302 corresponds to step 202 of the foregoing embodiment, and the detailed implementation manner may refer to the foregoing description of step 202, which is not described herein again.

Step 303, extracting a local depth information map corresponding to the position information of the bolt in the depth information map.

In this embodiment, the execution body may extract a partial depth information map corresponding to position information of the bolt in the depth information map. As an example, the executing body may find a bolt region in the depth information map corresponding to the image to be detected obtained in step 301 according to the position information of the bolt in the image to be detected obtained in step 302, and cut out a corresponding bolt local depth information map. The partial depth information graph corresponding to the bolt position is cut out for subsequent normalization processing.

And step 304, determining whether the bolt is loosened by using the local depth information map and a pre-trained classification model.

In this embodiment, the execution subject may determine whether the bolt is loose by using the local depth information map obtained in step 303 and a classification model trained in advance. The classification model is a binary classification model, and a judgment result of whether the bolt is loosened can be obtained by inputting a local depth information map of the bolt.

In some optional implementations of the present embodiment, the step 304 of determining whether the bolt is loose by using the local depth information map and a pre-trained classification model includes: normalizing the depth information in the local depth information map to obtain a normalized local information map; and inputting the normalized local information graph into a classification model trained in advance to obtain a judgment result of whether the bolt is loosened. On the bolt local depth information map, the normalization of the depth information is performed because the bolt area possibly occupies a small area on the whole image to be detected, the depth value distribution range of the whole depth information map is large, if the normalization is directly performed on the whole depth information map, the depth difference between the bolt surface and the bolt bottom plate is small, and the bolt is not beneficial to distinguishing whether the bolt is loosened, so that the local depth information map of the bolt is cut out based on the position information of the bolt, and then the normalization processing is performed on the local depth information map. And then inputting the normalized local information graph into a classification model trained in advance, so that a judgment result of whether the bolt is loosened can be obtained. In the local information graph obtained after normalization, the depth information difference between loosened bolts and unreleased bolts is obvious, so that whether the bolts are loosened or not can be accurately judged, and the judgment result of whether the bolts are loosened or not is obtained.

In some optional implementations of this embodiment, the normalized range is from the minimum value to the maximum value after removing the 0 value. When acquiring the depth information map, some errors may be generated in the acquired depth information map due to the accuracy problem of the depth camera, for example, the depth value of the shadow portion is 0. Therefore, in the present embodiment, the normalization range may adopt a range from the minimum value to the maximum value after the 0 value is removed, so as to more accurately determine whether the bolt is loosened. Referring to fig. 4, fig. 4 is an effect diagram after normalization of depth information, as shown in fig. 4, when global normalization (from the minimum value to the maximum value of the whole diagram) is performed, the difference between bolt loosening and bolt loosening is not obvious, and it is not beneficial to judge whether the bolt is loosened; when the local normalization is carried out (from the local minimum value to the maximum value), the difference between the loosening and the non-loosening of the bolt is not obvious, and the judgment on whether the bolt is loosened or not is not facilitated; when the bolt is locally normalized (the minimum value is to the maximum value after 0 value is locally removed), the difference between the bolt loosening and the bolt not loosening is obvious, and whether the bolt is loosened or not can be accurately judged.

In some optional implementations of this embodiment, the training of the classification model includes: acquiring a training sample set, wherein the training sample in the training sample set comprises a local information graph of a sample bolt and a state label of the bolt, and the state label of the bolt is a label for indicating whether the bolt is loosened; and taking the local information graph of the sample bolt as input, taking the state label of the bolt as output, and training to obtain a classification model.

According to the bolt looseness detection method provided by the embodiment of the application, firstly, an image to be detected containing a bolt and a depth information map corresponding to the image to be detected are obtained; inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected; then extracting a local depth information map corresponding to the position information of the bolt in the depth information map; and finally, determining whether the bolt is loosened or not by utilizing the local depth information graph and a pre-trained classification model. According to the bolt looseness detection method, the position of the bolt is accurately positioned by means of target detection, the classification model is designed according to the depth difference information between the surface of the bolt and the bottom plate, whether the bolt is loosened or not is further determined, the accuracy of a detection result is improved, the detection method is simple and easy to achieve, and the robustness is high.

With continued reference to fig. 5, fig. 5 is an application scenario of the bolt loosening detection method. As shown in fig. 5, firstly, an RGB image to be detected including a bolt and a depth information map corresponding to the image to be detected are obtained; inputting the RGB image into a pre-trained target detector so as to obtain the position information of the bolt in the image; based on the position information, a bolt local depth information map is cut out from the depth information map, and normalization processing is carried out on the local depth information map to obtain a normalized local information map; and finally, inputting the normalized local information graph into a classification model trained in advance, so as to obtain judgment information of whether the bolt is loosened. Fig. 5 shows RGB images of two states of bolt loosening and bolt loosening, and depth information maps and bolt local information maps corresponding to the RGB images, and it can be seen that, after normalization processing is performed on the bolt local depth information maps, in the obtained normalized local information maps, the difference between bolt loosening and bolt loosening is obvious, and whether the bolt is loosened can be accurately determined.

With further reference to fig. 6, as an implementation of the method shown in the above figures, the present application provides an embodiment of a bolt looseness detection apparatus, which corresponds to the method embodiment shown in fig. 2, and which can be applied to various electronic devices.

As shown in fig. 6, the bolt looseness detection apparatus 600 of the present embodiment may include: an acquisition module 601, an input module 602, and a determination module 603. The acquiring module 601 is configured to acquire an image to be detected containing a bolt and a depth information map corresponding to the image to be detected; an input module 602, configured to input an image to be detected into a pre-trained target detector, so as to obtain position information of a bolt in the image to be detected; a determining module 603 configured to determine whether the bolt is loose using the position information of the bolt, the depth information map, and a pre-trained classification model.

In the present embodiment, in the bolt looseness detection apparatus 600: the specific processing and the technical effects thereof of the obtaining module 601, the input module 602 and the determining module 603 can refer to the related descriptions of step 201 and step 203 in the corresponding embodiment of fig. 2, which are not repeated herein.

In some optional implementations of this embodiment, the determining module includes: an extraction submodule configured to extract a partial depth information map corresponding to position information of the bolt in the depth information map; and the determining submodule is configured to determine whether the bolt is loosened by using the local depth information map and a pre-trained classification model.

In some optional implementations of this embodiment, the determining sub-module is further configured to: normalizing the depth information in the local depth information map to obtain a normalized local information map; and inputting the normalized local information graph into a classification model trained in advance to obtain a judgment result of whether the bolt is loosened.

In some optional implementations of this embodiment, the normalized range is from the minimum value to the maximum value after removing the 0 value.

In some optional implementations of this embodiment, the training step of the classification model includes: acquiring a training sample set, wherein the training sample in the training sample set comprises a local information graph of a sample bolt and a state label of the bolt, and the state label of the bolt is a label for indicating whether the bolt is loosened; and taking the local information graph of the sample bolt as input, taking the state label of the bolt as output, and training to obtain a classification model.

There is also provided, in accordance with an embodiment of the present application, an electronic device, a readable storage medium, and a computer program product.

FIG. 7 illustrates a schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 7, the device 700 comprises a computing unit 701, which may perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM)702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Various components in the device 700 are connected to the I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, or the like; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

Computing unit 701 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 701 executes the respective methods and processes described above, such as the detection method of bolt looseness. For example, in some embodiments, the bolt loosening detection method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 700 via ROM 702 and/or communications unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the bolt loosening detection method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the bolt loosening detection method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A bolt loosening detection method comprises the following steps:

acquiring an image to be detected containing a bolt and a depth information map corresponding to the image to be detected;

inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected;

and determining whether the bolt is loosened or not by utilizing the position information of the bolt, the depth information map and a pre-trained classification model.

2. The method of claim 1, wherein the determining whether the bolt is loose using the position information of the bolt, the depth information map, and a pre-trained classification model comprises:

extracting a local depth information map corresponding to the position information of the bolt in the depth information map;

and determining whether the bolt is loosened or not by utilizing the local depth information map and a pre-trained classification model.

3. The method of claim 2, wherein the determining whether the bolt is loose using the local depth information map and a pre-trained classification model comprises:

normalizing the depth information in the local depth information map to obtain a normalized local information map;

and inputting the normalized local information graph into a classification model trained in advance to obtain a judgment result of whether the bolt is loosened.

4. The method of claim 3, wherein the normalized range is from a minimum value to a maximum value with 0 values removed.

5. The method according to any one of claims 1-4, wherein the training of the classification model comprises:

acquiring a training sample set, wherein the training sample in the training sample set comprises a local information graph of a sample bolt and a state label of the bolt, and the state label of the bolt is a label for indicating whether the bolt is loosened;

and taking the local information graph of the sample bolt as input, taking the state label of the bolt as output, and training to obtain the classification model.

6. A bolt loosening detection device, comprising:

the device comprises an acquisition module, a processing module and a display module, wherein the acquisition module is configured to acquire an image to be detected containing a bolt and a depth information map corresponding to the image to be detected;

the input module is configured to input the image to be detected into a pre-trained target detector to obtain position information of a bolt in the image to be detected;

a determination module configured to determine whether the bolt is loose using the position information of the bolt, the depth information map, and a pre-trained classification model.

7. The apparatus of claim 6, wherein the means for determining comprises:

an extraction sub-module configured to extract a partial depth information map corresponding to position information of the bolt in the depth information map;

a determining submodule configured to determine whether the bolt is loose using the partial depth information map and a pre-trained classification model.

8. The apparatus of claim 7, wherein the determination submodule is further configured to:

9. The apparatus of claim 8, wherein the normalized range is a minimum to maximum value with 0 values removed.

10. The apparatus according to any one of claims 6-9, wherein the training of the classification model comprises:

11. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-5.

13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-5.