CN113034456B

CN113034456B - Bolt loosening detection method, device, equipment and storage medium

Info

Publication number: CN113034456B
Application number: CN202110288604.0A
Authority: CN
Inventors: 陈路燕; 邹建法; 聂磊
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-03-18
Filing date: 2021-03-18
Publication date: 2023-07-28
Anticipated expiration: 2041-03-18
Also published as: CN113034456A

Abstract

The application discloses a bolt loosening detection method, device, equipment and storage medium, relates to the technical field of artificial intelligence, and particularly relates to the technical field of deep learning and image processing. One embodiment of the method comprises the following steps: acquiring an image to be detected containing a bolt and a depth information image corresponding to the image to be detected; inputting an image to be detected into a pre-trained target detector to obtain the position information of a bolt in the image to be detected; and determining whether the bolts are loosened or not by utilizing the position information, the depth information map and the pre-trained classification model of the bolts. According to the embodiment, the position of the bolt is accurately positioned by means of target detection, whether the bolt is loosened or not is further determined, the accuracy of a detection result is improved, and the detection method is simple and easy to implement and high in robustness.

Description

Bolt loosening detection method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the field of computers, in particular to the field of artificial intelligence such as deep learning, image processing and the like, and particularly relates to a bolt loosening detection method, device and equipment and a storage medium.

Background

Bolts are a very important component in the field of industrial production. Railway, windmill, factory machinery, etc. have a large number of bolt applications. In these application scenarios, once the bolt is loosened, the device is likely to fail to operate normally, and thus serious consequences that are difficult to retrieve are likely to occur. Therefore, bolt loosening detection is one of the very important demands in the field of industrial production.

Taking a railway train as an example, the train is taken as an important transportation means in daily travel of the national people, and the safety of the train cannot be ignored. In order to ensure that the locomotive can safely run, railway staff needs to check the state of the locomotive every day, and one of the check items is to check whether bolts are loosened. The number of trains traveling nationally in one day is extremely large, and each train has a plurality of carriages, each carriage has a very large number of bolts, and the manpower demand for maintenance is very large.

Disclosure of Invention

The embodiment of the application provides detection, device and equipment for bolt loosening and a storage medium.

In a first aspect, an embodiment of the present application provides a method for detecting loosening of a bolt, including: acquiring an image to be detected containing a bolt and a depth information image corresponding to the image to be detected; inputting an image to be detected into a pre-trained target detector to obtain the position information of a bolt in the image to be detected; and determining whether the bolts are loosened or not by utilizing the position information, the depth information map and the pre-trained classification model of the bolts.

In a second aspect, an embodiment of the present application provides a detection apparatus for bolt loosening, including: the acquisition module is configured to acquire an image to be detected containing the bolt and a depth information image corresponding to the image to be detected; the input module is configured to input an image to be detected into a pre-trained target detector to obtain position information of a bolt in the image to be detected; and the determining module is configured to determine whether the bolt is loosened by utilizing the position information, the depth information graph and the pre-trained classification model of the bolt.

In a third aspect, an embodiment of the present application provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of the implementations of the first aspect.

In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a method as described in any implementation of the first aspect.

In a fifth aspect, embodiments of the present application propose a computer program product comprising a computer program which, when executed by a processor, implements a method as described in any of the implementations of the first aspect.

According to the bolt loosening detection method, device and equipment and the storage medium, firstly, an image to be detected containing a bolt and a depth information image corresponding to the image to be detected are obtained; inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected; and finally, determining whether the bolts are loosened or not by utilizing the position information, the depth information map and the pre-trained classification model of the bolts. The method accurately positions the bolt by means of target detection, and designs a classification model according to depth difference information between the surface of the bolt and the bottom plate, so that whether the bolt is loosened or not is determined, accuracy of a detection result is improved, and the detection method is simple and easy to implement and high in robustness.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:

FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;

FIG. 2 is a flow chart of one embodiment of a method of detecting bolt loosening according to the present application;

FIG. 3 is a flow chart of another embodiment of a method of detecting bolt loosening according to the present application;

FIG. 4 is an effect diagram of normalizing depth information;

FIG. 5 is an application scenario diagram of a bolt looseness detection method;

FIG. 6 is a schematic structural view of one embodiment of a bolt looseness detection device according to the present application;

fig. 7 is a block diagram of an electronic device for implementing a method of detecting bolt looseness according to an embodiment of the present application.

Detailed Description

Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

FIG. 1 illustrates an exemplary system architecture 100 to which embodiments of a bolt-loosening detection method or a bolt-loosening detection device of the present application may be applied.

As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user can interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or transmit images to be detected or the like. Various client applications, such as photographing software, etc., may be installed on the terminal devices 101, 102, 103.

The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices including, but not limited to, smartphones, tablets, laptop and desktop computers, and the like. When the terminal devices 101, 102, 103 are software, they can be installed in the above-described electronic devices. Which may be implemented as a plurality of software or software modules, or as a single software or software module. The present invention is not particularly limited herein.

The server 105 may provide various services. For example, the server 105 may analyze and process the image to be detected acquired from the terminal devices 101, 102, 103, and generate a processing result (for example, a determination result of whether or not the bolts are loosened).

The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster formed by a plurality of servers, or as a single server. When server 105 is software, it may be implemented as a plurality of software or software modules (e.g., to provide distributed services), or as a single software or software module. The present invention is not particularly limited herein.

It should be noted that, the method for detecting bolt loosening provided in the embodiments of the present application is generally executed by the server 105, and accordingly, the device for detecting bolt loosening is generally disposed in the server 105.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow 200 of one embodiment of a method of detecting bolt loosening according to the present application is shown. The bolt loosening detection method comprises the following steps:

step 201, obtaining an image to be detected containing a bolt and a depth information map corresponding to the image to be detected.

In this embodiment, an execution body (for example, the server 105 shown in fig. 1) of the bolt loosening detection method may acquire an image to be detected including a bolt and a depth information map corresponding to the image to be detected. The image to be detected may be an image including a bolt collected by a camera, and the image format is not particularly limited in this application, for example, the image to be detected may be a color image in RGB format collected by a common camera. The corresponding depth information map may be a depth information map corresponding to an image to be detected acquired using a depth camera.

Step 202, inputting an image to be detected into a pre-trained target detector to obtain position information of a bolt in the image to be detected.

In this embodiment, the executing body may input the acquired image to be detected into a target detector trained in advance, so as to obtain the position information of the bolt in the image to be detected. As an example, the object detector may be obtained by: acquiring a training sample set, wherein the training sample in the training sample set comprises RGB images containing bolts and labeling results of manual bolts in the images; and taking an RGB image containing the bolt as input, taking a labeling result of the bolt position in the image as output, and training to obtain the target detector. The target detector is more selected, and the target detector based on deep learning is used in the application, and other target detectors can be adopted according to actual situations, which is not particularly limited in the application. Inputting the image to be detected obtained in the step 201 to a pre-trained target detector, and outputting the position information of the bolt in the image to be detected, wherein the position information of the bolt can be coordinate information of the bolt in the image to be detected.

And 203, determining whether the bolts are loosened or not by using the position information, the depth information map and the pre-trained classification model of the bolts.

In this embodiment, the executing body may determine whether the bolt is loosened by using the position information, the depth information map, and the pre-trained classification model of the bolt obtained in step 202. Optionally, according to the positional information of the bolt obtained in step 202, a bolt area may be found in the depth information map corresponding to the image to be detected, and a corresponding local depth information map of the bolt may be segmented. As an example, the classification model may be obtained by: acquiring a training sample set, wherein the training samples in the training sample set comprise a local depth information graph of a sample bolt and a state label of the bolt, and the state label of the bolt is a label for indicating whether the bolt is loosened; and taking the local depth information graph of the sample bolt as input, taking the state label of the bolt as output, and training to obtain the classification model. And obtaining a judging result of whether the bolt is loosened or not by using the local depth information map and the trained classification model.

According to the bolt loosening detection method, firstly, an image to be detected containing a bolt and a depth information image corresponding to the image to be detected are obtained; inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected; and finally, determining whether the bolts are loosened or not by utilizing the position information, the depth information map and the pre-trained classification model of the bolts. According to the bolt loosening detection method, the position of the bolt is accurately positioned by means of target detection, the classification model is designed according to the depth difference information between the surface of the bolt and the bottom plate, whether the bolt is loosened or not is further determined, the accuracy of a detection result is improved, and the detection method is simple and easy to achieve and high in robustness.

With continued reference to fig. 3, fig. 3 illustrates a flow 300 of another embodiment of a method of detecting bolt loosening according to the present application. The bolt loosening detection method comprises the following steps:

step 301, obtaining an image to be detected containing a bolt and a depth information map corresponding to the image to be detected.

In this embodiment, the execution body of the bolt loosening detection method may acquire the image to be detected including the bolt and the depth information map corresponding to the image to be detected. Step 301 corresponds to step 201 of the foregoing embodiment, and the specific implementation may refer to the foregoing description of step 201, which is not repeated here.

Step 302, inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected.

In this embodiment, the executing body may input the acquired image to be detected into a target detector trained in advance, so as to obtain the position information of the bolt in the image to be detected. Step 302 corresponds to step 202 of the foregoing embodiment, and the specific implementation may refer to the foregoing description of step 202, which is not repeated here.

Step 303, extracting a local depth information map corresponding to the position information of the bolt in the depth information map.

In this embodiment, the execution body may extract a partial depth information map corresponding to the position information of the bolt from the depth information map. As an example, the executing body may find a bolt area in the depth information map corresponding to the image to be detected, which is obtained in step 301, according to the position information of the bolt in the image to be detected, which is obtained in step 302, and cut out a corresponding local depth information map of the bolt. The partial depth information map corresponding to the bolt position is cut out for subsequent normalization processing.

And 304, determining whether the bolts are loosened or not by utilizing the local depth information map and the pre-trained classification model.

In this embodiment, the executing body may determine whether the bolt is loosened by using the local depth information map obtained in step 303 and the pre-trained classification model. The classification model is a classification model, and a judging result of whether the bolt is loosened can be obtained by inputting the local depth information graph of the bolt.

In some alternative implementations of the present embodiment, step 304 determines whether the bolt is loose using the local depth information map and the pre-trained classification model, including: normalizing the depth information in the local depth information map to obtain a normalized local information map; and inputting the normalized local information graph into a pre-trained classification model to obtain a judging result of whether the bolts are loosened. On the bolt local depth information map, the normalization of the depth information is performed because the bolt area may occupy a relatively small area on the whole image to be detected, and the depth value distribution range of the whole depth information map is larger, if the normalization is performed directly on the whole depth information map, the normalization is performed on the local depth information map based on the position information of the bolt because the depth difference between the surface of the bolt and the bolt bottom plate is smaller, so that whether the bolt loosens or not is not easily distinguished is not facilitated, the local depth information map of the bolt is firstly segmented based on the position information of the bolt, and then the normalization processing is performed on the local depth information map. And then inputting the normalized local information graph into a pre-trained classification model, and obtaining a judging result of whether the bolt loosens or not. In the partial information graph obtained after normalization, the difference between the loosened and unrelieved depth information of the bolt is obvious, so that whether the bolt is loosened or not can be accurately judged, and a judging result of whether the bolt is loosened or not can be obtained.

In some alternative implementations of the present embodiment, the normalized range is from a minimum value to a maximum value after removing the 0 value. In acquiring a depth information map, some errors may be generated in the acquired depth information map due to the accuracy problem of the depth camera, such as a depth value of 0 in a shadow portion. Therefore, in this embodiment, the normalization range may use the minimum value to the maximum value after removing the 0 value, so as to more accurately determine whether the bolt is loosened. Referring to fig. 4, fig. 4 is an effect diagram after normalizing depth information, as shown in fig. 4, when global normalization (the minimum value to the maximum value of the whole graph) is performed, the difference between loosening and loosening of the bolt is not obvious, which is not beneficial to judging whether the bolt is loosened; when local normalization (local minimum value to maximum value), the difference between loosening and loosening of the bolt is not obvious, and whether the bolt is loosened is not beneficial to judging; and when the local normalization (the minimum value to the maximum value after the 0 value is locally removed), the difference between loosening and unreleasing of the bolt is obvious, and whether the bolt is loosened can be accurately judged.

In some optional implementations of this embodiment, the training step of the classification model includes: acquiring a training sample set, wherein the training sample in the training sample set comprises a local information graph of a sample bolt and a state label of the bolt, and the state label of the bolt is a label for indicating whether the bolt is loosened; and taking a local information graph of the sample bolt as input, taking a state label of the bolt as output, and training to obtain a classification model.

According to the bolt loosening detection method, firstly, an image to be detected containing a bolt and a depth information image corresponding to the image to be detected are obtained; inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected; then extracting a local depth information map corresponding to the position information of the bolt in the depth information map; and finally, determining whether the bolts are loosened or not by utilizing the local depth information map and the pre-trained classification model. According to the bolt loosening detection method, the position of the bolt is accurately positioned by means of target detection, the classification model is designed according to the depth difference information between the surface of the bolt and the bottom plate, whether the bolt is loosened or not is further determined, the accuracy of a detection result is improved, and the detection method is simple and easy to achieve and high in robustness.

With continued reference to fig. 5, fig. 5 is an application scenario of the bolt loosening detection method. As shown in fig. 5, firstly, an RGB image to be detected containing a bolt and a depth information map corresponding to the image to be detected are obtained; inputting the RGB image into a pre-trained target detector, so as to obtain the position information of the bolt in the image; based on the position information, segmenting a local depth information map of the bolt from the depth information map, and carrying out normalization processing on the local depth information map to obtain a normalized local information map; and finally, inputting the normalized local information graph into a pre-trained classification model, so as to obtain judging information of whether the bolts are loosened. Fig. 5 shows RGB images of two states of loosening and unreleasing of the bolt and a corresponding depth information image and a bolt local information image, and it can be seen that when the normalization processing is performed on the bolt local depth information image, the difference between loosening and unreleasing of the bolt is obvious in the obtained normalized local information image, and whether the bolt is loosened can be accurately judged.

With further reference to fig. 6, as an implementation of the method shown in the foregoing drawings, the present application provides an embodiment of a bolt loosening detection device, where the embodiment of the device corresponds to the embodiment of the method shown in fig. 2, and the device may be specifically applied to various electronic devices.

As shown in fig. 6, the bolt looseness detecting apparatus 600 of the present embodiment may include: an acquisition module 601, an input module 602 and a determination module 603. The acquiring module 601 is configured to acquire an image to be detected containing a bolt and a depth information map corresponding to the image to be detected; the input module 602 is configured to input an image to be detected into a pre-trained target detector to obtain position information of a bolt in the image to be detected; the determining module 603 is configured to determine whether the bolt is loose using the positional information of the bolt, the depth information map, and the pre-trained classification model.

In the present embodiment, in the bolt loosening detection device 600: the specific processing and the technical effects of the obtaining module 601, the input module 602, and the determining module 603 may refer to the relevant descriptions of the steps 201 to 203 in the corresponding embodiment of fig. 2, and are not repeated herein.

In some optional implementations of this embodiment, the determining module includes: the extraction sub-module is configured to extract a local depth information map corresponding to the position information of the bolt in the depth information map; a determination sub-module configured to determine whether the bolt is loose using the local depth information map and the pre-trained classification model.

In some optional implementations of the present embodiment, the determination submodule is further configured to: normalizing the depth information in the local depth information map to obtain a normalized local information map; and inputting the normalized local information graph into a pre-trained classification model to obtain a judging result of whether the bolts are loosened.

In some alternative implementations of the present embodiment, the normalized range is the minimum value to the maximum value after removing the 0 value.

According to embodiments of the present application, there is also provided an electronic device, a readable storage medium and a computer program product.

Fig. 7 illustrates a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 7, the apparatus 700 includes a computing unit 701 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 may also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Various components in device 700 are connected to I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, etc.; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, an optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The computing unit 701 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The calculation unit 701 performs the respective methods and processes described above, for example, a bolt looseness detection method. For example, in some embodiments, the method of detecting bolt looseness may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 700 via ROM 702 and/or communication unit 709. When the computer program is loaded into the RAM 703 and executed by the calculation unit 701, one or more steps of the above-described bolt looseness detection method may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the bolt looseness detection method in any other suitable way (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A bolt loosening detection method comprises the following steps:

acquiring an image to be detected containing a bolt and a depth information image corresponding to the image to be detected;

inputting the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected;

determining whether the bolt is loose or not by using the position information of the bolt, the depth information map and a pre-trained classification model, including:

extracting a local depth information map corresponding to the position information of the bolt in the depth information map;

normalizing the depth information in the local depth information map to obtain a normalized local information map;

and inputting the normalized local information graph into a pre-trained classification model to obtain a judging result of whether the bolt is loosened, wherein the normalization range is from a minimum value to a maximum value after 0 value is removed, and the classification model is designed through depth difference information between the surface of the bolt and the bottom plate so as to determine whether the bolt is loosened.

2. The method of claim 1, wherein the training of the classification model comprises:

acquiring a training sample set, wherein the training samples in the training sample set comprise a local information graph of a sample bolt and a state label of the bolt, and the state label of the bolt is a label for indicating whether the bolt is loosened;

and taking the local information graph of the sample bolt as input, taking the state label of the bolt as output, and training to obtain the classification model.

3. A bolt loosening detection device, comprising:

the acquisition module is configured to acquire an image to be detected containing the bolt and a depth information image corresponding to the image to be detected;

the input module is configured to input the image to be detected into a pre-trained target detector to obtain the position information of the bolt in the image to be detected;

a determination module configured to determine whether the bolt is loose using the positional information of the bolt, the depth information map, and a pre-trained classification model, comprising:

an extraction sub-module configured to extract a local depth information map corresponding to the position information of the bolt in the depth information map;

a determination submodule configured to: normalizing the depth information in the local depth information map to obtain a normalized local information map; and inputting the normalized local information graph into a pre-trained classification model to obtain a judging result of whether the bolt is loosened, wherein the normalization range is from a minimum value to a maximum value after 0 value is removed, and the classification model is designed through depth difference information between the surface of the bolt and the bottom plate so as to determine whether the bolt is loosened.

4. The apparatus of claim 3, wherein the training of the classification model comprises:

5. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of claim 1 or 2.

6. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of claim 1 or 2.