CN111666958A - Method, device, equipment and medium for detecting equipment state based on image recognition - Google Patents
Method, device, equipment and medium for detecting equipment state based on image recognition Download PDFInfo
- Publication number
- CN111666958A CN111666958A CN201910164194.1A CN201910164194A CN111666958A CN 111666958 A CN111666958 A CN 111666958A CN 201910164194 A CN201910164194 A CN 201910164194A CN 111666958 A CN111666958 A CN 111666958A
- Authority
- CN
- China
- Prior art keywords
- image
- detected
- feature map
- target
- target equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000001514 detection method Methods 0.000 claims abstract description 30
- 238000000605 extraction Methods 0.000 claims abstract description 27
- 238000010586 diagram Methods 0.000 claims abstract description 20
- 238000012549 training Methods 0.000 claims abstract description 20
- 230000007246 mechanism Effects 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 11
- 230000001629 suppression Effects 0.000 claims abstract description 11
- 238000013507 mapping Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 11
- 230000004927 fusion Effects 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 5
- 238000007689 inspection Methods 0.000 abstract description 12
- 230000009286 beneficial effect Effects 0.000 abstract description 5
- 230000000694 effects Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The application discloses an equipment state detection method based on image recognition, which comprises the following steps: acquiring a feature map of an image to be detected by using a feature extraction network obtained by pre-training; according to a computer vision attention mechanism, increasing the pixel weight of the target equipment in the feature map, and reducing the pixel weight of the background information in the feature map; inputting the feature map with the adjusted pixel weight into a multi-scale prediction network, and marking target equipment on an image to be detected by using prediction frames with different sizes; and determining a target frame in the plurality of prediction frames by using the soft-interval non-maximum suppression processing network to obtain the state information of the target equipment. According to the method, the accuracy of identifying the state of the target equipment can be relatively improved and the false alarm rate of routing inspection can be reduced by adjusting the pixel weights of the target equipment and the background information in the characteristic diagram. The application also discloses an equipment state detection device based on image recognition, equipment and a computer readable storage medium, which have the beneficial effects.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for detecting a device status based on image recognition.
Background
With the rapid development of electric power utilities in China, the number of substations serving as hubs for electric power transmission of electric power systems is increasing. Since a large number of secondary devices such as meters, switches, and oil level gauges exist in a substation, it is necessary to periodically perform inspection of devices in the substation in order to ensure the safety of an electric power system. At present, the inspection method of the transformer substation is transiting from traditional manual inspection to automatic inspection. The inspection robot acquires the equipment image, so that the equipment image is automatically identified and detected, the equipment condition in the transformer substation is obtained, and the automatic inspection of the transformer substation is realized.
In the prior art, image features of an image to be detected are generally extracted through a feature extraction network to obtain a plurality of feature maps; then, determining prediction frames with different sizes of the target equipment through a multi-scale prediction network; determining a target frame of the target equipment by using a soft interval non-maximum suppression processing network; and obtaining the state information of each target device. According to the method in the prior art, the image characteristics of the image to be detected are equivalently processed, so that the accuracy of the state of the identified target equipment is not high enough, and the false report of the routing inspection result is caused.
Therefore, how to identify the accuracy of the target device state when detecting the target device state by using the image recognition technology is a technical problem that needs to be solved by those skilled in the art at present.
Disclosure of Invention
In view of this, the present invention provides an apparatus state detection method based on image recognition, which can relatively improve the accuracy of recognizing the state of a target apparatus, and achieve the effect of reducing the false alarm rate of inspection; another object of the present invention is to provide an apparatus, a device and a computer-readable storage medium for detecting device status based on image recognition, all of which have the above advantages.
In order to solve the above technical problem, the present invention provides an apparatus status detection method based on image recognition, including:
acquiring a feature map of an image to be detected by using a feature extraction network obtained by pre-training;
according to a computer vision attention mechanism, increasing the pixel weight of the target equipment in the feature map, and reducing the pixel weight of the background information in the feature map;
inputting the feature map with the adjusted pixel weight into a multi-scale prediction network, and marking the target equipment on the image to be detected by using prediction frames with different sizes;
and determining a target frame in the plurality of prediction frames by using a soft-interval non-maximum suppression processing network to obtain the state information of the target equipment.
Preferably, the inputting the feature map after the pixel weight adjustment to a multi-scale prediction network, and marking the target device on the image to be detected by using prediction frames with different sizes specifically includes:
inputting the feature map with the adjusted pixel weight to the multi-scale prediction network;
respectively fusing two adjacent layers of feature maps with different scales by using the multi-scale prediction network;
and marking the target equipment in the image to be detected by using prediction frames with different sizes according to the mapping relation between the fused feature map and the image to be detected.
Preferably, the marking the target device in the image to be detected by using the prediction frames with different sizes according to the mapping relationship between the fused feature map and the image to be detected specifically includes:
acquiring a sample image of the target equipment and counting the length-width ratio of each target equipment in the sample image;
classifying the length-width ratios, and determining a plurality of target sizes for marking the target equipment in the image to be detected according to a preset rule;
and marking the target equipment in the image to be detected by using a prediction frame of each target size according to the mapping relation between the fused feature map and the image to be detected.
Preferably, the obtaining of the feature map of the image to be detected by using the pre-trained feature extraction network specifically comprises:
and acquiring the characteristic diagram of the image to be detected by using a Dense Net network obtained by pre-training.
Preferably, the obtaining of the feature map of the image to be detected by using the previously trained Dense Net network specifically comprises:
and acquiring a characteristic diagram of the image to be detected by using a Dense Net network obtained by pre-training according to a preset time period.
In order to solve the above technical problem, the present invention further provides an apparatus for detecting a device status based on image recognition, including:
the characteristic extraction module is used for acquiring a characteristic diagram of the image to be detected by utilizing a characteristic extraction network obtained by pre-training;
the pixel weight adjusting module is used for increasing the pixel weight of the target equipment in the feature map and reducing the pixel weight of the background information in the feature map according to a computer visual attention mechanism;
the setting module is used for inputting the feature map after the pixel weight is adjusted into a multi-scale prediction network and marking the target equipment on the image to be detected by using prediction frames with different sizes;
and the determining module is used for determining a target frame in the plurality of prediction frames by using the soft interval non-maximum suppression processing network to obtain the state information of the target equipment.
Preferably, the setting module specifically includes:
the input unit is used for inputting the feature map after the pixel weight is adjusted to the multi-scale prediction network;
the fusion unit is used for fusing the feature maps of two adjacent layers with different scales by utilizing the multi-scale prediction network;
and the marking unit is used for marking the target equipment in the image to be detected by using the prediction frames with different sizes according to the mapping relation between the fused feature map and the image to be detected.
In order to solve the above technical problem, the present invention further provides an apparatus status detecting apparatus based on image recognition, including:
a memory for storing a computer program;
and the processor is used for realizing the steps of any one of the device state detection methods based on the image recognition when the computer program is executed.
In order to solve the above technical problem, the present invention further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements the steps of any one of the above-mentioned device status detection methods based on image recognition.
Compared with the prior art, the method for detecting the equipment state based on the image recognition is characterized in that after the feature extraction network obtained by pre-training is used for obtaining the feature map of the image to be detected, the feature map is further adjusted according to a computer vision attention system, the accuracy of feature extraction of the target equipment in the image to be detected can be relatively improved by increasing the pixel weight of the target equipment in the feature map and reducing the pixel weight of background information in the feature map, the accuracy of state recognition of the target equipment can be relatively improved, and the effect of reducing the false alarm rate of inspection is achieved.
In order to solve the technical problems, the invention also provides an equipment state detection device based on image recognition, equipment and a computer readable storage medium, which have the beneficial effects.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of an apparatus state detection method based on image recognition according to an embodiment of the present invention;
fig. 2 is a structural diagram of an apparatus state detection device based on image recognition according to an embodiment of the present invention;
fig. 3 is a structural diagram of an apparatus state detection apparatus based on image recognition according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The core of the embodiment of the invention is to provide an equipment state detection method based on image recognition, which can relatively improve the accuracy of recognizing the state of target equipment and achieve the effect of reducing the false alarm rate of routing inspection; another core of the present invention is to provide an apparatus for detecting device status based on image recognition, a device and a computer-readable storage medium, all having the above-mentioned advantages.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Fig. 1 is a flowchart of an apparatus state detection method based on image recognition according to an embodiment of the present invention. As shown in fig. 1, an apparatus state detection method based on image recognition includes:
s10: and acquiring a feature map of the image to be detected by using a feature extraction network obtained by pre-training.
Specifically, the purpose of this step is to extract the image features of the image to be detected through a feature extraction network obtained by pre-training, and obtain a feature map (feature map) of the image to be detected. It should be noted that the feature extraction network may be a VGG16 network or a Dense Net network, and in specific implementation, a network of a corresponding type may be selected according to actual requirements, and a feature extraction network capable of extracting target devices and background information in an image to be detected is obtained by training a preset sample image including the target devices in the substation.
S20: according to a computer visual attention mechanism, the pixel weight of the target device in the feature map is increased, and the pixel weight of the background information in the feature map is decreased.
It should be noted that there are two implementation methods for the attention mechanism of computer vision, one is implemented based on Reinforcement Learning (Reinforcement Learning), and the other is implemented based on Gradient descent (Gradient deletion), and the specific type of the attention mechanism of computer vision is not limited in this embodiment. In the step, the pixel weight of the target equipment in the feature map is increased, and the pixel weight of the background information in the feature map is reduced, so that the attention degree of the target equipment can be relatively improved in detection, and the attention degree of the background information is relatively reduced, and the feature information of the target equipment can be more conveniently extracted.
S30: and inputting the feature map with the adjusted pixel weight into a multi-scale prediction network, and marking target equipment on the image to be detected by using prediction frames with different sizes.
Specifically, in the step, after the pixel weights of the target device and the background information in the feature map are adjusted according to a computer visual attention mechanism, the feature map with the adjusted pixel weights is input to the multi-scale prediction network; and detecting target equipment by using a multi-scale prediction network, and respectively marking the target equipment on the image to be detected through prediction frames with different sizes.
The prediction frame refers to a frame for marking a target device on an image to be detected, and the size of the prediction frame is different, and the number of corresponding feature points in the prediction frame is different. Each prediction frame is provided with corresponding state information representing target equipment, and the specific state information comprises on or off. In specific implementation, corresponding identification information such as confidence level and coordinate values representing position information may be further set on each prediction box. Specifically, by setting corresponding state information and identification information using a real frame previously set on a sample Image and using a labeling tool such as a Label Image, and then performing sample training, when a target Image is marked with a prediction frame in an Image to be detected, the corresponding state information and identification information can also be set on the prediction frame.
S40: and determining a target frame in the plurality of prediction frames by using the soft-interval non-maximum suppression processing network to obtain the state information of the target equipment.
In a preferred embodiment, the feature points of non-target devices, such as background information, in the prediction frame are reduced as much as possible on the basis that the prediction frame includes all the feature points of the target devices, so that it is necessary to determine the target frame that is most suitable for marking the target device, and obtain the state information of the target device from the state information corresponding to the target frame.
In order to determine a target frame corresponding to a target device, in this embodiment, confidence values of all prediction frames of the same target device are obtained first; then, sequencing the confidence values of all the prediction frames to determine the highest confidence value and the corresponding prediction frame; and traversing the rest of the prediction frames except the highest confidence value, and reducing the confidence values of the rest of the prediction frames if the overlapping areas of the rest of the prediction frames and the prediction frames corresponding to the highest confidence value are larger than a preset threshold value, thereby achieving the purpose of removing the redundant prediction frames of the same target equipment.
It should be noted that, when a plurality of target devices exist in the image to be detected, all the prediction frames corresponding to different target devices are respectively obtained, and the target frames corresponding to the target devices are respectively determined by using the soft-interval non-maximum suppression processing network.
In another embodiment, the maximum value suppression method may be used to identify the target frame from the plurality of prediction frames and obtain the status information of the target device. However, in the method for determining the target frame by using the soft-interval non-maximum suppression processing network in the embodiment, the problem that the prediction frames of adjacent similar objects are removed can be relatively avoided, so that the accuracy of determining the target device is improved.
Compared with the prior art, the method for detecting the equipment state based on the image recognition provided by the embodiment of the invention has the advantages that after the characteristic diagram of the image to be detected is obtained by utilizing the pre-trained characteristic extraction network, the characteristic diagram is further adjusted according to the computer vision attention mechanism, the accuracy of the characteristic extraction of the target equipment in the image to be detected can be relatively improved by increasing the pixel weight of the target equipment in the characteristic diagram and reducing the pixel weight of background information in the characteristic diagram, the accuracy of the state recognition of the target equipment can be relatively improved, and the effect of reducing the false alarm rate of routing inspection is achieved.
On the basis of the foregoing embodiment, this embodiment further describes and optimizes the technical solution, and specifically, inputting the feature map after adjusting the pixel weights to a multi-scale prediction network, and marking target devices on an image to be detected by using prediction frames with different sizes specifically includes:
inputting the feature map with the adjusted pixel weight to a multi-scale prediction network;
respectively fusing two adjacent layers of feature maps with different scales by utilizing a multi-scale prediction network;
and marking target equipment in the image to be detected by using prediction frames with different sizes according to the mapping relation between the fused feature map and the image to be detected.
As a preferred embodiment, the feature maps after the pixel weight adjustment are firstly input into a multi-scale prediction network, and then the feature maps of two adjacent layers after resolution scaling are respectively fused by using the multi-scale prediction network. Specifically, the multi-scale prediction network can be a pyramid feature extraction network, and feature fusion is performed from bottom to top according to feature maps with different resolutions; the multi-scale prediction network can also be a multi-scale feature extraction network, and feature fusion is carried out from top to bottom according to feature graphs with different resolutions. The feature graphs used by each layer of multi-scale prediction network are fused with features with different resolutions, so that the accuracy of feature extraction of the target equipment can be improved.
After feature fusion is carried out, feature points of the feature target equipment in the feature graph can be determined, so that pixel points of the feature target equipment in the image to be detected are determined according to the feature mapping relation between the feature graph after fusion and the image to be detected, the pixel points are marked by prediction frames with different sizes, and the target equipment in the image to be detected is marked by the prediction frames.
Therefore, the method for detecting the equipment state based on the image recognition can further improve the accuracy of extracting the target equipment in the image to be detected.
On the basis of the foregoing embodiment, this embodiment further describes and optimizes the technical solution, and specifically, according to the mapping relationship between the fused feature map and the image to be detected, the marking of the target device in the image to be detected by using the prediction frames with different sizes specifically includes:
acquiring a sample image of the target equipment and counting the length-width ratio of each target equipment in the sample image;
classifying the length-width ratios, and determining a plurality of target sizes for marking target equipment in the image to be detected according to a preset rule;
and marking target equipment in the image to be detected by using a prediction frame of each target size according to the mapping relation between the fused feature map and the image to be detected.
In order to mark a target device with prediction frames of different sizes, it is first necessary to determine the sizes of the prediction frames for marking the target device in an image to be detected.
In the embodiment, the length-width ratio of the target equipment in each sample image is obtained through obtaining the sample image of the target equipment and counting; then, classifying the counted length-width ratios by using a clustering algorithm such as k _ means and the like, and determining a plurality of length-width ratios with the top frequency in the length-width ratios as target sizes according to a preset rule, namely obtaining a plurality of target sizes; and marking the target equipment on the image to be detected by using a prediction frame corresponding to each target size according to the mapping relation between the fused feature map and the image to be detected. For example, the aspect ratios 1:2 and 2:3 with the highest occurrence frequency and the next highest occurrence frequency are used as target sizes, and the target equipment in the image to be detected is correspondingly marked by using a prediction frame of the target sizes.
Therefore, compared with the method of using the preset fixed target size as the size of the prediction frame in the prior art, the method provided by the embodiment reduces the number of the target sizes by improving the adaptation degree of the target sizes and the target device, and can reduce the time consumption for determining the target frame on the basis of not reducing the accuracy of feature extraction.
On the basis of the above embodiment, the embodiment further describes and optimizes the technical solution, and specifically, the obtaining of the feature map of the image to be detected by using the pre-trained feature extraction network specifically includes:
and acquiring a characteristic diagram of the image to be detected by using a Dense Net network obtained by pre-training.
As a preferred embodiment, the present embodiment adopts a density Net network to perform feature extraction on an image to be detected. The Dense Net network consists of 4 Dense Block modules, and each layer of each Dense Block module is connected with the previous layer, so that the feature extraction can be carried out repeatedly, and the accuracy of extracting the features can be further improved; compared with a VGG16 network, each layer of the Dense Net network has fewer parameters to learn, so that the training and using process is more convenient.
In the specific implementation, after the Dense Net network is trained, a test image is input into the Dense Net network for testing, so as to determine whether the Dense Net network is effective and the performance of the Dense Net network. After determining that the characteristic diagram of the image to be tested obtained by the Dense Net network can reach the preset standard, the characteristic diagram of the image to be tested is obtained by the Dense Net network obtained by pre-training, so that the accuracy of obtaining the characteristic diagram of the image to be tested by the Dense Net network can be further ensured.
On the basis of the above embodiment, the embodiment further describes and optimizes the technical solution, and specifically, the obtaining of the feature map of the image to be detected by using the previously trained Dense Net network specifically includes:
and acquiring a characteristic diagram of the image to be detected by using a Dense Net network obtained by pre-training according to a preset time period.
In this embodiment, a feature map of an image to be detected is obtained according to a preset time period, that is, an operation of extracting features of the image to be detected by using a previously trained Dense Net network is periodically performed to obtain the feature map. In the embodiment, the duration of the time period is not limited, and the method can relatively reduce the misjudgment rate by acquiring the feature map of the image to be detected for multiple times according to the preset time period compared with the method for detecting the equipment state by acquiring the feature map only once.
The above detailed description is made on the embodiment of the method for detecting the device state based on image recognition, and the present invention also provides a device, and a computer-readable storage medium for detecting the device state based on image recognition corresponding to the method.
Fig. 2 is a structural diagram of an apparatus state detection device based on image recognition according to an embodiment of the present invention, and as shown in fig. 2, an apparatus state detection device based on image recognition includes:
the feature extraction module 21 is configured to obtain a feature map of the image to be detected by using a feature extraction network obtained through pre-training;
the pixel weight adjusting module 22 is configured to increase the pixel weight of the target device in the feature map and decrease the pixel weight of the background information in the feature map according to the computer visual attention mechanism;
the setting module 23 is configured to input the feature map after the pixel weight is adjusted to the multi-scale prediction network, and mark target devices on the image to be detected by using prediction frames with different sizes;
and the determining module 24 is configured to determine the target frame in the plurality of prediction frames by using the soft-interval non-maximum suppression processing network, so as to obtain the state information of the target device.
The device state detection device based on image recognition provided by the embodiment of the invention has the beneficial effects of the device state detection method based on image recognition.
As a preferred embodiment, the setting module specifically includes:
the input unit is used for inputting the feature map after the pixel weight is adjusted to the multi-scale prediction network;
the fusion unit is used for fusing two adjacent layers of feature maps with different scales by utilizing a multi-scale prediction network;
and the marking unit is used for marking the target equipment in the image to be detected by using the prediction frames with different sizes according to the mapping relation between the fused feature map and the image to be detected.
Fig. 3 is a structural diagram of an apparatus state detection apparatus based on image recognition according to an embodiment of the present invention, and as shown in fig. 3, an apparatus state detection apparatus based on image recognition includes:
a memory 31 for storing a computer program;
a processor 32 for implementing the steps of the device status detection method based on image recognition as described above when executing the computer program.
The equipment state detection equipment based on the image recognition provided by the embodiment of the invention has the beneficial effects of the equipment state detection method based on the image recognition.
In order to solve the above technical problem, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the device status detection method based on image recognition as described above.
The computer-readable storage medium provided by the embodiment of the invention has the beneficial effects of the equipment state detection method based on the image recognition.
The method, the device, the equipment and the computer readable storage medium for detecting the equipment state based on the image recognition provided by the invention are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are set forth only to help understand the method and its core ideas of the present invention. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Claims (9)
1. An apparatus state detection method based on image recognition is characterized by comprising the following steps:
acquiring a feature map of an image to be detected by using a feature extraction network obtained by pre-training;
according to a computer vision attention mechanism, increasing the pixel weight of the target equipment in the feature map, and reducing the pixel weight of the background information in the feature map;
inputting the feature map with the adjusted pixel weight into a multi-scale prediction network, and marking the target equipment on the image to be detected by using prediction frames with different sizes;
and determining a target frame in the plurality of prediction frames by using a soft-interval non-maximum suppression processing network to obtain the state information of the target equipment.
2. The method according to claim 1, wherein the inputting the feature map with the adjusted pixel weights to a multi-scale prediction network, and the marking the target device on the image to be detected by using prediction frames with different sizes specifically comprises:
inputting the feature map with the adjusted pixel weight to the multi-scale prediction network;
respectively fusing two adjacent layers of feature maps with different scales by using the multi-scale prediction network;
and marking the target equipment in the image to be detected by using prediction frames with different sizes according to the mapping relation between the fused feature map and the image to be detected.
3. The method according to claim 2, wherein the marking the target device in the image to be detected by using the prediction frames with different sizes according to the mapping relationship between the fused feature map and the image to be detected specifically comprises:
acquiring a sample image of the target equipment and counting the length-width ratio of each target equipment in the sample image;
classifying the length-width ratios, and determining a plurality of target sizes for marking the target equipment in the image to be detected according to a preset rule;
and marking the target equipment in the image to be detected by using a prediction frame of each target size according to the mapping relation between the fused feature map and the image to be detected.
4. The method according to claim 2, wherein the obtaining of the feature map of the image to be detected by using the pre-trained feature extraction network specifically comprises:
and acquiring the characteristic diagram of the image to be detected by using a Dense Net network obtained by pre-training.
5. The method according to any one of claims 1 to 4, wherein the obtaining of the feature map of the image to be detected by using the DenseNet network obtained by pre-training is specifically as follows:
and acquiring a characteristic diagram of the image to be detected by using a Dense Net network obtained by pre-training according to a preset time period.
6. An apparatus state detection device based on image recognition, comprising:
the characteristic extraction module is used for acquiring a characteristic diagram of the image to be detected by utilizing a characteristic extraction network obtained by pre-training;
the pixel weight adjusting module is used for increasing the pixel weight of the target equipment in the feature map and reducing the pixel weight of the background information in the feature map according to a computer visual attention mechanism;
the setting module is used for inputting the feature map after the pixel weight is adjusted into a multi-scale prediction network and marking the target equipment on the image to be detected by using prediction frames with different sizes;
and the determining module is used for determining a target frame in the plurality of prediction frames by using the soft interval non-maximum suppression processing network to obtain the state information of the target equipment.
7. The apparatus according to claim 6, wherein the setting module specifically includes:
the input unit is used for inputting the feature map after the pixel weight is adjusted to the multi-scale prediction network;
the fusion unit is used for fusing the feature maps of two adjacent layers with different scales by utilizing the multi-scale prediction network;
and the marking unit is used for marking the target equipment in the image to be detected by using the prediction frames with different sizes according to the mapping relation between the fused feature map and the image to be detected.
8. An apparatus state detection apparatus based on image recognition, characterized by comprising:
a memory for storing a computer program;
a processor for implementing the steps of a method for device status detection based on image recognition according to any one of claims 1 to 5 when executing said computer program.
9. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of a method for device state detection based on image recognition according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910164194.1A CN111666958A (en) | 2019-03-05 | 2019-03-05 | Method, device, equipment and medium for detecting equipment state based on image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910164194.1A CN111666958A (en) | 2019-03-05 | 2019-03-05 | Method, device, equipment and medium for detecting equipment state based on image recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111666958A true CN111666958A (en) | 2020-09-15 |
Family
ID=72381640
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910164194.1A Pending CN111666958A (en) | 2019-03-05 | 2019-03-05 | Method, device, equipment and medium for detecting equipment state based on image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111666958A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112308045A (en) * | 2020-11-30 | 2021-02-02 | 深圳集智数字科技有限公司 | Detection method and device for dense crowd and electronic equipment |
| CN116342571A (en) * | 2023-03-27 | 2023-06-27 | 中吉创新技术(深圳)有限公司 | State detection method and device for ventilation system control box and storage medium |
| CN116434478A (en) * | 2022-12-30 | 2023-07-14 | 深圳市地质局 | Intelligent early warning response method, device and system for geological disasters |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107742093A (en) * | 2017-09-01 | 2018-02-27 | 国网山东省电力公司电力科学研究院 | A kind of infrared image power equipment component real-time detection method, server and system |
| CN109255352A (en) * | 2018-09-07 | 2019-01-22 | 北京旷视科技有限公司 | Object detection method, apparatus and system |
| CN109272016A (en) * | 2018-08-08 | 2019-01-25 | 广州视源电子科技股份有限公司 | Target detection method, device, terminal equipment and computer readable storage medium |
-
2019
- 2019-03-05 CN CN201910164194.1A patent/CN111666958A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107742093A (en) * | 2017-09-01 | 2018-02-27 | 国网山东省电力公司电力科学研究院 | A kind of infrared image power equipment component real-time detection method, server and system |
| CN109272016A (en) * | 2018-08-08 | 2019-01-25 | 广州视源电子科技股份有限公司 | Target detection method, device, terminal equipment and computer readable storage medium |
| CN109255352A (en) * | 2018-09-07 | 2019-01-22 | 北京旷视科技有限公司 | Object detection method, apparatus and system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112308045A (en) * | 2020-11-30 | 2021-02-02 | 深圳集智数字科技有限公司 | Detection method and device for dense crowd and electronic equipment |
| CN112308045B (en) * | 2020-11-30 | 2023-11-24 | 深圳集智数字科技有限公司 | Method and device for detecting dense crowd and electronic equipment |
| CN116434478A (en) * | 2022-12-30 | 2023-07-14 | 深圳市地质局 | Intelligent early warning response method, device and system for geological disasters |
| CN116434478B (en) * | 2022-12-30 | 2023-11-21 | 深圳市地质局 | Intelligent early warning response method, device and system for geological disasters |
| CN116342571A (en) * | 2023-03-27 | 2023-06-27 | 中吉创新技术(深圳)有限公司 | State detection method and device for ventilation system control box and storage medium |
| CN116342571B (en) * | 2023-03-27 | 2023-12-22 | 中吉创新技术(深圳)有限公司 | State detection method and device for ventilation system control box and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107742093B (en) | Real-time detection method, server and system for infrared image power equipment components | |
| CN109118479B (en) | Capsule network-based insulator defect identification and positioning device and method | |
| CN109376605B (en) | Electric power inspection image bird-stab-prevention fault detection method | |
| CN108009515B (en) | Power transmission line positioning and identifying method of unmanned aerial vehicle aerial image based on FCN | |
| CN111797890A (en) | Method and system for detecting defects of power transmission line equipment | |
| CN109446925A (en) | A kind of electric device maintenance algorithm based on convolutional neural networks | |
| CN111666958A (en) | Method, device, equipment and medium for detecting equipment state based on image recognition | |
| CN108010025B (en) | Switch and indicator lamp positioning and state identification method of screen cabinet based on RCNN | |
| CN107563412A (en) | A kind of infrared image power equipment real-time detection method based on deep learning | |
| CN106022345B (en) | A kind of high voltage isolator state identification method based on Hough forest | |
| CN110070530A (en) | A kind of powerline ice-covering detection method based on deep neural network | |
| CN110929646A (en) | Power distribution tower reverse-off information rapid identification method based on unmanned aerial vehicle aerial image | |
| CN110570392A (en) | method, device, system, equipment and medium for detecting on-off state of substation equipment | |
| CN114445746A (en) | Model training method, railway contact net abnormity detection method and related device | |
| CN111815576B (en) | Method, device, equipment and storage medium for detecting corrosion condition of metal part | |
| CN112634254A (en) | Insulator defect detection method and related device | |
| CN109740654A (en) | A kind of tongue body automatic testing method based on deep learning | |
| CN115619778A (en) | Power equipment defect identification method and system, readable storage medium and equipment | |
| CN115471487A (en) | Insulator defect detection model construction and insulator defect detection method and device | |
| CN114581419A (en) | Transformer insulating sleeve defect detection method, related equipment and readable storage medium | |
| CN112270671B (en) | Image detection method, device, electronic equipment and storage medium | |
| CN114119528A (en) | Defect detection method and device for distribution line insulator | |
| CN113506290A (en) | Method and device for detecting defects of line insulator | |
| CN116229278B (en) | Method and system for detecting rust defect of vibration damper of power transmission line | |
| CN115937492B (en) | Feature recognition-based infrared image recognition method for power transformation equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 214105 No. 299 Dacheng Road, Xishan District, Jiangsu, Wuxi Applicant after: Zhongke Weizhi intelligent manufacturing technology Jiangsu Co.,Ltd. Address before: 214105 No. 299 Dacheng Road, Xishan District, Jiangsu, Wuxi Applicant before: ZHONGKE WEIZHI INTELLIGENT MANUFACTURING TECHNOLOGY JIANGSU Co.,Ltd. |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200921 Address after: 214105 No. 299 Dacheng Road, Xishan District, Jiangsu, Wuxi Applicant after: ZHONGKE WEIZHI INTELLIGENT MANUFACTURING TECHNOLOGY JIANGSU Co.,Ltd. Address before: Zuchongzhi road Kunshan city 215347 Suzhou City, Jiangsu province No. 1699 building 7 floor Applicant before: KUNSHAN BRANCH, INSTITUTE OF MICROELECTRONICS OF CHINESE ACADEMY OF SCIENCES |
|
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200915 |