CN114463395A - Monitoring equipment offset detection method, equipment and medium - Google Patents
Monitoring equipment offset detection method, equipment and medium Download PDFInfo
- Publication number
- CN114463395A CN114463395A CN202111676639.8A CN202111676639A CN114463395A CN 114463395 A CN114463395 A CN 114463395A CN 202111676639 A CN202111676639 A CN 202111676639A CN 114463395 A CN114463395 A CN 114463395A
- Authority
- CN
- China
- Prior art keywords
- image
- template
- feature points
- monitoring
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 122
- 238000001514 detection method Methods 0.000 title claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 47
- 238000005286 illumination Methods 0.000 claims abstract description 21
- 238000012806 monitoring device Methods 0.000 claims description 23
- 238000003860 storage Methods 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 11
- 230000009466 transformation Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 7
- 238000010219 correlation analysis Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000006073 displacement reaction Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004451 qualitative analysis Methods 0.000 description 2
- 238000004445 quantitative analysis Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000009413 insulation Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/532—Query formulation, e.g. graphical querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Library & Information Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a method, a device and a medium for detecting offset of monitoring equipment, wherein the method comprises the following steps: acquiring a monitoring image of equipment to be inspected, which is shot by monitoring equipment, within a preset time interval; searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images corresponding to the equipment to be inspected under different illumination conditions; respectively extracting the feature points of the monitoring image and the template image; performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image; and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment. The deviation of the monitoring equipment is qualitatively and quantitatively analyzed, so that the deviation detection of the monitoring equipment can be more efficiently carried out.
Description
Technical Field
The present application relates to the field of power technologies, and in particular, to a method, an apparatus, and a medium for detecting offset of a monitoring apparatus.
Background
With the gradual expansion of the scale of the power grid, deployed monitoring equipment is increased. Under the condition that the installation position of the monitoring equipment is loose, and the engineering personnel overhauls, the position of the monitoring equipment can be deviated. Based on this, the change of the angle of shooing of prison shooting equipment can make the installation of prison shooting equipment become nonconforming with the standard on the one hand, influences the user experience that intelligence was patrolled and examined, and on the other hand, the change of the angle of shooing can lead to the formation of image angle skew, can't shoot the target area even, influences the recognition accuracy of algorithm, also can make the incident that relies on artificial mark characteristic produce the error, leads to intelligence to patrol and examine and can't go on.
At present, a video stream-based scheme is generally adopted, and a feature matching or optical flow manner is adopted between frames to judge the deviation of the monitoring device. However, on the premise of not affecting insulation and safety of power equipment at all, the power industry adopts an active wireless or passive wireless mode, so that the power consumption is extremely high, the power consumption requirement is difficult to meet based on a video streaming mode, the monitoring equipment is in a dormant state for a long time, and whether the monitoring equipment is deviated or not is difficult to be monitored in real time by adopting a long-term video streaming mode, so that the deviation detection efficiency of the monitoring equipment is low.
Disclosure of Invention
The embodiment of the application provides a method, equipment and medium for detecting offset of monitoring equipment, and is used for solving the problem of low offset detection efficiency of the monitoring equipment.
The embodiment of the application adopts the following technical scheme:
in one aspect, an embodiment of the present application provides a method for detecting offset of a monitoring device, where the method includes: acquiring a monitoring image of equipment to be inspected, which is shot by monitoring equipment, within a preset time interval; searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images corresponding to the equipment to be inspected under different illumination conditions; respectively extracting the feature points of the monitoring image and the template image; performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image; and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment.
In one example, the retrieving, in a pre-constructed template set, a template image with the highest similarity to the monitored image specifically includes: respectively converting the monitoring image and the template images from RGB space to HSV space; extracting a first V component of the monitoring image and extracting second V components corresponding to the plurality of template images respectively according to the HSV space; extracting a first histogram feature of the first V component and a second histogram feature of the second V component; performing correlation analysis on the first histogram feature and the second histogram feature, and determining respective corresponding similarities between the monitoring image and the plurality of template images; and determining the template image with the highest similarity to the monitored image according to the similarity.
In one example, the respectively extracting feature points of the monitoring image and the template image specifically includes: acquiring an image watermark of the monitored image; wherein the template image does not include the image watermark; the image watermark is identification information of a manufacturer to which the monitoring equipment belongs; determining a pixel area occupied by the image watermark in the monitored image according to the resolution of the monitored image; removing the pixel area to extract a designated area which does not contain an image watermark from the monitored image; and respectively extracting the specified area of the monitoring image and the characteristic points of the template image through a rapid characteristic point extraction model ORB.
In one example, the extracting, by using the fast feature point extraction model ORB, the feature points of the specified region of the monitored image and the template image respectively includes: respectively constructing a scale image pyramid for the designated area and the template image; and carrying out grid processing on the pyramid of the scale image, carrying out feature point detection on each small grid in each layer of pyramid image through a FAST feature point detection algorithm, and respectively extracting feature points of the designated area and the template image.
In one example, the performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image specifically includes: carrying out feature matching on the feature points of the monitored image and the feature points of the template image through a fast nearest search matching algorithm to obtain a plurality of groups of feature points with matching relation; filtering the multiple groups of feature points through a grid motion statistics GMS filtering algorithm to determine multiple groups of matching feature points with correct matching relation; and calculating the offset distance of each group of matching feature points between the monitored image and the template image according to the coordinate parameters of each matching feature point in the plurality of groups of matching feature points with the correct matching relationship.
In one example, the analyzing the offset distance according to a preset rule to perform offset detection on the monitoring device specifically includes: calculating an average distance corresponding to a plurality of offset distances; if the average distance is smaller than a preset offset distance threshold, judging whether the similarity between the monitoring image and the template image is smaller than a preset similarity threshold; if yes, sending a detection notice to the user; and acquiring feedback information of the user, and determining whether the monitoring equipment deviates or not through the feedback information.
In one example, the method further comprises: if the average distance is larger than a preset offset distance threshold value, determining that the monitoring equipment is offset; generating a coordinate transformation matrix in the multiple groups of matching characteristic points with correct matching relation according to the coordinate parameters of the matching characteristic points; correcting the monitored image through the coordinate transformation matrix; and acquiring the horizontal direction offset, the vertical direction offset component and the offset rotation angle of the monitoring equipment through the coordinate transformation matrix so as to enable the user to correct the monitoring equipment.
In one example, the monitoring image includes a plurality of devices to be inspected, and the method further includes: determining areas of the plurality of devices to be inspected in the monitored image respectively according to the coordinate information of the plurality of devices to be inspected in the monitored image; and shifting the equipment to be patrolled and examined with the minimum area for multiple times so that the user can judge the distance of the equipment to be patrolled and examined with the minimum area, and taking the distance of the tolerance shift as the preset shift distance.
On the other hand, an embodiment of the present application provides a monitoring apparatus offset detection apparatus, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to: acquiring a monitoring image of equipment to be inspected within a preset time interval; searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images of the equipment to be inspected under different illumination conditions; respectively extracting the feature points of the monitoring image and the template image; performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image; and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment.
In another aspect, an embodiment of the present application provides a non-volatile computer storage medium for monitoring offset detection of a device, where the non-volatile computer storage medium stores computer-executable instructions configured to: acquiring a monitoring image of equipment to be inspected within a preset time interval; searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images of the equipment to be inspected under different illumination conditions; respectively extracting the feature points of the monitoring image and the template image; performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image; and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment.
The embodiment of the application adopts at least one technical scheme which can achieve the following beneficial effects:
according to the embodiment of the application, the monitoring image of the equipment to be patrolled and examined is acquired at preset time intervals, a video stream does not need to be acquired in real time, the power consumption of the monitoring equipment is reduced, the template sets corresponding to the template images under different illumination conditions are constructed in advance by comprehensively considering the change of the illumination conditions, the template image with the highest similarity with the monitoring image is retrieved, the detection accuracy can be improved, finally, the deviation distance of the monitoring image relative to the template image is analyzed, the deviation detection is carried out on the monitoring equipment, the qualitative and quantitative analysis on the deviation of the monitoring equipment is completed, and the deviation detection of the monitoring equipment can be carried out more efficiently.
Drawings
In order to more clearly explain the technical solutions of the present application, some embodiments of the present application will be described in detail below with reference to the accompanying drawings, in which:
fig. 1 is a schematic flowchart of a method for detecting offset of a monitoring device according to an embodiment of the present disclosure;
fig. 2 is a schematic structural diagram of a monitoring device offset detection device provided in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following embodiments and accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Some embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of a method for detecting offset of a monitoring device according to an embodiment of the present disclosure. Certain input parameters or intermediate results in the procedure allow for manual intervention adjustments to help improve accuracy.
The analysis method according to the embodiment of the present application may be implemented by a terminal device or a server, and the present application is not limited to this. For convenience of understanding and description, the following embodiments are described in detail by taking a server as an example.
The process in fig. 1 may include the following steps:
s101: and acquiring a monitoring image which is shot by the monitoring equipment and is related to the equipment to be inspected within a preset time interval.
Wherein, wait to patrol and examine equipment can be power equipment, for example, every certain time interval, for example, an hour, a day, three days, gather electronic equipment's supervision image.
S102: searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images corresponding to the equipment to be inspected under different illumination conditions.
It should be noted that the template image is used as a reference image of the monitoring image, that is, when the monitoring device does not shift, the device to be inspected is photographed, so as to obtain the template image.
In addition, the monitoring equipment shoots the equipment to be patrolled and examined under different illumination conditions, and the obtained monitoring image is different, however, the illumination condition of the monitoring image can influence the accuracy of characteristic point extraction. Therefore, when the monitoring apparatus captures two monitoring images having different illumination conditions without a shift, if feature point extraction is performed on each of the two monitoring images, some feature points may be substantially not matched.
Based on this, when offset detection is performed on the monitoring equipment, in order to enable the monitoring image to be matched with the template image with the highest similarity as much as possible, when the template image is collected, a plurality of template images corresponding to the equipment to be inspected under different illumination conditions are collected, the influence of the illumination conditions on feature point extraction is reduced, and therefore the accuracy of the offset detection of the monitoring equipment is improved.
Further, the template set may include template images captured by a plurality of monitoring devices, that is, each monitoring device captures a corresponding device to be inspected under different illumination conditions, so as to obtain a plurality of template images corresponding to the corresponding devices to be inspected.
For example, a plurality of monitoring devices in different power scenes respectively capture monitoring images of respective corresponding power devices under different lighting conditions, so as to obtain a plurality of template images respectively corresponding to the respective corresponding power devices.
S103: and respectively extracting the feature points of the monitoring image and the template image.
Based on the method, the feature points of the monitored image are automatically extracted, and the configuration of the artificial feature points is reduced.
S104: and performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image.
The matching relationship of the feature points in the two images can be determined through feature matching, and the offset distance of the feature points relative to the template image is determined through the matching relationship.
Note that, since the number of feature points is not only one, the number of offset distances may be plural.
S105: and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment.
If the number of the offset distances is multiple, whether the monitoring equipment is offset or not can be judged by analyzing the offset distances corresponding to the multiple feature points respectively.
According to the method of the figure 1, the monitoring images of the equipment to be inspected are acquired at preset time intervals, a video stream does not need to be acquired in real time, the power consumption of the monitoring equipment is reduced, the template sets corresponding to the template images under different illumination conditions are pre-constructed by comprehensively considering the change of the illumination conditions, the template image with the highest similarity to the monitoring image is retrieved, the detection accuracy can be improved, and finally the deviation distance of the monitoring image relative to the template image is analyzed to perform deviation detection on the monitoring equipment, so that the qualitative and quantitative analysis on the deviation of the monitoring equipment is completed, the deviation detection method of the monitoring equipment can be performed more efficiently, and the method is particularly suitable for the deviation detection method of the monitoring equipment of the non-video stream in the power scene.
It should be noted that, although the embodiment of the present application describes steps S101 to S105 in sequence with reference to fig. 1, this does not mean that steps S101 to S105 must be executed in strict sequence. The embodiment of the present application is described by sequentially describing step S101 to step S105 according to the sequence shown in fig. 1, so as to facilitate those skilled in the art to understand the technical solutions of the embodiment of the present application. In other words, in the embodiment of the present application, the sequence between step S101 and step S105 may be appropriately adjusted according to actual needs.
Based on the method of fig. 1, the examples of the present application also provide some specific embodiments and extensions of the method, and the following description is continued.
In some embodiments of the present application, in order to reduce misjudgment under illumination change and noise conditions and improve adaptability and robustness of an algorithm, a template image with the highest similarity to a current monitored image is selected in a template set, and then, a server needs to confirm the similarity between the monitored image and each template image in the template set, and then selects the template image with the highest similarity to the current monitored image.
Specifically, the server converts the monitoring image and the plurality of template images from the RGB space to the HSV space. Then, according to the HSV space, a first V component of the monitoring image is extracted, second V components corresponding to the plurality of template images are extracted, and a first histogram feature of the first V component and a second histogram feature of the second V component are extracted. Then, carrying out correlation analysis on the first histogram feature and the second histogram feature, and determining the corresponding similarity between the monitoring image and the plurality of template images. And finally, determining the template image with the highest similarity to the monitored image according to the similarity.
It should be noted that, since the similarity between the monitoring image and each of the plurality of template images may be negative as a result of the correlation analysis, the absolute value of the correlation analysis result is used as the similarity between the two images.
In some implementations of the present application, since the monitored image usually has identification information of a manufacturer to which the monitoring device belongs, that is, an image watermark, in order to filter interference of the image watermark, the server removes an image watermark region, and extracts feature points only in the image watermark region.
It should be noted that the image watermark region in the template image has been removed, and of course, the template image may also include the image watermark region, and when the image watermark region is removed from the monitoring image, the image watermark region of the template image is removed at the same time.
Specifically, the server acquires an image watermark of the monitored image, determines a pixel area occupied by the image watermark in the monitored image according to the resolution of the monitored image, and eliminates the pixel area so as to extract a specified area which does not contain the image watermark in the monitored image.
After the designated region is extracted, the designated region of the monitoring image and the feature points of the template image are respectively extracted by using a fast feature point extraction model ORB.
Among them, ORB feature extraction is an algorithm for fast feature point extraction and description. The method is divided into two parts, namely feature point extraction and feature point description. The feature extraction is developed by a FAST algorithm, and the feature point description is improved according to a BRIEF feature description algorithm. The ORB feature is to combine the detection method of FAST feature points with BRIEF feature descriptors and make improvements and optimization on the original basis. Has faster operation speed compared with SIFT and SURF.
Further, when the designated area of the monitoring image and the feature points of the template image are respectively extracted, the server respectively constructs a scale image pyramid for the designated area and the template image, performs grid processing on the scale image pyramid, performs feature point detection on each small grid in each layer of pyramid image through a FAST feature point detection algorithm, and respectively extracts the feature points of the designated area and the template image.
In some implementations of the present application, when feature matching is performed on feature points of a monitoring image and feature points of a template image, currently, a brute force search matching method BF is generally used for matching the feature points, but the method has low efficiency, and when the data dimension is high, the matching efficiency is drastically reduced. Since the ORB features are binary features, the fast nearest neighbor search matching algorithm FLANN can be used for feature matching.
Based on the method, the server performs feature matching on the feature points of the monitored image and the feature points of the template image through a fast nearest search matching algorithm to obtain a plurality of groups of feature points with matching relations.
Due to the influence of various factors, in the matching result, an incorrect matching relationship may be included, and therefore, in order to effectively ensure the accuracy of feature point matching, the incorrect matching relationship needs to be filtered, the corresponding relationship separation between correct matching and incorrect matching is realized, the judgment of the offset of the monitoring equipment is performed only by using the relevant information of the correct matching relationship, and the correction of the monitoring image is performed.
Specifically, a plurality of groups of feature points are filtered through a grid motion statistics GMS filtering algorithm, and a plurality of groups of matching feature points with correct matching relations are determined.
And calculating the offset distance of each group of matching characteristic points between the monitored image and the template image according to the coordinate parameters of each matching characteristic point in the plurality of groups of matching characteristic points with the correct matching relation.
Further, after obtaining the offset distance of the monitoring image relative to the template image, when analyzing the offset distance, the server may preset an offset distance threshold and compare the offset distance with the offset distance threshold, but since the number of the offset distances is plural, the server may calculate an average distance corresponding to the plural offset distances, compare the average distance with the preset offset distance threshold, and if the average distance is greater than the preset offset distance threshold, it indicates that the monitoring device is offset.
If the average distance is smaller than the preset offset distance threshold, it may indicate that the monitoring device is not offset, but in order to verify the confidence of the current detection result, it is considered that the template images in the template set are limited, and even if the template images in the template set have the highest similarity to the monitoring image, in fact, the similarity between the template images and the monitoring image may be low, and therefore, the template images having the highest similarity to the monitoring image are monitored.
And when the highest similarity is smaller than a preset similarity threshold, the confidence of the result of the offset detection of the current template image participating in the monitoring equipment is not high.
Based on this, if the average distance is smaller than a preset offset distance threshold, whether the similarity between the monitoring image and the template image is smaller than a preset similarity threshold is judged, if yes, a detection notice is sent to the user, feedback information of the user is obtained, and whether the monitoring device is offset is determined through the feedback information.
And if the similarity between the monitoring image and the template image is not smaller than a preset similarity threshold, determining that the monitoring equipment does not deviate.
It should be noted that the server outputs the template image with the similarity smaller than the preset similarity threshold to the template set update queue, so that the user can monitor the template set update queue and continuously update the template set.
Meanwhile, the monitoring images without deviation are stored in the template set so as to update the template set.
In some embodiments of the present application, when determining the preset offset distance, it is difficult to satisfy the tolerance of different sizes of devices to the offset of the monitoring device by using a fixed value manner at present.
Therefore, considering that the monitored image comprises a plurality of devices to be inspected, namely, a plurality of devices to be inspected may be included, the accuracy of the deviation detection result is improved by taking the inspection device with the minimum area in the current monitored scene as a reference, the tolerance of the devices to be inspected with different sizes to the deviation of the monitored device is met by adopting a self-adaptive threshold value mode, and the sensitivity of the deviation detection of the monitored device is improved.
Specifically, the server determines areas of the plurality of devices to be inspected in the monitored image according to coordinate information of the plurality of devices to be inspected in the monitored image, and then shifts the devices to be inspected with the minimum area for multiple times, so that a user can judge the shift tolerance distance of the devices to be inspected with the minimum area, and the shift tolerance distance is used as the preset shift distance. That is to say, based on the operation of the user, the device to be inspected with the minimum area is subjected to multiple times of deviation, and the deviation tolerance distance of the device to be inspected with the minimum area is judged.
In some embodiments of the present application, after determining that the monitoring device is offset, the monitoring device and the monitoring image need to be corrected.
Specifically, the server generates a coordinate transformation matrix according to the coordinate parameters of each matching feature point in a plurality of groups of matching feature points having a correct matching relationship.
The expression of the coordinate transformation matrix is as follows:
wherein d isxA horizontal displacement component, dyAnd (3) a vertical displacement component, theta is an offset rotation angle, and m is a coordinate transformation matrix.
Correcting the monitored image through a coordinate transformation matrix; and acquiring the horizontal direction offset, the vertical direction offset component and the offset rotation angle of the monitoring equipment through the coordinate transformation matrix so as to enable a user to correct the monitoring equipment.
Based on the same idea, some embodiments of the present application further provide a device and a non-volatile computer storage medium corresponding to the above method.
Fig. 2 is a schematic structural diagram of a monitoring device offset detection device provided in an embodiment of the present application, where the monitoring device includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
acquiring a monitoring image of equipment to be inspected within a preset time interval;
searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images of the equipment to be inspected under different illumination conditions;
respectively extracting the feature points of the monitoring image and the template image;
performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image;
and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment.
Some embodiments of the present application provide a non-volatile computer storage medium for monitoring device offset detection, storing computer-executable instructions configured to:
acquiring a monitoring image of equipment to be inspected within a preset time interval;
searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images of the equipment to be inspected under different illumination conditions;
respectively extracting the feature points of the monitoring image and the template image;
performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image;
and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment.
The embodiments in the present application are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the device and media embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference may be made to some descriptions of the method embodiments for relevant points.
The device and the medium provided by the embodiment of the application correspond to the method one to one, so the device and the medium also have the similar beneficial technical effects as the corresponding method, and the beneficial technical effects of the method are explained in detail above, so the beneficial technical effects of the device and the medium are not repeated herein.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the technical principle of the present application shall fall within the protection scope of the present application.
Claims (10)
1. A method for detecting offset of a monitoring device, the method comprising:
acquiring a monitoring image of equipment to be inspected, which is shot by monitoring equipment, within a preset time interval;
searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images corresponding to the equipment to be inspected under different illumination conditions;
respectively extracting the feature points of the monitoring image and the template image;
performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image;
and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment.
2. The method according to claim 1, wherein the retrieving, in the pre-constructed template set, the template image with the highest similarity to the monitored image specifically comprises:
respectively converting the monitoring image and the template images from RGB space to HSV space;
extracting a first V component of the monitoring image and extracting second V components corresponding to the plurality of template images respectively according to the HSV space;
extracting a first histogram feature of the first V component and a second histogram feature of the second V component;
performing correlation analysis on the first histogram feature and the second histogram feature, and determining respective corresponding similarities between the monitoring image and the plurality of template images;
and determining the template image with the highest similarity to the monitored image according to the similarity.
3. The method according to claim 1, wherein the extracting the feature points of the monitoring image and the template image respectively comprises:
acquiring an image watermark of the monitored image; wherein the template image does not include the image watermark; the image watermark is identification information of a manufacturer to which the monitoring equipment belongs;
determining a pixel area occupied by the image watermark in the monitored image according to the resolution of the monitored image;
removing the pixel area to extract a designated area which does not contain an image watermark from the monitored image;
and respectively extracting the specified area of the monitoring image and the characteristic points of the template image through a rapid characteristic point extraction model ORB.
4. The method according to claim 3, wherein the extracting the feature points of the specific region of the monitored image and the template image respectively through a fast feature point extraction model ORB specifically comprises:
respectively constructing a scale image pyramid for the designated area and the template image;
and carrying out grid processing on the pyramid of the scale image, carrying out feature point detection on each small grid in each layer of pyramid image through a FAST feature point detection algorithm, and respectively extracting feature points of the designated area and the template image.
5. The method according to claim 1, wherein the performing feature matching on the feature points of the surveillance image and the feature points of the template image and determining the offset distance of the feature points of the surveillance image with respect to the feature points of the template image specifically comprises:
carrying out feature matching on the feature points of the monitored image and the feature points of the template image through a fast nearest search matching algorithm to obtain a plurality of groups of feature points with matching relation;
filtering the multiple groups of feature points through a grid motion statistics GMS filtering algorithm to determine multiple groups of matching feature points with correct matching relation;
and calculating the offset distance of each group of matching feature points between the monitored image and the template image according to the coordinate parameters of each matching feature point in the plurality of groups of matching feature points with the correct matching relationship.
6. The method according to claim 5, wherein the analyzing the offset distance according to a preset rule to perform offset detection on the monitoring device specifically includes:
calculating an average distance corresponding to a plurality of offset distances;
if the average distance is smaller than a preset offset distance threshold, judging whether the similarity between the monitoring image and the template image is smaller than a preset similarity threshold;
if yes, sending a detection notice to the user;
and acquiring feedback information of the user, and determining whether the monitoring equipment deviates or not through the feedback information.
7. The method of claim 6, further comprising:
if the average distance is larger than a preset offset distance threshold value, determining that the monitoring equipment is offset;
generating a coordinate transformation matrix in the multiple groups of matching characteristic points with correct matching relation according to the coordinate parameters of the matching characteristic points;
correcting the monitored image through the coordinate transformation matrix; and acquiring the horizontal direction offset, the vertical direction offset component and the offset rotation angle of the monitoring equipment through the coordinate transformation matrix so as to enable the user to correct the monitoring equipment.
8. The method of claim 6, wherein the surveillance image includes a number of devices to be inspected, the method further comprising:
determining areas of the plurality of devices to be inspected in the monitored image respectively according to the coordinate information of the plurality of devices to be inspected in the monitored image;
and shifting the equipment to be patrolled and examined with the minimum area for multiple times so that the user can judge the distance of the equipment to be patrolled and examined with the minimum area, and taking the distance of the tolerance shift as the preset shift distance.
9. A monitoring device shift detection device, characterized in that the device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
acquiring a monitoring image of equipment to be inspected within a preset time interval;
searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images of the equipment to be inspected under different illumination conditions;
respectively extracting feature points of the monitoring image and the template image;
performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image;
and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment.
10. A surveillance device offset detection non-volatile computer storage medium storing computer-executable instructions configured to:
acquiring a monitoring image of equipment to be inspected within a preset time interval;
searching a template image with the highest similarity to the monitored image in a pre-constructed template set; the template set comprises a plurality of template images of the equipment to be inspected under different illumination conditions;
respectively extracting the feature points of the monitoring image and the template image;
performing feature matching on the feature points of the monitored image and the feature points of the template image, and determining the offset distance of the feature points of the monitored image relative to the feature points of the template image;
and analyzing the offset distance according to a preset rule so as to perform offset detection on the monitoring equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111676639.8A CN114463395A (en) | 2021-12-31 | 2021-12-31 | Monitoring equipment offset detection method, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111676639.8A CN114463395A (en) | 2021-12-31 | 2021-12-31 | Monitoring equipment offset detection method, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114463395A true CN114463395A (en) | 2022-05-10 |
Family
ID=81407694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111676639.8A Pending CN114463395A (en) | 2021-12-31 | 2021-12-31 | Monitoring equipment offset detection method, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114463395A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115393363A (en) * | 2022-10-31 | 2022-11-25 | 山东金帝精密机械科技股份有限公司 | Production early warning method, equipment and medium for bearing retainer |
| CN115760856A (en) * | 2023-01-10 | 2023-03-07 | 惟众信(湖北)科技有限公司 | Part spacing measuring method and system based on image recognition and storage medium |
-
2021
- 2021-12-31 CN CN202111676639.8A patent/CN114463395A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115393363A (en) * | 2022-10-31 | 2022-11-25 | 山东金帝精密机械科技股份有限公司 | Production early warning method, equipment and medium for bearing retainer |
| CN115760856A (en) * | 2023-01-10 | 2023-03-07 | 惟众信(湖北)科技有限公司 | Part spacing measuring method and system based on image recognition and storage medium |
| CN115760856B (en) * | 2023-01-10 | 2023-04-28 | 惟众信(湖北)科技有限公司 | Image recognition-based part spacing measurement method, system and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110853033B (en) | Video detection method and device based on inter-frame similarity | |
| CN114463395A (en) | Monitoring equipment offset detection method, equipment and medium | |
| WO2020233397A1 (en) | Method and apparatus for detecting target in video, and computing device and storage medium | |
| CN110674886B (en) | Video target detection method fusing multi-level features | |
| CN113344002B (en) | Target coordinate duplication eliminating method and system, electronic equipment and readable storage medium | |
| CN116797977A (en) | Method and device for identifying dynamic target of inspection robot and measuring temperature and storage medium | |
| Feng et al. | A novel saliency detection method for wild animal monitoring images with WMSN | |
| CN112116068A (en) | Annular image splicing method, equipment and medium | |
| WO2018121414A1 (en) | Electronic device, and target image recognition method and apparatus | |
| CN117576091B (en) | Cooling tower fan vibration detection method and system based on video detection | |
| CN114235815A (en) | Method for detecting surface defects of outdoor electrical equipment of converter station based on scene filtering | |
| CN112330618B (en) | Image offset detection method, device and storage medium | |
| CN117978966A (en) | Method for monitoring autonomous running state of kitchen video through multi-point access | |
| CN117456204A (en) | Target tracking method, device, video processing system, storage medium and terminal | |
| JP2022029125A (en) | Classification unit, generation unit, dataset generation device, frame image classification method, and frame image classification program | |
| CN104104911A (en) | Timestamp eliminating and resetting method in panoramic image generation process and system thereof | |
| US20240221426A1 (en) | Behavior detection method, electronic device, and computer readable storage medium | |
| CN111601108B (en) | Monitoring video processing method and device and video monitoring terminal | |
| CN112381024A (en) | Multi-mode fused unsupervised pedestrian re-identification and rearrangement method | |
| CN112015231A (en) | Method and system for processing surveillance video partition | |
| CN111553408A (en) | Automatic test method for video recognition software | |
| CN111882656A (en) | Graph processing method, equipment and storage medium based on artificial intelligence | |
| CN110826455A (en) | Target identification method and image processing equipment | |
| CN112585946A (en) | Image shooting method, image shooting device, movable platform and storage medium | |
| Liang et al. | Improving Accuracy of Small Object Detection in Screen based on YOLOv7 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |