CN110717932A - Method for detecting state of scissor type knife switch by real-time tracking - Google Patents

Method for detecting state of scissor type knife switch by real-time tracking Download PDF

Info

Publication number
CN110717932A
CN110717932A CN201910895598.8A CN201910895598A CN110717932A CN 110717932 A CN110717932 A CN 110717932A CN 201910895598 A CN201910895598 A CN 201910895598A CN 110717932 A CN110717932 A CN 110717932A
Authority
CN
China
Prior art keywords
knife switch
frame image
state
switch arm
arm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910895598.8A
Other languages
Chinese (zh)
Inventor
任大明
汪辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Xinhe Electronic Technology Co Ltd
Original Assignee
Nanjing Xinhe Electronic Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Xinhe Electronic Technology Co Ltd filed Critical Nanjing Xinhe Electronic Technology Co Ltd
Priority to CN201910895598.8A priority Critical patent/CN110717932A/en
Publication of CN110717932A publication Critical patent/CN110717932A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
    • G01R31/327—Testing of circuit interrupters, switches or circuit-breakers
    • G01R31/3271—Testing of circuit interrupters, switches or circuit-breakers of high voltage or medium voltage devices
    • G01R31/3275—Fault detection or status indication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Abstract

The invention provides a detection method of a scissor type switch state tracked in real time, which comprises the steps of collecting a real-time monitoring video, establishing a model file, and obtaining a training model marking positions of a switch arm and a switch arm joint point; automatically positioning the knife switch arm in the first frame of image by utilizing a training model and a model file and acquiring a knife switch estimation state, sequentially detecting until the knife switch estimation state in one frame of image is closed or virtual, marking the frame of image as an initial frame of image, calculating a joint point of the knife switch arm and an angular bisector of the knife switch arm in the initial frame of image, and simultaneously extracting and recording feature points; tracking and positioning the knife switch arm joint point of the current frame image by using the characteristic point of the previous frame image, calculating the knife switch arm angular bisector in the current frame image, calculating the included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state; and detecting that the knife switch stops moving, and verifying the final state of the movement of the knife switch in the video. The invention solves the problem of low real-time detection speed of the scissor type disconnecting link and improves the detection accuracy.

Description

Method for detecting state of scissor type knife switch by real-time tracking
Technical Field
The invention relates to the technical field of image recognition, in particular to a method for detecting the state of a scissor type knife switch by real-time tracking.
Background
The knife switch is an electric appliance which is frequently used in a high-voltage switch electric appliance, and plays an isolation role in a circuit. In practical applications, due to the long-term operation of the knife switch, the situation that the knife switch is not opened or closed in place can occur, and the situation can cause electric arcs to be generated between the upper knife switch arm and the lower knife switch arm of the knife switch. The electric arc is a gas discharge phenomenon, and the possibility of instantaneous sparks generated by current passing through certain insulating media can cause fire or threaten personal safety; in addition, the arc temperature is extremely high, and insulating materials are easily burnt, so that a current leakage event or damage to disconnecting link equipment is caused; therefore, the closed state of the knife switch needs to be accurately detected.
Whether the scissor type knife switch is in place or not is detected, and manual observation is mainly needed in practice; in the prior art, the degree of opening and closing of the knife switch is also judged by calculating the angle between the upper knife switch arm and the lower knife switch arm. At present, the angle between upper and lower knife switch arms is calculated, and a plurality of calculation schemes appear at home and abroad, one is to change the interior of a circuit and add a sensor or a signal lamp; one is a method based on digital image processing, which judges the on-off state of a knife switch through a monitoring image, and in the method based on digital image processing, due to the complex background, the knife switch is difficult to be correctly matched, and the detection precision is reduced. In addition, due to the influence of the shooting environment, the edge of the knife switch arm in the image is fuzzy, the detection effect is influenced, and the angle between the upper knife switch arm and the lower knife switch arm cannot be accurately detected.
Disclosure of Invention
The invention provides a method for detecting the state of a scissor type switch in real time tracking, which aims to solve the problems that whether the scissor type switch is in place or not is difficult to detect quickly and accurately and the real-time detection speed is low in the prior art.
The technical scheme of the invention is realized as follows:
a detection method of scissor type switch state of real-time tracking collects real-time monitoring video of the scissor type switch, obtains any frame image of the real-time video, establishes a boundary line model of a switch arm of the type switch according to the frame image and stores the boundary line model as a model file;
acquiring a sample image of the scissor type knife switch through multiple ways, and training the sample image by using a deep learning algorithm to acquire a training model for marking positions of joint points of a knife switch arm and the knife switch arm;
sequentially detecting from a first frame image of a video, automatically positioning a disconnecting link arm in the first frame image by using a training model and a model file and acquiring a disconnecting link estimation state, if the disconnecting link estimation state in the first frame image is open, directly outputting the disconnecting link state as open and continuously detecting a second frame image until the disconnecting link estimation state in the first frame image is closed or virtually closed, and marking the frame image as an initial frame image of a detection angle; acquiring a rectangular frame of a knife switch arm joint point in an initial frame image according to the model file and the training model, calculating the joint point of the knife switch arm and an angular bisector of the knife switch arm in the initial frame image, and simultaneously extracting and recording feature points which are favorable for tracking in the rectangular frame of the knife switch arm joint point in the initial frame image; performing edge detection on the knife switch in the initial frame image according to the positioning result of the knife switch arm in the initial frame image, determining the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm in the initial frame image, further calculating the included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state of the knife switch in the initial frame image;
tracking and positioning a joint point of a knife switch arm in a current frame image by using a characteristic point of a previous frame image, calculating an angular bisector of the knife switch arm in the current frame image according to a tracking and positioning result, automatically positioning a knife switch in the current frame image, performing edge detection according to a positioning result, determining a final edge line of an upper knife switch arm and an edge line of a lower knife switch arm in the current frame image, further calculating an included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state of the knife switch in the current frame image;
sequentially processing the next frame of image in the same way as the current frame of image, if the movement direction of the disconnecting link is from open to closed, checking the final state of the movement of the disconnecting link in the video until the disconnecting link is detected to stop moving; if the movement direction of the disconnecting link is from closing to opening, when the state of the disconnecting link is detected to be changed into opening, the opening state of the disconnecting link is directly output, the angle between the disconnecting link arms does not need to be calculated until the disconnecting link is detected to stop moving, and the final state of the movement of the disconnecting link in the video is verified.
Preferably, the method for acquiring any frame image of the real-time video, establishing a boundary line model of the switch arm of the switch of the model according to the frame image and storing the boundary line model as the model file comprises the following steps: acquiring any frame image of a video, setting a program on a computer, opening the frame image by using the program, and performing delineation on left and right boundaries and upper and lower boundaries of upper and lower knife gate arms of a knife gate to form a quadrilateral template, wherein the upper and lower knife gate arms are in the quadrilateral template, and four endpoint coordinates of the quadrilateral template are stored as a model file; if the knife switch in the frame image is in a closed state, performing edge tracing according to the actual position of the knife switch arm, and if the knife switch in the frame image is in an open or virtual closing state, estimating the position of the knife switch arm in the closed state according to the upper and lower fixed boundaries of the knife switch, and performing edge tracing on the position of the knife switch arm in the closed state.
Preferably, the specific way of automatically positioning the knife switch arm in the first frame image and acquiring the estimated state of the knife switch by using the training model and the model file is as follows:
detecting a current frame image by using a training model, and acquiring all knife switch areas and knife switch estimation states corresponding to each knife switch area in the current frame image, wherein the areas corresponding to the knife switches in the closed state and the virtual closed state are integral rectangular frame areas containing upper and lower knife switch arms and upper and lower end points, and the areas corresponding to the knife switches in the open state are folding rectangular frame areas containing the upper and lower knife switch arms and the lower end points;
acquiring a closed quadrilateral frame area, an upper midpoint and a lower midpoint of the disconnecting link in a closed state according to the model file, wherein the upper midpoint is the midpoint of the upper boundary of the closed quadrilateral frame, and the lower midpoint is the midpoint of the lower boundary of the closed quadrilateral frame, and calculating the overlapping areas of the integral rectangular frame area and the folding rectangular frame area and the closed quadrilateral frame area respectively;
traversing all areas where the switches in the closed state and the virtual closed state are located in the current frame image, giving an overlap area threshold value, and obtaining an area which contains an upper midpoint and a lower midpoint, has the largest overlap area and is larger than the overlap area threshold value, wherein the area is the final integral area of the upper and lower switch arms, so that the switch positioning in the closed state or the virtual closed state is realized;
if the area which contains the upper midpoint and the lower midpoint and has the largest overlapping area and is larger than the threshold of the overlapping area does not exist, the current state of the disconnecting link to be detected is not in a 'closed' state or a 'virtual closed' state, disconnecting link positioning is realized from all disconnecting links in the 'open' state, the areas where all disconnecting links in the 'open' state are located in the target image are traversed, the area which contains the lower midpoint and has the largest overlapping area and is larger than the threshold of the overlapping area is obtained, and if the area exists, the area is the final disconnecting link arm area in the open state, so that disconnecting link positioning in the 'open' state is realized.
Preferably, the specific way of determining the final edge line of the upper knife gate arm and the final edge line of the lower knife gate arm in the initial frame image by performing edge detection on the knife gate in the frame image according to the positioning result of the knife gate arm in the initial frame image is as follows:
performing edge detection on the initial frame image according to the positioning result of the knife gate arm to obtain an edge line set, calculating the distance from the knife gate arm joint point of the current frame image to each edge line in the edge line set, giving an upper distance threshold value and a lower distance threshold value, removing edge lines corresponding to the distances which are not within the threshold value range, and obtaining an updated edge line set;
and (3) utilizing an angular bisector to vertically distinguish the updated edge line set, respectively obtaining an upper knife switch arm edge line set and a lower knife switch arm edge line set, and symmetrically pairing to determine the final upper knife switch arm edge line and the final lower knife switch arm edge line.
Preferably, the rectangular frame of the joint point of the knife gate arm in the initial frame image is obtained according to the model file and the training model, and the specific way of calculating the joint point of the knife gate arm in the initial frame image is as follows: calculating the central point of the quadrilateral template according to the model file, setting an estimation threshold value R, corresponding the central point to an initial frame image, taking the central point as the center in the initial frame image, establishing an estimation joint point rectangular frame containing the joint point of the blade arm in a closed state in the initial frame image, wherein the width of the blade arm is wide, and the length of the blade arm is R times the width of the blade arm; and acquiring all corrected joint point rectangular frames containing the marked joint point positions after positioning in the initial frame image according to the training model, and calculating the corrected joint point rectangular frame with the largest overlapping area with the estimated joint point rectangular frame in the corrected joint point rectangular frames, wherein the corrected joint point rectangular frame is the knife gate arm joint point rectangular frame in the initial frame image, and the center of the corrected joint point rectangular frame is the knife gate arm joint point.
Preferably, the specific way of calculating the bisector of the blade arm in the initial frame image is as follows: acquiring a distortion coefficient of a camera for acquiring a real-time video, and calculating a camera matrix camera _ matrix [ -focal _ length,0, center.x ] according to the size of an initial frame image; 0, focal _ length, center.y; 0,0,1], where focal _ length is the initial frame image width, center.x and center.y are the x and y coordinates of the point in the initial frame image, respectively;
estimating corresponding world coordinates according to the coordinates of the four end points in the model file, acquiring a rotation matrix and a translation matrix of a coordinate system where the camera is located relative to a world coordinate system according to the coordinates of the four end points, the world coordinates, the camera matrix and the distortion coefficient in the model file, and acquiring a transformation matrix according to the rotation matrix and the translation matrix;
acquiring the joint point coordinates of the knife switch arm joint points in the ideal image according to the transformation matrix to obtain the angular bisector of the knife switch arm joint points of the ideal image, wherein the angular bisector is parallel to the ground;
and calculating the corresponding coordinate of any point on the angular bisector of the ideal image in the initial frame image through a transformation matrix, wherein the connecting line of the corresponding coordinate and the joint point of the knife-switch arm is the angular bisector of the knife-switch arm in the initial frame image.
Preferably, the specific way of tracking and positioning the joint point of the blade arm in the current frame image by using the feature point of the previous frame image is as follows: the feature points in the previous frame image are located in the joint point rectangular frame when the knife-arm joint points in the previous frame image are located, the feature points in the previous frame image are corresponding to the current frame image and the positions of the feature points in the current frame image are obtained, and the feature points in the previous frame image are corresponding to the tracking points in the current frame image; and acquiring a minimum rectangular frame containing all tracking points in the current frame image according to the tracking points, namely a joint point rectangular frame in the current frame image, wherein the midpoint of the joint point rectangular frame in the current frame image is a knife switch arm joint point, so that the knife switch arm joint point tracking and positioning are realized.
Preferably, the specific manner of determining the real-time status in the initial frame image is as follows: two angle thresholds T-o and T-c of the scissor type knife switch in the states of opening and closing are given according to the requirements of users;
and if the actual included angle between the upper knife switch arm and the lower knife switch arm is between T-o and T-c, judging that the knife switch in the initial frame image is in a closed state.
Preferably, the specific way of determining the real-time state of the knife switch in the current frame image is as follows:
giving two angle thresholds T-o and T-c of an opening state and a closing state of the scissor type knife switch according to user requirements, and setting a high threshold hT-o and a low threshold lT-o of the opening state, wherein the T-o is between the high threshold hT-o and the low threshold lT-o, and hT-o is greater than T-o; setting a high threshold hT-c and a low threshold lT-c of a closed state, wherein T-c is between the high threshold hT-c and the low threshold lT-c, hT-c > T-c > lT-c, and lT-o > hT-c;
if the correction included angle between the upper knife switch arm and the lower knife switch arm is larger than the hT-o, the knife switch in the current frame image is judged to be in an open state, if the correction included angle between the left knife switch arm and the right knife switch arm is smaller than the lT-c, the knife switch in the current frame image is judged to be in a closed state, and if the correction included angle between the left knife switch arm and the right knife switch arm is between the hT-c and the lT-o, the knife switch in the current frame image is judged to be in a virtual closed state; and if the correction included angle between the left knife switch arm and the right knife switch arm is between lT-o and hT-o or between lT-c and hT-c, judging that the state of the knife switch in the current frame image is the same as the state of the knife switch in the previous frame image.
Preferably, until the knife switch is detected to stop moving, the specific way of verifying the final state of the knife switch movement in the video is as follows: judging the real-time state of the knife switch in each continuous frame of image and recording the real-time state of the knife switch; giving a threshold value of the continuous frame number, if the state of the disconnecting link in the continuous frame images is kept unchanged and the continuous frame number is greater than the threshold value of the continuous frame number, indicating that the disconnecting link maintains the same state, and detecting that the disconnecting link stops moving;
if the state of the disconnecting link in the first frame of image is an open state, the operation is finished if the final state is verified to be a closed state, and if the final state is verified to be a virtual closed state, an alarm is given and staff is asked to confirm and process manually; if the state of the disconnecting link is a closed state, the operation is finished if the final state is verified to be an open state, and if the final state is verified to be a virtual closed state, the operation is prompted by an alarm and the manual confirmation and processing of workers are requested; if the state of the disconnecting link is a virtual closing state, the operation is finished when the final state is verified to be a closed state or an open state, and if the final state is verified to be the virtual closing state, the operation is prompted by an alarm and the worker is requested to confirm and process the operation manually.
The invention has the beneficial effects that: according to the method for detecting the state of the scissor type knife switch tracked in real time, the quadrilateral template of the knife switch arm in the closed state in the model file is correspondingly used as a reference in each frame of image of the real-time monitoring video, so that each frame of image of the real-time monitoring video can be conveniently processed subsequently. The method comprises the steps of firstly obtaining the position of a knife switch arm to be detected and the corresponding knife switch state in each frame of image according to a training model and a model file, directly outputting the state if the knife switch state is open, not calculating the angle between the knife switch arms, calculating the angle between the knife switch arms if the knife switch state is closed or virtual, accurately analyzing the real-time state of the knife switch according to the user requirement and the angle between the knife switch arms, simultaneously marking the frame of image as an initial frame of image, obtaining a knife switch arm joint point rectangular frame of the initial frame of image by using the training model and the model file, and improving the real-time detection speed by extracting characteristic points which are beneficial to tracking in the knife switch arm joint point rectangular frame in the initial frame of image and by tracking the characteristic points in real time when processing the subsequent frame of image.
Calculating a knife switch arm angular bisector according to the knife switch arm joint points, and acquiring an upper knife switch arm edge line set and a lower knife switch arm edge line set by using the knife switch arm angular bisector, so that the accuracy is improved; then, determining an upper knife switch arm edge line and a lower knife switch arm edge line in a pairing manner; and finally, calculating an included angle between the upper knife switch arm and the lower knife switch arm, judging the real-time state of the knife switch according to the included angle, verifying the final state of the movement of the knife switch in the video, improving the real-time detection precision and having important practical application value.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is an image of a build model file according to the present invention;
FIG. 2 is a schematic diagram of a structure for calculating the joint point of the knife arm;
fig. 3 is a schematic structural diagram of an updated edge line set divided up and down by using an angle bisector.
In the figure:
1. an upper knife switch arm; 2. a lower knife switch arm; 3. a knife gate arm articulation point; 4. a center point; 5. estimating a joint point rectangular frame; 6. correcting a joint point rectangular frame; 7. an angular bisector.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1: as shown in fig. 1, a method for detecting a state of a scissor-type switch with real-time tracking includes acquiring a real-time monitoring video of the scissor-type switch, acquiring any frame image of the real-time video, establishing a boundary line model of a switch arm of the scissor-type switch according to the frame image, and storing the boundary line model as a model file; acquiring a sample image of the scissor type knife switch through multiple ways, and training the sample image by using a deep learning algorithm to acquire a training model for marking positions of joint points of a knife switch arm and the knife switch arm; sequentially detecting from a first frame image of a video, automatically positioning a disconnecting link arm in the first frame image by using a training model and a model file and acquiring a disconnecting link estimation state, if the disconnecting link estimation state in the first frame image is open, directly outputting the disconnecting link state as open and continuously detecting a second frame image until the disconnecting link estimation state in the first frame image is closed or virtually closed, and marking the frame image as an initial frame image of a detection angle; acquiring a rectangular frame of a knife switch arm joint point in an initial frame image according to the model file and the training model, calculating the joint point of the knife switch arm and an angular bisector of the knife switch arm in the initial frame image, and simultaneously extracting and recording feature points which are favorable for tracking in the rectangular frame of the knife switch arm joint point in the initial frame image; performing edge detection on the knife switch in the initial frame image according to the positioning result of the knife switch arm in the initial frame image, determining the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm in the initial frame image, further calculating the included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state of the knife switch in the initial frame image; tracking and positioning a joint point of a knife switch arm in a current frame image by using a characteristic point of a previous frame image, calculating an angular bisector of the knife switch arm in the current frame image according to a tracking and positioning result, automatically positioning a knife switch in the current frame image, performing edge detection according to a positioning result, determining a final edge line of an upper knife switch arm and an edge line of a lower knife switch arm in the current frame image, further calculating an included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state of the knife switch in the current frame image; if the movement direction of the disconnecting link is from opening to closing, the final state of the disconnecting link movement in the video is verified until the disconnecting link is detected to stop moving; if the movement direction of the disconnecting link is from closing to opening, when the state of the disconnecting link is detected to be changed into opening, the opening state of the disconnecting link is directly output, the angle between the disconnecting link arms does not need to be calculated until the disconnecting link is detected to stop moving, and the final state of the movement of the disconnecting link in the video is verified.
If the state of the knife switch in the first frame image of the video is open and the state of the knife switch is virtually closed or closed after being changed, the movement direction of the knife switch is from open to closed; if the state of the knife switch in the first frame image of the video is closed and the state of the knife switch is virtual closed or opened after being changed, the movement direction of the knife switch is from closed to open. The real-time state of the knife switch in the initial frame image is an output state.
When each frame of image of the video is detected in sequence, the training model and the model file are utilized to automatically position the knife switch arm in the detected frame of image and obtain the knife switch state, wherein the obtained knife switch state is a more general state, and the subsequently obtained real-time state is the knife switch state which is accurately analyzed according to the angle between the knife switch arms. If the movement direction of the knife switch is from opening to closing, the state of the knife switch in the first frame image is opening, and the opening state can be detected according to the requirements of users without accurately calculating the angle between knife switch arms. When the initial frame image in the real-time video is detected to be in a closed state or a virtual closed state, the specific state of the knife switch needs to be determined specifically according to an included angle between the upper knife switch arm and the lower knife switch arm, and the real-time state of the scissor type knife switch is accurately detected. If the movement direction of the knife switch is from closing to opening, the state of the knife switch in the first frame image is closing, the first frame image is the initial frame image of the calculated angle, the specific state of the knife switch needs to be determined specifically according to the included angle between the upper knife switch arm and the lower knife switch arm, the real-time state of the scissor type knife switch is accurately detected until the state of the knife switch is changed into opening, the angle does not need to be accurately calculated, and the opening state is directly output.
In embodiment 1, the manner of "acquiring any frame image of a real-time video, establishing a boundary line model of a switch arm of a switch of the model according to the frame image, and storing the boundary line model as a model file" is as follows: acquiring any frame image of a video, setting a program on a computer, opening the frame image by using the program, and performing delineation on left and right boundaries and upper and lower boundaries of upper and lower knife gate arms of a knife gate to form a quadrilateral template, wherein the upper and lower knife gate arms are in the quadrilateral template, and four endpoint coordinates of the quadrilateral template are stored as a model file; if the knife switch in the frame image is in a closed state, performing edge tracing according to the actual position of the knife switch arm, and if the knife switch in the frame image is in an open or virtual closing state, estimating the position of the knife switch arm in the closed state according to the upper and lower fixed boundaries of the knife switch, and performing edge tracing on the position of the knife switch arm in the closed state. Therefore, the model file can be determined from any frame image of the real-time video.
In embodiment 1, a specific manner for automatically positioning the knife gate arm in the first frame image and obtaining the estimated state of the knife gate by using the training model and the model file is as follows: detecting a current frame image by using a training model, and acquiring all knife switch areas and knife switch estimation states corresponding to each knife switch area in the current frame image, wherein the areas corresponding to the knife switches in the closed state and the virtual closed state are integral rectangular frame areas containing upper and lower knife switch arms and upper and lower end points, and the areas corresponding to the knife switches in the open state are folding rectangular frame areas containing the upper and lower knife switch arms and the lower end points; acquiring a closed quadrilateral frame area, an upper midpoint and a lower midpoint of the disconnecting link in a closed state according to the model file, wherein the upper midpoint is the midpoint of the upper boundary of the closed quadrilateral frame, and the lower midpoint is the midpoint of the lower boundary of the closed quadrilateral frame, and calculating the overlapping areas of the integral rectangular frame area and the folding rectangular frame area and the closed quadrilateral frame area respectively; traversing all areas where the switches in the closed state and the virtual closed state are located in the current frame image, giving an overlap area threshold value, obtaining an area which contains an upper middle point and a lower middle point and has the largest overlap area, and obtaining the area which is the final integral area of the upper and lower switch arms, so as to realize the switch positioning in the closed state or the virtual closed state; if the area which contains the upper midpoint and the lower midpoint and has the largest overlapping area and is larger than the threshold of the overlapping area does not exist, the current state of the disconnecting link to be detected is not in a 'closed' state or a 'virtual closed' state, disconnecting link positioning is realized from all disconnecting links in the 'open' state, the areas where all disconnecting links in the 'open' state are located in the target image are traversed, the area which contains the lower midpoint and has the largest overlapping area and is larger than the threshold of the overlapping area is obtained, and if the area exists, the area is the final disconnecting link arm area in the open state, so that disconnecting link positioning in the 'open' state is realized. And acquiring the estimated state of the knife switch in each frame of image by using the training model and the model file.
As shown in fig. 2, a rectangular frame of the joint point of the knife gate arm in the initial frame image is obtained according to the model file and the training model, and the specific way of calculating the joint point of the knife gate arm in the initial frame image is as follows: calculating the central point of the quadrilateral template according to the model file, setting an estimation threshold value R, corresponding the central point to an initial frame image, taking the central point 4 as the center in the initial frame image, establishing an estimation joint point rectangular frame 5 containing the joint point of the blade arm in a closed state in the initial frame image, wherein the width of the blade arm is wide, and the length of the blade arm is R times of the width of the blade arm; and acquiring all corrected joint point rectangular frames 6 containing the positions of the marked joint points after positioning in the initial frame image according to the training model, and calculating the corrected joint point rectangular frame 6 with the largest overlapping area with the estimated joint point rectangular frame in the corrected joint point rectangular frames, wherein the corrected joint point rectangular frame is the knife gate arm joint point rectangular frame in the initial frame image, and the center of the corrected joint point rectangular frame is the knife gate arm joint point.
The mode of calculating the angle bisector of the knife switch arm in the initial frame image is as follows: acquiring a distortion coefficient of a camera for acquiring a real-time video, and calculating a camera matrix camera _ matrix [ -focal _ length,0, center.x ] according to the size of an initial frame image; 0, focal _ length, center.y; 0,0,1], where focal _ length is the initial frame image width, center.x and center.y are the x and y coordinates of the point in the initial frame image, respectively; estimating corresponding world coordinates according to Four end point coordinates in a model file, acquiring a rotation matrix and a translation matrix of a coordinate system where a camera is located relative to a world coordinate system according to the Four end point coordinates, the world coordinates, a camera matrix and a distortion coefficient in the model file by using a camera posture Estimation method proposed in a reference document 'position Estimation based on Four focus co-planar point coresponsors', acquiring a transformation matrix according to the rotation matrix and the translation matrix, setting the transformation matrix as R _ matrix and a knife gate arm joint point as artist _ p, and acquiring an artist point coordinate artist _ q (artist _ q.x and artist _ q.y) of the knife gate arm joint point in an ideal image according to the transformation matrix to obtain an angle line y (artist _ q.y/artist _ q.x) parallel to the ground and passing the closing point of the ideal image; and calculating the corresponding coordinate of any point angle _ p on the angular bisector of the ideal image in the initial frame image through a transformation matrix, wherein the connecting line of the corresponding coordinate angle _ q and the knife-blade arm joint point artis _ p is the angular bisector 7 of the initial frame image.
The ideal image theoretically refers to the knife switch image which is shot by a camera perpendicular to the knife switch, and the center point of the camera and the joint point of the knife switch arm are on the same straight line. In the actual shooting of the camera, the knife switch or the camera generally has a certain degree of position deviation, and the knife switch arm joint point of the ideal image acquired by using the knife switch arm joint point and the transformation matrix and the knife switch arm joint point of the theoretical ideal image have some errors which do not influence the calculation of the angular bisector.
The specific way of determining the final edge line of the upper knife gate arm and the final edge line of the lower knife gate arm in the initial frame image by performing edge detection on the knife gate in the initial frame image according to the positioning result of the knife gate arm in the initial frame image is as follows: performing edge detection on the initial frame image according to the positioning result of the knife gate arm to obtain an edge line set, calculating the distance from the knife gate arm joint point of the initial frame image to each edge line in the edge line set, giving an upper distance threshold value and a lower distance threshold value, removing edge lines corresponding to the distances which are not within the threshold value range, and obtaining an updated edge line set; and (3) utilizing an angular bisector to vertically distinguish the updated edge line set, respectively obtaining an upper knife switch arm edge line set and a lower knife switch arm edge line set, and symmetrically pairing to determine the final upper knife switch arm edge line and the final lower knife switch arm edge line.
When the edge line of the upper knife switch arm and the edge line of the lower knife switch arm in the initial frame image are determined, the updated edge line set is vertically distinguished by utilizing an angular bisector, and the specific mode of respectively acquiring the edge line set of the upper knife switch arm and the edge line set of the lower knife switch arm is as follows: calculating a unit direction vector which is perpendicular to an angular bisector and has an upward direction or a downward direction and takes a joint point as a starting point in the target image
Figure BDA0002210118910000101
Traversing and updating the edge lines in the edge line set, recording two end points of any edge line as T1 and T2, and recording the vector of T1 and the knife gate arm joint point as
Figure BDA0002210118910000102
The vector of the T2 and the knife arm joint point is recorded as
Figure BDA0002210118910000111
Computing
Figure BDA0002210118910000112
And
Figure BDA0002210118910000113
for unit direction vector of direction up
Figure BDA0002210118910000114
If it is
Figure BDA0002210118910000115
Then T1 is above the bisector, otherwise, below the bisector, if so
Figure BDA0002210118910000116
Then T2 is above the bisector, otherwise, below the bisector; for unit direction vector of downward direction
Figure BDA0002210118910000117
If it is
Figure BDA0002210118910000118
Then T1 is below the bisector, otherwise, above the bisector, if so
Figure BDA0002210118910000119
Then T2 is below the bisector, otherwise, above the bisector;
if T1 and T2 are both on the upper side of the angular bisector, the edge line is an upper knife gate arm edge line, if T1 and T2 are both on the lower side of the angular bisector, the edge line is a lower knife gate arm edge line, if T1 and T2 are respectively on both sides of the angular bisector, the length of the upper line segment and the lower line segment of the edge line with the angular bisector as a dividing line is calculated, the edge line is the edge line on the longer side of the upper line segment and the lower line segment, and an upper knife gate arm edge line set and a lower knife gate arm edge line set are obtained.
As shown in FIG. 3, n is a unit vector, T1 and T2 are two end points of an edge line, P1 and P2 are vectors formed by connecting a joint point of a knife switch arm with T1 and T2, and the included angles between P1 and P2 and n are acute angles, so that the included angles are acute angles, and the knife switch is simple in structure and convenient to use, and the knife switch is convenient to use
Figure BDA00022101189100001110
Figure BDA00022101189100001111
Therefore, T1 and T2 are both arranged above the angle bisector, and the edge line formed by T1 and T2 is the edge line of the upper knife switch arm.
Unit direction vector
Figure BDA00022101189100001114
The specific calculation method comprises the following steps: setting the angular bisector line of the knife switch arm on the target image as y ═ kx + b, and the coordinates of the joint point of the knife switch arm as (C)x,Cy) Setting the equation of a straight line which is perpendicular to the angle bisector and passes through the perpendicular line of the joint point of the knife brake arm as y ═ cx + d, C ═ 1/k, Cy＝-1/k*Cx+d，d＝Cy+1/k*Cx(ii) a Calculating a unit direction vector with the joint point of the knife switch arm as a starting point and in an upward direction or a downward direction according to a linear equation of the vertical line
Figure BDA00022101189100001112
Let the end point coordinate of the unit direction vector be (n)x,ny)，
Then (n)y-Cy)/(nx-Cx)＝-1/k,(ny-Cy)2+(nx-Cx)2When it is 1, n is calculatedxAnd nyUnit direction vector
The method for symmetrically pairing the upper knife switch arm edge line set and the lower knife switch arm edge line set and determining the final upper knife switch arm edge line and the final lower knife switch arm edge line in the initial frame image comprises the following steps: given an angular threshold, for any edge line in the upper knife gate arm edge line set relative to the horizontal positionIs recorded as MhH represents different upper knife switch arm edge lines, the angle difference values of the edge lines of any two upper knife switch arms within the angle threshold value are divided into the same group, and the upper knife switch arm edge lines in the same group are sorted from long to short; the angle of any edge line in the lower blade arm edge line set relative to the horizontal position is recorded as NmM represents different lower knife switch arm edge lines, the angle difference value of any two lower knife switch arm edge lines within the angle threshold value is divided into the same group, and the lower knife switch arm edge lines in the same group are sorted from long to short;
and traversing all the same groups of the upper knife switch arm edge line set and all the same groups of the lower knife switch arm edge line set, pairing one by one, calculating any two groups of corrected knife switch arm edge lines and pairing scores, and obtaining the corrected upper knife switch arm edge line and the corrected lower knife switch arm edge line corresponding to the combination with the highest score as the final upper knife switch arm edge line and the final lower knife switch arm edge line.
The mode of traversing all the same groups of the upper knife switch arm edge line set and all the same groups of the lower knife switch arm edge line set and pairing one by one and calculating any two groups of correction knife switch arm edge lines and pairing scores is as follows: as shown in fig. 2, calculating an upper midpoint between two top endpoints and a lower midpoint between two bottom endpoints according to a model file, calculating an average angle of each group, combining the average angle of each group in the upper knife gate arm edge line set with the upper midpoint to obtain a corrected upper knife gate arm edge line of each group in the upper knife gate arm edge line set, and marking an intersection point of the corrected upper knife gate arm edge line and the angular bisector as a point a; the average angle of each group in the lower knife switch arm edge line set is combined with the lower midpoint to obtain the corrected lower knife switch arm edge line of each group in the lower knife switch arm edge line set, the intersection point of the corrected lower knife switch arm edge line and the angular bisector is marked as a point B, and the distance d between the point A and the point B is calculatedAB(ii) a The distance from the upper fixed boundary to the lower fixed boundary of the knife switch arm is d0，dAB/d0If the number is more than 0.5, the same group where the point A is located and the same group where the point B is located are removed;
the calculation formula of any two groups of matching scores is set as
Figure BDA0002210118910000131
Wherein the coefficient STRepresenting the square sum of the edge line lengths in any one same group of the edge line sets of the upper knife gate arm; sBRepresenting the sum of the squares of edge line lengths in any one same group of the lower blade arm edge line sets; beta is an adjustable sensitivity parameter; and the edge line of the upper knife switch arm in the correction position corresponding to the point A and the edge line of the lower knife switch arm in the correction position corresponding to the point B corresponding to the highest pairing score S are the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm.
Assuming that the number of edge lines in any one same group of the edge line set of the upper knife gate arm is K, each edge line can be represented as LiCalculating the length of the edge line as LLEngth according to the coordinates of the starting point and the end point of the edge lineiK, where i is 0,1,. K, then ST＝LLength0 2+LLength1 2...+LLengthK 2(ii) a Assuming that the number of edge lines in any one same group in the edge line set of the lower blade arm is Q, each edge line can be represented as RjCalculating the length of the edge line as RLength according to the coordinates of the starting point and the ending point of the edge linejWherein j is 0,1, Q, then SB＝RLength0 2+RLength1 2...+RLengthQ 2(ii) a Beta is an adjustable sensitivity parameter, different scores can be obtained by taking different beta values in actual use, the matching result is determined according to the different scores, the beta value corresponding to the best matching result is selected as the final beta value, and different users can also determine the final beta value according to actual conditions; and the edge line of the upper knife switch arm in the correction position corresponding to the point A and the edge line of the lower knife switch arm in the correction position corresponding to the point B corresponding to the highest pairing score S are the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm.
The specific mode of tracking and positioning the joint point of the knife arm in the current frame image by using the characteristic point of the previous frame image is as follows: the feature points in the previous frame image are located in the joint point rectangular frame when the knife-arm joint points in the previous frame image are located, the feature points in the previous frame image are corresponding to the current frame image and the positions of the feature points in the current frame image are obtained, and the feature points in the previous frame image are corresponding to the tracking points in the current frame image; according toThe tracking point obtains a minimum rectangular frame containing all tracking points in the current frame image, namely a joint point rectangular frame in the current frame image, and the midpoint of the joint point rectangular frame in the current frame image is a knife switch arm joint point, so that the knife switch arm joint point in the current frame image is tracked and positioned. The specific mode of calculating the knife switch arm angle bisector in the current frame image according to the positioning result is the same as the specific mode of calculating the knife switch arm angle bisector in the initial frame image. The specific manner of determining the final upper knife gate arm edge line and the final lower knife gate arm edge line in the current frame image is the same as the specific manner of determining the final upper knife gate arm edge line and the final lower knife gate arm edge line in the initial frame image. Processing the subsequent image is the same as processing the current frame image. The way to "calculate the angle between the upper and lower knife arm" is: taking the final edge line of the upper knife switch arm and the edge line of the lower knife switch arm as a reference, and if the final angle of the edge line of the upper knife switch arm relative to the straight line of the angular bisector of the knife switch arm in the closed state is alpha1The final angle of the edge line of the lower knife switch arm relative to the straight line of the angular bisector of the knife switch arm in the closed state is alpha2The final angle between the upper and lower knife arms is theta ═ alpha2-α1|。
The specific way of judging the real-time state in the initial frame image is as follows: two angle thresholds T-o and T-c of the scissor type knife switch in the states of opening and closing are given according to the requirements of users; and if the actual included angle between the upper knife switch arm and the lower knife switch arm is between T-o and T-c, judging that the knife switch in the initial frame image is in a closed state. In actual use, the angle threshold is determined according to actual use conditions.
The specific way of judging the real-time state of the knife switch in the current frame image is as follows: giving two angle thresholds T-o and T-c of an opening state and a closing state of the scissor type knife switch according to user requirements, and setting a high threshold hT-o and a low threshold lT-o of the opening state, wherein the T-o is between the high threshold hT-o and the low threshold lT-o, and hT-o is greater than T-o; setting a high threshold hT-c and a low threshold lT-c of a closed state, wherein T-c is between the high threshold hT-c and the low threshold lT-c, hT-c > T-c > lT-c, and lT-o > hT-c; if the correction included angle between the upper knife switch arm and the lower knife switch arm is larger than the hT-o, the knife switch in the current frame image is judged to be in an open state, if the correction included angle between the left knife switch arm and the right knife switch arm is smaller than the lT-c, the knife switch in the current frame image is judged to be in a closed state, and if the correction included angle between the left knife switch arm and the right knife switch arm is between the hT-c and the lT-o, the knife switch in the current frame image is judged to be in a virtual closed state; and if the correction included angle between the left knife switch arm and the right knife switch arm is between lT-o and hT-o or between lT-c and hT-c, judging that the state of the knife switch in the current frame image is the same as the state of the knife switch in the previous frame image.
The method for judging the state of the knife switch in the next frame image is the same as the method for judging the state of the knife switch in the current frame image.
When a model file is established and the estimated state of the disconnecting link is obtained according to a training model, the closing state of the disconnecting link refers to the fact that the upper disconnecting link arm is in contact with the upper fixed boundary and the upper disconnecting link arm and the lower disconnecting link arm are on the same straight line, the virtual closing state refers to the fact that the upper disconnecting link arm is in contact with the upper fixed boundary but the upper disconnecting link arm and the lower disconnecting link arm are not on the same straight line, and the opening state refers to the fact that the upper disconnecting link arm is not in contact with the upper fixed boundary. The state of the knife switch is general, and is different from the state of the knife switch obtained according to the included angle between the knife switch arms, and the state of the knife switch obtained according to the included angle between the knife switch arms is the final state of the knife switch in the target image. Until the knife switch is detected to stop moving, the specific mode for verifying the final state of the knife switch movement in the video is as follows: judging the real-time state of the knife switch in each continuous frame of image and recording the real-time state of the knife switch; giving a threshold value of the continuous frame number, if the state of the disconnecting link in the continuous frame images is kept unchanged and the continuous frame number is greater than the threshold value of the continuous frame number, indicating that the disconnecting link maintains the same state, and detecting that the disconnecting link stops moving; if the state of the disconnecting link in the first frame of image is an open state, the operation is finished if the final state is verified to be a closed state, and if the final state is verified to be a virtual closed state, an alarm is given and staff is asked to confirm and process manually; if the state of the disconnecting link is a closed state, the operation is finished if the final state is verified to be an open state, and if the final state is verified to be a virtual closed state, the operation is prompted by an alarm and the manual confirmation and processing of workers are requested; if the state of the disconnecting link is a virtual closing state, the operation is finished when the final state is verified to be a closed state or an open state, and if the final state is verified to be the virtual closing state, the operation is prompted by an alarm and the worker is requested to confirm and process the operation manually.
The scissor type disconnecting link refers to a vertical telescopic disconnecting switch, the disconnecting switch is divided into an upper disconnecting link arm and a lower disconnecting link arm by taking a middle joint point as a reference, the disconnecting link arms are telescopically opened and closed by utilizing the joint point in the opening and closing process of the disconnecting link, the two disconnecting link arms are on the same straight line when the disconnecting link is completely closed, and the two disconnecting link arms are folded together when the disconnecting link is completely opened.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (10)

1. A detection method of a scissor type switch state tracked in real time is characterized in that a real-time monitoring video of the scissor type switch is collected, any frame image of the real-time video is obtained, a boundary line model of a switch arm of the scissor type switch is established according to the frame image, and the boundary line model is stored as a model file;
acquiring a sample image of the scissor type knife switch through multiple ways, and training the sample image by using a deep learning algorithm to acquire a training model for marking positions of joint points of a knife switch arm and the knife switch arm;
sequentially detecting from a first frame image of a video, automatically positioning a disconnecting link arm in the first frame image by using a training model and a model file and acquiring a disconnecting link estimation state, if the disconnecting link estimation state in the first frame image is open, directly outputting the disconnecting link state as open and continuously detecting a second frame image until the disconnecting link estimation state in the first frame image is closed or virtually closed, and marking the frame image as an initial frame image of a detection angle; acquiring a rectangular frame of a knife switch arm joint point in an initial frame image according to the model file and the training model, calculating the joint point of the knife switch arm and an angular bisector of the knife switch arm in the initial frame image, and simultaneously extracting and recording feature points which are favorable for tracking in the rectangular frame of the knife switch arm joint point in the initial frame image; performing edge detection on the knife switch in the initial frame image according to the positioning result of the knife switch arm in the initial frame image, determining the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm in the initial frame image, further calculating the included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state of the knife switch in the initial frame image;
tracking and positioning a joint point of a knife switch arm in a current frame image by using a characteristic point of a previous frame image, calculating an angular bisector of the knife switch arm in the current frame image according to a tracking and positioning result, automatically positioning a knife switch in the current frame image, performing edge detection according to a positioning result, determining a final edge line of an upper knife switch arm and an edge line of a lower knife switch arm in the current frame image, further calculating an included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state of the knife switch in the current frame image;
sequentially processing the next frame of image in the same way as the current frame of image, if the movement direction of the disconnecting link is from open to closed, checking the final state of the movement of the disconnecting link in the video until the disconnecting link is detected to stop moving; if the movement direction of the disconnecting link is from closing to opening, when the state of the disconnecting link is detected to be changed into opening, the opening state of the disconnecting link is directly output, the angle between the disconnecting link arms does not need to be calculated until the disconnecting link is detected to stop moving, and the final state of the movement of the disconnecting link in the video is verified.
2. The method for detecting the status of a scissor type switch tracked in real time according to claim 1, wherein a manner of acquiring any frame image of a real-time video, establishing a boundary line model of the switch arm of the type of the switch according to the frame image, and storing the boundary line model as a model file "is as follows: acquiring any frame image of a video, setting a program on a computer, opening the frame image by using the program, and performing delineation on left and right boundaries and upper and lower boundaries of upper and lower knife gate arms of a knife gate to form a quadrilateral template, wherein the upper and lower knife gate arms are in the quadrilateral template, and four endpoint coordinates of the quadrilateral template are stored as a model file; if the knife switch in the frame image is in a closed state, performing edge tracing according to the actual position of the knife switch arm, and if the knife switch in the frame image is in an open or virtual closing state, estimating the position of the knife switch arm in the closed state according to the upper and lower fixed boundaries of the knife switch, and performing edge tracing on the position of the knife switch arm in the closed state.
3. The method for detecting the status of a scissor type knife switch tracked in real time according to claim 2, wherein the specific way of automatically positioning the knife switch arm in the first frame image and obtaining the estimated status of the knife switch by using the training model and the model file is as follows:
detecting a current frame image by using a training model, and acquiring all knife switch areas and knife switch estimation states corresponding to each knife switch area in the current frame image, wherein the areas corresponding to the knife switches in the closed state and the virtual closed state are integral rectangular frame areas containing upper and lower knife switch arms and upper and lower end points, and the areas corresponding to the knife switches in the open state are folding rectangular frame areas containing the upper and lower knife switch arms and the lower end points;
acquiring a closed quadrilateral frame area, an upper midpoint and a lower midpoint of the disconnecting link in a closed state according to the model file, wherein the upper midpoint is the midpoint of the upper boundary of the closed quadrilateral frame, and the lower midpoint is the midpoint of the lower boundary of the closed quadrilateral frame, and calculating the overlapping areas of the integral rectangular frame area and the folding rectangular frame area and the closed quadrilateral frame area respectively;
traversing all areas where the switches in the closed state and the virtual closed state are located in the current frame image, giving an overlap area threshold value, and obtaining an area which contains an upper midpoint and a lower midpoint, has the largest overlap area and is larger than the overlap area threshold value, wherein the area is the final integral area of the upper and lower switch arms, so that the switch positioning in the closed state or the virtual closed state is realized;
if the area which contains the upper midpoint and the lower midpoint and has the largest overlapping area and is larger than the threshold of the overlapping area does not exist, the current state of the disconnecting link to be detected is not in a 'closed' state or a 'virtual closed' state, disconnecting link positioning is realized from all disconnecting links in the 'open' state, the areas where all disconnecting links in the 'open' state are located in the target image are traversed, the area which contains the lower midpoint and has the largest overlapping area and is larger than the threshold of the overlapping area is obtained, and if the area exists, the area is the final disconnecting link arm area in the open state, so that disconnecting link positioning in the 'open' state is realized.
4. The method for detecting the status of a scissor-type knife switch tracked in real time according to claim 2, wherein the edge detection of the knife switch in the initial frame image is performed according to the positioning result of the knife switch arm in the initial frame image, and the specific way for determining the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm in the initial frame image is as follows:
performing edge detection on the initial frame image according to the positioning result of the knife switch arm to obtain an edge line set, and calculating the knife switch arm joint point to edge line of the current frame image
Setting an upper distance threshold and a lower distance threshold for the distance of each edge line in the set, and eliminating the edge lines corresponding to the distances which are not within the threshold range to obtain an updated edge line set;
and (3) utilizing an angular bisector to vertically distinguish the updated edge line set, respectively obtaining an upper knife switch arm edge line set and a lower knife switch arm edge line set, and symmetrically pairing to determine the final upper knife switch arm edge line and the final lower knife switch arm edge line.
5. The method for detecting the status of a scissor type knife switch tracked in real time according to claim 2, wherein the rectangular frame of the joint points of the knife switch arm in the initial frame image is obtained according to the model file and the training model, and the specific way of calculating the joint points of the knife switch arm in the initial frame image is as follows: calculating the central point of the quadrilateral template according to the model file, setting an estimation threshold value R, corresponding the central point to an initial frame image, taking the central point as the center in the initial frame image, establishing an estimation joint point rectangular frame containing the joint point of the blade arm in a closed state in the initial frame image, wherein the width of the blade arm is wide, and the length of the blade arm is R times the width of the blade arm; and acquiring all corrected joint point rectangular frames containing the marked joint point positions after positioning in the initial frame image according to the training model, and calculating the corrected joint point rectangular frame with the largest overlapping area with the estimated joint point rectangular frame in the corrected joint point rectangular frames, wherein the corrected joint point rectangular frame is the knife gate arm joint point rectangular frame in the initial frame image, and the center of the corrected joint point rectangular frame is the knife gate arm joint point.
6. The method for detecting the status of a scissor-type knife switch tracked in real time according to claim 5, wherein the specific manner of calculating the angle bisector of the knife switch arm in the initial frame image is as follows: acquiring a distortion coefficient of a camera for acquiring a real-time video, and calculating a camera matrix camera _ matrix [ -focal _ length,0, center.x ] according to the size of an initial frame image; 0, focal _ length, center.y; 0,0,1], where focal _ length is the initial frame image width, center.x and center.y are the x and y coordinates of the point in the initial frame image, respectively;
estimating corresponding world coordinates according to the coordinates of the four end points in the model file, acquiring a rotation matrix and a translation matrix of a coordinate system where the camera is located relative to a world coordinate system according to the coordinates of the four end points, the world coordinates, the camera matrix and the distortion coefficient in the model file, and acquiring a transformation matrix according to the rotation matrix and the translation matrix;
acquiring the joint point coordinates of the knife switch arm joint points in the ideal image according to the transformation matrix to obtain the angular bisector of the knife switch arm joint points of the ideal image, wherein the angular bisector is parallel to the ground;
and calculating the corresponding coordinate of any point on the angular bisector of the ideal image in the initial frame image through a transformation matrix, wherein the connecting line of the corresponding coordinate and the joint point of the knife-switch arm is the angular bisector of the knife-switch arm in the initial frame image.
7. The method for detecting the status of a scissor type knife switch tracked in real time according to claim 5, wherein the specific way of tracking and positioning the joint point of the knife switch arm in the current frame image by using the feature point of the previous frame image is as follows: the feature points in the previous frame image are located in the joint point rectangular frame when the knife-arm joint points in the previous frame image are located, the feature points in the previous frame image are corresponding to the current frame image and the positions of the feature points in the current frame image are obtained, and the feature points in the previous frame image are corresponding to the tracking points in the current frame image; and acquiring a minimum rectangular frame containing all tracking points in the current frame image according to the tracking points, namely a joint point rectangular frame in the current frame image, wherein the midpoint of the joint point rectangular frame in the current frame image is a knife switch arm joint point, so that the knife switch arm joint point tracking and positioning are realized.
8. The method for detecting the status of a scissor type knife switch tracked in real time according to claim 5, wherein the real-time status in the initial frame image is determined by: two angle thresholds T-o and T-c of the scissor type knife switch in the states of opening and closing are given according to the requirements of users;
and if the actual included angle between the upper knife switch arm and the lower knife switch arm is between T-o and T-c, judging that the knife switch in the initial frame image is in a closed state.
9. The method for detecting the status of a scissor-type switch according to claim 5, wherein the real-time status of the switch in the current frame image is determined by:
giving two angle thresholds T-o and T-c of an opening state and a closing state of the scissor type knife switch according to user requirements, and setting a high threshold hT-o and a low threshold lT-o of the opening state, wherein the T-o is between the high threshold hT-o and the low threshold lT-o, and hT-o is greater than T-o; setting a high threshold hT-c and a low threshold lT-c of a closed state, wherein T-c is between the high threshold hT-c and the low threshold lT-c, hT-c > T-c > lT-c, and lT-o > hT-c;
if the correction included angle between the upper knife switch arm and the lower knife switch arm is larger than the hT-o, the knife switch in the current frame image is judged to be in an open state, if the correction included angle between the left knife switch arm and the right knife switch arm is smaller than the lT-c, the knife switch in the current frame image is judged to be in a closed state, and if the correction included angle between the left knife switch arm and the right knife switch arm is between the hT-c and the lT-o, the knife switch in the current frame image is judged to be in a virtual closed state; and if the correction included angle between the left knife switch arm and the right knife switch arm is between lT-o and hT-o or between lT-c and hT-c, judging that the state of the knife switch in the current frame image is the same as the state of the knife switch in the previous frame image.
10. The method for detecting the status of a scissor-type knife switch according to claim 5, wherein the specific way to verify the final status of the knife switch movement in the video until the knife switch movement is detected is: judging the real-time state of the knife switch in each continuous frame of image and recording the real-time state of the knife switch; giving a threshold value of the continuous frame number, if the state of the disconnecting link in the continuous frame images is kept unchanged and the continuous frame number is greater than the threshold value of the continuous frame number, indicating that the disconnecting link maintains the same state, and detecting that the disconnecting link stops moving;
if the state of the disconnecting link in the first frame of image is an open state, the operation is finished if the final state is verified to be a closed state, and if the final state is verified to be a virtual closed state, an alarm is given and staff is asked to confirm and process manually; if the state of the disconnecting link is a closed state, the operation is finished if the final state is verified to be an open state, and if the final state is verified to be a virtual closed state, the operation is prompted by an alarm and the manual confirmation and processing of workers are requested; if the state of the disconnecting link is a virtual closing state, the operation is finished when the final state is verified to be a closed state or an open state, and if the final state is verified to be the virtual closing state, the operation is prompted by an alarm and the worker is requested to confirm and process the operation manually.
CN201910895598.8A 2019-09-21 2019-09-21 Method for detecting state of scissor type knife switch by real-time tracking Pending CN110717932A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910895598.8A CN110717932A (en) 2019-09-21 2019-09-21 Method for detecting state of scissor type knife switch by real-time tracking

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910895598.8A CN110717932A (en) 2019-09-21 2019-09-21 Method for detecting state of scissor type knife switch by real-time tracking

Publications (1)

Publication Number Publication Date
CN110717932A true CN110717932A (en) 2020-01-21

Family

ID=69210767

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910895598.8A Pending CN110717932A (en) 2019-09-21 2019-09-21 Method for detecting state of scissor type knife switch by real-time tracking

Country Status (1)

Country Link
CN (1) CN110717932A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112819889A (en) * 2020-12-30 2021-05-18 浙江大华技术股份有限公司 Method and device for determining position information, storage medium and electronic device
CN114037960A (en) * 2022-01-11 2022-02-11 合肥金星智控科技股份有限公司 Flap valve state identification method and system based on machine vision
CN112819889B (en) * 2020-12-30 2024-05-10 浙江大华技术股份有限公司 Method and device for determining position information, storage medium and electronic device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102622615A (en) * 2012-02-24 2012-08-01 山东鲁能智能技术有限公司 Knife switch state closing reliability judging method based on distance between knife switch arm feature points
WO2016034008A1 (en) * 2014-09-04 2016-03-10 华为技术有限公司 Target tracking method and device
CN107944396A (en) * 2017-11-27 2018-04-20 国网安徽省电力有限公司经济技术研究院 A kind of disconnecting link state identification method based on improvement deep learning
CN109063764A (en) * 2018-07-26 2018-12-21 福建和盛高科技产业有限公司 A kind of judgment method of disconnecting switch closing operation in place based on machine vision
CN109406999A (en) * 2017-08-16 2019-03-01 云南电网有限责任公司保山供电局 Disconnecting link condition detection method and device
WO2019101220A1 (en) * 2017-12-11 2019-05-31 珠海大横琴科技发展有限公司 Deep learning network and average drift-based automatic vessel tracking method and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102622615A (en) * 2012-02-24 2012-08-01 山东鲁能智能技术有限公司 Knife switch state closing reliability judging method based on distance between knife switch arm feature points
WO2016034008A1 (en) * 2014-09-04 2016-03-10 华为技术有限公司 Target tracking method and device
CN109406999A (en) * 2017-08-16 2019-03-01 云南电网有限责任公司保山供电局 Disconnecting link condition detection method and device
CN107944396A (en) * 2017-11-27 2018-04-20 国网安徽省电力有限公司经济技术研究院 A kind of disconnecting link state identification method based on improvement deep learning
WO2019101220A1 (en) * 2017-12-11 2019-05-31 珠海大横琴科技发展有限公司 Deep learning network and average drift-based automatic vessel tracking method and system
CN109063764A (en) * 2018-07-26 2018-12-21 福建和盛高科技产业有限公司 A kind of judgment method of disconnecting switch closing operation in place based on machine vision

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
马啸川;李庆武;刘静;钱荣;: "变电站机器人自动巡检中的刀闸开合状态分析", 电子测量与仪器学报, no. 06, 15 June 2018 (2018-06-15) *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112819889A (en) * 2020-12-30 2021-05-18 浙江大华技术股份有限公司 Method and device for determining position information, storage medium and electronic device
CN112819889B (en) * 2020-12-30 2024-05-10 浙江大华技术股份有限公司 Method and device for determining position information, storage medium and electronic device
CN114037960A (en) * 2022-01-11 2022-02-11 合肥金星智控科技股份有限公司 Flap valve state identification method and system based on machine vision

Similar Documents

Publication Publication Date Title
CN110705571A (en) Scissor type knife switch opening and closing state detection method
CN107813310B (en) Multi-gesture robot control method based on binocular vision
CN105760826B (en) Face tracking method and device and intelligent terminal
CN110942443A (en) Real-time detection method for split type disconnecting link state
CN110032992A (en) A kind of detection method that cheats at one's exam based on posture
CN111191641B (en) Accurate detection method for motion state of split type disconnecting link
CN106650697B (en) A kind of meter dial recognition methods
CN107729853B (en) Automatic identification method suitable for narrow-scale pointer instrument of transformer substation
CN112017173B (en) Power equipment defect detection method based on target detection network and structured positioning
CN110378325B (en) Target pose identification method in robot grabbing process
JP6444283B2 (en) Posture determination device
CN104423569A (en) Pointing position detecting device, method and computer readable recording medium
CN107862713B (en) Camera deflection real-time detection early warning method and module for polling meeting place
CN110689528A (en) Opening and closing angle detection method for split type disconnecting link based on automatic positioning
CN104949658B (en) Wire ice-shedding trajectory measurement device based on technique of binocular stereoscopic vision
CN106595593B (en) Suspension insulator windage yaw monitoring method based on oval feature fitting
CN110717932A (en) Method for detecting state of scissor type knife switch by real-time tracking
CN111950440A (en) Method, device and storage medium for identifying and positioning door
Bei et al. Sitting posture detection using adaptively fused 3D features
CN110599446B (en) Method for judging switching-on position of isolating switch
CN110717409A (en) Real-time accurate detection method for split type disconnecting link state
CN114724042A (en) Automatic detection method for zero-value insulator in power transmission line
CN110675452A (en) Real-time tracking split type disconnecting link state detection method
CN105987665B (en) Early warning monitoring device and method for sag point change range of ultra-high voltage transmission line
CN111008951A (en) Method for calculating opening and closing angle of split type disconnecting link based on positioning identification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination