CN110717932B - Real-time tracking detection method for scissor type knife switch state - Google Patents

Real-time tracking detection method for scissor type knife switch state Download PDF

Info

Publication number
CN110717932B
CN110717932B CN201910895598.8A CN201910895598A CN110717932B CN 110717932 B CN110717932 B CN 110717932B CN 201910895598 A CN201910895598 A CN 201910895598A CN 110717932 B CN110717932 B CN 110717932B
Authority
CN
China
Prior art keywords
knife switch
frame image
state
disconnecting link
arm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910895598.8A
Other languages
Chinese (zh)
Other versions
CN110717932A (en
Inventor
任大明
汪辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanxi Jinzhi Hongyang Technology Co ltd
Original Assignee
Shanxi Jinzhi Hongyang Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanxi Jinzhi Hongyang Technology Co ltd filed Critical Shanxi Jinzhi Hongyang Technology Co ltd
Priority to CN201910895598.8A priority Critical patent/CN110717932B/en
Publication of CN110717932A publication Critical patent/CN110717932A/en
Application granted granted Critical
Publication of CN110717932B publication Critical patent/CN110717932B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
    • G01R31/327—Testing of circuit interrupters, switches or circuit-breakers
    • G01R31/3271—Testing of circuit interrupters, switches or circuit-breakers of high voltage or medium voltage devices
    • G01R31/3275—Fault detection or status indication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a detection method for a real-time tracking scissor type disconnecting link state, which comprises the steps of collecting a real-time monitoring video, establishing a model file, and obtaining a training model for marking disconnecting link arms and the joint points of the disconnecting link arms; automatically positioning a knife switch arm in a first frame image by using a training model and a model file, acquiring a knife switch estimated state, sequentially detecting until the knife switch estimated state in one frame image is closed or virtually closed, marking the frame image as an initial frame image, calculating an angular bisector of a closing point of the knife switch arm and the knife switch arm, and simultaneously extracting and recording characteristic points; tracking and positioning the joint points of the knife switch arms of the current frame image by utilizing the characteristic points of the previous frame image, calculating the angle bisector of the knife switch arms in the current frame image, calculating the included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state; and detecting that the disconnecting link stops moving, and verifying the final state of the disconnecting link moving in the video. The invention solves the problem of low real-time detection speed of the scissor type knife switch and improves the detection accuracy.

Description

Real-time tracking detection method for scissor type knife switch state
Technical Field
The invention relates to the technical field of image recognition, in particular to a detection method for a real-time tracking scissor type knife switch state.
Background
The disconnecting link is an electric appliance which is used frequently in the high-voltage switch electric appliance and plays an isolating role in a circuit. In practical application, due to the long-term operation of the disconnecting link, the situation that the disconnecting link is not opened or closed in place can occur, and the situation can cause electric arcs to be generated between the upper disconnecting link arm and the lower disconnecting link arm of the disconnecting link. Arcing is a gas discharge phenomenon, and instantaneous sparks generated by current passing through certain insulating media can cause fires or threaten personal safety; in addition, the arc temperature is extremely high, so that insulating materials are easy to burn out, and leakage events or damage to knife switch equipment are caused; therefore, the closing state of the knife switch needs to be accurately detected.
Detecting whether the scissor type knife switch is opened or closed in place or not, wherein manual observation is mainly needed in practice; in the prior art, the opening and closing degree of the disconnecting link is judged by calculating the angle between the upper disconnecting link arm and the lower disconnecting link arm. At present, the angle between the upper knife switch arm and the lower knife switch arm is calculated, and a plurality of calculation schemes appear at home and abroad, one is to change the inside of a circuit and add a sensor or a signal lamp; in the method based on digital image processing, the detection precision is reduced because the background is complex and the knife switch is difficult to match correctly. In addition, because of the influence of shooting environment, the edge of a knife switch arm in an image is fuzzy, the detection effect is influenced, and the angle between the upper knife switch arm and the lower knife switch arm cannot be accurately detected.
Disclosure of Invention
The invention provides a real-time tracking detection method for the state of a scissor type knife switch, which aims to solve the problems that whether the scissor type knife switch is opened or closed in place or not and the real-time detection speed is low in the prior art.
The technical scheme of the invention is realized as follows:
a real-time tracking detection method for the state of a scissor type disconnecting link acquires a real-time monitoring video of the scissor type disconnecting link, acquires any frame image of the real-time video, establishes a boundary line model of a disconnecting link arm of the model disconnecting link according to the frame image, and stores the boundary line model as a model file;
acquiring sample images of the scissor type disconnecting link through various ways, and training the sample images by using a deep learning algorithm to acquire training models marking the disconnecting link arms and the joint points of the disconnecting link arms;
Sequentially detecting from a first frame image of a video, automatically positioning a knife switch arm in the first frame image by using a training model and a model file, acquiring a knife switch estimation state, directly outputting the knife switch state to be opened and continuously detecting a second frame image if the knife switch estimation state in the first frame image is opened, until the knife switch estimation state in one frame image is detected to be closed or virtually closed, and marking the frame image as an initial frame image of a detection angle; acquiring a knife switch arm joint point rectangular frame in an initial frame image according to a model file and a training model, calculating a joint point of a knife switch arm in the initial frame image and an angular bisector of the knife switch arm, and simultaneously extracting and recording characteristic points which are beneficial to tracking in the knife switch arm joint point rectangular frame in the initial frame image; according to the positioning result of the knife switch arm in the initial frame image, carrying out edge detection on the knife switch in the frame image, determining the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm in the initial frame image, further calculating the included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state of the knife switch in the initial frame image;
Tracking and positioning the joint points of the disconnecting link arms in the current frame image by utilizing the characteristic points of the previous frame image, calculating the angle bisectors of the disconnecting link arms in the current frame image according to the tracking and positioning results, automatically positioning the disconnecting link in the current frame image, carrying out edge detection according to the positioning results, determining the final edge line of the upper disconnecting link arm and the edge line of the lower disconnecting link arm in the current frame image, calculating the included angle between the upper disconnecting link arm and the lower disconnecting link arm, and judging the real-time state of the disconnecting link in the current frame image;
Sequentially processing the next frame of image in the same way as the current frame of image, and if the movement direction of the knife switch is from open to closed, verifying the final state of the knife switch movement in the video until the knife switch is detected to stop moving; if the movement direction of the disconnecting link is from closed to open, when the disconnecting link state is detected to be changed into open, the disconnecting link opening state is directly output, the angle between disconnecting link arms does not need to be calculated until the disconnecting link is detected to stop moving, and the final state of the disconnecting link movement in the video is verified.
Preferably, the method of acquiring any frame image of the real-time video and establishing a boundary line model of the knife switch arm of the knife switch according to the frame image and storing the boundary line model as a model file is as follows: any frame image of a video is acquired, a program is set on a computer, the frame image is opened by the program, the left and right boundaries and the upper and lower boundaries of upper and lower knife switch arms of a knife switch are traced, a quadrilateral template is formed, the upper and lower knife switch arms are arranged in the quadrilateral template, and four end point coordinates of the quadrilateral template are stored as model files; and if the knife switch in the frame image is in a closed state, tracing the edge according to the actual position of the knife switch arm, and if the knife switch in the frame image is in an opened or virtual closing state, estimating the position of the knife switch arm in the closed state according to the upper and lower fixed boundaries of the knife switch, and tracing the position of the knife switch arm in the closed state.
Preferably, the specific way for automatically positioning the knife switch arm in the first frame image and obtaining the estimated state of the knife switch by using the training model and the model file is as follows:
Detecting a current frame image by using a training model, and acquiring all knife gate areas and knife gate estimation states corresponding to each knife gate area in the current frame image, wherein the areas corresponding to the knife gates in the 'closed' and 'virtual' states are integral rectangular frame areas comprising upper and lower knife gate arms and upper and lower endpoints, and the area corresponding to the knife gate in the 'open' state is a folding rectangular frame area comprising the upper and lower knife gate arms and lower endpoints;
Acquiring a closed quadrilateral frame area, an upper midpoint and a lower midpoint of a disconnecting link in a closed state according to a model file, wherein the upper midpoint is a midpoint of an upper boundary of the closed quadrilateral frame, the lower midpoint is a midpoint of a lower boundary of the closed quadrilateral frame, and calculating overlapping areas of the whole rectangular frame area and the folded rectangular frame area with the closed quadrilateral frame area respectively;
traversing the areas where all the knife switches in the 'closed' and 'virtual' states in the current frame image, giving an overlapping area threshold value, and obtaining an area which comprises an upper midpoint and a lower midpoint and has the largest overlapping area and is larger than the overlapping area threshold value, wherein the area is the final integral area of the upper and lower knife switch arms, so that the knife switch positioning in the 'closed' or 'virtual' state is realized;
If the region comprising the upper midpoint and the lower midpoint and having the largest overlapping area and being larger than the overlapping area threshold value does not exist, the fact that the current state of the disconnecting link to be detected is not the closed state or the virtual state is indicated, the disconnecting link positioning is realized from all disconnecting links in the open state, the region where the disconnecting links in all the open states are located in the target image is traversed, the region comprising the lower midpoint and having the largest overlapping area and being larger than the overlapping area threshold value is obtained, and if the region exists, the region is the disconnecting link arm region in the final open state, so that the disconnecting link positioning in the open state is realized.
Preferably, according to the positioning result of the knife switch arm in the initial frame image, edge detection is performed on the knife switch in the frame image, and the specific mode of determining the final upper knife switch arm edge line and lower knife switch arm edge line in the initial frame image is as follows:
Edge detection is carried out on the initial frame image according to the knife switch arm positioning result, an edge line set is obtained, the distance from a knife switch arm joint point of the current frame image to each edge line in the edge line set is calculated, an upper distance threshold value and a lower distance threshold value are given, the edge line corresponding to the distance which is not in the threshold value range is removed, and an updated edge line set is obtained;
and (3) distinguishing the updated edge line set up and down by utilizing the angular bisector, respectively acquiring an upper disconnecting link arm edge line set and a lower disconnecting link arm edge line set, and symmetrically pairing to determine final upper disconnecting link arm edge lines and lower disconnecting link arm edge lines.
Preferably, a rectangular frame of the articulation point of the knife switch arm in the initial frame image is obtained according to the model file and the training model, and the concrete mode for calculating the articulation point of the knife switch arm in the initial frame image is as follows: calculating a center point of a quadrilateral template according to a model file, setting an estimated threshold R, and corresponding the center point to an initial frame image, wherein the initial frame image is centered on the center point, the width of a knife switch arm is wide, and the length of R times of the width of the knife switch arm is long, so that an estimated gate point rectangular frame containing a gate point of the knife switch arm in a closed state is established in the initial frame image; and obtaining all the corrected joint point rectangular frames which are positioned in the initial frame image and contain the marked joint point positions according to the training model, and calculating a corrected joint point rectangular frame with the largest overlapping area with the estimated joint point rectangular frame in the corrected joint point rectangular frame, wherein the corrected joint point rectangular frame is a knife gate arm joint point rectangular frame in the initial frame image, and the center of the corrected joint point rectangular frame is a knife gate arm joint point.
Preferably, the specific way of calculating the angular bisector of the knife switch arm in the initial frame image is as follows: obtaining a distortion coefficient of a camera for acquiring real-time video, and calculating a camera matrix = [ focal_length,0, center. X according to an initial frame image size; 0, focal_length, center. 0,1], wherein focal_length is the width of the initial frame image, and center. X and center. Y are the x coordinate and y coordinate of the midpoint of the initial frame image, respectively;
Estimating world coordinates corresponding to the four endpoint coordinates in the model file according to the four endpoint coordinates, the world coordinates, the camera matrix and the distortion coefficient in the model file, acquiring a rotation matrix and a translation matrix of a coordinate system where the camera is located relative to the world coordinate system, and acquiring a transformation matrix according to the rotation matrix and the translation matrix;
acquiring joint point coordinates of the joint points of the disconnecting link arm in an ideal image according to the transformation matrix, and obtaining an angular bisector which passes through the joint points of the disconnecting link arm of the ideal image and is parallel to the ground;
And calculating the corresponding coordinates of any point on the angular bisector of the ideal image in the initial frame image through a transformation matrix, wherein the connecting line of the corresponding coordinates and the knife switch arm joint point is the knife switch arm angular bisector of the initial frame image.
Preferably, the specific way of tracking and positioning the joint point of the knife switch arm in the current frame image by utilizing the characteristic point of the previous frame image is as follows: characteristic points in a previous frame image are positioned in a joint point rectangular frame when a knife switch arm joint point in the previous frame image is positioned, the characteristic points in the previous frame image are corresponding to the current frame image, the positions of the characteristic points in the current frame image are obtained, and the characteristic points in the previous frame image are corresponding to tracking points in the current frame image; and acquiring a minimum rectangular frame containing all tracking points in the current frame image according to the tracking points, namely, a joint point rectangular frame in the current frame image, wherein the middle point of the joint point rectangular frame in the current frame image is a knife switch arm joint point, so that the knife switch arm joint point tracking and positioning are realized.
Preferably, the specific way of judging the real-time state in the initial frame image is as follows: two angle thresholds T-o and T-c of the scissor type knife switch in an 'open' state and a 'closed' state are given according to the user requirement;
If the included angle between the upper knife switch arm and the lower knife switch arm is larger than T-o, judging that the knife switch in the initial frame image is in an open state, if the actual included angle between the upper knife switch arm and the lower knife switch arm is smaller than T-c, judging that the knife switch in the initial frame image is in a closed state, and if the actual included angle between the upper knife switch arm and the lower knife switch arm is between T-o and T-c, judging that the knife switch in the initial frame image is in a virtual closing state.
Preferably, the specific mode for judging the real-time state of the knife switch in the current frame image is as follows:
According to the user requirement, two angle threshold values T-o and T-c of a scissor type knife switch in an 'open' state and a 'closed' state are given, a high threshold value hT-o and a low threshold value lT-o of the open state are set, the T-o is between the high threshold value hT-o and the low threshold value lT-o, and the hT-o is more than T-o and more than lT-o; setting a high threshold hT-c and a low threshold lT-c of the closed state, T-c being between the high threshold hT-c and the low threshold lT-c, hT-c > T-c > lT-c, and lT-o > hT-c;
If the correction included angle between the upper knife switch arm and the lower knife switch arm is larger than hT-o, judging that the knife switch in the current frame image is in an open state, if the correction included angle between the left knife switch arm and the right knife switch arm is smaller than lT-c, judging that the knife switch in the current frame image is in a closed state, and if the correction included angle between the left knife switch arm and the right knife switch arm is between hT-c and lT-o, judging that the knife switch in the current frame image is in a virtual closing state; if the correction included angle between the left knife switch arm and the right knife switch arm is between lT-o and hT-o or between lT-c and hT-c, judging that the state of the knife switch in the current frame image is the same as the state of the knife switch in the previous frame image.
Preferably, until the stop of the movement of the knife switch is detected, the specific way of verifying the final state of the movement of the knife switch in the video is as follows: judging the real-time state of the knife switch in each continuous frame of image and then recording the real-time state; setting a continuous frame number threshold, if the state of the disconnecting link in the continuous frame images is kept unchanged and the continuous frame number is greater than the continuous frame number threshold, indicating that the disconnecting link maintains the same state, and detecting that the disconnecting link stops moving;
If the state of the knife switch in the first frame image is an open state, the operation is completed when the final state is verified to be a closed state, and if the final state is verified to be a virtual state, an alarm is given and a worker is asked to confirm and process manually; if the state of the disconnecting link is a closed state, the operation is completed when the disconnecting link is in an open state in the final state, and the disconnecting link is in a virtual state in the final state, an alarm is given and a worker is asked to confirm and process manually; if the state of the disconnecting link is the virtual closing state, the operation is completed when the disconnecting link is in the closing or opening state in the final state, and the disconnecting link is in the virtual closing state in the final state, an alarm is given and the staff is asked to confirm and process manually.
The beneficial effects of the invention are as follows: according to the real-time tracking detection method for the state of the scissor type disconnecting link, the quadrilateral template of the disconnecting link arm in the closed state in the model file is corresponding to each frame of image of the real-time monitoring video to be used as a reference, so that each frame of image of the real-time monitoring video can be conveniently processed subsequently. According to the method, firstly, the position of a knife gate arm to be detected and the corresponding knife gate state in each frame of image are obtained according to a training model and a model file, if the knife gate state is opened, the state is directly output, the angle between the knife gate arms is not calculated, if the knife gate state is closed or virtually closed, the angle between the knife gate arms is calculated, the real-time state of the knife gate is accurately analyzed according to the user requirement and the angle between the knife gate arms, meanwhile, the frame image is marked as an initial frame image, the training model and the model file are utilized to obtain a knife gate joint point rectangular frame of the initial frame image, the characteristic points which are beneficial to tracking in the knife gate joint point rectangular frame of the initial frame image are extracted, and when the subsequent frame image is processed, the knife gate joint point is positioned through real-time tracking of the characteristic points, so that the real-time detection speed is improved.
Calculating a knife switch arm angle bisector according to the knife switch arm joint points, and acquiring an upper knife switch arm edge line set and a lower knife switch arm edge line set by utilizing the knife switch arm angle bisector, so that the accuracy is improved; then, the edge line of the upper disconnecting link arm and the edge line of the lower disconnecting link arm are determined in a pairing mode; and finally, calculating the included angle between the upper knife switch arm and the lower knife switch arm, judging the real-time state of the knife switch according to the included angle, verifying the final state of the knife switch motion in the video, improving the real-time detection precision, and having important practical application value.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.
FIG. 1 is an image of a build model file of the present invention;
FIG. 2 is a schematic diagram of a structure for calculating a knife switch arm articulation point;
fig. 3 is a schematic diagram of a structure for distinguishing the update edge line set up and down by using an angular bisector.
In the figure:
1. An upper knife switch arm; 2. a lower knife switch arm; 3. a knife switch arm joint point; 4. a center point; 5. estimating a rectangular frame of the node; 6. correcting the rectangular frame of the joint point; 7. angular bisector.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1: as shown in fig. 1, a method for detecting the state of a scissor type knife switch in real time tracking is provided, a real-time monitoring video of the scissor type knife switch is collected, any frame image of the real-time video is obtained, and a boundary line model of a knife switch arm of the model knife switch is built according to the frame image and is stored as a model file; acquiring sample images of the scissor type disconnecting link through various ways, and training the sample images by using a deep learning algorithm to acquire training models marking the disconnecting link arms and the joint points of the disconnecting link arms; sequentially detecting from a first frame image of a video, automatically positioning a knife switch arm in the first frame image by using a training model and a model file, acquiring a knife switch estimation state, directly outputting the knife switch state to be opened and continuously detecting a second frame image if the knife switch estimation state in the first frame image is opened, until the knife switch estimation state in one frame image is detected to be closed or virtually closed, and marking the frame image as an initial frame image of a detection angle; acquiring a knife switch arm joint point rectangular frame in an initial frame image according to a model file and a training model, calculating a joint point of a knife switch arm in the initial frame image and an angular bisector of the knife switch arm, and simultaneously extracting and recording characteristic points which are beneficial to tracking in the knife switch arm joint point rectangular frame in the initial frame image; according to the positioning result of the knife switch arm in the initial frame image, carrying out edge detection on the knife switch in the frame image, determining the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm in the initial frame image, further calculating the included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state of the knife switch in the initial frame image; tracking and positioning the joint points of the disconnecting link arms in the current frame image by utilizing the characteristic points of the previous frame image, calculating the angle bisector of the disconnecting link arms in the current frame image according to the tracking and positioning results, automatically positioning the disconnecting link in the current frame image, carrying out edge detection according to the positioning results, determining the final edge line of the upper disconnecting link arm and the edge line of the lower disconnecting link arm in the current frame image, calculating the included angle between the upper disconnecting link arm and the lower disconnecting link arm, and judging the real-time state of the disconnecting link in the current frame image; if the movement direction of the disconnecting link is from open to closed, stopping movement of the disconnecting link until the disconnecting link is detected, and verifying the final state of movement of the disconnecting link in the video; if the movement direction of the disconnecting link is from closed to open, when the disconnecting link state is detected to be changed into open, the disconnecting link opening state is directly output, the angle between disconnecting link arms does not need to be calculated until the disconnecting link is detected to stop moving, and the final state of the disconnecting link movement in the video is verified.
If the state of the knife switch in the first frame of video image is open, and the state of the knife switch is virtual or closed after being changed, the moving direction of the knife switch is from open to closed; if the state of the knife switch in the first frame of video image is closed and the state of the knife switch is virtual or opened after being changed, the moving direction of the knife switch is from closed to open. The real-time state of the knife switch in the initial frame image is the output state.
When each frame of image of the video is detected in sequence, the training model and the model file are utilized to automatically position the knife switch arms in the detected frame of image and acquire the knife switch state, wherein the acquired knife switch state is a general state, and the real-time state acquired later is the knife switch state accurately analyzed according to the angle between the knife switch arms. If the movement direction of the knife switch is from open to close, the knife switch state in the first frame of image is open, and the open state is detected according to the user requirement, so that the angle between the knife switch arms does not need to be accurately calculated. Under the condition that an initial frame image in the real-time video is detected to be in a closed state or a virtual closed state, the specific state of the knife switch is required to be determined specifically according to the included angle between the upper knife switch arm and the lower knife switch arm, and the real-time state of the scissor type knife switch is accurately detected. If the movement direction of the knife switch is from closed to open, the knife switch state in the first frame image is closed, the first frame image is an initial frame image for calculating the angle, the specific state of the knife switch is required to be determined specifically according to the included angle between the upper knife switch arm and the lower knife switch arm, the real-time state of the scissor type knife switch is accurately detected until the knife switch state is changed into open, the angle is not required to be accurately calculated, and the open state is directly output.
In embodiment 1, the manner of "acquiring any frame image of a real-time video and establishing a boundary line model of a knife switch arm of the type of knife switch according to the frame image and storing the boundary line model as a model file" is as follows: any frame image of a video is acquired, a program is set on a computer, the frame image is opened by the program, the left and right boundaries and the upper and lower boundaries of upper and lower knife switch arms of a knife switch are traced, a quadrilateral template is formed, the upper and lower knife switch arms are arranged in the quadrilateral template, and four end point coordinates of the quadrilateral template are stored as model files; and if the knife switch in the frame image is in a closed state, tracing the edge according to the actual position of the knife switch arm, and if the knife switch in the frame image is in an opened or virtual closing state, estimating the position of the knife switch arm in the closed state according to the upper and lower fixed boundaries of the knife switch, and tracing the position of the knife switch arm in the closed state. Therefore, the model file can be determined from any frame image of the real-time video.
In embodiment 1, the specific way to automatically position the knife switch arm in the first frame image and obtain the estimated state of the knife switch by using the training model and the model file is as follows: detecting a current frame image by using a training model, and acquiring all knife gate areas and knife gate estimation states corresponding to each knife gate area in the current frame image, wherein the areas corresponding to the knife gates in the 'closed' and 'virtual' states are integral rectangular frame areas comprising upper and lower knife gate arms and upper and lower endpoints, and the area corresponding to the knife gate in the 'open' state is a folding rectangular frame area comprising the upper and lower knife gate arms and lower endpoints; acquiring a closed quadrilateral frame area, an upper midpoint and a lower midpoint of a disconnecting link in a closed state according to a model file, wherein the upper midpoint is a midpoint of an upper boundary of the closed quadrilateral frame, the lower midpoint is a midpoint of a lower boundary of the closed quadrilateral frame, and calculating overlapping areas of the whole rectangular frame area and the folded rectangular frame area with the closed quadrilateral frame area respectively; traversing the areas where all the knife switches in the 'closed' and 'virtual' states are located in the current frame image, giving an overlapping area threshold value, and acquiring an area which comprises an upper midpoint and a lower midpoint and has the largest overlapping area, wherein the area is the final integral area of the upper and lower knife switch arms, so that the knife switch positioning in the 'closed' or 'virtual' state is realized; if the region comprising the upper midpoint and the lower midpoint and having the largest overlapping area and being larger than the overlapping area threshold value does not exist, the fact that the current state of the disconnecting link to be detected is not the closed state or the virtual state is indicated, the disconnecting link positioning is realized from all disconnecting links in the open state, the region where the disconnecting links in all the open states are located in the target image is traversed, the region comprising the lower midpoint and having the largest overlapping area and being larger than the overlapping area threshold value is obtained, and if the region exists, the region is the disconnecting link arm region in the final open state, so that the disconnecting link positioning in the open state is realized. And acquiring the estimation state of the disconnecting link in each frame of image by using the training model and the model file.
As shown in fig. 2, according to the model file and the training model, a rectangular frame of the articulation point of the knife switch arm in the initial frame image is obtained, and the specific way of calculating the articulation point of the knife switch arm in the initial frame image is as follows: calculating a center point of a quadrilateral template according to a model file, setting an estimated threshold R, corresponding the center point to an initial frame image, and establishing an estimated node rectangular frame 5 containing a switch arm node in a closed state in the initial frame image, wherein the width of a switch arm is wide and the length of R times of the width of the switch arm is long by taking the center point 4 as the center in the initial frame image; and acquiring all the corrected joint point rectangular frames 6 which are positioned in the initial frame image and contain the marked joint point positions according to the training model, and calculating the corrected joint point rectangular frame 6 with the largest overlapping area with the estimated joint point rectangular frame in the corrected joint point rectangular frame, wherein the corrected joint point rectangular frame is a knife gate arm joint point rectangular frame in the initial frame image, and the center of the corrected joint point rectangular frame is a knife gate arm joint point.
The mode of calculating the angle bisector of the knife switch arm in the initial frame image is as follows: obtaining a distortion coefficient of a camera for acquiring real-time video, and calculating a camera matrix = [ focal_length,0, center. X according to an initial frame image size; 0, focal_length, center. 0,1], wherein focal_length is the width of the initial frame image, and center. X and center. Y are the x coordinate and y coordinate of the midpoint of the initial frame image, respectively; estimating corresponding world coordinates according to four end point coordinates in a model file, and obtaining a rotation matrix and a translation matrix of a coordinate system of the camera relative to the world coordinate system according to the four end point coordinates, the world coordinates, a camera matrix and distortion coefficients in the model file by using a camera posture estimation method proposed in reference Pose Estimation based on Four Coplanar Point Correspondences, obtaining a transformation matrix according to the rotation matrix and the translation matrix, setting the transformation matrix as R_matrix, setting a knife arm joint point as artis _p, obtaining joint point coordinates artis _q= (artis _ q.x, artis _ q.y) of the knife arm joint point in an ideal image according to the transformation matrix, and obtaining an angular bisector y= (artis _ q.y/artis _ q.x x) of the passing joint point of the ideal image and parallel to the ground; and calculating the corresponding coordinates of any point angle_p on the angular bisector of the ideal image in the initial frame image through a transformation matrix, wherein the connecting line of the corresponding coordinates angle_q and the knife-blade arm joint point artis _p is the angular bisector 7 of the initial frame image.
The ideal image theoretically refers to the image of the knife switch, which is taken by the camera perpendicular to the knife switch, with the center point of the camera on the same straight line as the articulation point of the knife switch arm. In actual shooting, a disconnecting link or a camera generally has a certain degree of position offset, and a disconnecting link arm joint point of an ideal image obtained by using a disconnecting link arm joint point and a transformation matrix has some errors with a disconnecting link arm joint point of a theoretical ideal image, and the errors do not affect the calculation of an angular bisector.
According to the positioning result of the knife switch arm in the initial frame image, carrying out edge detection on the knife switch in the frame image, and determining the final specific modes of the edge line of the upper knife switch arm and the edge line of the lower knife switch arm in the initial frame image are as follows: edge detection is carried out on the initial frame image according to the knife switch arm positioning result, an edge line set is obtained, the distance from a knife switch arm joint point of the initial frame image to each edge line in the edge line set is calculated, an upper distance threshold value and a lower distance threshold value are given, the edge line corresponding to the distance which is not in the threshold value range is removed, and an updated edge line set is obtained; and (3) distinguishing the updated edge line set up and down by utilizing the angular bisector, respectively acquiring an upper disconnecting link arm edge line set and a lower disconnecting link arm edge line set, and symmetrically pairing to determine final upper disconnecting link arm edge lines and lower disconnecting link arm edge lines.
When an upper disconnecting link arm edge line and a lower disconnecting link arm edge line in an initial frame image are determined, the updated edge line set is distinguished up and down by utilizing an angular bisector, and the specific modes for respectively acquiring the upper disconnecting link arm edge line set and the lower disconnecting link arm edge line set are as follows: calculating a unit direction vector which is perpendicular to an angular bisector and has an upward direction or a downward direction with an articulation point as a starting point in the target imageTraversing edge lines in the updated edge line set, marking two end points of any one edge line as T1 and T2, and marking vectors of T1 and knife gate arm joint points as/>Let the vector of T2 and the knife switch arm joint be the/>Calculation/>AndFor unit direction vector/>, upward in directionIf/>T1 is above the bisector, otherwise below the bisector, if/>Then T2 is above the angular bisector, otherwise the angular bisector is below; for unit direction vector/>, downward in directionIf/>T1 is below the bisector, otherwise above the bisector, if/>Then T2 is on the lower side of the angular bisector, otherwise the angular bisector is on the upper side;
if T1 and T2 are both on the upper side of the angular bisector, the edge line is an upper disconnecting link arm edge line, if T1 and T2 are both on the lower side of the angular bisector, the edge line is a lower disconnecting link arm edge line, if T1 and T2 are respectively on two sides of the angular bisector, the lengths of the upper line segment and the lower line segment of the edge line, which take the angular bisector as a parting line, are calculated, the edge line is an edge line on the longer side of the upper side line segment and the lower side line segment, and an upper disconnecting link arm edge line set and a lower disconnecting link arm edge line set are obtained.
As shown in FIG. 3, n is a unit vector, T1, T2 are two end points of the edge line, P1, P2 are vectors connecting the knife-arm joint point and T1, T2, and the included angle between P1, P2 and n is an acute angle, so Therefore, the T1 and the T2 are arranged on the upper side of the angular bisector, and the edge line formed by the T1 and the T2 is the edge line of the upper disconnecting link arm.
Unit direction vectorThe specific mode of calculation is as follows: setting an angular bisector line of a knife switch arm on a target image as y=kx+b, setting coordinates of a knife switch arm joint point as (C x,Cy), and setting a linear equation of a perpendicular line which is perpendicular to the angular bisector and passes through the knife switch arm joint point as y=cx+d, c= -1/k, and C y＝-1/k*Cx+d,d＝Cy+1/k*Cx; calculating a unit direction vector/>, which takes a disconnecting link arm joint point as a starting point and is upward or downward, according to a straight line equation of a vertical lineThe end point coordinates of the unit direction vector are set to (n x,ny),
Then (ny-Cy)/(nx-Cx)＝-1/k,(ny-Cy)2+(nx-Cx)2＝1, finds n x and n y, unit direction vector
The method for symmetrically pairing the upper knife switch arm edge line set and the lower knife switch arm edge line set and determining the final upper knife switch arm edge line and the final lower knife switch arm edge line in the initial frame image is as follows: giving an angle threshold, for which the angle of any edge line in the upper disconnecting link edge line set relative to the horizontal position is marked as M h, h represents different upper disconnecting link edge lines, dividing the angle difference value of any two upper disconnecting link edge lines into the same group, and sorting the upper disconnecting link edge lines in the same group from long to short; for the angle of any edge line in the lower knife switch arm edge line set relative to the horizontal position is recorded as N m, m represents different lower knife switch arm edge lines, the angle difference value of any two lower knife switch arm edge lines is divided into the same group within the angle threshold value, and the lower knife switch arm edge lines in the same group are ordered from long to short;
Traversing all the same groups of the upper disconnecting link arm edge line sets and all the same groups of the lower disconnecting link arm edge line sets, performing one-to-one pairing, calculating any two groups of correction disconnecting link arm edge lines and pairing scores, and obtaining final upper disconnecting link arm edge lines and final lower disconnecting link arm edge lines corresponding to the highest-score combination.
The method for traversing all the same groups of upper knife arm edge line sets and all the same groups of lower knife arm edge line sets and performing one-to-one pairing is as follows: as shown in fig. 2, calculating an upper midpoint between the two end points at the top and a lower midpoint between the two end points at the bottom according to the model file, calculating an average angle of each group, and acquiring a corrected upper disconnecting link edge line of each group in the upper disconnecting link edge line set by combining the average angle of each group in the upper disconnecting link edge line set with the upper midpoint, wherein an intersection point of the corrected upper disconnecting link edge line and the angular bisector is denoted as a point a; the average angle of each group in the lower disconnecting link edge line set is combined with the lower midpoint to obtain the corrected lower disconnecting link edge line of each group in the lower disconnecting link edge line set, the intersection point of the corrected lower disconnecting link edge line and the angular bisector is marked as a point B, and the distance d AB between the point A and the point B is calculated; if the distance from the upper fixed boundary to the lower fixed boundary of the knife switch arm is d 0,dAB/d0 to be more than 0.5, removing the same group where the point A is located and the same group where the point B is located;
setting any two groups of calculation formulas of pairing scores as Wherein the coefficient S T represents the sum of squares of the lengths of any of the same group of inner edge lines of the upper knife gate arm edge line set; s B is the sum of squares of the lengths of any inner edge lines of the same group of edge lines of the lower disconnecting link arm; beta is an adjustable sensitivity parameter; the correction upper knife switch arm edge line where the point A corresponding to the highest pairing score S is located and the correction lower knife switch arm edge line where the point B is located are the final upper knife switch arm edge line and lower knife switch arm edge line.
Assuming that the number of edge lines in any same group of the upper switch arm edge line set is K, each edge line may be represented as L i, and the length thereof is LLength i according to the coordinates of the start point and the end point of the edge line, where i=0, 1,..k, then S T＝LLength0 2+LLength1 2...+LLengthK 2; assuming that the number of edge lines in any same group of the edge line set of the lower disconnecting link arm is Q, each edge line may be denoted as R j, and the length thereof is RLength j according to the coordinates of the start point and the end point of the edge line, where j=0, 1,..q, then S B＝RLength0 2+RLength1 2...+RLengthQ 2; beta is an adjustable sensitivity parameter, different scores can be obtained by taking different beta values in actual use, a pairing result is determined according to the different scores, the corresponding beta value with the best pairing result is selected as a final beta value, and different users can also determine the beta value according to actual conditions; the correction upper knife switch arm edge line where the point A corresponding to the highest pairing score S is located and the correction lower knife switch arm edge line where the point B is located are the final upper knife switch arm edge line and lower knife switch arm edge line.
The specific mode for tracking and positioning the joint points of the knife switch arm in the current frame image by utilizing the characteristic points of the previous frame image is as follows: characteristic points in a previous frame image are positioned in a joint point rectangular frame when a knife switch arm joint point in the previous frame image is positioned, the characteristic points in the previous frame image are corresponding to the current frame image, the positions of the characteristic points in the current frame image are obtained, and the characteristic points in the previous frame image are corresponding to tracking points in the current frame image; and acquiring a minimum rectangular frame containing all tracking points in the current frame image according to the tracking points, namely, a joint point rectangular frame in the current frame image, wherein the middle point of the joint point rectangular frame in the current frame image is a knife switch arm joint point, so that the knife switch arm joint point in the current frame image is tracked and positioned. The specific mode of calculating the angle bisector of the disconnecting link arm in the current frame image according to the positioning result is the same as the specific mode of calculating the angle bisector of the disconnecting link arm in the initial frame image. The specific manner of determining the final upper and lower blade edge lines in the current frame image is the same as the specific manner of determining the final upper and lower blade edge lines in the initial frame image. The processing of the subsequent image is the same processing as the processing of the current frame image. The mode of calculating the included angle between the upper knife switch arm and the lower knife switch arm is as follows: and taking the final upper knife switch arm edge line and the final lower knife switch arm edge line as references, if the angle of the final upper knife switch arm edge line relative to the straight line of the angle bisector of the closed state knife switch arm is alpha 1, the angle of the final lower knife switch arm edge line relative to the straight line of the angle bisector of the closed state knife switch arm is alpha 2, and the final included angle between the upper knife switch arm and the lower knife switch arm is theta= |alpha 2-α1 |.
The specific mode for judging the real-time state in the initial frame image is as follows: two angle thresholds T-o and T-c of the scissor type knife switch in an 'open' state and a 'closed' state are given according to the user requirement; if the included angle between the upper knife switch arm and the lower knife switch arm is larger than T-o, judging that the knife switch in the initial frame image is in an open state, if the actual included angle between the upper knife switch arm and the lower knife switch arm is smaller than T-c, judging that the knife switch in the initial frame image is in a closed state, and if the actual included angle between the upper knife switch arm and the lower knife switch arm is between T-o and T-c, judging that the knife switch in the initial frame image is in a virtual closing state. In actual use, the angle threshold is determined according to the actual use condition.
The specific mode for judging the real-time state of the knife switch in the current frame image is as follows: according to the user requirement, two angle threshold values T-o and T-c of a scissor type knife switch in an 'open' state and a 'closed' state are given, a high threshold value hT-o and a low threshold value lT-o of the open state are set, the T-o is between the high threshold value hT-o and the low threshold value lT-o, and the hT-o is more than T-o and more than lT-o; setting a high threshold hT-c and a low threshold lT-c of the closed state, T-c being between the high threshold hT-c and the low threshold lT-c, hT-c > T-c > lT-c, and lT-o > hT-c; if the correction included angle between the upper knife switch arm and the lower knife switch arm is larger than hT-o, judging that the knife switch in the current frame image is in an open state, if the correction included angle between the left knife switch arm and the right knife switch arm is smaller than lT-c, judging that the knife switch in the current frame image is in a closed state, and if the correction included angle between the left knife switch arm and the right knife switch arm is between hT-c and lT-o, judging that the knife switch in the current frame image is in a virtual closing state; if the correction included angle between the left knife switch arm and the right knife switch arm is between lT-o and hT-o or between lT-c and hT-c, judging that the state of the knife switch in the current frame image is the same as the state of the knife switch in the previous frame image.
The mode of judging the state of the knife switch in the next frame image is the same as that of judging the state of the knife switch in the current frame image.
When a model file is established and a disconnecting link estimation state is obtained according to a training model, the closing state of the disconnecting link means that contact exists between an upper disconnecting link arm and an upper fixed boundary, the upper disconnecting link arm and the lower disconnecting link arm are on the same straight line, the virtual closing state means that contact exists between the upper disconnecting link arm and the upper fixed boundary, but the upper disconnecting link arm and the lower disconnecting link arm are not on the same straight line, and the opening state means that contact does not exist between the upper disconnecting link arm and the upper fixed boundary. The state of the knife switch is general, and is different from the state of the knife switch obtained according to the included angle between the knife switch arms, and the state of the knife switch obtained according to the included angle between the knife switch arms is the final state of the knife switch in the target image. Until the stop motion of the disconnecting link is detected, the specific mode for verifying the final state of the disconnecting link motion in the video is as follows: judging the real-time state of the knife switch in each continuous frame of image and then recording the real-time state; setting a continuous frame number threshold, if the state of the disconnecting link in the continuous frame images is kept unchanged and the continuous frame number is greater than the continuous frame number threshold, indicating that the disconnecting link maintains the same state, and detecting that the disconnecting link stops moving; if the state of the knife switch in the first frame image is an open state, the operation is completed when the final state is verified to be a closed state, and if the final state is verified to be a virtual state, an alarm is given and a worker is asked to confirm and process manually; if the state of the disconnecting link is a closed state, the operation is completed when the disconnecting link is in an open state in the final state, and the disconnecting link is in a virtual state in the final state, an alarm is given and a worker is asked to confirm and process manually; if the state of the disconnecting link is the virtual closing state, the operation is completed when the disconnecting link is in the closing or opening state in the final state, and the disconnecting link is in the virtual closing state in the final state, an alarm is given and the staff is asked to confirm and process manually.
The scissors type disconnecting link refers to a vertical telescopic disconnecting link, the disconnecting link is divided into an upper disconnecting link arm and a lower disconnecting link arm by taking a middle articulation point as a reference, the disconnecting link arms utilize the articulation point to carry out telescopic opening and closing in the opening and closing process of the disconnecting link, the two disconnecting link arms are on the same straight line when being completely closed, and the two disconnecting link arms are folded together when being completely opened.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.

Claims (10)

1. A detection method of real-time tracking of the state of a scissor type disconnecting link is characterized by collecting real-time monitoring video of the scissor type disconnecting link, obtaining any frame image of the real-time video, establishing a boundary line model of a disconnecting link arm of the model disconnecting link according to the frame image and storing the boundary line model as a model file;
acquiring sample images of the scissor type disconnecting link through various ways, and training the sample images by using a deep learning algorithm to acquire training models marking the disconnecting link arms and the joint points of the disconnecting link arms;
Sequentially detecting from a first frame image of a video, automatically positioning a knife switch arm in the first frame image by using a training model and a model file, acquiring a knife switch estimation state, directly outputting the knife switch state to be opened and continuously detecting a second frame image if the knife switch estimation state in the first frame image is opened, until the knife switch estimation state in one frame image is detected to be closed or virtually closed, and marking the frame image as an initial frame image of a detection angle; acquiring a knife switch arm joint point rectangular frame in an initial frame image according to a model file and a training model, calculating a joint point of a knife switch arm in the initial frame image and an angular bisector of the knife switch arm, and simultaneously extracting and recording characteristic points which are beneficial to tracking in the knife switch arm joint point rectangular frame in the initial frame image; according to the positioning result of the knife switch arm in the initial frame image, carrying out edge detection on the knife switch in the frame image, determining the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm in the initial frame image, further calculating the included angle between the upper knife switch arm and the lower knife switch arm, and judging the real-time state of the knife switch in the initial frame image;
Tracking and positioning the joint points of the disconnecting link arms in the current frame image by utilizing the characteristic points of the previous frame image, calculating the angle bisector of the disconnecting link arms in the current frame image according to the tracking and positioning results, automatically positioning the disconnecting link in the current frame image, carrying out edge detection according to the positioning results, determining the final edge line of the upper disconnecting link arm and the edge line of the lower disconnecting link arm in the current frame image, calculating the included angle between the upper disconnecting link arm and the lower disconnecting link arm, and judging the real-time state of the disconnecting link in the current frame image;
Sequentially processing the next frame of image in the same way as the current frame of image, and if the movement direction of the knife switch is from open to closed, verifying the final state of the knife switch movement in the video until the knife switch is detected to stop moving; if the movement direction of the disconnecting link is from closed to open, when the disconnecting link state is detected to be changed into open, the disconnecting link opening state is directly output, the angle between disconnecting link arms does not need to be calculated until the disconnecting link is detected to stop moving, and the final state of the disconnecting link movement in the video is verified.
2. The method for detecting the state of a scissors type knife switch according to claim 1, wherein the method for acquiring any frame image of a real-time video and establishing a boundary line model of a knife switch arm of the knife switch according to the frame image and storing the boundary line model as a model file is as follows: any frame image of a video is acquired, a program is set on a computer, the frame image is opened by the program, the left and right boundaries and the upper and lower boundaries of upper and lower knife switch arms of a knife switch are traced, a quadrilateral template is formed, the upper and lower knife switch arms are arranged in the quadrilateral template, and four end point coordinates of the quadrilateral template are stored as model files; and if the knife switch in the frame image is in a closed state, tracing the edge according to the actual position of the knife switch arm, and if the knife switch in the frame image is in an opened or virtual closing state, estimating the position of the knife switch arm in the closed state according to the upper and lower fixed boundaries of the knife switch, and tracing the position of the knife switch arm in the closed state.
3. The method for detecting a real-time tracking scissor type knife switch state according to claim 2, wherein the specific way for automatically positioning the knife switch arm in the first frame of image and obtaining the estimated state of the knife switch by using a training model and a model file is as follows:
Detecting a current frame image by using a training model, and acquiring all knife gate areas and knife gate estimation states corresponding to each knife gate area in the current frame image, wherein the areas corresponding to the knife gates in the 'closed' and 'virtual' states are integral rectangular frame areas comprising upper and lower knife gate arms and upper and lower endpoints, and the area corresponding to the knife gate in the 'open' state is a folding rectangular frame area comprising the upper and lower knife gate arms and lower endpoints;
Acquiring a closed quadrilateral frame area, an upper midpoint and a lower midpoint of a disconnecting link in a closed state according to a model file, wherein the upper midpoint is a midpoint of an upper boundary of the closed quadrilateral frame, the lower midpoint is a midpoint of a lower boundary of the closed quadrilateral frame, and calculating overlapping areas of the whole rectangular frame area and the folded rectangular frame area with the closed quadrilateral frame area respectively;
traversing the areas where all the knife switches in the 'closed' and 'virtual' states in the current frame image, giving an overlapping area threshold value, and obtaining an area which comprises an upper midpoint and a lower midpoint and has the largest overlapping area and is larger than the overlapping area threshold value, wherein the area is the final integral area of the upper and lower knife switch arms, so that the knife switch positioning in the 'closed' or 'virtual' state is realized;
If the region comprising the upper midpoint and the lower midpoint and having the largest overlapping area and being larger than the overlapping area threshold value does not exist, the fact that the current state of the disconnecting link to be detected is not the closed state or the virtual state is indicated, the disconnecting link positioning is realized from all disconnecting links in the open state, the region where the disconnecting links in all the open states are located in the target image is traversed, the region comprising the lower midpoint and having the largest overlapping area and being larger than the overlapping area threshold value is obtained, and if the region exists, the region is the disconnecting link arm region in the final open state, so that the disconnecting link positioning in the open state is realized.
4. The method for detecting the state of a scissor type knife switch according to claim 2, wherein the specific mode of determining the final edge line of the upper knife switch arm and the final edge line of the lower knife switch arm in the initial frame image is as follows:
Edge detection is carried out on the initial frame image according to the knife switch arm positioning result, an edge line set is obtained, and the knife switch arm joint point to edge line of the current frame image is calculated
The distance of each edge line in the set is given an upper distance threshold value and a lower distance threshold value, and the edge line corresponding to the distance which is not in the threshold value range is removed to obtain an updated edge line set;
and (3) distinguishing the updated edge line set up and down by utilizing the angular bisector, respectively acquiring an upper disconnecting link arm edge line set and a lower disconnecting link arm edge line set, and symmetrically pairing to determine final upper disconnecting link arm edge lines and lower disconnecting link arm edge lines.
5. The method for detecting the state of a scissor type knife switch according to claim 2, wherein the method for calculating the articulation point of the knife switch arm in the initial frame image according to the model file and the training model is characterized in that: calculating a center point of a quadrilateral template according to a model file, setting an estimated threshold R, and corresponding the center point to an initial frame image, wherein the initial frame image is centered on the center point, the width of a knife switch arm is wide, and the length of R times of the width of the knife switch arm is long, so that an estimated gate point rectangular frame containing a gate point of the knife switch arm in a closed state is established in the initial frame image; and obtaining all the corrected joint point rectangular frames which are positioned in the initial frame image and contain the marked joint point positions according to the training model, and calculating a corrected joint point rectangular frame with the largest overlapping area with the estimated joint point rectangular frame in the corrected joint point rectangular frame, wherein the corrected joint point rectangular frame is a knife gate arm joint point rectangular frame in the initial frame image, and the center of the corrected joint point rectangular frame is a knife gate arm joint point.
6. The method for detecting a real-time tracking scissor type knife switch state according to claim 5, wherein the specific way of calculating the angle bisector of the knife switch arm in the initial frame image is as follows: obtaining a distortion coefficient of a camera for acquiring real-time video, and calculating a camera matrix = [ focal_length,0, center. X according to an initial frame image size; 0, focal_length, center. 0,1], wherein focal_length is the width of the initial frame image, and center. X and center. Y are the x coordinate and y coordinate of the midpoint of the initial frame image, respectively;
Estimating world coordinates corresponding to the four endpoint coordinates in the model file according to the four endpoint coordinates, the world coordinates, the camera matrix and the distortion coefficient in the model file, acquiring a rotation matrix and a translation matrix of a coordinate system where the camera is located relative to the world coordinate system, and acquiring a transformation matrix according to the rotation matrix and the translation matrix;
acquiring joint point coordinates of the joint points of the disconnecting link arm in an ideal image according to the transformation matrix, and obtaining an angular bisector which passes through the joint points of the disconnecting link arm of the ideal image and is parallel to the ground;
And calculating the corresponding coordinates of any point on the angular bisector of the ideal image in the initial frame image through a transformation matrix, wherein the connecting line of the corresponding coordinates and the knife switch arm joint point is the knife switch arm angular bisector of the initial frame image.
7. The method for detecting the state of a scissor type knife switch according to claim 5, wherein the specific way of tracking and positioning the joint point of the knife switch arm in the current frame image by utilizing the characteristic point of the previous frame image is as follows: characteristic points in a previous frame image are positioned in a joint point rectangular frame when a knife switch arm joint point in the previous frame image is positioned, the characteristic points in the previous frame image are corresponding to the current frame image, the positions of the characteristic points in the current frame image are obtained, and the characteristic points in the previous frame image are corresponding to tracking points in the current frame image; and acquiring a minimum rectangular frame containing all tracking points in the current frame image according to the tracking points, namely, a joint point rectangular frame in the current frame image, wherein the middle point of the joint point rectangular frame in the current frame image is a knife switch arm joint point, so that the knife switch arm joint point tracking and positioning are realized.
8. The method for detecting a real-time tracking scissors-type knife switch state according to claim 5, wherein the specific way of judging the real-time state in the initial frame image is as follows: two angle thresholds T-o and T-c of the scissor type knife switch in an 'open' state and a 'closed' state are given according to the user requirement;
If the included angle between the upper knife switch arm and the lower knife switch arm is larger than T-o, judging that the knife switch in the initial frame image is in an open state, if the actual included angle between the upper knife switch arm and the lower knife switch arm is smaller than T-c, judging that the knife switch in the initial frame image is in a closed state, and if the actual included angle between the upper knife switch arm and the lower knife switch arm is between T-o and T-c, judging that the knife switch in the initial frame image is in a virtual closing state.
9. The method for detecting the state of a scissor type knife switch according to claim 5, wherein the specific way for judging the real-time state of the knife switch in the current frame image is as follows:
According to the user requirement, two angle threshold values T-o and T-c of a scissor type knife switch in an 'open' state and a 'closed' state are given, a high threshold value hT-o and a low threshold value lT-o of the open state are set, the T-o is between the high threshold value hT-o and the low threshold value lT-o, and the hT-o is more than T-o and more than lT-o; setting a high threshold hT-c and a low threshold lT-c of the closed state, T-c being between the high threshold hT-c and the low threshold lT-c, hT-c > T-c > lT-c, and lT-o > hT-c;
If the correction included angle between the upper knife switch arm and the lower knife switch arm is larger than hT-o, judging that the knife switch in the current frame image is in an open state, if the correction included angle between the left knife switch arm and the right knife switch arm is smaller than lT-c, judging that the knife switch in the current frame image is in a closed state, and if the correction included angle between the left knife switch arm and the right knife switch arm is between hT-c and lT-o, judging that the knife switch in the current frame image is in a virtual closing state; if the correction included angle between the left knife switch arm and the right knife switch arm is between lT-o and hT-o or between lT-c and hT-c, judging that the state of the knife switch in the current frame image is the same as the state of the knife switch in the previous frame image.
10. The method for detecting the state of a scissors type knife switch in real time tracking according to claim 5, wherein the specific way of verifying the final state of the knife switch motion in the video until the knife switch is detected to stop moving is as follows: judging the real-time state of the knife switch in each continuous frame of image and then recording the real-time state; setting a continuous frame number threshold, if the state of the disconnecting link in the continuous frame images is kept unchanged and the continuous frame number is greater than the continuous frame number threshold, indicating that the disconnecting link maintains the same state, and detecting that the disconnecting link stops moving;
If the state of the knife switch in the first frame image is an open state, the operation is completed when the final state is verified to be a closed state, and if the final state is verified to be a virtual state, an alarm is given and a worker is asked to confirm and process manually; if the state of the disconnecting link is a closed state, the operation is completed when the disconnecting link is in an open state in the final state, and the disconnecting link is in a virtual state in the final state, an alarm is given and a worker is asked to confirm and process manually; if the state of the disconnecting link is the virtual closing state, the operation is completed when the disconnecting link is in the closing or opening state in the final state, and the disconnecting link is in the virtual closing state in the final state, an alarm is given and the staff is asked to confirm and process manually.
CN201910895598.8A 2019-09-21 2019-09-21 Real-time tracking detection method for scissor type knife switch state Active CN110717932B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910895598.8A CN110717932B (en) 2019-09-21 2019-09-21 Real-time tracking detection method for scissor type knife switch state

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910895598.8A CN110717932B (en) 2019-09-21 2019-09-21 Real-time tracking detection method for scissor type knife switch state

Publications (2)

Publication Number Publication Date
CN110717932A CN110717932A (en) 2020-01-21
CN110717932B true CN110717932B (en) 2024-06-14

Family

ID=69210767

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910895598.8A Active CN110717932B (en) 2019-09-21 2019-09-21 Real-time tracking detection method for scissor type knife switch state

Country Status (1)

Country Link
CN (1) CN110717932B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112819889B (en) * 2020-12-30 2024-05-10 浙江大华技术股份有限公司 Method and device for determining position information, storage medium and electronic device
CN114037960B (en) * 2022-01-11 2022-05-24 合肥金星智控科技股份有限公司 Method and system for identifying state of flap valve based on machine vision

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102622615A (en) * 2012-02-24 2012-08-01 山东鲁能智能技术有限公司 Knife switch state closing reliability judging method based on distance between knife switch arm feature points

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104376576B (en) * 2014-09-04 2018-06-05 华为技术有限公司 A kind of method for tracking target and device
CN109406999A (en) * 2017-08-16 2019-03-01 云南电网有限责任公司保山供电局 Disconnecting link condition detection method and device
CN107944396B (en) * 2017-11-27 2021-12-28 国网安徽省电力有限公司经济技术研究院 Knife switch state identification method based on improved deep learning
CN107818571B (en) * 2017-12-11 2018-07-20 珠海大横琴科技发展有限公司 Ship automatic tracking method and system based on deep learning network and average drifting
CN109063764A (en) * 2018-07-26 2018-12-21 福建和盛高科技产业有限公司 A kind of judgment method of disconnecting switch closing operation in place based on machine vision

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102622615A (en) * 2012-02-24 2012-08-01 山东鲁能智能技术有限公司 Knife switch state closing reliability judging method based on distance between knife switch arm feature points

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
变电站机器人自动巡检中的刀闸开合状态分析;马啸川;李庆武;刘静;钱荣;;电子测量与仪器学报;20180615(06);全文 *

Also Published As

Publication number Publication date
CN110717932A (en) 2020-01-21

Similar Documents

Publication Publication Date Title
CN110705571A (en) Scissor type knife switch opening and closing state detection method
CN110717932B (en) Real-time tracking detection method for scissor type knife switch state
CN110717409B (en) Real-time accurate detection method for state of split type disconnecting link
CN110689528B (en) Automatic positioning-based opposite-opening type disconnecting link opening and closing angle detection method
CN105654461B (en) A kind of machine vision detection method of multiple fission conductor conductor spacer fracture
CN110689574B (en) Method for detecting motion state of split type disconnecting link
CN111191641B (en) Accurate detection method for motion state of split type disconnecting link
CN107767374A (en) A kind of GIS disc insulators inner conductor hot-spot intelligent diagnosing method
CN112017173B (en) Power equipment defect detection method based on target detection network and structured positioning
CN108648169A (en) The method and device of high voltage power transmission tower defects of insulator automatic identification
CN112749656B (en) ORB feature matching and yolo-based air switch state detection method and device
JP6444283B2 (en) Posture determination device
CN109447949A (en) Insulated terminal defect identification method based on crusing robot
CN108537829A (en) A kind of monitor video personnel state recognition methods
CN104949658B (en) Wire ice-shedding trajectory measurement device based on technique of binocular stereoscopic vision
WO2023092760A1 (en) Diagnosis method and diagnosis apparatus based on thermal vision
CN111008951B (en) Calculation method of opening and closing angle of split type disconnecting link based on positioning identification
CN110717923B (en) Knife switch state detection method capable of rapidly identifying electric wire
CN113449688B (en) Power transmission tree obstacle recognition system based on image and laser point cloud data fusion
CN105987665B (en) Early warning monitoring device and method for sag point change range of ultra-high voltage transmission line
CN110675452A (en) Real-time tracking split type disconnecting link state detection method
Lu et al. Thermal Fault Diagnosis of Electrical Equipment in Substations Based on Image Fusion.
CN108748162A (en) A kind of Mechanical arm control method of the robot experimental teaching based on least square method
CN113076825A (en) Transformer substation worker climbing safety monitoring method
CN110942443B (en) Real-time detection method for state of split type disconnecting link

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20240515

Address after: Room 2307, Block C, Torch Building, No. 306 Changzhi Road, Xuefu Industrial Park, Taiyuan City, Shanxi Province, 030000, China

Applicant after: Shanxi Jinzhi Hongyang Technology Co.,Ltd.

Country or region after: China

Address before: 210032 room 501A, block a, software building, No.9 Xinghuo Road, high tech Zone, Nanjing City, Jiangsu Province

Applicant before: NANJING XINHEHUITONG ELECTRON TECHNOLOGY CO.,LTD.

Country or region before: China

GR01 Patent grant
GR01 Patent grant