WO2023078254A1

WO2023078254A1 - Method and apparatus for determining open/closed state of knife switch, and device, medium and product

Info

Publication number: WO2023078254A1
Application number: PCT/CN2022/129042
Authority: WO
Inventors: 方嘉聪; 肖少剑; 龚浩
Original assignee: 珠海优特电力科技股份有限公司
Priority date: 2021-11-08
Filing date: 2022-11-01
Publication date: 2023-05-11
Also published as: GB2616993A; CN114220068A; CN114220068B; GB202308674D0; AU2022381656A1

Abstract

The present disclosure provides a method and apparatus for determining the open/closed state of a knife switch, and a device, a medium and a product. The method comprises: acquiring a knife switch monitoring image; extracting image features of a plurality of preset regions in the knife switch monitoring image, wherein the plurality of preset regions at least comprise a closing position region in which a knife switch is located when same is in a closing position state and an opening position region in which the knife switch is located when same is in an opening position state; performing structural similarity comparison on the image features of the preset regions in the knife switch monitoring image with an image feature of a preset region corresponding to a preset knife switch template image; and determining the open/closed state of the knife switch according to a structural similarity comparison result.

Description

Method, device, equipment, medium and product for determining opening and closing state of knife switch

This disclosure claims the priority of the Chinese patent application with the application number 202111314122.4 and the title of the invention "method, device, equipment, medium and product for determining the opening and closing state of the knife gate" submitted to the China Patent Office on November 08, 2021, and its entire content Incorporated by reference in this disclosure.

technical field

The present disclosure relates to the technical field of rail transit, and in particular to a method, device, equipment, medium and product for determining the opening and closing state of a knife switch.

Background technique

In the field of rail transit, the status management of in-track equipment is closely related to the safety of rail transit. Among them, the monitoring and management of the knife switch equipment in the track is a more important part.

The traditional monitoring and management method of switch equipment is to judge the opening and closing status of the switch through whether the electricity is connected or physical and mechanical sensors. This kind of knife switch confirmation method needs to add a sensor to the equipment body. Once the sensor fails, the opening and closing status of the knife switch cannot be determined, and the operating equipment needs to be shut down during maintenance or renovation.

Therefore, a new method for confirming the opening and closing state of the knife switch is needed in the field of rail transit to solve the problem that the opening and closing state of the knife switch cannot be determined when the sensor fails.

Contents of the invention

The disclosure provides a method, device, equipment, medium and product for determining the opening and closing state of a knife switch, which are used to provide a method for confirming the opening and closing state of a knife switch to solve the problem that the opening and closing state of a knife switch cannot be determined when a sensor fails.

The first aspect of the present disclosure provides a method for determining the opening and closing state of a knife gate, including: acquiring a knife gate monitoring image; the knife gate monitoring image is that the camera device shoots the knife gate monitoring area according to the image sending instruction sent by the knife gate control device Generated and sent; the image sending instruction is generated by the knife gate control device within a preset time period for controlling the closing or opening of the knife gate; image features of multiple preset areas in the knife gate monitoring image are extracted The plurality of preset areas at least include the closed position area where the knife switch is in the closed position state and the divided position area where the knife switch is located when the knife switch is in the divided state; monitor the knife switch The image features of each preset area in the image are compared with the image features of the corresponding preset area of the preset knife gate template image for structural similarity; according to the result of the structural similarity comparison, the opening and closing status of the knife gate is determined.

The second aspect of the present disclosure provides a device for determining the opening and closing state of a knife gate, including: an acquisition module configured to acquire a knife gate monitoring image; the knife gate monitoring image is an image sending instruction sent by the camera device according to the knife gate control device It is generated and sent by photographing the knife gate monitoring area; the image sending instruction is generated by the knife gate control device within a preset time period for controlling the knife gate to close or disconnect; the extraction module is configured to extract the knife gate monitoring The image features of multiple preset areas in the image; the multiple preset areas at least include the closed position area where the knife switch is in the closed state and the branch position where the knife switch is in the divided state. bit position area; comparison module, configured to carry out structural similarity comparison between the image features of each of the preset areas in the knife gate monitoring image and the image features of the preset knife gate template image corresponding to the preset area; determine the module, set In order to determine the opening and closing state of the knife switch according to the result of the structural similarity comparison.

The third aspect of the embodiments of the present disclosure provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory , so as to realize the method for determining the opening and closing state of the knife switch according to any one of the first aspect.

The fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement any one of the first aspect when executed by a processor. The method for determining the opening and closing state of the knife switch described above.

The fifth aspect of the embodiments of the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method for determining the opening and closing state of the knife switch described in any one of the first aspect is implemented.

The present disclosure provides a method, device, equipment, medium and product for determining the opening and closing state of a knife gate, the method comprising: acquiring a knife gate monitoring image; the knife gate monitoring image is an image sent by a camera device according to a knife gate control device The instruction is generated and sent by taking pictures of the knife gate monitoring area; the image sending instruction is generated by the knife gate control device within the preset time period for controlling the knife gate to close or disconnect; extracting multiple images from the knife gate monitoring image The image characteristics of the preset area; the plurality of preset areas at least include the closing position area where the knife switch is in the closing state and the dividing position area where the knife switch is in the dividing state; Carry out structural similarity comparison between the image features of each of the preset areas in the knife gate monitoring image and the image features of the preset knife gate template image corresponding to the preset area; determine the opening and closing state of the knife gate according to the result of the structural similarity comparison . The method for determining the opening and closing state of the knife switch of the present disclosure compares the structural similarity between the preset area of the knife switch monitoring image and the preset area of the preset knife switch template image, because the preset area at least includes when the knife switch is in the closing state The closed position area where the knife switch is located and the divided position area where the knife switch is located when the knife switch is in the split state. Therefore, no matter whether the sensor is faulty or not, the opening and closing position of the knife switch can be determined according to the comparison result. state.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

FIG. 1 is a scene diagram that can realize the method for determining the opening and closing state of the knife switch of the present disclosure;

FIG. 2 is a schematic flowchart of a method for determining the opening and closing state of a knife switch provided by the first embodiment of the present disclosure;

FIG. 3 is a schematic flowchart of a method for determining the opening and closing state of a knife switch provided by the second embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a preset area of a method for determining the opening and closing state of a knife switch provided by the second embodiment of the present disclosure;

FIG. 5 is a schematic diagram of the image alignment process of the method for determining the opening and closing state of the knife switch provided by the third embodiment of the present disclosure;

Fig. 6 is a schematic diagram of the interaction process of the method for determining the opening and closing state of the knife switch provided by the fourth embodiment of the present disclosure;

FIG. 7 is a schematic diagram of the image alignment process of the method for determining the opening and closing state of the knife switch provided by the fourth embodiment of the present disclosure;

Fig. 8 is a schematic structural diagram of a device for determining the opening and closing state of a knife switch provided by the fifth embodiment of the present disclosure;

FIG. 9 is a schematic structural diagram of an electronic device provided by a sixth embodiment of the present disclosure.

By means of the above-mentioned drawings, certain embodiments of the present disclosure have been shown and will be described in more detail hereinafter. These drawings and written description are not intended to limit the scope of the disclosed concept in any way, but to illustrate the disclosed concept for those skilled in the art by referring to specific embodiments.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

The technical solution of the present disclosure will be described in detail below with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present disclosure will be described below with reference to the accompanying drawings.

In order to clearly understand the technical solutions of the present disclosure, the solutions of the prior art are first introduced in detail. Computer vision technology is a computer simulation of human visual process, which is a synthesis of image processing, artificial intelligence and pattern recognition technologies. With the continuous development of rail transit, the application of computer vision technology in the status management of equipment in the track is also increasing. The monitoring and management of traditional knife switch equipment in the track needs to add sensors to the equipment body. Once the sensor fails, the opening and closing status of the knife switch cannot be determined, and the operating equipment needs to be shut down during maintenance or renovation. Therefore, the rail transit field needs A new method for confirming the opening and closing state of the knife switch, combined with computer vision technology to solve the problem that the opening and closing state of the knife switch cannot be determined when the sensor fails.

Therefore, in the field of rail transit in the prior art, a new method for confirming the opening and closing state of the knife gate is needed to solve the problem that the opening and closing state of the knife gate cannot be determined when the sensor fails. The inventor found in the research that in order to solve this problem, the Obtain the monitoring image of the knife switch taken by the camera device for monitoring the operating state of the knife switch equipment, and determine the opening and closing state of the knife switch according to the monitoring image of the knife switch and the preset knife switch template image.

Specifically, when the knife gate control device controls the knife gate equipment to be closed or disconnected, the knife gate monitoring image generated by the camera device shooting the knife gate monitoring area is obtained, and the image features of multiple preset areas in the knife gate monitoring image are extracted. The plurality of preset areas at least include the closed position area where the knife switch is in the closed position and the divided position area where the knife switch is in the divided state when the knife switch is in the divided state. Then, the structural similarity comparison is performed between the image features of each preset area in the knife gate monitoring image and the image features of the preset knife gate template image corresponding to the preset area. According to the result of structural similarity comparison, the opening and closing state of the knife switch is determined. Therefore, no matter whether the sensor is faulty or not, the opening and closing state of the knife switch can be determined according to the comparison result.

The inventor proposes the technical solution of the present disclosure based on the above-mentioned inventive discoveries.

The following introduces the application scenarios of the method for determining the opening and closing state of the knife switch provided by the present disclosure. As shown in FIG. 1 , 1 is an electronic device, 2 is a knife switch control device, 3 is a knife switch device, and 4 is a camera device. The network architecture of the application scenario corresponding to the method for determining the opening and closing state of the knife switch provided in the present disclosure includes: an electronic device 1 , a knife switch control device 2 , a knife switch device 3 and a camera device 4 . The knife switch control device 2 can control the closing or opening of the knife switch device 3 , and the camera device 4 can photograph the knife switch monitoring area including the knife switch device 3 .

After the knife switch control device 2 controls the knife switch of the knife switch device 3 to close or open, the knife switch device 3 will feed back the operation information of the knife switch through the sensor set in the period. At the same time, the knife gate control device 2 will send an image sending instruction to the camera device 4 within a preset time period, such as 5 seconds, so that the camera device 4 will send the knife gate monitoring image generated by shooting the knife gate monitoring area to the electronic equipment 1. The electronic device 1 extracts image features of multiple preset regions in the knife gate monitoring image after acquiring the knife gate monitoring image. The multiple preset areas at least include the closed position area where the knife switch is in the closed position and the divided position area where the knife switch is in the divided state when the knife switch is in the divided state. At the same time, the electronic device 1 compares the image features of each preset area in the knife gate monitoring image with the image features of the preset knife gate template image corresponding to the preset area for structural similarity, and determines the knife gate classification according to the result of the structural similarity comparison. combined state. After the electronic device 1 determines the opening and closing state of the knife switch, it can send the opening and closing state of the knife switch to the knife control device 2 so that the knife control device 2 can further determine the opening and closing state of the knife switch according to the opening and closing state of the knife switch and the operation information of the knife switch. The gate opening and closing state.

Embodiments of the present disclosure will be described below in conjunction with the accompanying drawings.

Fig. 2 is a schematic flowchart of the method for determining the opening and closing state of the knife switch provided by the first embodiment of the present disclosure. As shown in Fig. The opening and closing state determination device can be integrated in the electronic device. Then the method for determining the opening and closing state of the knife switch provided in this embodiment includes the following steps:

Step S101, acquiring a monitoring image of a knife switch. The knife gate monitoring image is generated and sent by the camera device to photograph the knife gate monitoring area according to the image sending instruction sent by the knife gate control device. The image sending instruction is generated by the knife switch control device within a preset time period for controlling the closing or opening of the knife switch.

In this embodiment, the photographing device may photograph the knife gate monitoring area including the knife gate in real time, or may photograph the knife gate monitoring area after receiving an image sending instruction, which is not limited in this embodiment. During the preset period of time during which the knife gate control device controls the knife gate to close or disconnect, the knife gate control device will send an image sending instruction to the shooting device, so that the shooting device will send the knife gate monitoring image generated by shooting the knife gate monitoring area to The device for determining the opening and closing state of the knife switch.

The preset time period can be set according to actual needs. For example, it takes 4 seconds for the knife gate to be fully closed to completely separated, then the preset time period can be set to 4 seconds or 5 seconds, etc., thereby reducing the image taken by the shooting device. is the probability that the switch is in a state between fully closed and fully open.

Step S102, extracting image features of a plurality of preset areas in the knife gate monitoring image. The plurality of preset areas at least include the closed position area where the knife switch is in the closed position and the divided position area where the knife switch is in the divided state when the knife switch is in the divided state.

In this embodiment, the extracted image feature can be extracted scale-invariant feature transform (English full name: Scale-invariant feature transform, English abbreviation: SIFT) feature, so that the extracted feature is more stable, and it can also make the follow-up and The structural similarity comparison between preset knife gate template images is more stable.

The closed position area where the knife switch is located when the knife switch is in the closed position refers to the position area where the knife switch is located when it is fully closed. The division position area where the knife switch is located when the knife switch is in the divided state refers to the position area where the knife switch is located when the knife switch is in the complete division state.

The preset area is shown in dotted line boxes a and b in Fig. 4, wherein, the dotted line box a can refer to the area where the knife switch is located when the knife switch is in the divided state, and the dotted line box b can refer to the area where the knife switch is in the closed state. The closing position area where the knife switch is in the bit state. The preset area needs to include at least the closed position area where the knife switch is in the closed state and the divided position area where the knife switch is in the divided state, so that in the subsequent and preset knife gate template When the images are compared, it is judged whether the opening and closing state of the knife switch is the closing state, the dividing state, or whether it is neither in the closing state nor in the dividing state.

Step S103 , comparing the image features of each preset area in the knife gate monitoring image with the image features of the corresponding preset area of the preset knife gate template image for structural similarity.

In this embodiment, the preset knife gate template image refers to pre-captured images of the knife gate in various states, such as the template image taken when the knife gate is in the closing state and the template image taken when the knife gate is in the dividing state. After a single configuration of the preset knife gate template image, the original configuration can be reused without model training, which improves configuration efficiency.

Structural similarity is used to compare the similarity between two images. The English abbreviation of structural similarity is: SSIM, and the full English name is: structural similarity index, which is an index to measure the similarity of two images.

Step S104, determining the opening and closing state of the knife switch according to the result of the structural similarity comparison.

In this embodiment, the structural similarity can determine the similarity between the knife gate monitoring image and the preset knife gate template image, and can determine whether the knife gate in the knife gate monitoring image is in the preset knife gate template image according to the similarity The status of the knife switch, so as to determine the opening and closing status of the knife switch in the monitoring image of the knife switch.

An embodiment of the present disclosure provides a method for determining an opening and closing state of a knife switch, the method comprising: acquiring a monitoring image of a knife switch. The knife gate monitoring image is generated and sent by the camera device to photograph the knife gate monitoring area according to the image sending instruction sent by the knife gate control device. The image sending instruction is generated by the knife switch control device within a preset time period for controlling the closing or opening of the knife switch. The image features of multiple preset areas in the knife gate monitoring image are extracted. The plurality of preset areas at least include the closed position area where the knife switch is in the closed position and the divided position area where the knife switch is in the divided state when the knife switch is in the divided state. Structural similarity comparisons are made between the image features of each preset area in the knife gate monitoring image and the image features of the preset knife gate template image corresponding to the preset area. According to the result of structural similarity comparison, the opening and closing state of the knife switch is determined.

The method for determining the opening and closing state of the knife gate in the embodiment of the present disclosure compares the structural similarity between the preset area of the knife gate monitoring image and the preset area of the preset knife gate template image, because the preset area at least includes The close position area where the knife switch is in the state and the minute position area where the knife switch is in the split state. Therefore, no matter whether the sensor is faulty or not, the knife switch where the knife switch is located can be determined according to the comparison result. Split state.

Figure 3 is a schematic flow chart of the method for determining the opening and closing state of the knife switch provided by the second embodiment of the present disclosure. As shown in Figure 3, the method for determining the opening and closing state of the knife switch provided by this embodiment is provided in the previous embodiment of the present disclosure. On the basis of the determination method of the opening and closing state of the knife switch, each step is further refined. Then, the method for determining the opening and closing state of the knife switch provided in this embodiment includes the following steps.

Step S201, acquiring a monitoring image of a knife switch. The knife gate monitoring image is generated and sent by the camera device to photograph the knife gate monitoring area according to the image sending instruction sent by the knife gate control device. The image sending instruction is generated by the knife switch control device within a preset time period for controlling the closing or opening of the knife switch.

In this embodiment, the implementation manner of step 201 is similar to the implementation manner of step 101 in the previous embodiment of the present disclosure, and will not be repeated here.

Step S202, obtaining the coordinate range of each preset area in the preset knife gate template image in the preset knife gate template image. Wherein, the preset knife switch template image is a pre-shot closing template image when the knife switch is in a closing state and a dividing template image when the knife switch is in a separating state. The closing template image includes the closing position area when the knife switch is in the closing state and the dividing position area when the knife switch is in the closing state. The divider template image includes the closing position area when the knife switch is in the divider state and the divider position area when the knife switch is in the divider state.

In this embodiment, when configuring the preset knife gate template image, each preset area will be represented in the form of a coordinate range. The preset area can be in the form of a square area, as shown by dashed boxes a and b in FIG. 4 , or it can be set in a circle or other shapes.

In this embodiment, the closing template image may further include: an image of the closing position area when the knife switch is in the closing position area, and an image of the dividing position area when the knife switch is in the closing position area. The dividing template image may also include: an image of the closing position area when the knife switch is in the dividing position area, and an image of the dividing position area when the knife switch is in the dividing position area.

Step S203, extracting image features corresponding to preset areas in the knife gate monitoring image according to each coordinate range.

In this embodiment, since the preset area of the knife gate monitoring image corresponds to the preset area of the preset knife gate template image, the corresponding preset area in the knife gate monitoring image can be extracted according to each coordinate range of the preset knife gate template image. Image features of the set area. The image feature corresponding to the preset area in the knife gate monitoring image may also be extracted in other ways, which is not limited in this embodiment.

It should be noted that the preset area is the closed position area where the knife switch is in the closed state and the divided position area where the knife switch is in the divided state when the knife switch is in the closed state.

Step S204, comparing the closing position area in the knife gate monitoring image with the closing position area in the closing template image and the closing position area in the quantization template image respectively, so as to determine the closing position area comparison result.

In this embodiment, the purpose of comparing the structural similarity of the closing position area in the knife gate monitoring image with the closing position area in the closing template image and the closing position area in the sub-position template image is to determine the Whether there is a knife switch in the closed position area in the image. As shown in Figure 4, the dotted line box b area is the close position area in the preset area. When the knife switch is in the close position state, there is a knife switch in the dotted line box b area, and when the knife switch is in the split state, the dotted line There is no knife switch in the box b area.

At the same time, the specific structural similarity comparison process is as follows:

If the structural similarity between the closing position area in the knife gate monitoring image and the closing position area in the closing template image is greater than the closing position area in the knife gate monitoring image and the closing position area in the split template image If there is structural similarity between them, it is determined that there is a knife switch in the closing position area as a result of the comparison of the closing position area.

If the structural similarity between the closing position area in the knife gate monitoring image and the closing position area in the closing template image is smaller than the closing position area in the knife gate monitoring image and the closing position area in the split template image If there is structural similarity between them, it is determined that there is no knife gate in the closing position area as a result of the comparison.

Structural similarity is expressed in numerical form, generally the value is between 0 and 1, and the larger the value, the higher the degree of similarity. When the structural similarity between the closing position area in the knife gate monitoring image and the closing position area in the closing template image is greater, it means that there is a knife gate in the closing position area.

Step S205, performing structural similarity comparison between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image and the quantile position area in the quantile template image, so as to determine the quantile position area comparison result.

In this embodiment, the structural similarity comparison between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image and the quantile position area in the quantizing template image is to determine the Whether there is a knife gate in the quantile position area in the image. As shown in Figure 4, the dotted line box a area is the division position area in the preset area. When the knife switch is in the division state, there is a knife switch in the dotted line box a area, and when the knife switch is in the close position, the dotted line There is no knife gate in the box a area.

If the structural similarity between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image is greater than the quantile position area in the knife gate monitoring image and the quantile position area in the quantizing template image If there is structural similarity between them, it is determined that there is no knife gate in the quantile position area as a result of the comparison.

If the structural similarity between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image is smaller than the quantile position area in the knife gate monitoring image and the quantile position area in the quantizing template image If there is structural similarity between them, it is determined that there is a knife gate in the quantile position area as a result of the comparison of the quantile position area.

The quantile position area is determined in a similar way to the close position area. When the structural similarity between the quantile position area in the knife gate monitoring image and the quantile position area in the quantile template image is greater, it represents the quantile position. There are knife gates in the area.

Step S206, determining the opening and closing state of the knife switch according to the comparison result of the closed position area and the comparison result of the divided position area.

In this embodiment, the determination of the opening and closing state of the knife switch according to the comparison result of the closing position area and the comparison result of the dividing position area can be specifically as follows:

If the comparison result of the closing position area is that there is a knife switch in the closing position area, and the comparison result of the closing position area is that there is no knife switch in the closing position area, then it is determined that the knife switch is in the closing state.

If the comparison result of the close position area is that there is no knife switch in the close position area, and the comparison result of the split position area is that there is a knife switch in the split position area, then it is determined that the knife switch state is in the split state.

If the comparison result of the closing position area is that there is no knife switch in the closing position area, and the comparison result of the closing position area is that there is no knife switch in the closing position area, then it is determined that the knife switch state is in a non-separating and non-closing state.

Step S207, sending the opening and closing state of the knife switch to the knife switch control device, so that the knife switch control device determines the opening and closing state of the knife switch for a second time according to the opening and closing state of the knife switch and the operation information of the knife switch fed back by the sensor.

In this embodiment, the opening and closing state of the knife switch can be sent to the knife switch control device, so that the knife switch control device can determine the opening and closing state of the knife switch according to the opening and closing state of the knife switch and the knife switch operation information fed back by the sensor, so as to improve the certainty. The accuracy and stability of the opening and closing state of the knife switch.

FIG. 5 is a schematic diagram of the image alignment process of the method for determining the opening and closing state of the knife switch provided by the third embodiment of the present disclosure. As shown in FIG. Based on the method for determining the opening and closing state of the knife gate provided in the example, an image alignment process is added, and the image alignment process includes the following steps.

Step S301, extracting the image features of the monitor image of the knife gate and the image features of the preset knife gate template image.

In this embodiment, the SIFT feature is preferable for extracting the image feature of the knife gate monitoring image, which can better improve the matching degree of the image feature pair and the stability of the transformation matrix.

Step S302, matching the image features of the monitor image of the knife gate with the image features of the preset knife gate template image to generate a plurality of image feature pairs.

An image feature pair is a feature pair consisting of two image features that match.

The matching method can use FLANN (Chinese: fast nearest neighbor search package, English full name: Fast Library for Approximate Nearest Neighbors) or BF (Chinese is violence, English full name: Brute Force) algorithm to further improve the accuracy of matching image feature pairs sex and stability.

In step S303, a transformation matrix between the knife gate monitoring image and the preset knife gate template image is determined according to a plurality of image feature pairs and a preset projection transformation algorithm.

The preset projection transformation algorithm can use the RANSAC robust solution algorithm, which is called Random Sample Consensus in English, so as to determine a more stable and accurate transformation matrix.

At the same time, when determining the transformation matrix between the knife gate monitoring image and the preset knife gate template image according to multiple image feature pairs and the preset projection transformation algorithm, it is possible to first judge whether the number of multiple image feature pairs is greater than or equal to the preset threshold.

If it is determined that the number of multiple image feature pairs is greater than or equal to the preset threshold, the transformation matrix between the knife gate monitoring image and the preset knife gate template image is determined according to the multiple image feature pairs and a preset projection transformation algorithm.

If it is determined that the number of image feature pairs is less than the preset threshold, it can be determined that the matching degree between the knife gate monitoring image and the preset knife gate template image is too low, and the next knife gate monitoring image can be used for image alignment.

Step S304, correcting the monitoring image of the knife gate according to the transformation matrix, so that the corrected monitoring image of the knife gate is aligned with the preset knife gate template image.

In this embodiment, the correcting process is performed on the monitoring image of the knife gate because the rail vehicle is likely to generate relatively large vibrations during actual operation, thereby causing an angle deviation of the photographing device. If the shooting device produces an angular offset, the captured knife switch monitoring image will also have an angle offset problem, which will affect the subsequent determination of the opening and closing status of the knife switch. Therefore, the problem of angular deviation of the shooting device can be solved by correcting the monitoring image of the knife gate.

At the same time, in order to better understand the method for determining the opening and closing state of the knife switch of the present disclosure, further description will be given below in conjunction with the accompanying drawings. As shown in FIG. 6 , in this embodiment, the second determination of the status of the knife switch by the knife switch control device is used as an example for illustration. The knife switch control device adopts the control device in the commonly used one-key sequence control system, which is represented by one-key sequence control in the figure. The photographing device adopted in this embodiment is a camera.

The overall process is: the one-button sequential control system sends a signal to control the closing or opening of the knife switch, and after waiting for a few seconds, the knife switch moves in place, and the sensor in the knife switch device will feedback the operation information of the knife switch, that is, whether it is closed or divided. At the same time, the one-button sequential control system will send an image sending command to the camera, so that the camera can transmit the captured knife switch monitoring image to the knife switch opening and closing state determination device to judge the knife switch opening and closing state, and return the result to the one-button sequence control system. The one-button sequential control system integrates the status discrimination information of the sensor and the switch opening and closing status determination device, and obtains the opening and closing status of the switch comprehensively, and completes the double confirmation of the opening and closing of the switch.

specific process:

Before determining the opening and closing state analysis of the knife switch, the images of the knife switch in the opening and closing positions are intercepted for labeling configuration. After a single configuration is completed, the original configuration can be reused without model training.

Specifically, firstly, two photos of the knife switch in the split position and the closing position are saved as template images. And mark the two dotted line boxes at the position a and b of the active connection of the knife switch, as shown in Figure 4, the upper part of Figure 4 is the state when it is closed, and the lower part is the state when it is divided. In this way, four matching templates can be obtained, namely close position (with knife switch), close position (without knife switch), separate position (with knife switch) and separate position (without knife switch). These 4 areas will be used to identify the opening and closing state of the knife switch.

Then, as shown in FIG. 7 , the angle of the camera is corrected on the monitor image of the knife gate, so that the monitor image of the knife gate is aligned with the template image. Specifically: the device for determining the opening and closing state of the knife gate obtains the knife gate monitoring image that needs to determine the knife gate state from the camera, extracts the SIFT feature of the image, and then uses the FLANN or BF algorithm to match the SIFT features of the knife gate monitoring image and the template image, Get pairs of feature points. At this time, judge whether the number of feature point pairs is greater than or equal to 10, if not, wait for the next image input by the camera, if so, use the RANSAC robust solution algorithm to find the knife gate monitoring The transformation matrix M of the image to the template image. Perform affine or projective transformation on the knife gate monitoring image according to M, align the two images, correct the angle offset of the camera, and complete the image alignment.

After the image alignment is completed, the structural similarity comparison is carried out between the knife gate monitoring image and the template image. First, according to the coordinates of the dotted frame areas a and b of the template image, select the corresponding dotted frame areas a and b on the knife gate monitor image after alignment to compare with the dotted frame areas a and b of the saved template image. Through SSIM similarity comparison, it is obtained whether there is a knife switch in the a and b areas, so as to judge the opening and closing status of the knife switch. Discrimination of the opening and closing state is shown in Table 1.

Table 1 Knife switch opening and closing status table

a区域状态a area state	b区域状态b area status	刀闸状态Knife status
无刀闸No knife gate	有刀闸With knife switch	合位fit together
有刀闸With knife switch	无刀闸No knife gate	分位quantile
无刀闸No knife gate	无刀闸No knife gate	动作不到位The action is not in place
有刀闸With knife switch	有刀闸With knife switch	分析出错analysis error

After reading the corresponding state from Table 1, the device for determining the opening and closing state of the knife switch sends the opening and closing state of the knife switch to the one-key sequence control system.

Fig. 8 is a schematic structural diagram of the device for determining the opening and closing state of the knife switch provided by the fifth embodiment of the present disclosure. As shown in Fig. 8, in this embodiment, the device 400 for determining the opening and closing state of the knife switch includes:

The obtaining module 401 is configured to obtain the monitoring image of the knife gate. The knife gate monitoring image is generated and sent by the camera device to photograph the knife gate monitoring area according to the image sending instruction sent by the knife gate control device. The image sending instruction is generated by the knife switch control device within a preset time period for controlling the closing or opening of the knife switch.

The extraction module 402 is configured to extract image features of multiple preset areas in the knife gate monitoring image. The plurality of preset areas at least include the closed position area where the knife switch is in the closed position and the divided position area where the knife switch is in the divided state when the knife switch is in the divided state.

The comparison module 403 is configured to perform structural similarity comparison between the image features of each preset area in the knife gate monitoring image and the image features of the preset knife gate template image corresponding to the preset area.

The determination module 404 is configured to determine the opening and closing state of the knife switch according to the result of the structural similarity comparison.

The device for determining the opening and closing state of the knife switch provided in this embodiment can implement the technical solution of the method embodiment shown in FIG. 2 , and its realization principle and technical effect are similar to those of the method embodiment shown in FIG. 2 , so details will not be repeated here.

At the same time, the device for determining the opening and closing state of the knife switch provided in the present disclosure further refines the device for determining the opening and closing state of the knife switch 400 on the basis of the device for determining the opening and closing state of the knife switch provided in the previous embodiment.

Optionally, in this embodiment, the extraction module 402 is specifically set as:

The coordinate range of each preset area in the preset knife gate template image in the preset knife gate template image is acquired. The image features corresponding to the preset area in the knife gate monitoring image are extracted according to each coordinate range.

Optionally, in this embodiment, the preset area includes the closed position area where the knife switch is in the closed state and the divided position area where the knife switch is in the divided state when the knife switch is in the divided state.

The preset knife gate template image is a pre-shot closing template image when the knife gate is in a closing state and a dividing template image when the knife gate is in a dividing state. The closing template image includes the closing position area when the knife switch is in the closing state and the dividing position area when the knife switch is in the closing state. The divider template image includes the closing position area when the knife switch is in the divider state and the divider position area when the knife switch is in the divider state.

The comparison module 403 is set to:

Structural similarity comparisons were carried out between the closing position area in the knife gate monitoring image and the closing position area in the closing template image and the closing position area in the quantized template image, so as to determine the comparison result of the closing position area. Structural similarity comparisons are made between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image and the quantile position area in the quantile template image to determine the comparison result of the quantile position area.

Optionally, in this embodiment, the comparison module 403 compares the closing position area in the knife gate monitoring image with the closing position area in the closing template image and the closing position area in the sub-positioning template image respectively in a similar structure. Sex comparison to determine the zygomatic position area comparison result, set to:

If the structural similarity between the closing position area in the knife gate monitoring image and the closing position area in the closing template image is greater than the closing position area in the knife gate monitoring image and the closing position area in the split template image If there is structural similarity between them, it is determined that there is a knife switch in the closing position area as a result of the comparison of the closing position area. If the structural similarity between the closing position area in the knife gate monitoring image and the closing position area in the closing template image is smaller than the closing position area in the knife gate monitoring image and the closing position area in the split template image If there is structural similarity between them, it is determined that there is no knife gate in the closing position area as a result of the comparison.

Optionally, in this embodiment, the comparison module 403 compares the quantile position area in the knife gate monitoring image with the quantile position area in the closing template image and the quantile position area in the quantizing template image. To determine the quantile position area comparison results, the specific settings are:

If the structural similarity between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image is greater than the quantile position area in the knife gate monitoring image and the quantile position area in the quantizing template image If there is structural similarity between them, it is determined that there is no knife gate in the quantile position area as a result of the comparison. If the structural similarity between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image is smaller than the quantile position area in the knife gate monitoring image and the quantile position area in the quantizing template image If there is structural similarity between them, it is determined that there is a knife gate in the quantile position area as a result of the comparison of the quantile position area.

Optionally, in this embodiment, the determining module 404 is set to:

If the comparison result of the closing position area is that there is a knife switch in the closing position area, and the comparison result of the closing position area is that there is no knife switch in the closing position area, then it is determined that the knife switch is in the closing state. If the comparison result of the close position area is that there is no knife switch in the close position area, and the comparison result of the split position area is that there is a knife switch in the split position area, then it is determined that the knife switch state is in the split state. If the comparison result of the closing position area is that there is no knife switch in the closing position area, and the comparison result of the closing position area is that there is no knife switch in the closing position area, then it is determined that the knife switch state is in a non-separating and non-closing state.

Optionally, in this embodiment, the device 400 for determining the opening and closing state of the knife switch further includes:

The image alignment module is configured to extract the image features of the monitor image of the knife gate and the image features of the preset knife gate template image. Match the image features of the knife gate monitoring image with the image features of the preset knife gate template image to generate multiple image feature pairs. A transformation matrix between the knife gate monitoring image and the preset knife gate template image is determined according to a plurality of image feature pairs and a preset projection transformation algorithm. The knife gate monitoring image is corrected according to the transformation matrix, so that the corrected knife gate monitoring image is aligned with the preset knife gate template image.

Optionally, in this embodiment, when the image alignment module determines the transformation matrix between the knife gate monitoring image and the preset knife gate template image according to a plurality of image feature pairs and a preset projection transformation algorithm, it is set as:

It is judged whether the number of multiple image feature pairs is greater than or equal to a preset threshold. If it is determined that the number of multiple image feature pairs is greater than or equal to the preset threshold, the transformation matrix between the knife gate monitoring image and the preset knife gate template image is determined according to the multiple image feature pairs and a preset projection transformation algorithm.

The secondary determination module is configured to send the opening and closing state of the knife switch to the knife switch control device, so that the knife switch control device can secondarily determine the opening and closing state of the knife switch according to the opening and closing state of the knife switch and the operation information of the knife switch fed back by the sensor.

The device for determining the opening and closing state of the knife switch provided in this embodiment can implement the technical solution of the method embodiment shown in Fig. 2-Fig. 7, and its realization principle and technical effect are similar to those of the method embodiment shown in Fig. 2-Fig. Let me repeat them one by one.

The present disclosure also provides an electronic device, a computer-readable storage medium, and a computer program product.

As shown in FIG. 9 , FIG. 9 is a schematic structural diagram of an electronic device provided by a sixth embodiment of the present disclosure. The electronic device is intended to be in various forms of digital computers suitable for knife-opening state determination training, such as laptop computers, personal digital assistants, and other suitable computers. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 9 , the electronic device includes: a processor 501 and a memory 502 . The various components are interconnected using different buses and can be mounted on a common motherboard or otherwise as desired. A processor may process instructions for execution within an electronic device.

The memory 502 is a non-transitory computer-readable storage medium provided in the present disclosure. Wherein, the memory stores instructions executable by at least one processor, so that at least one processor executes the method for determining the opening and closing state of the knife switch provided in the present disclosure. The non-transitory computer-readable storage medium of the present disclosure stores computer instructions, and the computer instructions are used to make the computer execute the method for determining the opening and closing state of the knife switch provided in the present disclosure.

The memory 502, as a non-transitory computer-readable storage medium, can be configured to store non-transitory software programs, non-transitory computer-executable programs and modules, such as program instructions/ modules (for example, the acquisition module 401, the extraction module 402, the comparison module 403 and the determination module 404 shown in FIG. 8). The processor 501 executes various functional applications and data processing of the server by running non-transient software programs, instructions and modules stored in the memory 502, that is, implements the method for determining the opening and closing state of the switch in the above method embodiments.

At the same time, this embodiment also provides a computer product. When the instructions in the computer product are executed by the processor of the electronic device, the electronic device can execute the method for determining the opening and closing state of the knife switch in the above embodiment.

Other implementations of the disclosed embodiments will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. The present disclosure is intended to cover any modification, use or adaptation of the embodiments of the present disclosure. These modifications, uses or adaptations follow the general principles of the embodiments of the present disclosure and include those in the technical field not disclosed by the embodiments of the present disclosure. Common knowledge or common technical means. The specification and examples are to be considered as illustrative only.

It should be understood that the embodiments of the present disclosure are not limited to the precise structures that have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the disclosed embodiments is limited only by the appended claims.

Claims

A method for determining the opening and closing state of a knife switch, comprising:

Obtain the knife gate monitoring image; the knife gate monitoring image is generated and sent by the camera device according to the image sending instruction sent by the knife gate control device to shoot the knife gate monitoring area; the image sending instruction is that the knife gate control device is controlling the knife Generated within a preset period of time when the gate is closed or opened;

Extracting the image features of a plurality of preset areas in the monitor image of the knife switch; the multiple preset areas at least include the closed position area where the knife switch is in the closed position and the closed position area when the knife switch is in the split state The sub-position area where the knife switch is located;

Comparing the image features of each preset area in the knife gate monitoring image with the image features of the preset knife gate template image corresponding to the preset area for structural similarity, wherein the preset knife gate template image is a pre-shot The closing template image when the knife switch is in the closed position and the sub-position template image when the knife switch is in the sub-position state; the closing template image includes: the closing position area where there is a knife switch and the sub-position area where there is no knife switch Bit position area; the sub-position template image includes a position area where there is no knife switch and a bit position area where there is a knife switch;

According to the result of structural similarity comparison, the opening and closing state of the knife switch is determined.
The method according to claim 1, wherein said extracting image features of a plurality of preset areas in said knife gate monitoring image comprises:

Obtain the coordinate range of each preset area in the preset knife gate template image in the preset knife gate template image;

Image features corresponding to preset areas in the knife gate monitoring image are extracted according to each of the coordinate ranges.
The method according to claim 1, wherein the structural similarity comparison is performed between the image features of each preset area in the knife gate monitoring image and the image features of the preset knife gate template image corresponding to the preset area, include:

Comparing the closing position area in the knife gate monitoring image with the closing position area in the closing template image and the closing position area in the sub-position template image respectively to determine the closing position location area comparison results;

Structural similarity comparisons are performed between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image and the quantile position area in the quantile template image to determine the quantile position Location area comparison results.
The method according to claim 3, wherein said combining the closing position area in the knife gate monitoring image with the closing position area in the closing template image and the closing position area in the dividing template image respectively Structural similarity comparison of zygomatic location regions to determine the comparison results of zygomatic location regions, including:

If the structural similarity between the closing position area in the knife gate monitoring image and the closing position area in the closing template image is greater than the closing position area in the knife gate monitoring image and the dividing template image, If there is structural similarity between the closing position areas, it is determined that the comparison result of the closing position areas is that there is a knife gate in the closing position areas;

If the structural similarity between the closing position area in the knife gate monitoring image and the closing position area in the closing template image is smaller than the closing position area in the knife gate monitoring image and the dividing template image, If the structural similarity between the closing position areas is determined, the comparison result of the closing position areas is determined to be that there is no knife gate in the closing position areas.
The method according to claim 4, wherein said combining the grading position area in the knife gate monitoring image with the grading position area in the closing template image and the grading position area in the grading template image respectively Structural similarity comparison of the quantile position area to determine the quantile position area comparison results, including:

If the structural similarity between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image is greater than the fractional position area in the knife gate monitoring image and the quantizing template image If the structural similarity between the quantile position areas is determined, the comparison result of the quantile position areas is determined to be that there is no knife gate in the quantile position areas;

If the structural similarity between the quantile position area in the knife gate monitoring image and the quantile position area in the closing template image is smaller than the fractional position area in the knife gate monitoring image and the quantizing template image If the structural similarity between the quantile position areas is determined, the comparison result of the quantile position areas is determined to be that there is a knife gate in the quantile position area.
The method according to claim 5, wherein said determining the opening and closing state of the knife switch according to the result of the structural similarity comparison comprises:

If the comparison result of the closing position area is that there is a knife switch in the closing position area, and the comparison result of the minute position area is that there is no knife switch in the closing position area, then it is determined that the knife switch state is in the closing state;

If the comparison result of the close position area is that there is no knife switch in the close position area, and the comparison result of the split position area is that there is a knife switch in the split position area, then it is determined that the knife switch state is in the split state;

If the comparison result of the closing position area is that there is no knife switch in the closing position area, and the comparison result of the closing position area is that there is no knife switch in the closing position area, then it is determined that the knife switch state is in a non-separating and non-closing state.
The method according to any one of claims 1 to 6, wherein, before extracting the image features of a plurality of preset areas in the knife gate monitoring image, further comprising:

Extracting the image features of the monitor image of the knife gate and the image features of the preset knife gate template image;

Matching the image features of the knife gate monitoring image with the image features of the preset knife gate template image to generate a plurality of image feature pairs;

determining a transformation matrix between the knife gate monitoring image and the preset knife gate template image according to the plurality of image feature pairs and a preset projection transformation algorithm;

The knife gate monitoring image is corrected according to the transformation matrix, so that the corrected knife gate monitoring image is aligned with a preset knife gate template image.
The method according to claim 7, wherein the transformation matrix between the knife gate monitoring image and the preset knife gate template image is determined according to the plurality of image feature pairs and a preset projection transformation algorithm, include:

judging whether the number of image feature pairs is greater than or equal to a preset threshold;

If it is determined that the number of the multiple image feature pairs is greater than or equal to the preset threshold, then determine the difference between the knife gate monitoring image and the preset knife gate template image according to the multiple image feature pairs and the preset projection transformation algorithm. Transformation matrix between.
The method according to claim 7, wherein the correcting the knife gate monitoring image according to the transformation matrix so that the corrected knife gate monitoring image is aligned with the preset knife gate template image comprises:

Affine or projective transformation is performed on the knife gate monitoring image according to the transformation matrix, so that the knife gate monitoring image after the affine or projective transformation is aligned with the preset knife gate template image.
The method according to claim 1, wherein, after determining the opening and closing state of the knife switch according to the result of the structural similarity comparison, further comprising:

Sending the opening and closing state of the knife switch to the control device of the knife switch, so that the control device of the knife switch determines the opening and closing state of the knife switch for a second time according to the opening and closing state of the knife switch and the operation information of the knife switch fed back by the sensor.
A device for determining the opening and closing state of a knife switch, comprising:

The acquisition module is configured to acquire the knife gate monitoring image; the knife gate monitoring image is generated and sent by the camera device according to the image sending instruction sent by the knife gate control device to capture the knife gate monitoring area; the image sending instruction is the knife gate Generated by the control device within the preset time period for controlling the closing or opening of the knife switch;

The extraction module is configured to extract the image features of multiple preset areas in the knife switch monitoring image; the multiple preset areas include at least the closed position area where the knife switch is in the closed state and the knife switch The divisional position area where the knife switch is located when it is in the divisional state;

The comparison module is configured to perform structural similarity comparison between the image features of each preset area in the knife gate monitoring image and the image features of the preset knife gate template image corresponding to the preset area, wherein the preset knife gate The template image is the closing template image taken in advance when the knife switch is in the closed position and the sub-position template image when the knife switch is in the sub-position state; There is a sub-position area of a knife switch; the sub-position template image includes: a closing position area without a knife switch and a sub-position area with a knife switch;

The determination module is configured to determine the opening and closing state of the knife switch according to the result of the structural similarity comparison.
An electronic device, comprising: a processor, and a memory communicatively connected to the processor;

the memory stores computer-executable instructions;

The processor executes the computer-executed instructions stored in the memory, so as to realize the method for determining the opening and closing state of the knife switch according to any one of claims 1 to 10.
A computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to realize the knife-gate separation according to any one of claims 1 to 10 A method for determining the state of fit.
A computer program product, including a computer program, when the computer program is executed by a processor, the method for determining the opening and closing state of a switch according to any one of claims 1 to 10 is realized.