WO2023171068A1

WO2023171068A1 - Surveillance system

Info

Publication number: WO2023171068A1
Application number: PCT/JP2022/045759
Authority: WO
Inventors: 圭吾長谷川; 友輔生内
Original assignee: 株式会社日立国際電気
Priority date: 2022-03-07
Filing date: 2022-12-13
Publication date: 2023-09-14

Abstract

The purpose of the present invention is to provide a surveillance system capable of more accurately detecting dangerous events at railroad crossings. The surveillance system comprises an imaging device and an analysis device. The analysis device includes a state analysis control unit and a state analysis unit. When detecting the approach of a train by analyzing an image acquired by the imaging device or on the basis of information notified from a state notification unit, the state analysis control unit instructs the state analysis unit to start analyzing the image within the railroad crossing. The state analysis unit uses the image acquired by the imaging device to detect and track a moving object within the railroad crossing. When it is determined by using at least a current moving speed of the moving object that a detection target cannot reach a set virtual line until the blocking of the railroad crossing is completed, the state analysis unit determines that a dangerous event occurs, and performs alarm processing.

Description

Monitoring system

The present invention relates to a monitoring system, and particularly to a monitoring system that detects a dangerous event based on the results of detecting and tracking a detection target in an image.

For example, automatic level crossing alarms and traffic signals are installed at railroad crossings to ensure traffic safety by separating pedestrians, cars, and trains. However, accidents have occurred in which people, cars, wheelchairs, etc. are left behind at railroad crossings and collided with trains. Therefore, in order to improve safety, it is common practice to install laser sensors to detect pedestrians, cars, etc. remaining within railroad crossings.

On the other hand, in Patent Document 1, when it is determined that a vehicle is approaching a crossing road, it is determined whether or not the detected object is a moving object that is moving. If it is determined that there is a moving object, it predicts the change in the position of the moving object, and based on the predicted change in the position of the moving object, it predicts whether the moving object is likely to interfere with the vehicle's travel. An apparatus is disclosed.

In addition, Patent Document 2 describes a level crossing passability judgment that determines whether a passerby can pass through a level crossing by comparing the passerby's position and passing speed with the level crossing information, and notifies the passerby of the result of the judgment. A support system is disclosed.

Japanese Patent Application Publication No. 2006-111177 Japanese Patent Application Publication No. 2016-155405

However, since the method using a laser sensor detects a dangerous event after the barrier has been lowered, there is a short time lag from detection until the alarm is issued and the train stops. Another option is to install surveillance cameras to monitor the inside of railroad crossings and visually check the images, but this method requires a large number of railroad crossings to be monitored at the same time, which puts a heavy burden on the inspectors.

On the other hand, Patent Document 1 assumes the use of a laser irradiation type sensor. In this case, the range that can be detected by laser irradiation is limited, and the information obtained from the laser is also limited. Additionally, laser irradiation type sensors are more expensive than cameras. Furthermore, it is necessary to install a separate camera in order for the guard to visually check the situation inside the railroad crossing.

Furthermore, Patent Document 2 assumes that the location of a passerby is acquired by the passerby holding a mobile terminal. Therefore, if a passerby does not have a corresponding mobile terminal, the passerby cannot be detected.

In view of the above problems, the present invention aims to provide a monitoring system that can more accurately detect dangerous events within railroad crossings.

In order to achieve the above object, one of the typical monitoring systems of the present invention includes an imaging device and an analysis device, the analysis device has a state analysis control section and a state analysis section, and the analysis device includes a state analysis control section and a state analysis section. When the condition analysis control section detects that a train is approaching based on the analysis of the image acquired by the imaging device or the information notified from the condition notification section, the condition analysis control section instructs the condition analysis section to start analyzing the image inside the railroad crossing. and the state analysis unit detects and tracks a moving object within the railroad crossing using the image acquired by the imaging device, and uses at least the current moving speed of the moving object to detect the object to be detected by the time the crossing of the railroad crossing is completed. If it is determined that the set virtual line cannot be reached, it is determined that a dangerous event has occurred and an alarm process is performed.

According to the present invention, a dangerous event within a railroad crossing can be detected more accurately in a monitoring system.
Problems, configurations, and effects other than those described above will be clarified by the following embodiments.

FIG. 1 is a block diagram of a computer system for implementing aspects according to embodiments of the present disclosure. FIG. 2 is a block diagram showing an example of the configuration of the monitoring system of the present invention. FIG. 3 is a conceptual diagram showing an example of application of the monitoring system of the present invention. FIG. 4 shows an example of a flowchart of processing of the monitoring system of the present invention. FIG. 5 is a diagram for explaining the determination method of the monitoring system of the present invention. FIG. 6 is a diagram showing a first display example in the monitoring system of the present invention. FIG. 7 is a diagram showing a second display example in the monitoring system of the present invention.

A mode for carrying out the present invention will be described.

<Computer system for implementing aspects according to embodiments>
FIG. 1 is a block diagram of a computer system 1 for implementing aspects according to embodiments of the present disclosure. The mechanisms and apparatus of the various embodiments disclosed herein may be applied to any suitable computing system. The main components of computer system 1 include one or more processors 2 , memory 4 , terminal interface 12 , storage interface 14 , I/O (input/output) device interface 16 , and network interface 18 . These components may be interconnected via a memory bus 6, an I/O bus 8, a bus interface unit 9, and an I/O bus interface unit 10.

The computer system 1 may include one or

more processing devices

2A and 2B, collectively referred to as processors 2. Each processor 2 executes instructions stored in memory 4 and may include onboard cache. In some embodiments, computer system 1 may include multiple processors, and in other embodiments, computer system 1 may be a single processing unit system. Processing devices include CPU (Central Processing Unit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and DSP (Digital Processing Unit). al Signal Processor) etc. can be applied.

In some embodiments, memory 4 may include random access semiconductor memory, storage devices, or storage media (either volatile or non-volatile) for storing data and programs. In some embodiments, memory 4 represents the entire virtual memory of computer system 1 and may include virtual memory of other computer systems connected to computer system 1 via a network. Although the memory 4 may be conceptually considered as a single entity, in other embodiments this memory 4 may be a more complex arrangement, such as a hierarchy of caches and other memory devices. . For example, memory may exist as multiple levels of caches, and these caches may be divided by function. As a result, one cache may hold instructions while the other cache holds non-instruction data used by the processor. Memory is so-called NUMA (Non-Uniform)
(Memory Access) computer architecture may be distributed and associated with a variety of different processing devices.

Memory 4 may store all or some of the programs, modules, and data structures that perform the functions described herein. For example, the memory 4 may store a latent factor identification application 50. In some embodiments, latent factor identification application 50 may include instructions or writings that perform functions described below on processor 2, or may include instructions or writings that are interpreted by other instructions or writings. In some embodiments, latent factor identification application 50 may be applied to semiconductor devices, chips, logic gates, circuits, circuit cards, and/or other physical hardware instead of or in addition to processor-based systems. It may also be implemented in hardware via a device. In some embodiments, latent factor identification application 50 may include data other than instructions or descriptions. In some embodiments, cameras, sensors, or other data input devices (not shown) may be provided to communicate directly with bus interface unit 9, processor 2, or other hardware of computer system 1. . Such an arrangement may reduce the need for processor 2 to access memory 4 and the latent factor identification application.

The computer system 1 may include a processor 2 , a memory 4 , a display system 24 , and a bus interface unit 9 for communicating between the I/O bus interface unit 10 . I/O bus interface unit 10 may be coupled to I/O bus 8 for transferring data to and from various I/O units. I/O bus interface unit 10 connects via I/O bus 8 to a plurality of I/

O interface units

12, 14, 16, also known as I/O processors (IOPs) or I/O adapters (IOAs). and 18. Display system 24 may include a display controller, display memory, or both. A display controller may provide video, audio, or both data to display device 26. Computer system 1 may also include devices such as one or more sensors configured to collect data and provide the data to processor 2 . For example, computer system 1 may include environmental sensors that collect humidity data, temperature data, pressure data, etc., motion sensors that collect acceleration data, motion data, etc., and the like. Other types of sensors can also be used. The display memory may be a dedicated memory for buffering video data. Display system 24 may be connected to a display device 26, such as a standalone display screen, a television, a tablet, or a handheld device. In some embodiments, display device 26 may include speakers to render audio. Alternatively, a speaker for rendering audio may be connected to the I/O interface unit. In other embodiments, the functionality provided by display system 24 may be implemented by an integrated circuit that includes processor 2. Similarly, the functionality provided by the bus interface unit 9 may be realized by an integrated circuit including the processor 2.

The I/O interface unit has the ability to communicate with various storage or I/O devices. For example, the terminal interface unit 12 may include a user output device such as a video display device, a speaker television, or a user input device such as a keyboard, mouse, keypad, touch pad, trackball, button, light pen, or other pointing device. It is possible to attach the user I/O device 20 as shown in FIG. Using the user interface, the user inputs input data and instructions to the user I/O device 20 and the computer system 1 by operating the user input device, and receives output data from the computer system 1. Good too. The user interface may be displayed on a display device, played via a speaker, or printed via a printer, for example, via the user I/O device 20.

Storage interface 14 may include one or more disk drives or direct access storage devices 22 (typically magnetic disk drive storage devices, but an array of disk drives or other storage devices configured to appear as a single disk drive). ) can be installed. In some embodiments, storage device 22 may be implemented as any secondary storage device. The contents of the memory 4 may be stored in the storage device 22 and read from the storage device 22 as needed. Network interface 18 may provide a communication path so that computer system 1 and other devices can communicate with each other. This communication path may be, for example, the network 30.

The computer system 1 shown in FIG. 1 includes a bus structure that provides a direct communication path between a processor 2, a memory 4, a bus interface 9, a display system 24, and an I/O bus interface unit 10; In embodiments, computer system 1 may include point-to-point links in a hierarchical, star, or web configuration, multiple hierarchical buses, and parallel or redundant communication paths. Further, although I/O bus interface unit 10 and I/O bus 8 are shown as a single unit, in reality computer system 1 may include multiple I/O bus interface units 10 or multiple I/O A bus 8 may also be provided. Also, although a plurality of I/O interface units are shown for separating the I/O bus 8 from various communication paths connecting various I/O devices, in other embodiments, one of the I/O devices may be Some or all may be directly connected to one system I/O bus.

In some embodiments, computer system 1 is a device that receives requests from other computer systems (clients) that do not have a direct user interface, such as a multi-user mainframe computer system, a single-user system, or a server computer. There may be. In other embodiments, computer system 1 may be a desktop computer, a portable computer, a laptop, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable electronic device.

<Block diagram of monitoring system>
FIG. 2 is a block diagram showing an example of the configuration of the monitoring system of the present invention.

The monitoring system 100 shown in FIG. 2 includes an imaging device 101, an analysis device 102, an alarm device 103, a display device 104, and a status notification section 105.

The imaging device 101 is configured by, for example, a network camera, and is used to acquire images of monitoring areas such as railroad crossings. The imaging device 101 can have a camera configuration that obtains information by forming an image of incident light on an image sensor through a lens or an aperture. Examples of the image sensor here include a CCD (Charge-Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and the like. The imaging device 101 captures video at, for example, 3 frames per second (3 fps) or more, and the information is sent to the analysis device 102 . A plurality of imaging devices 101 can be installed depending on the situation. As the camera, a visible light camera, an infrared camera, or the like can be used.

The analysis device 102 includes a state analysis control section 106 and a state analysis section 107. The analysis device 102 is realized by, for example, a computer equipped with a processor such as a CPU, GPU, FPGA, or DSP, and the computer system 1 of FIG. 1 can be applied, for example. The condition analysis control section 106 controls the start of detection of a dangerous event by the condition analysis section 107 based on the image obtained by the imaging device 101 and the train approach information notified from the condition notification section 105. The state analysis unit 107 analyzes the image obtained by the imaging device 101 and predicts whether a dangerous event such as a moving object such as a person or a car being left behind at a railroad crossing will occur. A specific determination method will be described later.

The warning device 103 is composed of, for example, a speaker, and when the state analysis unit 107 detects a dangerous event, it notifies the surrounding area of the railroad crossing of the danger by sound. Further, as another aspect, it is configured by a protective radio device, and when the state analysis unit 107 detects a dangerous event, it transmits an emergency stop signal to the relevant train, etc.

The display device 104 is a display device that can display surveillance video captured by the imaging device 101. At this time, the details of the dangerous event detected by the state analysis unit 107 can be displayed. The display device 104 can be configured with, for example, a liquid crystal display, an organic EL (OLED) display, or the like. Further, operation means such as a switcher, a keyboard, a mouse, etc. may be attached. The display device 104 may aggregate images from each imaging device 101 to a monitoring center or the like. The display device 104 may be configured at the same location as the analysis device 102, or may be configured at a different location. The display device 104 can be applied as the display device 26 in the example of FIG.

The status notification unit 105 is configured using a railroad crossing controller, an automatic train stop (ATS), and the like. The status notification unit 105 detects the approach of a train to the relevant railroad crossing, and notifies the status analysis control unit 106 of the approach information. This approach information can be determined based on, for example, the position of the train to the relevant level crossing, whether the estimated time until reaching the relevant level crossing is within a predetermined value, etc.

<Example of application of monitoring system>
FIG. 3 is a conceptual diagram showing an example of application of the monitoring system of the present invention.

In FIG. 3, a moving object 130, exemplified by a wheelchair, is shown crossing a crossing area 160 of a railroad crossing 150. The crossing area 160 is the area between the front and rear blocking rods 151 and crosses the track 170. When the moving object 130 crosses the front boundary line 161, it can pass through the crossing area 160 and reach a safe area. A boundary line 161 of the crossing area 160 is a boundary line near the front blocking rod 151.

The imaging device 101 can be installed in a railroad crossing warning device 152, etc., and is installed so that it can photograph the entire crossing area 160. For this reason, it is best to install it at a higher position than a certain level. For example, the height is 2 m or more, which is higher than a person's height. Further, instead of just one imaging device 101, a plurality of imaging devices 101 may be installed diagonally, etc., to photograph the entire railroad crossing 150 including the crossing area 160 from multiple angles.

The warning device 103 is installed near the railroad crossing 150, and can issue a warning to the railroad crossing 150 and people near the railroad crossing 150. In the example of FIG. 3, it is installed in a railroad crossing warning device 152.

The analysis device 102 can be installed at a location different from the railroad crossing 150. Furthermore, the condition analysis unit 107 may be installed near the railroad crossing 150 or may be installed integrally with the analysis device 102.

The display device 104 can be installed at a location different from the railroad crossing 150. For example, it is possible to aggregate the information together with information from the imaging devices 101 at other railroad crossings at a monitoring center or the like.

FIG. 3 shows a case where a dangerous event occurs in which the moving object 130 is likely to be left behind in the crossing area 160 of the railroad crossing 150. In this case, the information is analyzed by the analysis device 102 based on the image photographed by the imaging device 101, and the event is determined to be a dangerous event. Based on this, the alarm device 103 issues a predetermined alarm. Further, on the display device 104, a supervisor can check the video of the event determined to be a dangerous event.

<Flowchart> FIG. 4 shows an example of a flowchart of processing of the monitoring system of the present invention. The processing flow of the monitoring system 100 will be explained using FIG. 4.

First, in step S201, the state analysis control unit 106 detects the approach of a train. The detection here can be carried out by detecting the start of the warning bell or the start of the crossing at the railroad crossing 150. Furthermore, the imaging device 101 constantly transmits railroad crossing surveillance video to the state analysis control unit 106. Therefore, the state analysis control unit 106 can detect the start of shutoff by image analysis. Examples of methods for detecting the start of a cutoff using surveillance video include a method of detecting blinking of a railroad crossing warning from the railroad crossing alarm 152 based on changes in color and brightness, and a method of detecting movement of the cutoff rod 151. In addition to this, the approach of a train may be detected by the status notification unit 105 acquiring information from a level crossing controller, an ATS, or the like. When the approach of a train is detected, the condition analysis control section 106 instructs the condition analysis section 107 to start analyzing the level crossing condition.

Next, in step S202, the state analysis unit 107 detects and tracks moving objects 130 such as people, cars, bicycles, motorcycles, wheelchairs, strollers, canes, and white canes inside the railroad crossing 150. The detection and tracking here can be realized by combining machine learning technology, pattern matching, etc. based on the video captured by the imaging device 101. Machine learning technology can be performed by image analysis using AI (artificial intelligence), deep learning, etc. Pattern matching can be performed, for example, by image analysis, such as by comparing with a template image. Once the moving object 130 is detected, it moves frame by frame on the screen. At this time, the movement history of the detected moving object 130 can be tracked using machine learning or pattern matching to determine whether the object is the same as the detected moving object 130. Furthermore, algorithms other than machine learning and pattern matching may be used to detect and track the moving object 130.

At this time, it is preferable to perform a process of attaching an identification ID to each moving object to identify the detection target. Furthermore, a detection object type ID may be attached to each moving object to identify the type of the object, such as a person, a car, a bicycle, a motorcycle, a wheelchair, a stroller, a cane, a white cane, etc. By associating the ID and the detection target in this way, it can be useful for later-described determination and later analysis of dangerous events.

Next, in step S203, the speed of the moving object 130 is estimated. The speed is estimated by the state analysis unit 107 through image analysis of the video captured by the imaging device 101. A specific example of the speed estimation method will be explained in the determination method described later.

Next, in step S204, it is determined whether the moving object 130 can cross the railroad crossing 150. This determination can be made by the state analysis unit 107 using the speed estimated in step S203. Whether the moving object 130 has crossed the railroad crossing 150 can be determined by, for example, whether it can pass through the crossing area 160 before the crossing is completed. A specific example of the determination method will be described later. In step S204, if the moving object 130 can cross the railroad crossing 150, the process ends. If crossing is not possible (true), the process advances to step S205. In this case, the state analysis unit 107 transmits information on the occurrence of a dangerous event.

In step S205, alarm processing is performed. For example, the warning device 103 alerts the area around the railroad crossing 150 of danger by voice or the like. In addition, the display device 104 may display a warning about the occurrence of a dangerous event. These operations can be performed based on information received from the state analysis unit 107 regarding the occurrence of a dangerous event.

<First determination method>
FIG. 5 is a diagram for explaining the determination method of the monitoring system of the present invention. The first determination method will be explained using FIG. 5.

FIG. 5 shows a state in which a moving object 130 (a wheelchair is exemplified in FIG. 5) is crossing a railroad crossing 150. An imaginary line 301 is determined near the rear boundary in the transverse direction of the crossing area 160 of the railroad crossing 150, and an imaginary line 302 is determined near the front boundary. These can be determined by image analysis. The moving object 130 is present in the traversal area 160, the current time is t _n , and the current position is P _n (x(t _n ), y(t _n )).

The first determination method is to determine that the moving object 130 cannot cross the railroad crossing (true) if the time it takes to cross the railroad crossing 150 is predicted to be longer than the remaining time until the barrier 151 of the railroad crossing 150 closes. This is a judgment method.

If the destination point where the moving object 130 reaches the imaginary line 302 in front is PG (x _G _, y _G ), then the destination point from the current position t _n (x (t _n ), y (t _n )) is PG (x G , y _G ). The distance d(t _n ) to P _G (x _G , y _G ) is expressed by Equation 1 below.

Here, the destination point P _G (x _G , y _G ) is, for example, the position of the virtual line 302 that is the shortest distance ahead from the current position P _n (x (t _n ), y (t _n )). can be taken as the destination point _PG . In addition, since the most recent direction of movement can be calculated by moving from the previous position, the intersection with the virtual line 302 on the extension of the movement vector calculated from this may be set as the destination point _PG .

Next, the current moving speed v(t _n ) is expressed by the following equation 2.

Here, from the position P _n-1 (x(t _n-1 ₎ , y(t _{n-1 )) at the previous time t n-1} to the position P _n (x(t _{n )} ) at the current time t _n , y(t _n )) and the time difference. The immediately preceding time t _n-1 is a predetermined time before the current time t _n , and may be, for example, a predetermined frame interval such as a difference from one frame before or several frames before.

The following equation 3 is a determination condition.

Here, it is determined whether Equation 3 is true or false using the distance d(t _n ) obtained from Equation 1 and the moving speed v(t _n ) obtained from Equation 2 above. Here, τ is the remaining time from the current time t _n until the crossing of the railroad crossing 150 is completed. This τ can be calculated by the state analysis unit 107 based on the information on the railroad crossing, since the time from the start of the railroad crossing to the completion of the railroad crossing is determined in advance.

If the above condition of Equation 3 is satisfied, the moving object 130 will not reach the virtual line 301 before the crossing of the railroad crossing 150 is completed. Therefore, if this condition is met, it becomes true and the moving object 130 cannot cross the railroad crossing 150, so it can be determined that a dangerous event has occurred.

Note that the above example shows an example in which the determination is made only for the virtual line 302 located in front of the moving direction of the moving object 130, but the determination is made for the virtual line 301 located behind the moving object 130 in the moving direction. It's okay. In addition to this, the distance d(t _n ) to the virtual line is applied to both the backward virtual line 301 and the forward virtual line 302, and if both are true, it is determined that crossing is not possible. It's okay. This also takes into account the possibility that the moving object 130 can turn back.

<Second determination method>
The second determination method determines whether the amount of movement of the detected object (moving object 130) at a certain time is less than or equal to a threshold value under predetermined conditions. In this case, it can be assumed that the vehicle is turning back, wandering around, or moving at low speed, so this method determines that crossing is not possible (true) in such a case. Specifically, it can be expressed by the following equation 4.

Of the above equations 4, equation 4-1, which is the first condition shown below, will be explained.

Here, v(t _n ) is the speed at the current time calculated using Equation 2 above. V _th is a threshold value of the amount of movement from time t=t _m to t=t _n . The time t _m is a time before the current time t _n , for example, 2 seconds ago, further 5 seconds ago, or even 10 seconds ago. In other words, it can be a value between about 1 second and 10 seconds ago. Equation 4-1 indicates that the total amount of movement from the previous time t _m to the current time t _n is less than or equal to a predetermined movement amount threshold. As a result, if the amount of movement within the predetermined time is less than or equal to the predetermined value, this formula is met. Equation 4-1 uses the total amount of movement for simplicity, but a velocity vector that takes the direction of movement into consideration may be used. In this case, the resultant vector is the position P _m (x (t _m ), y (t _m )) at time t = t _m and the position P _n (x (t _{n ), y (t n} ₎ ) at t = t _n . ) represents the movement direction and movement distance, so if the movement distance is short, it can be considered that the movement is staying near the position _Pm .

Of the above equations 4, equation 4-2, which is the next condition shown below, will be explained.

Here, τ is the time from the current time t _n until the crossing of the railroad crossing 150 is completed. T _th1 is a threshold value for the remaining time to complete the cutoff. In other words, if there is little time remaining until the crossing of the level crossing 150 is completed, the condition of this equation is satisfied because the situation is dangerous.

Of the above equations 4, equation 4-3, which is the last condition shown below, will be explained.

Here, M(x(t _n ), y(t _n )) indicates whether the position of the detection target is in the detection area at the current time. The detection area M is assumed to be, for example, the crossing area 160, and is assumed to be an area between the virtual line 301 and the virtual line 302 shown in FIG. 5. Here, if it is the detection area M(x, y), it will be "1", and if it is any other area, it will be "0". Equation 4-3 is 1 when the detection target (moving object 130) is within the detection area M, and the condition of this equation is satisfied. Note that the detection area M(x, y) may be set in a predetermined area within the railroad crossing, but may also be set to include a prohibited area such as a track area outside the railroad crossing.

The above equation 4 is determined to be true when all of the above equations 4-1, 4-2, and 4-3 are satisfied, and it can be determined that a dangerous event has occurred. However, the degree of risk may be indicated in stages when some of these three equations are satisfied. For example, if Equations 4-1 and 4-3 are satisfied, it is determined that a dangerous event has occurred, and if Equation 4-2 is further satisfied, it is determined that a dangerous event has occurred.

<Third determination method>
The third determination method is a method of determining that some abnormality has occurred and that crossing is not possible (true) when tracking disappears. For example, this may occur if the moving object 130 to be detected moves out of the detection area by intruding onto the tracks outside the crossing area 160 of the railroad crossing 150, or if it hides behind a structure such as a fence or utility pole. . In this case, since the imaging device 101 cannot capture the detection target, tracking may be lost. At this time, it is assumed that the detection target has not moved from the position where the detection target was last detected, and it is treated as v=0. Then, when the state of v=0 continues for a predetermined time T _th2 , it is determined that crossing is not possible (true), and it is determined that a dangerous event has occurred. The predetermined time T _th2 can be determined in advance.

Note that the method of treating the disappearance of tracking as v=0 can also be applied to the first determination method and the second determination method. In the first determination method, for example, if v(t _n )=0 in equation 3, the value on the left side of the equation becomes infinite, and the condition of equation 3 is satisfied. This allows it to be determined that a dangerous event has occurred. In the second determination method, for example, if v(t _n )=0 in Equation 4, the left side of Equation 4-1 becomes 0, and the condition of Equation 4 is satisfied. This allows it to be determined that a dangerous event has occurred.

<Fourth determination method>
The fourth determination method is the determination using the detection target type described in step S202. The detection target type can be determined by image analysis using AI (artificial intelligence), deep learning, or the like. At this time, a determination is made according to the type of detection target. Examples of detection target types include people (adults, children, elderly people), cars, bicycles, motorbikes, wheelchairs, strollers, canes, white canes, and the like. For example, the threshold values (V _{th ,} T _th1 , T _th2 ) used in the second and third determination methods may be changed and set in advance for each detection target type.

Furthermore, for each detection target type, based on the current moving speed v (t _n ) and position P _n (x (t _n ), y (t _n )), the arrival to the

virtual line

301 or 302 is determined based on machine learning. Anticipate the time. Then, if the expected arrival time is longer (slower) than the completion time of the railroad crossing 150, or if the probability that it will be longer (slower) is greater than a predetermined value, it may be determined that a dangerous event has occurred. . In this case, the arrival time for that type can be predicted by using past data at the relevant level crossing and data at other level crossings.

Additionally, if a specific event occurs for each detection target type, it may be determined that a dangerous event has occurred. For example, in the case of a car, if the car is stopped at a railroad crossing for a predetermined period of time, it is possible that the car in front of the car is stuck in a traffic jam and cannot move, or that the car is unable to move due to a problem such as a breakdown of the car. For example, if the wheelchair is stopped near a track groove or in the direction of the track in a crossing zone, there is a possibility that the wheelchair has fallen off and is unable to move. For example, if a child is smaller than a predetermined age and is alone at a railroad crossing, there is a possibility that the child may not be able to cross the railroad crossing on his or her own judgment. In such a case, regardless of the first to third determination methods, the event may be determined to be a dangerous event or a state with a high possibility of becoming a dangerous event.

<Display example>
FIG. 6 is a diagram showing an example of display on the first screen in the monitoring system of the present invention. FIG. 7 is a diagram showing an example of display on the second screen in the monitoring system of the present invention.

The first screen shown in FIG. 6 and the second screen shown in FIG. 7 are displayed by the display device 104. These examples show an example of highlighting or enlarging a video of interest from among a plurality of displayed surveillance videos. Although any display method is possible, the following is a configuration for the observer to notice the danger.

The first display method is the method shown in FIG. In the display screen 400 of FIG. 6, a main display 410 in which a plurality of images for each imaging device 101 are arranged in tiles is displayed on the left side. In other words, images of different railroad crossings are displayed side by side. The main display 410 in FIG. 6 shows an example in which nine videos are displayed in three horizontal columns and three vertical columns. Further, a thumbnail display 420 is provided on the right side of the main display 410. The thumbnail display 420 is displayed in a smaller area than the main display 410, and each video display is also smaller than the main display 410. In the thumbnail display 420, for example, images from the imaging device 101 that cannot be displayed on the main display 410 are displayed in an enlarged manner. The thumbnail display 420 in FIG. 6 shows an example in which three videos are displayed vertically side by side.

There are two possible display stages. One is when the approach of a train is detected in step S201 in FIG. This may also be the start of shutoff. The other is the stage in which a dangerous event is determined in step S204 in FIG. It is possible to display a different emphasis on a stage where a train is approaching and a stage where a dangerous event has been determined as compared to a normal stage. For example, at the stage when the approach of a train is detected, the image from the corresponding imaging device 101 may be emphasized by some kind of highlight display. In this case, the corresponding video may be moved from the thumbnail display 420 in FIG. 6 to the main display 410. Then, a specific highlight display 411 is performed at the stage where the event is determined to be a dangerous event. The highlight display 411 is divided into colors, such as a yellow frame for an approaching train and a red frame for a dangerous event, and displays a warning using different colored frames.

Additionally, images that cannot be displayed on the main display 410 may be highlighted and displayed on the thumbnail display 420 in the same manner as the highlight display 411. Further, images showing a train approaching and a dangerous event may be displayed side by side so that they are displayed higher than other images. For example, such images may be arranged from the left side of the upper row of the main display 410. In this case, the video of the dangerous event may be displayed at a higher priority than the video of the approaching train.

In the second display method, a highlight display 411 of an image of an approaching train as shown in FIG. 6 is performed in the same way as in the first display method. However, when it is determined that it is a dangerous event, the screen changes to a second display screen 500 as shown in FIG. Here, the video determined to be a dangerous event is displayed as an enlarged display 511 on the main display 510 as shown in FIG. For example, as shown in FIG. 7, the enlarged display 511 uses four normal image display spaces to display one image in an enlarged manner. Further, the thumbnail display 420 on the right side in FIG. 7 is displayed in the same manner as in FIG. 6 .

Note that the first display method and the second display method described above may have a function of canceling the emphasis or enlarged display of the dangerous event after detecting the dangerous event. This assumes that safety has been confirmed by a supervisor or the like. The display device 104 may be provided with an interface for this cancellation operation. In this case, once the highlighted or enlarged display is started, the highlighted or enlarged display may be continued until it is manually canceled.

In addition, in the first display method and the second display method, videos in which a shutdown is being started and videos of dangerous events are preferentially displayed on the

main displays

410 and 510, and videos in other states are displayed as thumbnail displays 420. You may. This allows videos of particular interest to be displayed on the

main displays

410, 510. In addition, the

main displays

410 and 510 may exclusively allocate the video that is starting to be cut off and the video in other states to each tile-shaped display portion. Furthermore, each tile-shaped display portion can be allocated for cyclic display.

<Effect>
According to the embodiments described above, it is possible to accurately detect a dangerous event at a railroad crossing by analyzing the image captured by the imaging device. At this time, by detecting the approach of a train, it can be detected before the crossing is completed. In addition, accurate detection is possible by detecting whether a moving object can cross a railroad crossing. Furthermore, since an imaging device is used, it is possible to detect the entire condition inside the railroad crossing. Furthermore, it can be realized without using other sensors such as a laser irradiation type sensor, and costs can be reduced. Furthermore, the images from the imaging device can be viewed directly by surveillance personnel to visually confirm the situation. In addition, by emphasizing or enlarging and displaying a video of interest from the surveillance video on the display device, it is possible to display the video so that the surveillance staff can easily notice the video.

Note that the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the embodiments described above are described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add, delete, or replace a part of the configuration of each embodiment with other configurations.

For example, although the above embodiment has been described with respect to a railroad crossing, it is also applicable to other situations in which the predicted arrival time to the virtual line is used. For example, it can be applied to signal control at intersections, automatic door opening/closing control, elevator door opening/closing control, etc.

Furthermore, although the above embodiment shows an example in which the detection target is tracked using images from a camera, it is also possible to use a configuration in which various sensors such as LiDAR and millimeter wave radar are additionally used for detection and tracking to improve detection accuracy. good.

DESCRIPTION OF SYMBOLS 1... Computer system, 2... Processor, 2A, 2B... Processing device, 4... Memory, 6... Memory bus, 8... I/O bus, 9... Bus interface unit, 10... I/O bus interface unit, 12... Terminal Interface unit, 14...Storage interface, 16...I/O device interface, 18...Network interface, 20...User I/O device, 22...Storage device, 24...Display system, 26...Display device, 30...Network, 50... Latent factor identification application, 100... Monitoring system, 101... Imaging device, 102... Analysis device, 103... Alarm device, 104... Display device, 105... Status notification unit, 106... Status analysis control unit, 107... Status analysis unit, 130 ...moving object, 150...level crossing, 151...blocking rod, 152...level crossing alarm, 160...crossing area, 161...boundary line, 170...railway, 301, 302...virtual line, 400...display screen, 410...main display, 411...Highlight display, 420...Thumbnail display, 500...Second display screen, 510...Main display, 511...Enlarged display

Claims

Equipped with an imaging device and an analysis device,
The analysis device includes a state analysis control section and a state analysis section,
When the condition analysis control section detects that a train is approaching based on the analysis of the image acquired by the imaging device or the information notified from the condition notification section, the condition analysis control section causes the condition analysis section to start analyzing the image inside the railroad crossing. instruct the
The state analysis unit detects and tracks a moving object within the railroad crossing using the image acquired by the imaging device, and uses at least the current moving speed of the moving object to determine whether the detection target has been set by the time the crossing of the railroad crossing is completed. A monitoring system that determines that a dangerous event has occurred and issues an alarm if it is determined that the virtual line cannot be reached.
The monitoring system according to claim 1,
The state analysis unit links and tracks an ID that identifies the detection target and an ID that identifies the type of the detection target by identifying the type by image analysis for each detection target,
A monitoring system characterized in that the types include at least a person, a car, a bicycle, a wheelchair, a stroller, a cane, and a white cane.
The monitoring system according to claim 1,
The state analysis unit calculates the time required for the moving object to reach the virtual line from the current moving speed of the moving object and the distance from the current position of the moving object to the virtual line, and calculates the time required for the moving object to reach the virtual line. A monitoring system that determines that a dangerous event has occurred if the time is longer than the remaining time until the crossing is completed.
The monitoring system according to claim 1,
The state analysis unit is configured to determine that the moving object exists within the railroad crossing, the amount of movement of the moving object at the current time is less than or equal to a predetermined threshold, and the time until the crossing of the railroad crossing is completed is within the predetermined threshold. A monitoring system that determines that a dangerous event has occurred.
The monitoring system according to claim 1,
The state analysis unit predicts the arrival time to the virtual line based on the current position and speed of the moving object according to the type, and if the predicted arrival time is longer than the crossing completion time of the crossing, it is determined that a dangerous event has occurred. A monitoring system characterized by determining that an occurrence has occurred.
The monitoring system according to claim 1,
comprising a display device, the display device displays images from the plurality of image pickup devices side by side, and highlights or enlarges the image of the image pickup device determined by the condition analysis unit to be in the occurrence of a dangerous event; A monitoring system characterized by: