JP2019008474A - Monitor supporting system and monitor supporting method - Google Patents

Abstract

To sufficiently monitor a road facility.SOLUTION: A monitor supporting system comprises: multiple cameras 4 that are installed in a tollgate T; an overhead view image generation part 12 for generating overhead view images from camera images that are picked up by the multiple cameras 4; a combined overhead view image generation part 13 for generating a combined overhead view image by combining the overhead view images that are generated by the overhead view image generation part 12; an abnormality detection processing part 22 for detecting an abnormal state of an object from the camera images that are picked up by the multiple cameras 4; and an image processing part 33 for emphasizing and displaying the object, in which the abnormal state is detected, in the combined overhead view image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technology for a monitoring support system and a monitoring support method for supporting monitoring of road facilities.

  In recent years, with the development of network technology, the efficiency of operation and maintenance has been increasing in various fields. For example, in the road field, an ETC (Electronic Toll Collection) system is already widely used as a toll collection system, and there is no need to place a toll member in a booth inside the island. Remote monitoring is performed at such toll booths where there are no tollers.

  In Patent Document 1, “a plurality of monitoring cameras that monitor the first lane (11a) of the first toll booth 10 on the toll road (1) and output monitoring image real-time data that is real-time data of the monitoring image ( 501 to 506), the first toll collection device (3) installed in the first lane, the first toll collection device and the plurality of monitoring cameras, and the first toll gate A first monitoring server (6) installed in a remote monitoring device (7) connected to the first monitoring server via a network (999), the remote monitoring device comprising: The toll road toll gate remote monitoring system includes a first display unit (74) for displaying whether or not a failure has occurred in the first toll collection device, and a first operation unit (75). Disclosed (see summary).

  Patent Document 2 states that “a plurality of in-vehicle cameras 11, 12, 13, and 14 capture images around the vehicle, and a plurality of captured images captured by these in-vehicle cameras 11, 12, 13, and 14, respectively. By converting and connecting to a bird's-eye view image, the first to fourth sonars 21, 22, 23, and 24 are used to display the images as a continuous bird's-eye view image on the display device 40. An overhead image display system and a method for displaying an overhead image are disclosed in which, when it is detected that an obstacle exists in the corresponding area, the position of the joint portion of the overhead image is changed. .

JP 2007-114829 A JP 2007-41791 A

  However, when trying to implement the technique described in Patent Document 1 for the purpose of further efficiency, the following problems arise. That is, in the technique described in Patent Document 1, since the shooting range is narrow only with the lane monitoring camera that is shooting a portion where the vehicle is passing, a camera for shooting a wide area around the toll gate is separately provided. Many need to be installed. In a control system that distributes video from multiple cameras, monitoring can be performed using video that is simply reduced in size due to restrictions on the network line of the system, and further divided into multiple screens using the reduced video. Done. In that case, the video monitored by the supervisor only displays each part of the toll gate. That is, it is difficult for the monitoring staff to grasp where each screen shows the toll booth.

  For this reason, when an observer occurs in a remote control room and an abnormality occurs in the delivered video and the detailed location of the abnormality occurrence point is specified, the camera arrangement diagram of each toll gate must be stored in advance. There must be. In addition, when monitoring a plurality of toll gates, it is necessary to keep an eye on the camera video that is unilaterally distributed, which increases the burden of management work for the supervisor.

  The problem of Patent Document 2 will be described in the embodiment.

  The present invention has been made in view of such a background, and an object of the present invention is to efficiently monitor road facilities.

  In order to solve the above-described problem, the present invention provides a plurality of imaging units provided in a road facility, an overhead image generation unit that generates an overhead image from captured images captured by each of the plurality of imaging units, A combined overhead image generation unit that generates a combined overhead image by combining the overhead image generated by the overhead image generation unit, and an abnormality that detects an abnormal state of the object from the captured images captured by the plurality of imaging units And a video processing unit that highlights the object in which the abnormal state has been detected in the synthesized overhead view video.

  According to the present invention, it is possible to efficiently monitor road facilities.

It is a figure which shows the example of whole structure of the monitoring assistance system used by 1st Embodiment. It is a figure which shows the structure of the computer used by 1st Embodiment. It is the figure which showed the example of arrangement | positioning of the camera in the toll booth used in 1st Embodiment. It is the figure which showed the example of the image | video for a display which the video server of the monitoring assistance system used by 1st Embodiment produces | generates. It is a figure which shows the example of a screen which a monitoring person confirms in the control room of the monitoring assistance system used by 1st Embodiment. It is a flowchart which shows the process sequence in the bird's-eye view image processing apparatus used by 1st Embodiment. It is a flowchart which shows the process sequence in the abnormality detection server used by 1st Embodiment. It is a flowchart which shows the process sequence in the video server used by 1st Embodiment. It is a figure which shows the example of a whole structure of the monitoring assistance system used by 2nd Embodiment. It is a figure (the 1) which shows the example of the synthetic | combination overhead view image | video used by 2nd Embodiment. It is FIG. (1) for demonstrating the movement of a boundary. It is FIG. (2) for demonstrating the movement of a boundary. It is a figure (the 2) which shows the example of the synthetic | combination overhead view image | video used in 2nd Embodiment. It is a flowchart which shows the process sequence of the bird's-eye view image processing apparatus used by 2nd Embodiment. It is a flowchart which shows the process sequence of the abnormality detection server used by 2nd Embodiment. It is a block diagram which shows the example of whole structure of the monitoring assistance system used by 3rd Embodiment. It is a flowchart which shows the process sequence of the abnormality detection server used by 3rd Embodiment. It is a flowchart which supports the process sequence of the video server used by 3rd Embodiment. It is a figure which shows the image | video for a display in the monitoring assistance system used in 5th Embodiment.

  Hereinafter, modes for carrying out the present invention (hereinafter referred to as “embodiments”) will be described with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description will be omitted as appropriate.

<< First Embodiment >>
[Monitoring support system Z]
FIG. 1 is a diagram illustrating an example of the overall configuration of a monitoring support system Z used in the first embodiment.
In the example shown in FIG. 1, a form in which a plurality of (three in the example of FIG. 1) toll gates T (T1 to T3) are monitored in the control room C is shown.
In the example of FIG. 1, the case where there is one control room C and three toll booths T is illustrated. However, one control room C and three toll gates T have no special meaning, and the number of control rooms C and the number of toll gates T are not limited to the example in FIG. That is, the viewpoints of the functions of the monitoring support system Z are basically the same even if the number of control rooms C and the number of toll gates T are other numbers.

  In the toll gate T, a plurality of cameras (imaging units) 4, an overhead video processing device 1, an abnormality detection server 2, and a video server 3 are installed.

  The plurality of cameras 4 are installed in such a manner that there is no blind spot with respect to the necessary monitoring area at the toll booth T and the shooting ranges of the cameras 4 overlap as much as possible.

  Camera images (captured images) captured by each camera 4 are input to the overhead view image processing apparatus 1. The overhead video processing apparatus 1 generates an overhead video based on the camera video input from each camera 4. Note that the generated overhead view video is not limited to the overhead view as seen from directly above the toll booth T, but it is sufficient that the entire toll booth T can be seen at a glance. Such a bird's-eye view image may be used. Note that the overhead view video is generated by a known method such as projective transformation.

  Furthermore, the bird's-eye view video processing device 1 synthesizes the bird's-eye view video generated from the respective camera videos to form one synthesized bird's-eye view video.

  In addition, the camera video captured by the camera 4 is input to the abnormality detection server 2. The abnormality detection server 2 detects a moving object and an abnormal situation from each input camera image by using an image recognition technique. Further, the abnormality detection server 2 performs a process of calculating the position of the detected abnormal object in the camera video. Objects and events that are detected abnormally are, for example, vehicles that have stopped on the road before and after passing through the toll gate T, fallen objects on the road, people entering and leaving the lane, vehicles that run backward in the lane, etc. . As a specific method of abnormality detection, an optimum method is used depending on the object or event to be detected. Various known methods such as a background subtraction method and pattern matching based on feature amount extraction are used as an anomaly detection method.

  Based on the abnormality detection result transmitted from the abnormality detection server 2, the position information of the abnormal object, and the like, the video server 3 displays a frame and the like for emphasizing the abnormality occurrence position, and the combined overhead transmitted from the overhead video processing apparatus 1. Superimpose on video. Thereby, the video server 3 generates the highlighted display video. Further, the video server 3 performs display video transmission processing to the control server 6 via the network 5.

The video server 3 at each toll gate T is connected to the control server 6 in the control room C via the network 5.
The control server 6 displays the display video transmitted from the video server 3 of each toll gate T on a monitor (display unit) 61 installed in the control room 6. In the example of FIG. 1, a monitor 61 is installed to correspond to each toll booth T.

  Next, details of the overhead video processing apparatus 1, the abnormality detection server 2, and the video server 3 will be described.

(Overhead video processing device 1)
The overhead video processing apparatus 1 includes an information receiving unit 11, an overhead video generation unit 12, a combined overhead video generation unit 13, and an information transmission unit 14.
The information receiving unit 11 receives information (camera images and the like) from other devices (camera 4 and the like).
The overhead view video generation unit 12 generates an overhead view video from each received camera video.
The synthesized bird's-eye view video generation unit 13 connects each bird's-eye view video generated by the bird's-eye view video generation unit 12 to generate one synthesized bird's-eye view video.
The information transmission unit 14 transmits information (such as a composite overhead image) to another device (such as the video server 3).

(Abnormality detection server 2)
The abnormality detection server 2 includes an information reception unit 21, an abnormality detection processing unit 22, and an information transmission unit 23.
The information receiving unit 21 receives information (camera video and the like) from other devices (camera 4 and the like).
The anomaly detection processing unit 22 detects an anomaly from each camera video using the above-described image recognition process, and further extracts anomaly information including the position information of the abnormal object from the camera video.
The information transmission unit 23 transmits information (abnormal information or the like) to another device (video server 3 or the like).

(Video server 3)
The video server 3 includes an information receiving unit 31, a determination processing unit 32, a video processing unit 33, and an information transmission unit 34.
The information receiving unit 31 receives information (composite overhead image, abnormality information, etc.) from other devices (overhead image processing device 1, abnormality detection server 2, etc.).
The determination processing unit 32 performs various determinations.
The video processing unit 33 generates a display video in which an abnormal object is highlighted on the synthesized overhead video based on the abnormal information.
The information transmission unit 34 transmits information (display video or the like) to another device (control server 6 or the like).

[Hardware configuration]
FIG. 2 is a diagram illustrating a configuration of the computer 100 used in the first embodiment.
The computer 100 corresponds to the overhead video processing device 1, the abnormality detection server 2, and the video server 3 of FIG.
The computer 100 includes a memory 101, a CPU (Central Processing Unit) 102, a storage device 103 such as an HD (Hard Disk), a communication device 104, and the like.
The communication device 104 is a device that communicates with other devices.
The units 11 to 14, 21 to 23, and 31 to 34 in the overhead view video processing device 1, the abnormality detection server 2, and the video server 3 in FIG. 1 are the programs stored in the storage device 103 in each of the devices 1 to 3. The program is implemented by being loaded into the memory 101 and executed by the CPU 102.
The same applies to the overhead video processing apparatus 1, the abnormality detection server 2, and the video server 3 in the second and third embodiments.

(Camera placement)
FIG. 3 is a diagram illustrating an arrangement example of the cameras 4 at the toll gate T used in the first embodiment.
FIG. 3 shows an example of a toll gate T having a total of four lanes, two entrance lanes and two exit lanes. As shown in FIG. 3, four cameras 4 </ b> A to 4 </ b> D (4) are installed on the illumination columns PA to PD (P). Moreover, four cameras 4E-4H (4) are installed under the roof. That is, in the example of FIG. 3, a total of eight cameras 4 are installed. Although FIG. 3 shows a configuration in which the camera 4 is installed under the roof of the lighting column P and the toll gate T, the installation location is not limited to these. For example, a dedicated pillar of the camera 4 may be installed, or the camera 4 may be installed on the roof of the toll gate T.

(Example of display video)
FIG. 4 is a diagram illustrating an example of the display video 201 generated by the video server 3 of the monitoring support system Z used in the first embodiment.
The display video 201 is a video of the toll gate T shown in FIG.
FIG. 4 shows an example of a display image 201 generated from the camera images of the eight cameras 4A to 4H (4) arranged in the arrangement example of FIG. Here, in the display video 201, a broken-line frame M for highlighting an abnormal object is displayed on the composite overhead video 251 generated by connecting the eight overhead videos 200A to 200H (200). It is what. Each of the bird's-eye view images 200A to 200H is generated based on the camera images captured by the cameras 4A to 4H in FIG.

  Here, the synthesized bird's-eye view image 251 is one in which the frame line M is not displayed. The display video 201 is obtained by displaying the frame line M on the synthesized overhead video 251. That is, the difference between the synthesized overhead view image 251 and the display image 201 is the presence or absence of the frame line M.

  A frame line M indicates a portion where the abnormal state detected in the abnormality detection processing unit 22 (see FIG. 1) in the abnormality detection server 2 has occurred. In the example of the display video 201 shown in FIG. 4, an example is shown in which a vehicle stopped in the stop prohibited area after passing through the exit of the toll gate T is detected as abnormal.

(Video in control room C)
FIG. 5 is a diagram illustrating an example of a screen that a supervisor checks in the control room C of the monitoring support system Z used in the first embodiment.
Here, an example is shown in which three toll gates T1 to T3 (see FIG. 1) are monitored in the control room C (see FIG. 1).
In the control room C, three monitors 61a to 61c (61) are installed. Display images 201a to 201c (201) of the toll gates T1 to T3 are displayed on the respective monitors 61a to 61c. The display videos 201a to 201c correspond to the display videos 201 of the toll gates T1 to T3, respectively.

  For example, when a certain abnormality is detected at the toll gate T1, a frame line M that highlights the abnormality occurrence location appears in the display video 201a displayed on the monitor 61a. By the appearance of the frame line M, the monitor can instantly recognize that an abnormality has occurred at the toll booth T1.

Further, since each of the display images 201a to 201c is a bird's-eye view of the monitoring range at each toll booth T, the monitor can quickly determine not only where the abnormality has occurred but also at which toll booth T where the abnormality has occurred, Can be easily grasped.
As described above, according to the first embodiment, the monitor can quickly and easily grasp the abnormality occurrence and the occurrence position at the toll gate T. Therefore, according to the first embodiment, it is possible to reduce the load and increase the efficiency of monitoring work, such as shortening the transition time to necessary work after occurrence of an abnormality at the toll booth T.

  In the example shown in FIG. 5, the number of monitors 61 (61a to 61c) corresponding to the number of toll booths T is installed in the control room C, but the number of monitors 61 is not limited. For example, the display video 201 corresponding to a plurality of toll booths T may be divided and displayed on one monitor 61, or the display video 201 of each toll booth T on one monitor 61 is time-shared. May be displayed. The time division display is a display in which the display video 201 of each toll booth T is replaced.

[flowchart]
Hereinafter, processing performed in the overhead video processing apparatus 1, the abnormality detection server 2, and the video server 3 will be described. In the description of each flowchart, FIG. 1 is referred to as appropriate.
(Overhead video processing device 1)
FIG. 6 is a flowchart showing a processing procedure in the overhead video processing apparatus 1 used in the first embodiment.
First, the information receiving unit 11 acquires a camera video from each camera 4 (S101).
Next, the bird's-eye view image generation unit 12 generates a bird's-eye view image from each camera image acquired from each camera 4 using a known technique such as projective transformation (S102).
And the synthetic | combination bird's-eye video production | generation part 13 synthesize | combines each bird's-eye view video produced | generated by step S102, and produces | generates one synthetic | combination bird's-eye video (S103).
Thereafter, the information transmission unit 14 transmits the generated synthesized overhead video to the video server 3 (S104).
Thereafter, the overhead image processing apparatus 1 returns the process to step S101.

(Abnormality detection server 2)
FIG. 7 is a flowchart showing a processing procedure in the abnormality detection server 2 used in the first embodiment.
First, the information receiving unit 21 acquires a camera video from each camera 4 (S201).
Then, the abnormality detection processing unit 22 determines whether an abnormality has been detected in the camera video (S202). Abnormal is whether there is an object running at a speed outside the predetermined range based on the difference between frames of the camera image, whether there is an object running in the reverse direction, the size is larger than the predetermined size Whether there is a small moving object (such as a person).

When no abnormality is detected as a result of step S202 (S202 → No), the abnormality detection server 2 returns the process to step S201.
If an abnormality is detected as a result of step S202 (S202 → Yes), the abnormality detection processing unit 22 extracts abnormality information from the camera video in which the abnormality is detected (S203). The abnormality information is the camera number of the camera 4 that detected the abnormality, the coordinate information of the video range in which the abnormality was detected, and the like.
Thereafter, the information transmission unit 23 transmits the extracted abnormality information to the video server 3 (S204), and the abnormality detection server 2 returns the process to step S201.

(Video server 3)
FIG. 8 is a flowchart showing a processing procedure in the video server 3 used in the first embodiment.
First, the information receiving unit 31 of the video server 3 receives the composite overhead video from the overhead video processing device 1 (S301).
And the determination process part 32 determines whether abnormality information was received from the abnormality detection server 2 (S302).
If the abnormality information is received as a result of step S302 (S302 → Yes), the video processing unit 33 calculates the position of the object (abnormal object) in which the abnormality is detected on the combined overhead image (S303). That is, the video processing unit 33 determines where in the composite overhead view video the object from which an abnormality has been detected corresponds based on the camera number included in the abnormality information and the coordinate information of the video range in which the abnormality has been detected (on the composite overhead video). Position).
Then, the video processing unit 33 generates a display video in which the video range calculated in step S303 is highlighted (S304).

On the other hand, when the abnormality information is not received as a result of step S302 (S302 → No), the video processing unit 33 generates the display video by using the synthesized overhead video as it is as the display video (S305).
Then, the information transmitting unit 34 transmits the display video to the control server 6, and the video server 3 returns the process to step S301.

  The control server 6 displays the display video transmitted to the monitor 61.

In conventional monitoring support systems, since the camera video is simply displayed, it is difficult to grasp which part of the toll booth T the captured video is. For this reason, it is difficult for the supervisor to specify where the occurrence of the abnormality occurs at the toll gate T.
In the monitoring support system Z of the first embodiment, as shown in FIG. 6, since all of the toll booth T can be monitored with an image seen from above, an abnormality occurs at the toll booth T. Can be easily grasped.

In addition, it is possible to easily grasp where the abnormality has occurred at the toll booth T by highlighting the abnormality-occurring object such as the frame line M (see FIG. 4) on the synthesized overhead view video.
By doing in this way, the toll gate T (road facility) can be monitored efficiently.
That is, according to the monitoring support system Z of the first embodiment, the monitor can accurately and easily grasp the occurrence of an abnormality at the toll booth T and the occurrence position of the abnormality. Therefore, by using the monitoring support system Z of the first embodiment, it is possible to reduce the load and increase the efficiency of the monitoring work, such as shortening the transition time to the necessary work after that.

  Furthermore, by applying the monitoring support system Z to the toll gate T, the monitoring efficiency at the toll gate T can be improved.

<< Second Embodiment >>
Next, as a second embodiment of the present invention, as shown in FIG. 9, a boundary (boundary part) of a display range in each camera video in the overhead view video is provided to the overhead view video processing device 1a and the abnormality detection server 2a. A monitoring support system Za to which a function to be adjusted is added will be described.

[Monitoring support system Za]
FIG. 9 is a diagram illustrating an example of the overall configuration of the monitoring support system Za used in the second embodiment. In FIG. 9, the same components as those in FIG.
In the toll booth Ta (T1a to T3a) of the monitoring support system Za shown in the second embodiment, when the abnormal object is at the boundary portion in the overhead image, the abnormality information extracted by the abnormality detection server 2a is sent to the overhead image processing device 1a. Has been sent.
Then, the overhead image processing apparatus 1a adjusts the boundary by moving the boundary when generating the overhead image based on the input abnormality information. The movement of the boundary will be described later.
Next, portions of the overhead image processing device 1a and the abnormality detection server 2a according to the second embodiment that are different from the configuration shown in FIG. 1 will be described. The configuration of the video server 3 is the same as that shown in FIG.

(Overhead video processing apparatus 1a)
The overhead video processing apparatus 1a is different from the overhead video processing apparatus 1 shown in FIG. 1 in that it includes a determination processing unit 15 and a boundary adjustment unit 16.
The determination processing unit 15 performs various determinations.
The boundary adjustment unit 16 adjusts the boundary by moving the boundary at the time of generating the overhead view video based on the abnormality information transmitted from the abnormality detection server 2a. The movement of the boundary will be described later.

(Abnormality detection server 2a)
The abnormality detection server 2a is different from the abnormality detection server 2 shown in FIG.
The determination processing unit 24 performs various determinations.
Further, the abnormality detection processing unit 22 determines whether or not the detected moving object is on the boundary when the overhead view video is generated.

[Example of composite overhead image 251]
With reference to FIGS. 10-13, the synthetic | combination overhead view image 251 used in 2nd Embodiment is demonstrated. 10 and 13, similar to FIG. 4, the overhead view images 200 </ b> A to 200 </ b> H are generated based on camera images taken by the cameras 4 </ b> A to 4 </ b> H of FIG. 3.
The synthesized bird's-eye view image 251a (251) shown in FIG. 10 is in a state where the processing used in the second embodiment is not performed. FIG. 10 illustrates an example in which the vehicle 203 is abnormally stopped at the boundary position 202 between the overhead view video 200A and the overhead view video 200B. The vehicle 203 is detected as an abnormal object because it is stopped on the center line.

In the processing shown in the first embodiment, there is a defect that the height direction information of the object at the boundary position 202 is lost as in the synthesized overhead view image 251a of FIG. For this reason, at first glance, it appears that the distorted vehicle is stopped. If the object is displayed in a distorted shape as in the vehicle 203, the observer will feel uncomfortable.
The boundary 202a will be described later.

This will be described with reference to FIG. 11 and FIG. Reference is made to FIG. 10 as appropriate.
An image B1 shown in FIG. 11 is a schematic example of a camera image taken by the camera 4B of FIG. 3, and an image B2 shown in FIG. 12 is a schematic example of a camera image taken by the camera 4A of FIG. 11 and 12, for the sake of simplicity, it is assumed that the side of the vehicle is reflected, but in reality, the images are images of the vehicle taken from diagonally above.

  When the boundary is not moved (the method of the first embodiment), the overhead image generation unit 12 combines the images (parts indicated by dots) obtained by cutting the images B1 and B2 at the boundaries L11 and L12, thereby combining the overhead images 251a. Is generated. Therefore, the synthesized bird's-eye view image 251a is an image in which a predetermined portion of the vehicle (object) is cut as indicated by reference numeral 203 in FIG. For this reason, the object is displayed in a distorted shape like the vehicle 203 in FIG.

  Therefore, in the second embodiment, the overhead video generation unit 12 moves the boundary L12 of the video B2 to the boundary L22. That is, the bird's-eye view image generation unit 12 moves the boundary L12 to the boundary L22 so that the entire abnormal object enters. And with this movement, the overhead view video generation unit 12 moves the boundary L11 of the video B1 to the boundary L21. Then, the bird's-eye view image generation unit 12 generates a bird's-eye view image based on the hatched portion images in FIGS. 11 and 12 generated along with the movement of the boundary, and further generates a composite bird's-eye view image 251. By doing in this way, the synthetic | combination bird's-eye view image 251 can be produced | generated by connecting the bird's-eye view image 200, without cut | disconnecting the image | video of a vehicle.

FIG. 13 shows a composite bird's-eye view image 251b generated using the bird's-eye view image whose boundary has been moved.
In the synthesized bird's-eye view image 251b in FIG. 13, the boundary 202 between the bird's-eye view image 200A and the bird's-eye view image 200B moves in the upward direction on the paper. The movement of the boundary is performed based on the abnormality information transmitted from the abnormality detection server 2 as described later. That is, the boundary 202 is moved similarly to the process of moving the boundary L12 of the video B2 in FIG. 12 to the boundary L22 and moving the boundary L11 of the video B1 in FIG. 11 to the boundary L21. By doing in this way, as shown in FIG.11 and FIG.12, the bird's-eye view image | video 200 is connected, without the image | video of a vehicle (object) cutting out.
By doing in this way, as shown to the code | symbol 203a, the distortion of the shape of an object (vehicle) can be made low and displayed.
Note that the boundary 202 may move not only above the page but also below the page. Alternatively, the boundary 202a between the bird's-eye view images 200E and 200F and the bird's-eye view image 200C in FIG. 10 can also move in the left-right direction in FIG.

[flowchart]
Hereinafter, processing performed by the overhead image processing apparatus 1a and the abnormality detection server 2a will be described. In the description of each flowchart, FIG. 9 is appropriately referred to.
(Overhead video processing apparatus 1a)
FIG. 14 is a flowchart showing a processing procedure of the overhead image processing apparatus 1a used in the second embodiment.
In FIG. 14, the same processes as those in FIG. 6 are denoted by the same step numbers and the description thereof is omitted.

After acquiring the camera image in step S101, the determination processing unit 15 determines whether or not boundary movement information has been received from the abnormality detection server 2a (S111). When the anomaly detection server 2a detects an anomaly at the boundary portion of the camera image, the boundary movement information is sent to the overhead image processing apparatus 1a based on the extracted anomaly information, using the boundary moving direction / distance in the camera image as the boundary movement information. Send. This process will be described later with reference to FIG.
As a result of step S111, when the boundary movement information is not received (S111 → No), the overhead image processing apparatus 1a advances the process to step S102.

  When the boundary movement information is received as a result of step S111 (S111 → Yes), the overhead view video generation unit 12 moves the boundary of the camera image to be moved according to the boundary movement information (S112).

  Then, the bird's-eye view image generation unit 12 generates a bird's-eye view image from each camera image including the camera image whose boundary is moved (S102). The subsequent processing is the same as steps S103 and S104 in FIG.

(Abnormality detection server 2a)
FIG. 15 is a flowchart showing a processing procedure of the abnormality detection server 2a used in the second embodiment.
In FIG. 15, the same steps as those in FIG.
After extracting the abnormality information in step S203, the abnormality detection processing unit 22 determines whether or not the video range in which the abnormality is detected is on the boundary of the camera video (S211). The anomaly detection server 2a shares with the overhead video processing apparatus 1 where in the camera video is the boundary.
As a result of step S211, when the video range where the abnormality is detected is not at the boundary (S211 → No), the abnormality detection server 2a performs the processes of step S203 and step S204.
As a result of step S211, when the video range in which the abnormality is detected is at the boundary (S211 → Yes), the abnormality detection processing unit 22 sets the area (size) of the abnormal range in the camera video to the camera video in which the abnormality is detected. It calculates for every (S212).
Then, the abnormality detection processing unit 22 identifies which camera image has the largest abnormality range among the camera images from which the abnormality is detected (S213). The camera video specified here corresponds to the video B2 in FIG.

  Subsequently, the abnormality detection processing unit 22 calculates the boundary moving direction / distance based on the information of the camera video specified in step S213 and the area of the abnormal range calculated in step S212 (S214). That is, the abnormality detection processing unit 22 determines how much the boundary is moved in which direction in which camera image. The moving distance of the boundary is such a distance that the entire object showing abnormality is included in the camera image specified in step S213. Therefore, the moving distance of the boundary is a distance according to the size of the abnormal object. Further, the abnormality detection processing unit 22 also captures the boundary of the camera image (image B1 in FIG. 11) that is not the camera image (image B2 in FIG. 12) identified in step S213, although an object indicating an abnormality is captured. Calculate the moving direction and distance.

  Subsequently, the information transmission unit 23 transmits the boundary movement information to the overhead image processing apparatus 1a (S215). The boundary movement information is information including the camera number corresponding to the camera image that has moved the boundary and how much the boundary has been moved in which direction of the camera image.

Then, the abnormality detection processing unit 22 re-extracts the abnormality information after moving the boundary of the camera video to be the boundary movement according to the boundary movement information (S216).
When step S216 is executed, the information transmission unit 23 transmits abnormality information including the abnormality information re-extracted in step S216 to the video server 3 (S204), and the abnormality detection server 2a performs the process to step S201. return.

Note that priorities may be assigned to all the cameras 4 in advance. In this case, the process of step S212 in FIG. 15 may be omitted to move the boundary of the camera image having a high priority order regardless of the size of the area of the abnormal range.
As described above, according to the second embodiment, even when an object is present at the boundary of the overhead view video, it is possible to display the object with a reduced degree of distortion.

  Patent Document 2 also describes a method for synthesizing an overhead view video when an obstacle exists in the boundary area when the overhead view video is generated. However, in Patent Document 2, a sonar is arranged at a position corresponding to the boundary region, and an obstacle is detected by the sonar. When an obstacle is detected, the boundary of the camera image is moved by a certain distance.

In addition, it is not preferable to install sonar on a road facility from the viewpoint of autonomous driving or the like.
In the second embodiment, such a sonar is unnecessary.

  Further, as in the technique described in Patent Document 2, when an obstacle is detected with sonar, it is possible to detect what distance the obstacle is located. I do n’t know if that ’s true. Therefore, the technique described in Patent Document 2 is not suitable for monitoring, and it is not known how much the boundary should be moved. Therefore, the moving distance of the boundary is a constant distance.

  In the monitoring support system Za of the second embodiment, not only can the monitoring person easily identify what kind of object is the abnormal object, but also the boundary can be moved according to the size of the object. In general, when a bird's-eye view image is generated from a camera image, a wide-angle camera is used. Therefore, the larger the boundary is moved, the greater the distortion of the image at the time of overhead view generation. Since the monitoring support system Za according to the second embodiment moves the boundary according to the size of the object and the like, it is possible to minimize the distortion of the video when the overhead view is generated.

«Third embodiment»
Next, as a third embodiment, a monitoring support system Zb provided with a sensor (sensor unit) 7 such as a human sensor or a fire alarm will be described as shown in FIG.

(Monitoring support system Zb)
FIG. 16 is a configuration diagram illustrating an example of the overall configuration of the monitoring support system Zb used in the third embodiment.
In FIG. 16, the same components as those in FIG. 9 are denoted by the same reference numerals, and description thereof is omitted.
As shown in FIG. 16, the toll gate Tb (T1b to T3b) in the monitoring support system Zb has a configuration in which a plurality of sensors 7 are added to the configuration of the monitoring support system Za shown in FIG. Although an example in which the sensor 7 is added to the configuration of the monitoring support system Za shown in FIG. 9 is shown here, the addition of the sensor 7 to the monitoring support system Z shown in FIG. It is.
Next, parts of the abnormality detection server 2b according to the third embodiment that are different from the configuration shown in FIG. 9 will be described. The configurations of the overhead video processing apparatus 1a and the video server 3 are the same as those shown in FIG.

(Abnormality detection server 2b)
The abnormality detection server 2b is different from the abnormality detection server 2a shown in FIG. 9 in that it includes a sensor determination processing unit 25.
The sensor determination processing unit 25 monitors whether or not the value of each sensor 7 is abnormal.

In the first embodiment and the second embodiment, the abnormality detection process is performed based on camera images from a plurality of cameras 4. In the third embodiment, the detection result by the sensor 7 is added in addition to the abnormality detection by the first embodiment and the second embodiment.
Specifically, the sensor 7 is a known sensor such as a human sensor using infrared rays, a fire alarm, or a vehicle detection sensor using magnetism. A sensor mounted on the ETC may be used as the vehicle detection sensor.
Which sensor 7 is used differs depending on the object or event to be detected. By adding the sensor 7 in this way, the accuracy of detecting a moving object or an abnormal situation is improved.

[flowchart]
Hereinafter, processing performed in the abnormality detection server 2b and the video server 3 will be described. In the description of each flowchart, FIG. 16 is referred to as appropriate.
(Abnormality detection server 2b)
FIG. 17 is a flowchart showing a processing procedure of the abnormality detection server 2b used in the third embodiment.
In FIG. 17, processing similar to that in FIG. 15 is denoted by the same step number and description thereof is omitted.
After step S204, the sensor determination processing unit 25 determines whether or not an abnormality (sensor abnormal value) by the sensor 7 is detected (S221).
If no abnormality is detected by the sensor 7 as a result of step S221 (S221 → No), the abnormality detection server 2 returns the process to step S201.

  As a result of step S221, when an abnormality is detected by the sensor 7 (S221 → Yes), the information transmission unit 23 transmits a sensor number that is the number of the sensor 7 that detected the abnormality to the video server 3 (S222). Then, the abnormality detection server 2 returns the process to step S201.

  Note that the timing at which step S221 and step S222 are executed is not limited to the timing illustrated in FIG. 17, and may be performed at any timing between step S201 and step S204.

(Video server 3)
FIG. 18 is a flowchart for supporting the processing procedure of the video server 3 used in the third embodiment.
In FIG. 18, the same processes as those in FIG.
After step S304 and step S305, the determination processing unit 32 determines whether or not the sensor number transmitted in step S222 of FIG. 17 has been received (S311).
If the sensor number is not received as a result of step S311 (S311 → No), the video server 3 returns the process to step S301.

When the sensor number is received as a result of step S311 (S311 → Yes), the video processing unit 33 generates a display video in which a sensor warning display is displayed at a location corresponding to the received sensor number in the synthesized overhead video (S312). ).
Then, the video server 3 returns the process to step S301.

(Screen example)
FIG. 19 is a diagram showing a display image 201d in the monitoring support system Zb used in the fifth embodiment.
19, the same elements as those in FIG. 13 are denoted by the same reference numerals as those in FIG.
In the display video 201d shown in FIG. 19, in addition to the frame line M2 indicating an abnormal object, a sensor warning display 301 indicating that there is a person and a sensor warning display 302 indicating that a fire has occurred are displayed. Yes.
The warning display indicating that there is a person indicated by the sensor warning display 301 is an abnormality detected by the human sensor. The warning display indicating that a fire has occurred in the sensor warning display 301 is an abnormality detected by the fire alarm. The video server 3 stores the type of the sensor 7 and the warning display pattern in association with each other, thereby enabling warning display according to the warning content as shown in FIG.

  The monitoring support system Zb of the third embodiment can improve monitoring accuracy because it can monitor not only by video but also by the sensor 7.

  In addition, for example, even if the display according to the output value of the sensor 7 is performed such that the sensor warning display 301 is displayed in red as the temperature detected by the temperature sensor provided in the fire alarm is higher. Good.

  For example, the present embodiment shows an example applied to a toll gate T, Ta, Tb on a highway, but it is applied to a road facility such as a main road of a highway, a service area, a parking area, a gas station, etc. can do.

  The present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to having all the configurations described. In addition, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of a certain embodiment. Moreover, it is possible to add / delete / replace other configurations for a part of the configurations of the embodiments.

  Any two of the overhead video processing device 1 (1a), the abnormality detection server 2 (2a, 2b), and the video server 3 may be one device. Alternatively, the overhead video processing device 1 (1a), the abnormality detection server 2 (2a, 2b), and the video server 3 may be combined into one device.

  In the present embodiment, highlighting is performed by surrounding an abnormal object with a frame line, but the present invention is not limited to the frame line. Further, when an abnormal object is detected, the abnormal object may be highlighted and a warning sound or the like may be generated to notify the monitoring staff that the abnormal object has been detected.

Further, the video server 3 may be installed in the control room C, or the control server 6 may have the function of the video server 3.
The monitor 61 may be a mobile terminal such as a smartphone or a tablet terminal.

In addition, each of the above-described configurations, functions, units 11 to 16, 21 to 25, 31 to 34, the storage device 103, and the like are realized by hardware by designing a part or all of them with, for example, an integrated circuit. May be. Further, as shown in FIG. 5, the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by a processor such as the CPU 102. In addition to storing information such as programs, tables, and files for realizing each function in an HD (Hard Disk), a memory 101, a recording device such as an SSD (Solid State Drive), or an IC (Integrated Circuit) card It can also be stored in a recording medium such as an SD (Secure Digital) card or a DVD (Digital Versatile Disc).
In each embodiment, control lines and information lines are those that are considered necessary for explanation, and not all control lines and information lines are necessarily shown on the product. In practice, it can be considered that almost all configurations are connected to each other.

1, 1a Overhead video processing device 2, 2a, 2b Anomaly detection server 3 Video server 4, 4A-4H Camera (imaging unit)
6 Control server 7 Sensor (sensor part)
DESCRIPTION OF SYMBOLS 11 Information reception part 12 Overhead video production | generation part 13 Composite overhead view video production | generation part 14 Information transmission part 15 Judgment processing part 16 Boundary adjustment part 21 Information reception part 22 Abnormality detection processing part 23 Information transmission part 24 Judgment processing part 31 Information reception part 32 Determination Processing unit 33 Video processing unit 34 Information transmission unit 61, 61a to 61c Monitor (display unit)
200, 200A to 200H overhead image 201, 201a to 201d display image 202, L11, L12, L21, L22 boundary (boundary part)
251, 251a, 251b Composite bird's-eye view image 301, 301 Sensor warning display M Frame (highlighted)
T, T1-T3, Ta, T1a-T3a, Tb, T1b-T3b Tollgate (road facility)
Z, Za, Zb monitoring support system

Claims (9)

A plurality of imaging units provided in a road facility;
An overhead video generation unit that generates an overhead video from captured images captured by each of the plurality of imaging units;
A synthesized overhead view video generation unit that generates a synthesized overhead view video by synthesizing the overhead view video generated by the overhead view video generation unit;
An abnormality detection processing unit that detects an abnormal state of an object from the captured images captured by the plurality of imaging units; and a video processing unit that highlights the object in which the abnormal state is detected in the synthesized overhead view image;
A monitoring support system characterized by comprising: The monitoring support according to claim 1, further comprising: a boundary adjusting unit that moves the boundary when the object in which the abnormal state is detected is present at the boundary for generating the overhead view video. system. The boundary adjustment unit
The area / size of the object existing at the boundary is calculated in each captured image where the object is present, and the boundary is moved so that the entire object is captured in the predetermined captured image. The monitoring support system according to claim 2. Having a sensor part,
The monitoring support system according to claim 1, wherein the video processing unit displays an abnormal state detected by the sensor unit in the synthesized overhead view video. The monitoring support system according to any one of claims 1 to 4, wherein the road facility is a toll gate. The display unit according to any one of claims 1 to 5, further comprising: a display unit configured to display, for each road facility, an image generated by the image processing unit based on captured images captured by the plurality of road facilities. The monitoring support system according to any one of the above. A monitoring support system having a plurality of imaging units provided in a road facility,
An overhead image is generated from the captured image captured by each of the plurality of imaging units,
A synthesized overhead view video is generated by synthesizing the generated overhead view video,
Detecting an abnormal state of an object from the captured images captured by the plurality of imaging units;
The monitoring support method, wherein the object in which the abnormal state is detected is highlighted in the synthesized overhead view video. The monitoring support system is
The monitoring support method according to claim 7, wherein when the object in which the abnormal state is detected is present at a boundary part for generating the overhead view video, the boundary part is moved. The monitoring support system is
The area / size of the object existing at the boundary is calculated in each captured image where the object is present, and the boundary is moved so that the entire object is captured in the predetermined captured image. The monitoring support method according to claim 8, wherein:

Images