JP7456210B2 - Information processing method, information processing device, program and information processing system - Google Patents

Description

The present invention relates to an information processing method, an information processing device, a program, and an information processing system.

In recent years, in order to reduce accidents caused by vehicles, functions have been developed to recognize pedestrians etc. around the vehicle, follow-up functions that follow the vehicle in front, and automatic systems that slow down or stop depending on the distance between the vehicle and the vehicle in front. Vehicles are equipped with various technologies such as brake functions. Patent Document 1 describes how a pedestrian terminal carried by a pedestrian or the like and an in-vehicle terminal mounted on a vehicle perform direct communication through pedestrian-to-vehicle communication and indirect communication through a roadside device installed on the road. A configured system is disclosed. The system disclosed in Patent Document 1 is configured to select whether to perform direct communication or indirect communication between the pedestrian terminal and the vehicle-mounted terminal, depending on the state of the pedestrian or the vehicle.

International Publication No. 2019/44208

In the system disclosed in Patent Document 1, each of the pedestrian terminal and the vehicle-mounted terminal acquires the position information of its own terminal and transmits it to the other terminal, and receives the position information of other pedestrians or vehicles received from the other terminal. Based on this, it is determined whether there is a risk of collision with another pedestrian or vehicle. Such a system has a problem in that pedestrians who are not carrying pedestrian terminals and vehicles which are not equipped with on-vehicle terminals are not subject to determination as to whether or not they are in a dangerous state.

The present invention was made in consideration of these circumstances, and its purpose is to provide an information processing method etc. that can determine the presence or absence of a dangerous situation for all pedestrians and vehicles etc. within a monitoring area.

In one aspect of the present invention, an information processing method includes a computer that acquires a captured image obtained by photographing a surveillance area on a road, inputs the captured image to a moving object discrimination model that has been trained to output information indicating whether or not the image contains a moving object performing a predetermined action that may pose a danger to people in the vicinity when the image is input, detects whether or not the surveillance area captured in the captured image contains a moving object performing the predetermined action based on the information output by the moving object discrimination model, and, when a moving object performing the predetermined action is detected, outputs notification information to notify a user in the surveillance area captured in the captured image of the presence of a moving object that may pose a danger.

In one aspect of the present invention, it is possible to determine the presence or absence of a dangerous situation for all pedestrians, vehicles, etc. within a monitoring area, and to appropriately notify when it is determined that there is a dangerous situation. I can do it.

1 is a schematic diagram showing a configuration example of an information processing system. 1 is a schematic diagram showing a configuration example of an information processing system. FIG. 2 is a block diagram showing a configuration example of a monitoring device and a user terminal. FIG. 2 is a schematic diagram showing a configuration example of a vehicle discrimination model. FIG. 2 is a block diagram illustrating a configuration example of an in-vehicle terminal and a vehicle management server. FIG. 2 is a schematic diagram showing a configuration example of a vehicle information DB. 3 is a flowchart illustrating an example of a notification processing procedure. 7 is a flowchart illustrating an example of a notification processing procedure in Embodiment 2. FIG. 12 is a flowchart illustrating an example of a notification processing procedure according to the third embodiment. FIG. 3 is a schematic diagram showing a configuration example of an information processing system according to a fourth embodiment. FIG. 2 is a schematic diagram showing a configuration example of a dangerous vehicle discrimination model. 12 is a flowchart illustrating an example of a notification processing procedure according to the fourth embodiment. 12 is a flowchart illustrating an example of a notification processing procedure according to the fourth embodiment. 12 is a flowchart illustrating an example of a notification processing procedure according to the fifth embodiment. 12 is a flowchart illustrating an example of a notification processing procedure according to the fifth embodiment. 12 is a flowchart illustrating an example of a notification processing procedure according to a sixth embodiment. 12 is a flowchart illustrating an example of a notification processing procedure according to the seventh embodiment.

Below, an information processing method, an information processing device, a program, and an information processing system of the present disclosure will be described in detail based on drawings showing embodiments thereof.

(Embodiment 1)
An information processing system that alerts passersby who need to be alerted when a vehicle is about to turn right or left at a road intersection will be described. 1 and 2 are schematic diagrams showing an example of the configuration of an information processing system. The information processing system 100 of this embodiment is provided in each monitoring area, for example, a road intersection. The information processing system 100 includes a monitoring device 10 that monitors traffic conditions based on captured images obtained by photographing a monitoring area, a display device 20 provided within the monitoring area, and a display device 20 that is carried by pedestrians or the like within the monitoring area. The monitoring area includes a user terminal 30 installed in the monitoring area, an in-vehicle terminal 40 installed in a vehicle within the monitoring area, and the like. The monitoring device 10 and the display device 20 are communicatively connected via, for example, a LAN (Local Area Network), and each monitoring device 10 or any one of the monitoring devices 10 outputs a caution message to each display device 20. Perform the processing to display it. Note that the monitoring device 10 is connected to a network N such as the Internet, and the monitoring device 10 and the display device 20 may be communicably connected via the network N.

A plurality of monitoring devices 10 (photographing devices) may be provided so as to be able to photograph everything within the surveillance area without creating a blind spot, and a camera 13 (Fig. 3), one monitoring device 10 may be provided. The display device 20 is a liquid crystal display, an organic EL display, electronic paper, or the like, and displays a caution message according to instructions from the monitoring device 10 to notify passersby in the monitoring area of the caution message. Note that the display device 20 may be configured with, for example, a light embedded in the road, and in this case, it can be configured to notify a caution message by lighting or blinking the light. Further, the display device 20 may include a light emitting unit such as a lamp or a revolving light, and may be configured to call attention by lighting or blinking the light emitting unit. The display device 20 is preferably provided at a location where a caution message can be visually recognized by passersby who are crossing a crosswalk in the monitoring area. Passersby crossing the crosswalk include pedestrians, bicycles, tricycles, skateboards, kick skaters, electric wheelchairs, senior cars, etc.

The user terminal 30 is a terminal device carried by a user within the monitoring area. The user terminal 30 is preferably configured with a wearable device such as a smart watch, but may also be configured with a smartphone, a tablet terminal, a personal computer, or the like. The user terminal 30 is configured to be able to communicate wirelessly with each monitoring device 10. The user terminal 30 and the monitoring device 10 are configured to enable wireless communication using, for example, Bluetooth (registered trademark), and the user terminal 30 can communicate with the monitoring device 10 when it is within a communicable range centered on the monitoring device 10. Receive the output signal. Users within the monitoring area include pedestrians on crosswalks, sidewalks, and roads within the monitoring area, and people using motorcycles, bicycles, tricycles, skateboards, kick skaters, electric wheelchairs, senior cars, etc. Note that the user refers to a person who is carrying the user terminal 30, and the passerby includes not only the user who is carrying the user terminal 30 but also the person who is not carrying the user terminal 30. The in-vehicle terminal 40 is a terminal device mounted on a vehicle such as a car or a motorcycle within the monitoring area, and is, for example, a car navigation device. The in-vehicle terminal 40 is connectable to the network N, and is communicatively connected to each monitoring device 10 via the network N.

A plurality of monitoring areas are provided on the road, and each monitoring area is provided with the information processing system 100 having the above-described configuration. Further, in this embodiment, a vehicle management server 50 is connected to the network N, and the vehicle management server 50 acquires and manages the current location information of each vehicle from the in-vehicle terminal 40 of each vehicle registered in advance. . The vehicle management server 50 is an information processing device capable of various information processing and information transmission/reception processing, and is, for example, a server computer or a personal computer. A plurality of vehicle management servers 50 may be provided, or may be realized by a plurality of virtual machines provided within one server device, or may be realized using a cloud server. The vehicle management server 50 holds information regarding vehicles that have been registered in advance, and performs processing to acquire current location information of the vehicle from the in-vehicle terminal 40 via the network N, and is located at a predetermined location in response to a request from the monitoring device 10. It performs processes such as providing information about the vehicle (in-vehicle terminal 40) to the monitoring device 10.

In the information processing system 100 of the present embodiment, the monitoring device 10 takes pictures with the camera 13, and based on the obtained captured images, a vehicle that is about to turn right or left at an intersection within the monitoring area (the shooting range) Detects the presence or absence of a vehicle (vehicle or left-turning vehicle). When the monitoring device 10 detects a right-turning vehicle or a left-turning vehicle within the monitoring area, the monitoring device 10 detects a right-turning vehicle or a left-turning vehicle within the monitoring area. 20) performs processing to display a caution message. Furthermore, the monitoring device 10 performs a process of outputting a caution signal via wireless communication when a passerby is crossing a crosswalk into which a right-turning vehicle or a left-turning vehicle is entering. Furthermore, the monitoring device 10 performs a process of outputting a caution signal to the in-vehicle terminal 40 of the right-turning vehicle or left-turning vehicle when there is a passerby who is crossing a crosswalk into which a right-turning vehicle or a left-turning vehicle is entering.

FIG. 3 is a block diagram showing a configuration example of the monitoring device 10 and the user terminal 30. The monitoring device 10 of this embodiment includes a control section 11, a storage section 12, a camera 13, a communication section 14, a wireless communication section 15, a reading section 16, etc., and these sections are interconnected via a bus. . The control unit 11 includes one or more processors such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), or a GPU (Graphics Processing Unit). The control unit 11 causes the monitoring device 10 to perform various information processing and control processing that should be performed by the information processing device of the present disclosure by appropriately executing the control program 12P stored in the storage unit 12.

The storage unit 12 includes a RAM (Random Access Memory), a flash memory, a hard disk, an SSD (Solid State Drive), and the like. The storage unit 12 stores in advance a control program 12P executed by the control unit 11 and various data necessary for executing the control program 12P. The storage unit 12 also temporarily stores data and the like generated when the control unit 11 executes the control program 12P. Furthermore, the storage unit 12 stores a vehicle discrimination model M1, a pedestrian discrimination model M2, etc., which are learned models trained by machine learning. The vehicle discrimination model M1 and the pedestrian discrimination model M2 are assumed to be used as program modules that function as part of artificial intelligence software.

The camera 13 is an imaging device having a lens, an image sensor, etc., and acquires image data of a subject image through the lens. The camera 13 performs photography according to instructions from the control unit 11, and sequentially acquires one piece (one frame) of image data (photographed image). The camera 13 is configured to acquire, for example, 60 frames, 30 frames, or 15 frames of video data per second, and the video data acquired by the camera 13 is sequentially stored in the storage unit 12. Note that the camera 13 may have a configuration that is built into the monitoring device 10, or may be configured to be externally attached to the monitoring device 10. In this case, the monitoring device 10 includes a connection section to which an external camera can be connected or a wireless communication section capable of wireless communication with the external camera, and transmits video data captured by the external camera via the connection section or the wireless communication section. get.

The communication unit 14 has an interface for connecting to the LAN by wired or wireless communication, and an interface for connecting to the network N by wired or wireless communication. Thereby, the communication unit 14 can transmit and receive information to and from the display device 20 via the LAN, and can transmit and receive information to and from other devices via the network N. The wireless communication unit 15 is an interface for performing wireless communication using Bluetooth, for example, and communicates with other devices within a communicable range (in this embodiment In this case, a control signal is transmitted to the user terminal 30).

The reading unit 16 reads information stored in a portable storage medium 1a including a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disc)-ROM, a USB (Universal Serial Bus) memory, an SD (Secure Digital) card, etc. read. The control program 12P and various data stored in the storage unit 12 may be read by the control unit 11 from the portable storage medium 1a via the reading unit 16 and stored in the storage unit 12. Further, the control program 12P and various data stored in the storage unit 12 may be downloaded by the control unit 11 from another device via the communication unit 14 and stored in the storage unit 12.

In addition to the above-described configuration, the monitoring device 10 of the present embodiment includes an input section that receives operation input from a user who operates the monitoring device 10 and sends a control signal corresponding to the operation content to the control section 11, and outputs a voice message. It may also be equipped with a speaker or the like.

The user terminal 30 includes a control section 31, a storage section 32, a wireless communication section 33, a notification section 34, etc., and these sections are interconnected via a bus. Each of the control unit 31, storage unit 32, and wireless communication unit 33 of the user terminal 30 has the same configuration as the control unit 11, storage unit 12, and wireless communication unit 15 of the monitoring device 10, so a detailed description thereof will be omitted. Note that the wireless communication unit 33 of the user terminal 30 can communicate with other devices (in this embodiment, the monitoring device 10) within a communicable range centered on a Bluetooth antenna (not shown) provided in the user terminal 30. The output control signal is received via the Bluetooth antenna.

The notification unit 34 includes a lamp, a buzzer, a speaker, a display unit, a vibrator, etc., and can control lighting or flashing of a lamp, sounding of a buzzer, audio output of a message by a speaker, display of a caution message by a display unit, vibration by a vibrator, etc. By doing so, the user of the user terminal 30 is notified of a predetermined situation (a situation where there is a right-turning vehicle or a left-turning vehicle). Note that the notification unit 34 implements notification processing by performing predetermined processing according to instructions from the control unit 31.

In addition to the above-described configuration, the user terminal 30 of the present embodiment includes an input section that receives operation input from a user who operates the user terminal 30 and sends a control signal corresponding to the operation content to the control section 31, a liquid crystal display, or an organic display. It may also include a display unit such as an EL display.

Figure 4 is a schematic diagram showing an example of the configuration of the vehicle discrimination model M1. The vehicle discrimination model M1 of this embodiment is configured, for example, as shown in Figure 4, with a CNN (Convolution Neural Network) model. In addition to the CNN model, the vehicle discrimination model M1 may be configured with an RNN (Recurrent Neural Network) model or a LSTM (Long Short-Term Memory) model. In addition, the vehicle discrimination model M1 may be configured with any object detection algorithm (neural network) such as R-CNN (Regions with CNN), Fast R-CNN, Faster R-CNN, Mask R-CNN, SSD (Single Shot Multibook Detector), YOLO (You Only Look Once), etc., or may be configured by combining several of these models. The vehicle discrimination model M1 (moving object discrimination model) is a trained model that takes as input a photographed image (vehicle image) of a vehicle such as an automobile or motorcycle, calculates whether the photographed vehicle is, for example, a right-turning vehicle, a left-turning vehicle, or a straight-moving vehicle based on the input vehicle image, and outputs the calculated result.

The vehicle discrimination model M1 shown in FIG. 4 is composed of an input layer, an intermediate layer, and an output layer. The intermediate layers include convolutional layers, pooling layers, and fully connected layers. In the vehicle discrimination model M1 of this embodiment, a vehicle image (image data) obtained by photographing a running vehicle is input via the input layer. Each pixel in the vehicle image is input to each node of the input layer, and the vehicle image input via each node of the input layer is input to the intermediate layer. The vehicle image input to the intermediate layer is subjected to filter processing or the like in the convolutional layer to extract the feature amount of the image to generate a feature map, and then compressed in the pooling layer to reduce the amount of information. A plurality of convolutional layers and pooling layers are repeatedly provided, and feature maps generated by the plurality of convolutional layers and pooling layers are input to the fully connected layer. The fully connected layer has multiple layers (two layers in Figure 4), and based on the input feature map, the output value of the node in each layer is calculated using various functions, thresholds, etc., and the calculated output value are sequentially input to nodes in subsequent layers. The fully connected layer sequentially inputs the output value of the node of each layer to the node of the next layer, thereby finally giving each output value to each output node of the output layer.

In the vehicle discrimination model M1 shown in FIG. 4, the output layer has three output nodes, and each output node outputs discrimination probabilities for each direction of travel: right turn, left turn, and straight ahead. The discrimination probability output by each output node is that the vehicle in the vehicle image input to the input layer is a vehicle traveling in the direction of travel associated with each output node (specifically, a right-turning vehicle, Indicates the possibility that the vehicle is turning left or going straight. The output value of each output node in the output layer is, for example, a value between 0 and 1.0, and the sum of the discrimination probabilities output from all output nodes is 1.0. In the vehicle discrimination model M1, the number of input nodes in the input layer, the number of each of the convolution layer, pooling layer, and fully connected layer in the intermediate layer, and the number of output nodes in the output layer are not limited to the example shown in FIG. 4. For example, the output layer may have, in addition to the three output nodes described above, an output node that outputs the discrimination probability for a backward vehicle that is traveling backwards, a dangerous vehicle that is driving in a meandering manner, or a dangerous vehicle that is driving in a swerving manner.

The vehicle discrimination model M1 is trained using teacher data including a vehicle image and the traveling direction (correct label) of the vehicle shown in the vehicle image. In the vehicle discrimination model M1, when a vehicle image included in the teacher data is input, an output value of 1.0 is output from the output node corresponding to the traveling direction indicated by the correct label included in the teacher data, and other output values are output. Learn so that the node outputs an output value of 0.0. In the learning process, the vehicle discrimination model M1 learns to optimize the weighting coefficients and functions that connect the nodes of each layer in the intermediate layer. As a result, when a vehicle image is input, a trained vehicle discrimination model M1 that is trained to identify a vehicle in the vehicle image from among right-turning vehicles, left-turning vehicles, and straight-going vehicles is obtained. . Note that, as the vehicle image used for the training data, for example, an image taken of a vehicle traveling in a right-turn lane, a vehicle indicating a right turn with a turn signal, etc. can be used as an image of a right-turning vehicle. Further, as the vehicle image used for the training data, for example, an image of a vehicle traveling in a left-turn lane, a vehicle indicating a left turn with a turn signal, etc. can be used as an image of a left-turn vehicle. Although learning of the vehicle discrimination model M1 is performed by another learning device, it may also be performed by the monitoring device 10. The vehicle discrimination model M1 is not limited to a neural network (deep learning) as shown in FIG. 4, and learning models constructed by various machine learning algorithms can be used.

The pedestrian discrimination model M2 is a trained model that receives a photographed image as an input and has learned to output information indicating whether or not there is a pedestrian in the photographed image when the photographed image is input. Note that pedestrians here include not only people walking, but also people using bicycles, tricycles, skateboards, kick skaters, electric wheelchairs, senior cars, etc. The pedestrian discrimination model M2 is constructed with the same configuration as the vehicle discrimination model M1 shown in FIG. Outputs the discrimination probability for the state where there is no one present. The pedestrian discrimination model M2 has a plurality of output nodes, and each output node is configured to output a discrimination probability for each of pedestrians, bicycles, tricycles, skateboards, kick skaters, electric wheelchairs, and senior cars. may have been done. When the pedestrian discrimination model M2 is configured in this way, the discrimination probability output by each output node is determined based on the state in which the person in the input captured image is associated with each output node (specifically indicates the possibility that the person is a pedestrian, or a person using a bicycle, tricycle, skateboard, kick skater, electric wheelchair, or senior car).

The pedestrian discrimination model M2 is trained using teacher data including a photographed image and information (correct label) indicating whether a pedestrian is present in the photographed image. The pedestrian discrimination model M2 generates an output of 1.0 from the output node corresponding to the state (whether there is a pedestrian or not) indicated by the correct label included in the teacher data when a captured image included in the teacher data is input. This value is output, and the other output nodes learn to output an output value of 0.0. As a result, when a captured image of a pedestrian, or a person using a bicycle, tricycle, skateboard, kick skater, electric wheelchair, senior car, etc. is input, it is possible to determine whether or not there is a pedestrian in the captured image. A trained pedestrian discrimination model M2 that has been trained to discriminate is obtained. Note that the captured images used as training data for the pedestrian discrimination model M2 include, for example, images of pedestrians on crosswalks, people using bicycles, tricycles, skateboards, kick skaters, electric wheelchairs, senior cars, etc. It can be used as an image containing pedestrians. Learning of the pedestrian discrimination model M2 may also be performed by another learning device, or may be performed by the monitoring device 10. Note that if the pedestrian discrimination model M2 is configured to output the discrimination probability for each of pedestrians, bicycles, tricycles, skateboards, kick skaters, electric wheelchairs, and senior cars, Learning may be performed using teacher data including information (correct label) indicating whether the subject in the photograph is a pedestrian, bicycle, tricycle, skateboard, kick skater, electric wheelchair, or senior car. Furthermore, the pedestrian discrimination model M2 is not limited to a neural network as shown in FIG. 4, and learning models constructed by various machine learning algorithms can be used.

FIG. 5 is a block diagram showing a configuration example of the in-vehicle terminal 40 and the vehicle management server 50. The in-vehicle terminal 40 includes a control section 41, a storage section 42, a communication section 43, a positioning section 44, a notification section 45, etc., and these sections are interconnected via a bus. The control unit 41 and storage unit 42 of the in-vehicle terminal 40 have the same configuration as the control unit 11 and the storage unit 12 of the monitoring device 10, and the notification unit 45 of the in-vehicle terminal 40 has the same configuration as the notification unit 34 of the user terminal 30. Since the configuration is as follows, a detailed explanation will be omitted. Note that the communication unit 43 of the in-vehicle terminal 40 is an interface for connecting to the network N by wireless communication, and transmits and receives information to and from other devices via the network N.

The positioning unit 44 detects the current location of the in-vehicle terminal 40 and obtains current location information (e.g., longitude and latitude coordinate values) indicating the current location. The positioning unit 44 receives radio waves transmitted from, for example, a GPS (Global Positioning System) satellite via a GPS antenna (not shown) provided in the in-vehicle terminal 40, and detects the current location based on the received radio waves. Note that the method of detecting the current location is not limited to a method based on radio waves from a GPS satellite. In addition to the above-mentioned configuration, the in-vehicle terminal 40 of this embodiment may also include an input unit that accepts operation input by a user operating the in-vehicle terminal 40 and sends a control signal corresponding to the operation content to the control unit 41, a display unit such as a liquid crystal display or an organic EL display, etc.

The vehicle management server 50 includes a control section 51, a storage section 52, a communication section 53, an input section 54, a display section 55, etc., and these sections are interconnected via a bus. The control unit 51, storage unit 52, and communication unit 53 of the vehicle management server 50 have the same configuration as the control unit 41, storage unit 42, and communication unit 43 of the in-vehicle terminal 40, so a detailed explanation will be omitted. Note that the storage unit 52 of the vehicle management server 50 stores a vehicle information DB (database) 52a, which will be described later, in addition to a control program 52P executed by the control unit 51. Furthermore, the communication unit 53 of the vehicle management server 50 may be configured to be connected to the network N through wired communication.

The input unit 54 accepts operation input by a user who operates the vehicle management server 50, and sends a control signal corresponding to the operation content to the control unit 51. The display unit 55 is a liquid crystal display or an organic EL display, etc., and displays various information according to instructions from the control unit 51. The input unit 54 and the display unit 55 may be a touch panel configured as an integrated unit.

FIG. 6 is a schematic diagram showing a configuration example of the vehicle information DB 52a. The vehicle information DB 52a stores information regarding vehicles registered in advance. The vehicle information DB 52a shown in FIG. 6 includes a vehicle ID string, a vehicle information string, a destination information string, a current location information string, etc., and stores each piece of information regarding the vehicle and the in-vehicle terminal 40 installed in the vehicle in association with the vehicle ID. Remember. The vehicle ID column stores identification information (vehicle ID) assigned to each registered vehicle. The vehicle information string stores information that can identify the vehicle, such as the vehicle type and body color. The destination information string stores destination information when communicating with the in-vehicle terminal 40 mounted on the vehicle via the network N. The current location information stores information indicating the current location of the vehicle obtained from the on-vehicle terminal 40. The vehicle ID stored in the vehicle information DB 52a is issued and stored by the control unit 51 when new vehicle information is registered in the vehicle information DB 52a. Other information stored in the vehicle information DB 52a is added or changed by the control unit 51 each time the control unit 51 receives an instruction for addition or change via the input unit 54 or the communication unit 53. The storage contents of the vehicle information DB 52a are not limited to the example shown in FIG. 6. For example, the vehicle's current traveling direction, traveling speed, traveling route history, etc. may be stored in the vehicle information DB 52a.

Below, in the information processing system 100 of the present embodiment, processing performed by each device when the monitoring device 10 detects a right-turning vehicle or a left-turning vehicle within the monitoring area and notifying passersby in the monitoring area. I will explain about it. FIG. 7 is a flowchart showing an example of a notification processing procedure. In FIG. 7, the left side shows the process performed by the in-vehicle terminal 40 mounted on the right-turning vehicle or the left-turning vehicle, the center shows the process performed by the monitoring device 10, and the right side shows the process performed by the user terminal 30. The following processes are executed by the control unit 11 according to the control program 12P stored in the storage unit 12 of the monitoring device 10, and are executed by the control unit 31 according to the control program 32P stored in the storage unit 32 of the user terminal 30. , is executed by the control unit 41 according to the control program 42P stored in the storage unit 42 of the vehicle-mounted terminal 40. Part of the following processing may be realized by a dedicated hardware circuit.

In the information processing system 100 of this embodiment, the control unit 11 of the monitoring device 10 uses the camera 13 to photograph a surveillance area that can be photographed (S11). The control unit 11 constantly takes pictures with the camera 13. Further, when the information processing system 100 includes a moving object sensor that detects the presence or absence of a moving object within the monitoring area, the control unit 11 may start photographing by the camera 13 after the presence of the moving object is detected by the moving object sensor. The control unit 11 (image acquisition unit) acquires a photographed image by photographing with the camera 13, and determines whether a right-turning vehicle or a left-turning vehicle is included in the photographed image based on the acquired photographic image (S12). . Specifically, the control unit 11 (detection unit) inputs the captured image to the vehicle discrimination model M1, and identifies the traveling direction of the vehicle shown in the captured image based on the output information from the vehicle discrimination model M1. (detect). For example, in the vehicle discrimination model M1, when the maximum output value (discrimination probability) is greater than or equal to a predetermined value (for example, 0.7), the control unit 11 controls the progress of the vehicle corresponding to the output node that outputs the maximum output value. The direction is specified as the traveling direction of the vehicle shown in the photographed image.

If the control unit 11 determines that no right-turning vehicle or left-turning vehicle is included in the photographed image based on the determination result of the traveling direction of the vehicle using the vehicle determination model M1 (S12: NO), the process of step S11 is performed. Return to , and continue photographing the surveillance area. In this embodiment, the monitoring device 10 uses the vehicle discrimination model M1 to determine the presence or absence of a right-turning vehicle or a left-turning vehicle in the photographed image, but a predetermined server connected to the network N determines whether there is a right-turning vehicle or a left-turning vehicle in the photographed image. The presence or absence of a left-turning vehicle may also be determined. In this case, the control unit 11 transmits the sequentially acquired captured images to a predetermined server via the network N, and acquires the judgment result (presence or absence of a right-turning vehicle or a left-turning vehicle in the captured image) determined by the predetermined server. According to the acquired detection result, it may be determined whether a right-turning vehicle or a left-turning vehicle is included in the photographed image.

When it is determined that a right-turning vehicle or a left-turning vehicle is included in the photographed image (S12: YES), the control unit 11 (specification unit) detects a right-turning vehicle or a left-turning vehicle in the monitoring area based on the right-turning vehicle or left-turning vehicle in the photographed image. An area (notification area) where left-turning vehicles need to be alerted is identified (S13). For example, a crosswalk that a right-turning vehicle or a left-turning vehicle enters in the photographed image is specified as a notification area. Specifically, when the vehicle shown in FIG. 1 (right-turning vehicle) is photographed, the control unit 11 specifies the upper crosswalk into which the right-turning vehicle enters as the notification area. When the notification area is identified, the control unit 11 displays a caution message on the display device 20 provided at a position corresponding to the notification area (S14). For example, when a crosswalk where a right-turning vehicle is entering is specified as a notification area, the control unit 11 displays a message such as "Beware of right-turning vehicle!" on the display device 20 installed at a position visible to passersby crossing the crosswalk. Display a caution message. The caution message to be displayed on the display device 20 is stored in the storage unit 12, for example. Note that when the display device 20 has a light emitting section, the control section 11 may call attention by lighting or blinking the light emitting section. Further, when the monitoring device 10 or the display device 20 has a speaker, the control unit 11 may alert the user by outputting an audio message from the speaker.

Next, the control unit 11 determines whether there are passersby such as pedestrians in the notification area specified in step S13 (S15). Here, the control unit 11 determines whether or not a passerby is included in the captured image based on the captured image of the notification area captured by the camera 13. Specifically, the control unit 11 inputs the captured image to the pedestrian discrimination model M2, and determines whether there is a passerby (such as a pedestrian) in the captured image based on the output information from the pedestrian discrimination model M2. Determine. For example, in the pedestrian discrimination model M2, the control unit 11 determines the state (presence or absence of a passerby such as a pedestrian) corresponding to the output node that outputs the maximum (large value) output value (discrimination probability) in the photographed image. Be state specific. Note that the control unit 11 may determine whether or not there is a passerby in the image taken by the camera 13 of the specified notification area by pattern matching using a template. In this case, templates of captured images of people using general pedestrians, bicycles, tricycles, skateboards, kick skaters, electric wheelchairs, senior cars, etc. are stored in advance in the storage unit 12, and the control unit 11 can determine the presence or absence of a passerby in a captured image by determining the presence or absence of an area that matches the template from the captured image. Further, the control unit 11 may calculate a difference between time-series captured images of the notification area, and determine whether there is a passerby within the notification area based on the difference information. Furthermore, the control unit 11 determines whether or not there are not only passersby such as pedestrians, but also objects such as obstacles or moving objects such as animals in the captured image (within the notification area). Good too.

When the control unit 11 (output unit) determines that there is a passerby in the notification area (S15: YES), the control unit 11 (output unit) transmits notification information that notifies that there is a right-turning vehicle or a left-turning vehicle by wireless communication via the wireless communication unit 15. is output (S16). When the control unit 31 of the user terminal 30 within a communicable range with the monitoring device 10 receives vehicle notification information output from the monitoring device 10 via the wireless communication unit 33, the control unit 31 causes the notification unit 34 to perform notification processing. (S17). For example, if the notification unit 34 has a lamp, the control unit 31 lights or blinks the lamp, if the notification unit 34 has a buzzer, sounds the buzzer, and if the notification unit 34 has a speaker, sends a message using the speaker. If the notification unit 34 has a display unit, a warning message is displayed on the display unit, and if the notification unit 34 has a vibrator, the vibrator vibrates. Thereby, the user of the user terminal 30 can be notified of the presence of a right-turning vehicle or a left-turning vehicle.

Next, the control unit 11 acquires destination information of the in-vehicle terminal 40 mounted on the right-turning vehicle or left-turning vehicle detected based on the captured image in step S12 (S18). For example, the control unit 11 transmits position information (information on the installation location) of the monitoring device 10 to the vehicle management server 50 via the network N, and transmits the position information (information on the installation location) of the monitoring device 10 to the vehicle management server 50, and transmits the position information (installation location information) of the monitoring device 10 to the vehicle management server 50, and transmits the location information (installation location information) of the monitoring device 10 to the vehicle management server 50. The destination information assigned to is acquired from the vehicle management server 50. At this time, the control unit 51 of the vehicle management server 50 identifies a vehicle whose current location information is the location information received from the monitoring device 10 from among the vehicles stored in the vehicle information DB 52a, and a vehicle-mounted terminal corresponding to the identified vehicle. 40 destination information is read from the vehicle information DB 52a and transmitted to the monitoring device 10. Note that the control unit 11 calculates the position information of the detected right-turn vehicle or left-turn vehicle based on the position information of the monitoring device 10, and transmits the position information of the vehicle-mounted terminal 40 of the vehicle whose current location information is the position information of the right-turn vehicle or left-turn vehicle. The destination information may be acquired from the vehicle management server 50. The position information of the monitoring device 10 may be stored in the storage unit 12, for example, and if the monitoring device 10 has a positioning unit, the current location information (for example, coordinate values of longitude and latitude) of the monitoring device 10 is determined by the positioning unit. You may obtain it. The control unit 11 also extracts vehicle information (for example, vehicle type or body color) of a right-turning vehicle or a left-turning vehicle from the photographed image, and transmits destination information of the right-turning vehicle or left-turning vehicle from the vehicle management server 50 based on the extracted vehicle information. You may obtain it.

The control unit 11 (output unit) informs the in-vehicle terminal 40 mounted on the right-turning vehicle or the left-turning vehicle via the communication unit 14 based on the destination information acquired in step S18 that pedestrians etc. are present in the notification area. Notification information notifying that there is a passerby is output (S19). When the control unit 41 of the in-vehicle terminal 40 receives the passerby notification information output from the monitoring device 10 via the communication unit 43, the control unit 41 executes notification processing by the notification unit 45 (S20). Here too, if the notification unit 45 has a lamp, the control unit 41 lights or flashes the lamp, if the notification unit 45 has a buzzer, sounds the buzzer, and if the notification unit 45 has a speaker, the control unit 41 lights up or blinks the lamp. If the notification unit 45 has a display unit, the warning message is displayed on the display unit, and if the notification unit 45 has a vibrator, the vibrator vibrates. Thereby, the user (driver) driving the vehicle in which the in-vehicle terminal 40 is mounted can be notified that there is a passerby such as a pedestrian in the direction of travel of the vehicle. Note that, when outputting the notification information to the in-vehicle terminal 40, the control unit 11 includes the photographed image used when detecting that there is a passerby in the notification area in step S15, and outputs the notification information to the in-vehicle terminal 40. You can also send it. In this case, the in-vehicle terminal 40 that has received the notification information displays the photographed image included in the notification information on a display unit (not shown) to indicate that there is a passerby in the direction of travel of the vehicle. person can be notified. Therefore, for example, even if a passerby is located in a position that is difficult to see from a right-turning vehicle or a left-turning vehicle, the occupants in the vehicle can grasp the passerby's position from the photographed image taken by the monitoring device 10. Further, the control unit 11 may include, in the notification information, information indicating that the right-turning vehicle or the left-turning vehicle is in a right-turning state or a left-turning state, and transmit the notification information to the in-vehicle terminal 40 . In this case, the in-vehicle terminal 40 that has received the notification information can notify the occupants in the vehicle that the vehicle is in a state of turning right or turning left, for example, using the notification unit 45, based on the information included in the notification information. Note that when the information processing system 100 is provided with a transmitter that can pinpoint information to the in-vehicle terminals 40 of vehicles within the monitoring area, the control unit 11 of the monitoring device 10 uses the transmitter to Passerby notification information may be sent in a pinpoint manner to the on-vehicle terminal 40 of a right-turning vehicle or a left-turning vehicle.

In addition, the control unit 11 of the monitoring device 10 determines whether the right-turning vehicle or left-turning vehicle is stopped or running based on the captured image of the right-turning vehicle or left-turning vehicle, and if the vehicle is stopped, Different notification information may be output to the in-vehicle terminal 40 depending on whether the vehicle is running or when the vehicle is running. Then, the control unit 41 of the in-vehicle terminal 40 determines whether the own vehicle (right-turning vehicle or left-turning vehicle) is stopped or running by performing notification processing according to the notification information from the monitoring device 10. This allows for notification processing. For example, when a right-turning vehicle or a left-turning vehicle is stopped, there is a high possibility that the driver of this vehicle recognizes a pedestrian or the like in front of the vehicle. On the other hand, when a right-turning vehicle or a left-turning vehicle is running, the driver of this vehicle is unlikely to recognize pedestrians or the like in front of the vehicle. Therefore, by varying the content of notification processing by the on-vehicle terminal 40 depending on the state of the vehicle, it is possible to switch the caution level depending on whether the driver recognizes a pedestrian or the like. Further, the control unit 11 of the monitoring device 10 detects the traveling speed of the right-turning vehicle or the left-turning vehicle based on the photographed image of the right-turning vehicle or the left-turning vehicle, and sends different notification information to the in-vehicle terminal according to the detected traveling speed. 40 may be output. In this case, it is possible to switch the caution level depending on the traveling speed of the vehicle that is about to turn right or left.

When the control unit 11 determines that there are no passersby in the notification area (S15: NO), it skips the processes of steps S16 and S18 to S19. Next, the control unit 11 determines whether the right-turning vehicle or left-turning vehicle detected in step S12 has passed through the monitoring area, based on the photographed image of the monitoring area (S21). Specifically, the control unit 11 performs a process of tracking the right-turning vehicle or left-turning vehicle detected in step S12 based on the photographed images sequentially acquired by the camera 13, and detects the right-turning vehicle or left-turning vehicle from among the photographed images. If it disappears, it is determined that this right-turning vehicle or left-turning vehicle has passed through the monitoring area. If it is determined that the right-turning vehicle or the left-turning vehicle has not passed through the monitoring area (S21: NO), the control unit 11 returns to the process of step S14, and displays a caution message on the display device 20, and displays a warning message within the notification area. The process of determining whether there are any passersby is continued. As a result, when there is a right-turning vehicle or a left-turning vehicle within the monitoring area, a caution message is displayed on the display device 20 provided at the crosswalk (notification area) that the right-turning vehicle or left-turning vehicle is scheduled to enter, and the If there is a passerby on the crosswalk, the user carrying the user terminal 30 is notified that there is a right-turning vehicle or a left-turning vehicle, and the right-turning vehicle or left-turning vehicle is notified that there is a passerby on the crosswalk they are planning to enter. Can be notified.

If it is determined that the right-turning vehicle or the left-turning vehicle has passed through the monitoring area (S21: YES), the control unit 11 uses wireless communication via the wireless communication unit 15 to determine whether the right-turning vehicle or the left-turning vehicle has passed and is in a safe state. Outputs safety information to notify that there is something (S22). Then, when the control unit 31 of the user terminal 30 within a communicable range with the monitoring device 10 receives the safety information output from the monitoring device 10 via the wireless communication unit 33, the control unit 31 of the user terminal 30 that is within a communicable range 34 ends (S23). As a result, after the right-turn vehicle or left-turn vehicle passes through the monitoring area, the notification process by the notification unit 34 of the user terminal 30 ends. Note that at this time, the control unit 31 may be configured to notify the user of the user terminal 30 of the safe state by performing other notification processing by the notification unit 34.

The control unit 11 also ends the display of the warning message that was displayed on the display device 20 in step S14 (S24). Furthermore, the control unit 11 outputs safety information to the in-vehicle terminal 40 of the right-turning vehicle or the left-turning vehicle based on the destination information acquired in step S18, notifying the vehicle (right-turning vehicle or left-turning vehicle) that the vehicle has passed the monitoring area and is in a safe state (S25). When the control unit 41 of the in-vehicle terminal 40 receives the safety information output from the monitoring device 10 via the communication unit 43, it ends the notification process by the notification unit 45 that was started in step S20 (S26). As a result, after the vehicle has passed the monitoring area, the notification process by the notification unit 45 of the in-vehicle terminal 40 of the vehicle ends. Here again, the control unit 41 may be configured to notify the passengers in the vehicle that the vehicle is in a safe state by performing other notification processes by the notification unit 45.

Through the above-described processing, the monitoring device 10 monitors the presence or absence of a right-turning vehicle or a left-turning vehicle within the monitoring area based on a photographed image of the monitoring area. When the presence of a right-turning vehicle or a left-turning vehicle is detected, for example, a crosswalk into which the right-turning vehicle or left-turning vehicle will enter is specified as a notification area, and a caution message is displayed on a display device 20 provided in the notification area. The monitoring device 10 also determines the presence or absence of a passerby within the notification area based on a captured image of the notification area, and if there is a passerby within the notification area, it is possible that the passerby is carrying a cell phone. The presence of a right-turning vehicle or a left-turning vehicle is notified to the user terminal 30, which has a right-turning vehicle or a left-turning vehicle, by wireless communication such as Bluetooth. Further, when there is a passerby within the notification area, the monitoring device 10 notifies the on-vehicle terminal 40 of the right-turning vehicle or left-turning vehicle that there is a passerby, such as a pedestrian, in the approach direction. The caution message displayed on the display device 20 in this manner can alert passersby who visually recognize the display device 20, and can also alert users carrying the user terminal 30 and vehicles equipped with the in-vehicle terminal 40. It is also possible to call attention to drivers inside the vehicle. This is expected to suppress the occurrence of accidents involving right-turning vehicles or left-turning vehicles.

In the process described above, when there is a passerby such as a pedestrian in the notification area, the monitoring device 10 uses wireless communication by the wireless communication unit 15 to output notification information to the user terminal 30 in the notification area to alert the user. However, the configuration is not limited to this configuration. For example, the monitoring device 10 may perform wireless communication by the wireless communication unit 15 regardless of the presence or absence of passersby in the notification area. That is, the monitoring device 10 may transmit notification information notifying that there is a right-turning vehicle or a left-turning vehicle by wireless communication without determining whether there are passersby in the notification area. It becomes possible to send notification information to the user terminal 30 at the same time.

In the present embodiment, the monitoring device 10 has a configuration in which the monitoring device 10 performs a process of monitoring the situation within the monitoring area based on the captured image taken by the camera 13 of the monitoring device 10, but the monitoring device 10 is not limited to this configuration. For example, when the information processing system 100 provided in one monitoring area includes a plurality of monitoring devices 10, captured images taken by the plurality of monitoring devices 10 are transmitted to one monitoring device 10, and one monitoring device 10 The configuration can be such that monitoring processing is performed based on a plurality of captured images. In this case, one monitoring device 10 that performs the monitoring process, as a result of the monitoring process, displays a warning message on the display device 20, processes a notification to the user terminal 30 of a pedestrian or the like in the notification area, and sends a notification to a vehicle turning right or turning left. What is necessary is to perform a notification process to the vehicle-mounted terminal 40 of the vehicle. Further, it is also possible to configure such that the photographed images taken by each monitoring device 10 are aggregated on a cloud server via the network N, and the cloud server performs monitoring processing based on the photographed images of each monitoring area. .

(Embodiment 2)
In the information processing system 100 of Embodiment 1 described above, an information processing system that notifies you when an accident occurs in which a right-turning vehicle or a left-turning vehicle being monitored in a monitoring area collides with another vehicle or a pedestrian will be described. . Since the information processing system 100 of this embodiment can be realized by the same device as the information processing system 100 of Embodiment 1, description of the configuration will be omitted.

Figure 8 is a flowchart showing an example of a notification process procedure in embodiment 2. The process shown in Figure 8 is the process shown in Figure 7 with steps S31 to S33 added between steps S19 and S21. Explanations of the same steps as in Figure 7 will be omitted. Note that steps S11 to S13 and S22 to S26 in Figure 7 are not shown in Figure 8.

In the information processing system 100 of this embodiment, the control units 11, 31, and 41 of the monitoring device 10, the user terminal 30, and the in-vehicle terminal 40 each perform the same processing as steps S11 to S20 in FIG. 7. Thereby, the monitoring device 10 monitors the presence or absence of a right-turning vehicle or a left-turning vehicle within the monitoring area, and if there is a right-turning vehicle or a left-turning vehicle, displays a caution message on the display device 20 and sends a warning message to the user terminal 30 via wireless communication. and notification to the in-vehicle terminal 40. Therefore, when there is a right-turning vehicle or a left-turning vehicle within the monitoring area, it is possible to alert passersby within the notification area and passengers inside the vehicle.

After processing in step S19, the control unit 11 determines whether an accident involving a right-turning vehicle or a left-turning vehicle detected in step S12 or a passerby detected in step S15 has occurred, based on the sequentially photographed images of the monitoring area. (S31). For example, the control unit 11 performs a process of tracking the right-turning vehicle or left-turning vehicle detected in step S12 and the passerby detected in step S15 based on the captured images sequentially acquired by the camera 13, and tracks the right-turning vehicle or left-turning vehicle. However, if the vehicle makes contact with any passerby or other vehicle, it is determined that an accident has occurred. When determining that an accident has not occurred (S31: NO), the control unit 11 moves to the process of step S21 and performs the processes from step S21 onwards.

If it is determined that an accident has occurred (S31: YES), the control unit 11 performs a process of reporting to a police station, for example (S32). For example, the control unit 11 outputs report information including information on the location where the traffic accident occurred via the network N to a pre-registered terminal at a police station or security company. Then, the control unit 11 stores an image taken by the camera 13, that is, an accident video obtained by photographing the accident scene with the camera 13, in the storage unit 12 (S33). Note that the control unit 11 determines whether or not the processing related to the accident has been completed based on the captured images sequentially captured by the camera 13, and when it is determined that the processing related to the accident has been completed, the control unit 11 displays the accident video captured by the camera 13. Ends amnestic processing. In addition, when the control unit 11 receives an instruction to end the recording process of the accident video via the communication unit 14 or the input unit (not shown), the control unit 11 may terminate the storage process of the accident video captured by the camera 13. good.

Through the above-described process, in this embodiment as well, when there is a right-turning vehicle or a left-turning vehicle within the monitoring area, it is possible to alert passersby within the notification area with a caution message displayed on the display device 20. Additionally, users within the notification area can be notified via the user terminal 30 that there is a right-turning vehicle or left-turning vehicle, and users inside the right-turning vehicle or left-turning vehicle can be notified via the in-vehicle terminal 40 that there is a vehicle approaching. It can notify you of the presence of pedestrians and other passersby. By alerting each user within the monitoring area in this way, it is expected that the occurrence of accidents within the monitoring area will be suppressed. Furthermore, in this embodiment, when an accident occurs within the monitoring area, it is possible to notify the user and to accumulate captured images of the accident site (accident video). Accident videos accumulated in this manner can be provided to the police, insurance companies, etc., and can be used as images for understanding the circumstances of the accident. In addition, if the contact information (phone number, etc.) of the family of the driver of each vehicle is registered in the vehicle management server 50 (vehicle information DB 52a), when the occurrence of an accident is detected, the driver of the accident vehicle, etc. The notification may be configured to notify the family members of the person. Furthermore, when the occurrence of an accident is detected, the monitoring device 10 obtains destination information of vehicles around the accident site from the vehicle management server 50, and guides the surrounding vehicles to evacuate from the accident site. The notification information may be configured to be transmitted via network N.

In this embodiment, the same effects as in the first embodiment described above can be obtained. Furthermore, in this embodiment, when a traffic accident such as a collision occurs within the monitoring area, it is possible to report it to a police station, security company, etc., and furthermore, it is possible to collect photographed images of the accident scene. Also in this embodiment, the modifications described in the above-described first embodiment can be applied.

(Embodiment 3)
An information processing system that warns users (passengers, vehicle occupants) who need to be alerted when there is a pedestrian crossing a red light, a pedestrian crossing a crosswalk without traffic lights, or a pedestrian crossing a road at a location without a crosswalk will be described. Note that pedestrians crossing a red light, a pedestrian crossing a crosswalk without traffic lights, and a pedestrian crossing a road at a location without a crosswalk are collectively referred to as "dangerous pedestrians." The dangerous pedestrians here include pedestrians, people using bicycles, tricycles, skateboards, kick scooters, electric wheelchairs, senior cars, etc. The information processing system 100 of this embodiment can be realized by a device similar to the information processing system 100 of the first embodiment, so a description of the configuration will be omitted. The information processing system 100 of this embodiment is provided in each monitoring area, not only for road intersections, but also for crosswalks without traffic lights, and places where pedestrians frequently cross the road at places without traffic lights and crosswalks, etc. The information processing system 100 of this embodiment can perform the same processes as those of the above-described embodiments, and when the same processes are performed, the same effects can be obtained.

Below, in the information processing system 100 of the present embodiment, when the monitoring device 10 detects a dangerous cross-crosser in the monitoring area, processing for notifying users in a predetermined notification area will be described. The processing performed by each device will be explained. FIG. 9 is a flowchart illustrating an example of a notification processing procedure according to the third embodiment. In FIG. 9, the left side shows the processing performed by the in-vehicle terminal 40, and the right side shows the processing performed by the monitoring device 10. The following processes are executed by the control unit 11 according to the control program 12P stored in the storage unit 12 of the monitoring device 10, and are executed by the control unit 41 according to the control program 42P stored in the storage unit 42 of the in-vehicle terminal 40. Ru. Part of the following processing may be realized by a dedicated hardware circuit.

In the information processing system 100 of this embodiment, the control unit 11 of the monitoring device 10 photographs the monitoring area using the camera 13 (S41). Also in this embodiment, the control unit 11 may always take pictures with the camera 13, and if the information processing system 100 has a moving body sensor that detects the presence or absence of a moving body within the monitoring area, the moving body sensor detects the presence of a moving body. Photographing by the camera 13 may be started after the camera 13 is set. The control unit 11 determines whether or not a dangerous cross-crosser is included in the captured image based on the captured image captured by the camera 13 (S42). Here, the control unit 11 uses, for example, the pedestrian discrimination model M2 to determine whether or not a dangerous cross-crosser is included in the photographed image. Specifically, the control unit 11 inputs the photographed image to the pedestrian discrimination model M2, and determines whether or not there is a dangerous cross-crosser in the photographed image based on the output information from the pedestrian discrimination model M2. Note that the pedestrian discrimination model M2 of this embodiment is trained to discriminate not only whether there are passersby such as pedestrians in an image, but also whether there is a dangerous cross-crosser. Therefore, in the pedestrian discrimination model M2, when the maximum output value (discrimination probability) is greater than or equal to a predetermined value (for example, 0.7), the control unit 11 sets the state corresponding to the output node that outputs the maximum output value. (Presence or absence of passersby or dangerous crossers) is identified in the photographed image. Note that the control unit 11 may determine whether there is a passerby or a dangerous cross-crosser in the photographed image by pattern matching using a template.

When the control unit 11 determines that there is no dangerous cross-crosser within the monitoring area (S42: NO), the process returns to step S41. If it is determined that there is a dangerous crosser in the monitoring area (S42: YES), the control unit 11 specifies a notification area that requires a warning regarding the dangerous crosser (S43). For example, the control unit 11 specifies roads within a predetermined range around the dangerous cross-crosser as the notification area. When the control unit 11 identifies the notification area, it determines whether there is a vehicle within the notification area (S44). Here, if the specified notification area can be photographed by the camera 13, the control unit 11 determines whether or not there is a vehicle within the notification area based on the photographed image taken by the camera 13. On the other hand, if the specified notification area cannot be photographed with the camera 13, the control unit 11 transmits, for example, position information indicating the notification area to the vehicle management server 50 via the network N, and the vehicle management server 50 The presence or absence of a vehicle may also be determined.

If it is determined that there is a vehicle within the notification area (S44: YES), the control unit 11 acquires destination information assigned to the in-vehicle terminal 40 of the vehicle within the notification area from the vehicle management server 50 (S45). . At this time, the control unit 11 transmits the location information of the monitoring device 10 or the location information of the notification area calculated from the location information of the monitoring device 10 to the vehicle management server 50 via the network N, and The destination information of the in-vehicle terminal 40 is acquired from the vehicle management server 50. The control unit 51 of the vehicle management server 50 reads destination information corresponding to a vehicle whose current location is within a predetermined range from the position indicated by the position information received from the monitoring device 10 from the vehicle information DB 52a, and sends the destination information to the monitoring device 10. Send. At this time, if the traveling direction of each vehicle is registered in the vehicle information DB 52a, the control unit 51 identifies the vehicle whose traveling direction is in the direction of the dangerous cross-country person among the vehicles within the notification area, and specifies the vehicle. The destination information of the vehicle may be transmitted to the monitoring device 10.

When the control unit 11 acquires the destination information of the vehicles within the notification area, the control unit 11 sends information to the in-vehicle terminal 40 of each vehicle within the notification area via the communication unit 14 based on the destination information to notify the in-vehicle terminal 40 of each vehicle within the notification area that there is danger in the surroundings or in the direction of travel. Notification information notifying that there is a person crossing the road is output (S46). When the control unit 41 of the in-vehicle terminal 40 receives the notification information of the dangerous cross-crosser output from the monitoring device 10, the control unit 41 executes notification processing by the notification unit 45 (S47). Thereby, it is possible to notify a user driving a vehicle within the notification area that there is a dangerous cross-crosser. Here, too, the control unit 11 may include the photographed image in which it is detected that there is a dangerous cross-crosser in the monitoring area in step S<b>42 in the notification information, and transmit the notification information to the in-vehicle terminal 40 . In this case, the photographed image included in the notification information is displayed on the display unit (not shown) of the vehicle-mounted terminal 40, and the occupants in the vehicle can be notified that there is a dangerous cross-crosser in the direction of travel of the vehicle.

When the control unit 11 determines that there is no vehicle in the notification area (S44: NO), it skips the processing of steps S45 to S46. Next, the control unit 11 determines whether the dangerous crosser detected in step S42 has passed through the monitoring area, based on the photographed image of the monitoring area (S48). Here, the control unit 11 performs a process of tracking the dangerous cross-crosser detected in step S42 based on the photographed images sequentially acquired by the camera 13, and when the dangerous cross-crosser disappears from the photographed images, or When the dangerous crosser completes crossing the road, it may be determined that the dangerous crosser has passed through the monitoring area. When determining that the dangerous crosser has not passed through the monitoring area (S48: NO), the control unit 11 returns to the process of step S44 and continues the process of determining whether there is a vehicle within the notification area. . As a result, when there is a person crossing the road dangerously within the monitoring area, it is possible to notify vehicles within the notification area that there is a person crossing the road dangerously.

If it is determined that the dangerous crosser has passed through the monitoring area (S48: YES), the control unit 11 sends a safe message to the in-vehicle terminal 40 of the vehicle within the notification area based on the destination information acquired in step S45. Safety information notifying the state is output (S49). When the control unit 41 of the in-vehicle terminal 40 receives the safety information output from the monitoring device 10, it ends the notification process by the notification unit 45 started in step S47 (S50). Here too, the control unit 41 may notify the user in the vehicle of the safe state through other notification processing by the notification unit 45.

Through the above-described processing, the monitoring device 10 of the present embodiment monitors the presence or absence of dangerous crossers in the monitoring area based on the photographed image of the monitoring area. When a dangerous crosser is detected, for example, the vehicle-mounted terminal 40 of a vehicle traveling in the direction of the dangerous crosser is notified that there is a dangerous crosser in the direction of travel. This is expected to alert drivers and others in the vehicle, and reduce the occurrence of accidents involving dangerous crossers. Note that the monitoring device 10 may display a caution message such as "No crossing the road!" or "Please do not cross the road here" on the display device 20 provided within the monitoring area. In this case, it is possible to notify the person crossing the road and the people around him/her of the danger. Furthermore, the monitoring device 10 may notify the user terminals 30 within the range where wireless communication is possible that there is a dangerous cross-crosser. In this case, people around the person crossing the road can be notified of the danger. Particularly, when the dangerous cross-crosser is carrying the user terminal 30, it is possible to notify the dangerous cross-crosser himself/herself of the danger and issue a warning.

In this embodiment, the monitoring device 10 is configured to monitor, in addition to dangerous crossers as described above, parks and playgrounds located in places facing the road, and to monitor jumping out of parks and playgrounds. You can also do that. Specifically, when a photographed image is input, the pedestrian discrimination model M2 is trained to output information indicating whether or not there is a person who is about to run out onto the road in the photographed image. By using such a pedestrian discrimination model M2, it is possible to determine whether a person in a photographed image is about to run out onto the road, and when it is detected that a child or the like has run out from a park or playground, etc. , it is possible to notify passengers of vehicles traveling nearby.

In this embodiment, the same effects as in each of the above-described embodiments can be obtained. In addition, in this embodiment, not only crossers who are crossing the road dangerously, such as crosswalks with red lights and crosswalks where there are no crosswalks, etc. Even if there is a crosswalk at a crosswalk without a traffic light, the occupants of nearby vehicles can be notified of the presence of the dangerous crosswalk in the direction of travel. Therefore, it is expected that accidents that may occur due to dangerous crossers can be suppressed. Also in this embodiment, the modifications described in each of the above-described embodiments can be applied.

(Embodiment 4)
A description will be given of an information processing system in which a plurality of monitoring areas are set on a road, and when a vehicle (hereinafter referred to as a dangerous vehicle) is driving dangerously in one monitoring area, the information processing system warns users (passersby and vehicle occupants) in another monitoring area in the direction of travel of the dangerous vehicle. The dangerous vehicles here are assumed to be vehicles such as bicycles, automobiles, and motorcycles, but may also be skateboards, kick scooters, electric wheelchairs, senior cars, etc. The dangerous vehicles also include vehicles that ignore lanes (lane ignoring), vehicles that drive at a certain speed or faster (speeding), vehicles that drive in a zigzag manner (zigzag driving), vehicles that ignore traffic signals (signal ignoring), vehicles that drive aggressively (tailgating), etc.

Figure 10 is a schematic diagram showing an example of the configuration of the information processing system 100 of the fourth embodiment. The information processing system 100 of this embodiment is provided in each monitoring area, with road intersections and the like as the monitoring area, as in each of the above-mentioned embodiments. For example, the first monitoring area and the second monitoring area in Figure 10 are adjacent monitoring areas and are connected by roads extending in the vertical direction. In this situation, in this embodiment, for example, when a dangerous vehicle is detected in the first monitoring area, if the traveling direction of this dangerous vehicle is the second monitoring area, the presence of the dangerous vehicle is notified to users (passersby and vehicle occupants) in the second monitoring area. Specifically, when a monitoring device 10 (first monitoring device 10) provided in the first monitoring area detects a dangerous vehicle in the first monitoring area based on a captured image, it notifies the monitoring device 10 (second monitoring device 10) provided in the second monitoring area, and the second monitoring device 10 alerts users in the second monitoring area. At that time, the first monitoring device 10 transmits the captured image of the dangerous vehicle to the second monitoring device 10, and if the second monitoring device 10 detects that a dangerous vehicle is in or approaching the second monitoring area based on the captured image of the second monitoring area and the captured image of the dangerous vehicle acquired from the first monitoring device 10, it alerts users in the second monitoring area.

In this embodiment, the information processing system 100 provided in each monitoring area can be realized by the same device as the information processing system 100 of the first embodiment, so a description of the configuration will be omitted. Further, the information processing system 100 of this embodiment can perform the same processing as each of the embodiments described above, and when the same processing is performed, the same effects can be obtained. In addition to the configuration shown in FIG. 3, the monitoring device 10 of this embodiment stores in the storage unit 12 a dangerous vehicle discrimination model M3 learned by machine learning.

FIG. 11 is a schematic diagram showing a configuration example of the dangerous vehicle discrimination model M3. The dangerous vehicle discrimination model M3 of this embodiment has the same configuration as the vehicle discrimination model M1 shown in FIG. This is a trained model that has been trained to calculate whether a vehicle is driving in a meandering manner, a vehicle that ignores traffic lights, or a vehicle that is driving erratically, and outputs the calculated results. The dangerous vehicle discrimination model M3 has, for example, five output nodes in the output layer, and each output node has a discrimination probability for each of a vehicle ignoring lanes, a vehicle exceeding speed, a vehicle driving in a meandering manner, a vehicle ignoring traffic lights, and a vehicle driving erratically. Output. The discrimination probability output by each output node is that the vehicle in the captured image input to the input layer is a vehicle associated with each output node (specifically, a vehicle ignoring lanes, a vehicle exceeding speed, or a meandering vehicle). Indicates the possibility that the vehicle is a driving vehicle, a vehicle ignoring a traffic light, or a vehicle driving in a reckless manner.

The dangerous vehicle discrimination model M3 is trained using training data that includes captured images and information (correct label) indicating the behavior of the vehicle in the captured images (ignoring lanes, speeding, meandering driving, ignoring traffic lights, tailgating). do. Dangerous vehicle discrimination model M3 outputs an output value of 1.0 from the output node corresponding to the behavior indicated by the correct label included in the teacher data when a captured image included in the teacher data is input, and outputs an output value of 1.0 from other output nodes. Learn so that the node outputs an output value of 0.0. As a result, when a photographed image is input, a trained dangerous vehicle discrimination model M3 that has been trained to discriminate the behavior of a vehicle in the photographed image is obtained. Learning of the dangerous vehicle discrimination model M3 may also be performed by another learning device, or may be performed by the monitoring device 10. Furthermore, the dangerous vehicle discrimination model M3 is not limited to the neural network shown in FIG. 4, but learning models constructed by various machine learning algorithms can be used.

In the following, in the information processing system 100 of the present embodiment, when the first monitoring device 10 performs a process of notifying (calling attention to) a user in the second monitoring area when the first monitoring device 10 detects a dangerous vehicle in the first monitoring area, The processing performed by each device will be explained below. 12 and 13 are flowcharts showing an example of a notification processing procedure according to the fourth embodiment. In FIG. 12, the left side shows the process performed by the first monitoring device 10, and the right side shows the process performed by the second monitoring device 10. In FIG. 13, the left side shows the process performed by the in-vehicle terminal 40 in the second monitoring area, and the center The processing performed by the second monitoring device 10 is shown on the right, and the processing performed by the user terminal 30 in the second monitoring area is shown on the right.

In the information processing system 100 of this embodiment, the control unit 11 of the first monitoring device 10 captures an image of the first monitoring area (S61) and determines whether or not a reckless vehicle is included in the captured image based on the captured image (S62). Here, the control unit 11 (detection unit) inputs the captured image to the reckless vehicle discrimination model M3 and determines whether or not a reckless vehicle is included in the captured image based on the output information from the reckless vehicle discrimination model M3. For example, when the maximum output value (discrimination probability) in the reckless vehicle discrimination model M3 is equal to or greater than a predetermined value (e.g., 0.7), the control unit 11 identifies that the captured image contains a vehicle whose behavior (lane ignoring, speeding, meandering, running red lights, tailgating) corresponds to the output node that outputs the maximum output value.

When the control unit 11 determines that there is no dangerous vehicle within the monitoring area (S62: NO), the process returns to step S61. If it is determined that there is a dangerous vehicle within the monitoring area (S62: YES), the control unit 11 (identifying unit) specifies a notification area where a warning regarding the dangerous vehicle is required, in this case a second monitoring area (S63). . For example, the control unit 11 identifies the traveling direction of the detected dangerous vehicle based on the captured image, and identifies an adjacent monitoring area (second monitoring area) in the traveling direction. Note that the positional information of each monitoring area is stored, for example, in the storage unit 12 of the monitoring device 10, and the control unit 11 specifies the second monitoring area based on the positional information of each monitoring area. Further, the control unit 11 may acquire the position information of each monitoring area from a predetermined server via the network N, and specify the second monitoring area based on the obtained position information. 2 information on the monitoring area may be acquired.

When the second monitoring area is identified, the control unit 11 of the first monitoring device 10 transmits the captured image of the dangerous vehicle to the second monitoring device 10 provided in the second monitoring area via the network N ( S64). Note that when the first monitoring device 10 identifies a plurality of second monitoring areas as notification areas that require alerting regarding dangerous vehicles, the first monitoring device 10 transmits captured images of the dangerous vehicle to the second monitoring device 10 of each second monitoring area. Send. Further, when a plurality of second monitoring devices 10 are provided in the second monitoring area, the first monitoring device 10 can detect dangerous vehicles from any second monitoring device 10 or a preset second monitoring device 10. Send the captured image. The control unit 11 of the first monitoring device 10 also transmits the type of dangerous vehicle (ignoring lanes, speeding, meandering, ignoring traffic lights, or tailgating) along with the photographed image of the dangerous vehicle to the second monitoring device 10. Good too.

The control unit 11 of the second monitoring device 10 photographs the second monitoring area (S65), and based on the photographed image of the second monitoring area and the photographed image of the dangerous vehicle received from the first monitoring device 10, It is determined whether the vehicle is within or approaching the second monitoring area (S66). Here, the control unit 11 determines whether or not there is an area that matches the photographed image of the dangerous vehicle in the photographed image of the second monitoring area by pattern matching, thereby detecting the dangerous vehicle in or near the second monitoring area. Determine whether or not there is. Further, the control unit 11 of the second monitoring device 10 extracts the license plate information of the dangerous vehicle from the photographed image of the dangerous vehicle received from the first monitoring device 10, and the vehicle with the extracted license plate is photographed in the second monitoring area. Depending on whether there is a dangerous vehicle in the image, it may be determined whether there is a dangerous vehicle in or near the second monitoring area. Note that the first monitoring device 10 extracts the information on the license plate of the dangerous vehicle from the photographed image of the dangerous vehicle and transmits the extracted license plate information to the second monitoring device 10. It may be configured to notify two monitoring devices 10. Thereby, the control unit 11 (determination unit) of the second monitoring device 10 can determine whether there is a vehicle identical to the dangerous vehicle detected within the first monitoring area. If it is determined that there is no dangerous vehicle in or near the second monitoring area (S66: NO), the control unit 11 of the second monitoring device 10 ends the process.

If it is determined that there is a dangerous vehicle in or near the second monitoring area (S66: YES), the control unit 11 of the second monitoring device 10 determines whether there is a dangerous vehicle in the second monitoring area or in the vicinity of the second monitoring area (S66: YES). It is determined whether there are other vehicles (S67). For example, the control unit 11 inputs the photographed image of the second monitoring area to the vehicle discrimination model M1, and determines whether or not there is a vehicle in the second monitoring area based on the output information from the vehicle discrimination model M1. Good too. If it is determined that there is a vehicle within the second monitoring area (S67: YES), the control unit 11 acquires destination information of the in-vehicle terminal 40 of the vehicle within the second monitoring area from the vehicle management server 50 (S68). The process here is similar to the process of acquiring vehicle destination information from the vehicle management server 50, which was described in each of the embodiments described above.

The control unit 11 (output unit) of the second monitoring device 10 determines whether a dangerous vehicle is nearby, based on the destination information acquired from the vehicle management server 50, with respect to the in-vehicle terminal 40 of each vehicle in the second monitoring area. The notification information notifying the user is output via the network N (S69). When the control unit 41 of the in-vehicle terminal 40 receives the notification information of a dangerous vehicle output from the monitoring device 10, the control unit 41 executes notification processing by the notification unit 45 (S70). Thereby, passengers of vehicles within the second monitoring area can be notified of the presence of a dangerous vehicle nearby. Here, too, the control unit 11 may include the photographed image of the dangerous vehicle acquired from the first monitoring device 10 in the notification information and transmit it to the in-vehicle terminal 40. In addition, if the second monitoring device 10 is notified of the type of dangerous vehicle (ignoring lanes, speeding, meandering driving, ignoring traffic lights, or tailgating) from the first monitoring device 10, the control unit of the second monitoring device 10 11 may transmit notification information that notifies the presence of a dangerous vehicle nearby and the type of dangerous vehicle to the in-vehicle terminal 40 of each vehicle. In this case, the in-vehicle terminal 40 can also notify the passenger of the type of dangerous vehicle through the notification unit 45, and the passenger can understand what kind of dangerous vehicle is approaching, so that the passenger can take prompt action. It becomes possible to take evacuation actions.

When the control unit 11 determines that there is no vehicle in the second monitoring area (S67: NO), it skips the processing of steps S68 to S69. Next, the control unit 11 determines whether there is a passerby such as a pedestrian in the second monitoring area based on the photographed image of the second monitoring area (S71). If it is determined that there is a passerby within the second monitoring area (S71: YES), the control unit 11 (output unit) notifies you that a dangerous vehicle is nearby by wireless communication via the wireless communication unit 15. Notification information is output (S72). When the control unit 31 of the user terminal 30 within the communicable range of the second monitoring device 10 receives notification information of a dangerous vehicle output from the second monitoring device 10, the control unit 31 causes the notification unit 34 to execute a notification process. (S73). Thereby, the user of the user terminal 30 can be notified of the presence of a dangerous vehicle.

When the control unit 11 determines that there are no passersby in the second monitoring area (S71: NO), the control unit 11 skips the process of step S72 and determines whether the dangerous vehicle is in the second monitoring area based on the captured image of the second monitoring area. It is determined whether the area has been passed (S74). If it is determined that the dangerous vehicle has not passed through the second monitoring area (S74: NO), the control unit 11 returns to the process of step S67 and determines the presence or absence of a vehicle and the presence or absence of a passerby in the second monitoring area. Continue processing. As a result, when there is a dangerous vehicle in or near the second monitoring area, it is possible to notify passengers of vehicles and passersby in the second monitoring area that there is a dangerous vehicle.

If it is determined that the dangerous vehicle has passed through the second monitoring area (S74: YES), the control unit 11 outputs safety information by wireless communication via the wireless communication unit 15 (S75), and outputs safety information to the control unit of the user terminal 30. 31 ends the notification process started in step S73 when safety information is received from the second monitoring device 10 (S76). As a result, after the dangerous vehicle passes through the second monitoring area, the notification process by the user terminal 30 ends. Furthermore, the control unit 11 outputs safety information via the network N to the in-vehicle terminal 40 of the vehicle in the second monitoring area based on the destination information acquired in step S68 (S77), and the control unit of the in-vehicle terminal 40 41 ends the notification process started in step S70 when safety information is received from the second monitoring device 10 (S78).

Through the above-described process, in the present embodiment, when a dangerous vehicle is detected in the first monitoring area, it is possible to alert passersby and vehicle occupants in the second monitoring area. Therefore, it is possible to notify passersby in the second monitoring area and vehicle occupants who are unaware of the presence of the dangerous vehicle. Note that the second monitoring device 10 not only transmits notification information of a dangerous vehicle to the user terminal 30 and the in-vehicle terminal 40 in the second monitoring area, but also displays a caution message on the display device 20 in the second monitoring area. It's okay. In this case, even people who do not carry the user terminal 30 can be notified of the presence of a dangerous vehicle.

In this embodiment, the second monitoring device 10 determines whether or not this dangerous vehicle is in or near the second monitoring area based on the photographed image of the dangerous vehicle acquired from the first monitoring device 10, and if the dangerous vehicle is present, Processing was performed to notify passersby and vehicle occupants within the second monitoring area. In addition, the second monitoring device 10 performs a process of notifying passersby and vehicle occupants in the second monitoring area without determining whether a dangerous vehicle is in or near the second monitoring area. It's okay. That is, when the first monitoring device 10 detects a dangerous vehicle within the first monitoring area, it specifies the second monitoring area and notifies the second monitoring device 10 of the presence of the dangerous vehicle. When the second monitoring device 10 receives the notification of the dangerous vehicle from the first monitoring device 10, the second monitoring device 10 transmits the notification information of the dangerous vehicle to the in-vehicle terminal 40 in the second monitoring area via the network N. Dangerous vehicle notification information is transmitted to user terminals 30 within the area by wireless communication. With this configuration, it is possible to notify passersby and vehicle occupants in the second monitoring area before the dangerous vehicle enters the second monitoring area, so early notification is possible and the notification can be received. This allows users to evacuate quickly.

Furthermore, in this embodiment, the images taken by each monitoring device 10 are aggregated on the cloud server via the network N, and the cloud server performs processing to detect dangerous vehicles within each monitoring area. When detected, it may be configured to perform a process of notifying passersby in the surrounding monitoring area and vehicle passengers (user terminal 30 and vehicle-mounted terminal 40). In the case of such a configuration, the monitoring device 10 may perform a photographing process using the camera 13 and a process of transmitting the photographed image to the cloud server via the network N.

In this embodiment, the same effects as in each of the above-described embodiments can be obtained. Furthermore, in this embodiment, when a dangerous vehicle is detected in the first monitoring area, nearby passersby in the second monitoring area and the occupants of the vehicle are notified. It is also possible to alert the occupants of the vehicle. This is expected to suppress the occurrence of accidents caused by dangerous vehicles detected in the first monitoring area in the second monitoring area. Also in this embodiment, the modifications described in each of the above-described embodiments can be applied.

(Embodiment 5)
An information processing system that alerts users (passersby and vehicle occupants) in the direction of travel of an emergency vehicle when the emergency vehicle is traveling on a road will be described. Emergency vehicles include, for example, police vehicles, fire trucks, ambulances, etc., and refer to vehicles that are driving in an emergency with sirens sounding and red warning lights turned on. Since the information processing system 100 of this embodiment can be realized by the same device as the information processing system 100 of Embodiment 1, description of the configuration will be omitted. The information processing system 100 of this embodiment can perform the same processing as each of the embodiments described above, and when the same processing is performed, the same effects can be obtained.

Below, in the information processing system 100 of this embodiment, when the monitoring device 10 detects an emergency vehicle, each device performs a process of notifying a user within a predetermined area (notification area) in the traveling direction of the emergency vehicle. We will explain the processing performed by . 14 and 15 are flowcharts illustrating an example of a notification processing procedure according to the fifth embodiment. 14 and 15, the left side shows the process performed by the in-vehicle terminal 40, the center shows the process performed by the monitoring device 10, and the right side shows the process performed by the user terminal 30.

In the information processing system 100 of the present embodiment, the control unit 11 of the monitoring device 10 photographs the monitoring area (S91), and determines whether an emergency vehicle running in an emergency is included in the photographed image based on the photographed image. is determined (S92). Here, the control unit 11 uses, for example, the vehicle discrimination model M1 to determine whether an emergency vehicle is included in the photographed image. The vehicle discrimination model M1 of this embodiment is trained to discriminate not only whether a vehicle in an image is a right-turning vehicle, a left-turning vehicle, or a straight-going vehicle, but also whether or not it is an emergency vehicle. That is, when a captured image (vehicle image) is input, the vehicle discrimination model M1 outputs the probability (discrimination probability) that the vehicle in the captured image is a right-turning vehicle, a left-turning vehicle, a straight-going vehicle, or an emergency vehicle. . Therefore, the control unit 11 can determine whether an emergency vehicle is included in the captured image based on the discrimination probability output from the vehicle discrimination model M1. Photographed images (vehicle images) used as training data for the vehicle discrimination model M1 include, for example, a vehicle type used as an emergency vehicle, an emergency vehicle with a warning light on, an emergency vehicle running in an emergency, etc. A photographed image can be used. Note that the control unit 11 may determine whether an emergency vehicle is included in the captured image by pattern matching using a template. In addition, when the emergency vehicle has a transmitter that transmits a signal indicating the approach of its own vehicle (emergency vehicle) by wireless communication, the control unit 11 receives the signal transmitted from the transmitter of the emergency vehicle to The approach of a vehicle may also be detected.

When the control unit 11 determines that there is no emergency vehicle within the monitoring area (S92: NO), the process returns to step S91. If it is determined that there is an emergency vehicle within the monitoring area (S92: YES), the control unit 11 specifies a notification area that requires alerting regarding the emergency vehicle (S93). For example, the control unit 11 specifies a crosswalk in the direction of travel of the emergency vehicle as the notification area. The control unit 11 displays a message notifying the presence of an emergency vehicle on the display device 20 provided in the specified notification area (S94), and notifies passersby in the notification area of the approach of the emergency vehicle. Note that if the display device 20 has a light emitting section, the light emitting section may be turned on or blinking, and if the monitoring device 10 or the display device 20 has a speaker, an audio message may be output from the speaker to alert the user.

Next, the control unit 11 determines whether or not there is a vehicle in the notification area (S95). If it is determined that there is a vehicle in the notification area (S95: YES), it acquires destination information of the vehicle (vehicle-mounted terminal 40) in the notification area from the vehicle management server 50 (S96). Then, based on the acquired destination information, the control unit 11 outputs notification information notifying the approach of an emergency vehicle to the vehicle-mounted terminal 40 of the vehicle in the notification area via the network N (S97). If the control unit 41 of the vehicle-mounted terminal 40 receives the notification information of the emergency vehicle output from the monitoring device 10, it executes notification processing by the notification unit 45 (S98). Here, the control unit 41 may output, for example, a voice message notifying the approach of the emergency vehicle together with a voice message to guide the vehicle to a safe place, and in this case, it is possible to guide the passenger to evacuate the vehicle to a safe place. If the control unit 11 determines that there is no vehicle in the notification area (S95: NO), it skips the processing of steps S96 to S97.

Next, the control unit 11 judges whether or not there is a pedestrian or other passerby in the notification area (S99). If it is judged that there is a passerby in the notification area (S99: YES), the control unit 11 outputs notification information notifying the approach of an emergency vehicle by wireless communication via the wireless communication unit 15 (S100). When the control unit 31 of the user terminal 30 receives the notification information of the emergency vehicle output from the monitoring device 10, it executes notification processing by the notification unit 34 (S101). If the control unit 11 judges that there is no passerby in the notification area (S99: NO), it skips the processing of step S100 and judges whether or not the emergency vehicle has passed the monitoring area based on the captured image of the monitoring area (S102). If it is judged that the emergency vehicle has not passed the monitoring area (S102: NO), the control unit 11 returns to the processing of step S94. As a result, when an emergency vehicle is traveling in the monitoring area, it is possible to notify the passengers of the vehicle and passersby in the notification area in the direction of travel of the approach of the emergency vehicle.

When determining that the emergency vehicle has passed through the monitoring area (S102: YES), the control unit 11 outputs safety information by wireless communication via the wireless communication unit 15 (S103), and receives safety information from the monitoring device 10. The control unit 31 of the user terminal 30 that has done so ends the notification process started in step S101 (S104). Further, the control unit 11 ends the display of the message displayed on the display device 20 in step S94 (S105), and sends messages to vehicles (in-vehicle terminals 40) in the monitoring area based on the destination information acquired in step S96. Safety information is output via network N (S106). When the control unit 41 of the in-vehicle terminal 40 receives the safety information from the monitoring device 10, it ends the notification process started in step S98 (S107).

Through the processing described above, the monitoring device 10 of this embodiment monitors the presence or absence of an emergency vehicle within the monitoring area based on the captured image of the monitoring area. When the monitoring device 10 detects an emergency vehicle, the monitoring device 10 alerts passersby and vehicles within the notification area in the direction of travel of the emergency vehicle, thereby guiding the passersby and vehicles within the notification area to evacuate. can do. Moreover, when the monitoring device 10 of this embodiment detects an emergency vehicle that is running in an emergency, it may notify the monitoring devices 10 in other monitoring areas in the traveling direction of the emergency vehicle. In this case, the monitoring device 10 in the other monitoring area can alert passersby and vehicles in the other monitoring area.

In this embodiment, the same effects as those of the above-mentioned embodiments can be obtained. In addition, in this embodiment, it is possible to notify passersby and vehicle occupants in the direction of travel of the emergency vehicle of the approach of the emergency vehicle. Therefore, it is possible to urge passersby such as pedestrians and vehicles in the direction of travel of the emergency vehicle to quickly evacuate to a safe place. In this embodiment, too, it is possible to apply the modified examples described as appropriate in each of the above-mentioned embodiments.

(Embodiment 6)
An information processing system that warns a passenger of a vehicle traveling in the wrong direction when there is a vehicle traveling in the wrong direction on a road will be described. A wrong-way vehicle is a vehicle such as a car or a motorcycle, and refers to a vehicle that is traveling in the opposite direction to the direction of travel determined with respect to the road. Since the information processing system 100 of this embodiment can be realized by the same device as the information processing system 100 of Embodiment 1, description of the configuration will be omitted. The information processing system 100 of this embodiment can perform the same processing as each of the embodiments described above, and when the same processing is performed, the same effects can be obtained.

Below, in the information processing system 100 of this embodiment, when the monitoring device 10 detects a wrong-way vehicle, it warns the occupants (driver, etc.) of the wrong-way vehicle, and warns surrounding vehicles (general vehicles). The process performed by each device when performing the process of notifying the passenger will be explained. FIG. 16 is a flowchart illustrating an example of a notification processing procedure according to the sixth embodiment. In FIG. 16, the left side shows the processing performed by the on-vehicle terminal 40 of a vehicle traveling the wrong way, the center shows the processing performed by the monitoring device 10, and the right side shows the processing performed by the on-vehicle terminal 40 of a general vehicle.

In the information processing system 100 of the present embodiment, the control unit 11 of the monitoring device 10 photographs the monitoring area (S111), and based on the photographed image, determines whether or not the photographed image includes a vehicle driving in the wrong direction. (S112). Here, the control unit 11 uses, for example, the vehicle discrimination model M1 to determine whether or not a vehicle running the wrong way is included in the photographed image. The vehicle discrimination model M1 of the present embodiment is trained to discriminate not only whether a vehicle in an image is a right-turning vehicle, a left-turning vehicle, or a vehicle traveling straight, but also whether a vehicle is traveling in the wrong direction. That is, when a captured image (vehicle image) is input, the vehicle discrimination model M1 outputs the probability (discrimination probability) that the vehicle in the captured image is a right-turning vehicle, a left-turning vehicle, a vehicle going straight, or a vehicle driving the wrong way. do. Therefore, the control unit 11 can determine whether or not a wrong-way vehicle is included in the photographed image based on the discrimination probability output from the vehicle discrimination model M1. As the photographed image (vehicle image) used as training data for such a vehicle discrimination model M1, for example, a photographed image of a vehicle traveling in the wrong direction on a general road, an expressway, etc. can be used. Note that the control unit 11 may determine whether a vehicle driving the wrong way is included in the captured image by pattern matching using a template.

When the control unit 11 determines that there is no wrong-way vehicle within the monitoring area (S112: NO), the process returns to step S111. If it is determined that there is a vehicle running the wrong way within the monitoring area (S112: YES), the control unit 11 acquires the destination information of the on-vehicle terminal 40 of the vehicle running the wrong way from the vehicle management server 50 (S113). Based on the acquired destination information, the control unit 11 outputs warning information to notify and warn that the vehicle is traveling in the wrong direction via the network N to the in-vehicle terminal 40 of the vehicle traveling in the wrong direction (S114). When the control unit 41 of the in-vehicle terminal 40 of the wrong-way vehicle receives the warning information output from the monitoring device 10, the control unit 41 executes the warning process by the notification unit 45 (S115). For example, the control unit 41 issues a warning by outputting a voice message notifying that the vehicle is traveling in the wrong direction, a voice message for guiding the vehicle to a safe place, and the like. As a result, it is possible to notify the driver of the wrong-way vehicle that the vehicle is traveling in the wrong direction, and to guide the wrong-way vehicle to retreat to a safe place immediately.

Next, the control unit 11 specifies a notification area that requires a warning regarding a wrong-way vehicle (S116). For example, the control unit 11 specifies, as the notification area, a crosswalk in the direction of travel of the wrong-way vehicle. The control unit 11 determines whether there is a vehicle (general vehicle) within the specified notification area (S117), and if it is determined that there is a general vehicle within the notification area (S117: YES), the control unit 11 determines whether or not there is a vehicle (general vehicle) within the specified notification area. The destination information of the general vehicle (in-vehicle terminal 40) is acquired from the vehicle management server 50 (S118). Based on the acquired destination information, the control unit 11 outputs notification information notifying the presence of the wrong-way vehicle to the on-vehicle terminal 40 of the general vehicle within the notification area via the network N (S119). When the control unit 41 of the in-vehicle terminal 40 of the general vehicle receives the notification information of the wrong-way vehicle output from the monitoring device 10, the control unit 41 executes notification processing by the notification unit 45 (S120). When the control unit 11 determines that there is no general vehicle in the notification area (S117: NO), it skips the processing of steps S118 to S119.

Next, the control unit 11 judges whether the wrong-way vehicle has passed the monitoring area based on the captured image of the monitoring area (S121), and if it judges that the wrong-way vehicle has not passed (S121: NO), the process returns to step S117. This allows passengers of general vehicles in the direction of travel of the wrong-way vehicle to be notified of the presence of the wrong-way vehicle when the wrong-way vehicle is traveling in the monitoring area. If it judges that the wrong-way vehicle has passed the monitoring area (S121: YES), the control unit 11 outputs safety information to the general vehicle (vehicle-mounted terminal 40) in the notification area via the network N based on the destination information acquired in step S118 (S122). If the control unit 41 of the vehicle-mounted terminal 40 of the general vehicle receives safety information from the monitoring device 10, it ends the notification process started in step S120 (S123). In addition, if the control unit 11 detects that the wrong-way vehicle has stopped reverse driving based on the captured image, it may output safety information to the wrong-way vehicle notifying that it is no longer in a reverse driving state based on the destination information of the wrong-way vehicle acquired in step S113. As a result, the vehicle-mounted terminal 40 of the wrong-way vehicle may end the warning process that began in step S115.

Through the above-described processing, the monitoring device 10 of the present embodiment monitors the presence or absence of a wrong-way vehicle within the monitoring area based on the photographed image of the monitoring area. When the monitoring device 10 detects a wrong-way vehicle, it not only warns the occupants of the wrong-way vehicle, but also notifies the occupants of regular vehicles in the direction of travel of the wrong-way vehicle to alert them to the presence of the wrong-way vehicle. can do. Furthermore, when the monitoring device 10 of this embodiment detects a wrong-way vehicle, it may notify the monitoring devices 10 in other monitoring areas in the direction of movement of the wrong-way vehicle. In this case, the monitoring device 10 in the other monitoring area can alert general vehicles in the other monitoring area.

In this embodiment, the same effects as in each of the above-described embodiments can be obtained. Further, in this embodiment, it is possible to warn the occupants of the vehicle traveling the wrong way, and also to notify the occupants of surrounding general vehicles of the approach of the vehicle traveling the wrong way. Therefore, it is possible to notify the driver of the wrong-way vehicle of the situation in which the vehicle is traveling the wrong way, and to urge surrounding general vehicles to quickly evacuate to a safe location. Also in this embodiment, the modifications described in each of the above-described embodiments can be applied.

(Embodiment 7)
When a dangerous situation occurs on a narrow road, such as a situation where a vehicle is approaching from behind a pedestrian, or a situation where a bicycle is riding through a narrow gap between a pedestrian and a vehicle, An information processing system that alerts users (passersby and vehicle occupants) who need to be alerted will be described. Since the information processing system 100 of this embodiment can be realized by the same device as the information processing system 100 of Embodiment 1, description of the configuration will be omitted. Furthermore, the information processing system 100 of the present embodiment is provided in each monitoring area, not only at road intersections but also at places where collisions are likely to occur on narrow roads. Note that the information processing system 100 of this embodiment can perform the same processing as in each embodiment described above, and when the same processing is performed, the same effects can be obtained. In addition to the configuration shown in FIG. 3, the monitoring device 10 of this embodiment stores in the storage unit 12 a dangerous situation discrimination model (not shown) learned by machine learning.

The dangerous situation discrimination model of this embodiment has the same configuration as the vehicle discrimination model M1 shown in FIG. This is a trained model that has learned to determine whether or not it is true and output the determination result. For example, the dangerous situation discrimination model has two output nodes in the output layer, and each output node has a discrimination probability that indicates the possibility that the situation shown in the photographed image is a dangerous situation, and a discrimination probability that indicates the possibility that the situation shown in the captured image is a dangerous situation. The discriminant probability indicating the gender is output. Such a dangerous situation discrimination model is trained using teacher data including a photographed image and information (correct label) indicating whether the situation in the photographed image is a dangerous situation. The dangerous situation discrimination model generates an output of 1.0 from the output node corresponding to the situation (dangerous situation or not) indicated by the correct label included in the teaching data when a captured image included in the teaching data is input. This value is output, and the other output nodes learn to output an output value of 0.0. As a result, when a photographed image is input, a trained dangerous situation discrimination model that is trained to determine whether or not the situation in the photographed image is a dangerous situation is obtained.

Below, in the information processing system 100 of this embodiment, when the monitoring device 10 detects a dangerous situation within the monitoring area, various steps are taken when performing a process of notifying a user who may be involved in the dangerous situation. The processing performed by the device will be explained. FIG. 17 is a flowchart illustrating an example of a notification processing procedure according to the seventh embodiment. In FIG. 17, the left side shows the process performed by the in-vehicle terminal 40, the center shows the process performed by the monitoring device 10, and the right side shows the process performed by the user terminal 30.

In the information processing system 100 of the present embodiment, the control unit 11 of the monitoring device 10 photographs the monitoring area (S131), and determines whether or not the photographed image includes a dangerous situation based on the photographed image (S131). S132). Here, the control unit 11 uses the dangerous situation determination model to determine whether the situation shown in the photographed image is a dangerous situation. Specifically, the control unit 11 inputs the photographed image into a dangerous situation determination model, and determines whether or not the situation in the photographed image is dangerous based on output information from the dangerous situation determination model. For example, in the dangerous situation determination model, the control unit 11 specifies the situation (whether or not it is a dangerous situation) corresponding to the output node that outputs the maximum (larger) output value as the situation in the photographed image. Note that the control unit 11 may determine whether the situation shown in the photographed image is a dangerous situation by pattern matching using a template.

When the control unit 11 determines that no dangerous situation has occurred within the monitoring area (S132: NO), the process returns to step S131. If it is determined that a dangerous situation has occurred within the monitoring area (S132: YES), the control unit 11 outputs danger information notifying the dangerous situation by wireless communication via the wireless communication unit 15 (S133). . For example, the control unit 11 transmits danger information to the user terminal 30 that notifies the user who is carrying the user terminal 30 that a vehicle is approaching from behind. When the control unit 31 of the user terminal 30 receives danger information from the monitoring device 10, the control unit 31 executes notification processing by the notification unit 34 (S134). Next, the control unit 11 acquires the destination information of the vehicle (in-vehicle terminal 40) causing the dangerous situation from the vehicle management server 50 (S135). Note that the control unit 11 obtains the destination information of this vehicle by using the location information of the monitoring device 10, the location information of the vehicle calculated from the location information of the monitoring device 10, or the traveling direction or vehicle type of this vehicle. can be acquired from the vehicle management server 50.

Based on the acquired destination information, the control unit 11 outputs danger information notifying the vehicle-mounted terminal 40 of the vehicle causing the dangerous situation via the network N (S136). When the control unit 41 of the vehicle-mounted terminal 40 receives the danger information output from the monitoring device 10, the control unit 11 executes a notification process by the notification unit 45 (S137). Based on the captured images of the monitoring area, the control unit 11 judges whether the dangerous situation detected in step S132 has been resolved (S138). Here, the control unit 11 inputs the captured images sequentially acquired by the camera 13 to a dangerous situation determination model, and judges whether the dangerous situation in the captured images has been resolved based on the output information from the dangerous situation determination model. Note that the control unit 11 may perform a process of tracking pedestrians, vehicles, etc. in the captured images based on the captured images sequentially acquired by the camera 13, and judge that the dangerous situation has been resolved when there is no longer a possibility of contact between the pedestrians and vehicles, etc.

If it is determined that the dangerous situation has not been resolved (S138: NO), the control unit 11 returns to the process of step S133 and continues the notification process to the user terminal 30 and the in-vehicle terminal 40. As a result, when a dangerous situation occurs within the monitoring area, pedestrians and vehicle occupants involved in the dangerous situation can be notified of the occurrence of the dangerous situation, and it is possible to prevent the dangerous situation from developing into a collision accident. Be expected. If it is determined that the dangerous situation has been resolved (S138: YES), the control unit 11 outputs safety information by wireless communication via the wireless communication unit 15 (S139), and the control unit 31 of the user terminal 30 outputs safety information to the monitoring device. 10, the notification process started in step S133 is ended (S140). Further, the control unit 11 outputs safety information via the network N to the on-vehicle terminal 40 of the vehicle that caused the dangerous situation based on the destination information acquired in step S135 (S141), and controls the on-vehicle terminal 40. When the unit 41 receives safety information from the monitoring device 10, it ends the notification process started in step S137 (S142).

Through the above-described processing, the monitoring device 10 of this embodiment monitors the occurrence of a dangerous situation within the monitoring area based on the captured image of the monitoring area. When the occurrence of a dangerous situation is detected, users such as pedestrians (user terminal 30) and vehicles (vehicle terminal 40) involved in the dangerous situation are notified of the dangerous situation. As a result, even if a pedestrian is not aware of a vehicle approaching from behind on a narrow road, for example, the pedestrian can be notified via the user terminal 30, so that the pedestrian is not aware of the vehicle approaching from behind. vehicles can be noticed. Furthermore, for example, when a vehicle is traveling on a narrow road, there may be pedestrians, bicycles, etc. very close to it. The information allows the situation of pedestrians and bicycles around the vehicle to be grasped. Therefore, the pedestrians involved, vehicle drivers, etc. can grasp the situation before it develops into a dangerous situation, such as a collision, so it can be expected to suppress the occurrence of accidents.

In this embodiment, the same effects as in each of the above-described embodiments can be obtained. Furthermore, in this embodiment, by setting a narrow road as the monitoring area, even when pedestrians and vehicles are moving back and forth at close distances, each other's presence can be known, thereby reducing the occurrence of collisions. It is expected that Also in this embodiment, the modifications described in each of the above-described embodiments can be applied.

In each of the embodiments described above, by using the vehicle discrimination model M1, it is possible to determine whether a vehicle within the monitoring area in the captured image is a right-turning vehicle, a left-turning vehicle, a straight-going vehicle, an emergency vehicle, a wrong-way vehicle, etc. . Furthermore, by using the pedestrian discrimination model M2, it is possible to discriminate whether there are pedestrians or other passersby in the monitoring area in the photographed image, and whether there are people crossing the road dangerously. Further, by using the dangerous vehicle discrimination model M3, it is possible to determine whether the vehicle in the photographed image is a dangerous vehicle such as a vehicle ignoring lanes, a vehicle exceeding speed, a vehicle driving in a meandering manner, a vehicle ignoring traffic lights, or a vehicle driving erratically. Furthermore, by using the dangerous situation determination model, it can be determined whether the monitoring area in the photographed image is in a dangerous situation. Such discrimination models may be combined into one model, or may be divided into a plurality of models. In addition, the content to be determined by each model is not limited to what is described above, but includes moving objects (moving objects) that perform behaviors (predetermined actions) that may pose a danger to passersby in or near the monitoring area. It suffices if the configuration is such that it is possible to determine the presence or absence of the . With such a configuration, it is possible to monitor all pedestrians, vehicles, etc. in each monitoring area to see if they are in a dangerous state or are likely to become a dangerous state. Then, when a dangerous condition is detected, it is possible to alert a user who needs to be alerted.

The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is indicated by the claims, not by the meaning described above, and is intended to include all modifications within the scope and meaning equivalent to the claims.

10 Monitoring device 11 Control unit 12 Storage unit 13 Camera 15 Wireless communication unit 20 Display device 30 User terminal 31 Control unit 33 Wireless communication unit 34 Notification unit 40 In-vehicle terminal 41 Control unit 43 Communication unit 45 Notification unit 50 Vehicle management server M1 Vehicle discrimination Model M2 Pedestrian discrimination model

Claims (13)

Obtain images obtained by photographing the surveillance area on the road,
For a moving object discrimination model trained to output information indicating whether or not the image contains a moving object that performs a predetermined action that may pose a danger to surrounding people when an image is input. , inputting the acquired photographic image and detecting whether or not a moving object performing the predetermined action is included in the monitoring area photographed in the photographic image based on the information output by the moving object discrimination model; A process of outputting notification information for notifying a user in the monitoring area photographed in the captured image of the presence of a moving body that may pose a danger when a moving body performing an action is detected,
A plurality of monitoring devices are provided that are associated with each of the plurality of monitoring areas on the road and obtain captured images obtained by photographing each of the plurality of monitoring areas,
The first monitoring device associated with the first monitoring area is
When a moving object performing the predetermined action is detected in the first monitoring area based on information outputted by the moving object discrimination model when captured images of the first monitoring area are input to the moving object discrimination model, the Identifying the direction of movement of the moving object performing the predetermined action based on the photographed image of the first monitoring area;
identifying a second monitoring area different from the first monitoring area in the specified direction of travel;
performing a process of transmitting the captured image of the first monitoring area to a second monitoring device associated with the identified second monitoring area;
The second monitoring device includes:
Based on the captured image of the moving object performing the predetermined action in the first monitoring area received from the first monitoring device, the predetermined action detected in the first monitoring area is included in the captured image of the second monitoring area. Detects the presence or absence of a moving object that is the same as the moving object being performed,
If the same moving object is detected, perform a process of outputting notification information to the user within the second monitoring area.
Information processing method. The moving object discrimination model is trained to output information indicating whether a right-turning vehicle or a left-turning vehicle is included in the image;
The first monitoring device,
identifying a notification area according to a right-turning vehicle or a left-turning vehicle detected based on information output by the moving object discrimination model;
Output the notification information to users within the specified notification area
The information processing method according to claim 1. The first monitoring device,
The acquired captured image is input to a pedestrian discrimination model that has been trained to output information indicating whether or not a pedestrian is included in the image when the image is input, and the pedestrian discrimination model is trained to output information indicating whether or not a pedestrian is included in the image. Based on the information output by the model, it is detected whether or not the monitoring area captured in the captured image includes pedestrians who may pose a danger to the moving object performing the predetermined action.
The information processing method according to claim 1 or 2. The first monitoring device,
When the pedestrian is detected within the monitoring area in the photographed image based on the information output by the pedestrian discrimination model, a moving object that may pose a danger to the pedestrian is displayed on the user terminal of the detected pedestrian. Outputs notification information via wireless communication to notify the existence of
The information processing method according to claim 3. The first monitoring device,
transmitting the photographed image in which the pedestrian in the monitoring area is detected based on the information output by the pedestrian discrimination model to the moving object performing the predetermined action;
The information processing method according to claim 3 or 4. The first monitoring device,
Sending information indicating that the moving object is performing the predetermined action to the moving object that is performing the predetermined action.
An information processing method according to any one of claims 1 to 5. The moving object discrimination model is trained to output information indicating whether or not a pedestrian crossing a road with a red light is included in the image when an image is input;
The first monitoring device,
Based on the information output by the moving object discrimination model, when a pedestrian crossing a road with a red light is detected in the acquired photographic image, specifying a vehicle to which notification information should be output;
Output notification information to the in-vehicle terminal installed in the specified vehicle
An information processing method according to any one of claims 1 to 6 . The moving object discrimination model is trained to output information indicating whether or not an emergency vehicle running in an emergency is included in the image when an image is input;
The first monitoring device,
Based on the information output by the moving object discrimination model, when an emergency vehicle running in an emergency is detected in the acquired photographic image, specifying a monitoring area in the direction of travel of the emergency vehicle;
Output notification information to users within the specified monitoring area
An information processing method according to any one of claims 1 to 7 . the moving object discrimination model is trained to output, when an image is input, information indicating whether or not a wrong-way vehicle is included in the image;
The first monitoring device,
When the wrong-way vehicle is detected based on the information output by the moving object discrimination model, notification information is output to an in-vehicle terminal mounted on the wrong-way vehicle.
9. An information processing method according to any one of claims 1 to 8 . The first monitoring device,
Detecting the occurrence of an accident within the monitoring area based on the acquired captured images,
When an accident is detected, images taken in the monitoring area are stored.
An information processing method according to any one of claims 1 to 9 . In an information processing device having a control unit,
The control unit is
Acquire a captured image of a surveillance area on a road,
The acquired captured image is input to a moving object discrimination model that has been trained to output information indicating whether or not a moving object performing a predetermined action that may be dangerous to surrounding people is included in the image when the image is input, and based on the information output by the moving object discrimination model, it is detected whether or not a moving object performing the predetermined action is included in a surveillance area captured in the captured image;
a process of outputting notification information notifying a user in the surveillance area captured in the captured image of the presence of a potentially dangerous moving object when a moving object performing the predetermined action is detected,
when a moving object performing the predetermined action is detected in the first monitoring area based on information output by the moving object discrimination model when a photographed image of the first monitoring area is input to the moving object discrimination model, a moving direction of the moving object performing the predetermined action is identified based on the photographed image of the first monitoring area;
Identifying a second monitoring area different from the first monitoring area in the identified traveling direction;
Transmitting a captured image of the first monitoring area to a second information processing device associated with the identified second monitoring area;
when receiving a photographed image of the moving object performing the predetermined action in another monitoring area from another information processing device associated with the other monitoring area, detecting the presence or absence of a moving object identical to the moving object performing the predetermined action detected in the other monitoring area in the photographed image of the first monitoring area based on the received photographed image of the other monitoring area;
When the same moving object is detected, notification information is output to a user in the first monitoring area.
Information processing device. Obtain images obtained by photographing the surveillance area on the road,
For a moving object discrimination model trained to output information indicating whether or not the image contains a moving object that performs a predetermined action that may pose a danger to surrounding people when an image is input. , inputting the acquired photographic image, and detecting whether or not a moving object performing the predetermined action is included in the monitoring area photographed in the photographic image, based on the information output by the moving object discrimination model;
When a moving object performing the predetermined action is detected, a process of outputting notification information that notifies a user in the monitoring area captured in the photographed image of the presence of a moving object that may pose a danger,
When a moving object performing the predetermined action is detected in the first monitoring area based on information outputted by the moving object discrimination model when captured images of the first monitoring area are input to the moving object discrimination model, the Identifying the direction of movement of the moving object performing the predetermined action based on the photographed image of the first monitoring area;
identifying a second monitoring area different from the first monitoring area in the specified direction of travel;
transmitting the captured image of the first monitoring area to a second information processing device associated with the identified second monitoring area;
When a captured image of a moving object performing the predetermined action in another monitoring area is received from another information processing device associated with the other monitoring area, based on the received captured image of the other monitoring area. , detecting the presence or absence of a moving object that is the same as a moving object performing the predetermined action detected in the other monitoring area in the photographed image of the first monitoring area;
If the same moving object is detected, output notification information to the user within the first monitoring area.
A program that causes an information processing device to execute a process. Equipped with multiple photographic devices that photograph multiple monitoring areas on the road,
The first photographing device that photographs the first monitoring area is
For a moving object discrimination model trained to output information indicating whether or not the image contains a moving object that performs a predetermined action that may pose a danger to surrounding people when an image is input. , a detection unit that receives an image taken of a first monitoring area and detects whether or not a moving object that performs the predetermined action is included in the first monitoring area based on information output by the moving object discrimination model; and,
a specifying unit that specifies a notification area where the presence of a potentially dangerous moving object should be notified when a moving object performing the predetermined action is detected in the first monitoring area;
an output unit that outputs notification information that notifies users in the specified notification area of the presence of a moving object that may pose a danger ;
When the detection unit detects a moving object that performs the predetermined action in the first monitoring area, identifying the traveling direction of the moving object that performs the predetermined action based on a photographed image of the first monitoring area. Department and
an area specifying unit that specifies a second monitoring area different from the first monitoring area in the specified direction of travel;
a transmitting unit that transmits the photographed image of the first monitoring area to a second photographing device that photographs the specified second monitoring area,
The second photographing device is
Based on the captured image of the moving object performing the predetermined action in the first monitoring area received from the first imaging device, the image of the moving object detected in the first monitoring area is included in the captured image of the second monitoring area. a determination unit that determines whether there is a moving object that is the same as the moving object that performs a predetermined action;
a specifying unit that specifies the second monitoring area as a notification area when the determining unit detects the same object in the second monitoring area;
and an output unit that outputs notification information that notifies users in the specified notification area of the presence of a moving object that may pose a danger.
Information processing system.

Images