JP7428839B2 - Information processing method - Google Patents

Description

The present disclosure relates to an information processing method and an information processing system.

Using wireless communication such as wireless LAN (Local Area Network) or mobile phone lines, an operator in a remote location can operate a vehicle without a driver or a vehicle that is not operated by a driver and is capable of autonomous driving. There is a remote control system that operates indirectly.

In such remote control systems, the vehicle (hereinafter referred to as the operated vehicle) communicates the sensing results obtained by observing the surroundings of the vehicle using various sensors installed on the vehicle, such as millimeter wave radar, laser radar, and cameras. Communicate to the operator via means. Based on the conveyed sensing results, the operator grasps the state of the operated vehicle and the state of the surroundings of the operated vehicle, determines how to control the operated vehicle, and determines the method of controlling the vehicle. Consider. Then, the operator remotely controls the operated vehicle by transmitting a control signal regarding the running of the vehicle to the operated vehicle.

For example, Patent Document 1 discloses a technique for remotely controlling a vehicle existing in a remote control target area, which is an area where a vehicle should be remotely controlled. As a result, the operator only has to remotely control the vehicle within a limited area as the remote control target area, and the time required to continuously remotely control the vehicle can be shortened, reducing the burden on the operator of remote control. Can be done.

Patent No. 2017-147626

In the conventional technology such as Patent Document 1, prior to the operation of the operated vehicle by the operator, the operated vehicle is monitored by the operator or a monitor who does not perform operations other than the operator (hereinafter also referred to collectively as a monitor). will be held. Supervisors do not know when or where a dangerous situation such as an accident will occur, so they are required to monitor all scenes as much as possible, which places a heavy burden on them. Therefore, it is desirable to reduce the monitoring burden on the supervisor. However, although Patent Document 1 discloses an invention for reducing the burden on an operator who remotely operates a vehicle, it does not disclose a method for reducing the monitoring burden on a supervisor who remotely monitors a vehicle.

Therefore, the present disclosure provides an information processing method that can reduce the monitoring burden on the monitor.

An information processing method according to an aspect of the present disclosure uses a computer to obtain first position information indicating a first position of a vehicle to be monitored, and second position information indicating a position where the vehicle needs to be monitored. obtaining second position information indicating a position, determining a monitoring priority for the vehicle according to movement information regarding movement by the vehicle from the first position to the second position, and determining a monitoring priority for the vehicle based on the monitoring priority. Presentation information for monitoring the vehicle is generated, and the presentation information is output to a presentation device.

Further, an information processing system according to an aspect of the present disclosure includes a first acquisition unit that uses a computer to acquire first position information indicating a first position that is a position of a vehicle to be monitored; a second acquisition unit that acquires second position information indicating a second position that requires a second position; and a second acquisition unit that determines a monitoring priority for the vehicle according to movement information regarding movement from the first position to the second position. and a presentation control section that generates presentation information for monitoring the vehicle based on the monitoring priority and causes a presentation device to output the presentation information.

Further, a program according to one aspect of the present disclosure is a program for causing a computer to execute the above information processing method.

An information processing method according to an aspect of the present disclosure uses a computer communicably connected to a vehicle to acquire first position information indicating a first position that is a position of the vehicle that is a monitoring target, and obtaining second position information indicating a second position that is a position that requires monitoring, and determining a monitoring priority for the vehicle according to movement information regarding movement by the vehicle from the first position to the second position. Then, the amount of communication with the vehicle is controlled based on the monitoring priority.

Further, an information processing method according to an aspect of the present disclosure uses a computer to acquire first position information indicating a first position that is a position of a vehicle to be monitored, and the first position information is a position where the vehicle needs to be monitored. obtaining second position information indicating a second position; obtaining work information regarding a supervisor's work according to the degree of difficulty of traveling at the second position; Determine a monitoring priority for the vehicle according to movement information regarding movement and the work information, generate presentation information for monitoring the vehicle based on the monitoring priority, and send the presentation information to a presentation device. Output to .

Further, an information processing method according to an aspect of the present disclosure uses a computer to acquire first position information indicating a first position that is a position of a vehicle to be monitored, and the first position information is a position where the vehicle needs to be monitored. acquiring second position information indicating a second position, and causing a presentation device to output presentation information for monitoring the vehicle based on the first position information and the second position information; At least first information indicating the reason why the vehicle is determined to be a vehicle to be monitored, second information regarding a method for monitoring the vehicle, and third information regarding unnecessary monitoring when there is no vehicle to be monitored. Contains one.

Further, an information processing method according to an aspect of the present disclosure includes using a computer to acquire first position information indicating a first position of each of a plurality of vehicles to be monitored by a remote monitor; Obtaining second position information indicating a second position that requires monitoring by the remote monitor for each of the plurality of vehicles, and responding to movement information regarding movement by the vehicle from the first position to the second position. Presentation information that determines a monitoring priority for each of the plurality of vehicles, and causes the remote monitor to monitor a vehicle having a relatively high monitoring priority among the plurality of vehicles based on the monitoring priority. is generated, and the presentation information is output to the presentation device.

Note that these comprehensive or specific aspects may be realized in a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, and the system, integrated circuit, computer program, and recording It may be implemented with any combination of media.

According to the information processing method of the present disclosure, it is possible to reduce the monitoring burden on the monitor.

FIG. 1 is a block diagram showing the configuration of a monitoring system according to an embodiment. FIG. 2 is an overview diagram of an example of a monitoring room according to the embodiment. FIG. 3 is a block diagram showing the functional configuration of the server device according to the embodiment. FIG. 4 is a diagram showing an example of a priority table stored in the operation DB according to the embodiment. FIG. 5 is a diagram showing an example of a work information table stored in the operation DB according to the embodiment. FIG. 6 is a block diagram showing the functional configuration of the recommendation unit according to the embodiment. FIG. 7 is an explanatory diagram showing a first example of presentation information according to the embodiment. FIG. 8 is an explanatory diagram showing a second example of presentation information according to the embodiment. FIG. 9 is an explanatory diagram showing a third example of presentation information according to the embodiment. FIG. 10 is an explanatory diagram showing a fourth example of presentation information according to the embodiment. FIG. 11 is an explanatory diagram showing a fifth example of presentation information according to the embodiment. FIG. 12 is an explanatory diagram showing a sixth example of presentation information according to the embodiment. FIG. 13 is an explanatory diagram showing a seventh example of presentation information according to the embodiment. FIG. 14 is a sequence diagram showing processing of the monitoring system according to the embodiment. FIG. 15 is a flow diagram illustrating processing for determining monitoring priority in the recommendation unit according to the embodiment. FIG. 16 is a flow diagram showing the processing of the monitoring control unit according to the embodiment. FIG. 17 is a sequence diagram showing processing of a monitoring system according to a modification of the embodiment.

An information processing method according to an aspect of the present disclosure uses a computer to obtain first position information indicating a first position of a vehicle to be monitored, and second position information indicating a position where the vehicle needs to be monitored. obtaining second position information indicating a position, determining a monitoring priority for the vehicle according to movement information regarding movement by the vehicle from the first position to the second position, and determining a monitoring priority for the vehicle based on the monitoring priority. Presentation information for monitoring the vehicle is generated, and the presentation information is output to a presentation device.

Thereby, the supervisor can determine whether or not it is necessary to monitor the vehicle by checking the presentation information output to the presentation device. In other words, the supervisor can determine whether or not monitoring is necessary without constantly monitoring the vehicle or analyzing the vehicle situation in detail. Furthermore, the driving environment of the vehicle varies depending on the location, which affects safe driving of the vehicle. Therefore, by presenting the presentation information based on the monitoring priority according to the relationship between the position requiring monitoring and the position of the vehicle, the observer can use the presentation information as a reference for monitoring. Therefore, according to the information processing method according to one aspect of the present disclosure, the monitoring burden on the monitor can be reduced. Specifically, the burden of monitoring all images and the burden of determining the necessity of monitoring a vehicle can be reduced.

Further, the movement information may include at least one of arrival time, distance, and travel route from the first position to the second position.

Thereby, the monitoring priority can be determined using at least one of arrival time, distance, and travel route as movement information. Therefore, the monitoring priority can be determined according to the amount of time until the vehicle reaches the position where monitoring is required. Therefore, by checking the presentation information based on the monitoring priority, the supervisor can identify the vehicle that should be monitored preferentially, and the monitoring burden on the supervisor can be reduced. Note that the arrival time means the time required for the vehicle to move from the first position to the second position.

Further, the movement information may include the arrival time, and the monitoring priority may be determined according to the arrival time.

Thereby, by checking the presentation information corresponding to the arrival time, the monitor can easily determine whether or not monitoring is required as a result. In other words, the monitor can specify vehicles that should be monitored with priority from a time perspective. Therefore, the monitoring efficiency of the supervisor can be improved, and the monitoring burden on the supervisor can be further reduced.

Furthermore, the degree of difficulty of traveling at the second position may be acquired, and the monitoring priority may be determined according to the degree of difficulty and the arrival time.

As a result, the observer can more accurately identify the vehicle that should be monitored with priority according to the situation at the second position by checking the presented information that takes into consideration the degree of difficulty of driving at the second position. Can be done. Therefore, the monitoring burden on the supervisor can be further reduced.

Further, the second location may include a point specified from at least one of geographic information and traffic information.

This makes it possible to determine monitoring priorities that take geographic information or traffic information into consideration. The degree to which monitoring is required is influenced by static geographical conditions or dynamic traffic conditions. Therefore, by presenting the presentation information based on the monitoring priority that takes geographic information or traffic information into consideration, the monitor can use the presentation information as a reference for monitoring. For example, the geographic information includes road shapes (for example, intersections or curves, merging, gradients, accident-prone locations, etc.), and the traffic information includes dynamic road conditions (for example, traffic congestion information). In other words, as a result, the observer can more appropriately determine which vehicle should be monitored with priority, based on at least one of the static information at the second position and the dynamic information at the second position. can. Therefore, the monitoring burden on the supervisor can be further reduced.

Further, the second location may include a point where a person gets on or off the vehicle.

As a result, presentation information is generated that takes into consideration movement information to a point on the vehicle travel route where there is a risk of contact with a person (for example, a bus stop or a point where the user gets on or off the vehicle). By checking the presented information, the supervisor can determine which vehicles should be prioritized to be monitored, taking into consideration information on movement to a point where there is a risk of contact with people. Therefore, the monitoring burden on the supervisor can be further reduced.

Further, the second position may include a point where an object that satisfies a predetermined condition is located.

Thereby, it is possible to determine the monitoring priority according to the positional relationship with the object that satisfies the predetermined conditions. The degree to which monitoring is required is influenced not only by geographical or traffic conditions, but also by the nature of the object located. Therefore, by presenting the presentation information that takes into account the position of the object that satisfies the predetermined condition, the observer can use the presentation information as a reference for monitoring. Therefore, the monitoring burden on the supervisor can be further reduced.

Further, the predetermined condition may include that the object has a possibility of interfering with the traveling of the vehicle, and the object may be identified from sensing data of the vehicle or a vehicle around the vehicle.

Thereby, it is possible to present presentation information that takes into consideration the position of objects that may impede the running of the vehicle. Vehicle monitoring is likely to be required in areas around objects that may impede the vehicle's movement. Therefore, by presenting the presentation information, the monitor can use the presentation information as a reference for monitoring. For example, objects that may impede the running of a vehicle include vehicles that are allowed to run before other vehicles (e.g., emergency vehicles), vehicles that are difficult to move (e.g., vehicles that have been involved in an accident), or objects that violate driving regulations. There are vehicles that may interfere with vehicle sensing (for example, large vehicles), and vehicles that are of a special type (for example, special work vehicles).

Furthermore, work information regarding the work of the supervisor according to the degree of difficulty of traveling at the second position may be acquired, and the monitoring priority may be determined according to the movement information and the work information.

Thereby, the supervisor can determine whether or not the vehicle needs to be monitored as a result by checking the presented information in which the work information is further taken into consideration. In other words, the supervisor can specify the vehicle to be monitored with priority in addition to the viewpoint of work for monitoring the vehicle. Therefore, the monitoring efficiency of the supervisor can be improved, and the monitoring burden on the supervisor can be further reduced.

Further, the work information includes at least one of a work time required for monitoring the vehicle and a work time required for operating the vehicle, and the movement information includes the arrival time, and the movement information includes the arrival time and the work time. The monitoring priority may be determined accordingly.

As a result, the supervisor can determine whether or not it is necessary to monitor the vehicle by checking the presented information that takes the arrival time and work time into consideration. In other words, the monitor can specify the vehicle to be monitored with priority in terms of the total time from the vehicle's current location to the second location until the monitoring or operation is completed. For example, it becomes possible to perform monitoring while ensuring the time required for monitoring or operation to reach the second position. Therefore, the monitoring burden on the supervisor can be further reduced.

Furthermore, the degree of difficulty of traveling at the second position may be acquired, and the monitoring priority may be determined according to the difference between the arrival time and the working time and the degree of difficulty.

As a result, the supervisor can check the presented information, which takes into account the difference between the arrival time and the work time, and the degree of difficulty of traveling at the second position, to determine whether it is necessary to monitor the vehicle as a result. You can judge whether or not. Therefore, the monitoring burden on the supervisor can be further reduced.

Further, the first position information is acquired from each of the plurality of vehicles, the second position information indicating a second position that is a position that requires monitoring for each of the plurality of vehicles, and the second position information is acquired from each of the plurality of vehicles. A monitoring priority is determined using the first location information and the second location information for each of the plurality of vehicles, and the presentation information is the monitoring priority of the vehicle specified according to the height of the monitoring priority for each of the plurality of vehicles. It may also include information for

Thereby, the observer can know which vehicle requires monitoring among the plurality of vehicles by checking the presentation information output to the presentation device. In other words, the monitor can know which vehicles require monitoring without constantly monitoring all of the multiple vehicles and without having to analyze in detail all the situations of the multiple vehicles. Therefore, when there are a plurality of vehicles to be monitored, the effect of reducing the monitoring burden on the observer can be further enjoyed. Additionally, by allowing one person to monitor multiple vehicles, it becomes possible to monitor multiple vehicles with fewer people than the number of vehicles, reducing costs such as personnel and equipment costs associated with monitoring. .

Further, the presentation information may include information indicating the identified vehicle as a monitoring target.

Thereby, the presentation device presents information about a vehicle that requires monitoring by a supervisor among the plurality of vehicles. That is, vehicles that require monitoring are directly indicated. Therefore, the supervisor can easily identify the vehicle that requires monitoring, thereby further reducing the monitoring burden on the supervisor.

In addition, the presentation information may include information for monitoring a vehicle that is identified according to a high monitoring priority among the plurality of vehicles, rather than information for monitoring other vehicles other than the identified vehicle. It may also include information that is presented in an emphasized manner.

Thereby, even if information about a plurality of vehicles is presented on the presentation device, the observer can easily know the information about the vehicle of interest among them. For example, if images of a plurality of vehicles are each displayed in a size that corresponds to the monitoring priority, it is possible to visually know which vehicle is of interest based on the size of the image. Therefore, even if information about a plurality of vehicles is presented on the presentation device, the monitoring burden on the observer can be reduced. Note that a noteworthy vehicle means a vehicle that has a high monitoring priority.

In the acquisition of the first location information, the first location information may be acquired through communication with the vehicle, and the information processing method may further include controlling the amount of communication according to the monitoring priority. good.

Thereby, communication with the vehicle can be optimized according to the monitoring priority. For example, it is possible to reduce the amount of communication, the number of times of communication, the frequency of communication, the communication time, etc. with a vehicle with a low monitoring priority. Therefore, it is possible to reduce the monitoring burden on the supervisor while suppressing the use of the network band with the vehicle according to the monitoring priority.

Furthermore, the presentation information may be output to the presentation device before the vehicle reaches the second position.

Thereby, the monitor can grasp the vehicles that will require monitoring before the timing that requires monitoring arrives.

Further, an information processing system according to an aspect of the present disclosure includes a first acquisition unit that uses a computer to acquire first position information indicating a first position that is a position of a vehicle to be monitored; a second acquisition unit that acquires second position information indicating a second position that requires a second position; and a second acquisition unit that determines a monitoring priority for the vehicle according to movement information regarding movement from the first position to the second position. and a presentation control section that generates presentation information for monitoring the vehicle based on the monitoring priority and causes a presentation device to output the presentation information.

This produces the same effects as the information processing method described above.

Note that these comprehensive or specific aspects may be realized by a system, a device, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, and the system, device, integrated circuit, computer program Alternatively, it may be realized using any combination of recording media.

Hereinafter, embodiments will be specifically described with reference to the drawings.

Note that the embodiments described below are all inclusive or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, order of steps, etc. shown in the following embodiments are examples, and do not limit the present disclosure. Further, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the most significant concept will be described as arbitrary constituent elements.

(Embodiment)
The information processing method and the like of this embodiment will be explained with reference to FIGS. 1 to 16.

A remote monitoring system (hereinafter also simply referred to as a monitoring system) acquires sensing results from a monitored vehicle via a communication means and transmits them to a supervisor. Based on the transmitted sensing results, the supervisor grasps the state of the monitored vehicle and the state of the surroundings of the monitored vehicle, and determines whether intervention by an operator is necessary for the monitored vehicle. Note that the supervisor may consider appropriate intervention methods including indirect control by the operator. If the supervisor determines that it is difficult for the monitored vehicle to travel autonomously, the supervisor requests the operator to remotely control the vehicle. An operator remotely controls a monitored vehicle by transmitting a control signal regarding the running of the vehicle to the monitored vehicle (that is, the operated vehicle). Note that if the operator is a monitor, the operator may directly shift from monitoring to operation.

[Monitoring system configuration]
First, a monitoring system including a server device that executes the information processing method according to the present embodiment will be described with reference to FIGS. 1 to 13.

FIG. 1 is a block diagram showing the configuration of a monitoring system 1 according to this embodiment. FIG. 2 is an overview diagram of an example of a monitoring room according to the present embodiment.

A monitoring system 1 shown in FIG. 1 is an information processing system for monitoring vehicles M1, M2, . . . , MN that can be operated vehicles. Hereinafter, the vehicles M1, M2, ..., MN are also referred to as "vehicles M1, etc.". The monitoring room shown in FIG. 2 is a room for a monitoring person to monitor the vehicle M1 and the like. In the monitoring room, information obtained from the vehicle M1 etc. and information for monitoring the vehicle M1 etc. are presented to the monitor, and the monitor can perform operations for monitoring. In addition to the operation for monitoring by the supervisor, the operation for remote control of the operated vehicle may be performed. Note that the supervisor may be an operator who operates the operated vehicle, or may be a person different from the operator. Furthermore, although an example will be described in which the monitoring system 1 is an information processing system that monitors a plurality of vehicles M1, etc., the number of vehicles that the monitoring system 1 monitors is not particularly limited. For example, the monitoring system 1 may be an information processing system that monitors one vehicle.

As shown in FIG. 1, the monitoring system 1 includes a server device 10, an input device 20, and a display device 30.

The server device 10 is a processing device that makes decisions regarding monitoring by a supervisor. Specifically, the server device 10 makes decisions regarding monitoring through information processing by a computer using the content input to the input device 20 and vehicle information obtained from the vehicle M1 and the like. In the present disclosure, the server device 10 uses the location information of the vehicle M1, etc. (an example of first location information) included in the vehicle information acquired from the vehicle M1, etc. to indicate the monitoring priority level for each of the vehicles M1, etc. The present invention is characterized in that presentation information for monitoring the vehicle M1 etc. is generated and output based on the monitoring priority. Details will be described later.

The server device 10 is installed at a location where it can be communicably connected to the network N, and its physical location may be anywhere. As an example, if the server device 10 is placed inside or near the monitoring room, there is an advantage that communication with the input device 20 and the display device 30 can be performed in a short time.

The input device 20 is an input device into which an operation by a supervisor is input. The input device 20 may be a keyboard, a mouse, a touch panel, or the like connected to the server device 10, or may be a button that can be operated by pressing or touching. The input device 20 provides the server device 10 with information regarding the input operation. The input device 20 is installed in the monitoring room. When the server device 10 is located at a remote location, the input device 20 and the server device 10 may be connected via a network.

The display device 30 is a device that acquires presentation information from the server device 10 and outputs the presentation information as an image. Images include photographs, illustrations, characters, etc. The display device 30 is a liquid crystal display or the like. The image output by the display device 30 is visually recognized by a supervisor, and is used when the supervisor decides whether or not to monitor a vehicle, which vehicle to monitor, or how to monitor a vehicle. used for. Note that the display device 30 is an example of a presentation device. The monitoring system 1 may include a device that outputs sound as a presentation device together with the display device 30 or in place of the display device 30. The display device 30 is installed in the monitoring room. When the server device 10 is located at a remote location, the display device 30 and the server device 10 may be connected via a network.

The vehicle M1 is a vehicle with or without a person on board, and is, for example, an automatic driving vehicle. Vehicle M1 is a vehicle that can be monitored by a supervisor. The vehicle M1 exists in an external space different from the monitoring room, such as a road or a parking lot.

Vehicle M1 includes a communication interface, is connected to network N through communication, and is communicably connected to server device 10 via network N. The vehicle M1 includes one or more cameras that photograph the surroundings of the vehicle M1, as well as the position, speed, acceleration, jerk, steering angle, remaining amount of fuel or battery, operating state of the direction indicator, Detects the operating status of ABS (Anti-lock Braking System) or AEB (Automatic Emergency Braking), the position of surrounding objects, the existence, number and type of surrounding objects, distance from surrounding objects, speed difference, acceleration difference, etc. Equipped with multiple sensors. The vehicle M1 transmits information including image data captured by one or more cameras and sensing data such as the position of the vehicle M1 and the positions of surrounding objects acquired by a plurality of sensors via a communication interface and a network N. It is transmitted to the server device 10. Note that the above information transmitted by the vehicle M1 is also referred to as vehicle information. The above-mentioned various types of information transmitted as vehicle information are also referred to as types of vehicle information. That is, there may be multiple types of vehicle information itself. Note that acceleration/deceleration control, steering angle control, etc. of the vehicle M1 can be controlled by a supervisor via the network N as necessary. Such control by a supervisor is also called remote control.

The vehicle information further includes error information such as life and death information indicating whether or not a plurality of sensors are operating or information regarding a system error of the vehicle M1 itself, and call information for calling a supervisor from the passenger of the vehicle M1. May be included. In addition, if the vehicle M1 stores map information, the vehicle information may include the legal speed at the location where the vehicle M1 is traveling, information indicating the positional relationship of the vehicle M1 with respect to the lane, or information set by the vehicle M1. Information regarding the travel route may also be included.

Further, among the vehicle information, information such as the position of the vehicle M1, error information, call information, legal speed, information indicating the positional relationship with respect to the lane, travel route, etc. are examples of information indicating the driving state of the vehicle M1. Information such as the presence or absence of surrounding objects is an example of information indicating the external state of the vehicle M1. Note that the vehicle information only needs to include at least information indicating the position of the vehicle M1. Note that the position of the vehicle M1 is an example of a first position, and the information indicating the first position is an example of first position information. The first position means, for example, the current position of the vehicle M1.

Furthermore, the vehicle M1 may further include a sound collection device (not shown) as a sensor that collects sounds around the vehicle M1. The sound collection device includes, for example, a microphone. From the sounds around the vehicle M1 (for example, the sound of a siren) picked up by the microphone, it is possible to detect emergency vehicles (for example, ambulances, fire engines, or police cars) that are running or stopped around the vehicle M1. etc.) may be detected. It should be noted that the emergency vehicle is an example of an object that satisfies a predetermined condition, and the predetermined condition includes that the object has the possibility of interfering with the traveling of the vehicle M1, and here, the emergency vehicle is given priority over general vehicles when traveling. That's true. Furthermore, the emergency vehicle is not limited to being detected by a sound pickup device, and may be detected by, for example, image analysis of images captured by one or more cameras.

Furthermore, a vehicle that is difficult to drive (for example, an accident vehicle or a vehicle that has stopped due to trouble) may be identified from images captured by one or more cameras or sensing data acquired by a plurality of sensors. Note that a vehicle that is difficult to travel is an example of an object that satisfies a predetermined condition, and the predetermined condition includes that the object has the possibility of interfering with the travel of the vehicle M1. It is difficult.

The identification of the object that satisfies the above predetermined conditions may be performed by the vehicle M1 or the like. In this case, the vehicle information may include information such as the type, position, and speed of the object that satisfies the predetermined conditions. Note that the point where the object that satisfies the predetermined condition is located is an example of a second position that is a position that requires monitoring of the vehicle M1 etc. by a supervisor. Further, detection of an object that satisfies the above predetermined conditions may be performed by the server device 10. In this case, the vehicle information may include audio information or the like in the sensing data. Further, the predetermined condition may be other than the above as long as it has a possibility of affecting the running of the vehicle M1. For example, the predetermined condition may be that the vehicle travels at a predetermined speed or higher. In this case, the object that satisfies the predetermined condition may be a dangerous vehicle that travels dangerously.

As described above, the second position may be a position specified from sensing data. Although details will be described later, the second location may be, for example, a point specified from at least one of geographic information and traffic information. The second location may include, for example, a railroad crossing, crossing, merging, diversion, curve, or apex point of a slope. Further, the second location may include a location where the vehicle stops erroneously, a location where a person gets on or off the vehicle, and a location at a station. A station is a facility where vehicles are refueled or batteries are charged. Note that the second position is a point on the travel route of the vehicle.

Vehicles M2 to MN are each similar to vehicle M1, and move independently of vehicle M1. Furthermore, vehicles M2 to MN each transmit image data and sensing data generated by their own devices to the server device 10 in the same way as vehicle M1.

The network N is a communication network to which the server device 10, the vehicle M1, etc. are connected. The communication standard or communication method of the network N is, for example, a mobile phone network, a satellite communication network, a wide area communication network using Wi-Fi, etc., but is not limited to these. The part connected to vehicle M1 etc. is wireless.

An input device 20 and a display device 30 are arranged in the monitoring room. The input device 20 and the display device 30 are arranged at positions where input and visual confirmation by the observer are easy. For example, the input device 20 and the display device 30 are placed on a desk in front of a chair where a supervisor is seated. Further, a display device 31 that displays images such as image data acquired by the vehicle M1 and the like and a map showing the position of the vehicle M1 is arranged at a position that can be viewed from the entire monitoring room. For example, the display device 31 displays images based on image data acquired from all vehicles M1, etc. of vehicles M1, M2, . . . , MN that can be operated vehicles. The display device 31 is connected to, for example, the server device 10 and displays images of all vehicles transmitted from the server device 10. The displayed image is visually recognized by a supervisor.

Although there may be a plurality of monitors, the number of monitors may be smaller than the number of vehicles M1 and the like. It is assumed that the monitor does not always monitor all vehicles such as vehicle M1, but only selects and monitors the vehicles that are judged to require monitoring, and does not monitor other vehicles. . This has the advantage of reducing the number of personnel required for monitoring and allowing the supervisor to concentrate on monitoring the vehicles that require monitoring. Monitoring is deemed necessary, for example, when the vehicle is in a dangerous position, in a dangerous condition, approaching a dangerous position, or when the vehicle is in a dangerous condition. For example, when it is predicted that Note that the number of monitors may be one, or the number of monitors may be the same as the number of vehicles M1 and the like.

The monitor visually recognizes the image presented by the display device 30 and decides which vehicle among the vehicles M1 etc. to monitor, how to monitor the vehicle, and the like. Then, the supervisor inputs the vehicle to be monitored or the monitoring method to the server device 10 via the input device 20. For example, the display device 30 displays an image of a vehicle to be monitored by a supervisor among vehicles M1, M2, ..., MN that can be operated vehicles, or an image for selecting a vehicle to monitor. Ru.

Hereinafter, a process in which the server device 10 generates information for recommending to the monitor which vehicle should be monitored will be described in detail. In this case, the observer determines which vehicle to monitor based on the information presented by the display device 30. If the monitoring target recommended by the server device 10 is appropriate, the burden on the supervisor of searching for the monitoring target among the vehicles M1 etc. can be reduced.

FIG. 3 is a block diagram showing the functional configuration of server device 10 according to this embodiment.

As shown in FIG. 3, the server device 10 includes a communication section 11, a vehicle management section 12, an operation DB (database) 13, an operation management section 14, a recommendation section 15, a reception section 16, and a monitoring and control section 14. 17 and a video management section 18.

The communication unit 11 is a communication interface that is connected to the network N and is communicably connected to the vehicle M1 and the like via the network N. The communication unit 11 acquires vehicle information from the vehicle M1 and the like.

The vehicle management unit 12 is a processing unit that manages the vehicle M1 and the like. The vehicle management unit 12 acquires the vehicle information transmitted by the vehicle M1, etc., and manages the position, speed, running state, etc. of each of the vehicles M1, etc. based on the acquired vehicle information.

The operation DB 13 is a database that stores information for operation of the vehicle M1 and the like. The operation DB 13 holds information on maps, traffic, and operation plans as information for the operation of the vehicle M1 and the like. The operation plan includes information indicating a travel route on which the vehicle M1 etc. is scheduled to travel, and information indicating the planned position of the vehicle M1 etc. on the route at each point in time during travel.

The map information includes geographic information such as topography, information about road layout, and position information of road ancillary elements (for example, positions specified by route and latitude). Road ancillary elements include marking lines displayed on the road that divide the width of the road (roadway center line, lane boundary line, outside roadway line, etc.) and structures near the road (traffic lights, road signs, guardrails, etc.) ) etc.

The geographical information may include statistical risk information on driving (for example, historical information on accidents or near-misses) accumulated along with position information on a map. Further, the geographic information may include risk information (eg, intersections, curves, etc.) based on the map shape (eg, road shape). The geographical information here includes so-called static information. Moreover, the geographical information may further include a point where a person gets on or off the vehicle. The geographical information may include information such as the location of a bus stop or the like or the destination of the own vehicle. A point on the map where there is a risk based on driving or the shape of the map, or a point where a person gets on or off the vehicle is an example of the second location. Furthermore, the geographic information includes information regarding the risk value at the second location. The geographic information may include, for example, a priority indicating the degree of priority of monitoring at the second location.

The traffic information includes traffic congestion information on a map, event information, and the like. The event information may include information regarding events such as festivals or sports days, information regarding time such as commuting and returning times for nearby schools, and information regarding driving such as traffic regulations. The traffic information here includes dynamic information that can change from moment to moment or in real time.

The server device 10 may acquire the above geographical information and traffic information from a device external to the server device 10 and store the acquired information in the operation DB 13.

In addition, the operation DB 13 stores a table (priority table) in which a plurality of pieces of situation information about the vehicle M1, etc. based on the vehicle information are associated with the degree to which a supervisor needs to monitor the vehicle M1, etc. based on the plurality of situation information. I remember. The operation DB 13 also stores a table (work information table) in which a plurality of pieces of situation information are associated with the time required for monitoring the vehicle M1, etc. (also referred to as work time) in the plural pieces of situation information.

Here, various tables stored in the operation DB 13 will be explained with reference to FIGS. 4 and 5. Note that the operation DB 13 only needs to store at least the work information table T2 among the priority table T1 and the work information table T2 described below.

FIG. 4 is a diagram showing an example of the priority table T1 stored in the operation DB 13 according to the present embodiment.

As shown in FIG. 4, the priority table T1 is a table in which priorities are associated with vehicle status information. A plurality of pieces of vehicle situation information are set in the priority table T1, and a priority is set for each piece of vehicle situation information.

Here, the priority is a numerical value that corresponds to the degree to which the vehicle requires monitoring by a supervisor, and for example, the higher the numerical value, the higher the degree to which the supervisor requires monitoring. In other words, the priority is a numerical value that corresponds to the degree of hindrance to automatic driving, and indicates the magnitude of the risk caused by automatic driving or continuing automatic driving. The priority is a numerical value based on vehicle information.

The vehicle status information may be information indicating the vehicle status based on vehicle information. For example, "an accident has occurred in one's own vehicle" is information indicating the situation of the vehicle, which is determined based on the driving state, image, acceleration, and the like. For example, if the own vehicle is in contact with an object around the own vehicle in the image, an "occurrence of an accident involving the own vehicle" is detected. Further, for example, "sudden start" is information indicating the situation of the vehicle, which is determined based on an image, acceleration, and the like. For example, a "sudden start" is detected when the acceleration within a predetermined period when the vehicle starts moving from a stopped state is equal to or greater than a predetermined value.

In addition to the vehicle information, the vehicle status information may also be information based on an operation plan that includes the vehicle's travel route. For example, "time schedule deviation" is information indicating the status of the vehicle, which is determined based on the vehicle position information included in the vehicle information, the operation plan, and the like. For example, if the current position of the vehicle is a predetermined distance or more from the current scheduled position of the vehicle included in the operation plan, a "time schedule deviation" is detected. Further, for example, "railroad crossing" is information indicating the situation of the vehicle, which is determined based on vehicle position information and geographic information included in the vehicle information. For example, a "level crossing" is detected when the current position of the vehicle is within a predetermined range from the level crossing. Note that the detection of the situation information is performed by the recommendation unit 15.

The five situation information (hereinafter also referred to as danger information) from "an accident occurred in own vehicle" to "time schedule deviation" shown in Figure 4 is information to indicate that the vehicle is in a dangerous situation. Yes, the two situation information (hereinafter also referred to as caution information) from "level crossing" to "traffic jam caused by own vehicle" are information to indicate that the vehicle may be in a dangerous situation from now on. It is. Caution information does not indicate that there is a problem with the vehicle's operation, but rather indicates that it is possible to drive the vehicle automatically.

Note that the priorities shown in FIG. 4 may be values weighted with weights (coefficients) depending on the situation information. For example, "an accident has occurred in one's own vehicle" is a situation that requires immediate monitoring by a supervisor, and has a high weight value, so a high priority is set. On the other hand, "time schedule deviation" is a situation that requires less monitoring than "an accident occurs in one's own vehicle" and has a lower weight value, so it is set to have a lower priority. Specifically, the priority is set based on the degree to which monitoring by a supervisor is required and the weight corresponding to each of the plurality of situation information corresponding to each of the plurality of situation information indicating the situation of the vehicle based on the vehicle information. may be done. The weight may be determined, for example, depending on the severity of an accident that may occur or the incidence rate of an accident for each piece of situation information. Incidentally, the accident occurrence rate is the probability that if the vehicle situation (for example, "occurrence of traffic congestion caused by own vehicle") is left unchecked, an event (for example, an accident) that must be dealt with by the supervisor will occur.

FIG. 5 is a diagram showing an example of the work information table T2 stored in the operation DB 13 according to the present embodiment.

As shown in FIG. 5, the work information table T2 is a table in which work time and vehicle status information are associated with each other. A plurality of pieces of vehicle situation information are set in the work information table T2, and a work time is set for each piece of vehicle situation information. The type of vehicle situation information set in the work information table T2 indicates a situation that can be obtained from, for example, geographic information or traffic information, and is at least the same as the vehicle situation information set in the priority table T1. Some parts may be the same. The position corresponding to the situation indicated by the situation information shown in FIG. 5 on the travel route of the vehicle M1 etc. is the second position.

The work time is an example of work information regarding the work of the supervisor according to the degree of difficulty of driving for each situation information, and includes at least one of the time required for the supervisor to judge the situation and the time required for the supervisor to respond. . The time it takes for the observer to judge the situation is, for example, the time it takes for the observer to judge the situation around the vehicle after checking the image taken by the vehicle. For example, it is the time required to understand the movements of the person at the crosswalk in front of the vehicle. Further, the time required for the supervisor to respond is, for example, the time required to bring the vehicle to an emergency stop or to stop the vehicle on the shoulder of the road. For example, stopping a vehicle on the roadside is more complex and time-consuming than stopping the vehicle in an emergency. Although FIG. 5 shows an example in which the longer the working time is, the higher the degree of difficulty in traveling is, but the present invention is not limited to this.

Note that the work information table T2 may be set for each of a plurality of supervisors. This makes it possible to determine the monitoring priority according to the work time of each supervisor. Although the work information table T2 has been described as an example including work time, it may also include work amount or work difficulty level. That is, the work information table T2 may be any table in which at least one of work time, work amount, and work difficulty is associated with situation information.

Referring again to FIG. 3, the operation management unit 14 is a processing unit that manages the operation of the vehicle M1 and the like. The operation management unit 14 acquires information such as the operation plan of the vehicle M1 etc. from the operation DB 13, and transmits instruction information including the travel route of the vehicle M1 etc. to the vehicle M1 etc. The operation management unit 14 also determines operation information including how much the operation of the vehicle M1 etc. is delayed with respect to the operation plan. The operation management unit 14 also acquires at least one of geographic information and traffic information from the operation DB 13 and presents the acquired geographical information and traffic information to the recommendation unit 15.

The recommendation unit 15 is a processing unit that generates information for recommending a vehicle to be monitored by the supervisor to the supervisor. The recommendation unit 15 acquires vehicle information transmitted by the vehicle M1 etc. from the vehicle management unit 12, and acquires operation information regarding the operation of the vehicle M1 etc. and at least one of geographical information and traffic information from the operation management unit 14. . Then, the recommendation unit 15 generates recommendation information for recommending a vehicle to be monitored by the supervisor based on the acquired vehicle information, operation information, and at least one of geographic information and traffic information. generate. The recommendation information includes information that identifies a vehicle that is a recommended target for monitoring. The recommendation information is indicated by, for example, a monitoring priority indicating the degree to which the vehicle should be prioritized in monitoring compared to other vehicles. Further, the recommendation information may be a monitoring recommendation degree that indicates the degree to which each vehicle is recommended to be monitored. Note that the recommendation unit 15 may determine the monitoring priority based on at least vehicle information.

Further, the recommendation unit 15 generates information indicating that monitoring is not required when there is no vehicle to be monitored by the supervisor (for example, when there is no vehicle whose monitoring priority is equal to or higher than a predetermined threshold). You can. Information indicating that monitoring is unnecessary is included in the recommendation information. Note that the recommendation unit 15 is an example of a determination unit.

Here, details of the recommendation section 15 will be explained with further reference to FIG. 6.

FIG. 6 is a block diagram showing the functional configuration of the recommendation unit 15 according to this embodiment.

As shown in FIG. 6, the recommendation section 15 includes a driving risk calculation section 15a, a route risk calculation section 15b, an integrated risk calculation section 15c, and a recommendation target determination section 15d.

The driving risk calculation unit 15a is a processing unit that calculates a driving risk value of the vehicle M1 etc. from at least one of vehicle information and operation information. The driving risk calculation unit 15a calculates a driving risk value based on at least one of the vehicle information acquired from the vehicle management unit 12 and the operation information acquired from the operation management unit 14. The driving risk value indicates the degree to which a supervisor needs to monitor the driving of the vehicle M1, and is calculated as a priority in this embodiment.

The driving risk calculation unit 15a calculates the driving risk value of the vehicle M1 etc. based on the driving situation of the vehicle M1 etc. based on the above information and the priority shown in FIG. 4.

The route risk calculation unit 15b is a processing unit that calculates a route risk value, which is a risk value on the route of the vehicle M1, based on position-based information and at least one of geographic information and traffic information. The route risk calculation unit 15b calculates the route risk value of the vehicle M1 etc. from the location information of the vehicle M1 etc. (an example of first location information) and at least one of geographical information and traffic information (an example of second location information). do. The route risk calculation unit 15b calculates a route according to a first position indicated by the first position information acquired from the operation management unit 14 and a second position that is a point specified from at least one of geographic information and traffic information. Calculate the risk value. The route risk calculation unit 15b calculates a route risk value according to movement information (an example of position-based information) regarding movement of the vehicle M1 etc. from the first position to the second position. The movement information is information obtained from the first position and the second position, and includes, for example, at least one of the arrival time, distance, and travel route from the first position to the second position. In the following, an example will be described in which the movement information includes the arrival time. The route risk value indicates the degree to which the travel route of the vehicle M1 requires monitoring by a supervisor, and is calculated based on the arrival time in this embodiment. Note that the arrival time in this specification means the time required for the vehicle M1 to move from the first position to the second position, and the distance on the traveling route from the first position to the second position and the time required for the vehicle M1 to move from the first position to the second position. It is calculated from the speed etc. Furthermore, if the second position is a point where a predetermined object (for example, satisfies a predetermined condition described later) is located, and the predetermined object is moving, the arrival time is calculated from the speed of the predetermined object, etc. be done.

The route risk calculation unit 15b calculates first position information of the vehicle M1, etc., second position information based on the travel route and at least one of geographic information and traffic information, and work information (for example, work time) shown in FIG. Based on this, the route risk value for the vehicle M1 etc. may be calculated. That is, the route risk calculation unit 15b may calculate the route risk value based on the work information in addition to the first position information and the second position information.

The integrated risk calculation unit 15c calculates an integrated risk value, which is the risk value of each vehicle M1, etc., based on the driving risk value calculated by the driving risk calculation unit 15a and the route risk value calculated by the route risk calculation unit 15b. This is the processing section. The integrated risk calculation unit 15c calculates one risk value for each vehicle M1 etc. by integrating the driving risk value and the route risk value by a predetermined calculation. Note that the operation includes at least one of addition, subtraction, multiplication, and division. The integrated risk calculation unit 15c calculates the integrated risk value for the vehicle M1 etc., for example, by adding the driving risk value and the route risk value. In the present embodiment, the integrated risk calculation unit 15c determines the monitoring priority obtained by adding the driving priority and the route priority as the integrated risk value. Further, when integrating the driving risk value and the route risk value by a predetermined calculation, the integrated risk calculation unit 15c may multiply at least one of the driving risk value and the route risk value by a predetermined weight (coefficient). good.

The recommendation target determining unit 15d is a processing unit that determines a vehicle to be recommended as a monitoring target from the integrated risk value of each vehicle M1 etc. obtained from the integrated risk calculating unit 15c. The recommendation target determining unit 15d may determine that a vehicle having the highest monitoring priority or higher than a threshold (an example of a vehicle of interest) among the vehicles M1 etc. is the vehicle to be recommended as a monitoring target.

The number of vehicles recommended by the recommendation target determining unit 15d may be one, two or more, and may be determined depending on the monitoring ability of the observer, for example. Further, if there is no vehicle with a monitoring priority higher than a threshold value, the recommendation target determining unit 15d may determine, for example, that there is no vehicle to recommend. In this case, the recommendation information includes information indicating that there is no vehicle to recommend.

Note that the recommendation unit 15 only needs to have a component for determining a recommendation target based on the route risk value. In other words, the recommendation unit 15 only needs to include the route risk calculation unit 15b and the recommendation target determination unit 15d. Further, the monitoring priority may be determined according to at least the route risk value. That is, the monitoring priority may be determined according to movement information (for example, arrival time) regarding the movement of the vehicle from the first position to the second position.

Referring again to FIG. 3, the receiving unit 16 is a processing unit that receives operation information input into the input device 20 and indicating an input operation by the supervisor. The operation information includes, for example, an operation for switching a monitoring target.

The monitoring control unit 17 is a processing unit that determines a vehicle to be monitored. The monitoring control unit 17 determines a vehicle to be monitored based on the recommendation information generated by the recommendation unit 15 and the operation information received by the receiving unit 16, and sends monitoring information indicating the vehicle to be monitored. generate. The monitoring control unit 17 provides the generated monitoring information to the video management unit 18.

The monitoring information generated by the monitoring control unit 17 includes information that identifies a vehicle to be monitored. Further, the monitoring information may include information indicating the reason why the vehicle is determined to be a subject of monitoring. The monitoring information includes, for example, information indicating which situation information among the plurality of situation information shown in FIG. 5 has been detected. Further, the monitoring information may include, for example, information indicating which situation information among the plurality of situation information shown in FIG. 4 has been detected. The monitoring information may further include monitoring priority.

The monitoring control unit 17 has an automatic update mode and a manual update mode as operating modes related to determining a vehicle to be monitored. The automatic update mode is a mode in which a vehicle to be monitored is determined according to recommendation information and monitoring information is generated, regardless of whether or not operation information has been received or the content of the received operation information. The manual update mode is a mode in which a vehicle to be monitored is determined according to operation information and monitoring information is generated, regardless of the presence or absence of recommendation information or the content of the recommendation information.

The video management unit 18 is a processing unit that manages video to be displayed on the display device 30. The video management unit 18 generates presentation information including image data related to a video to be displayed on the display device 30 and outputs it to the display device 30. The video management unit 18 acquires monitoring information from the monitoring control unit 17, and when the monitoring information includes information specifying a vehicle to be monitored, the image management unit 18 acquires the image received by the communication unit 11 from the vehicle to be monitored. Include data in presentation information. Note that the video management unit 18 is an example of a presentation control unit that generates presentation information and causes the display device 30 to output the generated presentation information.

Specifically, the presentation information may include information indicating a vehicle that is a target of monitoring indicated in the monitoring information. Furthermore, when the monitoring information includes information indicating one or more vehicles that can be monitored, the presentation information may include the monitoring priority of each of the one or more vehicles included in the monitoring information. Moreover, when the monitoring information includes information indicating the direction of monitoring, the presentation information may include an image taken from the vehicle in the direction of monitoring included in the monitoring information. Further, the presentation information may include information indicating that monitoring is not required when there is no vehicle to be monitored by the observer.

Next, an example of presentation information generated by the video management unit 18, that is, displayed by the display device 30 will be described.

FIG. 7 is an explanatory diagram showing a first example of presentation information according to the present embodiment. FIG. 7 shows an example in which monitoring priority is presented as presentation information.

As shown in FIG. 7, the image 32 displayed by the display device 30 includes an image 34 indicating the monitoring priority of each vehicle M1 and the like. That is, the video management unit 18 causes the display device 30 to present the image 32 in which the vehicle M1 etc. are associated with the monitoring priority. The monitoring priority is information indicating the degree to which a vehicle should be monitored with priority. The monitoring priority is shown in three levels, for example, A (high), B (medium), and C (low). For example, it is shown that the monitoring priority of vehicle M1 is A, and the monitoring priority of vehicle M2 is B. The image 32 also includes an image 33 of a button for starting monitoring of each of the vehicles M1 and the like. Note that the monitoring priority may be a numerical value. The monitoring priority may be, for example, a value obtained by adding up priorities corresponding to detected situation information among a plurality of pieces of situation information. Note that the initial value of the monitoring priority may be a predetermined value (for example, 0). The initial value of the monitoring priority means that no situation requiring immediate or future monitoring has occurred.

Thereby, the monitor can easily determine which vehicle should be monitored by checking the monitoring priority of each of the plurality of vehicles. The monitor can know which vehicles require monitoring without having to analyze in detail all the situations of multiple vehicles.

Note that the presentation information is, for example, information for monitoring the vehicle M1, etc., which is specified according to the height of the monitoring priority of each of the plurality of vehicles M1, etc. The presentation information may include information for preferentially presenting a vehicle with a high monitoring priority. The presentation information may include information indicating each of the plurality of vehicles M1 etc. in order of monitoring priority. The presentation information includes information for monitoring a vehicle that is specified according to a high monitoring priority among the plurality of vehicles M1, etc., in order of the monitoring priority of each of the plurality of specified vehicles. Good too. In addition, the presentation information includes information indicating a vehicle that is specified according to a high monitoring priority among the plurality of vehicles M1, etc., and information indicating a correspondence between the monitoring priority corresponding to the specified vehicle. May contain. Furthermore, the presentation information may include information for displaying the vehicles in order of monitoring priority, starting from the highest or lowest monitoring priority. FIG. 7 shows an example in which the monitoring priorities are displayed in descending order of priority.

The presentation information also includes information for monitoring a vehicle (an example of first information) that is specified according to the high monitoring priority among the plurality of vehicles M1, etc., and information for monitoring other vehicles other than the specified vehicle. Information for monitoring (an example of second information) may be presented in a more emphasized manner. For example, the presentation information includes information in which information indicating a vehicle with the highest monitoring priority or a vehicle higher than a threshold value (an example of first information) is presented with emphasis over information on other vehicles (an example of second information). It's okay to stay. For example, the presentation information may include information that presents the button image 33 corresponding to the vehicle M1 with a high monitoring priority among the button images 33 in a different manner from the button image 33 corresponding to other vehicles. good.

FIG. 8 is an explanatory diagram showing a second example of presentation information according to the present embodiment. FIG. 8 shows an example in which the driving risk value and route risk value on which the monitoring priority is determined are presented as presentation information.

As shown in FIG. 8, the image 32a displayed by the display device 30 includes an image 34a showing the travel risk value and route risk value for each vehicle M1 and the like. In other words, the video management unit 18 causes the display device 30 to present an image 32a in which each vehicle M1 etc. is associated with a driving risk value and a route risk value. The driving risk value and the route risk value are shown in three levels, for example, A (high), B (medium), and C (low). For example, it is shown that the driving risk value and the route risk value of the vehicle M1 are A, the driving risk value of the vehicle M2 is A, and the route risk value is B. The image 32a also includes an image 33 of a button for starting monitoring of each of the vehicles M1 and the like. Note that the driving risk value and the route risk value may be numerical values. The driving risk value and the route risk value may be, for example, a value obtained by adding up priorities corresponding to detected situation information among a plurality of situation information.

FIG. 9 is an explanatory diagram showing a third example of presentation information according to the present embodiment. FIG. 9 shows an example in which an image based on image data acquired from vehicle M1 or the like is presented as presentation information.

As shown in FIG. 9, the image 32b displayed by the display device 30 includes images 35 to 37 for each vehicle M1 and the like. The image 35 is, for example, an image based on image data captured by one or more cameras of the vehicle M1 (an example of a vehicle of interest) with a high monitoring priority. Images 36 and 37 are similar, and are images based on image data taken by one or more cameras of a vehicle (an example of another vehicle) that has a lower monitoring priority than vehicle M1, such as vehicle M2. In this way, the presentation information may include information that displays the image 35 of the vehicle M1 with a high monitoring priority in a larger area than the images 36 and 37 of the vehicle M2 etc. with a lower monitoring priority than the vehicle M1. Note that the image 35 is an example of the first image included in the first information, and the images 36 and 37 are examples of the second images included in the second information. That is, the presentation information may include information for displaying the first image in a larger area than the second image. Also, presenting information indicating the vehicle of interest (for example, image 35) in a larger size than information indicating other vehicles (images 36 and 37) is included in emphatically presenting.

The image 32b may include an image of a button (see button image 33 shown in FIG. 7) for starting monitoring of each of the vehicles M1 and the like.

The monitor visually recognizes the images 32, 32a, or 32b, selects a vehicle to be monitored, and operates the button image 33 corresponding to the selected vehicle. This operation is accepted by the input device 20, and operation information indicating this operation is received by the receiving unit 16.

In FIGS. 7 to 9, an example has been described in which an image for the observer to select a vehicle to be monitored is displayed on the display device 30, that is, an example is not automatically switched to the image of the monitoring target. In the following, an example in which the presentation information is information for displaying an image of a vehicle to be monitored by a supervisor, that is, an example in which the image is automatically switched to the image to be monitored will be described.

FIG. 10 is an explanatory diagram showing a fourth example of presentation information according to the present embodiment. FIG. 10 shows an example in which an image based on image data acquired from vehicle M1 is presented as presentation information.

As shown in FIG. 10, the image 38 displayed by the display device 30 is an image of the vehicle M1 having a high (for example, highest) monitoring priority. In this way, the presentation information includes information for monitoring the vehicle M1 having the highest monitoring priority or higher than a threshold value among the plurality of vehicles M1, etc. The presentation information includes an image 38 (see FIG. 8) taken by the vehicle M1 having the highest monitoring priority or higher than a threshold value among the plurality of vehicles M1, etc., or an image in which the vehicle M1 is shown. In other words, the presentation information includes information indicating a vehicle (for example, vehicle M1) specified based on the monitoring priority as a monitoring target, and includes, for example, an image 38 of the image displayed on the display device 30 taken by vehicle M1. , or may include information for switching to an image in which the vehicle is shown. This allows the supervisor to omit the task of selecting a monitoring target, thereby further reducing the monitoring burden on the supervisor.

FIG. 11 is an explanatory diagram showing a fifth example of presentation information according to the present embodiment. FIG. 11 shows an example in which the reason why vehicle M1 was selected is presented in the image 38 shown in FIG. 10 as presentation information. That is, the presentation information includes information indicating the degree of contribution of at least one of the plurality of situation information to the monitoring priority.

As shown in FIG. 11, the image 38 displayed by the display device 30 includes an image 39 indicating the reason why the vehicle M1 was determined to be a monitoring target. The image 39 is included in the information indicating the degree of contribution. In FIG. 11, the vehicle M1 was determined to be the monitoring target because the "level crossing" had the highest degree of contribution. Note that work information (for example, work time) corresponding to "Railroad Crossing" may be presented superimposed on the image 38, or if situation information other than "Railroad Crossing" is detected, the detected situation information may be presented superimposed on the image 38. That is, two or more pieces of situation information corresponding to the vehicle M1 may be presented in a superimposed manner on the image 38. In this case, two or more pieces of situation information and their corresponding priorities may be presented. The two or more situation information and the priority are included in the information indicating the degree of contribution.

Note that if there is no vehicle with a monitoring priority equal to or higher than a predetermined threshold (for example, if there is no vehicle to be monitored), presentation information indicating that monitoring is not required may be presented. For example, information indicating that there is no monitoring target, information urging a break in monitoring work, information urging a work different from monitoring work, etc. may be presented. FIG. 12 is an explanatory diagram showing a sixth example of presentation information according to the present embodiment. Specifically, an image 38 including an image 40 indicating that there is no monitoring target is displayed as information indicating that monitoring is not required. An example is shown below.

Further, the time when there is no vehicle with a monitoring priority equal to or higher than a predetermined threshold may be recorded, that is, the time when monitoring is unnecessary, and statistical information about the time when monitoring is unnecessary may be presented. For example, statistical information may be presented, such as the mean, median, or mode of unmonitored time, or distribution over a particular period of time, such as a day, week, or month. Note that, for example, the process of recording the time when monitoring is not required is performed by the monitoring control unit 17, for example. Further, statistical information may be included in the monitoring information.

Further, a proposal for a period of time away from monitoring work, such as a break, may be presented from the statistical information on the time when monitoring is not required. For example, information that urges the user to take a break may be presented during a time period during which monitoring is statistically long during the day. Note that the process of proposing a period of time away from monitoring work, such as a break, is performed by the monitoring control unit 17, for example. Further, the suggestion (for example, information urging a person to take a break) may be included in the monitoring information.

FIG. 13 is an explanatory diagram showing a seventh example of presentation information according to the present embodiment. Specifically, the average value of the non-monitoring time and the information that prompts the user to take a break include the average value of the non-monitoring time and the information that prompts the user to take a break. An example is shown in which an image 38 including an image 41 indicating this is displayed.

[Monitoring system operation]
Next, processing by the monitoring system 1 configured as described above will be described below.

FIG. 14 is a sequence diagram showing the processing of the monitoring system 1 according to this embodiment. The sequence diagram shown in FIG. 14 is a diagram showing the flow of processing for determining recommended monitoring targets using position information. Note that in FIG. 14, vehicle M1 is illustrated as a representative of vehicle M1, etc., but similar processing is performed for other vehicles M2 to MN.

As shown in FIG. 14, in step S101, the operation management unit 14 of the server device 10 sets a travel route for the vehicle M1 based on the operation plan for the vehicle M1 stored in the operation DB 13. Then, the operation management unit 14 transmits instruction information including the set travel route to the vehicle M1. Note that the operation management unit 14 similarly sets travel routes for the other vehicles M2 to MN and transmits instruction information.

In step S121, vehicle M1 receives the instruction information transmitted by operation management section 14 in step S101. Then, the vehicle M1 travels according to the travel route set by the operation management section 14 based on the received instruction information.

In step S122, the vehicle M1 acquires vehicle information (i.e., sensing data and image data) indicating the position, speed, acceleration, steering angle, etc. while traveling.

In step S123, vehicle M1 transmits the vehicle information (ie, sensing data and image data) acquired in step S122 to server device 10. Note that even after step S123, the vehicle M1 acquires vehicle information (that is, sensing data and image data) indicating the position, speed, acceleration, steering angle, etc. while traveling, and transmits it to the server device 10. Note that the vehicle information includes at least first position information indicating a first position, which is the position of vehicle M1.

Note that the processes of steps S121 to S123 are similarly performed in other vehicles M2 to MN.

In step S102, the communication unit 11 of the server device 10 receives vehicle information (that is, sensing data and image data) including the first position information transmitted by the vehicle M1 and the like in step S123. The communication unit 11 provides the vehicle management unit 12 with the received vehicle information (ie, sensing data and image data). Note that in step S102, the communication unit 11 acquires vehicle information (that is, sensing data and image data) through communication.

In step S103, the operation management unit 14 of the server device 10 determines whether the first location information included in the vehicle information received in step S102 conforms to the operation plan, and displays the result of the determination. Generate information.

In step S104, the recommendation unit 15 of the server device 10 acquires second position information indicating the second position from the operation management unit 14. Specifically, the route risk calculation unit 15b of the recommendation unit 15 acquires the second position by acquiring at least one of geographic information and traffic information from the operation management unit 14. Note that the recommendation unit 15 may directly acquire the second location information from the operation DB 13.

In step S105, the recommendation unit 15 determines the monitoring priority for each of the plurality of vehicles M1, etc. Specifically, the recommendation unit 15 acquires the first location information included in the vehicle information received by the communication unit 11 in step S102 from the vehicle management unit 12, and receives the operation information generated by the operation management unit 14 in step S103. Then, in step S104, second position information is acquired from the operation management unit 14.

Then, the recommendation unit 15 determines a monitoring priority for the vehicle according to the degree to which the vehicle requires monitoring by a supervisor based on the acquired vehicle information, operation information, first location information, and second location information. Specifically, the driving risk calculation unit 15a of the recommendation unit 15 calculates a driving risk value based on the acquired vehicle information and operation information and a table (for example, priority table T1) stored in the operation DB 13. do. Further, the route risk calculation unit 15b of the recommendation unit 15 calculates a route risk value based on the acquired first location information and second location information. Then, the integrated risk calculation unit 15c determines an integrated risk value (an example of monitoring priority) based on the driving risk value and the route risk value.

Note that in step S105, the recommendation unit 15 only needs to acquire at least the first location information and the second location information. That is, in step S105, the monitoring priority may be determined based on the route risk value based on the first location information and the second location information. Furthermore, in step S105, monitoring priority is determined for each of the plurality of vehicles M1 using the first position information and the second position information.

In step S106, the recommendation unit 15 determines a recommendation target vehicle that is required to be monitored by a supervisor based on the monitoring priority determined in step S105. Specifically, the recommendation target determining unit 15d of the recommendation unit 15 selects a vehicle with the highest monitoring priority or a vehicle with a monitoring priority equal to or higher than a predetermined threshold value from among the plurality of vehicles M1, etc. as a recommended vehicle. It may be determined that Note that the number of recommended vehicles determined in step S106 is not limited to one, but from the perspective of further reducing the monitoring burden on the observer, it is better to have a smaller number of recommended vehicles determined in step S106. , for example, may be one.

Furthermore, in step S105, the recommendation unit 15 may determine that there is no recommendation target vehicle that requires monitoring by the supervisor, based on the monitoring priority determined in step S102. For example, if there is no vehicle whose monitoring priority is equal to or higher than a predetermined threshold among the plurality of vehicles M1, etc., the recommendation unit 15 determines that there is no recommended vehicle that requires monitoring by a supervisor. good. That is, in step S105, the recommendation unit 15 may determine whether there is a recommendation target vehicle that requires monitoring by a supervisor, based on the monitoring priority determined in step S102.

Then, the monitoring control unit 17 is provided with information specifying the recommendation target vehicle or recommendation information including the fact that the recommendation target vehicle does not exist.

In step S141, the input device 20 receives an operation by the monitor to switch the target of monitoring. This operation may be an operation based on a judgment made by the observer after visually checking the information displayed on the display device 31 and considering which vehicle should be monitored. Then, the input device 20 transmits operation information indicating the content of the accepted operation to the server device 10. The operation information includes information indicating which vehicle among the vehicles M1 and the like has been switched to be a monitoring target by the supervisor. Note that if the monitor has not performed any operation, operation information indicating that no operation has been performed may be transmitted to the server device 10, or no operation information may be transmitted.

In step S107, the receiving unit 16 of the server device 10 receives the operation information transmitted by the input device 20 in step S141.

In step S108, the monitoring control unit 17 determines the object to be monitored by the supervisor, and generates monitoring information that can specify the object to be monitored. When determining the monitoring target, the monitoring control unit 17 determines the monitoring target based on the recommendation information provided by the recommendation unit 15 in step S106 and the operation information received by the receiving unit 16. The monitoring control unit 17 provides the generated monitoring information to the video management unit 18.

Further, when the monitoring control unit 17 acquires recommendation information including that the recommendation target vehicle does not exist, in step S108, the monitoring control unit 17 includes information indicating that monitoring is not required, statistical information on the time when monitoring is not required, and information on the period of time away from monitoring work. Monitoring information including at least one of the proposals may be generated and provided to the video management unit 18. Note that, for example, the monitoring control unit 17 can control the time from when the recommendation information indicating that the recommendation target vehicle does not exist to when the recommendation information including the information specifying the recommendation target vehicle is first acquired, which does not require monitoring. It may also be recorded as time.

In step S109, the video management unit 18 generates presentation information to be presented to the observer based on the monitoring information provided from the monitoring control unit 17 in step S108.

In step S110, the video management unit 18 transmits the presentation information generated in step S109 to the display device 30. That is, in steps S109 and S110, the video management unit 18 generates presentation information for monitoring the vehicle based on the monitoring priority, and transmits the generated presentation information to the display device 30.

In step S131, the display device 30 receives the presentation information transmitted by the monitoring control unit 17 in step S109, and displays information regarding the vehicle to be monitored based on the presentation information. That is, the video management unit 18 causes the display device 30 to output the presentation information by transmitting the presentation information to the display device 30.

Note that if the current operating mode of the monitoring control unit 17 is the manual update mode, the processes of steps S107 and S141 may not be performed.

If monitoring priorities are determined based on the various situations of multiple vehicles and the decision on which vehicle to monitor is left to the supervisor, it is difficult for the supervisor to recognize signs of multiple vehicles and make decisions. Have difficulty. On the other hand, in the present application, since the server device 10 automatically changes the priority according to the time required to reach a high-risk point (that is, a point where it is difficult to drive), the supervisor can Vehicles to be monitored can be appropriately selected depending on the time until the situation becomes difficult. Furthermore, the video management unit 18 may cause the display device 30 to output the presentation information before the vehicle M1 reaches the second position.

FIG. 15 is a flow diagram showing a process for determining monitoring priority in the recommendation unit 15 according to the present embodiment. The process shown in FIG. 15 shows in detail the process executed in step S105 shown in FIG. 14.

In step S201, the recommendation unit 15 acquires vehicle information, operation information, first location information, and second location information. Specifically, the driving risk calculation unit 15a of the recommendation unit 15 acquires vehicle information from the vehicle management unit 12, and acquires operation information from the operation management unit 14. Further, the route risk calculation unit 15b of the recommendation unit 15 acquires first location information and second location information from the operation management unit 14.

In step S202, the driving risk calculation unit 15a calculates a driving risk value of the vehicle based on the vehicle information and operation information. Specifically, the driving risk calculation unit 15a determines the monitoring priority of the vehicle based on the vehicle information and operation information of the vehicle as the driving risk value. That is, the driving risk calculation unit 15a determines the degree to which a supervisor needs to monitor the driving of the vehicle based on the vehicle information and the operation information. The driving risk calculation unit 15a determines the priority corresponding to the situation information detected in the vehicle from among the plurality of situation information included in the priority table T1 as the degree to which the vehicle requires monitoring. For example, when two or more pieces of situation information are detected from among a plurality of pieces of situation information, the driving risk calculation unit 15a adds the priorities corresponding to the two or more pieces of situation information as the driving priority of the vehicle. decide.

When the driving risk calculation unit 15a determines that an accident has occurred in the vehicle based on the driving condition, camera image, acceleration, etc., the driving risk calculation unit 15a detects that "an accident has occurred in the own vehicle" and prioritizes the vehicle. Set the degree to "23". Further, when receiving a call from a passenger of a vehicle, the recommendation unit 15 detects "call occurrence" and determines the priority of the vehicle to be "18". Furthermore, when the recommendation unit 15 determines that the vehicle is located near a railroad crossing based on the vehicle's location information and geographical information, it detects a "level crossing" and sets the priority of the vehicle to "4". do.

Note that if the priority shown in the priority table T1 is an unweighted value, the degree to which monitoring by a supervisor is required may be determined using the weight in step S202. For example, the degree to which monitoring by a supervisor is required may be determined by multiplying the priority corresponding to the situation information and the weight corresponding to the situation information. That is, the degree to which monitoring by a supervisor is required may be determined by calculation. In this case, a predetermined value is used as the weight.

Note that the weights corresponding to each of the plurality of pieces of situation information may be determined sequentially depending on the vehicle or the surrounding situation of the vehicle. The weight may be determined according to information other than the situation information corresponding to the weight. Other information may be determined by the observer depending on the content of the other information (e.g., the number of surrounding objects of the vehicle included in the vehicle information), for example, in the situation information corresponding to the weight (e.g., "crosswalk"). This is information whose degree of monitoring changes. In other words, the other information is information that affects the degree to which the situation information requires monitoring by the monitor.

For example, the degree to which situation information "crosswalk" (an example of first situation information) requires monitoring by a supervisor changes depending on the number of objects surrounding the vehicle. For example, the greater the number of surrounding objects (for example, the number of people), the greater the risk of vehicle travel (for example, the higher the incidence of accidents), and therefore the greater the need for monitoring by a supervisor. Therefore, the recommendation unit 15 may increase the weight value corresponding to "crosswalk" as the number of surrounding objects increases. Note that the other information is not limited to the vehicle information indicated by the number of surrounding objects, but includes situation information other than the relevant situation information among a plurality of situation information (this is an example of the second situation information, for example, "sudden start"). ), operation information, or a combination thereof. For example, if a vehicle that has been stopped near a crosswalk starts moving, the danger of running the vehicle at the crosswalk increases if the vehicle starts moving suddenly, so it is considered that the degree to which a supervisor needs to monitor the vehicle increases. Therefore, the recommendation unit 15 may increase the weight corresponding to the "crosswalk" when the vehicle is "suddenly starting" compared to when the vehicle is not "suddenly starting".

As described above, the plurality of situation information includes first situation information (for example, "crosswalk") and second situation information different from the first situation information (for example, "sudden start"). The weight corresponding to the first situation information may be determined according to at least one of the second situation information, vehicle information, and operation information.

Further, the weight corresponding to each of the plurality of pieces of situation information may be determined according to the time period after the situation information is detected that the monitor is not responding (for example, not monitoring). Taking the situation information "Call Occurrence" as an example, when determining the weight corresponding to the situation information "Call Occurrence", the server device 10 acquires a call from a vehicle passenger, and then the supervisor responds to the call. The determination may be made in accordance with the time during which the request was not made (hereinafter also referred to as non-response time). For example, the longer the unresponsive time for a "call occurrence", the more dangerous the vehicle's driving becomes, and therefore the higher the degree of necessity for monitoring by a supervisor. Therefore, the recommendation unit 15 may increase the value of the weight corresponding to "call occurrence" as the unresponsive time for the call increases.

Moreover, the weight corresponding to each of the plurality of situation information may be determined according to the difference between the situation information and the reference. Taking the situation information "time schedule deviation" as an example, when determining the weight corresponding to the situation information "time schedule deviation", the time difference (for example, may be determined depending on the amount of time (time behind schedule). For example, the greater the time difference from the operation plan, the greater the risk of vehicle travel, and therefore the greater the need for monitoring by a supervisor. Therefore, the recommendation unit 15 may increase the value of the weight corresponding to the "time schedule deviation" as the time difference from the operation plan becomes larger.

In addition, as another example of determining the weight according to the difference from the reference value in the situation information, the situation information "abnormal speed" (not shown) will be described. When determining the weight corresponding to the situation information "abnormal speed", the weight may be determined according to the speed difference from the legal speed of the road on which the vehicle is traveling (an example of a reference in the situation information). For example, the greater the speed difference from the legal speed, the more dangerous the vehicle's driving becomes, and therefore the greater the need for monitoring by a supervisor. Therefore, the recommendation unit 15 may increase the value of the weight corresponding to "speed abnormality" as the speed difference from the legal speed increases. Note that the legal speed is acquired based on, for example, vehicle position information and geographic information.

In step S203, the route risk calculation unit 15b calculates the route risk value of the vehicle based on the first position information and the second position information. The route risk calculation unit 15b determines the monitoring priority of the vehicle as a route risk value according to the movement information regarding the movement of the vehicle from the first position indicated by the first position information to the second position indicated by the second position information. decide. That is, the route risk calculation unit 15b determines the degree to which the supervisor should monitor the route of the vehicle based on the first position information and the second position information. For example, when the movement information is arrival time, the route risk calculation unit 15b determines the priority to be a higher value as the arrival time is shorter. The route risk calculation unit 15b, for example, acquires geographical information from the operation management unit 14, and determines the priority of the route based on the priority of the second position included in the acquired geographical information and the arrival time. You may. For example, if a situation indicated by two or more pieces of situation information occurs at the second position, for example, if a traffic jam occurs at a confluence point, the route risk calculation unit 15b calculates whether only one of the conditions is occurring. The priority may be set to a higher value than in the case of That is, the route risk calculation unit 15b may determine the priority of the route according to the number of situations at the second position.

Note that the second position may be a point where an object that satisfies a predetermined condition, such as an accident vehicle or an emergency vehicle, is located. In this case, the route risk calculation unit 15b may acquire vehicle information from the vehicle management unit 12 and identify an object that satisfies a predetermined condition from sensing data included in the acquired vehicle information. The route risk calculation unit 15b may then set the point where the identified object is located as the second position. For example, when calculating the route risk value for the vehicle M1, the route risk calculation unit 15b may specify the second position using the sensing data acquired from the vehicle M1, or may identify the second position using the sensing data acquired from the vehicle M1. The second position may be specified using the following.

In step S204, the integrated risk calculation unit 15c calculates an integrated risk value based on the driving risk value and the route risk value as the monitoring priority. Specifically, the integrated risk calculation unit 15c calculates and integrates the priority calculated as the driving risk value and the priority calculated as the route risk value, thereby calculating the monitoring priority for the vehicle. decide. The integrated risk calculation unit 15c may determine the monitoring priority by, for example, adding or multiplying the priority calculated as the driving risk value and the priority calculated as the route risk value. The integrated risk calculation unit 15c may add or multiply after weighting at least one of the two priorities.

As described above, when there is a point on the route that has a high risk of a situation requiring response occurring, that is, when there is a second position on the route, the recommendation unit 15 recommends that The time or distance (an example of movement information) is calculated, and the monitoring priority is varied according to the calculated arrival time or distance. That is, when a plurality of vehicles exist, the recommendation unit 15 determines the monitoring priority based on the movement information.

Note that the route risk calculation unit 15b may further acquire the degree of difficulty of traveling at the second position, and determine the priority of the vehicle in the route according to the acquired degree of difficulty and arrival time. The route risk calculation unit 15b may obtain the degree of difficulty of traveling, for example, by reading out from the operation DB 13 a table (not shown) in which the situation at the second position is associated with the degree of difficulty of traveling. For example, information regarding the difficulty level may be included in the work information table T2 shown in FIG. Note that the longer the work time included in the work information table T2, the higher the difficulty level. The route risk calculation unit 15b determines the priority to be a higher value as the degree of difficulty of traveling is higher.

The degree of difficulty in driving indicates the degree of difficulty in automatically driving the vehicle, and may include, for example, the degree of difficulty caused by road conditions that are different from usual. For example, when construction requires a detour or one-way traffic, the difficulty level of traveling may be set to be high.

Note that the route risk calculation unit 15b may further acquire work information regarding the monitoring work at the second position, and determine the priority of the vehicle in the route according to the acquired work information and movement information. If the movement information includes the arrival time and the work information includes the work time (for example, see FIG. 5), the route risk calculation unit 15b may determine the priority according to the arrival time and the work time. For example, the route risk calculation unit 15b may determine the priority to be a higher value as the arrival time is shorter or the work time is longer. The route risk calculation unit 15b determines the priority to be a higher value as the difference between the arrival time and the work time (specifically, the value obtained by subtracting the work time from the arrival time, also referred to as grace time) is shorter. You can. The route risk calculation unit 15b calculates the priority in the route by weighting the priority corresponding to the situation of the second position (for example, the priority obtained from geographical information) according to the grace period. You may decide.

Thereby, the observer can secure the necessary time to monitor the vehicle until it reaches the second position. For example, the observer can grasp the situation of the vehicle and perform operations if necessary before reaching the second position. In other words, the observer can monitor the target vehicle more appropriately. Note that, when the grace period becomes negative, the route risk calculation unit 15b may output information indicating that the grace period is negative. Then, for example, if the grace period is negative, the recommendation target determining unit 15d may determine the vehicle as the recommendation target vehicle.

Note that the route risk calculation unit 15b may further acquire the degree of difficulty of traveling at the second position, and determine the grace time, which is the difference between the arrival time and the work time, and the priority according to the degree of difficulty. good. For example, the route risk calculation unit 15b may determine the priority to be a higher value as the grace period is shorter or the degree of difficulty is higher. The route risk calculation unit 15b weights the priority corresponding to the situation of the second position (for example, the priority obtained from geographical information) according to the grace period and the degree of difficulty, thereby determining the route. You may decide the priority of

Note that the communication unit 11 or the vehicle management unit 12 is an example of a first acquisition unit that acquires first position information, and the operation management unit 14 is an example of a second acquisition unit that acquires second position information.

FIG. 16 is a flow diagram showing the processing of the monitoring control unit 17 according to the present embodiment. The process shown in FIG. 16 shows in detail the process executed in step S108 shown in FIG. 14.

In step S301, the monitoring control unit 17 determines whether the receiving unit 16 has received operation information from the input device 20. If it is determined that the operation information has been received (Yes in step S301), the process advances to step S302, and if not (No in step S301), step S301 is executed again. That is, the monitoring control unit 17 enters a standby state in step S301 until it receives the operation information.

In step S302, the monitoring control unit 17 acquires the details of the operation by the supervisor based on the operation information received by the receiving unit 16.

In step S303, the monitoring control unit 17 determines whether an operation for switching the monitoring target has been performed. If it is determined that the above operation has been performed (Yes in step S303), the process proceeds to step S321, and if it is determined that the above operation has not been performed (No in step S303), the process proceeds to step S304.

In step S304, the monitoring control unit 17 determines whether the current operating mode is the automatic update mode. If the automatic update mode is in effect (Yes in step S304), the process advances to step S305; otherwise (No in step S304), the process advances to step S311.

In step S305, the monitoring control unit 17 acquires the recommendation information provided by the recommendation unit 15 in step S106.

In step S306, the monitoring control unit 17 determines whether the recommendation information includes information specifying the recommendation target vehicle. If the recommendation information includes information specifying the recommended vehicle (Yes in S306), that is, if the recommendation unit 15 determines that the recommended vehicle exists, the process advances to step S307; otherwise (No in step S306). Then, the process advances to step S308.

In step S307, the monitoring control unit 17 determines the recommendation target vehicle recommended based on the recommendation information as the monitoring target vehicle.

In step S308, the monitoring control unit 17 determines that the monitoring target vehicle does not exist based on the recommendation information.

In step S311, the monitoring control unit 17 determines to maintain the vehicle currently being monitored.

In step S321, the supervisory control unit 17 determines that the vehicle to which the supervisor performs the switching operation is to be the vehicle to be monitored.

In step S309, the monitoring control unit 17 generates monitoring information indicating the target of monitoring based on the determination in step S307, S308, S311, or S321. When the process of step S309 is finished, the series of processes shown in FIG. 16 ends. The monitoring information generated in step S309 is provided to the video management unit 18 and used in the process of step S109.

In addition, although the case where the recommendation part 15 recommends a vehicle to be monitored has been described above, the recommendation part 15 may further recommend including how to monitor the vehicle. "How to monitor the vehicle" includes, for example, monitoring how fast the vehicle is traveling, and which direction to monitor when viewed from the vehicle. In that case, the recommendation information includes not only information that identifies the vehicle, but also information that identifies how the vehicle is to be monitored. If the monitoring information further includes information specifying the vehicle monitoring method, data corresponding to the monitoring method among the image data and sensing data received by the communication unit 11 from the vehicle is included in the presentation information, and the monitoring information is present to the person concerned. More specifically, when recommending which direction to monitor when viewed from the vehicle as a vehicle monitoring method, the camera in the direction that is the target of monitoring among the one or more cameras equipped on the vehicle is acquired. The displayed image may be included in the presentation information.

(Modified example of embodiment)
The information processing method and the like of this modification will be explained with reference to FIG. 17. Note that the configuration of the monitoring system according to this modification is the same as that in the embodiment, and a description thereof will be omitted.

FIG. 17 is a sequence diagram showing the processing of the monitoring system 1 according to this modification. In FIG. 17, vehicles M1 and M2 are illustrated as representative vehicles such as vehicle M1, but similar processing is performed for other vehicles as well. An example is shown in which the server device 10 acquires vehicle information etc. from vehicles M1 and M2. Further, steps S401 to S410 performed by the server device 10 are similar to steps S101 to S110 shown in FIG. 14, and a description thereof will be omitted. Furthermore, steps S421 to S423 performed by vehicle M1 and steps S424 to S426 performed by vehicle M2 are similar to steps S121 to S123 shown in FIG. 14, and their explanation will be omitted. Further, step S431 performed by the display device 30 is similar to step S131 shown in FIG. 14, and a description thereof will be omitted. Further, step S441 performed by the input device 20 is the same as step S141 shown in FIG. 14, and a description thereof will be omitted. In the following description, it will be assumed that the vehicle M1 is determined to be the monitoring target from among the vehicles M1 and the like in step S408.

In step S411, the server device 10 transmits communication amount control information for controlling the amount of communication with the vehicle to the vehicle M2. That is, the server device 10 transmits the communication amount control information to the vehicle M2 that was not determined to be a monitoring target in step S408. Here, the communication amount control information is information for reducing the amount of communication between the vehicle M2 and the server device 10.

The amount of communication between vehicle M2 and server device 10 is controlled according to the monitoring priority. The monitoring priority of vehicle M2 is lower than the monitoring priority of vehicle M1. Therefore, the server device 10 may generate the communication amount control information so that the amount of communication between the vehicle M2 and the server device 10 is smaller than the amount of communication between the vehicle M1 and the server device 10. Further, the server device 10 may generate communication amount control information such that the amount of communication between the vehicle M2 and the server device 10 is smaller after the monitoring target is determined than before the monitoring target is determined. good.

Note that when the monitoring target is determined to be the vehicle M1, the communication amount control information is transmitted to all vehicles other than the vehicle M1 (also referred to as other vehicles) among the plurality of vehicles M1 and the like. The communication amount control information transmitted by another vehicle may be determined according to the monitoring priority of the vehicle. A vehicle with a lower monitoring priority among other vehicles may be controlled to have a smaller amount of communication than a vehicle with a higher monitoring priority among other vehicles. Thereby, it is possible to suppress the amount of communication between the vehicle M1 and the like and the server device 10, while transmitting more information about a vehicle that is likely to become a monitoring target later on among other vehicles. Note that the server device 10 may uniformly reduce the amount of communication of other vehicles.

In step S427, vehicle M2 receives the communication amount control information.

In step S428, vehicle M2 changes the amount of communication with the server device 10 based on the received amount of communication control information. The vehicle M2 reduces the amount of communication by at least one of lowering the image quality of the image data transmitted from the vehicle M2, lowering the communication speed for transmitting the image data, and lowering the frame rate for capturing the image data. may be made smaller. Furthermore, when the vehicle M2 is equipped with two or more cameras that photograph the surroundings of the vehicle M2, the amount of communication may be reduced by transmitting only image data photographed by a predetermined camera to the server device 10. The predetermined camera is determined based on, for example, the moving route of the vehicle M2, and is, for example, a camera that photographs the direction in which the vehicle M2 is traveling.

Note that the process in step S411 may be performed by any component included in the server device 10. For example, the monitoring control unit 17 or the video management unit 18 may perform the process of step S611. For example, after transmitting the presentation information to the display device 30, the video management section 18 may transmit the communication amount control information to the vehicle M2 via the communication section 11.

(Other embodiments)
Although the present disclosure has been described above based on embodiments and modified examples (hereinafter also referred to as embodiments, etc.), the present disclosure is not limited to the above-described embodiments.

For example, in the above embodiments, the vehicle selected by the operation of the observer and the recommended vehicle of the recommendation unit are determined as the vehicle to be monitored with priority. has been described, but is not limited thereto. The recommendation target vehicle may be determined as the monitoring target vehicle with priority over the vehicle selected by the operation by the supervisor. For example, when the monitoring priority is equal to or higher than a predetermined value, the monitoring control unit may prioritize the recommended vehicle and determine the recommended vehicle as the monitoring target vehicle.

Furthermore, in the above embodiments and the like, an example has been described in which the recommendation target determining unit of the recommendation unit determines a vehicle with the highest monitoring priority or a vehicle higher than a threshold value as a recommendation target vehicle. It is not limited to this. The recommendation unit may, for example, determine a vehicle in which predetermined situation information is detected from among a plurality of pieces of situation information as the recommendation target vehicle. For example, a vehicle in which the status information "an accident occurred in the own vehicle" has been detected may be determined as a recommendation target vehicle. Further, for example, the recommendation target determining unit may determine, as the recommendation target vehicle, a vehicle for which it is detected that the grace period is less than or equal to a predetermined time.

Furthermore, in the above embodiments and the like, when there are multiple monitors, the recommendation unit may change the priority or order of priority depending on the status of the monitor. The "merging" shown in FIG. 5 will be explained as an example. Assume that supervisor A has handled "merging" more often than supervisor B in the past. In this case, supervisor A can handle the "merging" more smoothly than supervisor B. Therefore, the work time for "merging" for supervisor A may be shorter than for supervisor B. For example, when there are multiple supervisors and it becomes necessary to monitor a "merging", the supervisory control unit 17 may preferentially allocate the monitoring to a supervisor who has a short working time. Thereby, when there are multiple observers, it is possible to have the observer who can smoothly deal with the situation information at the second position monitor the vehicle.

Further, in the above embodiments, for example, when two or more pieces of situation information are detected from among a plurality of pieces of situation information, the driving risk calculation unit calculates a value obtained by adding the priorities corresponding to the two or more pieces of situation information. Although an example has been described in which the priority is determined for the traveling of the vehicle, the priority is not limited thereto. The driving risk calculation unit converts statistical values such as the average value, median value, maximum value, and minimum value of two or more priorities corresponding to two or more pieces of situation information into the driving priority (driving risk value) of the vehicle. It may be determined as

Further, in the above embodiments, the route risk calculation unit calculates a value obtained by adding the priorities corresponding to the two or more pieces of situation information to the route of the vehicle, for example, when the number of pieces of information at the second position is two or more. Although an example has been described in which the priority is determined as the priority, the priority is not limited to this. The route risk calculation unit calculates statistical values such as the average value, median value, maximum value, and minimum value of two or more priorities corresponding to two or more pieces of situation information as a priority (route risk value) for the route of the vehicle. It may be determined as

Further, in the above embodiments and the like, the priority in driving and the priority in a route may be determined using, for example, a machine learning model. A machine learning model trained using learning data is implemented inside the recommendation unit. The recommendation unit may obtain a priority output as a result of inputting the acquired vehicle information and operation information as input data into a machine learning model as a priority in driving. Further, the recommendation unit may obtain the priority output as a result of inputting the acquired first location information and second location information as input data to the machine learning model as the priority in the route.

Further, the priority level or the degree of difficulty of traveling indicated by the priority table in the above embodiments etc. may be determined as appropriate depending on the surrounding situation at the second position, for example. The route risk calculation unit corrects the priority indicated by the priority table according to the number of surrounding objects (for example, including pedestrians or bicycles) or the type of surrounding objects around the second position, or The difficulty level of running may also be determined. The route risk calculation unit may increase the priority or difficulty level as the number of peripheral objects around the second position increases.

Furthermore, in the above embodiments and the like, it has been described that there are multiple types of vehicle information, but weights may be associated with multiple types of vehicle information. For example, a weight may be associated with each of the steering angle or speed (an example of vehicle information). Then, the monitoring priority may be determined based on the weight.

Furthermore, the monitoring priority may be determined using not only information on the vehicle to be monitored but also information on other vehicles. Specifically, the monitoring control unit acquires movement information of a vehicle other than the vehicle to be monitored, and determines whether the other vehicle is the destination of the vehicle to be monitored or is heading toward a second position on the movement route. Determine whether or not there is. If it is determined that another vehicle is heading to the second position, the monitoring control unit determines the monitoring priority to be higher than in the case where this is not the case. Further, the monitoring control unit determines the monitoring priority according to the difference between the time when another vehicle heading to the second position reaches the second position and the time when the vehicle to be monitored reaches the second position. You may. Note that another vehicle may also be a monitoring target.

Further, there is no particular limitation on the method of communication between devices included in the monitoring system in the above embodiments and the like. Wireless communication or wired communication may be performed between the devices. Additionally, wireless communication and wired communication may be combined between devices.

Further, each processing unit (for example, a vehicle management unit, a traffic management unit, a recommendation unit, etc.) of the monitoring system according to the above embodiments is typically realized as an LSI, which is an integrated circuit. These may be integrated into one chip individually, or may be integrated into one chip including some or all of them.

Further, circuit integration is not limited to LSI, and may be realized using a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI may be used.

Note that in the above embodiments and the like, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

Moreover, all the numbers used above are exemplified to specifically explain the present disclosure, and each embodiment of the present disclosure is not limited to the illustrated numbers.

Furthermore, the division of functional blocks in the block diagram is just an example; multiple functional blocks can be realized as one functional block, one functional block can be divided into multiple functional blocks, or some functions can be moved to other functional blocks. You can. Further, functions of a plurality of functional blocks having similar functions may be processed in parallel or in a time-sharing manner by a single piece of hardware or software.

Further, in the above embodiments, the server device is realized by a single device, but it may be realized by a plurality of devices connected to each other.

Further, the order in which the steps in the flowchart are executed is for illustrative purposes to specifically explain the present disclosure, and may be in an order other than the above. Further, some of the above steps may be executed simultaneously (in parallel) with other steps.

Other forms may be obtained by making various modifications to the above embodiments etc. that those skilled in the art would think of, or may be realized by arbitrarily combining the constituent elements and functions of the embodiments etc. without departing from the spirit of the present disclosure. Forms are also included in this disclosure.

The present disclosure can be used in an information processing method for a supervisor to monitor an operated vehicle.

1 Monitoring system (information processing system)
10 Server device 11 Communication department 12 Vehicle management department 13 Operation DB
14 Operation Management Department 15 Recommendation Department (Decision Department)
15a Driving risk calculation unit 15b Route risk calculation unit 15c Integrated risk calculation unit 15d Recommendation target determination unit 16 Receiving unit 17 Monitoring control unit 18 Video management unit (presentation control unit)
20 Input device 30, 31 Display device 32, 32a, 32b, 33-39 Image M1, M2,..., MN Vehicle N Network T1 Priority table T2 Work information table

Claims (22)

Using a computer communicably connected to the vehicle,
obtaining first position information indicating a first position that is the position of the vehicle to be monitored;
obtaining second position information indicating a second position that is a position that requires monitoring of the vehicle;
determining a monitoring priority for the vehicle according to movement information regarding movement by the vehicle from the first location to the second location;
An information processing method that controls the amount of communication with the vehicle based on the monitoring priority. The information processing method according to claim 1, wherein when another vehicle different from the vehicle has a lower monitoring priority than the vehicle, the amount of communication with the other vehicle is made smaller than the amount of communication with the vehicle. further generating presentation information for monitoring the vehicle based on the monitoring priority;
The information processing method according to claim 1 or 2, further comprising causing a presentation device to output the presentation information. using a computer,
Obtaining first location information indicating a first location that is the location of the vehicle to be monitored;
obtaining second position information indicating a second position that is a position that requires monitoring of the vehicle;
Obtaining work information regarding the work of the supervisor according to the degree of difficulty of traveling at the second position,
determining a monitoring priority for the vehicle according to movement information regarding movement by the vehicle from the first position to the second position and the work information;
generating presentation information for monitoring the vehicle based on the monitoring priority;
An information processing method that causes a presentation device to output the presentation information. The work information includes at least one of a first work time required for monitoring the vehicle and a second work time required for operating the vehicle,
The movement information includes the arrival time from the first position to the second position,
The information processing method according to claim 4, wherein the monitoring priority is determined according to the arrival time and at least one of the first work time and the second work time. Furthermore, acquiring the difficulty level of traveling at the second position,
The information processing method according to claim 5, wherein the monitoring priority is determined according to the difference between the arrival time and at least one of the first work time and the second work time, and the degree of difficulty. The information processing method according to claim 5 or 6, wherein the longer at least one of the first working time and the second working time is, the higher the monitoring priority is. The information processing method according to claim 5, wherein the shorter the grace period obtained by subtracting at least one of the first work time and the second work time from the arrival time, the higher the monitoring priority is. using a computer,
Obtaining first location information indicating a first location that is the location of the vehicle to be monitored;
obtaining second position information indicating a second position that is a position that requires monitoring of the vehicle;
causing a presentation device to output presentation information for monitoring the vehicle based on the first location information and the second location information;
The presentation information includes first information indicating the reason why the vehicle is determined to be a vehicle to be monitored, second information regarding a method for monitoring the vehicle, and whether monitoring is unnecessary when there is no vehicle to be monitored. An information processing method including at least one third information regarding. determining a monitoring priority for the vehicle according to movement information regarding movement by the vehicle from the first location to the second location;
The information processing method according to claim 9, wherein the presentation information is generated based on the monitoring priority. The presentation information includes at least the first information,
The information processing method according to claim 9, wherein the first information includes information indicating which situation information among a plurality of situation information indicating the situation of the vehicle has been detected in the vehicle. determining a monitoring priority for the vehicle according to movement information regarding movement by the vehicle from the first location to the second location;
The presentation information includes at least the first information,
The information processing method according to claim 11, wherein the first information includes information indicating a degree of contribution of at least one of the plurality of situation information to the monitoring priority. The presentation information includes at least the first information,
The information processing method according to claim 10, wherein the first information includes the monitoring priority. The presentation information includes at least the second information,
The information processing method according to claim 9, wherein the second information includes an image taken in a direction in which the vehicle is monitored by a remote monitor of the vehicle. The presentation information includes at least the second information,
The information processing method according to claim 9, wherein the second information includes information specifying how to monitor the vehicle. The presentation information includes at least the third information,
The information processing method according to claim 9, wherein the third information includes information indicating that the monitoring target does not exist. Further, recording a time when no monitoring is required, which is a time when there is no vehicle with the monitoring priority equal to or higher than a predetermined threshold;
The presentation information includes at least the third information,
The information processing method according to claim 10, wherein the third information includes statistical information about the time when monitoring is not required. The information processing method according to any one of claims 9 to 17, wherein the presentation information is information obtained by superimposing and displaying at least one of the first information, the second information, and the third information on an image. . The presentation information includes information for monitoring one or more of the vehicles to be monitored, the number of which is determined according to the monitoring ability of a remote monitor of the vehicle. 18. The information processing method according to any one of 17. using a computer,
obtaining first position information indicating a first position of each of a plurality of vehicles to be monitored by a remote monitor;
obtaining second position information indicating a second position that requires monitoring by the remote monitor for each of the plurality of vehicles;
determining a monitoring priority for each of the plurality of vehicles according to movement information regarding movement by the vehicle from the first position to the second position;
Based on the monitoring priority, generating presentation information that allows the remote monitor to monitor a vehicle with a relatively high monitoring priority among the plurality of vehicles;
An information processing method that causes a presentation device to output the presentation information. Further, it is determined whether or not the operation mode of the vehicle is an automatic update mode, and if the operation mode is the automatic update mode and an operation by the remote monitor indicating switching of the monitoring target is not received, the The information processing method according to claim 20, wherein the presentation information is generated based on monitoring priority. Further, when an operation by the remote monitor indicating switching of the monitoring target is received, the presentation information that causes the remote monitor to monitor a vehicle based on the operation among the plurality of vehicles is generated. 22. The information processing method described in 21.

Images