JP2019016118A - Monitoring program, monitoring method, and monitoring device - Google Patents

Abstract

To appropriately obtain correspondence between an image and sound concerning a remotely-monitored mobile body.SOLUTION: A monitoring program allows a computer to execute an acquisition procedure and a notification procedure. The acquisition procedure is the procedure to acquire multiple pieces of image information and multiple pieces of sound information detected in a mobile body that is the mobile body to be remotely monitored by a monitoring person and operated by automatic driving. The notification procedure is the procedure to notify a monitoring person of object sound information being the sound information satisfying a predetermined condition which requires the confirmation of an operation state among multiple pieces of sound information acquired in the acquisition procedure.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a monitoring program, a monitoring method, and a monitoring apparatus.

  Conventionally, techniques for monitoring various objects have been provided. For example, a technique related to a monitoring system for monitoring a target to be monitored such as a building or a house at a remote position and performing safety measures such as crime prevention is known.

JP 2004-287922 A

  However, with the above-described conventional technology, it is difficult to appropriately monitor a moving body such as a bus operated by an automatic driving technology. For example, when a monitor located remotely from a moving body monitors the moving body based on a moving image (image) or sound, when a sound detected in the moving body is output to the monitoring person, the monitoring device It may be difficult for the supervisor to know which image is displayed on the screen.

  The present application has been made in view of the above, and an object of the present invention is to provide a monitoring program, a monitoring method, and a monitoring apparatus that can appropriately grasp the correspondence between an image and sound related to a remotely monitored mobile object. To do.

  The monitoring program according to the present application is a moving body that is remotely monitored by a supervisor, an acquisition procedure for acquiring a plurality of audio information detected in a moving body that operates by automatic driving, and a plurality of acquisition steps acquired by the acquisition procedure And a notification procedure for notifying the monitor of target audio information, which is audio information satisfying a predetermined condition that requires confirmation of the operation status, among audio information.

  According to the aspect of the embodiment, there is an effect that it is possible to appropriately grasp the correspondence between the image and the sound related to the mobile body to be remotely monitored.

FIG. 1 is a diagram illustrating an example of a monitoring process according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the monitoring system according to the embodiment. FIG. 3 is a diagram illustrating a configuration example of the monitoring apparatus according to the embodiment. FIG. 4 is a diagram illustrating an example of the mobile object information storage unit according to the embodiment. FIG. 5 is a diagram illustrating a configuration example of the determination device according to the embodiment. FIG. 6 is a diagram illustrating an example of the mobile object information storage unit according to the embodiment. FIG. 7 is a diagram illustrating an example of a supervisor information storage unit according to the embodiment. FIG. 8 is a diagram illustrating an example of the monitoring correspondence information storage unit according to the embodiment. FIG. 9 is a diagram illustrating an example of the monitoring process according to the embodiment. FIG. 10 is a diagram illustrating an example of a display change according to the embodiment. FIG. 11 is a diagram illustrating an example of a monitoring process according to the embodiment. FIG. 12 is a diagram illustrating an example of a display change according to the embodiment. FIG. 13 is a flowchart illustrating an example of the monitoring process according to the embodiment. FIG. 14 is a diagram illustrating an example of a monitoring process for one moving object. FIG. 15 is a hardware configuration diagram illustrating an example of a computer that implements the functions of the monitoring device and the determination device.

  Hereinafter, modes for implementing a monitoring program, a monitoring method, and a monitoring apparatus according to the present application (hereinafter referred to as “embodiments”) will be described in detail with reference to the drawings. Note that the monitoring program, the monitoring method, and the monitoring apparatus according to the present application are not limited to the embodiment. In the following embodiments, the same portions are denoted by the same reference numerals, and redundant description is omitted.

(Embodiment)
[1. (Monitoring process)
First, an example of the monitoring process according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a monitoring process according to the embodiment. In the example shown in FIG. 1, the determination device 100 (see FIG. 5) provides various information to a monitor (operator) that remotely monitors an automatic driving bus (hereinafter also simply referred to as “bus”) that automatically travels on a road. The case of changing (deciding) based on In the example illustrated in FIG. 1, the determination apparatus 100 changes a monitor (operator) who remotely monitors a bus that automatically travels on a road at a timing based on various information. Specifically, the determination apparatus 100 changes the monitor who remotely monitors each of the buses 20-1 to 20-5 at a timing based on various conditions for the plurality of buses 20-1 to 20-5. Show the case. FIG. 1 shows a case where each monitor (operator) remotely monitors a plurality of buses.

  Moreover, in the example shown in FIG. 1, the case where bus | baths 20-1 to 20-5 operate | move based on the operation route matched with each is shown. For example, the buses 20-1 to 20-5 travel on public roads along a predetermined operation route. The buses 20-1 to 20-5 are not limited to travel based on the operation route, but may travel on a route according to the designation of the destination of the passenger. Although FIG. 1 shows a case where a bus that is a vehicle that automatically travels is shown, the mobile body is not limited to a bus, and may be various mobile bodies. For example, the target is not limited to a bus, and may be various business vehicles such as a taxi or a truck such as a truck. Further, for example, the mobile body may be a private car, a so-called private car. For example, what is transported by the moving object to be monitored in the monitoring system 1 (see FIG. 2) is not limited to a person and may be various objects. In other words, various objects including tangible objects such as a person or an article other than a person may be used as an object for transporting the moving object in the monitoring system 1.

[Configuration of the monitoring system]
Prior to the description of FIG. 1, the configuration of the monitoring system 1 will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the monitoring system according to the embodiment. As shown in FIG. 2, the monitoring system 1 includes a monitoring device 10-1, a monitoring device 10-2, a bus 20-1, a bus 20-2, and a determination device 100. For example, the monitoring system 1 may be a so-called remote automatic traveling system. The remote automatic driving system here refers to an automatic operation that allows a driver (monitorer or operator) who is remote from a car (moving body) to operate the car using telecommunications technology. It means a system related to driving technology.

  Hereinafter, the monitoring devices 10-1 and 10-2 will be referred to as the monitoring device 10 when they are not particularly distinguished. In the monitoring system 1 illustrated in FIG. 2, two monitoring devices 10-1 and 10-2 are illustrated, but the monitoring system 1 may include three or more monitoring devices 10. Further, in the monitoring system 1 shown in FIG. 2, two buses 20-1 and 20-2 are illustrated, but the monitoring system 1 includes five buses 20-1 to 20- as shown in FIG. 5 may be included. Hereinafter, the buses 20-1 to 20-5 will be referred to as the bus 20 when they are not particularly distinguished. Note that the monitoring system 1 may include six or more buses 20. As shown in FIG. 2, the monitoring device 10, the bus 20, and the determination device 100 are connected via a predetermined network N so that they can communicate with each other by wire or wirelessly. Note that the monitoring system 1 illustrated in FIG. 2 may include a plurality of determination devices 100.

  The monitoring device 10 is an information processing device used by a monitor who remotely monitors the bus 20. The monitoring device 10 is a terminal device for a monitor to monitor a moving body. For example, the supervisors such as the supervisor A1 and the supervisor A2 may be employees of a business operator (company) that conducts a passenger transportation business using the bus 20. For example, the supervisor A1 or the supervisor A2 may be an employee of a transportation organization that operates a bus 20 transportation business. For example, the supervisor A1 and the supervisor A2 may be workers engaged in remote monitoring work at the location (building or the like) of a transportation organization that operates the bus 20 transportation business. When “monitoring person A * (* is an arbitrary numerical value)” is described, it indicates that the monitoring person is a user identified by the monitoring person ID “A *”. For example, when “monitoring person A1” is described, the monitoring person is a monitoring person identified by the monitoring person ID “A1”.

  The monitoring device 10 is realized by, for example, a notebook PC or a desktop PC. Further, the monitoring device 10 may be an information processing device such as a smartphone, a tablet terminal, a mobile phone, or a PDA. Moreover, the monitoring apparatus 10 may be a wearable terminal such as a glasses-type terminal. In other words, the monitoring device 10 may be any information processing device as long as the monitor can perform a remote monitoring operation. In the example of FIG. 1, the case where the monitoring apparatus 10 is a notebook PC is shown for the sake of simplicity.

  The bus 20 is a moving body that travels by automatic driving. For example, the bus 20 is a moving body that automatically travels by appropriately using various conventional techniques related to automatic driving. In addition, the bus 20 is a mobile body that allows a remote monitor to remotely monitor or remotely operate the bus 20 using telecommunications technology. Further, the bus 20 is assumed to be a mobile body that has a communication function (not shown) and can perform information communication with the monitoring device 10 and the determination device 100. In addition, the bus 20 is assumed to be a moving body that has an information processing function (not shown) and is controlled based on information received from the monitoring device 10 or the determination device 100.

  The bus 20 includes various sensors such as a camera (not shown) and a microphone (not shown), and can detect information indicating the surroundings of the bus 20 and the situation in the bus 20. The camera mounted on the bus 20 may be any sensor as long as it can capture images and moving images around the bus 20 and in the bus 20. For example, the camera may be various sensors such as a CCD (Charge-Coupled Device) image sensor. The microphone mounted on the bus 20 may be any sensor as long as it can collect sound around the bus 20 or in the bus 20.

  In addition, the bus 20 transmits a moving image detected (captured) by the camera to the monitoring device 10 or the determination device 100 used by the supervisor. For example, the bus 20 transmits the moving image detected by the camera in real time to the monitoring device 10 or the determination device 100 used by the monitor. In addition, the bus 20 transmits the sound detected (sound collection) by the microphone to the monitoring device 10 and the determination device 100 used by the monitor. For example, the bus 20 transmits the sound detected by the microphone in real time to the monitoring device 10 and the determination device 100 used by the supervisor.

  The bus 20 has a sensor (hereinafter also referred to as “position sensor”) that can detect position information such as a GPS (Global Positioning System) sensor, and can detect the position of the bus 20. For example, the bus 20 transmits the position information detected by the position sensor to the monitoring device 10 or the determination device 100 used by the monitor. For example, the bus 20 transmits the position information detected by the position sensor in real time to the monitoring device 10 and the determination device 100 used by the monitor.

  Further, the bus 20 is assumed to be able to travel autonomously on a designated route based on information detected by a camera or a microphone while receiving no instruction from the outside by a supervisor or the like. The bus 20 is not limited to cameras and microphones, and may travel autonomously using various conventional techniques such as radar. In addition, it is assumed that an operator who operates the bus 20 in the bus 20, a so-called driver, is not on the bus 20 in the example illustrated in FIG. 1.

  The determining apparatus 100 is an information processing apparatus that determines an aspect related to automatic driving of a plurality of moving bodies based on whether or not the monitoring status of one monitoring condition satisfies a predetermined condition based on information regarding the monitoring person. Further, the determination device 100 is an information processing device that determines an aspect related to automatic driving of a plurality of moving bodies when the state of the plurality of moving bodies satisfies a predetermined condition based on information about the plurality of moving bodies. Moreover, the determination apparatus 100 is an information processing apparatus that determines the timing for changing a monitor who monitors a plurality of moving objects based on information related to remote monitoring. For example, the information related to the monitor may include information indicating whether the monitor can be monitored, information indicating the monitoring mode of the monitor, or information related to the mobile object that is monitored by the monitor. For example, the information related to the moving body may include sensor information (moving image, sound, etc.) detected in the moving body and collected information. For example, the information related to remote monitoring may include various information such as information related to a monitor and information related to a moving object.

  From here, the monitoring process will be described with reference to FIG. In the example of FIG. 1, a case where two monitoring personnel A1 and monitoring personnel A2 remotely monitor the five buses 20-1 to 20-5 is illustrated. In the monitoring system 1, three or more (for example, 100) supervisors may monitor five or more (for example, 1,000) buses 20.

  First, the map information MP11-1 shown in FIG. 1 will be briefly described. The map information MP11-1 illustrated in FIG. 1 is a diagram schematically illustrating the positions of the buses 20-1 to 20-5, the relationship between the bus 20 and the supervisor, and the like. The map information MP11-1 shown in FIG. 1 is for illustrating the change in the relationship between the bus 20 and the supervisor, that is, the change of the supervisor of the bus 20.

  Hereinafter, the map information MP11-1 and MP11-2 (see FIGS. 9 and 11) will be referred to as map information MP11 when they are described without distinction. In the example of FIG. 1, a dotted line connecting the bus 20 and the monitoring device 10 shown in the map information MP11 indicates the relationship between the bus 20 and the monitor. In the map information MP11-1 in FIG. 1, since the buses 20-1 to 20-3 and the monitoring device 10-1 used by the monitor A1 are connected by a dotted line, the map information MP11- in FIG. 1 indicates that the buses 20-1 to 20-3 are remotely monitored by the supervisor A1. 1 indicates that the monitoring apparatus 10 acquires information collected on the bus 20 (hereinafter also referred to as “target information”).

  From here, a monitoring process in the case where an abnormal sound indicating a failure sign occurs in one bus 20 will be described with reference to FIG. As described above, at the time corresponding to the map information MP11-1 in FIG. 1, the buses 20-1 to 20-3 are remotely monitored by the monitor A1. In addition, at the time corresponding to the map information MP11-1 in FIG. 1, the buses 20-4 and 20-5 are remotely monitored by the supervisor A2. That is, in the example of FIG. 1, at the time before the monitoring process by the determination apparatus 100, the monitor A1 remotely monitors the three buses 20-1 to 20-3, and the monitor A2 has the two buses 20- 4, 20-5 are monitored remotely. In the example of FIG. 1, as an example, a case where the sound information detected on the bus 20 includes a different sound (hereinafter also referred to as “abnormal sound”) as a predetermined condition that requires confirmation of the operation status of the bus 20. explain. The abnormal sound includes various sounds as long as the sound suggests an unusual situation on the bus 20. For example, in addition to abnormal noise that indicates a malfunction of the engine, abnormal noise occurs outside the moving body, such as sound that indicates sudden braking of surrounding vehicles, sound that indicates warnings such as horns, sound that indicates collision, etc. A sound may be included that identifies what has been done. Moreover, the sound which identifies that abnormality generate | occur | produced inside the moving body, such as a passenger's scream and a hoarse voice, may be contained. Note that audio information that satisfies a predetermined condition may be described as “target audio information”.

  First, the monitoring device 10-1 used by the monitor A1 acquires target information on the bus 20-1 (step S11-1). In the example of FIG. 1, the monitoring apparatus 10-1 acquires various information such as a moving image and sound detected on the bus 20-1 shown in the map information MP11-1 as target information of the bus 20-1. Although not shown, the monitoring device 10-1 displays a moving image of the bus 20-1 on a screen (corresponding to the output unit 14 in FIG. 3; the same applies hereinafter), or a voice (in FIG. 3). Or the like, and so forth, etc.), the monitor A1 can remotely monitor the bus 20-1. For example, immediately after acquisition, the monitoring device 10-1 displays the moving image of the bus 20-1 on the screen or outputs sound through a speaker or the like, so that the supervisor A1 makes the bus 20-1 in real time. Enables remote monitoring.

  In addition, the monitoring device 10-1 used by the monitor A1 acquires target information of the bus 20-2 (Step S11-2). In the example of FIG. 1, the monitoring apparatus 10-1 acquires various information such as moving images and sounds detected on the bus 20-2 indicated by the map information MP11-1 as the target information of the bus 20-2. Although not shown in the figure, the monitoring device 10-1 displays the moving image of the bus 20-2 on the screen or outputs sound through a speaker or the like, so that the supervisor A1 uses the bus 20-2. Enables remote monitoring. For example, immediately after the acquisition, the monitoring device 10-1 displays the moving image of the bus 20-2 on the screen or outputs sound through a speaker or the like so that the supervisor A1 can display the bus 20-2 in real time. Enables remote monitoring.

  In addition, the monitoring device 10-1 used by the supervisor A1 acquires the target information of the bus 20-3 (Step S11-3). In the example of FIG. 1, the monitoring apparatus 10-1 acquires various information such as moving images and sounds detected on the bus 20-3 shown in the map information MP11-1 as target information of the bus 20-3. Although not shown, the monitoring device 10-1 displays the moving image of the bus 20-3 on the screen or outputs sound through a speaker or the like, so that the supervisor A1 uses the bus 20-3. Enables remote monitoring. For example, immediately after acquisition, the monitoring device 10-1 displays the moving image of the bus 20-3 on the screen or outputs sound through a speaker or the like, so that the supervisor A1 can display the bus 20-3 in real time. Enables remote monitoring.

  In addition, the monitoring device 10-2 used by the supervisor A2 acquires the target information of the bus 20-4 (Step S11-4). In the example of FIG. 1, the monitoring apparatus 10-2 acquires various information such as moving images and sounds detected on the bus 20-4 indicated by the map information MP11-1 as the target information of the bus 20-4. Although not shown, the monitoring device 10-2 displays the moving image of the bus 20-4 on the screen or outputs sound through a speaker or the like, so that the supervisor A2 uses the bus 20-4. Enables remote monitoring. For example, immediately after acquisition, the monitoring device 10-2 displays the moving image of the bus 20-4 on the screen or outputs sound through a speaker or the like, so that the supervisor A2 makes the bus 20-4 in real time. Enables remote monitoring.

  In addition, the monitoring device 10-2 used by the supervisor A2 acquires the target information of the bus 20-5 (Step S11-5). In the example of FIG. 1, the monitoring device 10-2 acquires various information such as a moving image and sound detected on the bus 20-5 indicated by the map information MP11-1 as target information on the bus 20-5. Although not shown in the drawing, the monitoring device 10-2 displays the moving image of the bus 20-5 on the screen or outputs sound through a speaker or the like, so that the supervisor A2 uses the bus 20-5. Enables remote monitoring. For example, immediately after acquisition, the monitoring device 10-2 displays the moving image of the bus 20-5 on the screen or outputs sound through a speaker or the like so that the supervisor A2 can display the bus 20-5 in real time. Enables remote monitoring.

  Hereinafter, when it demonstrates without distinguishing step S11-1-S11-5, it will generically call step S11. Steps S11-1 to S11-5 are for showing the correspondence between the bus 20 and the monitoring device 10, and the monitoring device 10 sets the target information of the bus 20 to be monitored a plurality of times. You may acquire over.

  Note that the monitoring device 10 may acquire target information from the bus 20 that is a monitoring target, or may acquire target information from the bus 20 via an external device such as the determination device 100. For example, the determination apparatus 100 may acquire target information such as a moving image or sound from the bus 20 and transmit the target information to the monitoring apparatus 10 used by a monitor who monitors the bus 20. For example, the determination apparatus 100 may acquire target information such as a moving image and sound from the bus 20 and store the target information in the mobile body information storage unit 121 (see FIG. 6).

  In addition, the monitor A1 remotely monitors the bus 20 that is the monitoring target based on the target information acquired in step S11 (step S12-1). In the example of FIG. 1, the supervisor A1 remotely monitors the three buses 20-1 to 20-3 by the monitoring device 10-1. The monitoring person A1 sees the moving images of the buses 20-1 to 20-3 displayed on the screen by the monitoring device 10-1, or the buses 20-1 to 20-3 output by the monitoring device 10-1 through a speaker or the like. By listening to the voice, the three buses 20-1 to 20-3 are remotely monitored in real time.

  In the example of FIG. 1, the monitoring device 10-1 displays a list of moving images MV11 corresponding to the bus 20-1, moving images MV12 corresponding to the bus 20-2, and moving images MV13 corresponding to the bus 20-3. The For example, the moving image MV11 includes a moving image MV11-1 captured in front of the bus 20-1, that is, a traveling direction during normal traveling, a moving image MV11-2 captured in the rear of the bus 20-1, and the like. .

  Further, for example, the moving image MV12 includes a moving image MV12-1 captured in front of the bus 20-2, a moving image MV12-2 captured in the rear of the bus 20-2, and the like. Although not shown in the example of FIG. 1, the moving image MV13 includes a moving image obtained by imaging the front of the bus 20-3, a moving image obtained by imaging the rear of the bus 20-3, and the like.

  In the example of FIG. 1, for the sake of simplicity, the case where moving images corresponding to two locations on the front and rear of the bus 20 are displayed is shown. However, the moving images on the bus 20 such as the moving images MV11 to MV15 are shown. In addition to the front and rear, moving images of various places may be included. For example, the moving image on the bus 20 may include a moving image obtained by imaging the side of the bus 20, that is, the left and right sides, and a moving image obtained by imaging the inside of the bus 20. Moreover, the monitoring apparatus 10 may display the moving image corresponding to each bus | bath 20 so that a supervisor's desired location can be switched. The monitoring device 10 may display a moving image corresponding to the front and the inside of the bus 20 when the monitoring person desires a moving image corresponding to the front and the inside of the bus 20.

  Further, the monitor A2 remotely monitors the bus 20 that is the monitoring target based on the target information acquired in step S11 (step S12-2). In the example of FIG. 1, the supervisor A2 remotely monitors two buses 20-4 and 20-5 by the monitoring device 10-2. The monitoring person A2 sees the moving images of the buses 20-4 and 20-5 displayed on the screen by the monitoring device 10-2, or the buses 20-4 and 20-5 output by the monitoring device 10-2 through a speaker or the like. By listening to voice, the two buses 20-4 and 20-5 are remotely monitored in real time. In the example of FIG. 1, the monitoring device 10-2 displays a list of moving images MV14 corresponding to the bus 20-4 and moving images MV15 corresponding to the bus 20-5.

  Although not shown in the example of FIG. 1, the moving image MV14 includes a moving image captured in front of the bus 20-4, a moving image captured behind the bus 20-4, and the like. Includes a moving image captured in front of the bus 20-5, a moving image captured in the rear of the bus 20-5, and the like.

  Hereinafter, when step S12-1 and step S12-2 are described without distinction, they are collectively referred to as step S12. Steps S12-1 and S12-2 are for indicating that the monitor A1 and the monitor A2 are remotely monitoring the bus 20 by using the monitoring device 10, and the monitor A1 and the monitor A1. A2 continuously monitors the bus 20 to be monitored over a predetermined period (for example, during business hours).

  Here, a predetermined event occurs in the bus 20-1 (step S13). In the example of FIG. 1, an abnormal noise indicating the possibility of failure is generated from the bus 20-1. And since the abnormal sound is contained in the audio | voice information acquired from the bus | bath 20-1, the monitoring apparatus 10-1 determines the audio | voice information of the bus | bath 20-1 as object audio | voice information. Note that the determination device 100 may determine whether or not the sound information acquired from the bus 20-1 includes an abnormal sound. In this case, the monitoring device 10 may acquire information indicating the correspondence between the target audio information and the image information from the determination device 100. For example, the monitoring device 10 and the determination device 100 may determine whether or not the sound information includes abnormal sound by comparing the sound information acquired from the bus 20-1 and the sound pattern during normal operation. . In addition, for example, the monitoring device 10 or the determination device 100 compares the audio information acquired from the bus 20-1 with the list information of the sounds determined to be abnormal sounds, so that the audio information includes abnormal sounds. You may determine whether. For example, a normal model is learned from feature values obtained from normal sound data (learning data) inside and outside the mobile body, and the feature information of speech information acquired from the bus 20-1 is separated from the normal model. Normality and abnormality may be determined. Note that the above is an example, and any information can be used as long as the monitoring device 10 and the determination device 100 can determine whether or not the sound information acquired from the bus 20-1 includes sound that needs to be confirmed. It may be determined whether or not the sound information includes abnormal sounds.

  In the example of FIG. 1, since abnormal noise has occurred in the bus 20-1, the monitoring apparatus 10-1 can display that the image information corresponding to the bus 20-1 can be identified as corresponding to the target audio information. The mode is changed (step S13-1). In other words, the monitoring device 10-1 assumes that the audio information acquired from the bus 20-1 is the target audio information that requires confirmation of the operation status, and sends the target audio to the monitor A1 who is remotely monitoring the bus 20-1. The display mode is changed so that it is possible to grasp that the information is voice information acquired from the bus 20-1.

  For example, the monitoring apparatus 10-1 changes the color of the moving image MV11 corresponding to the bus 20-1 to a color different from that of the other moving images MV12 and MV13, so that the source of the target audio information is the bus 20- A plurality of moving images MV11 to MV13 are displayed in a display mode in which it can be identified as 1. For example, the monitoring device 10-1 changes the frame color of the moving image MV11 corresponding to the bus 20-1 to a color that draws attention, such as red or yellow, so that the source of the target audio information is the bus 20-1. A plurality of moving images MV11 to MV13 are displayed in a display mode in which it can be identified. Further, along with the change of the display mode, the monitoring device 10-1 outputs sound including abnormal noise generated in the bus 20-1 (step S13-2).

  As described above, when a certain bus 20 satisfies a predetermined condition that requires confirmation of the operation status, the monitoring device 10 changes the moving image of the bus 20 to a display mode different from the moving image of the other bus 20. Therefore, it is possible to appropriately grasp the correspondence between the image and the sound related to the mobile body to be remotely monitored. In addition, when there are a plurality of target audio information satisfying a predetermined condition, the monitoring device 10 has an image corresponding to each of the plurality of target audio information in a display mode that varies according to each level of the plurality of target audio information. Information may be displayed. For example, the monitoring device 10 may display image information corresponding to each of the plurality of target sound information in a manner like a heat map in order from the higher level among the plurality of target sound information. For example, the monitoring apparatus 10 may display image information corresponding to each of the plurality of target sound information in different colors such as red, yellow, and blue in order from the highest level among the plurality of target sound information. Good. For example, the monitoring device 10 displays image information corresponding to each of the target audio information exceeding the threshold in different colors such as red, yellow, and blue in order from the larger threshold based on a plurality of thresholds having different sizes. May be.

(1-1. System configuration)
As described above, the association between the target audio information and the image information may be performed in any configuration in the monitoring system 1. For example, the monitoring device 10 itself may associate the target audio information with the image information. Further, the determination device 100 and the bus 20 may associate the target audio information with the image information. In this case, the monitoring device 10 may acquire information indicating the correspondence between the target audio information and the image information from the determination device 100 or the bus 20. As described above, the monitoring system 1 can display the image information (moving image) corresponding to the target audio information in a display mode that can identify that the monitoring device 10 corresponds to the target audio information together with the output of the target audio information. Any configuration may be used.

(1-2. Target of association)
In the example of FIG. 1, the case where the bus 20 in which the target audio information is detected among the plurality of buses 20 is displayed so as to be identifiable is shown. It may be displayed in an identifiable manner at which part of the bus 20 in which is detected. In this case, the bus 20 may have a plurality of microphones corresponding to each location that is desired to be identified as a location where the target audio information is detected. Further, the location where the target audio information is detected may be specified by estimating the direction of the audio detected by the microphone provided on the bus 20 using various conventional techniques as appropriate.

  For example, when microphones are installed in the front, rear, inside, and engine portions of the bus 20, the monitoring device 10 determines whether the microphone detects the target audio information in front, rear, inside, and the bus 20. Which part of the engine portion is detected may be displayed so as to be identifiable. For example, the monitoring device 10 may display a moving image corresponding to each of the front, rear, inside, and engine portion of the bus 20, and may display in which location the target audio information is detected. For example, the monitoring device 10 may display a 20 bird's-eye view of the bus and display a location where an abnormal sound is generated.

  For example, when the target audio information is detected behind the bus 20, the monitoring apparatus 10 displays a moving image (for example, the moving image MV11-2) corresponding to the rear of the bus 20 in a mode different from other moving images. Thus, it may be displayed in an identifiable manner whether the target audio information is detected. Further, for example, when the target audio information is detected in the engine portion of the bus 20, the monitoring device 10 displays the moving image corresponding to the engine portion of the bus 20 in a manner different from other moving images, thereby Whether the information is detected may be displayed in an identifiable manner. Further, for example, the monitoring device 10 may display the location of the bus 20 where the target voice information is detected so that it can be identified by character information or the like.

  The above points will be described with reference to FIG. FIG. 14 is a diagram illustrating an example of a monitoring process for one moving object. The example of FIG. 14 shows a case where four moving images and sounds in the front, rear, in-car, and engine room of the bus 20-50 are detected. For example, cameras and microphones are respectively installed at four locations on the bus 20-50 in the front, rear, in the car, and in the engine room, so that 4 in the front, rear, in the car, and in the engine room of the bus 20-50. The moving image and sound of the location are detected.

  In the example illustrated in FIG. 14, the monitoring device 10 displays a moving image MV50 corresponding to the bus 20-50. That is, the example of FIG. 14 shows a state in which a supervisor who uses the monitoring device 10 is monitoring the bus 20-50.

  For example, the moving image MV50 includes a moving image MV50-1 obtained by imaging the forward direction of the bus 20-50, that is, a traveling direction during normal traveling, a moving image MV50-2 obtained by imaging the rear of the bus 20-50, and the bus 20 For example, a moving image MV50-3 obtained by imaging the inside of the vehicle of −50, a moving image MV50-4 obtained by imaging the inside of the engine room of the bus 20-50, and the like are included.

  Here, in the example of FIG. 14, a case where abnormal noise occurs in the engine room of the bus 20-50 is shown. In this case, the monitoring device 10 determines that the sound information in the engine room of the bus 20-50 is the target sound information because the sound information detected in the engine room of the bus 20-50 includes an abnormal sound. Therefore, the monitoring apparatus 10 displays the moving image MV50-4 obtained by imaging the inside of the engine room of the bus 20-50 in a display mode that can identify that it corresponds to the target audio information. That is, the monitoring apparatus 10 displays the moving image MV50-4 as another moving image MV50- so that the monitor can grasp that the sound information in the engine room of the bus 20-50 is sound information including abnormal sounds. 1 to MV50-3 are displayed in a different mode.

  In the example of FIG. 14, the monitoring device 10 adds the identification sign SN50 to the area where the moving image MV50-4 obtained by imaging the inside of the engine room of the bus 20-50 is displayed. The moving images MV50-1 to MV50-3 are displayed in a different manner. Specifically, the monitoring apparatus 10 displays the moving image MV50-4 by surrounding it with the identification sign SN50 including “Non-occurring noise”, thereby displaying the moving image MV50-4 with the other moving images MV50-1 to MV50-1. It is displayed in a mode different from MV50-3. In this way, the monitoring device 10 enables identification of the correspondence between the target audio information detected at a certain location on one bus 20 and the image at that location, thereby enabling the image and audio related to the mobile body to be remotely monitored. Can be properly grasped.

(1-3. Display mode)
In the example of FIG. 1, it is possible to grasp which bus 20 corresponds to the target audio information by making the color of the moving image corresponding to the target audio information different from other moving images. Depending on various display modes, it may be possible to grasp which bus 20 corresponds to the target audio information. For example, the monitoring device 10 displays any moving image corresponding to the target audio information larger than other moving images or blinks and displays the moving image corresponding to the target audio information. You may be able to grasp | ascertain whether it corresponds to audio | voice information.

  For example, the monitoring apparatus 10 may visualize and display the sound corresponding to each moving image. For example, the monitoring device 10 may make it possible to grasp which bus 20 corresponds to the target audio information by displaying the waveform of the audio corresponding to each moving image in association with the moving image. For example, when a loud sound is detected in a certain bus 20, among the sound waveforms displayed on the monitoring device 10, the sound waveform corresponding to the bus 20 changes greatly. Can correspond to the target voice information.

(1-4. Monitoring by the observer)
Moreover, the monitoring apparatus 10 may change aspects, such as an audio | voice output, based on the information which shows the area | region which the monitoring person is gazing at. For example, the monitoring apparatus 10 may acquire information indicating which area on the screen the monitor is gazing using, as appropriate, various conventional techniques such as eye tracking that detects the line of sight of the monitor.

  For example, the monitoring device 10 may also be referred to as image information (hereinafter referred to as “gaze image information”) that is estimated to be watched by the monitor according to the movement of the line of sight of the monitor among a plurality of pieces of image information (moving images). Audio information corresponding to the image information may be output in a different output mode from the audio information corresponding to other image information. For example, the monitoring device 10 may output only audio information corresponding to the gaze image information among the plurality of image information. For example, the monitoring device 10 may make the volume of the audio information corresponding to the gaze image information larger than the volume of the audio information corresponding to the other image information. For example, the monitoring apparatus 10 may output audio information corresponding to gaze image information at a volume larger than that of other audio information among a plurality of pieces of image information. Note that the above is an example, and the monitoring apparatus 10 may determine the output mode of audio information and image information using various information as appropriate.

(1-5. Audio output)
Further, the monitoring device 10 may cause the speaker 16 to output sound so that sound information can be heard from a direction corresponding to the display position of the screen. For example, the monitoring device 10 may output surround sound by having a plurality of speakers 16. For example, the monitoring device 10 may output sound corresponding to the image displayed on the right side of the screen from the speaker 16 located on the image side (right side). For example, the monitoring device 10 may output sound corresponding to the image displayed on the left side of the screen from the speaker 16 located on the image side (left side). For example, the monitoring device 10 may output sound corresponding to the image displayed in the central portion of the screen from the speaker 16 located in the central portion. For example, the above-described processing may be realized by disposing the speakers 16 in front of the monitoring device 10 or in the left-right direction. Note that the above is an example, and the monitoring apparatus 10 may output sound so that the image and the direction in which the sound is output can be associated by appropriately using various conventional techniques. For example, the monitoring apparatus 10 may implement the above processing using various conventional techniques even when a single speaker 16 or an earphone is used.

(1-6. Target focus)
Further, when the monitor designates the bus 20, the monitoring device 10 may output only the audio information and image information of the bus 20 designated by the monitor. For example, when the monitoring device 10 designates an area where a certain bus 20 is displayed, the monitoring device 10 may output only audio information and image information of the bus 20 designated by the monitoring device. In addition, the monitoring device 10 may request the determination device 100 to change the monitor of the bus 20 other than the bus 20 for which the monitor has designated the bus 20 to another monitor. In this case, the determination apparatus 100 may change the monitor of the bus 20 other than the bus 20 for which the monitor has designated the bus 20 to another monitor as required. In addition, when there are a plurality of pieces of target voice information, the monitoring apparatus 10 determines that the supervisor of the bus 20 corresponding to the predetermined target voice information among the plurality of pieces of target voice information is changed to another supervisor. You may request. For example, when there are two pieces of target voice information, the monitoring device 10 changes the monitor of the bus 20 corresponding to one of the two pieces of target voice information to another monitor. You may request | require to the determination apparatus 100. FIG.

[2. (Configuration of monitoring device)
Next, the configuration of the monitoring apparatus 10 according to the embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration example of the monitoring apparatus according to the embodiment. As illustrated in FIG. 3, the monitoring device 10 includes a communication unit 11, a storage unit 12, an input unit 13, an output unit 14, and a control unit 15. In addition, when performing eye tracking for detecting the observer's line of sight, the monitoring device 10 may include a sensor or the like for detecting the observer's line of sight although not shown.

(Communication unit 11)
The communication unit 11 is realized by, for example, a communication circuit or a NIC (Network Interface Card). The communication unit 11 is connected to the network N by wire or wirelessly and transmits / receives information to / from the determination device 100. Moreover, the communication part 11 may transmit / receive information between the bus | bath 20, the determination apparatus 100, etc. FIG.

(Storage unit 12)
The storage unit 12 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 12 stores, for example, information related to applications installed in the monitoring device 10, such as a program (monitoring program). As shown in FIG. 3, the storage unit 12 according to the embodiment includes a moving body information storage unit 12A.

(Moving object information storage unit 12A)
12 A of moving body information storage parts which concern on embodiment memorize | store various information regarding a moving body. For example, the mobile object information storage unit 12A stores various types of information related to a mobile object that is being remotely monitored by a monitor who uses the monitoring device 10. FIG. 4 is a diagram illustrating an example of the mobile object information storage unit according to the embodiment. The moving body information storage unit 12A shown in FIG. 4 includes items such as “bus ID”, “latest position”, “moving image”, “voice”, “operation route”, and “operation area”. A moving body information storage unit 12A illustrated in FIG. 4 indicates a moving body information storage unit corresponding to the monitor A1 in FIG.

  “Bus ID” indicates identification information for identifying a bus that is a mobile body to be remotely monitored. For example, the bus identified by the bus ID “20-1” corresponds to the bus 20-1 in FIG. The “latest position” indicates the latest position information of the bus identified by the bus ID. In the example illustrated in FIG. 4, “latest position” illustrates an abstract code such as “position LC11”, but may be information indicating latitude and longitude.

  The “moving image” indicates a moving image (also simply referred to as “moving image”) acquired from the bus. FIG. 4 shows an example in which conceptual information such as “moving image MV11” is stored in “moving image”, but actually, moving image information or a file path name indicating the storage location is stored. .

  “Voice” indicates voice information (also simply referred to as “voice”) acquired from the bus. FIG. 4 shows an example in which conceptual information such as “speech AD11” is stored in “speech”, but actually, speech information or a file path name indicating the storage location is stored.

  “Operating route” indicates the operating route of the bus identified by the bus ID. For example, “operation route” indicates a route on which the bus identified by the bus ID operates by automatic operation. In the example illustrated in FIG. 4, the “operating route” is illustrated with an abstract code such as “route R11”, but may be information indicating the latitude and longitude of a road or a bus stop that stops. The “operation area” indicates a bus operation area identified by the bus ID. For example, the “operation area” indicates an area where the bus identified by the bus ID operates by automatic driving. In the example shown in FIG. 4, “operating area” indicates an abstract code such as “area A”, but indicates a specific area name such as “XX city” or “XX ward”. It may be information.

  In the example of FIG. 4, the latest position of the bus identified by the bus ID “20-1” is “position LC11”. The moving image of the bus identified by the bus ID “20-1” is “moving image MV11”. The bus voice identified by the bus ID “20-1” is “voice AD11”. In addition, the bus operation route identified by the bus ID “20-1” is “route R11”. The bus operation area identified by the bus ID “20-1” is “area A”.

  As described above, the moving body information storage unit 12A illustrated in FIG. 4 is the moving body of the monitoring device 10-1 in a state where the monitor A1 in FIG. 1 is monitoring the three buses 20-1 to 20-3. An information storage unit 12A is shown. That is, the mobile body information storage unit 12A shown in FIG. 4 stores mobile body information relating to the three buses 20 of the buses 20-1 to 20-3 monitored by the supervisor A1 in FIG. The mobile body information storage unit 12A is not limited to the above, and may store various information according to the purpose.

  In addition, for example, in the terminal device 10-2 used by the supervisor A2 in FIG. 1, the moving body information storage unit 12A has the buses 20-4 and 20-5 monitored by the supervisor A2 in FIG. The moving body information regarding the two buses 20 is stored.

  Moreover, the memory | storage part 12 may memorize | store various information besides the above. For example, the storage unit 12 may store information stored in the storage unit 120 of the determination apparatus 100. For example, the storage unit 12 may store information shown in the supervisor information storage unit 122 (see FIG. 7), the monitoring correspondence information storage unit 123 (see FIG. 8), and the like. For example, the storage unit 12 may store information corresponding to the bus 20 on which the monitoring device 10 is arranged. For example, in the case of the monitoring device 10-1 arranged on the bus 20-1, the storage unit 12 of the monitoring device 10-1 corresponds to the bus 20-1 among the information stored in the storage unit 120 of the determination device 100. Information to be stored may be stored.

  For example, the storage unit 12 may store various information displayed by a notification unit 154 described later. For example, the storage unit 12 may store specific information, explicit information, and the like. For example, the storage unit 12 may store information on a face photograph of a monitor who can be a monitor of the bus 20 on which the monitoring device 10 is arranged, a business history, and the like.

(Input unit 13)
The input unit 13 acquires various operations from the user. For example, the input unit 13 may accept various operations from the user via a display surface (for example, the output unit 14) by a touch panel function. Further, the input unit 13 may accept various operations from buttons provided on the monitoring device 10 or a keyboard or mouse connected to the monitoring device 10.

(Output unit 14)
The output unit 14 is a display screen such as a smartphone or a tablet terminal realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display unit for displaying various information. For example, the output unit 14 outputs (displays) various information according to instructions from the notification unit 154.

(Control unit 15)
The control unit 15 is a controller, and for example, various programs stored in a storage device such as the storage unit 12 in the monitoring device 10 by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. This is realized by executing the RAM as a work area. For example, the various programs include installed application programs. The control unit 15 is a controller, and is realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  As illustrated in FIG. 3, the control unit 15 includes a transmission unit 151, an acquisition unit 152, and a notification unit 154, and realizes or executes the monitoring processing functions and operations described below. Note that the internal configuration of the control unit 15 is not limited to the configuration illustrated in FIG. 3, and may be another configuration as long as it performs a monitoring process described later. Further, the connection relationship between the processing units included in the control unit 15 is not limited to the connection relationship illustrated in FIG. 3, and may be another connection relationship.

(Transmitter 151)
The transmission unit 151 transmits various information. For example, the transmission unit 151 transmits various types of information to an external information processing apparatus. For example, the transmission unit 151 transmits various requests to the bus 20 and the determination device 100 according to the operation acquired by the input unit 13. For example, the transmission unit 151 may request information regarding the response monitor from the determination apparatus 100 at a predetermined timing.

(Acquisition unit 152)
The acquisition unit 152 acquires various types of information. For example, the acquisition unit 152 acquires various types of information from the storage unit 12. For example, the acquisition unit 152 acquires various types of information from an external information processing apparatus. The acquisition unit 152 receives various types of information from an external information processing apparatus. For example, the acquisition unit 152 acquires various types of information from the bus 20 and the determination device 100.

  For example, the acquisition unit 152 acquires a plurality of pieces of image information and a plurality of pieces of audio information detected by a moving body that is remotely monitored by a monitor and that operates by automatic driving. For example, the acquisition unit 152 acquires the plurality of image information and the plurality of audio information corresponding to each of the plurality of moving objects. For example, the acquisition unit 152 may acquire image information and audio information from each of a plurality of moving objects that are monitoring targets. For example, the acquisition unit 152 may acquire image information and audio information of a plurality of moving objects that are monitoring targets from the storage unit 12 and the determination device 100.

  For example, the acquisition unit 152 acquires the plurality of image information and the plurality of audio information corresponding to each location in one moving body. For example, the acquisition unit 152 may acquire image information and audio information corresponding to each location in one mobile body from one mobile body that is a monitoring target as needed. For example, the acquisition unit 152 may acquire image information and audio information corresponding to each location in one moving body that is a monitoring target from the storage unit 12 or the determination device 100. For example, the acquisition unit 152 may acquire information indicating the correspondence between the target audio information and the image information from the determination device 100.

(Determination unit 153)
The determination unit 153 determines various types of information. For example, the determination unit 153 determines various types of information based on the various types of information acquired from the external device by the acquisition unit 152. For example, the determination unit 153 determines various types of information based on the various types of information acquired from the determination device 100. For example, the determination unit 153 determines various types of information based on the various types of information acquired from the bus 20. For example, the determination unit 153 determines various types of information based on the information stored in the storage unit 12. For example, the determination unit 153 determines various types of information based on information stored in the mobile body information storage unit 12A or the like.

  For example, when the sound information acquired from the bus 20-1 includes abnormal sound, the determination unit 153 determines that the sound information on the bus 20-1 is the target sound information. For example, the determination unit 153 determines whether or not the sound information includes abnormal sound by comparing the sound information acquired from the bus 20-1 with the sound pattern during normal operation. Further, for example, the determination unit 153 determines whether or not the sound information includes abnormal sound by comparing the sound information acquired from the bus 20-1 and the list information of sound determined to be abnormal sound. .

(Notification unit 154)
The notification unit 154 notifies various information. For example, the notification unit 154 notifies various information via the output unit 14 and the speaker 16 (sound output unit). For example, the notification unit 154 displays various information via the output unit 14. For example, the notification unit 154 outputs audio information via the speaker 16. For example, the notification unit 154 outputs a plurality of audio information via the speaker 16. For example, the notification unit 154 displays various information on the output unit 14. For example, the notification unit 154 notifies various information acquired by the acquisition unit 152. For example, the notification unit 154 notifies various information stored in the storage unit 12. For example, the notification unit 154 notifies various information received from the bus 20 or the determination device 100. For example, the notification unit 154 notifies various information according to the determination by the determination unit 153. For example, the notification unit 154 notifies the monitor of target audio information that is audio information that satisfies a predetermined condition that requires confirmation of the operation status among a plurality of audio information. For example, the notification unit 154 displays a plurality of pieces of image information in a display mode that can identify that the audio information determined as the corresponding audio information by the determination unit 153 corresponds to the image information.

  For example, the notification unit 154 includes image information corresponding to target audio information, which is audio information satisfying a predetermined condition that requires confirmation of an operation state, among the plurality of audio information acquired by the acquisition unit 152, target audio information, A plurality of pieces of image information are displayed in a display mode in which it is possible to identify that they correspond.

  For example, the notification unit 154 displays image information corresponding to the target audio information and display information related to the target audio information in association with each other. For example, the notification unit 154 displays image information corresponding to the target audio information and display information obtained by visualizing the target audio information in association with each other. For example, the notification unit 154 makes the volume of the audio information corresponding to the gaze image information larger than the volume of the audio information corresponding to the other image information.

  For example, when there are a plurality of target audio information satisfying a predetermined condition, the notification unit 154 displays an image corresponding to each of the plurality of target audio information in a display mode that varies according to each level of the plurality of target audio information. Display information. For example, the notification unit 154 can identify that the target audio information corresponds to the mobile object corresponding to the target audio information satisfying a predetermined condition among the plurality of audio information acquired by the acquisition unit 152. To display multiple image information. For example, the notification unit 154 identifies that the target audio information corresponds to a location in one moving body corresponding to the target audio information satisfying a predetermined condition among the plurality of audio information acquired by the acquisition unit 152. A plurality of pieces of image information are displayed in a possible display mode.

  In the example of FIG. 1, the notification unit 154 changes the image information corresponding to the bus 20-1 to a display mode that can identify that it corresponds to the target audio information. For example, the notification unit 154 assumes that the voice information acquired from the bus 20-1 is target voice information that requires confirmation of the operation status, and the target voice information is sent to the supervisor A1 who remotely monitors the bus 20-1. The display mode is changed so that it can be grasped that the audio information is acquired from the bus 20-1.

  In the example of FIG. 1, the notification unit 154 changes the color of the moving image MV11 corresponding to the bus 20-1 to a color different from that of the other moving images MV12 and MV13, so that the generation source of the target audio information is the bus. The plurality of moving images MV11 to MV13 are displayed in a display mode that can identify that the image is 20-1. For example, the notification unit 154 changes the frame color of the moving image MV11 corresponding to the bus 20-1 to a color that draws attention, such as red or yellow, so that the source of the target audio information is the bus 20-1. A plurality of moving images MV11 to MV13 are displayed in a display mode in which this can be identified.

  Note that the processing such as the monitoring processing by the control unit 15 described above may be realized by, for example, JavaScript (registered trademark). Further, when the monitoring process described above is performed by a predetermined application or when the monitoring process is performed by a dedicated application, the control unit 15 may include, for example, an application control unit that controls the predetermined application or the dedicated application. .

(Speaker 16)
Further, the monitoring device 10 illustrated in FIG. 3 includes a speaker 16 that is a sound output unit that outputs audio information. For example, the speaker 16 outputs audio information detected by the moving body. For example, the speaker 16 outputs various audio information according to instructions from the notification unit 154. When the output unit 14 can also output audio information, the monitoring apparatus 10 does not have the speaker 16 and the output unit 14 may output audio information. The speaker 16 may not be integrated with the monitoring device 10, and may be a speaker, a headphone, or an earphone that is externally attached to the monitoring device 10. That is, the speaker 16 may be realized in any manner as long as it can output audio information in the moving body to the monitor.

  The speaker 16 outputs a plurality of audio information acquired by the acquisition unit 152. For example, the speaker 16 outputs a plurality of audio information in an output mode based on information related to the line of sight of the supervisor. For example, the speaker 16 outputs audio information corresponding to gaze image information, which is image information estimated to be watched by the monitor according to the movement of the monitor's line of sight, among the plurality of pieces of image information. The audio information corresponding to the information is output in a different output mode. For example, the speaker 16 outputs only audio information corresponding to gaze image information among a plurality of image information. In the example of FIG. 1, the speaker 16 outputs sound including abnormal noise generated on the bus 20-1.

[3. Configuration of decision device]
Next, the configuration of the determination apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 5 is a diagram illustrating a configuration example of the determination device according to the embodiment. As illustrated in FIG. 5, the determination apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The determination device 100 includes an input unit (for example, a keyboard and a mouse) that receives various operations from an administrator of the determination device 100 and an output (for example, a liquid crystal display) for displaying various information. Also good.

(Communication unit 110)
The communication unit 110 is realized by a communication circuit, a NIC, or the like, for example. The communication unit 110 is connected to the network N by wire or wireless, and transmits and receives information to and from the monitoring device 10 and the bus 20.

(Storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As illustrated in FIG. 5, the storage unit 120 according to the embodiment includes a moving body information storage unit 121, a supervisor information storage unit 122, and a monitoring correspondence information storage unit 123.

(Moving object information storage unit 121)
The mobile body information storage unit 121 according to the embodiment stores various types of information regarding the user. For example, the mobile body information storage unit 121 stores various types of information related to user attributes. FIG. 6 is a diagram illustrating an example of the mobile object information storage unit according to the embodiment. The mobile body information storage unit 121 illustrated in FIG. 6 includes items such as “bus ID”, “latest position”, “moving image”, “voice”, “operation route”, and “operation area”.

  “Bus ID” indicates identification information for identifying a bus that is a mobile body to be remotely monitored. For example, the bus identified by the bus ID “20-1” corresponds to the bus 20-1 in FIG. The “latest position” indicates the latest position information of the bus identified by the bus ID. In the example illustrated in FIG. 6, the “latest position” indicates an abstract code such as “position LC11”, but may be information indicating latitude and longitude.

  The “moving image” indicates a moving image (also simply referred to as “moving image”) acquired from the bus. Although FIG. 6 shows an example in which conceptual information such as “moving image MV11” is stored in “moving image”, actually, moving image information or a file path name indicating the storage location is stored. .

  “Voice” indicates voice information (also simply referred to as “voice”) acquired from the bus. FIG. 6 shows an example in which conceptual information such as “speech AD11” is stored in “speech”, but actually, speech information or a file path name indicating the storage location is stored.

  “Operating route” indicates the operating route of the bus identified by the bus ID. For example, “operation route” indicates a route on which the bus identified by the bus ID operates by automatic operation. In the example illustrated in FIG. 6, the “operating route” indicates an abstract code such as “route R11”, but may be information indicating the latitude and longitude of a road to pass or a bus stop to stop. The “operation area” indicates a bus operation area identified by the bus ID. For example, the “operation area” indicates an area where the bus identified by the bus ID operates by automatic driving. In the example shown in FIG. 6, “service area” indicates an abstract code such as “area A”, but indicates a specific area name such as “XX city” or “XX ward”. It may be information.

  In the example of FIG. 6, the latest position of the bus identified by the bus ID “20-1” is “position LC11”. The moving image of the bus identified by the bus ID “20-1” is “moving image MV11”. The bus voice identified by the bus ID “20-1” is “voice AD11”. In addition, the bus operation route identified by the bus ID “20-1” is “route R11”. The bus operation area identified by the bus ID “20-1” is “area A”.

  The mobile body information storage unit 121 is not limited to the above, and may store various types of information according to the purpose.

(Monitorer information storage unit 122)
The supervisor information storage unit 122 according to the embodiment stores various types of information related to the user context. FIG. 7 is a diagram illustrating an example of a supervisor information storage unit according to the embodiment. For example, the supervisor information storage unit 122 stores various information related to the supervisor. The monitor information storage unit 122 illustrated in FIG. 7 includes items such as “monitor ID”, “situation”, “working time information”, “working skill”, and “maximum number of monitored units”.

  The “monitorer ID” indicates identification information for identifying a monitor who remotely monitors a bus moving in automatic operation. For example, the supervisor identified by the supervisor ID “A1” corresponds to the supervisor A1 who uses the monitoring apparatus 10-1 in FIG. “Status” indicates the business status of the supervisor identified by the supervisor ID.

  The “business hours information” indicates the business hours of the monitor identified by the monitor ID. In the example shown in FIG. 7, “working time information” shows an abstract code such as “working time SF11”, but is information indicating a specific working time such as “weekdays from 9:00 to 17:00”. There may be.

  “Business skill” indicates a skill related to the business of the supervisor identified by the supervisor ID. FIG. 7 illustrates an abstract code such as “business skill SK11 (5 years of service)” and “year of service” in “business skill”. Etc. may be included.

  The “maximum number of monitored units” indicates the number of mobile units that can be remotely monitored simultaneously by the monitor identified by the monitor ID. Note that the maximum number of monitors of each monitor may be determined in accordance with work experience such as years of service of each monitor and individual abilities.

  In the example of FIG. 7, the status of the supervisor identified by the supervisor ID “A1” indicates “on duty”. Further, the work time information of the monitor identified by the monitor ID “A1” indicates “work time SF11”. Further, the business skill of the supervisor identified by the supervisor ID “A1” indicates “business skill SK11 (5 years of service)”. In addition, the maximum number of monitoring persons identified by the monitoring person ID “A1” is “5”.

  The supervisor information storage unit 122 is not limited to the above, and may store various types of information according to the purpose.

(Monitoring correspondence information storage unit 123)
The monitoring correspondence information storage unit 123 according to the embodiment stores various types of information related to the correspondence between the monitor and the bus that is the target of remote monitoring. FIG. 8 is a diagram illustrating an example of the monitoring correspondence information storage unit according to the embodiment. For example, the monitoring correspondence information storage unit 123 stores various types of information related to the correspondence between the monitor and the bus that is the object of remote monitoring, the state (status) of the bus, and the like. The monitoring correspondence information storage unit 123 illustrated in FIG. 8 includes items such as “monitorer ID” and “monitoring target”. The “monitoring target” includes items such as “bus ID” and “status”.

  The “monitorer ID” indicates identification information for identifying a monitor who remotely monitors a bus moving in automatic operation. The “monitoring target” includes various types of information on the bus to be monitored. “Bus ID” indicates identification information for identifying a bus that is a mobile body to be remotely monitored. “Status” indicates the state (status) of the bus identified by the bus ID.

  In the example of FIG. 8, the supervisor identified by the supervisor ID “A1” includes the bus identified by the bus ID “20-1”, the bus identified by the bus ID “20-2”, and the bus ID “ This indicates that the three buses identified by “20-3” are being remotely monitored. That is, the supervisor A1 indicates that the three buses 20-1 to 20-3 are being remotely monitored. The status of the bus identified by the bus ID “20-1” indicates “normal”. For example, the status “normal” indicates that the corresponding bus is operating as usual. The status of the bus identified by the bus ID “20-2” indicates “normal”. Further, the status of the bus identified by the bus ID “20-3” indicates “normal”.

  In the example of FIG. 8, the supervisor identified by the supervisor ID “A5” indicates that only the bus identified by the bus ID “20-15” is being remotely monitored. That is, the supervisor A5 indicates that one bus of the buses 20-15 is being remotely monitored. The status of the bus identified by the bus ID “20-15” indicates “event occurrence”. For example, the status “event occurrence” indicates that a predetermined event has occurred in the corresponding bus. That is, in the example of FIG. 8, the supervisor A5 indicates that a predetermined event such as a failure has occurred in the bus 20-15, so that only one bus 20-15 is being handled.

  Note that the monitoring correspondence information storage unit 123 is not limited to the above, and may store various types of information according to the purpose. For example, the status is not limited to “event occurrence”, and may include a plurality of types according to the event. For example, the status may be more detailed contents such as “approach to emergency vehicle”, “failure”, “occurrence of abnormal noise”, “presence of obstacle” corresponding to each event.

(Control unit 130)
Returning to the description of FIG. 5, the control unit 130 is a controller, for example, various programs (corresponding to an example of a determination program) stored in a storage device inside the determination device 100 by a CPU, an MPU, or the like. Is implemented by using the RAM as a work area. The control unit 130 is a controller, and is realized by an integrated circuit such as an ASIC or FPGA, for example.

  As illustrated in FIG. 5, the control unit 130 includes an acquisition unit 131, a determination unit 132, a determination unit 133, and a transmission unit 134, and realizes or executes information processing functions and operations described below. . Note that the internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 5, and may be another configuration as long as information processing described later is performed.

(Acquisition part 131)
The acquisition unit 131 acquires various types of information. For example, the acquisition unit 131 acquires various types of information from an external device. For example, the acquisition unit 131 acquires various types of information from the monitoring device 10. For example, the acquisition unit 131 acquires various information from the bus 20. For example, the acquisition unit 131 may acquire various types of information shown in the mobile body information storage unit 121, the supervisor information storage unit 122, the monitoring correspondence information storage unit 123, and the like from an external information processing apparatus.

  For example, the acquisition unit 131 acquires various types of information from the storage unit 120. For example, the acquisition unit 131 acquires various types of information from the mobile object information storage unit 121, the supervisor information storage unit 122, the monitoring correspondence information storage unit 123, and the like.

  For example, the acquisition unit 131 acquires information related to a single supervisor who remotely monitors a plurality of moving objects that operate by automatic driving. For example, the acquisition unit 131 acquires information related to one supervisor who remotely monitors a plurality of vehicles traveling on a road by automatic driving.

  For example, the acquisition unit 131 acquires information on a plurality of moving bodies that are remotely monitored by a monitor and that operate by automatic driving. For example, the acquisition unit 131 acquires information related to remote monitoring of a plurality of moving bodies that are remotely monitored by one monitor and that operate by automatic driving.

  For example, the acquisition unit 131 acquires target information of the bus 20. For example, the acquisition unit 131 acquires various information such as a moving image and sound detected on the bus 20. For example, the acquisition unit 131 acquires information indicating that a predetermined event has occurred in the bus 20-1 from the monitoring device 10-1.

  For example, the acquisition unit 131 acquires information indicating that it is difficult for the monitor A1 to monitor all the monitoring targets from the monitoring device 10-1. For example, the acquisition unit 131 acquires an image on the screen side of the monitoring device 10-1 captured by the monitoring device 10-1.

(Determination unit 132)
The determination unit 132 determines various types of information. For example, the determination unit 132 determines various types of information based on the various types of information acquired from the external device by the acquisition unit 131. For example, the determination unit 132 determines various types of information based on the various types of information acquired from the monitoring device 10. For example, the determination unit 132 determines various types of information based on the various types of information acquired from the bus 20. For example, the determination unit 132 determines various types of information based on information stored in the storage unit 120. For example, the determination unit 132 determines various types of information based on information stored in the mobile body information storage unit 121, the supervisor information storage unit 122, the monitoring correspondence information storage unit 123, and the like.

  For example, the determination unit 132 determines whether the monitoring status of one monitor satisfies a predetermined condition based on the information regarding the one monitor acquired by the acquisition unit 131. For example, the determination unit 132 determines whether it has become difficult for one monitor to monitor a plurality of moving objects. For example, the determination unit 132 determines whether or not the monitoring status of one monitor based on information regarding one monitor satisfies a predetermined condition.

  For example, the determination unit 132 determines whether or not a response by a single monitor to a predetermined moving body has occurred. For example, the determination unit 132 determines whether or not a response by a single monitor to a predetermined moving body included in the plurality of moving bodies has occurred.

  For example, the determination unit 132 determines whether the load of one monitor in monitoring a plurality of moving objects satisfies a predetermined standard. For example, the determination unit 132 determines whether or not the load on one monitor in monitoring a plurality of moving objects is greater than or equal to a predetermined threshold. For example, the determination unit 132 determines whether or not the load on one monitor in monitoring a plurality of moving objects is less than a predetermined threshold.

  For example, the determination unit 132 determines whether it has become difficult for one monitor to monitor a plurality of moving objects. For example, the determination unit 132 determines whether one monitoring person cannot visually recognize a monitoring device for monitoring a plurality of moving objects. For example, the determination unit 132 determines whether one monitor has left a seat where a plurality of moving objects are monitored.

  For example, the determination unit 132 determines whether or not the situation of the plurality of moving objects satisfies a predetermined condition based on the information on the plurality of moving objects acquired by the acquisition unit 131. For example, the determination unit 132 determines whether a predetermined event has occurred in one mobile body among the plurality of mobile bodies.

  For example, the determination unit 132 determines whether or not an event that prevents the movement of one moving body has occurred. For example, the determination unit 132 determines whether or not a failure has occurred in one moving body. For example, the determination unit 132 determines whether a failure has occurred in the traveling direction of one moving body.

  For example, the determination unit 132 determines whether it is time to change the monitor who monitors a plurality of moving objects based on the information related to remote monitoring acquired by the acquisition unit 131. For example, the determination unit 132 determines whether or not the state of one mobile body among the plurality of mobile bodies satisfies a predetermined condition.

  For example, the determination unit 132 determines whether or not the moving body satisfies a predetermined condition based on information on the moving body acquired from the monitoring device 10-1. For example, the determination unit 132 determines that a predetermined event has occurred in the bus 20-1 based on information indicating that a predetermined event has occurred in the bus 20-1 acquired from the monitoring device 10-1. May be. For example, the determination unit 132 determines that the bus 20-1 satisfies a predetermined condition based on information indicating that a predetermined event has occurred in the bus 20-1 acquired from the monitoring device 10-1. May be.

  For example, the determination unit 132 determines that the monitor A1 has left the monitoring device 10-1 based on the image on the screen side of the monitoring device 10-1 captured by the monitoring device 10-1. For example, the determination unit 132 determines that the monitor A1 has left the monitoring device 10-1 by analyzing an image acquired from the monitoring device 10-1. Further, for example, the determination unit 132 determines that the monitor A1 has moved away from the monitoring device 10-1 based on the position information of the monitor A1. Further, for example, the determination unit 132 may determine that monitoring of all the monitoring targets has become difficult by determining that the monitor A1 has moved away from the monitoring device 10-1. For example, as long as the determination unit 132 can grasp the positional relationship between the monitoring person A1 and the monitoring device 10-1, the determination unit 132 may determine what kind of information makes monitoring difficult by the monitoring person A1. Good.

  For example, when the sound information acquired from the bus 20-1 includes abnormal sound, the determination unit 132 determines the sound information on the bus 20-1 as the target sound information. For example, the determination unit 132 may determine whether or not the sound information includes abnormal sound by comparing the sound information acquired from the bus 20-1 and the sound pattern during normal operation. Further, for example, the determination unit 132 determines whether or not the sound information includes abnormal sound by comparing the sound information acquired from the bus 20-1 and the sound list information determined to be abnormal sound. May be.

(Determining unit 133)
The determination unit 133 determines various types of information. For example, the determination unit 133 determines various types of information based on the various types of information acquired from the external device by the acquisition unit 131. For example, the determination unit 133 determines various types of information based on the various types of information acquired from the monitoring device 10. For example, the determination unit 133 determines various types of information based on various types of information acquired from the bus 20. For example, the determination unit 133 determines various types of information based on the information stored in the storage unit 120. For example, the determination unit 133 determines various types of information based on information stored in the mobile body information storage unit 121, the supervisor information storage unit 122, the monitoring correspondence information storage unit 123, and the like. The determination unit 133 determines various types of information based on the determination by the determination unit 132.

  For example, the determination unit 133 may automatically operate a plurality of moving objects based on information about one monitor acquired by the acquisition unit 131 depending on whether the monitoring status of the one monitor satisfies a predetermined condition. The aspect regarding is determined. For example, when it becomes difficult for one monitor to monitor a plurality of moving bodies, the determining unit 133 determines to change the aspect related to the automatic driving of some of the moving bodies. To do. For example, the determination unit 133 determines an aspect related to the automatic driving of the plurality of moving bodies according to the determination as to whether or not the monitoring situation of one monitoring person satisfies a predetermined condition based on information related to one monitoring person.

  For example, the determination unit 133 determines to change the aspect related to the automatic operation of some of the plurality of moving bodies when the response by a single monitor to a predetermined moving body occurs. For example, the determination unit 133 changes the aspect related to the automatic operation of some of the plurality of moving bodies when a response to a predetermined moving body included in the plurality of moving bodies occurs. Then decide.

  For example, the determination unit 133 determines to change the aspect related to the automatic operation of some of the moving bodies so that the load on one monitoring person in the monitoring of the plurality of moving bodies satisfies a predetermined standard. For example, the determination unit 133 determines to change the aspect related to the automatic driving of some of the moving bodies so that the load of one monitoring person in monitoring a plurality of moving bodies is averaged with other monitoring persons. For example, the determination unit 133 determines to change the aspect related to the automatic operation of some of the mobile objects when the load on one monitor in monitoring a plurality of mobile objects is equal to or greater than a predetermined threshold. For example, the determination unit 133 determines to change the aspect related to the automatic driving of some of the moving bodies so that the load on one monitoring person in monitoring a plurality of moving bodies is less than a predetermined threshold.

  For example, the determination unit 133 changes the monitor of a mobile body other than some mobile bodies from one monitor to another monitor, or stops the operation of mobile bodies other than some mobile bodies. And decide. For example, when it becomes difficult for one monitor to monitor a plurality of moving bodies, the determining unit 133 determines to change the aspect related to the automatic driving of the plurality of moving bodies.

  For example, when one monitoring person cannot visually recognize a monitoring device for monitoring a plurality of moving objects, the determining unit 133 changes the monitoring persons of the plurality of moving objects from one monitoring person to another monitoring person. It is determined to change to a person or to stop the operation of a plurality of moving objects. For example, when one monitor moves away from a seat where a plurality of moving objects are monitored, the determination unit 133 changes the monitoring persons of the plurality of moving objects from one monitoring person to another monitoring person, or It is determined that the operations of the plurality of moving objects are stopped.

  For example, the determination unit 133 determines an aspect related to the automatic driving of the plurality of moving bodies when the situation of the plurality of moving bodies satisfies a predetermined condition based on the information about the plurality of moving bodies acquired by the acquisition unit 131. To do. For example, when a predetermined event occurs in one moving body among the plurality of moving bodies, the determining unit 133 determines to change the aspect related to the automatic operation of the one moving body among the plurality of moving bodies.

  For example, when an event that prevents the movement of one moving body occurs, the determination unit 133 determines to change the aspect related to the automatic operation of the one moving body among the plurality of moving bodies. For example, when a problem occurs in one moving body, the determining unit 133 determines to change the aspect related to the automatic operation of the one moving body among the plurality of moving bodies. For example, when a failure occurs in the traveling direction of one moving body, the determining unit 133 determines to change the aspect related to the automatic operation of the one moving body among the plurality of moving bodies.

  For example, the determination unit 133 determines to change the aspect related to the automatic operation of a moving body other than one moving body among the plurality of moving bodies. For example, the determination unit 133 determines to change the monitor of a mobile body other than one mobile body to another monitor or to stop the operation of a mobile body other than the one mobile body. For example, the determination part 133 determines to change the aspect regarding the automatic driving | operation of one moving body. For example, the determination unit 133 determines to change the aspect related to the automatic operation of one moving body or to stop the operation of one moving body.

  For example, the determination unit 133 determines the timing for changing the monitor who monitors a plurality of moving objects based on the information regarding the remote monitoring acquired by the acquisition unit 131. For example, the determination unit 133 determines to change the monitor who monitors a plurality of moving bodies based on the timing at which the situation of one moving body among the plurality of moving bodies satisfies a predetermined condition.

  For example, the determination unit 133 determines to change the monitor who monitors a plurality of moving bodies based on the timing when one moving body stops. For example, the determination unit 133 determines to change the supervisor who monitors a plurality of moving bodies at the timing when one moving body stops. For example, the determination unit 133 determines to change the monitoring person who monitors a plurality of moving objects at a timing when a predetermined time has elapsed since one moving object stopped.

  For example, the determination unit 133 determines to change the supervisor who monitors one moving body based on the timing. For example, the determination unit 133 determines to change a monitor who monitors a mobile body other than one mobile body among the plurality of mobile bodies based on the timing. For example, when one mobile unit is remotely monitored by a plurality of monitors including one monitor, the determination unit 133 determines to change the monitor who monitors the one mobile unit based on the timing.

  For example, when one mobile unit is remotely monitored by a plurality of monitors including one monitor, the determination unit 133 determines to change one monitor that monitors the one mobile unit based on the timing. . For example, when one mobile unit is remotely monitored by a plurality of monitors including a single monitor, the determination unit 133 selects one monitor from a plurality of monitors that monitor the single mobile unit based on timing. Is determined to be excluded. For example, when one mobile unit is remotely monitored by a plurality of monitors including one monitor, the determination unit 133 selects one of the plurality of monitors that monitor the one mobile unit based on the timing. Decide to change the watcher to another watcher.

  In the example of FIG. 9, the determination unit 133 selects other monitors of the bus 20 other than the bus 20-1 from among the monitoring targets of the monitor A 1 who remotely monitors the bus 20-1 in which the predetermined event has occurred. Decide to change to a supervisor. For example, the determination unit 133 determines to change the monitor of the buses 20-2 and 20-3 to the monitor A2 among the buses 20-1 to 20-3 monitored by the monitor A1. For example, the determination unit 133 determines to change the monitor of the bus 20 other than the bus 20-1 among the monitoring targets of the monitor A1 at the timing when the bus 20-1 determines that the predetermined condition is satisfied. Then, the determination unit 133 changes the monitor of the buses 20-2 and 20-3, which are the monitoring targets of the monitor A1, to the monitor A2. In the example of FIG. 9, as shown in map information MP11-2, the determination part 133 determines the aspect regarding the automatic driving | operation of a several mobile body by changing the monitoring object of each supervisor A1 and A2.

  In the example of FIG. 11, the determination unit 133 determines to change the monitor of all the buses 20 to be monitored by the monitor A1 who becomes difficult to monitor all the monitor targets to another monitor. For example, the determination unit 133 determines to change the monitor of the buses 20-1 to 20-3, which is the monitoring target of the monitor A1, to the monitor A2. For example, the determination unit 133 determines to change the monitor of the monitoring target bus 20 of the monitor A1 at the timing when the monitor A1 acquires information indicating that monitoring of all the monitoring targets is difficult. Then, the determination unit 133 changes the monitor of the buses 20-1 to 20-3, which is the monitoring target of the monitor A1, to the monitor A2. In the example of FIG. 11, as shown in map information MP11-2, the determination part 133 determines the aspect regarding the automatic driving | operation of a some mobile body by changing the monitoring object of each monitor A1 and A2.

(Transmitter 134)
The transmission unit 134 transmits various information to the external device. For example, the transmission unit 134 transmits various types of information to the monitoring device 10. For example, when the bus 20 and the monitoring device 10 directly communicate with each other, the transmission unit 134 may transmit information specifying a destination to which the target information is transmitted to the bus 20. For example, when the bus 20-1 is monitored by the supervisor A1 using the monitoring device 10-1, the transmission unit 134 sets the transmission destination so that the target information is transmitted to the monitoring device 10-1 on the bus 20-1. Information to be specified may be transmitted.

  For example, the transmission unit 134 may transmit target information regarding the bus 20 to be monitored to the monitoring device 10. For example, the transmission unit 134 may transmit various information such as a moving image and sound detected on the bus 20 to the monitoring device 10.

[4. (Change display)
The display change in the monitoring apparatus 10 will be described with reference to FIGS. 9 and 11 are diagrams illustrating an example of the monitoring process according to the embodiment. 10 and 12 are diagrams illustrating an example of display change according to the embodiment. The map information MP11-1 and MP11-2 shown in FIGS. 9 and 11 will be briefly described. The map information MP11-1 and MP11-2 shown in FIGS. 9 and 11 are diagrams schematically showing the positions of the buses 20-1 to 20-5, the relationship between the bus 20 and the supervisor, and the like. The map information MP11-1 shown in FIGS. 9 and 11 corresponds to the map information MP11-1 shown in FIG. The map information MP11-1 and map information MP11-2 shown in FIGS. 9 and 11 are for illustrating changes in the relationship between the bus 20 and the supervisor, that is, changes in the supervisor of the bus 20, and map information. The ranges (areas) shown in MP11-1 and map information MP11-2 are the same area. Further, when the map information MP11-1 and MP11-2 are described without particular distinction, they are described as map information MP11.

  First, an example of a display change in the monitoring apparatus 10-1 will be described with reference to FIG. Steps S11 to S13 in FIG. 9 are the same as steps S11 to S13 in FIG. In the example of FIG. 9, in step S13, an event indicating the possibility of failure such as abnormal noise occurs from the bus 20-1. In addition, the monitor A1 who remotely monitors the bus 20-1 by the outputs in steps S13-1 and S13-2 in FIG. 1 indicates that the target voice information including abnormal sounds output by the monitoring device 10-1 is the bus. It is assumed that the sound corresponds to the voice 20-1, and it is estimated that a predetermined event has occurred in the bus 20-1. And the supervisor A1 transmits the information which shows that the predetermined | prescribed event has generate | occur | produced to the bus | bath 20-1 to the determination apparatus 100 by operating the monitoring apparatus 10-1.

  For example, the determining apparatus 100 determines whether or not the moving body satisfies a predetermined condition based on information about the moving body acquired from the monitoring apparatus 10-1. For example, the determination device 100 determines that a predetermined event has occurred in the bus 20-1 based on information indicating that a predetermined event has occurred in the bus 20-1 acquired from the monitoring device 10-1. May be. For example, the determination apparatus 100 determines that the bus 20-1 satisfies a predetermined condition based on information indicating that a predetermined event has occurred in the bus 20-1 acquired from the monitoring apparatus 10-1. May be. The predetermined condition is not limited to whether or not a predetermined event has occurred in the bus 20, and may be various conditions.

  The determination device 100 that determines that the bus 20-1 satisfies the predetermined condition determines the monitoring mode of the plurality of moving objects (step S14). For example, the determination apparatus 100 determines an aspect related to automatic driving of a plurality of moving objects. For example, the determining apparatus 100 determines the timing for changing the supervisor who monitors a plurality of moving objects. In the example of FIG. 9, the determination apparatus 100 selects other supervisors of the buses 20 other than the bus 20-1 from among the monitoring targets of the supervisor A1 who remotely monitors the bus 20-1 in which the predetermined event has occurred. Decide to change to a supervisor. Specifically, the determination device 100 determines to change the monitor of the buses 20-2 and 20-3 to the monitor A2 among the buses 20-1 to 20-3 that are the monitoring targets of the monitor A1. For example, the determination apparatus 100 determines to change the monitor of the bus 20 other than the bus 20-1 among the monitoring targets of the monitor A1 at the timing when the bus 20-1 determines that the predetermined condition is satisfied. Then, the determination apparatus 100 changes the monitor of the buses 20-2 and 20-3, which is the monitoring target of the monitor A1, to the monitor A2. Note that the supervisor A2 is a supervisor who can monitor a plurality of buses 20 (see FIG. 8). Further, for example, when the determination apparatus 100 cannot change the supervisor to another supervisor, the determination apparatus 100 may stop the operation (running) of either the bus 20-1 or the buses 20-2 and 20-3. Good. For example, the determination apparatus 100 may determine to change the monitor of the bus 20-1 from the monitor A1 to another monitor (for example, a monitor whose monitoring target is 0). The “aspects relating to automatic driving” referred to here include not only the change of the supervisor and the stop of the moving body as described above, but also various aspects. For example, “an aspect related to automatic driving” may include reducing (decreasing) the speed of the moving object. Further, for example, the “aspect related to automatic driving” may include turning on (flashing) a predetermined lamp of the moving body, for example, issuing a hazard of the moving body. Further, for example, the “aspect related to automatic driving” may include outputting a predetermined sound from the moving body, for example, sounding a horn of the moving body.

  As shown in map information MP11-2 in FIG. 9, the determination apparatus 100 determines the aspect regarding the automatic driving | operation of a several mobile body by changing the monitoring object of each monitor A1 and A2. In the example of FIG. 9, at the time after the monitoring process by the determination apparatus 100, the monitor A1 remotely monitors one bus 20-1, and the monitor A2 monitors four buses 20-2 to 20-5. Remote monitoring.

  Here, as shown in FIG. 10, since the monitoring target of the supervisor A1 is changed to only the bus 20-1, the display of the monitoring device 10-1 is changed (step S14-1). Specifically, among the moving images MV11 to MV13 displayed on the monitoring device 10-1, only the moving image MV11 corresponding to the bus 20-1 remains and corresponds to the other buses 20-2 and 20-3, respectively. The moving images MV12 and MV13 to be displayed are excluded from the display. In this way, the display of the monitoring device 10-1 is changed.

  In this case, only the moving image MV11 corresponding to the bus 20-1 is displayed on the monitoring device 10-1. Thereby, the supervisor A1 can centrally monitor the bus 20-1. In addition, since the monitoring target of the supervisor A2 is changed to the buses 20-2 to 20-5, moving images MV12 and MV13 respectively corresponding to the buses 20-2 and 20-3 are added to the monitoring device 10-2. The moving images MV12 to MV15 respectively corresponding to the buses 20-2 to 20-5 are displayed, but the illustration is omitted.

  The monitoring device 10-1 used by the monitor A1 acquires the target information of the bus 20-1 (Step S15-1). In the example of FIG. 9, the monitoring device 10-1 acquires various information such as moving images and sounds detected on the bus 20-1 shown in the map information MP11-2 as target information of the bus 20-1.

  In addition, the monitoring device 10-2 used by the supervisor A2 acquires the target information of the bus 20-2 (Step S15-2). In the example of FIG. 9, the monitoring device 10-2 acquires various information such as a moving image and sound detected on the bus 20-2 shown in the map information MP11-2 as target information of the bus 20-2.

  In addition, the monitoring device 10-2 used by the monitor A2 acquires the target information of the bus 20-3 (Step S15-3). In the example of FIG. 9, the monitoring device 10-2 acquires various information such as a moving image and sound detected on the bus 20-3 indicated by the map information MP11-2 as target information of the bus 20-3.

  In addition, the monitoring device 10-2 used by the supervisor A2 acquires the target information of the bus 20-4 (Step S15-4). In the example of FIG. 9, the monitoring device 10-2 acquires various information such as moving images and sounds detected on the bus 20-4 indicated by the map information MP11-2 as the target information of the bus 20-4.

  In addition, the monitoring device 10-2 used by the monitor A2 acquires the target information of the bus 20-5 (Step S15-5). In the example of FIG. 9, the monitoring device 10-2 acquires various information such as moving images and sounds detected on the bus 20-5 shown in the map information MP11-2 as the target information of the bus 20-5.

  Hereinafter, when it demonstrates without distinguishing step S15-1-S15-5, it will be named step S15 generically. Steps S15-1 to S15-5 are for indicating the correspondence relationship between the bus 20 and the monitoring device 10, and the monitoring device 10 sets the target information of the bus 20 to be monitored a plurality of times. You may acquire over.

  In addition, the monitor A1 remotely monitors the bus 20 that is the monitoring target based on the target information acquired in step S15 (step S16-1). In the example of FIG. 9, the supervisor A1 remotely monitors only the bus 20-1 by the monitoring device 10-1. The monitoring person A1 views the moving image of the bus 20-1 displayed on the screen by the monitoring device 10-1 or listens to the sound of the bus 20-1 output by the monitoring device 10-1 through a speaker or the like. The bus 20-1 in which a predetermined event has occurred is remotely monitored in real time.

  In addition, the monitor A2 remotely monitors the bus 20 that is the monitoring target based on the target information acquired in step S15 (step S16-2). In the example of FIG. 9, the supervisor A2 remotely monitors the four buses 20-2 to 20-5 by the monitoring device 10-2. The monitoring person A2 sees the moving images of the buses 20-2 to 20-5 displayed on the screen by the monitoring device 10-2, or the buses 20-2 to 20-5 output by the monitoring device 10-2 through speakers or the like. By listening to voice, the four buses 20-2 to 20-5 are remotely monitored in real time.

  Hereinafter, when the steps S16-1 and S16-2 are described without distinction, they are collectively referred to as step S16. Steps S16-1 and S16-2 are for indicating that the monitoring person A1 or the monitoring person A2 remotely monitors the bus 20 using the monitoring device 10, and the monitoring person A1 or the monitoring person A1. A2 continuously monitors the bus 20 to be monitored over a predetermined period (for example, during business hours).

  Then, based on the target information acquired in step S15, the monitor A1 responds to an event that has occurred in the bus 20-1 that is the target of monitoring (step S17). For example, the supervisor A1 uses the bus 20-1 based on the moving image of the bus 20-1 displayed on the monitoring device 10-1, the sound of the bus 20-1 output by the monitoring device 10-1, and the like. Corresponds to a predetermined event that occurred. For example, the supervisor A1 remotely operates the bus 20-1 or stops the bus 20-1 at a predetermined position (for example, a road shoulder) according to the degree of influence of an event that has occurred on the bus 20-1. . As described above, the determination device 100 is remotely monitored by determining an aspect related to the automatic operation of a plurality of moving bodies based on information indicating that a predetermined event has occurred on the bus 20-1. Appropriate operation of a plurality of moving bodies can be enabled. In the example of FIG. 9, the case where the supervisor is changed when a predetermined event occurs in the bus 20 is shown. However, the determination device 100 relates to various automatic driving operations when a predetermined event occurs. Aspects may be determined. For example, when a predetermined event occurs in the bus 20, the determination device 100 stops the bus 20, decreases the speed of the bus 20, lights a predetermined lamp of the bus 20, or performs a predetermined event from the bus 20. You may determine the aspect regarding various automatic driving | operations, such as a sound output. Further, for example, when a predetermined event occurs on the bus 20, the determination apparatus 100 may confirm whether or not it is possible to deal with a monitor who monitors the bus 20. In this case, when the monitoring person who monitors the bus 20 replies that the monitoring is not possible, the determining apparatus 100 determines to change the monitoring of the bus 20 or another bus 20 monitored by the monitoring person to another monitoring person. May be. Moreover, the determination apparatus 100 may determine not to change the monitoring person when the monitoring person who monitors the bus 20 replies that it can respond. Further, for example, when a predetermined event occurs on the bus 20, the determination device 100 may determine to increase the number of monitoring personnel who monitor the bus 20. Further, for example, when a predetermined event occurs in the bus 20 monitored by one monitor, the determination device 100 determines to increase the number of monitors monitoring the bus 20 to two. May be. Note that the above is an example, and the determination device 100 may make various determinations when a predetermined event occurs in the bus 20 depending on the situation at that time.

  Next, an example of display change in the monitoring device 10-2 will be described with reference to FIG. As described above, at the time corresponding to the map information MP11-1 in FIG. 11, the buses 20-1 to 20-3 are remotely monitored by the monitor A1. Further, at the time corresponding to the map information MP11-1 in FIG. 11, the buses 20-4 and 20-5 are remotely monitored by the monitor A2. That is, in the example of FIG. 11, at the time before the monitoring process by the determination device 100, the monitor A1 remotely monitors the three buses 20-1 to 20-3, and the monitor A2 has the two buses 20- 4, 20-5 are monitored remotely.

  First, the monitoring device 10-1 used by the monitor A1 acquires target information on the bus 20-1 (step S21-1). In the example of FIG. 11, the monitoring device 10-1 acquires various information such as moving images and sounds detected on the bus 20-1 shown in the map information MP11-1 as the target information of the bus 20-1.

  In addition, the monitoring device 10-1 used by the monitor A1 acquires target information of the bus 20-2 (step S21-2). In the example of FIG. 11, the monitoring device 10-1 acquires various information such as moving images and sounds detected on the bus 20-2 shown in the map information MP11-1 as target information of the bus 20-2.

  In addition, the monitoring device 10-1 used by the supervisor A1 acquires target information of the bus 20-3 (step S21-3). In the example of FIG. 11, the monitoring device 10-1 acquires various information such as moving images and sounds detected on the bus 20-3 shown in the map information MP11-1 as the target information of the bus 20-3.

  In addition, the monitoring device 10-2 used by the supervisor A2 acquires the target information of the bus 20-4 (Step S21-4). In the example of FIG. 11, the monitoring device 10-2 acquires various information such as moving images and sounds detected on the bus 20-4 indicated by the map information MP11-1 as the target information of the bus 20-4.

  In addition, the monitoring device 10-2 used by the supervisor A2 acquires the target information of the bus 20-5 (Step S21-5). In the example of FIG. 11, the monitoring device 10-2 acquires various information such as moving images and sounds detected on the bus 20-5 shown in the map information MP11-1 as the target information of the bus 20-5.

  Hereinafter, when it demonstrates without distinguishing step S21-1-S21-5, it is named step S21 generically. Steps S21-1 to S21-5 are for indicating the correspondence between the bus 20 and the monitoring device 10, and the monitoring device 10 sets the target information of the bus 20 to be monitored a plurality of times. You may acquire over.

  In addition, the monitor A1 remotely monitors the bus 20 that is the monitoring target based on the target information acquired in step S21 (step S22-1). In the example of FIG. 11, the supervisor A1 remotely monitors three buses 20-1 to 20-3 by the monitoring device 10-1. The monitoring person A1 sees the moving images of the buses 20-1 to 20-3 displayed on the screen by the monitoring device 10-1, or the buses 20-1 to 20-3 output by the monitoring device 10-1 through a speaker or the like. By listening to the voice, the three buses 20-1 to 20-3 are remotely monitored in real time.

  In addition, the supervisor A2 remotely monitors the bus 20 that is the monitoring target based on the target information acquired in step S21 (step S22-2). In the example of FIG. 11, the supervisor A2 remotely monitors two buses 20-4 and 20-5 by the monitoring device 10-2. The monitoring person A2 sees the moving images of the buses 20-4 and 20-5 displayed on the screen by the monitoring device 10-2, or the buses 20-4 and 20-5 output by the monitoring device 10-2 through a speaker or the like. By listening to voice, the two buses 20-4 and 20-5 are remotely monitored in real time.

  Hereinafter, when it demonstrates without distinguishing step S22-1 and S22-2, it will be named step S22 generically. Steps S22-1 and S22-2 are for indicating that the monitor A1 or the monitor A2 is remotely monitoring the bus 20 by using the monitoring device 10, and the monitor A1 or the monitor A2 A2 continuously monitors the bus 20 to be monitored over a predetermined period (for example, during business hours).

  Here, it is assumed that it is difficult for the monitor A1 to remotely monitor the bus 20 (step S23). In the example of FIG. 11, it is assumed that it is difficult for the monitor A1 to monitor all the monitoring targets. For example, it is assumed that it is difficult to monitor all the monitoring targets due to a sudden change in the physical condition of the monitoring person A1 or the like due to leaving the monitoring apparatus 10-1. For example, it is assumed that it is difficult to monitor all the monitoring targets due to a sudden change in the physical condition of the monitoring person A1 or the like due to leaving the desk where the monitoring device 10-1 is placed. The monitoring person A1 may transmit information indicating that it becomes difficult for the monitoring person A1 to monitor all the monitoring targets by operating the monitoring apparatus 10-1.

  Further, the determination device 100 determines that the monitor A1 has moved away from the monitoring device 10-1 based on the image on the screen side of the monitoring device 10-1 captured by the monitoring device 10-1. It may be determined that monitoring has become difficult. In other words, the determination device 100 determines that the monitoring device 10 for monitoring a plurality of moving objects has become invisible. For example, the determination device 100 acquires the image on the screen side of the monitoring device 10-1 captured by the monitoring device 10-1 in real time and analyzes the image, so that the monitor A1 is away from the monitoring device 10-1. By determining, it may be determined that monitoring of all monitoring targets has become difficult. Further, for example, the determination apparatus 100 determines that monitoring of all the monitoring targets is difficult by determining that the monitor A1 is away from the monitoring apparatus 10-1 based on the position information of the monitor A1. Also good. It should be noted that the determination device 100 can determine what kind of information the monitoring by the supervisor A1 has become difficult if the positional relationship between the supervisor A1 and the monitoring device 10-1 can be grasped. Good. For example, the determination apparatus 100 grasps the positional relationship between the monitor A1 and the monitoring apparatus 10-1 based on information acquired from the RFID (Radio frequency identification) attached to the monitor A1 or the monitoring apparatus 10-1. May be.

  The determination apparatus 100 is not limited to whether or not the monitor is difficult to monitor, and various conditions regarding the monitoring status of the monitor may be set as the predetermined conditions. For example, the determination apparatus 100 may determine whether or not the monitoring level of the monitor is equal to or higher than a predetermined threshold as a predetermined condition regarding the monitoring status of the monitor.

  The determination apparatus 100 that has acquired the information indicating that it is difficult for the monitor A1 to monitor all the monitoring targets determines the monitoring mode of the plurality of moving objects (step S24). For example, the determination apparatus 100 determines an aspect related to automatic driving of a plurality of moving objects. For example, the determining apparatus 100 determines the timing for changing the supervisor who monitors a plurality of moving objects. In the example of FIG. 11, the determination apparatus 100 determines to change the monitor of all the buses 20 to be monitored by the monitor A1 who has become difficult to monitor all the monitor targets to another monitor. Specifically, the determination device 100 determines to change the monitor of the buses 20-1 to 20-3, which is the monitoring target of the monitor A1, to the monitor A2. For example, the determination device 100 determines to change the monitor of the bus 20 to be monitored by the monitor A1 at the timing when the monitor A1 acquires information indicating that monitoring of all the monitoring targets has become difficult. Then, the determination apparatus 100 changes the monitor of the buses 20-1 to 20-3, which is the monitoring target of the monitor A1, to the monitor A2. For example, the determination apparatus 100 may stop the operation (running) of the buses 20-1 to 20-3 when the monitoring person cannot be changed to another monitoring person.

  As shown in map information MP11-2 in FIG. 11, the determination apparatus 100 determines the aspect regarding the automatic driving | operation of a some mobile body by changing the monitoring object of each monitor A1 and A2. In the example of FIG. 11, at the time after the monitoring process by the determination apparatus 100, the supervisor A2 remotely monitors the five buses 20-1 to 20-5.

  Here, as shown in FIG. 12, since the monitoring target of the supervisor A2 is changed to the buses 20-1 to 20-5, the display of the monitoring device 10-2 is changed (step S24-1). Specifically, moving images MV11 to MV13 respectively corresponding to the buses 20-1 to 20-3 are added to the monitoring device 10-2, and moving images MV11 to MV11 corresponding to the buses 20-1 to 20-5 are respectively added. MV15 is displayed. In this way, the display of the monitoring device 10-2 is changed.

  In this case, moving images MV11 to MV15 corresponding to the five buses 20 of the buses 20-1 to 20-5 are displayed on the monitoring device 10-2. Thereby, the supervisor A2 can monitor the bus 20 including the buses 20-1 to 20-3 taken over from the supervisor A1. In addition, since the monitoring target of the monitoring person A1 is changed to 0, no moving image corresponding to any bus 20 is displayed on the monitoring device 10-1, but the illustration is omitted.

  In addition, the monitoring device 10-2 used by the monitor A2 acquires the target information of the bus 20-1 (Step S25-1). In the example of FIG. 11, the monitoring device 10-2 acquires various information such as a moving image and sound detected on the bus 20-1 shown in the map information MP11-2 as the target information of the bus 20-1.

  In addition, the monitoring device 10-2 used by the monitor A2 acquires target information of the bus 20-2 (Step S25-2). In the example of FIG. 11, the monitoring device 10-2 acquires various information such as moving images and sounds detected on the bus 20-2 shown in the map information MP11-2 as the target information of the bus 20-2.

  In addition, the monitoring device 10-2 used by the monitor A2 acquires the target information of the bus 20-3 (Step S25-3). In the example of FIG. 11, the monitoring device 10-2 acquires various information such as a moving image and sound detected on the bus 20-3 shown in the map information MP11-2 as target information of the bus 20-3.

  In addition, the monitoring device 10-2 used by the supervisor A2 acquires the target information of the bus 20-4 (Step S25-4). In the example of FIG. 11, the monitoring device 10-2 acquires various information such as moving images and sounds detected on the bus 20-4 indicated by the map information MP11-2 as target information of the bus 20-4.

  In addition, the monitoring device 10-2 used by the supervisor A2 acquires the target information of the bus 20-5 (Step S25-5). In the example of FIG. 11, the monitoring device 10-2 acquires various information such as moving images and sounds detected on the bus 20-5 indicated by the map information MP11-2 as the target information of the bus 20-5.

  Hereinafter, when the steps S25-1 to S25-5 are described without distinction, they are collectively referred to as step S25. Steps S25-1 to S25-5 are for showing the correspondence between the bus 20 and the monitoring device 10, and the monitoring device 10 sets the target information of the bus 20 to be monitored multiple times. You may acquire over.

  In addition, the supervisor A2 remotely monitors the bus 20 that is the monitoring target based on the target information acquired in step S25 (step S26). In the example of FIG. 11, the supervisor A2 remotely monitors five buses 20-1 to 20-5 by the monitoring device 10-2. The monitoring person A2 sees the moving images of the buses 20-1 to 20-5 displayed on the screen by the monitoring device 10-2, or the buses 20-1 to 20-5 output by the monitoring device 10-2 through speakers or the like. By listening to voice, five buses 20-1 to 20-5 are remotely monitored in real time.

  Step S26 is for indicating that the monitor A1 or the monitor A2 is remotely monitoring the bus 20 using the monitoring device 10, and the monitor A2 is the bus to be monitored. 20 is continuously monitored over a predetermined period (for example, during business hours). Further, when the monitor A1 returns to a state where monitoring is possible, the determination apparatus 100 may change the monitor on the buses 20-1 to 20-3 to the monitor A1. For example, when the monitor A1 returns to a state where monitoring is possible by returning to the front of the monitoring device 10-1, the determination device 100 changes the monitor of the buses 20-1 to 20-3 to the monitor A1. Also good.

(4-1. Monitoring mode)
In addition, about the monitoring aspect of the bus | bath 20 in the monitoring system 1, and a supervisor's change aspect, not only the example mentioned above but a various aspect may be sufficient. In the example described above, a case where one supervisor monitors each bus 20 has been described, but a mode in which a plurality of supervisors monitor one bus 20 may be used. For example, one bus 20 may be supervised by a plurality of supervisors. In this case, the plurality of supervisors who monitor one bus 20 may be of different types such as a primary supervisor (first supervisor) and a secondary supervisor (second supervisor). For example, the first monitor and the second monitor may differ in operation content, responsibility, and the like. For example, the first monitor may mainly perform remote monitoring of the bus 20, and the second monitor may perform operation management of the first monitor and the entire bus 20. In this case, one second monitor may be associated with a plurality of first monitors.

  Further, when one bus 20 is supervised by a plurality of supervisors, the determination apparatus 100 may be changeable by the supervisor even while the bus 20 is traveling. For example, when one bus 20 is supervised by a plurality of supervisors, the determining apparatus 100 changes the supervisor even if the bus 20 is traveling, assuming that a predetermined condition regarding the change timing is satisfied. May be. For example, when one bus is remotely monitored by a plurality of supervisors, the determination apparatus 100 excludes one supervisor from a plurality of supervisors monitoring the bus at a timing satisfying a predetermined condition. You may decide. In addition, for example, when one bus is remotely monitored by a plurality of supervisors, the determination apparatus 100 selects one supervisor among a plurality of supervisors who monitor the bus at a timing satisfying a predetermined condition. You may decide to change to another supervisor. For example, the determination apparatus 100 may make a change related to the monitoring person who monitors the traveling bus 20 at a desired timing, as long as the number of monitoring persons who monitor the traveling bus 20 is not zero.

(4-2. Business sharing)
Further, the determining device 100 may determine the number of buses 20 to be monitored by each monitor according to the work load, skill, etc. of each monitor. For example, the determination apparatus 100 may change the aspect related to the automatic driving of the moving body so that the load on each monitoring person in the monitoring of the plurality of moving bodies satisfies a predetermined standard. For example, the determination device 100 may determine the number of buses 20 to be monitored by each monitor so that the loads on each monitor are averaged. For example, when monitoring 100 buses 20 by 20 supervisors, the determination apparatus 100 may cause each supervisor to monitor five buses 20 so that the load on each supervisor is averaged. . Further, for example, the determination apparatus 100 determines that a supervisor with low skill or a supervisor with the maximum number of monitors less than 5 monitors the bus 20 with less than 5, and the load of the remaining monitors is averaged. As such, the number of buses 20 monitored by each monitor may be determined.

  Moreover, the determination apparatus 100 may determine the load of each monitor based on various information without being limited to the number of monitors. For example, the determination device 100 may determine a monitoring load for monitoring the bus 20 according to the operation route of each bus 20. For example, the determination apparatus 100 determines the load score of the bus 20 according to the operation route of each bus 20, and each monitor so that the sum of the load scores of the bus 20 to be monitored by each monitor is averaged. May determine the bus 20 to be monitored. For example, the determination apparatus 100 may determine the load score of the bus 20 of the operation route with a large traffic such as a pedestrian or a bicycle higher than that of the bus 20 of another operation route. For example, the determination apparatus 100 may determine the load score of the bus 20 on the operation route with few branches such as a single road lower than the buses 20 on other operation routes.

  For example, the determination apparatus 100 may determine the load score according to the status of each bus 20 (see FIG. 8). For example, the determination apparatus 100 may determine the load score of the bus 20 whose status is “event occurrence” higher than that of the bus 20 whose status is “normal”. For example, when the load on one monitor is equal to or greater than a predetermined threshold (upper limit threshold), the determining apparatus 100 may change the monitor on the bus 20 to be monitored by the one monitor to another monitor. Good. For example, the determination apparatus 100 changes the monitor of the bus 20 to be monitored by the one monitor to another monitor so that the load on the one monitor is less than a predetermined threshold (upper limit threshold). Also good. Note that the above is an example, and the determination device 100 may determine the bus 20 to be monitored by each monitor based on various information.

(4-3. Predetermined events)
Note that the predetermined event in the bus 20 that is a moving body is not limited to the occurrence of a failure or abnormal noise, and may be various events. The predetermined event in the bus 20 may be various events such as an emergency vehicle approach and an obstacle. For example, the determination apparatus 100 may determine to change an aspect related to the automatic operation of the bus 20 by setting an event that prevents the movement of the bus 20 as a predetermined event. For example, when a failure occurs in the traveling direction of the bus 20, the determination device 100 may change the aspect related to the automatic operation of the bus 20. For example, when an obstacle such as construction or falling rocks occurs in the traveling direction of one bus 20 (referred to as “required bus”), the determination apparatus 100 becomes a monitoring target of a monitor who monitors the required bus. The other bus 20 monitor may be changed to another monitor.

(4-4. Change timing)
Moreover, the determination apparatus 100 may change the supervisor of the bus 20 at various timings. For example, the determination apparatus 100 may change the supervisor who monitors the bus 20 at the timing when the bus 20 stops. Further, for example, the determination apparatus 100 may change the monitor who monitors the bus 20 at a timing when a predetermined time has elapsed since the bus 20 stopped. For example, the determination apparatus 100 may change the supervisor who monitors the bus 20 when the signal of the bus 20 is stopped. Further, the timing for changing the supervisor of the bus 20 here is not limited to the time when the bus 20 is stopped, but includes various timings. For example, the determination apparatus 100 may change the supervisor who monitors the bus 20 when the bus 20 travels at a low speed. For example, the determination apparatus 100 may change the supervisor who monitors the bus 20 when the bus 20 travels at a speed less than a predetermined threshold. For example, the determination apparatus 100 may change the supervisor who monitors the bus 20 when the bus 20 is traveling stably. For example, the determination apparatus 100 may change the supervisor who monitors the bus 20 when the bus 20 travels satisfying a predetermined condition regarding stable travel. For example, the determination apparatus 100 may change the supervisor who monitors the bus 20 when the bus 20 travels on a single lane road over a predetermined distance. For example, the determination apparatus 100 may change the supervisor who monitors the bus 20 when the bus 20 travels on a road having no branch such as an intersection over a predetermined distance. For example, the determination apparatus 100 may change the supervisor who monitors the bus 20 when the bus 20 travels on a straight road. For example, the determination apparatus 100 may change the monitor who monitors the bus 20 when the bus 20 travels on a road having no turning radius curve equal to or less than a predetermined threshold over a predetermined distance.

[5. (Monitoring process flow)
Next, the monitoring process by the monitoring system 1 according to the embodiment will be described with reference to FIG. FIG. 13 is a flowchart illustrating an example of the monitoring process according to the embodiment.

  First, the monitoring device 10 outputs a plurality of image information and a plurality of audio information detected by a plurality of moving bodies (step S101).

  And when there exists target audio | voice information which satisfy | fills the predetermined condition which needs confirmation of an operating condition (step S102: Yes), the monitoring apparatus 10 displays the image information corresponding to object audio | voice information in the display mode which can be identified (step). S103).

  If there is no target audio information that satisfies a predetermined condition that requires confirmation of the operation status (step S102: No), the determining apparatus 100 returns to step S102 and repeats the process.

[6. effect〕
As described above, the monitoring device 10 according to the embodiment includes the acquisition unit 152 and the notification unit 154. The acquisition unit 152 acquires a plurality of audio information detected by a moving body that is remotely monitored by a monitor and that operates by automatic driving. The notification unit 154 notifies the monitor of target audio information that is audio information that satisfies a predetermined condition that requires confirmation of the operation status among the plurality of audio information acquired by the acquisition unit 152.

  As described above, the determination apparatus 100 according to the embodiment notifies the supervisor of target audio information that is audio information satisfying a predetermined condition that requires confirmation of an operation state from among a plurality of audio information. It is possible to appropriately grasp the correspondence between the image and sound related to the monitored moving body.

  Moreover, in the determination apparatus 100 according to the embodiment, the acquisition unit 152 is a moving body that is remotely monitored by a monitor, and includes a plurality of pieces of image information and a plurality of audio information detected by the moving body that operates by automatic driving. get. The notification unit 154 corresponds to the target audio information and the image information corresponding to the target audio information, which is the audio information satisfying a predetermined condition that requires confirmation of the operation status, among the plurality of audio information acquired by the acquisition unit 152. A plurality of pieces of image information are displayed in a display mode that can be identified.

  As described above, the determination apparatus 100 according to the embodiment can identify that the target audio information corresponds to the image information corresponding to the target audio information that is the audio information that satisfies the predetermined condition that requires confirmation of the operation state. By displaying a plurality of pieces of image information in the display mode, it is possible to appropriately grasp the correspondence between the image and sound related to the remotely monitored mobile body.

  In the determination device 100 according to the embodiment, the notification unit 154 displays image information corresponding to the target audio information and display information related to the target audio information in association with each other.

  As described above, the determination apparatus 100 according to the embodiment displays the image information corresponding to the target sound information and the display information regarding the target sound information in association with each other, thereby displaying the image and sound regarding the mobile object to be remotely monitored. Can be properly grasped.

  In the determination device 100 according to the embodiment, the notification unit 154 displays image information corresponding to the target audio information and display information obtained by visualizing the target audio information in association with each other.

  As described above, the determination apparatus 100 according to the embodiment displays the image information related to the mobile object to be remotely monitored by displaying the image information corresponding to the target audio information and the display information obtained by visualizing the target audio information in association with each other. It is possible to appropriately understand the correspondence between the voice and the voice.

  In the determination device 100 according to the embodiment, the notification unit 154 outputs a plurality of pieces of audio information acquired by the acquisition unit 152.

  As described above, the determination apparatus 100 according to the embodiment can appropriately grasp the correspondence between the image and the sound regarding the mobile body that is remotely monitored even when outputting a plurality of sound information.

  Moreover, in the determination apparatus 100 according to the embodiment, the notification unit 154 outputs a plurality of pieces of audio information in an output mode based on information regarding the line of sight of the supervisor among a plurality of pieces of image information.

  As described above, the determination apparatus 100 according to the embodiment outputs the plurality of audio information in the output mode based on the information regarding the line of sight of the monitor among the plurality of image information, and thereby the image regarding the mobile body that is remotely monitored. It is possible to appropriately understand the correspondence between the voice and the voice.

  Further, in the determination device 100 according to the embodiment, the notifying unit 154 includes gaze image information that is image information that is estimated to be watched by the monitor according to the movement of the line of sight of the monitor among the plurality of pieces of image information. Is output in a different output mode from the audio information corresponding to the other image information.

  As described above, the determination apparatus 100 according to the embodiment corresponds to gaze image information that is image information that is estimated to be watched by the monitor according to the movement of the gaze of the monitor among the plurality of pieces of image information. By outputting the audio information in an output mode different from the audio information corresponding to the other image information, it is possible to appropriately grasp the correspondence between the image and the audio regarding the mobile body to be remotely monitored.

  In the determination device 100 according to the embodiment, the notification unit 154 outputs only the audio information corresponding to the gaze image information among the plurality of pieces of image information.

  As described above, the determination apparatus 100 according to the embodiment appropriately handles the correspondence between the image and the sound related to the remotely monitored moving body by outputting only the sound information corresponding to the gaze image information among the plurality of pieces of image information. Can be made graspable.

  Moreover, in the determination apparatus 100 according to the embodiment, the notification unit 154 increases the volume of the audio information corresponding to the gaze image information higher than the volume of the audio information corresponding to the other image information.

  As described above, the determination apparatus 100 according to the embodiment relates to a mobile body that is remotely monitored by increasing the volume of audio information corresponding to gaze image information higher than the volume of audio information corresponding to other image information. It is possible to appropriately grasp the correspondence between the image and the sound.

  Moreover, in the determination apparatus 100 according to the embodiment, when there are a plurality of target audio information satisfying a predetermined condition, the notification unit 154 has a display mode that varies depending on each level of the plurality of target audio information, Image information corresponding to each target audio information is displayed.

  As described above, when there are a plurality of pieces of target audio information satisfying a predetermined condition, the determination apparatus 100 according to the embodiment displays the plurality of pieces of target audio information in a display mode that varies according to each level of the plurality of pieces of target audio information. By displaying the image information corresponding to each of the images, it is possible to appropriately grasp the correspondence between the image and sound related to the remotely monitored mobile object.

  In the determination device 100 according to the embodiment, the acquisition unit 152 acquires a plurality of image information and a plurality of audio information corresponding to each of the plurality of moving objects. The notification unit 154 is a plurality of display modes that can identify that the target audio information corresponds to the mobile object corresponding to the target audio information that satisfies the predetermined condition among the plurality of audio information acquired by the acquisition unit 152. Displays image information for.

  As described above, the determination apparatus 100 according to the embodiment has a display mode in which it is possible to identify that the mobile object corresponding to the target voice information satisfying the predetermined condition among the plurality of voice information corresponds to the target voice information. By displaying a plurality of pieces of image information, it is possible to appropriately grasp the correspondence between images and sounds related to a plurality of mobile bodies that are remotely monitored.

  In the determination device 100 according to the embodiment, the acquisition unit 152 acquires a plurality of pieces of image information and a plurality of pieces of audio information corresponding to the respective locations in one moving body. The notification unit 154 can identify that the location in the one mobile object corresponding to the target audio information that satisfies the predetermined condition among the plurality of audio information acquired by the acquisition unit 152 corresponds to the target audio information. A plurality of pieces of image information are displayed in a display mode.

  As described above, the determination apparatus 100 according to the embodiment can identify that the target voice information corresponds to a location in one moving body corresponding to the target voice information that satisfies the predetermined condition among the plurality of voice information. By displaying a plurality of pieces of image information in such a display manner, it is possible to appropriately grasp the correspondence between an image and sound related to one mobile body that is remotely monitored.

[7. program〕
The processing by the monitoring device 10 described above is realized by the monitoring program according to the present application. For example, the notification unit 154 related to the monitoring device 10 is realized by the CPU or MPU included in the monitoring device 10 executing, for example, a notification procedure related to the monitoring program using the RAM as a work area by the monitoring program. For example, the notification unit 154 related to the monitoring device 10 is executed by the CPU or MPU included in the monitoring device 10 so that the monitoring program included in the display application, for example, executes the notification procedure related to the monitoring program using the RAM as a work area. It is realized by. Similarly, the other processing units related to the monitoring device 10 are realized by executing each procedure by the monitoring program.

  Note that the processing executed by the monitoring apparatus 10 according to the present application may not necessarily be realized by the monitoring program. For example, information outside the monitoring device 10 may be acquired by an OS (Operating System) included in the monitoring device 10. That is, the monitoring program itself does not execute the processing executed by the monitoring apparatus 10 as described above, but data acquired by the OS (for example, data used to display a moving image of a moving object, movement The above-described processing of the monitoring device 10 may be realized by receiving data used for outputting body sounds.

[8. Hardware configuration)
The monitoring device 10 and the determination device 100 according to the above-described embodiments are realized by a computer 1000 having a configuration as shown in FIG. 15, for example. FIG. 15 is a hardware configuration diagram illustrating an example of a computer that implements the functions of the monitoring device and the determination device. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. The communication interface 1500 receives data from other devices via the network N (network N in FIG. 2), sends the data to the CPU 1100, and transmits the data collected by the CPU 1100 to other devices via the network N.

  The CPU 1100 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the collected data to the output device via the input / output interface 1600.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  For example, when the computer 1000 functions as the monitoring device 10 or the determination device 100 according to the embodiment, the CPU 1100 of the computer 1000 implements the functions of the control units 15 and 130 by executing a program loaded on the RAM 1200. . The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them. However, as another example, these programs may be acquired from other devices via the network N.

  As described above, some of the embodiments and modifications of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various aspects can be made based on the knowledge of those skilled in the art including the aspects described in the disclosure line of the invention. It is possible to carry out the present invention in other forms that have been modified and improved.

[9. Others]
In addition, among the processes described in the above-described embodiments and modifications, all or a part of the processes described as being automatically performed can be manually performed, or are described as being performed manually. All or part of the processing can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified. For example, the various types of information illustrated in each drawing is not limited to the illustrated information.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  In addition, the above-described embodiments and modifications can be combined as appropriate within a range that does not contradict processing contents.

  In addition, the “section (module, unit)” described above can be read as “means” or “circuit”. For example, the transmission unit can be read as transmission means or a transmission circuit.

DESCRIPTION OF SYMBOLS 1 Monitoring system 10 Monitoring apparatus 12 Memory | storage part 12A Mobile body information storage part 13 Input part 14 Output part 15 Control part 151 Transmission part 152 Acquisition part 153 Determination part 154 Notification part 16 Speaker (sound output part)
DESCRIPTION OF SYMBOLS 100 Determination apparatus 121 Mobile body information storage part 122 Monitoring person information storage part 123 Monitoring corresponding | compatible information storage part 130 Control part 131 Acquisition part 132 Determination part 133 Determination part 134 Transmission part 20 Bus (mobile body)
N network

Claims (14)

An acquisition procedure for acquiring a plurality of audio information detected in a moving body that is remotely monitored by a monitor and operated by automatic driving;
A notification procedure for notifying a monitor of target voice information that is voice information satisfying a predetermined condition that requires confirmation of an operation state among a plurality of voice information acquired by the acquisition procedure;
A monitoring program for causing a computer to execute. The acquisition procedure is as follows:
A moving body that is remotely monitored by a monitor, and obtains a plurality of image information and the plurality of audio information detected in the moving body that operates by automatic driving,
The notification procedure includes:
In a display mode capable of identifying that the target audio information corresponds to the image information corresponding to the target audio information that is the audio information satisfying the predetermined condition among the plurality of audio information acquired by the acquisition procedure. The monitoring program according to claim 1, wherein the plurality of pieces of image information are displayed. The notification procedure includes:
The monitoring program according to claim 2, wherein image information corresponding to the target audio information and display information related to the target audio information are displayed in association with each other. The notification procedure includes:
The monitoring program according to claim 3, wherein image information corresponding to the target audio information and the display information obtained by visualizing the target audio information are displayed in association with each other. The notification procedure includes:
The monitoring program according to any one of claims 2 to 4, wherein a plurality of audio information acquired by the acquisition procedure is output. The notification procedure includes:
The monitoring program according to claim 5, wherein the plurality of pieces of audio information are output in an output mode based on information on the line of sight of the monitor. The notification procedure includes:
Among the plurality of image information, audio information corresponding to gaze image information that is image information that is estimated to be watched by the monitor according to the movement of the gaze of the monitor corresponds to other image information The monitoring program according to claim 6, wherein the monitoring information is output in an output mode different from that of the audio information. The notification procedure includes:
The monitoring program according to claim 7, wherein only the audio information corresponding to the gaze image information is output from the plurality of image information. The notification procedure includes:
The monitoring program according to claim 7, wherein a volume of audio information corresponding to the gaze image information is set larger than a volume of audio information corresponding to other image information. The notification procedure includes:
When there are a plurality of target audio information satisfying the predetermined condition, image information corresponding to each of the plurality of target audio information is displayed in a display mode that varies according to the level of each of the plurality of target audio information. The monitoring program according to any one of claims 2 to 9, wherein: The acquisition procedure is as follows:
Obtaining the plurality of image information and the plurality of audio information corresponding to each of a plurality of moving bodies;
The notification procedure includes:
Among the plurality of audio information acquired by the acquisition procedure, the mobile object corresponding to the target audio information satisfying the predetermined condition and the plurality of audio information in a display mode capable of identifying that the target audio information corresponds. Image information is displayed. The monitoring program of any one of Claims 2-10 characterized by the above-mentioned. The acquisition procedure is as follows:
Obtaining the plurality of pieces of image information and the plurality of pieces of audio information corresponding to the respective locations in one moving body;
The notification procedure includes:
Display capable of identifying that the target voice information corresponds to a location in the one moving body corresponding to the target voice information satisfying the predetermined condition among the plurality of voice information acquired by the acquisition procedure. The monitoring program according to any one of claims 2 to 10, wherein the plurality of pieces of image information are displayed in an aspect. A monitoring method performed by a computer,
An acquisition step of acquiring a plurality of audio information detected in a moving body that is remotely monitored by a monitor and operated by automatic driving;
A notification step of notifying a monitor of target audio information that is audio information satisfying a predetermined condition that requires confirmation of an operation state among the plurality of audio information acquired by the acquisition step;
The monitoring method characterized by including. An acquisition unit that acquires a plurality of audio information detected in a moving body that is remotely monitored by a monitor and operates by automatic driving;
A notification unit notifying unit for notifying a monitor of target audio information that is audio information satisfying a predetermined condition that requires confirmation of an operation state among the plurality of audio information acquired by the acquisition unit;
A monitoring device comprising:

Images