JP2024009255A - Output device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To cite it as one example that a smooth automatic operation is realized by smoothly performing mutual understanding with the other moving body.

SOLUTION: An output device 10 comprises a first acquisition part 110, a second acquisition part 120, and a first output part 150. The first acquisition part 110 acquires first behavioral information indicating first behavior of an occupant in a first moving body. The second acquisition part 120 acquires second behavioral information indicating second behavior of a second moving body around the first moving body within a predetermined period from occurrence of the first behavior. The first output part 150 outputs control information for controlling traveling of the first moving body, based on the first behavioral information and the second behavioral information.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an output device, an output method, and a computer program.

In autonomous driving, the next action of the own vehicle is determined while estimating the next movement of surrounding vehicles, pedestrians, etc. using various sensing results.

Patent Document 1 discloses that automatic steering control by a system is executed with reference to the driver's driving operation after detecting a pedestrian, etc., thereby increasing the degree of the driver's involvement in driving the vehicle. is listed.

Further, Patent Document 2 describes that when a specific message is received from another mobile body, a response message including intention information indicating the intention to perform a specific behavior is transmitted to that mobile body. .

Japanese Patent Application Publication No. 2017-206040 Japanese Patent Application Publication No. 2016-186721

However, with the technique of Patent Document 1, it was not possible to communicate with pedestrians and the like.

Further, in the technique of Patent Document 2, since information transmission and reception are involved, smooth communication with other moving bodies may be impaired. Furthermore, if messages etc. are not sent from other mobile units, it is not possible to communicate with each other.

An example of the problem to be solved by the present invention is to smoothly communicate with other moving bodies and realize smooth automatic driving.

The first invention is
a first acquisition unit that acquires first behavior information indicating a first behavior of a passenger of the first moving object;
a second acquisition unit that acquires second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior;
a first output unit that outputs control information for controlling travel of the first mobile body based on the first behavior information and the second behavior information;
It is an output device equipped with.

The second invention is
a first acquisition step of acquiring first behavior information indicating a first behavior of a passenger of the first moving object;
a second acquisition step of acquiring second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior;
a first output step of outputting control information for controlling travel of the first mobile body based on the first behavior information and the second behavior information;
This is an output method that includes.

The third invention is
A computer program for realizing an output device,
computer,
a first acquisition means for acquiring first behavior information indicating a first behavior of a passenger of the first moving body;
a second acquisition means for acquiring second behavior information indicating a second behavior of a second moving object around the first moving object within a predetermined period from the occurrence of the first behavior; a first output means for outputting control information for controlling travel of the first moving body based on the information and the second behavior information;
It is a computer program that functions as

FIG. 2 is a block diagram illustrating a functional configuration of an output device according to an embodiment. 3 is a flowchart illustrating an output method according to an embodiment. 1 is a block diagram illustrating a functional configuration of an output device according to a first embodiment. FIG. FIG. 2 is a diagram illustrating a hardware configuration of an output device. 5 is a flowchart illustrating processing performed by the output device according to the first embodiment. 7 is a flowchart illustrating a process performed by an output device according to a second embodiment. FIG. 7 is a block diagram illustrating a functional configuration of an output device according to a fifth embodiment. 12 is a block diagram illustrating a functional configuration of an output device 10 according to a sixth embodiment. FIG. 12 is a flowchart illustrating processing performed by the output device 10 according to the sixth embodiment.

Embodiments of the present invention will be described below with reference to the drawings. Note that in all the drawings, similar components are denoted by the same reference numerals, and descriptions thereof will be omitted as appropriate.

In the description below, the first acquisition section 110, second acquisition section 120, first detection section 112, second detection section 122, third acquisition section 130, determination section 140, first output section 150, The second output section 152, the first input section 160, the second input section 162, and the storage section 170 are not configurations in hardware units but blocks in functional units. The first acquisition section 110, the second acquisition section 120, the first detection section 112, the second detection section 122, the third acquisition section 130, the determination section 140, the second output section 152, the first output section 150 of the output device 10, The second input section 162, the first input section 160, and the storage section 170 mainly include a CPU of an arbitrary computer, a memory, a program loaded into the memory, a storage medium such as a hard disk that stores the program, and an interface for network connection. can be realized by any combination of hardware and software. There are various modifications of the method and device for realizing this.

FIG. 1 is a block diagram illustrating the functional configuration of an output device 10 according to an embodiment. The output device 10 according to this embodiment includes a first acquisition section 110, a second acquisition section 120, and a first output section 150. The first acquisition unit 110 acquires first behavior information indicating a first behavior of a passenger of a first moving object. The second acquisition unit 120 acquires second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period T1 from the occurrence of the first behavior. The first output unit 150 outputs control information for controlling travel of the first moving object based on the first behavior information and the second behavior information. This will be explained in detail below.

In a mobile object that runs partially or completely using automatic driving, an automatic driving control device determines the surrounding situation, and the operation of the mobile object is controlled based on the determination result. However, if it is difficult to predict the next behavior of other nearby moving objects, autonomous driving may not be able to run smoothly.

On the other hand, a person riding in a self-driving vehicle can communicate with, for example, passengers in surrounding vehicles and pedestrians. According to this embodiment, the surrounding situation and the next behavior of other moving objects are estimated from the behavior of the occupant of the own vehicle and the behavior of the occupants of surrounding vehicles and pedestrians, and the next behavior of other moving objects is estimated. Running is maintained.

In this embodiment, the first moving object is a vehicle such as a car, a motorized bicycle, or a light vehicle. The output device 10 is mounted on the first moving body. The first moving body is at least partially automatically driven by the control information output from the first output unit 150. The first moving body has at least one passenger.

The first behavior is the behavior of the passenger of the first moving body, and is a gesture or vocalization toward the second moving body. Specifically, the first behavior includes the behavior of holding out the palm facing upward, the behavior of pointing the palm toward the second moving object, the behavior of nodding, the behavior of conveying intentions by voice, and the headlight of the first moving object. , behavior that causes the hazard lights or brake lights to flash one or more times, behavior that causes the horn of the first moving object to sound more than once, behavior that causes the first moving object to decelerate, etc. It will be done. The first behavior information is an ID or the like that indicates which behavior is the first behavior. The first behavior may be a behavior toward the second moving body, or may not be a behavior toward the second moving body. Note that the first behavior is preferably one that can be directly or indirectly recognized by the second moving body.

The second moving body is a moving body located around the first moving body. The area around the first moving body is, for example, a range in which behavior can be recognized and detected from the first moving body. The area around the first moving body is not particularly limited, but is, for example, a range within a distance of 50 m or less from the first moving body.

The second moving object is, for example, a pedestrian or a bicycle. In this case, examples of the second behavior include behavior in which the pedestrian or bicycle rider nods, stops, or raises one hand. The second behavior information is an ID or the like that indicates which behavior is the second behavior. The second behavior may be a behavior toward the first moving object, or may not be a behavior toward the first moving object. Note that the second behavior is preferably one that can be directly or indirectly recognized by the first moving object.

The generation and acquisition method of the first behavior information and the second behavior information will be explained in detail in the examples described later.

For example, when a second moving object is a pedestrian on one side of a crosswalk, a passenger of the first moving object points his palm upward at face level (first behavior) and approaches the pedestrian. Encourage them to cross at the crosswalk. Then, the pedestrian nods his head (second behavior), indicating his intention to accept the offer from the passenger of the first moving object. The first output unit 150 outputs control information for stopping the first moving body based on first behavior information and second behavior information indicating these behaviors, respectively.

Also, for example, when a pedestrian on one side of a crosswalk is a second moving object, a passenger of the first moving object may face the pedestrian with his/her palm facing up at face level (first behavior). encourage people to cross at the crosswalk. Then, the pedestrian shakes his head (second behavior), indicating his intention to decline the offer of the passenger of the first moving object. The first output unit 150 outputs control information for causing the first mobile object to travel based on first behavior information and second behavior information indicating these behaviors, respectively. In addition, the second behavior at this time may be a behavior in which the pedestrian holds his/her hand vertically and waves it horizontally, or a behavior in which the pedestrian moves horizontally in the direction of travel of the vehicle with the palm facing upward, or the pedestrian moves his or her hand horizontally in the direction of travel of the vehicle. It is also possible to behave in a way that does not react.

The second moving body may be an oncoming vehicle of the first moving body. In that case, the second behavior is that the second moving object flashes its headlights, hazard lights, or brake lights one or more times, honks its horn more than once, decelerates, or approaches the edge of the road. Examples include behavior. Furthermore, the second behavior may be the behavior of a passenger of an oncoming vehicle that is a second moving object. Behaviors of the occupants of the oncoming vehicle include bowing, raising one hand, and the like. The second behavior information is an ID or the like that indicates which behavior is the second behavior.

For example, when the oncoming vehicle on a narrow road is a second moving object, the passenger of the first moving object points his/her palm toward the second moving object to ask the second moving object to wait for passage (first behavior). Then, the second moving object approaches the edge of the road and gives way (second behavior). Based on the first behavior information and the second behavior information respectively indicating these behaviors, control information is output so as to cause the first moving body to run while passing the second moving body.

Another example of a scene in which the first output unit 150 outputs a control signal based on the first behavior information and the second behavior information is when the first moving object or the second moving object makes a right turn or merges into a road. Examples include situations where

The second behavior is performed within a predetermined period T1 from the occurrence of the first behavior. The predetermined period T1 is predetermined based on the time expected for the second moving body to react in response to the first behavior. The predetermined period T1 is, for example, within 5 seconds or within 3 seconds.

FIG. 2 is a flowchart illustrating the output method according to the embodiment. The method includes a first acquisition step (S10), a second acquisition step (S20), and a first output step (S50). In the first acquisition step (S10), first behavior information indicating the first behavior of the passenger of the first moving body is acquired. In the second acquisition step (S20), second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period T1 from the occurrence of the first behavior is acquired. In the first output step (S50), control information for controlling the travel of the first moving object is output based on the first behavior information and the second behavior information.

The output method according to this embodiment is realized, for example, by the output device 10 as described above.

As described above, according to the present embodiment, the first output unit 150 outputs control information for controlling the travel of the first moving object based on the first behavior information and the second behavior information. Therefore, communication with other moving bodies is carried out smoothly, and smooth automatic driving is realized.

(Example 1)
FIG. 3 is a block diagram illustrating the functional configuration of the output device 10 according to the first embodiment. The output device 10 according to this example has the configuration of the output device 10 according to the embodiment. The output device 10 according to this embodiment further includes a first input section 160, a storage section 170, a first detection section 112, a second detection section 122, and a determination section 140. The functional configuration of the output device 10 will be described in detail later.

FIG. 4 is a diagram illustrating the hardware configuration of the output device 10. In this figure, the output device 10 is implemented using an integrated circuit 40. The integrated circuit 40 is, for example, an SoC (System On Chip).

Integrated circuit 40 has a bus 402, a processor 404, a memory 406, a storage device 408, an input/output interface 410, and a network interface 412. The bus 402 is a data transmission path through which the processor 404, memory 406, storage device 408, input/output interface 410, and network interface 412 exchange data with each other. However, the method of connecting the processors 404 and the like to each other is not limited to bus connection. The processor 404 is an arithmetic processing device implemented using a microprocessor or the like. The memory 406 is a memory implemented using RAM (Random Access Memory) or the like. The storage device 408 is a storage device implemented using ROM (Read Only Memory), flash memory, or the like.

The input/output interface 410 is an interface for connecting the integrated circuit 40 with peripheral devices. In this figure, a first camera 420, a second camera 421, a monitor 422, an input panel 423, a microphone 424, and a speaker 425 are connected to the input/output interface 410.

Network interface 412 is an interface for connecting integrated circuit 40 to a communication network. This communication network is, for example, a CAN (Controller Area Network) communication network. Note that the method for connecting the network interface 412 to the communication network may be a wireless connection or a wired connection.

The storage device 408 realizes the functions of the first input section 160, the first detection section 112, the second detection section 122, the first acquisition section 110, the second acquisition section 120, the determination section 140, and the first output section 150. It stores program modules for each. The processor 404 reads this program module into the memory 406 and executes it, thereby controlling the first input section 160, the first detection section 112, the second detection section 122, the first acquisition section 110, the second acquisition section 120, and the determination section. 140 and the functions of the first output section 150 are realized.

The hardware configuration of the integrated circuit 40 is not limited to the configuration shown in this figure. For example, program modules may be stored in memory 406. In this case, integrated circuit 40 may not include storage device 408.

The functional configuration of the output device 10 will be described in detail below with reference to FIG. 3. In the output device 10 according to the present embodiment, the first acquisition unit 110 is capable of receiving input from the passenger to the first input unit 160. An example of the first behavior is input to the first input unit 160.

In the example of FIG. 4, the input panel 423 and microphone 424 function as the first input section 160. Further, the first input unit 160 may be a mechanical handle, switch, pedal, etc. The first behavior is, for example, an input operation on a touch panel or an input by voice, and the input by the passenger is, for example, an operation that causes the headlights, hazard lamps, or brake lights of the first moving object to blink one or more times. These are an operation of sounding the horn of the first moving body one or more times, and an operation of decelerating the first moving body. At least one of the passengers of the first moving body is on board as a driver in a state where the first input unit 160 can be accessed. For example, a passenger serving as a driver is seated at an operating position where the first input section 160, such as a touch panel and a microphone, is provided. Further, the passenger serving as the driver holds an operation terminal, which is the first input section 160, and is equipped with a touch panel, a microphone, and the like. When the passenger performs an input operation on the first input unit 160 from among the plurality of input operations determined as candidates for the first behavior, the first acquisition unit 110 converts the information indicating the operation content into the first behavior. Obtain as information. Further, the first input unit 160 may be capable of receiving an input from the passenger indicating that the first behavior has been performed. For example, a message such as "Did you urge the pedestrian to cross the street?" may be displayed on the screen or a voice may be output to prompt the passenger to perform touch input or voice input.

Since the first behavior is input to the first input unit 160, the first acquisition unit 110 can directly acquire the first behavior information, and the first behavior can be recognized with high accuracy.

As described above, the output device 10 according to this embodiment includes the first detection section 112. The first detection unit 112 detects the behavior of the first moving object. The first acquisition unit 110 acquires first behavior information from the first detection unit 112. The first detection unit 112 acquires an image that includes at least a portion of the passenger of the first moving object, detects a first behavior using the image, and generates first behavior information. The first detection unit 112 acquires an image captured by the first camera 420. The first camera 420 is provided to capture an image of at least a portion of the driver of the first moving object. In addition, instead of the first camera 420, images may be acquired using a ToF (Time of Flight) camera or a lidar (LIDAR: Laser Imaging Detection and Ranging, Laser Illuminated Detection and Ranging, or LiDAR: Light Detection and Ranging). Good too.

The first detection unit 112 detects the first behavior of the passenger by, for example, extracting and analyzing feature points in the acquired image. Information indicating the pattern of feature points regarding the first behavior is stored in advance in the storage unit 170 for each behavior content, and the first detection unit 112 can read it and use it for detection. When the first detection unit 112 detects the first behavior, it generates information indicating the content of the behavior as first behavior information. When the first behavior information is generated by the first detection unit 112, the first acquisition unit 110 acquires the first behavior information. Note that the first detection unit 112 may detect the first behavior based on a plurality of images such as a moving image.

Specifically, feature amount information indicating the feature amount of a person's body is stored in the storage unit 170 in advance. The first detection unit 112 uses feature information from the image to identify the position of the human body part in the image. The passenger's posture is estimated from this identified position and human body model information, and behavior is estimated based on changes in posture over time.

Furthermore, the first detection unit 112 may utilize deep learning. In this case, the storage unit 170 stores in advance video data of the behavior and information indicating what kind of behavior the video data represents. The first detection unit 112 estimates the passenger's behavior based on the passenger's video data and pre-stored video data.

Note that the output device 10 does not need to include one of the first input section 160 and the first detection section 112. The first acquisition unit 110 only needs to be able to acquire the first behavior information from at least one of the first input unit 160 and the first detection unit 112.

As described above, the output device 10 according to this embodiment includes the second detection section 122. The second detection unit 122 detects the behavior of the second moving object. The second acquisition unit 120 acquires the second behavior information from the second detection unit 122. The second detection unit 122 acquires an image that includes at least a portion of the second moving object, detects the second behavior using the image, and generates second behavior information.

The second detection unit 122 acquires an image captured by the second camera 421. The second camera 421 is provided to image the surroundings of the first moving body. The second detection unit 122 extracts and analyzes feature points in the image acquired from the second camera 421, for example, and extracts the second moving object. For example, a pedestrian at the edge of a crosswalk, an oncoming vehicle located diagonally in front of the first moving object, etc. are extracted as the second moving object.

The second detection unit 122 detects the second behavior of the second moving object by extracting and analyzing feature points in the acquired image, for example. Information indicating the pattern of feature points regarding the second behavior is stored in advance in the storage unit 170 for each content of the behavior, and the second detection unit 122 can read it and use it for detection. When the second detection unit 122 detects the second behavior, it generates information indicating the content of the behavior as first behavior information. When the second behavior information is generated by the second detection unit 122, the second acquisition unit 120 acquires the second behavior information.

Note that the second detection unit 122 may extract the second moving object and detect the second behavior based on a plurality of images such as a moving image. Further, like the first detection unit 112, the second detection unit 122 may detect the second behavior by using a ToF camera or lidar, or may use deep learning.

Here, the second acquisition unit 120 acquires second behavior information indicating the second behavior within a predetermined period T1 from the occurrence of the first behavior. The second detection unit 122 may detect, for example, only the behavior within a predetermined period T1 after the first detection unit 112 detects the first behavior as the second behavior.

Further, when the second detection unit 122 detects the second behavior regardless of the timing, the second acquisition unit 120 detects the second behavior if the second behavior is performed for a predetermined period exceeding T1 after detecting the first behavior. In this case, it is not necessary to acquire second behavior information indicating the second behavior. In this case, the second acquisition unit 120 acquires the second behavior information and information indicating the time when the second behavior indicated by the second behavior information was performed. Further, the second acquisition unit 120 acquires information indicating the time at which the first behavior occurs. Note that the time at which the first behavior occurs and the time at which the first acquisition unit 110 acquires the first behavior information are approximately the same, and the time at which the second behavior occurs and the time at which the second acquisition unit 120 acquires the second behavior information are approximately the same. If the time to be acquired is almost the same, the second acquisition unit 120 uses the time when the first acquisition unit 110 acquired the first behavior information and the time at which the second acquisition unit 120 attempts to acquire the second behavior information. It may be determined whether the second behavior is performed within a predetermined period T1 after the first behavior is detected by comparing the time.

Furthermore, the first acquisition unit 110 and the second acquisition unit 120 may acquire the first behavior information and the second behavior information, respectively, regardless of the timing. This case will be explained in detail later.

As described above, the output device 10 includes the determination section 140. The determination unit 140 determines whether the first output unit 150 should output control information based on the first behavior information and the second behavior information. Then, when the determination unit 140 determines that the information should be output, the first output unit 150 outputs control information based on the first behavior information and the second behavior information.

Depending on the determination result of the determination unit 140, the output device 10 is switched between a standby state and a non-standby state. That is, when the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information, the output device 10 enters the standby state, and the first output unit 150 outputs the control information based on the first behavior information and the second behavior information. When it is determined that there is no need to output control information based on the first behavior information and the second behavior information, the output device 10 enters a non-standby state. The first output unit 150 outputs control information based on the first behavior information and the second behavior information when in the standby state. On the other hand, the first output unit 150 does not output control information based on the first behavior information and the second behavior information when in the non-standby state.

For example, in a non-standby state, the first detection unit 112 and the second detection unit 122 do not need to detect behavior. Further, in the non-standby state, the first acquisition unit 110 and the second acquisition unit 120 do not need to acquire the first behavior information and the second behavior information, respectively. Further, the first output unit 150 does not need to generate control information based on the first behavior information and the second behavior information. Note that the first output unit 150 may generate and output other control information that is not based on the first behavior information and the second behavior information in the non-standby state.

By including the determination unit 140 that determines whether or not the first output unit 150 should output control information based on the first behavior information and the second behavior information, the first behavior information and the second behavior can be outputted only when necessary. Control information is generated based on the information, and the processing load on the output device 10 is reduced. For example, when the first detection unit 112 and the second detection unit 122 do not perform detection in a non-standby state, the first detection unit 112 and the second detection unit 122 do not need to continue processing images.

In this embodiment, the determination unit 140 causes the first output unit 150 to output control information based on the first behavior information and the second behavior information when the speed of the first moving object becomes equal to or lower than a predetermined speed V. Determine that it should be output. The speed V is not particularly limited, but is, for example, 2 km/h or more and 20 km/h or less. The speed of the first moving body may be reduced by automatic operation, or may be reduced by instructions or operations from the driver. For example, if the automatic driving control device cannot determine the next behavior of the second moving object based on the behavior of the second moving object alone, the speed may be reduced. In addition, the speed may be reduced if the passenger of the first moving object discovers the second moving object and believes that communication is necessary, or if the surrounding environment requires caution, such as a narrow road. be. Information indicating the predetermined speed V is held in the storage unit 170, and the determination unit 140 can read it and use it for determination.

Note that the determination unit 140 determines that when the acceleration of the first moving object becomes equal to or less than a predetermined acceleration A, the first output unit 150 should output control information based on the first behavior information and the second behavior information. It may be determined that By determining the negative acceleration as the acceleration A, the determination unit 140 can put the output device 10 into a standby state when the first moving body decelerates.

When the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information and the output device 10 enters the standby state, the monitor indicates that the output device 10 is in the standby state. The information may be presented to the passenger of the first moving body by display on a display such as 422 or by sound from a speaker 425. Upon receiving the presentation, the passenger can perform the first behavior.

Furthermore, when the output device 10 is in a standby state, the first output unit 150 may output control information for slowing down or stopping the first moving body. Furthermore, when the output device 10 enters a standby state and the presence of the second moving object is detected, the first output unit 150 may output control information for slowing or stopping the first moving object. Note that the second moving object can be detected by the second detection unit 122 in the same manner as described above.

Further, in the output device 10, the determination unit 140 determines that the first output unit 150 should output the control information based on the first behavior information and the second behavior information, and the second detection unit 122 The output device 10 may be in a standby state when the presence of a moving object is detected. In this case, the second detection unit 122 may always perform the process of detecting the second moving body while the output device 10 is activated, or the first output unit 150 may perform the process of detecting the second moving object, or the first output unit 150 may detect the first behavior information and It may start after determining that control information should be output based on behavior information.

Further, in the case where the first mobile body is equipped with an automatic driving device, when the first mobile body acquires assistance request information from the automatic driving device, the determination unit 140 determines that the first output unit 150 It may be determined that the control information should be output based on the behavior information and the second behavior information, and the output device 10 may enter a standby state. The automatic driving device outputs assistance request information when it is difficult to determine the action that the first mobile object should perform under automatic driving control.

The determination unit 140 performs determination every predetermined time T3. The time T3 is not particularly limited, but is, for example, 1 second or more and 1 minute or less. The determination unit 140 determines that when the speed of the first moving object exceeds a predetermined speed V, the first output unit 150 does not need to output the control information based on the first behavior information and the second behavior information. The determination is made and the output device 10 is placed in a non-standby state.

The first output unit 150 outputs control information based on the first behavior information and the second behavior information. In the standby state, when the first acquisition unit 110 acquires the first behavior information and the second acquisition unit 120 acquires the second behavior information, the first output unit 150 outputs the acquired first behavior information and second behavior information. Control information is generated and output based on the combination. The relationship between the combination of the first behavior information and the second behavior information and the control information to be generated is determined in advance based on the pattern of communication assumed between the first mobile body and the second mobile body. It is being Reference information indicating the relationship between the combination of the first behavior information and the second behavior information and the control information to be generated is stored in advance in the storage unit 170, and the first output unit 150 refers to the relationship between the combination of the first behavior information and the second behavior information and the control information to be generated. Information can be read and used to generate control information.

When the first acquisition unit 110 and the second acquisition unit 120 respectively acquire the first behavior information and the second behavior information regardless of the timing, the second behavior is detected within a predetermined period T1 after detecting the first behavior. The first output unit 150 may determine whether or not it has been performed. The first output unit 150 outputs the time when the first behavior information and the second behavior information are respectively generated, and the time when the first behavior information and the second behavior information are respectively acquired by the first acquisition unit 110 and the second acquisition unit 120. The determination can be made using at least one of the following. The first output unit 150 outputs the first behavior information and the second behavior information only when the second behavior information indicating the second behavior is acquired by the second acquisition unit 120 within a predetermined period T1 from the occurrence of the first behavior. Generate and output control information based on behavior information.

The first moving body is controlled by the control information output from the first output unit 150. The control information includes, for example, information for stopping the first moving body, information for starting the first moving body, and the like.

FIG. 5 is a flowchart illustrating processing performed by the output device 10 according to the first embodiment. In the following, an example will be described in which the first output unit 150 determines whether or not the second behavior has been performed within a predetermined period T1 after detecting the first behavior. The processing is not limited to this example. In the output device 10, the determination unit 140 switches between a standby state and a non-standby state. First, in step S100, the first output unit 150 checks whether the output device 10 is in a standby state. The first output unit 150 can obtain information indicating the current state from the determination unit 140, for example. When the output device 10 is not in a standby state, that is, in a non-standby state, the first output unit 150 performs step S100 again after a predetermined time T4 has elapsed. On the other hand, when the output device 10 is in a standby state, the process moves to step S200. The time T4 is not particularly limited, but is, for example, 1 second or more and 1 minute or less.

In step S200, the first output unit 150 checks whether the first behavior information is acquired by the first acquisition unit 110. If the first behavior information is not acquired by the first acquisition unit 110 during the predetermined waiting time T5, the process returns to step S100. On the other hand, when the first behavior information is acquired by the first acquisition unit 110 during the predetermined waiting time T5, the process moves to step S300. The waiting time T5 is not particularly limited, but is, for example, 1 second or more and 1 minute or less.

In step S300, the first output unit 150 checks whether the second behavior information is acquired by the second acquisition unit 120. If the second behavior information is not acquired by the second acquisition unit 120 within the predetermined period T1 after the first behavior information is acquired, the process returns to step S100. On the other hand, if the second behavior information is acquired by the second acquisition unit 120 within the predetermined period T1 after the first behavior information is acquired, the process moves to step S500. Note that in this example, the time at which the first behavior occurs and the time at which the first acquisition unit 110 acquires the first behavior information are approximately the same, and the time at which the second behavior occurs and the time at which the second acquisition unit 120 acquires the first behavior information are approximately the same. It is assumed that the times at which the two behavior information are acquired are almost the same.

In step S500, the first output unit 150 generates control information using the acquired first behavior information and second behavior information and reference information, and outputs the generated control information. Then, the process returns to step S100 again.

As described above, according to the present example, similarly to the embodiment, the first output unit 150 outputs control information for controlling the traveling of the first moving object based on the first behavior information and the second behavior information. . Therefore, communication with other moving bodies is carried out smoothly, and smooth automatic driving is realized.

In addition, according to this embodiment, when the determination unit 140 determines that the control information should be output, the first output unit 150 outputs the control information. Therefore, control information is generated based on the first behavior information and the second behavior information only when necessary, and the processing load on the output device 10 is reduced.

(Example 2)
FIG. 6 is a flowchart illustrating processing performed by the output device 10 according to the second embodiment. The output device 10 according to this embodiment is the same as the output device 10 according to the first embodiment except for the points described below. In the output device 10 according to the present embodiment, the first acquisition unit 110 further acquires third behavior information indicating the third behavior of the passenger within a predetermined period T2 from the occurrence of the second behavior. The first output unit 150 outputs control information further based on the third behavior information. This will be explained in detail below.

The third behavior is the behavior of the passenger of the first moving body who has received the second behavior by the second moving body. The passenger of the first moving object that received the second behavior may, for example, express gratitude to the second moving object, or respond to the second behavior and direct his or her attention toward the next movement of the first moving object. take action. The third behavior is, for example, the behavior of the passenger such as raising one hand or nodding, the behavior related to the passenger's line of sight such as checking the safety of the surrounding area or watching pedestrians, or the first movement. This is a behavior that causes the body's headlights, hazard lights, or brake lights to flash one or more times, or a behavior that causes the horn of the first moving body to sound one or more times.

In the output device 10 according to the present embodiment, the first output unit 150 outputs control information based on the third behavior information in addition to the first behavior information and the second behavior information. Therefore, the result of the communication between the passenger of the first moving body and the second moving body can be grasped more accurately, and the first moving body can be controlled.

In the output device 10 according to the present embodiment, the first acquisition unit 110 acquires the third behavior in the same way as acquiring the first behavior. The third behavior information is an ID or the like indicating which behavior is the third behavior. The third behavior may be a behavior toward the second moving body, or may not be a behavior toward the second moving body. Note that the third behavior is preferably a behavior that can be recognized directly or indirectly from the second moving object.

Furthermore, the third behavior may be an input to the first input unit 160, similar to the first behavior. For example, when the passenger performs an input operation on the first input unit 160 out of a plurality of input operations determined as candidates for the third behavior, the first acquisition unit 110 transfers information indicating the operation contents to the third input unit 160. Obtained as behavior information.

Furthermore, like the first behavior, the third behavior may be a behavior detected by the first detection unit 112. The first detection unit 112 can detect the third behavior in the same manner as the detection of the first behavior. Information indicating the pattern of feature points for the third behavior is stored in advance in the storage unit 170 for each content of the behavior, and the first detection unit 112 can read it and use it for detection. When the first detection unit 112 detects the third behavior, the first detection unit 112 generates information indicating the content of the behavior as third behavior information. When the first detection unit 112 generates the third behavior information, the first acquisition unit 110 acquires the third behavior information.

Here, the first acquisition unit 110 acquires third behavior information indicating the third behavior within a predetermined period T2 from the occurrence of the second behavior. Note that even if the predetermined period T1 between the occurrence of the first behavior and the occurrence of the second behavior is the same as the predetermined period T2 between the occurrence of the second behavior and the occurrence of the third behavior, It's okay and it's okay to be different. The predetermined period T2 is predetermined based on the time expected for the passenger of the first moving object to react in response to the second behavior. The predetermined period T2 is, for example, within 5 seconds or within 3 seconds.

The first detection unit 112 may detect, for example, only the behavior within a predetermined period T2 after the second detection unit 122 detects the second behavior as the third behavior.

In addition, when the first detection unit 112 detects the first behavior regardless of the timing, the first acquisition unit 110 detects the second behavior and detects that the third behavior is performed after a predetermined period T2. In this case, it is not necessary to acquire third behavior information indicating the third behavior. In this case, the second acquisition unit 120 acquires the third behavior information and information indicating the time when the third behavior indicated by the third behavior information was performed. Further, the second acquisition unit 120 acquires information indicating the time when the third behavior occurs. Note that the occurrence time of the second behavior and the time when the second acquisition unit 120 acquires the second behavior information are almost the same, and the occurrence time of the third behavior and the time when the first acquisition unit 110 acquires the third behavior information are almost the same. If the times at which the first acquisition unit 110 attempts to acquire the second behavior information are almost the same, the first acquisition unit 110 attempts to acquire the third behavior information at the time at which the second acquisition unit 120 acquires the second behavior information. It may be determined whether the third behavior is performed within a predetermined period T2 after the second behavior is detected by comparing the time.

Further, the second acquisition unit 120 and the first acquisition unit 110 may acquire the second behavior information and the third behavior information, respectively, regardless of the timing. In this case, the first output unit 150 may determine whether the third behavior is performed within the predetermined period T2 after detecting the second behavior. The first output unit 150 outputs the time when the second behavior information and the third behavior information are respectively generated, and the time when the second behavior information and the third behavior information are respectively acquired by the second acquisition unit 120 and the first acquisition unit 110. The determination can be made using at least one of the following. The first output unit 150 outputs the first behavior information and the first behavior information only when the first acquisition unit 110 acquires the third behavior information indicating the third behavior within the predetermined period T2 from the occurrence of the second behavior. Based on the second behavior information and the third behavior information, control information is generated and output.

The output method according to this embodiment will be described below with reference to FIG. 6. In this embodiment, steps S100 to S300 are performed in the same manner as in the first embodiment. In this embodiment, in step S300, if the second behavior information is acquired by the second acquisition unit 120 within a predetermined period T1 after the first behavior information is acquired, the process moves to step S400.

In step S400, the first output unit 150 checks whether the first acquisition unit 110 acquires the third behavior information. If the third behavior information is not acquired by the first acquisition unit 110 during the predetermined period T2 after the second behavior information is acquired, the process returns to step S100. On the other hand, if the third behavior information is acquired by the first acquisition unit 110 during the predetermined period T2 after the second behavior information is acquired, the process moves to step S500. In this example, the occurrence time of the second behavior and the time when the second acquisition unit 120 acquires the second behavior information are almost the same, and the occurrence time of the third behavior and the time when the first acquisition unit 110 acquires the second behavior information are almost the same. 3. It is assumed that the times at which the behavior information is acquired are almost the same.

In step S500, the first output unit 150 generates control information using the acquired first behavior information, second behavior information, and third behavior information and reference information, and outputs the generated control information. . Then, the process returns to step S100 again.

In this embodiment, the reference information is information indicating the relationship between the combination of the first behavior information, the second behavior information, and the third behavior information and the control information to be generated.

As described above, according to the present example, similarly to the embodiment, the first output unit 150 outputs control information for controlling the traveling of the first moving object based on the first behavior information and the second behavior information. . Therefore, communication with other moving bodies is carried out smoothly, and smooth automatic driving is realized.

In addition, according to this embodiment, the first output unit 150 outputs control information further based on the third behavior information. Therefore, the first moving object is controlled by grasping the result of communication between the passenger of the first moving object and the second moving object with higher accuracy.

(Example 3)
In the output device 10 according to the third embodiment, when the first moving object is located within a predetermined range from a crosswalk or an intersection, the first output section 150 outputs the first behavior information and the second behavior. The output device 10 is the same as the output device 10 according to at least one of the first embodiment and the second embodiment, except that it is determined that control information should be output based on the information. This will be explained in detail below.

In the vicinity of a crosswalk or an intersection, there is a high possibility that communication between the passenger of the first moving object and the second moving object will be necessary. Therefore, when the first moving object is located within a predetermined range from a crosswalk or an intersection, the determination unit 140 puts the output device 10 on standby, thereby allowing smooth automatic driving near the crosswalk or intersection. will be held.

The determination unit 140 according to this embodiment acquires the position information of the first moving object and at least map information of the area where the first moving object is located. The first mobile body is equipped with, for example, a GPS (Global Positioning System) receiver, and the determination unit 140 receives position information indicating the position of the first mobile body from the GPS device installed in the first mobile body. get. Further, the map information is stored in advance in the storage unit 170, and the determination unit 140 can read and use it. Further, the determination unit 140 may obtain map information from an external server via a communication network. The map information includes information indicating the location of a crosswalk and information indicating the location of an intersection.

The determining unit 140 uses the acquired position information and map information to determine whether the first moving object is located within a predetermined range from a crosswalk or an intersection. The predetermined range is determined assuming a range in which communication between the passenger of the first moving body and the second moving body is required. "Within a predetermined range" means, for example, that the distance between the first moving object and the crosswalk or intersection is within L [m]. L is, for example, 2 or more and 100 or less. Note that the determination unit 140 determines that the first mobile object is located within a predetermined range from the crosswalk or intersection only when the crosswalk or intersection is located in front of the first mobile object in the direction of travel. Also good.

The determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the first moving object is located within a predetermined range from a crosswalk or an intersection. judge. That is, the output device 10 is placed in a standby state. On the other hand, when the first moving object is not located within a predetermined range from a crosswalk or an intersection, the first output unit 150 determines that there is no need to output the control information based on the first behavior information and the second behavior information. do. That is, the output device 10 is placed in a non-standby state.

Further, in this embodiment, the reference information is prepared for each fact that is the basis for determining that the first output unit 150 should output control information based on the first behavior information and the second behavior information. It's okay if it's done. In this case, when the first output unit 150 determines that control information should be output based on the first behavior information and the second behavior information, the determination unit 140 determines that the fact that is the basis for the determination (for example, “from the crosswalk The first moving object generates information indicating that the first moving object is located within a predetermined range, and that the speed of the first moving object has become less than a predetermined speed.

In step S500, the first output unit 150 acquires information indicating the fact that is the basis of the determination from the determination unit 140, and reads reference information corresponding to the fact indicated by the information from the storage unit 170. The first output unit 150 then generates control information using the read reference information and outputs it.

The content of communication between the passenger of the first moving object and the second moving object may vary depending on the scene in which the communication takes place. Therefore, by having the first output unit 150 output control information using reference information prepared for each fact as a basis, smooth automatic driving is performed in each scene.

As described above, according to the present example, similarly to the embodiment, the first output unit 150 outputs control information for controlling the traveling of the first moving object based on the first behavior information and the second behavior information. . Therefore, communication with other moving bodies is carried out smoothly, and smooth automatic driving is realized.

In addition, according to the present embodiment, when the first moving object is located within a predetermined range from a crosswalk or an intersection, the first output section 150 outputs the first behavior information and the second behavior information. It is determined that control information should be output based on the information. Therefore, automatic driving can be performed smoothly near crosswalks and intersections.

(Example 4)
In the output device 10 according to the fourth embodiment, the determination unit 140 determines that when the width of the road on which the first moving body travels satisfies a predetermined condition, the first output unit 150 outputs the first behavior information and the second behavior information. This is the same as the output device 10 according to at least one of the first to third embodiments, except that it is determined that control information should be output based on behavior information. This will be explained in detail below.

In this embodiment, the first moving body is a vehicle, and the second moving body is an oncoming vehicle of the first moving body. When two vehicles pass each other on a narrow road, the two opposing vehicles may communicate and give way to each other. Therefore, when the width of the road on which the first moving object travels satisfies a predetermined condition, the determination unit 140 puts the output device 10 into the standby state, so that automatic driving can be performed smoothly even on a narrow road.

The determination unit 140 can make the determination using, for example, the road width included in the map information. Further, the determination unit 140 may make the determination using the results of actually measuring the surroundings of the first moving body.

An example in which the determination unit 140 makes a determination using the road width indicated in the map information will be described. The determination unit 140 acquires the map information and the position information of the first moving body in the same manner as in the third embodiment. In this example, the map information includes information indicating the width of each road. Then, the determination unit 140 determines whether the width of the road on which the first mobile object travels satisfies a predetermined condition using the acquired position information and map information. This condition indicates that the space is narrow enough to require caution when two moving objects pass each other. Specifically, for example, if the width of the road on which the first moving object travels is less than or equal to W [m], the determination unit 140 determines that the width of the road satisfies a predetermined condition. W is, for example, 7 or more and 10 or less.

A case where the determination unit 140 makes a determination using the results of actually measuring the surroundings of the first moving body will be described below. In this example, the first moving body is provided with a sensor that measures the distance between the lateral side of the first moving body and the structure facing the lateral side. Then, the determination unit 140 acquires the distance between the lateral side surface of the first moving body and the structure from the sensor. The structure is, for example, a building, wall, or guardrail provided on the side of the road. If the acquired distance is less than or equal to a predetermined reference distance, the determination unit 140 determines that the width of the road on which the first moving object travels satisfies a predetermined condition.

The determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the width of the road on which the first moving object travels satisfies a predetermined condition. It is determined that That is, the output device 10 is placed in a standby state. On the other hand, when the width of the road on which the first moving body travels does not satisfy a predetermined condition, there is no need for the first output unit 150 to output control information based on the first behavior information and the second behavior information. It is determined that That is, the output device 10 is placed in a non-standby state.

In addition, in this example, as in Example 3, the determination unit 140 determines each fact that is the basis for determining that the first output unit 150 should output control information based on the first behavior information and the second behavior information. Reference information may also be prepared. Then, the first output unit 150 may generate control information using reference information corresponding to the fact that is the basis for the determination, and output the control information.

As described above, according to the present example, similarly to the embodiment, the first output unit 150 outputs control information for controlling the traveling of the first moving object based on the first behavior information and the second behavior information. . Therefore, communication with other moving bodies is carried out smoothly, and smooth automatic driving is realized.

In addition, according to the present embodiment, when the width of the road on which the first moving body travels satisfies a predetermined condition, the first output unit 150 outputs the first behavior information and the second behavior information. It is determined that control information should be output based on the behavior information. Therefore, automatic driving can be performed smoothly even on narrow roads.

(Example 5)
FIG. 7 is a block diagram illustrating the functional configuration of the output device 10 according to the fifth embodiment. The output device 10 according to this embodiment is the same as the output device 10 according to at least one of the first to fourth embodiments, except for the points described below.

The output device 10 according to this embodiment further includes a third acquisition section 130. The third acquisition unit 130 acquires information indicating a caution area to which the first output unit 150 should output control information based on the first behavior information and the second behavior information. The determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the first moving object is located within the caution area. This will be explained in detail below.

Places where it may be necessary for passengers of a first moving object to communicate with a second moving object, such as intersections, crosswalks, narrow roads, highways, or merging at store entrances and exits, can be predicted to some extent in advance. be. By designating such locations as areas requiring attention, the area in which automated driving can be performed smoothly can be expanded.

The caution area is an area where control information should be output based on the first behavior information and the second behavior information. The caution areas include, for example, areas near intersections and crosswalks, narrow roads, expressways, and areas near confluences at store entrances and exits. As the areas requiring attention, a plurality of areas where communication between the passenger of the first moving body and the second moving body is expected to occur can be estimated and determined in advance. The information indicating the caution areas is, for example, a list indicating the range of each area in terms of latitude and longitude. Further, the information indicating the caution area may be information indicating an area on a map that is linked to map information.

The third acquisition unit 130 acquires information indicating a caution area from an external server via a communication network, for example. Further, the third acquisition unit 130 may read and acquire information indicating a caution area stored in the storage unit 170 in advance.

The determination unit 140 acquires the position information of the first moving object as in the third embodiment. Then, using the position information and the information indicating the caution area acquired by the third acquisition unit 130, it is determined whether the first moving object is located within the caution area. Then, when the first moving object is located within the caution area, the first output unit 150 determines that control information should be output based on the first behavior information and the second behavior information. That is, the output device 10 is placed in a standby state. On the other hand, when the first moving object is not located within the caution area, the first output unit 150 determines that there is no need to output the control information based on the first behavior information and the second behavior information. That is, the output device 10 is placed in a non-standby state.

In addition, in this example, as in Example 3, the determination unit 140 determines each fact that is the basis for determining that the first output unit 150 should output control information based on the first behavior information and the second behavior information. Reference information may also be prepared. Then, the first output unit 150 may generate control information using reference information corresponding to the fact that is the basis for the determination, and output the control information.

As described above, according to the present example, similarly to the embodiment, the first output unit 150 outputs control information for controlling the traveling of the first moving object based on the first behavior information and the second behavior information. . Therefore, communication with other moving bodies is carried out smoothly, and smooth automatic driving is realized.

In addition, according to the present embodiment, when the first moving object is located within the caution area, the first output section 150 outputs control information based on the first behavior information and the second behavior information. It is determined that it should be output. Therefore, it is possible to expand the range in which automatic driving can be performed smoothly.

(Example 6)
FIG. 8 is a block diagram illustrating the functional configuration of the output device 10 according to the sixth embodiment. The output device 10 according to this embodiment is the same as the output device 10 according to at least one of the first to fifth embodiments, except for the points described below.

The output device 10 according to the present embodiment further includes a second output section 152 and a third output section 153 for notifying the passenger. Furthermore, the output device 10 according to the present embodiment further includes a second input section 162 that can receive input from the passenger. In the example of FIG. 4, the monitor 422 and speaker 425 function as the second output section 152. Further, the monitor 422 and the speaker 425 also function as the third output section 153. The input panel 423 and the microphone 424 function as the second input section 162. The output of the second output unit 152 and the output of the third output unit 153 may be audio, screen display, AR (Augmented Reality) display, or the like.

In this embodiment, the second output unit 152 outputs notification information for notifying the passenger of the content of the control information generated by the first output unit 150. The third output unit 153 outputs information for prompting the occupant of the first moving object to behave. Further, the second input unit 162 can accept input that changes the content of the control information. If there is a problem with the next operation indicated by the control information generated by the first output unit 150, the passenger of the first moving object inputs to the second input unit 162 that the next operation is to be canceled. Can be done. Thereby, a smooth handover is performed between the first moving body and the second moving body. This will be explained in detail below.

In this embodiment, the first moving body is equipped with an automatic driving device. For example, if the first mobile object approaches a crosswalk and it is detected that there is a pedestrian (second mobile object) near the crosswalk, the first mobile object will prevent the pedestrian from crossing based on automatic driving. Stop before the stop line near the crosswalk to wait.

Here, if the pedestrian does not start crossing the crosswalk, the automatic driving device outputs assistance request information. Then, the determination unit 140 puts the output device 10 into a standby state.

In this embodiment, when the output device 10 enters the standby state, the third output unit 153 outputs information urging the passenger of the first moving object to communicate with the second moving object. Information that promotes communication will also be referred to as "request information" below. The request information is information for prompting the passenger of the first moving object to behave. For example, the request information is output by voice guidance or screen display such as "A pedestrian has been detected near the sidewalk. Could you please encourage the pedestrian to cross?" Further, the third output unit 153 may output a combination of information indicating the state of the first moving object, such as "The vehicle will be stopped to wait for a pedestrian to cross." Further, the output of the third output unit 153 may be an AR display that emphasizes the second moving object.

Based on the request information, the passenger of the first moving object attempts to communicate with the second moving object. Then, the output device 10 detects the first behavior and the second behavior. Here, the first acquisition unit 110 may acquire information indicating the behavior of the passenger that occurs within a predetermined period from the output of the request information as the first behavior information.

Next, the first output unit 150 generates control information for controlling the travel of the first moving object based on the first behavior information and the second behavior information.

FIG. 9 is a flowchart illustrating processing performed by the output device 10 according to the sixth embodiment. The processing performed by the output device 10 is the same as the processing performed by the output device 10 according to at least one of the first to fifth embodiments, except for the points described below.

In this embodiment, if the output device 10 is in a standby state in step S100, the third output unit 153 presents request information to the passenger of the first moving object in step S150. Then, communication is performed between the passenger of the first moving object and the second moving object, and step S200 and step S300 are performed similarly to the first embodiment. However, in step S200, the first acquisition unit 110 may acquire, as the first behavior information, information indicating the behavior of the passenger that occurs within a predetermined period from the output of the request information.

When the second behavior information is acquired in step S300 (Y in step S300), the first output unit 150 generates control information in step S501 in the same manner as step S500 in the first embodiment. Then, in step S502, the second output unit 152 outputs notification information for notifying the passenger of the content of the control information generated by the first output unit 150.

Next, in step S503, the passenger inputs into the second input section 162 whether or not to cancel the next operation indicated in the output notification information. If an input to cancel is made to the second input unit 162 (Y in step S503), the first output unit 150 stops outputting the control information (step S505), and returns to step S100 again. In this case, the second output unit 152 may further notify the passenger to input an alternative action. Upon receiving the notification, the passenger makes an input indicating an alternative action to the second input section 162, and the first output section 150 outputs control information based on the input contents to the second input section 162.

On the other hand, when the second input unit 162 is inputted to not cancel (N in step S503), the first output unit 150 outputs the generated control information (step S504). Then, the process returns to step S100 again.

Note that in step S503, the passenger inputs into the second input unit 162 whether or not to cancel the next operation indicated in the output notification information. The user may input into the second input section 162 whether or not to approve the next action indicated in the information.

As described above, according to the present example, similarly to the embodiment, the first output unit 150 outputs control information for controlling the traveling of the first moving object based on the first behavior information and the second behavior information. . Therefore, communication with other moving objects is carried out smoothly, and smooth automatic driving is realized.

In addition, according to this embodiment, the second output unit 152 outputs notification information for notifying the passenger of the content of the control information generated by the first output unit 150. Therefore, the first moving object is controlled by grasping the result of communication between the passenger of the first moving object and the second moving object with higher accuracy.

Although the embodiments and examples have been described above with reference to the drawings, these are merely illustrative of the present invention, and various configurations other than those described above may be adopted. For example, in the sequence diagrams and flowcharts used in the above description, a plurality of steps (processes) are described in order, but the order in which the steps are executed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed within a range that does not affect the content. Further, each of the embodiments and examples described above can be combined as long as the contents do not conflict with each other.

Below, examples of reference forms will be added.
1-1. a first acquisition unit that acquires first behavior information indicating a first behavior of a passenger of the first moving object;
a second acquisition unit that acquires second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior;
a first output unit that outputs control information for controlling travel of the first mobile body based on the first behavior information and the second behavior information;
An output device comprising:
1-2. 1-1. In the output device described in
The first acquisition unit further acquires third behavior information indicating a third behavior of the passenger within a predetermined period from the occurrence of the second behavior,
The first output unit is an output device that outputs the control information based on the third behavior information.
1-3. 1-1. Or 1-2. In the output device described in
further comprising a determination unit that determines whether the first output unit should output the control information based on the first behavior information and the second behavior information,
The first output unit is an output device that outputs the control information when the determination unit determines that the control information should be output.
1-4. 1-3. In the output device described in
The determination unit is configured such that when the first moving object is located within a predetermined range from a crosswalk or an intersection, the first output unit outputs the control information based on the first behavior information and the second behavior information. Output device that is determined to be output.
1-5. 1-3. Or 1-4. In the output device described in
The determination unit is configured to cause the first output unit to output the control information based on the first behavior information and the second behavior information when the speed of the first moving object becomes equal to or lower than a predetermined speed. Output device that is determined to be appropriate.
1-6. 1-3. From 1-5. In the output device described in any one of
further comprising a third acquisition unit that acquires information indicating a caution area where the first output unit should output the control information based on the first behavior information and the second behavior information,
The determination unit determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the first moving object is located within the caution area. Output device to judge.
1-7. 1-3. From 1-6. In the output device described in any one of
The determination unit is configured to cause the first output unit to output the control information based on the first behavior information and the second behavior information when the width of the road on which the first moving body travels satisfies a predetermined condition. Output device that determines that should be output.
1-8. 1-1. From 1-7. In the output device described in any one of
In the output device, the first behavior is a gesture or utterance by a passenger of the first moving body toward the second moving body.
1-9. 1-1. From 1-8. In the output device described in any one of
The first moving body is a vehicle, the second moving body is an oncoming vehicle of the first moving body,
The second behavior is a behavior in which the second moving body flashes its headlights, hazard lamps, or brake lights one or more times, or sounds a horn at least once.
1-10. 1-1. From 1-9. In the output device described in any one of
The first acquisition unit is capable of accepting input from the passenger to the first input unit,
an output device in which the first behavior is an input to the first input section;
1-11. 1-1. From 1-10. In the output device described in any one of
further comprising a detection unit that detects behavior of the second moving body,
The second acquisition unit acquires the second behavior information from the detection unit,
The detection unit is
acquiring an image that includes at least a portion of the second moving body;
An output device that detects the second behavior using the image and generates the second behavior information.
1-12. 1-1. From 1-11. In the output device described in any one of
The output device further includes a second output unit that outputs notification information for notifying the passenger of the content of the control information.
1-13. 1-1. From 1-12. In the output device described in any one of
An output device further comprising a second input section capable of receiving an input for changing the content of the control information.
1-14. 1-1. From 1-13. In the output device described in any one of
The output device further includes a third output unit that outputs request information for prompting the passenger to behave.
1-15. 1-14. In the output device described in
The first acquisition unit is an output device that acquires information indicating behavior of the passenger that occurs within a predetermined period from output of the request information as the first behavior information.
2-1. a first acquisition step of acquiring first behavior information indicating a first behavior of a passenger of the first moving object;
a second acquisition step of acquiring second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior;
a first output step of outputting control information for controlling travel of the first mobile body based on the first behavior information and the second behavior information;
Output methods including.
2-2. 2-1. In the output method described in
further acquiring third behavior information indicating a third behavior of the passenger within a predetermined period from the occurrence of the second behavior;
In the first output step, the output method further includes outputting the control information based on the third behavior information.
2-3. 2-1. Or 2-2. In the output method described in
further comprising a determination step of determining whether or not the control information should be output in the first output step based on the first behavior information and the second behavior information,
An output method for outputting the control information in the first output step when it is determined in the determination step that the control information should be output.
2-4. 2-3. In the output method described in
In the determination step, when the first moving object is located within a predetermined range from a crosswalk or an intersection, the first output step outputs the control information based on the first behavior information and the second behavior information. Output method determined to be output.
2-5. 2-3. or 2-4. In the output method described in
In the determination step, when the speed of the first moving object becomes equal to or lower than a predetermined speed, the first output step outputs the control information based on the first behavior information and the second behavior information. Output method to judge as exponent.
2-6. 2-3. From 2-5. In the output method described in any one of
further comprising a third acquisition step of acquiring information indicating a caution area for which the control information should be output based on the first behavior information and the second behavior information in the first output step,
In the determining step, when the first moving body is located within the caution area, the control information should be output in the first output step based on the first behavior information and the second behavior information. Output method to judge.
2-7. 2-3. From 2-6. In the output method described in any one of
In the determination step, when the width of the road on which the first mobile object travels satisfies a predetermined condition, the first output step outputs the control information based on the first behavior information and the second behavior information. Output method that determines that should be output.
2-8. 2-1. From 2-7. In the output method described in any one of
In the output method, the first behavior is a gesture or a voice by a passenger of the first moving body toward the second moving body.
2-9. 2-1. From 2-8. In the output method described in any one of
The first moving body is a vehicle, the second moving body is an oncoming vehicle of the first moving body,
The output method is such that the second behavior is a behavior in which the second moving body flashes its headlights, hazard lamps, or brake lights one or more times, or sounds a horn at least once.
2-10. 2-1. From 2-9. In the output method described in any one of
In the first acquisition step, it is possible to receive input from the passenger into a first input section,
An output method in which the first behavior is an input to the first input unit.
2-11. 2-1. From 2-10. In the output method described in any one of
further comprising a detection step of detecting behavior of the second moving body,
In the second acquisition step, acquire the second behavior information generated in the detection step,
In the detection step,
acquiring an image that includes at least a portion of the second moving body;
An output method that detects the second behavior using the image and generates the second behavior information.
2-12. 2-1. From 2-11. In the output method described in any one of
The output method further includes the step of outputting notification information for notifying the passenger of the content of the control information.
2-13. 2-1. From 2-12. In the output method described in any one of
An output method further comprising the step of accepting an input for changing the content of the control information.
2-14. 2-1. From 2-13. In the output method described in any one of
An output method further comprising a third output step of outputting request information for prompting the passenger to behave.
2-15. 2-14. In the output method described in
In the first acquisition step, information indicating the behavior of the passenger occurring within a predetermined period from output of the request information is acquired as the first behavior information.
3-1. A computer program for realizing an output device,
computer,
a first acquisition means for acquiring first behavior information indicating a first behavior of a passenger of the first moving body;
a second acquisition means for acquiring second behavior information indicating a second behavior of a second moving object around the first moving object within a predetermined period from the occurrence of the first behavior; a first output means for outputting control information for controlling travel of the first moving body based on the information and the second behavior information;
A computer program that functions as
3-2. 3-1. In the computer program described in
The first acquisition means further acquires third behavior information indicating a third behavior of the passenger within a predetermined period from the occurrence of the second behavior,
The first output means is a computer program that outputs the control information further based on the third behavior information.
3-3. 3-1. Or 3-2. In the computer program described in
further causing the computer to function as a determining means for determining whether the first output means should output the control information based on the first behavior information and the second behavior information;
A computer program in which the first output means outputs the control information when the determination means determines that the control information should be output.
3-4. 3-3. In the computer program described in
The determination means is configured such that when the first moving object is located within a predetermined range from a crosswalk or an intersection, the first output means outputs the control information based on the first behavior information and the second behavior information. A computer program that determines what should be output.
3-5. 3-3. or 3-4. In the computer program described in
The determining means may cause the first outputting means to output the control information based on the first behavior information and the second behavior information when the speed of the first moving object becomes equal to or lower than a predetermined speed. A computer program that determines what should be done.
3-6. 3-3. From 3-5. In the computer program described in any one of
further causing the computer to function as a third acquisition means for acquiring information indicating a caution area for which the first output means should output the control information based on the first behavior information and the second behavior information;
The determination means determines that the first output means should output the control information based on the first behavior information and the second behavior information when the first moving object is located within the caution area. A computer program that makes a determination.
3-7. 3-3. From 3-6. In the computer program described in any one of
The determination means is configured to cause the first output means to output the control information based on the first behavior information and the second behavior information when the width of the road on which the first moving body travels satisfies a predetermined condition. A computer program that determines that should be output.
3-8. 3-1. From 3-7. In the computer program described in any one of
The computer program, wherein the first behavior is a gesture or utterance by a passenger of the first moving body toward the second moving body.
3-9. 3-1. From 3-8. In the computer program described in any one of
The first moving body is a vehicle, the second moving body is an oncoming vehicle of the first moving body,
The second behavior is a computer program in which the second moving object flashes its headlights, hazard lamps, or brake lights one or more times, or sounds its horn one or more times.
3-10. 3-1. From 3-9. In the computer program described in any one of
The first acquisition means is capable of accepting input from the passenger into the input means,
A computer program in which the first behavior is an input to the input means.
3-11. 3-1. From 3-10. In the computer program described in any one of
further causing the computer to function as a detection means for detecting behavior of the second moving object,
The second acquisition unit acquires the second behavior information from the detection means,
The detection means includes:
acquiring an image that includes at least a portion of the second moving body;
A computer program that detects the second behavior using the image and generates the second behavior information.
3-12. 3-1. From 3-11. In the computer program described in any one of
A computer program that causes a second output means to output notification information for notifying the passenger of the content of the control information.
3-13. 3-1. From 3-12. In the computer program described in any one of
A computer program capable of accepting input to a second input means for changing the content of the control information.
3-14. 3-1. From 3-13. In the computer program described in any one of
A computer program that causes a third output means to output request information for prompting the passenger to behave.
3-15. 3-14. In the computer program described in
The first acquisition means is a computer program that acquires, as the first behavior information, information indicating the behavior of the passenger that occurs within a predetermined period from the output of the request information.

10 Output device 40 Integrated circuit 110 First acquisition section 112 First detection section 120 Second acquisition section 122 Second detection section 130 Third acquisition section 140 Determination section 150 First output section 152 Second output section 153 Third output section 160 First input section 162 Second input section 170 Storage section 402 Bus 404 Processor 406 Memory 408 Storage device 410 Input/output interface 412 Network interface 420 First camera 421 Second camera 422 Monitor 423 Input panel 424 Microphone 425 Speaker

Claims (1)

a first acquisition unit that acquires first behavior information indicating a first behavior of a passenger of the first moving object;
a second acquisition unit that acquires second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior;
a first output unit that outputs control information for controlling travel of the first mobile body based on the first behavior information and the second behavior information;
An output device comprising:

Images