JP7459821B2 - Remote support system, remote support method, and remote support program - Google Patents

Description

The present disclosure relates to a remote support system, a remote support method, and a remote support program.

As shown in Non-Patent Document 1, automatic door opening/closing technology is used in self-driving vehicles. Automatic opening/closing basically opens and closes the door autonomously, and constantly monitors the situation based on central command. Additionally, in the event of an emergency, the central control center intervenes to open and close the doors, allowing operators to directly operate them remotely.

“Facilities, Mechanisms, and Systems – Yurikamome Co., Ltd.” URL: https://www.yurikamome.co.jp/feature/comfortable/system.html (Retrieved February 9, 2021)

When applying automatic door opening and closing technology to an autonomous bus, there is a risk of collisions when passengers disembark depending on the location of the bus stop. In addition, the operator may decide to suspend the opening and closing of the doors, and this decision is made by the operator based on the video transmitted to the remote support system. However, there are cases where there is a large delay in the transmitted video, and if there is a delay in the video, the operator's decision to suspend the operation will be delayed, which could put passengers disembarking in a dangerous situation.

In addition, when operating an LV4 unmanned autonomous vehicle as a service, it is necessary to autonomously open the door after confirming the safety of the surrounding area, but autonomous door opening may not be possible due to misrecognition by the recognition mechanism. It is assumed that this will become possible. At that time, it is thought that the operator will need to view the video from a remote location, confirm the safety of the area around the door, and then directly operate the door to open it. However, if the provided video is delayed, it becomes difficult to perform operations quickly while ensuring safety.

The present disclosure has been made in view of the above circumstances, and aims to provide a remote support system, a remote support method, and a remote support program that can control a door to support operator operations even if a video delay occurs. purpose.

The remote support system according to the present disclosure includes a calculation unit that calculates the delay amount of the video transmitted from the vehicle to the center, and a calculation unit that calculates the amount of delay of the video transmitted from the vehicle to the center, and the calculation unit that calculates the door opening method regarding at least one of the door opening speed and the door opening timing. and a control unit that performs control such that the delay increases as the amount of delay increases.

The remote assistance system, remote assistance method, and remote assistance program disclosed herein can control the door to assist the operator in operating the device even if there is a video delay.

FIG. 13 is a diagram showing an example of a change in door opening depending on the amount of video delay. FIG. 1 is a block diagram showing the configuration of a remote support system according to a first embodiment of the present disclosure. FIG. 13 is a diagram showing an example of a graph showing a change in door opening speed with respect to the amount of delay. FIG. 2 is a block diagram showing a hardware configuration of the remote support system. It is a flowchart showing the flow of remote support processing by the remote support system of the first embodiment. FIG. 2 is a diagram schematically showing the physique of passengers when viewing an image inside the car from a bird's eye view. FIG. 11 is a block diagram showing a configuration of a remote assistance system according to a second embodiment of the present disclosure. FIG. 13 is a diagram showing an example of a graph of changes in damping rate with respect to the occupant's body size. 13 is a flowchart showing a flow of a remote support process by the remote support system of the second embodiment. FIG. 2 is a block diagram showing the configuration of a remote support system according to a third embodiment of the present disclosure. It is a figure which shows an example of the graph of the change of a decay rate with respect to risk. It is a flow chart which shows the flow of remote support processing by a remote support system of a 3rd embodiment. FIG. 13 is a block diagram showing a configuration of a remote assistance system according to a fourth embodiment of the present disclosure. FIG. 13 is a diagram showing an example of a case where a moving object is present near the disembarking area. It is a figure which shows an example when a moving object does not exist in the vicinity of an alighting part. FIG. 13 is a diagram showing an example of a graph showing changes in damping rate for each parameter of a moving object; It is a flow chart which shows the flow of remote support processing by a remote support system of a 4th embodiment.

Each embodiment of the present disclosure will be described below with reference to the drawings.

In the technology disclosed herein, the speed at which the door is opened (hereinafter, door opening speed) is changed according to the amount of delay in the video in relation to the door opening control (hereinafter, door opening). For example, the greater the amount of delay in the video, the slower the door opening speed is set, which allows time for interruption before an accident or injury occurs, such as a collision with a moving object such as a bicycle traveling near the disembarking area, or being caught in the vehicle. In addition, as a variation of the embodiment, the door opening speed can be changed based on each of the passenger's physique, the risk of the disembarking area, and the parameters of the moving object present near the disembarking area. Note that the timing at which the door is opened (hereinafter, door opening timing) may be changed in relation to the way the door is opened, rather than the door opening speed. This allows the way the door is opened to be controlled so that the door opens within a range that allows the operator to interrupt the operation according to the communication delay. In addition, even if there is a communication delay in the video, the operator can quickly perform the operation while ensuring safety.

Here, the basic flow will be explained. Figure 1 is a diagram showing an example of changing the door opening according to the delay amount of the video. In Figure 1, it is assumed that there are flows (1) to (3) when opening the door. In (1), a request for assistance is made to the operator due to difficulties in automatic opening. Here, it is assumed that a delay amount d occurs in the video transmitted to the remote support system. In (2), it is assumed that the operator checks the safety around the door based on the video and remotely instructs the door to open and close. In (3), when the door is opened by the vehicle according to the instruction, the door opening speed s is changed according to the delay amount d, and the door opening is controlled by the changed door opening speed s. The door opening speed is determined by the function s = f(d). The door opening speed may be changed by either a person or a system that has learned human behavior. In addition, if the door opening timing is used for control instead of the door opening speed, the door opening timing is determined by the function t = f(d) like the door opening speed.

First Embodiment
The configuration of the remote support system according to the first embodiment of the present disclosure will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the remote support system according to the first embodiment of the present disclosure. As shown in Fig. 2, the remote support system 100 includes a storage unit 102, a transmission/reception unit 110, a delay calculation unit 112, and a control unit 120. The remote support system 100 communicates with a vehicle 90 via a network N. The vehicle 90 is, for example, an autonomous driving bus.

The storage unit 102 stores a door opening speed (hereinafter referred to as reference speed) that is a fixed reference. Note that door opening speeds may be set for each vehicle type and location. The operator selects the reference speed stored in the storage unit 102 during remote support. Note that the door opening speed here is an example of the initial door opening speed _sinit described later. When timing is used, the reference timing for opening the door is stored.

The transmitting/receiving unit 110 receives a support request and an image inside the vehicle from the vehicle 90 at appropriate timings. The transmitter/receiver 110 also receives operation instructions from the operator. The transmitting/receiving unit 110 transmits an operation instruction and a door opening command including a door opening speed to the vehicle 90. Note that in the remote support system 100, when there is a request for support, necessary notification is given to the operator, but since this is not the main control, the explanation will be omitted here.

The delay calculation unit 112 acquires the communication conditions when the video is received, and calculates the delay amount d of the video transmitted from the vehicle 90 to the center. The delay amount may be calculated as a delay time appropriately based on the communication volume in the communication conditions and noise in the video, etc. The delay amount may be the degree of delay.

The control unit 120 calculates the door opening speed s based on the delay amount d and generates a door opening command. FIG. 3 is a diagram showing an example of a graph of changes in door opening speed with respect to delay amount. As shown in FIG. 3, the function f(d) for determining the door opening speed s is designed so that the door opening speed s becomes slower from the initial door opening speed s _init in proportion to the delay amount d, that is, the delay amount d It may be defined so that the speed decreases in proportion to. Note that the degree of decrease in speed due to an increase in the amount of delay d is merely an example, and may be appropriately defined to decrease in stages for each certain threshold value. In this way, the control unit 120 controls the opening of the door to be delayed as the delay amount d becomes larger.

In vehicle 90, the door is controlled to open at door opening speed s based on the received door opening command.

The remote support system 100 can be realized, for example, by a server, and the type of server can be an appropriate configuration such as cloud. FIG. 4 is a block diagram showing the hardware configuration of the remote support system 100. As shown in FIG. 4, the remote support system 100 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, storage 14, an input unit 15, a display unit 16, and a communication interface (I/F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads the remote assistance program from the ROM 12 or the storage 14, and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various calculation processes according to the remote assistance program stored in the ROM 12 or the storage 14. In this embodiment, the remote assistance program is stored in the ROM 12 or the storage 14.

The ROM 12 stores various programs and data. The RAM 13 temporarily stores programs or data as a work area. The storage 14 is constituted by a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system and various data.

The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

The display unit 16 is, for example, a liquid crystal display, and displays various information. The display section 16 may employ a touch panel system and function as the input section 15.

The communication interface 17 is an interface for communicating with other devices such as terminals, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

The above is an example of the hardware configuration of the remote support system 100.

Next, the operation of the remote support system 100 will be described. FIG. 5 is a flowchart showing the flow of remote support processing by the remote support system 100 of the first embodiment. The CPU 11 reads out a remote support program from the ROM 12 or storage 14, expands it into the RAM 13, and executes it to perform the remote support processing. The CPU 11 functions as each part of the remote support system 100.

In step S100, the CPU 11 receives a request for assistance from the vehicle 90.

In step S102, the CPU 11 receives video from the vehicle 90 in response to the support request.

In step S104, the CPU 11 acquires the communication conditions at the time of receiving the video and calculates the delay d of the video transmitted from the vehicle 90 to the center.

In step S106, the CPU 11 obtains the door opening reference speed selected by the operator.

In step S108, the CPU 11 calculates the door opening speed s based on the reference door opening speed and the delay amount d, and generates a door opening command.

In step S110, the CPU 11 sends a door open command to the vehicle 90.

As described above, according to the remote support system 100 according to the first embodiment of the present disclosure, the door can be controlled to support the operator's operation even if a video delay occurs.

Second Embodiment
Next, a second embodiment will be described. In the second embodiment, door opening is controlled according to the physique of the passenger. The physique differs from person to person, not limited to attributes such as child or gender. If the person is large, he/she cannot get off the vehicle unless the door is fully opened, whereas if the person is small, he/she can get off even if the door is not fully opened. Therefore, the door is controlled to open quickly for large people and to open slowly for small people. Note that the same reference numerals are used for parts similar to those in the first embodiment, and description thereof will be omitted.

FIG. 6 is a diagram schematically showing the physiques of passengers when viewing an image inside the car from a bird's eye view. As shown in the upper part of FIG. 6, passengers are grouped into groups with different physiques: large, medium, and small, and presented to the operator to select the passenger expected to arrive first. Alternatively, the moving direction and speed of the passenger may be calculated from the video, and the passenger predicted to arrive first may be calculated and automatically selected by the system. Then, depending on the physique of the selected passenger, the door opening speed is calculated by taking into account s=f(d)+s _b and the attenuation speed s _b of the physique. For example, s _b is determined as s _fast , s _mid , s _slow , etc. for large, medium, and small physiques. Note that, as shown in the lower part of FIG. 6, when all physiques are grouped within the same physique, the physique may be automatically selected without requiring the operator to select the physique. In this manner, selection by the operator and automatic selection may be used depending on the grouping result. Further, depending on the operating status of the operator, if the operator is unable to cope with the situation, such as when the operator is under pressure, it is possible to select the physique as appropriate, such as automatically selecting it by calculation. Alternatively, automatic selection may be performed all the time.

FIG. 7 is a block diagram showing the configuration of a remote support system according to a second embodiment of the present disclosure. As shown in FIG. 7, the remote support system 200 includes a storage section 102, a transmitter/receiver section 110, a delay calculation section 112, a physique calculation section 214, and a control section 220.

The physique calculation unit 214 groups the physique of each passenger from the video. Then, the physique calculation unit 214 selects the physique of the passenger who will disembark. Here, the physique of each grouped passenger is presented to the operator, and the operator is prompted to select the passenger who is expected to disembark first from among those heading toward the door. The physique calculation unit 214 may also calculate the direction and speed of movement of the passengers from the video, and calculate and automatically select the passenger who is expected to arrive first. In this way, the physique is selected and controlled by the operator's selection of the passenger, or by automatic selection based on the calculation of the passenger's movement.

The control unit 220 calculates the door opening speed s based on the selected physique in addition to the delay amount, and generates a door opening command. Here, the door opening speed s is calculated by taking into account s=f(d)+s _b and the attenuation speed s _b of the body. In the case of addition, the attenuation speed s _b may be a negative value, and may be calculated after being weighted. FIG. 8 is a diagram showing an example of a graph of changes in damping speed with respect to the passenger's physique. In this case, the attenuation speed is reduced as the body size increases. Note that the degree of decrease in the attenuation speed is just an example, and can be defined as appropriate in the same way as the speed shown in FIG.

FIG. 9 is a flowchart showing the flow of remote support processing by the remote support system 200 of the second embodiment.

After step S104, in step S200, the CPU 11 groups the physique of each passenger from the video.

In step S202, the CPU 11 selects the physique of the passenger getting off the vehicle. The physique of the passenger is selected by the operator's selection of the passenger to alight (or automatic selection based on calculation of the passenger's movement). Thereafter, in step S108, in addition to the delay amount, a door opening speed s is calculated in consideration of the attenuation speed b based on the selected physique, and a door opening command is generated.

As described above, the remote assistance system 200 according to the second embodiment of the present disclosure can control the doors to assist the operator in operating the vehicle, taking into account video delay and the physique of the passenger.

(Third embodiment)
Next, a third embodiment will be described. The third embodiment is a case where door opening is controlled depending on the risk of the exit location. If it is a self-driving bus, depending on the type of bus stop it stops at, it may not be necessary to pay attention to ensuring safety after getting off the bus. Therefore, by assigning a risk r to each bus stop and taking it into account when calculating the door opening speed s, if it is known in advance that it is possible to get off the bus safely, smooth operation can be achieved without reducing the door opening speed too much. It becomes possible. Note that parts that are similar to those in the first and second embodiments are denoted by the same reference numerals, and description thereof will be omitted.

FIG. 10 is a block diagram showing the configuration of a remote support system according to a third embodiment of the present disclosure. As shown in FIG. 10, the remote support system 300 includes a storage section 102, a transmitter/receiver section 110, a delay calculation section 112, a physique calculation section 214, and a control section 320. The storage unit 102 includes a risk storage unit 304. Note that the speed may or may not be calculated by combining the attenuation speed s _b of the physique.

The risk for each alighting location is stored in the risk storage unit 304. The risk is r, and the decay rate of the risk r is _sr . If the risk of opening the door at a bus stop where it is safe to alight, such as a bus terminal, is r, and the risk of opening the door at a bus stop with heavy traffic is r', then the risk is defined as s=f(d)+ _sr >s=f(d)+sr _' .

Note that the transmitting/receiving unit 110 receives the support request including the drop-off location.

The control unit 320 calculates the door opening speed s based on the risk of the alighting location in addition to the amount of delay, and generates a door opening command. Here, the door opening speed s is calculated taking into account s=f(d)+ _sr and the risk r. Alternatively, it may be calculated as s=f(d)+s _b /g+s _r /g (g=s _b +s _r ), taking into consideration the selected physique. In this case, the physique attenuation rate s _b and the risk attenuation rate s _r may be weighted respectively.

The risk of the drop-off location may be obtained from the risk memory unit 304, or may be recalculated by taking into account the obtained risk and the current situation from images of the drop-off location, etc. Figure 11 shows an example of a graph of the change in the decay rate relative to risk. In this case, the decay rate is decreased as the safety level, which is an example of risk, increases. Note that the degree of decrease in the decay rate is an example, and can be defined appropriately in the same way as the rate shown in Figure 3.

FIG. 12 is a flowchart showing the flow of remote support processing by the remote support system 300 of the third embodiment.

After step S202, in step S300, the CPU 11 acquires the risk of the drop-off location from the risk memory unit 304. Then, in step S108, the CPU 11 calculates the door opening speed s in consideration of the risk-based attenuation speed _sr in addition to the delay amount, and generates a door opening command.

As described above, according to the remote support system 300 according to the third embodiment of the present disclosure, the door can be controlled to support the operator's operation while taking into account the video delay and the risk of the exit location.
(Fourth embodiment)
Next, a fourth embodiment will be described. The fourth embodiment is a case where door opening is controlled depending on a moving object existing near the exit section. If there are passersby or moving objects such as bicycles near the exit area, there is a risk of a collision. Therefore, by calculating the moving object parameters such as the speed, acceleration, and distance of the moving object and taking them into account when calculating the door opening speed s, it becomes possible to operate the vehicle in consideration of the situation at the exit section. Note that parts that are the same as those in the first to third embodiments are given the same reference numerals and descriptions thereof will be omitted.

Fig. 13 is a block diagram showing a configuration of a remote support system according to a fourth embodiment of the present disclosure. As shown in Fig. 13, a remote support system 400 includes a storage unit 102, a transmission/reception unit 110, a delay calculation unit 112, a physique calculation unit 214, a moving object parameter calculation unit 416, and a control unit 320. Note that the speed may or may not be calculated by combining the physique decay speed s _b and the risk decay speed s _r .

When a moving object is present within the adjustment range near the disembarkation area, the moving object parameter calculation unit 416 calculates a decay rate s _p of the moving object parameter. The moving object parameter p is calculated by combining the decay rates defined for the parameters of the moving object's speed s _obs , the moving object's acceleration a _obs , and the distance d _obs .

FIG. 14 is a diagram illustrating an example of a case where a moving object is present near the alighting section. The remote support system 400 acquires images or sensor information from a center or the like near the disembarkation area. The moving object parameter calculation unit 416 calculates the distance d _obs to the moving object from the sensor information. Then, the moving object parameter calculation unit 416 determines whether the distance d _obs of the moving object is within the adjustment range j of the disembarking section of the vehicle 90. If it is within the range, the attenuation speed _sp of the moving object parameter is calculated, and if it is not within the range, no calculation is performed. FIG. 15 is a diagram illustrating an example of a case where there is no moving object near the exit section. In this way, if it is outside the adjustment range j, the attenuation speed _sp of the moving body parameter is not calculated. Note that if there are multiple moving objects, the calculation may be performed for the closest moving object.

Fig. 16 is a diagram showing an example of a graph of the change in damping speed with respect to each parameter of the moving object. In this case, the damping speed is defined so that the door opening speed s decreases as i) the speed increases, ii) the acceleration increases, and iii) the distance from the door decreases. Note that the degree of decrease in damping speed is an example, and can be appropriately defined in the same manner as the speed shown in Fig. 3. Note that the damping speed s _p can be calculated by combining the parameters of the moving object without using all of them.

The control unit 420 calculates the door opening speed _s based on the delay amount and the decay speed s of the moving object parameter, and generates a door opening command. Here, the door opening speed s is calculated in consideration of s=f(d)+ _sp and the decay speed _s of the moving object parameter p. Note that the selected physique and risk may also be taken into consideration and the calculation may be s=f(d)+ _sb /g+ _sr /g+ _sp /g (g= _sb + _sr + _sp ). In this case, the decay speed _sb of the physique, the decay speed _sr of the risk, and the decay speed _sp of the moving object parameter may each be weighted.

Figure 17 is a flowchart showing the flow of remote support processing by the remote support system 400 of the fourth embodiment.

After step S300, in step S400, the CPU 11 acquires sensor information and calculates the distance d _obs to the moving object from the sensor information.

In step S402, the CPU 11 determines whether the distance d _obs of the moving object is within the adjustment range j of the disembarking section of the vehicle 90. If it is within the range, the process moves to step S404, and if it is not within the range, the process moves to step S106.

In step S404, the CPU 11 calculates the decay speed _sp of the moving object parameter.

Thereafter, in step S108, if the decay speed s _p of the moving object parameter has been calculated, the door opening speed s is calculated taking into consideration the decay speed s _p of the moving object parameter in addition to the delay amount, and a door opening command is generated. If the decay speed s _p of the moving object parameter has not been calculated, the door opening speed s is calculated without taking it into consideration.

As described above, according to the remote support system 400 according to the fourth embodiment of the present disclosure, the door can be controlled to support the operator's operation, taking into account the video delay and the moving object parameters.

Note that the present disclosure is not limited to the embodiments described above, and various modifications and applications are possible without departing from the gist of the present invention.

Furthermore, although the present specification has been described as an embodiment in which a program is installed in advance, it is also possible to provide the program by storing it in a computer-readable recording medium.

100, 200, 300, 400 Remote support system 102 Storage unit 110 Transmitter/receiver unit 112 Delay calculation unit 120, 220, 320, 420 Control unit 214 Physical size calculation unit 304 Risk storage unit 416 Mobile object parameter calculation unit

Claims (8)

a calculation unit that calculates a delay amount of a video transmitted from the vehicle to a center;
a control unit that controls a door opening manner related to at least one of a door opening speed and a door opening timing so as to delay the door opening manner as the calculated delay amount increases;
A remote support system including: The control unit includes:
When the door opening method is set to the speed, the speed is calculated to be slower from the reference as the delay amount becomes larger;
2. The remote support system according to claim 1, wherein when the timing is set to be the timing for opening the door, the timing is calculated to be delayed from a reference as the amount of delay increases. 3. The remote support system according to claim 1, wherein the control unit controls how to open the door based on the physique of the passenger heading toward the door, in addition to the amount of delay. When using the physique for the control,
4. The remote support system according to claim 3, wherein the control is performed by selecting the physique through passenger selection by an operator or automatic selection based on calculation of passenger movement. 5. The control unit controls, in addition to the delay amount, based on the risk of the place where the vehicle gets off the vehicle, the greater the risk, the later the opening of the door is delayed. Remote support system as described in section. The remote assistance system according to any one of claims 1 to 5, wherein the control unit controls the door opening to be delayed based on a moving object parameter including at least one of the speed, acceleration, and distance of a moving object present near the disembarking area of the vehicle in addition to the delay amount. Calculate the delay of the video transmitted from the vehicle to the center,
A door opening method related to at least one of a door opening speed and a door opening timing is controlled so as to be delayed as the calculated delay amount increases.
A remote support method for causing a computer to execute processing. Calculate the delay of the video transmitted from the vehicle to the center,
A door opening method related to at least one of a door opening speed and a door opening timing is controlled so as to be delayed as the calculated delay amount increases.
A program that causes a computer to carry out processing.