WO2024252447A1 - 情報提示方法，情報提示システム、及びプログラム - Google Patents

情報提示方法，情報提示システム、及びプログラム Download PDF

Info

Publication number
WO2024252447A1
WO2024252447A1 PCT/JP2023/020764 JP2023020764W WO2024252447A1 WO 2024252447 A1 WO2024252447 A1 WO 2024252447A1 JP 2023020764 W JP2023020764 W JP 2023020764W WO 2024252447 A1 WO2024252447 A1 WO 2024252447A1
Authority
WO
WIPO (PCT)
Prior art keywords
robot
display screen
unit
telepresence robot
person
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/020764
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
充裕 後藤
聡一郎 内田
仁志 瀬下
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NTT Inc
Original Assignee
Nippon Telegraph and Telephone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Telegraph and Telephone Corp filed Critical Nippon Telegraph and Telephone Corp
Priority to PCT/JP2023/020764 priority Critical patent/WO2024252447A1/ja
Priority to JP2025525426A priority patent/JPWO2024252447A1/ja
Publication of WO2024252447A1 publication Critical patent/WO2024252447A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • One aspect of the present invention relates to an information presentation method, an information presentation system, and a program for remote communication.
  • Hybrid work is a work style that combines traditional office work, where people come to the office, with remote work from home or in a shared office. Remote communication is also becoming more common in stores and facilities in commercial spaces.
  • Telepresence robots are attracting attention as a remote communication tool. Telepresence robots are robots that combine videoconferencing functions with remote control technology. Some telepresence robots have mobility capabilities. By using a physical device such as a telepresence robot, the person speaking to the robot can more easily sense the presence of the person operating the telepresence robot in a remote location.
  • Non-Patent Document 1 shows an example of using a telepresence robot in an office space.
  • Non-Patent Document 2 shows an example of using a telepresence robot in a commercial space (store).
  • Non-Patent Document 3 shows the results of a survey on acceptability of robots.
  • the present invention has been made in light of these points.
  • the purpose of the present invention is to reduce psychological resistance to conversation with a robot in an information presentation method for remote communication using a robot.
  • One aspect of the information presentation method is a method of presenting information using an information presentation system including a robot having a moving mechanism, a display screen, and a video calling function, and a remote control device for the robot, and includes displaying an audio waveform of audio acquired by the remote control device on the display screen, moving the robot to a first distance away from the person being spoken to while the audio waveform is displayed on the display screen, displaying a live-action video of the operator of the remote control device on the display screen, moving the robot to a position where the angle with respect to the person being spoken to is a first angle while the live-action video is displayed on the display screen, displaying an avatar of the operator on the display screen, and moving the robot to a second distance away from the position away from the person being spoken to that is shorter than the first distance while the avatar is displayed on the display screen.
  • the information presentation method can control the display screen and the method of approaching a person when the telepresence robot approaches a person and attempts to converse with them in remote communication using a telepresence robot having a screen display function and a movement function. This can improve the person who is spoken to by the telepresence robot's acceptance of the telepresence robot.
  • FIG. 1 is a block diagram showing an example of an overall configuration of an information presentation system according to an embodiment.
  • FIG. 2 is a block diagram showing an example of a circuit configuration of a remote control device included in the information presentation system according to an embodiment.
  • FIG. 3 is a block diagram showing an example of a circuit configuration of a telepresence robot included in an information presentation system according to an embodiment.
  • FIG. 4 is a block diagram illustrating an example of a functional configuration of an information presentation system according to an embodiment.
  • FIG. 5 is a diagram showing a specific example of a screen display on a display screen of a remote control device included in an information presentation system according to an embodiment.
  • FIG. 6 is a diagram showing a specific example of a screen display on a display screen of a telepresence robot included in an information presentation system according to an embodiment.
  • FIG. 7 is a diagram showing a specific example of a direction in which a telepresence robot included in an information presentation system according to an embodiment approaches a local speaker.
  • FIG. 8 is a diagram showing a specific example of the moving direction of a telepresence robot and a screen display on a display screen when the telepresence robot included in an information presentation system according to an embodiment approaches a local speaker.
  • FIG. 9 is a flowchart showing an overall flow of an information presentation method in an information presentation system according to an embodiment.
  • Fig. 1 is a block diagram showing an example of an overall configuration of the information presentation system 1.
  • the information presentation system 1 includes a remote control device 10 and a telepresence robot 20 (hereinafter simply referred to as "robot") that can communicate with the remote control device 10.
  • robot a telepresence robot 20
  • the location where the remote control device 10 is operated is referred to as the "remote location.”
  • the location where the telepresence robot 20 is operating is referred to as the "local location.”
  • the remote control device 10 is, for example, a personal computer.
  • the remote control device 10 may be a tablet terminal such as a smartphone, or may be another computer device.
  • the remote control device 10 has a function of remotely controlling the telepresence robot 20.
  • the remote control device 10 also has a function of performing a video call (video chat) with the telepresence robot 20. For example, an image of the local area is displayed on the display screen 1000 of the remote control device 10.
  • a person operating the remote control device 10 hereinafter referred to as "operator HM1" may operate the telepresence robot 20 via the remote control device 10.
  • the operator HM1 operates the telepresence robot 20 to search for a person to talk to on-site, or to move the telepresence robot 20 closer to the person to talk to.
  • the operator HM1 uses the video call function between the remote control device 10 and the telepresence robot 20 to talk to the person to be spoken to who is facing the telepresence robot 20 (hereinafter referred to as the "local speaker HM2").
  • the telepresence robot 20 includes a moving mechanism 21, a support 22, and a video calling unit 23. Note that the telepresence robot 20 may be a mobile humanoid robot.
  • the moving mechanism 21 moves the telepresence robot 20 based on the operation of the remote control device 10.
  • the moving mechanism 21 includes a base, wheels, a motor for driving (rotating) the wheels, etc.
  • the wheel drive motor (not shown) is installed in the base.
  • two wheels are provided on the sides (left and right) of the cylindrical base. Separate wheel drive motors are attached to the two wheels. The two wheels are controlled independently of each other. By controlling the rotation of the left and right wheels, the telepresence robot 20 can run and turn.
  • the configuration and arrangement of the base and wheels are not limited to this.
  • the base may be a rectangular parallelepiped.
  • three or more wheels may be provided.
  • the support 22 is provided on the moving mechanism 21.
  • a video calling unit 23 is provided on the support 22.
  • the support 22 may be provided with an extension mechanism, and may also be provided with a rotation mechanism for rotating the video calling unit 23 (display screen 2000).
  • the telepresence robot 20 may adjust the length of the support 22 to match the height of the local speaker HM2.
  • the telepresence robot 20 may also change the orientation of the video calling unit 23 (display screen 2000) depending on the traveling direction of the moving mechanism 21 and the positional relationship with the local speaker HM2.
  • the video call unit 23 is used for video calls.
  • the video call unit 23 includes a display screen 2000.
  • An image of a remote location (an image of the operator HM1) is displayed on the display screen 2000.
  • Fig. 2 is a block diagram showing an example of the circuit configuration of the remote control device 10.
  • the remote control device 10 includes a control circuit 11, a memory 12, an input unit 13, a display unit 14, a camera 15, a microphone 16, a speaker 17, and a communication unit 18. These are connected to each other so that they can transmit and receive data.
  • the control circuit 11 is, for example, a CPU (Central Processing Unit).
  • the control circuit 11 controls the entire remote control device 10.
  • the control circuit 11 executes a program stored in the program storage section 121 of the memory 12.
  • the memory 12 includes a program storage unit 121 and a data storage unit 122.
  • the memory 12 may also include various types of storage devices as external storage, such as a magnetic disk storage device (HDD: Hard Disk Drive) or a solid state drive (SSD: Solid State Drive).
  • HDD Hard Disk Drive
  • SSD Solid State Drive
  • the program storage unit 121 may be configured as a non-transient storage medium, for example, a non-volatile memory such as a ROM (Read Only Memory).
  • the program storage unit 121 stores programs used to execute the information presentation system 1.
  • the programs include an operation program for the telepresence robot 20 and a video call application with the telepresence robot 20.
  • the data storage unit 122 may be configured as a storage medium, for example, a volatile memory such as a RAM (Random Access Memory).
  • the data storage unit 122 stores data necessary for executing the information presentation system 1.
  • the input unit 13 includes an input device such as a keyboard, a mouse, a touch panel, or a button switch.
  • the operator HM1 can use the input device to operate the telepresence robot 20.
  • the presentation unit 14 includes a display screen 1000 (e.g., an LCD (Liquid Crystal Display) or an EL (Electroluminescence) display) as a display device.
  • a display screen 1000 e.g., an LCD (Liquid Crystal Display) or an EL (Electroluminescence) display
  • the presentation unit 14 displays an image captured on-site (hereinafter referred to as "on-site image") on the display screen 1000 based on the control of the control circuit 11.
  • Camera 15 is used to capture images of a remote location, i.e., of operator HM1, under the control of control circuit 11.
  • images captured by camera 15 at a remote location will be referred to as "remote location images.”
  • the microphone 16 is used to acquire voice from a remote location (hereinafter referred to as "remote voice"), i.e., the voice of the operator HM1, based on the control of the control circuit 11.
  • remote voice i.e., the voice of the operator HM1
  • the speaker 17 is used to play back audio captured locally (hereinafter referred to as "local audio") under the control of the control circuit 11.
  • the communication unit 18 is used for wireless communication with the telepresence robot 20.
  • the communication unit 18 transmits remote video and remote audio (video and audio of the operator HM1) and operation signals of the telepresence robot 20 to the telepresence robot 20.
  • the communication unit 18 also receives local video and audio from the telepresence robot 20, as well as information acquired by the telepresence robot 20.
  • the microphone 16 and speaker 17 may be built into the remote control device 10, or may be earphones or headphones with a microphone connected to the remote control device 10.
  • Fig. 3 is a block diagram showing an example of the circuit configuration of the telepresence robot 20.
  • the telepresence robot 20 includes a control circuit 30, a memory 31, an input unit 32, a presentation unit 33, a camera 34, a microphone 35, a speaker 36, a communication unit 37, a sensor 38, and a drive control unit 39. These are connected so that they can transmit and receive information to each other.
  • the presentation unit 33, the camera 34, the microphone 35, and the speaker 36 are provided in the video calling unit 23.
  • the drive control unit 39 is provided in the moving mechanism 21.
  • the control circuit 30, the memory 31, the communication unit 37, and the sensor 38 may be provided in the moving mechanism 21 or in the video calling unit 23.
  • the control circuit 30 is, for example, a CPU.
  • the control circuit 30 controls the entire telepresence robot 20. More specifically, the control circuit 30 controls the movement of the telepresence robot 20.
  • the control circuit 30 also controls the video call with the operator HM1.
  • the control circuit 30 executes a program stored in the program storage unit 41 of the memory 31.
  • the memory 31 includes a program storage unit 41 and a data storage unit 42. Note that the memory 31 may also include various types of storage devices as external storage, such as a magnetic disk storage device (HDD) or a solid state drive (SSD).
  • HDD magnetic disk storage device
  • SSD solid state drive
  • the program storage unit 41 may be configured as a non-transient storage medium, for example, a non-volatile memory such as a ROM (Read Only Memory).
  • the program storage unit 41 stores the execution program of the telepresence robot 20.
  • the program includes a movement control program for the telepresence robot 20, a video processing application, and a video call application with the remote control device 10.
  • the data storage unit 42 may be configured as a storage medium, for example, a volatile memory such as a RAM (Random Access Memory).
  • the data storage unit 42 stores data necessary for the operation of the telepresence robot 20.
  • the input unit 32 includes input devices such as a touch panel or button switches.
  • the local speaker HM2 may use these input devices to send and receive information to and from the operator HM1. These input devices may also be used during maintenance of the telepresence robot 20.
  • the presentation unit 33 includes a display screen 2000 (e.g., an LCD or EL display) as a display device.
  • the presentation unit 33 presents a remote image (image of the operator HM1) based on the control of the control circuit 30.
  • the image displayed on the presentation unit 33 is switched based on the positional relationship between the telepresence robot 20 and the local speaker HM2.
  • Camera 34 is used to capture local video including local speaker HM2 under the control of control circuit 30.
  • the microphone 35 is used to acquire local voice, including the voice of the local speaker HM2, based on the control of the control circuit 30.
  • the speaker 36 is used to play remote audio (the voice of the operator HM1) based on the control of the control circuit 30.
  • the communication unit 37 is used for wireless communication with the remote control device 10. For example, the communication unit 37 transmits local video and audio, as well as information acquired by the telepresence robot 20, to the remote control device 10. The communication unit 37 also receives remote video and audio, as well as operation signals for the telepresence robot 20, from the remote control device 10.
  • the telepresence robot 20 includes a plurality of sensors 38.
  • the telepresence robot 20 includes a range sensor (laser sensor) or a depth sensor (infrared sensor) to measure the distance to a person or an obstacle.
  • the telepresence robot 20 also includes a gyro sensor or an acceleration sensor to measure the inclination of the support 22, the position and moving speed of the telepresence robot 20, etc.
  • Information from these sensors 38 is transmitted to the control circuit 30.
  • the control circuit 30 controls the moving mechanism 21 (drive control unit 39) based on the information from the sensors 38.
  • the drive control unit 39 controls the drive of the wheel drive motor of the moving mechanism 21 based on the control of the control circuit 30.
  • Fig. 4 is a block diagram showing an example of the functional configuration of the information presentation system 1.
  • the remote control device 10 includes an operation acquisition unit 111, a video acquisition unit 112, and a voice acquisition unit 113 as processing functions for presenting information.
  • the operation acquisition unit 111, the video acquisition unit 112, and the voice acquisition unit 113 are realized by causing the control circuit 11 to execute a program stored in the program storage unit 121.
  • part or all of the operation acquisition unit 111, the video acquisition unit 112, and the voice acquisition unit 113 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).
  • the operation acquisition unit 111 acquires operation instructions for operating the telepresence robot 20 from the operator HM1. More specifically, the operation acquisition unit 111 acquires local information from the local information presentation unit 304 of the telepresence robot 20.
  • the local information includes local audio, local video, and person information detected from the local video.
  • the local information may also include position information of the telepresence robot 20 and distance information between the telepresence robot 20 and the detected person.
  • the operation acquisition unit 111 displays the local information on the display screen 1000.
  • the operator HM1 refers to the local information displayed on the display screen 1000 and instructs the operation of the telepresence robot 20.
  • the operator HM1 decides whether or not to select that person as the person to be spoken to, that is, the local speaker HM2. Furthermore, when there are multiple people detected from the local video, the operator HM1 can select the local speaker HM2 from among them.
  • the operation acquisition unit 111 generates an operation signal that instructs the operator HM1 to approach the local speaker HM2 (the person to be spoken to) selected by the operator HM1, and transmits the operation signal to the local information presentation unit 304.
  • the video acquisition unit 112 acquires remote location video, for example of the operator HM1, from the camera 15 of the remote control device 10.
  • the video acquisition unit 112 transmits the acquired remote location video to the remote location video acquisition unit 307 of the telepresence robot 20.
  • the voice acquisition unit 113 acquires remote voice, including the conversation of the operator HM1, from the microphone 16 of the remote control device 10.
  • the voice acquisition unit 113 transmits the acquired remote voice to the remote voice acquisition unit 308 of the telepresence robot 20.
  • the telepresence robot 20 includes, as information presentation processing functions, a local voice acquisition unit 301, a local video acquisition unit 302, a person information acquisition unit 303, a local information presentation unit 304, a movement control unit 305, a movement processing unit 306, a remote location video acquisition unit 307, a remote location audio acquisition unit 308, a remote location audio output unit 309, a display screen synthesis unit 310, and a screen display unit 311.
  • the local voice acquisition unit 301, the local video acquisition unit 302, the person information acquisition unit 303, the local information presentation unit 304, the movement control unit 305, the movement processing unit 306, the remote location video acquisition unit 307, the remote location audio acquisition unit 308, the remote location audio output unit 309, the display screen synthesis unit 310, and the screen display unit 311 are realized by causing the control circuit 30 to execute a program stored in the program storage unit 41.
  • local audio acquisition unit 301 may be realized using hardware such as an LSI or ASIC.
  • the local voice acquisition unit 301 acquires local voice from the microphone 35.
  • the local voice acquisition unit 301 transmits the acquired local voice to the local information presentation unit 304.
  • the local image acquisition unit 302 acquires local image from the camera 34.
  • the local image acquisition unit 302 transmits the local image to the person information acquisition unit 303.
  • the person information acquisition unit 303 analyzes the on-site video to acquire person information.
  • the person information includes information about the person area and the face direction of the person.
  • the person area is an area in the on-site video where a person is determined to be included.
  • the person information acquisition unit 303 can detect multiple people from the on-site video. For example, YOLO (You Only Look Once) or the like may be used as an algorithm for person (object) detection.
  • the person information acquisition unit 303 also detects the face direction of the person in the person area by face direction detection.
  • the person information acquisition unit 303 may detect the moving direction of the person.
  • the person information acquisition unit 303 can detect the body direction (front direction) of the person by detecting the face direction or the moving direction.
  • This person information is used by the operator HM1 to grasp the on-site situation and to process the movement of the telepresence robot 20.
  • the person information acquisition unit 303 transmits the on-site video and the person information to the on-site information presentation unit 304 and the movement processing unit 306.
  • the local information presentation unit 304 generates local information including local audio, local video, and person information.
  • the local information presentation unit 304 presents (transmits) the local information to the operation acquisition unit 111 of the remote control device 10.
  • the local information presentation unit 304 also transmits the local information and the operation signal received from the operation acquisition unit 111 to the movement control unit 305.
  • the movement control unit 305 controls the movement of the telepresence robot 20. More specifically, the movement control unit 305 recognizes the local speaker HM2 (the person to be spoken to) based on the local information and operation signal received from the local information presentation unit 304. The movement control unit 305 obtains, for example, a result of measuring the distance between the telepresence robot 20 and the local speaker HM2 (hereinafter referred to as the "face-to-face distance") from the sensor 38. The movement control unit 305 transmits information about the local speaker HM2 to the movement processing unit 306. The movement control unit 305 controls the movement of the telepresence robot 20 based on the movement instruction signal received from the movement processing unit 306. More specifically, the movement control unit 305 transmits a drive signal for the wheel drive motor of the movement mechanism 21 to the drive control unit 39.
  • the local speaker HM2 the person to be spoken to
  • the movement control unit 305 obtains, for example, a result of measuring the distance between the telepresence robot 20 and the local speaker HM
  • the movement processing unit 306 selects the method of approaching the local speaker HM2 and the contents of the screen display on the display screen 2000 based on the positional relationship between the telepresence robot 20 and the local speaker HM2. More specifically, the movement processing unit 306 receives information about the local speaker HM2 from the movement control unit 305. The movement processing unit 306 also receives local video and person information. The movement processing unit 306 calculates the positional relationship between the telepresence robot 20 and the local speaker HM2 from this information.
  • the information on the positional relationship includes the face-to-face distance between the telepresence robot 20 and the local speaker HM2, and the angle at which the telepresence robot 20 faces from the front of the local speaker HM2.
  • the movement processing unit 306 determines the method of movement of the telepresence robot 20 and the contents of the screen display to be presented on the display screen 2000 based on the information on the positional relationship. The details of the method of approaching the local speaker HM2 and the contents of the screen display will be described later.
  • the movement processing unit 306 determines the movement direction of the telepresence robot 20 based on the approach method.
  • the movement processing unit 306 transmits a movement instruction signal that instructs the movement direction of the telepresence robot 20 to the movement control unit 305.
  • the movement processing unit 306 also transmits a display instruction signal that instructs the contents of the screen display to the display screen synthesis unit 310.
  • the remote location video acquisition unit 307 acquires remote location video, i.e., video of the operator HM1, from the video acquisition unit 112 of the remote control device 10.
  • the remote location video acquisition unit 307 transmits the remote location video to the display screen synthesis unit 310.
  • the remote voice acquisition unit 308 acquires remote voice, i.e., the voice of the operator HM1, from the voice acquisition unit 113 of the remote operation device 10.
  • the remote voice acquisition unit 308 transmits the remote voice to the remote voice output unit 309 and the display screen synthesis unit 310.
  • the remote location audio output unit 309 outputs the remote location audio to the speaker 36 of the telepresence robot 20.
  • the display screen synthesis unit 310 receives remote video, i.e., an image of the operator HM1, from the remote video acquisition unit 307.
  • the display screen synthesis unit 310 receives remote audio, i.e., the audio of the operator HM1, from the remote audio acquisition unit 308.
  • the display screen synthesis unit 310 also receives a display instruction signal from the movement processing unit 306.
  • the display screen synthesis unit 310 reads out screen display data 421 from the data storage unit 42 based on the display instruction signal.
  • the display screen synthesis unit 310 synthesizes an image to be displayed on the display screen 2000 based on the video and audio of the operator HM1, the display instruction signal, and the screen display data 421.
  • the screen display unit 311 displays the image acquired from the display screen synthesis unit 310 as a remote location image on the display screen 2000 of the telepresence robot 20.
  • Fig. 5 is a diagram showing a specific example of screen displays on the display screen 1000 of the remote control device 10.
  • a local image display window 1100, a remote image display window 1200, and a controller display window 1300 are displayed on the display screen 1000 of the presentation unit 14.
  • Local image display window 1100 displays local image and person information.
  • an image of a shopping mall is shown as the local image.
  • the person area is shown by frame FR.
  • three frames FRa, FRb, and FRc are shown.
  • one frame FR may contain multiple people.
  • frame FRa contains one woman.
  • Frame FRb contains three parents and their children.
  • Frame FRc contains a male-female duo.
  • the direction of the face (movement direction) of the detected person is indicated by an arrow.
  • frame FRb which is indicated by a double frame, is shown in a selected state.
  • the remote location video display window 1200 displays the video of operator HM1.
  • the controller display window 1300 is used to select the frame FR (local speaker HM2).
  • the remote control device 10 transmits an operation signal to the telepresence robot 20 to instruct it to approach the selected local speaker HM2 (the person to be spoken to).
  • Fig. 6 is a diagram showing a specific example of a screen display on a display screen 2000 of the telepresence robot 20.
  • the display screen 2000 of the presentation unit 33 displays one of a live-action image of the face of the operator HM1 (remote image), an image of the avatar of the operator HM1, and an image of the voice waveform of the operator HM1 (voice waveform of the remote voice).
  • a live-action image of the face of the operator HM1 remote image
  • an image of the avatar of the operator HM1 image of the voice waveform of the remote voice
  • voice waveform of the remote voice For example, an avatar that displays facial elements such as the eyes, nose, and mouth is applied.
  • the voice waveform does not need to be displayed as a circular waveform as shown in FIG. 6, so long as it presents the state of the speech voice of the operator HM1.
  • the display screen synthesis unit 310 synthesizes one of the images based on a display instruction signal received from the movement processing unit 306.
  • the display screen synthesis unit 310 synthesizes an avatar based on the live-action image of the operator HM1 and the screen display data 421.
  • the display screen synthesis unit 310 then changes the facial expression of the avatar based on the voice of the operator HM1.
  • the anthropomorphism of the images decreases in the order of live-action images, avatars, and audio waveforms.
  • the information presentation system 1 can control the acceptance of the local speaker HM2 by displaying images with adjusted anthropomorphism on the display screen 2000 based on the positional relationship between the telepresence robot 20 and the local speaker HM2.
  • Fig. 7 is a diagram showing a specific example of a direction in which the telepresence robot 20 approaches the local speaker HM2.
  • the telepresence robot 20 can control the direction of approach when approaching the local speaker HM2, i.e., the approach angle.
  • the approach angle ⁇ is set to 0° when the telepresence robot 20 approaches the local speaker from the front.
  • the approach angle ⁇ is set to 45° when the telepresence robot 20 approaches the local speaker from diagonally to the right.
  • the approach angle ⁇ is set to 90° when the telepresence robot 20 approaches the local speaker from the right side.
  • the information presentation system 1 can control the sense of acceptance of the local speaker HM2 by adjusting the approach angle ⁇ based on the positional relationship between the telepresence robot 20 and the local speaker HM2.
  • Fig. 8 is a diagram showing a specific example of the moving direction of the telepresence robot 20 and the screen display of the display screen 2000 when the telepresence robot 20 approaches the local speaker HM2.
  • the direction of movement of the telepresence robot 20 and the contents of the screen display of the display screen 2000 are divided into, for example, three stages depending on the facing distance D between the telepresence robot 20 and the local speaker HM2.
  • the first stage is a stage where the face-to-face distance D between the telepresence robot 20 and the local speaker HM2 is longer (farther) than a preset first distance d th_app (D>d th_app ).
  • the telepresence robot 20 approaches the local speaker HM2 from a relatively long distance.
  • a voice waveform with low anthropomorphism is displayed on the display screen 2000 so that the local speaker HM2 is not aware of the conversation. When the voice waveform is displayed, the local speaker HM2 is more receptive than when a live-action video or an avatar is displayed.
  • the telepresence robot 20 adjusts the approach angle ⁇ .
  • the approach angle ⁇ is preferably 30° ⁇ 60° (diagonal right) or ⁇ 30° ⁇ 60° (diagonal left).
  • the telepresence robot 20 is positioned diagonally right or diagonally left with respect to the local speaker HM2, thereby reducing the psychological resistance of the local speaker HM2 to the telepresence robot 20.
  • the telepresence robot 20 moves such that the face-to-face distance D is not shorter than the first distance d th_app .
  • the telepresence robot 20 moves on an arc with a radius d th_app centered on the local speaker HM2.
  • a real-life image of the operator HM1 is displayed on the display screen 2000.
  • the third stage is a case where the telepresence robot 20 approaches the local speaker HM2 from the position (first distance d th_app and approach angle ⁇ ) to the second distance d th_fin (d th_app >D ⁇ d th_fin ).
  • the second distance d th_fin is shorter than the first distance d th_app .
  • the telepresence robot 20 stops when it approaches the second distance d th_fin .
  • the telepresence robot 20 approaches the local speaker HM2 from an oblique position, so that the psychological resistance of the local speaker HM2 is reduced.
  • an avatar with intermediate anthropomorphism is displayed on the display screen 2000.
  • the telepresence robot 20 talks to the local speaker HM2 while the avatar is displayed on the display screen 2000. Displaying an avatar will be more acceptable to local speakers HM2 than displaying live video or audio waveforms.
  • the moving speed of the telepresence robot 20 in each stage may be different.
  • the moving speed in the third stage may be slower than the first and second stages.
  • Fig. 9 is a flowchart showing the overall flow of the information presentation method.
  • the telepresence robot 20 acquires the local conditions (S1). More specifically, the local audio acquisition unit 301 acquires local audio. Also, the local video acquisition unit 302 acquires local video.
  • the person information acquisition unit 303 acquires person information (information on person area, facial direction, etc.) from the local video (S2).
  • the local information presentation unit 304 presents local information (local audio, local video, and person information) to the operation acquisition unit 111 of the remote control device 10 (S3).
  • the local information presentation unit 304 acquires an operation signal from the remote control device 10 (S4). More specifically, local information is displayed on the display screen 1000 of the remote control device 10.
  • the operator HM1 refers to the display screen 1000 and selects a local speaker HM2.
  • the operation acquisition unit 111 transmits an operation signal for the telepresence robot 20 to the local information presentation unit 304 based on the selection result of the operator HM1.
  • the telepresence robot 20 starts moving the telepresence robot 20 and displaying the display screen 2000 based on the operation signal (S5). More specifically, the movement processing unit 306 checks the positional relationship between the telepresence robot 20 and the local speaker HM2. If the facing distance D is longer than the first distance d th_app , the movement processing unit 306 selects the first stage of the approach method. The movement processing unit 306 transmits a movement instruction signal to the movement control unit 305 and transmits a display instruction signal to the display screen synthesis unit 310.
  • the display screen synthesis unit 310 When the display screen synthesis unit 310 receives a display instruction signal for the first stage of screen display from the movement processing unit 306, it synthesizes the audio waveform.
  • the screen display unit 311 displays the audio waveform on the display screen 2000 (S6).
  • the movement control unit 305 When the movement control unit 305 receives the first stage movement instruction signal from the movement processing unit 306, it moves the telepresence robot 20 to the first distance d th_app (S7). That is, in the first stage, the telepresence robot 20 approaches the local speaker HM2 to the first distance d th_app while displaying the voice waveform on the display screen 2000.
  • the movement processing unit 306 moves to the second stage of the approach method.
  • the display screen synthesis unit 310 receives a display instruction signal for the second stage of screen display from the movement processing unit 306, it selects the live-action video of the operator HM1.
  • the screen display unit 311 displays the live-action video of the operator HM1 on the display screen 2000 (S8).
  • the movement control unit 305 When the movement control unit 305 receives a second-stage movement instruction signal from the movement processing unit 306, it moves the telepresence robot 20 to a position where the approach angle ⁇ is 30° ⁇ 60° (right diagonal) or -30° ⁇ -60° (left diagonal) (S9). That is, in the second stage, the telepresence robot 20 moves to a preset approach angle ⁇ while displaying a live-action image of the operator HM1 on the display screen 2000.
  • the movement processing unit 306 moves to the third stage of the approach method.
  • the display screen synthesis unit 310 receives a display instruction signal for the third stage screen display from the movement processing unit 306, it synthesizes an avatar of the operator HM1 based on the live video and audio of the operator HM1.
  • the screen display unit 311 displays the avatar of the operator HM1 on the display screen 2000 (S10).
  • the movement control unit 305 When the movement control unit 305 receives a third-stage movement instruction signal from the movement processing unit 306, the movement control unit 305 moves the telepresence robot 20 to a second distance d th_fin from the position to which the telepresence robot 20 was moved in the second stage (S11). That is, in the third stage, the telepresence robot 20 approaches the local speaker HM2 to the second distance d th_fin while displaying an avatar on the display screen 2000.
  • the operator HM1 speaks to the local speaker HM2 via the telepresence robot 20 (S12). Note that the operator HM1 may start speaking to the local speaker HM2 from the first or second stage.
  • the information presentation system 1 can switch the moving direction of the telepresence robot 20 and the contents of the screen display of the display screen 2000 according to the distance between the telepresence robot 20 and the local speaker HM2 when the telepresence robot 20 approaches the local speaker HM2.
  • the telepresence robot 20 can approach the local speaker HM2 up to a first distance d th_app while displaying a voice waveform on the display screen 2000.
  • the telepresence robot 20 can move to a position where 30° ⁇ 60° (right diagonal) or ⁇ 30° ⁇ 60° (left diagonal) while displaying a live-action image of the operator HM1 on the display screen 2000.
  • the telepresence robot 20 can approach the local speaker HM2 up to a second distance d th_fin while displaying an avatar on the display screen 2000.
  • the telepresence robot 20 can then speak to the local speaker HM2 at a position where the second distance d th_fin is satisfied and 0° ⁇ 60° (right diagonal) or -30° ⁇ -60° (left diagonal).
  • a movement direction and screen display contents that pose relatively little psychological resistance to the local speaker HM2 can be selected. This can improve the receptiveness of the local speaker HM2 who is spoken to by the telepresence robot 20 to the telepresence robot 20.
  • the present invention is not limited to the above-described embodiment, and various modifications can be made in the implementation stage without departing from the gist of the invention. Furthermore, the embodiments may be implemented in appropriate combinations as far as possible, in which case the combined effects can be obtained. Furthermore, the above-described embodiment includes inventions at various stages, and various inventions can be extracted by appropriate combinations of the multiple components disclosed.
  • PROGRAM MEMORY UNIT 42 DATA MEMORY UNIT 111... OPERATION ACQUISITION UNIT 112... VIDEO ACQUISITION UNIT 113... AUDIO ACQUISITION UNIT 121... PROGRAM MEMORY UNIT 122... DATA MEMORY UNIT 301... ON-SITE AUDIO ACQUISITION UNIT 302... ON-SITE VIDEO ACQUISITION UNIT 303... PERSONAL INFORMATION ACQUISITION UNIT 304... ON-SITE INFORMATION PRESENTATION UNIT 305...
  • MOVING CONTROL UNIT 306 Movement processing unit 307: Remote location video acquisition unit 308: Remote location audio acquisition unit 309: Remote location audio output unit 310: Display screen synthesis unit 311: Screen display unit 421: Screen display data 1000: Display screen 1100: Local location video display window 1200: Remote location video display window 1300: Controller display window 2000: Display screen HM1: Operator HM2: Local speaker

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Manipulator (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/JP2023/020764 2023-06-05 2023-06-05 情報提示方法，情報提示システム、及びプログラム Ceased WO2024252447A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2023/020764 WO2024252447A1 (ja) 2023-06-05 2023-06-05 情報提示方法，情報提示システム、及びプログラム
JP2025525426A JPWO2024252447A1 (https=) 2023-06-05 2023-06-05

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/020764 WO2024252447A1 (ja) 2023-06-05 2023-06-05 情報提示方法，情報提示システム、及びプログラム

Publications (1)

Publication Number Publication Date
WO2024252447A1 true WO2024252447A1 (ja) 2024-12-12

Family

ID=93795221

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/020764 Ceased WO2024252447A1 (ja) 2023-06-05 2023-06-05 情報提示方法，情報提示システム、及びプログラム

Country Status (2)

Country Link
JP (1) JPWO2024252447A1 (https=)
WO (1) WO2024252447A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013059856A (ja) * 2005-09-30 2013-04-04 Irobot Corp 個人の相互交流のためのコンパニオンロボット
JP2020066094A (ja) * 2018-10-24 2020-04-30 トヨタ自動車株式会社 コミュニケーションロボットおよびコミュニケーションロボットの制御プログラム
JP2022074052A (ja) * 2020-10-29 2022-05-17 株式会社ユピテル 情報処理装置、通信端末、及びプログラム等

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013059856A (ja) * 2005-09-30 2013-04-04 Irobot Corp 個人の相互交流のためのコンパニオンロボット
JP2020066094A (ja) * 2018-10-24 2020-04-30 トヨタ自動車株式会社 コミュニケーションロボットおよびコミュニケーションロボットの制御プログラム
JP2022074052A (ja) * 2020-10-29 2022-05-17 株式会社ユピテル 情報処理装置、通信端末、及びプログラム等

Also Published As

Publication number Publication date
JPWO2024252447A1 (https=) 2024-12-12

Similar Documents

Publication Publication Date Title
US10462421B2 (en) Projection unit
US9900688B2 (en) Beamforming audio with wearable device microphones
US9294716B2 (en) Method and system for controlling an imaging system
TWI694904B (zh) 機器人語音操控系統及方法
CN110583013B (zh) 远程呈现系统
JP4949969B2 (ja) コミュニケーションロボットとその動作方法
JP4512830B2 (ja) コミュニケーションロボット
US12202121B2 (en) Robot
JP2009166184A (ja) ガイドロボット
US11659041B2 (en) Systems and methods for remote presence
JP4315872B2 (ja) 移動ロボットの制御装置
US20230217202A1 (en) Audio integration of portable electronic devices for enclosed environments
CN115552891A (zh) 远程控制系统和其远程作业装置、影像处理装置及程序
JPWO2019155735A1 (ja) 情報処理装置、情報処理方法及びプログラム
JP7106097B2 (ja) テレプレゼンスシステム
CN110202586A (zh) 与用户交流的装置、机器人、方法以及程序
WO2024252447A1 (ja) 情報提示方法，情報提示システム、及びプログラム
US20240051147A1 (en) Entertainment system and robot
WO2012008553A1 (ja) ロボットシステム
US20220301130A1 (en) Image processing device, telepresence system, image processing method, and image processing program
JP7402784B2 (ja) 遠隔表示システム、ロボット、及び表示端末
JP2006075948A (ja) ロボットの制御装置
JP7639780B2 (ja) 操縦システム、操縦方法及びプログラム
JP2024123375A (ja) 情報処理装置、制御プログラムおよび制御方法
US20250110630A1 (en) User interfaces and techniques for displaying information

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23940555

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2025525426

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE