WO2024162080A1 - 提案方法、提案装置および提案プログラム - Google Patents

提案方法、提案装置および提案プログラム Download PDF

Info

Publication number
WO2024162080A1
WO2024162080A1 PCT/JP2024/001732 JP2024001732W WO2024162080A1 WO 2024162080 A1 WO2024162080 A1 WO 2024162080A1 JP 2024001732 W JP2024001732 W JP 2024001732W WO 2024162080 A1 WO2024162080 A1 WO 2024162080A1
Authority
WO
WIPO (PCT)
Prior art keywords
scenario
robot
proposed
alternative
executing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2024/001732
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
琴子 山口
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to JP2024574463A priority Critical patent/JPWO2024162080A1/ja
Publication of WO2024162080A1 publication Critical patent/WO2024162080A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls

Definitions

  • This disclosure relates to a proposed method, a proposed device, and a proposed program for robot control.
  • robots that can behave in a variety of ways when given arbitrary commands are becoming more common. Users can make robots behave as they wish by incorporating into the robot a control program (hereafter referred to as a "scenario") that pre-determines how the robot will behave under certain conditions. For example, users can create scenarios of their choice by using a program for creating scenarios known as a scenario editor.
  • scenario a control program that pre-determines how the robot will behave under certain conditions.
  • users can create scenarios of their choice by using a program for creating scenarios known as a scenario editor.
  • a technology has been proposed that reduces the amount of work required to create a scenario by creating a scenario based on instructions entered by an operator.
  • a scenario can be used to operate robots of different models, as long as the robots have the same platform (hereafter referred to as "robotics PF"), which is the basic control function for loading the scenario.
  • robot PF the basic control function for loading the scenario.
  • Robots equipped with the same Robotics PF may lack the functionality required to execute a scenario due to differences in the sensors and operating mechanisms they are equipped with, or a sensor may be broken and unable to execute the scenario. For this reason, certain scenarios can only be executed on certain machines, and users are required to create scenarios for each machine.
  • this disclosure proposes a proposed method, a proposed device, and a proposed program that can improve user convenience regarding robot control.
  • a proposed method includes a control unit executing a reception procedure in which the control unit receives a scenario for controlling the behavior of a device, the scenario being used in common among different devices, a determination procedure in which the control unit determines whether a function or mechanism installed in the device in which the control unit is installed satisfies the requirements for executing the received scenario, and a proposal procedure in which, if it is determined that the scenario is difficult to execute, an alternative means for executing the scenario is proposed.
  • FIG. 11 is a diagram illustrating an overview of a proposal process according to the embodiment.
  • 10 is a flowchart illustrating an example of a procedure of a proposal process according to the embodiment.
  • FIG. 1 is a diagram for explaining a proposal process according to an embodiment;
  • FIG. 13 is a diagram (2) for explaining the proposal process according to the embodiment.
  • FIG. 4 is a diagram (3) for explaining the proposal process according to the embodiment.
  • FIG. 11 is a diagram illustrating an example of a data table related to the proposal process.
  • a figure showing an example of a user interface in the scenario determination process according to the embodiment. 13 is a diagram showing an example of a user interface in the scenario execution process according to the embodiment.
  • FIG. 1 is a diagram illustrating an example of the configuration of a robot according to an embodiment.
  • FIG. 13 is a diagram showing an overview of a proposal process according to a modified example.
  • FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of a proposed
  • Embodiment 1-1 Overview of proposal process according to embodiment 1-2.
  • Procedure of proposal process according to embodiment 1-3 Specific example of proposal process according to embodiment 1-4.
  • Example of user interface of proposal process 1-5 Configuration example of robot according to embodiment 1-6.
  • Modification 1-6-1 Device configuration 1-6-2.
  • EMBODIMENTS (1-1. Overview of proposal process according to embodiment) 1 is a diagram showing an overview of a proposal process according to an embodiment.
  • the proposal process according to the embodiment is realized by a robot 100 and a user 20 shown in FIG.
  • the robot 100 is an example of a proposal device that executes the proposal process according to the embodiment.
  • the robot 100 may be of any type, such as an autonomous type or a stationary type, as long as it has the functionality of an autonomous information device that can read a scenario and execute behavior in accordance with the scenario.
  • the user 20 is a person who creates a scenario for operating the robot 100, and is, for example, a manager or operator of the robot. Note that in this disclosure, the user 20 may also refer to an information processing terminal operated by the user 20. For example, in order to give the robot 100 a desired behavior, the user 20 creates a scenario using a dedicated app (scenario editor) or the like, and transmits the created scenario to the robot.
  • a dedicated app scenario editor
  • the user 20 can create scenarios by combining simple commands and conditions. Specifically, the user 20 creates scenarios incorporating desired actions by combining actions such as "waiting for a person to come within a certain distance” and “greeting a person when they come within a certain distance.”
  • the scenario can be executed if the user 20 creates a scenario for each sensor equipped in each machine or sets up operations that assume a failure, but this requires a lot of work from the user 20.
  • the scenarios prepared by the user 20 are usually static information, and it is difficult to replace them with scenarios that correspond to the robot's sensors or the functions (recognizers) that operate the sensors. In other words, in controlling a robot, there is a challenge in making the scenario run generically regardless of the machine, thereby increasing the convenience of the user 20 in terms of control.
  • the robot 100 which is an example of a proposal device according to the present disclosure, solves the above problem by a proposal process described below. Specifically, the robot 100 acquires a scenario that is commonly used among different robots, and determines whether the functions installed in the robot 100 satisfy the requirements for executing the acquired scenario. Then, if it is determined that it is difficult to execute the scenario, the robot 100 proposes an alternative means for executing the scenario. Furthermore, the robot 100 may propose the alternative means and automatically update the scenario based on the proposed alternative means.
  • the robot 100 will suggest a branch using a voice recognition function as an alternative, or suggest acquiring a human recognition function via a network as an alternative.
  • This allows the robot 100 to execute the scenario on any machine as long as it has a common basic function (the same robotics PF) that can read the created scenario. Therefore, the robot 100 can increase the convenience of the user 20 regarding robot control.
  • FIG. 1 it is assumed that a user 20 creates a general-purpose scenario and transmits the created scenario to a robot 100 via a network or the like.
  • the robot 100 has the functional configuration shown in FIG. 1 and executes a scenario obtained from the user 20.
  • the control unit 130 is a component for causing the robot 100 to function as an information processing device, and is, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU, etc.
  • the control unit 130 includes a management unit 131 and a robotics PF 135.
  • the robotics PF 135 also includes a reception unit 136, a judgment unit 137, an execution unit 138, and a proposal unit 139.
  • the robot 100 also stores mounting information and execution information.
  • the on-board information indicates information about the mechanisms and functions installed in the robot 100.
  • the sensor information 141 is information about the sensors equipped in the robot 100.
  • the sensor information 141 includes information about the sensors installed in the robot 100, such as whether the robot 100 is equipped with an image sensor, an audio sensor (such as a microphone), or a sensor that measures acceleration, temperature, humidity, etc.
  • the sensor information 141 may also include detailed information such as the specifications of each sensor, in addition to the presence or absence of the sensor.
  • the sensor information 141 may also include information such as the presence or absence of input/output devices such as a touch panel.
  • the sensor information 141 may also include information regarding the presence or absence and specifications of various known sensors.
  • the recognizer information 142 indicates information regarding the recognition function of the robot 100.
  • the recognizer information 142 includes information such as whether the robot 100 has a human recognition function that uses an image sensor to recognize the presence of a person, whether it has a face recognition function that recognizes a person's face, and whether it has a pose recognition function that recognizes a person's pose.
  • the recognizer information 142 may also include information such as whether the robot 100 has a voice recognition function or a recognition function based on characters input from a touch panel.
  • the recognizer information 142 may also include information regarding the presence and specifications of various known recognizers.
  • the operation information 143 is information that indicates the operating status of the robot 100, such as whether the sensors and recognizers are functioning normally.
  • the execution information is information about various execution results when a scenario is actually executed.
  • the recognition result information 145 includes the results of recognition by the recognizer (for example, whether or not the robot 100 recognized a person).
  • the sensor acquisition information 146 includes results such as numerical values actually acquired and measured by the sensor.
  • the management unit 131 associated with the control unit 130 manages the various types of information described above. For example, the management unit 131 manages information on the sensors and recognizers equipped in the robot 100. The management unit 131 also scans the inside of the robot 100 at regular intervals and manages whether the sensors and recognizers are operating normally.
  • the robotics PF 135 receives a scenario from the user 20 and performs various processes to execute the received scenario on the robot 100.
  • the reception unit 136 receives a scenario from the user 20.
  • the determination unit 137 determines whether or not the received scenario can be executed by the robot 100 based on the mounting information. For example, the determination unit 137 acquires information on sensors and recognizers used in the scenario, and determines whether or not the scenario can actually be executed by checking whether the robot 100 is equipped with the sensors and recognizers, whether the robot 100 is operating normally, etc.
  • the execution unit 138 executes various processes to realize the accepted scenario with the robot 100.
  • the suggestion unit 139 proposes an alternative to the user 20 when it is determined that the scenario is difficult to execute. For example, when the robot 100 receives a scenario that requires an image sensor, but the robot 100 is not equipped with an image sensor, the suggestion unit 139 proposes an alternative means for executing the scenario without using an image sensor.
  • FIG. 2 is a flowchart showing an example of a procedure for a proposal process according to an embodiment.
  • the robot 100 performs processes such as determining a scenario by itself, but the following process may be remotely executed on a terminal visible to the user 20 under the operation of the user 20.
  • the robot 100 that has received a scenario determines whether the robot 100 itself is equipped with a function for executing the scenario, but such a determination process and execution control of the robot 100 based on the scenario may be performed on the robot 100 from a terminal operated by the user 20.
  • the user 20 connects to the robot 100 and transmits the created scenario to the robot 100 (step S11).
  • the robot 100 accepts the scenario, it acquires the mounting information of the robot 100 (step S12).
  • the robot 100 analyzes the scenario and determines whether the robot 100 is equipped with functions for executing the scenario. First, the robot 100 determines whether the robot 100 is equipped with a recognizer for executing the scenario (step S13).
  • step S13 the robot 100 determines whether there is an unconsidered recognizer that could be an alternative (step S14). If a recognizer is found (step S14; Yes), the robot 100 determines whether the recognizer can be an alternative (step S15). The robot 100 determines whether the found recognizer can be an alternative based on preset condition information, etc., as will be described in detail below.
  • step S16 determines whether there is an unconsidered sensor that can be an alternative. If a sensor is found (step S16; Yes), the robot 100 determines whether the sensor is necessary for the alternative (step S17). If the sensor is not necessary for the alternative (step S17; No), the robot 100 searches for another sensor (step S16), and if no sensor that can be used for an alternative is found even after searching all sensors has been completed (step S16; No), it determines that the accepted scenario cannot be executed (step S18).
  • step S19 the robot 100 determines whether or not the sensor is operating normally. Note that, if it is determined in step S13 that the robot 100 is equipped with a recognizer for scenario execution (step S13; Yes), the robot 100 similarly determines whether or not the sensor for functioning the recognizer is operating normally (step S19).
  • step S19 If the sensor is not operating normally (step S19; No), the sensor or recognizer will not operate, and the robot 100 will search for an alternative (step S14).
  • step S19 if the sensor is operating normally (step S19; Yes), the robot 100 determines whether or not it is necessary to present the alternative to the scenario to the user 20 (step S20). For example, when the processing procedure has shifted from step S13 to step S19, the robot 100 determines that it is possible to execute the scenario without requiring an alternative, and therefore determines that it is unnecessary to present an alternative (step S20; No).
  • step S20 determines that it is necessary to present an alternative.
  • the robot 100 presents an appropriate alternative to the user 20 (step S21).
  • the robot 100 determines whether or not to update the scenario with the alternative in response to a request from the user 20, etc. (step S22). Note that the robot 100 may automatically determine whether or not to adopt the alternative, regardless of a response from the user 20. If it is determined that the scenario is not to be updated with the alternative (step S22; No), the robot 100 returns the process to step S14 to search for a different alternative.
  • step S22 if it is determined that the scenario should be updated with the alternative (step S22; Yes), the robot 100 updates the accepted scenario with the alternative and then executes the scenario (step S23). Note that if it is determined in step S20 that an alternative is not necessary (step S20; No), the robot 100 executes the scenario with the originally accepted content (step S23).
  • FIG. 3 is a diagram (1) for explaining the proposal process according to the embodiment.
  • Figure 3 shows a scenario 200.
  • scenario 200 is a definition document in which the behavior of robot 100 when guiding a person is preset, and is configured by combining each node that indicates the behavior of robot 100.
  • scenario 200 shown in Figure 3 shows that action processing and judgment processing move sequentially from left to right.
  • scenario 200 includes, as a robot action, "waiting for a person to come within a certain distance from the activation point" (step S30).
  • robot 100 continues to wait for a person to approach until the person comes within a certain distance set in advance.
  • the robot 100 judges whether the shortest distance between the robot 100 and the person exceeds 1.5 meters (step S40). If the distance between the robot 100 and the person exceeds 1.5 meters, the robot 100 takes the action of "greeting" as defined in the scenario 200. The robot 100 also waits to receive some kind of voice input from the person.
  • the robot 100 determines whether or not it has received input from a person (step S50). If the robot 100 has not received input from a person, it continues to wait. When the robot 100 receives input from a person, it decides on the subsequent processing based on the content of the voice input. For example, the robot 100 takes one of the following actions based on the analysis result of the voice input from the person: "guide to destination A", “guide to destination B", or "inform that the destination is unknown". For example, if a person gives voice input to the robot 100 such as "I want to go to destination A", the robot 100 decides to take the action of "guide to destination A" based on the voice input.
  • the robot 100 takes the action determined in step S50. That is, the robot 100 starts traveling to the starting point (in the example of scenario 200, a pre-set destination such as destination A or destination B) (step S60).
  • scenario 200 includes the actions and branching of robot 100 desired by user 20, who is the scenario creator.
  • robot 100 when robot 100 accepts scenario 200, it can guide people and move to a destination in accordance with the content defined in scenario 200.
  • FIG. 1 is a diagram (2) for explaining the proposal process according to the embodiment.
  • the robot 100 determines whether the scenario 200 can be executed without any problems. Specifically, when the robot 100 executes the action specified in step S30 ("wait for a person to come within a certain distance of the starting point"), the robot 100 determines whether the recognizer and sensors of the robot 100 will not impede the execution.
  • the robot 100 acquires onboard information about the robot 100 and determines whether it is equipped with recognizers and sensors necessary for the action, and whether they are operating without problems. For example, the robot 100 acquires information that indicates that it is equipped with an RGB camera, LiDAR (Light Detection and Ranging), and a speaker as sensors and mechanisms, but does not have a microphone. In this case, the robot 100 determines, based on the recognizer information, that the human recognition function is available, but voice recognition (input) is not possible. Also, the robot 100 determines that, although it is equipped with a ToF (Time of Flight) sensor, the sensor is defective and distance measurement functions, etc. are not available. The robot 100 collects this internal information about the robot 100 and then presents an alternative to the user 20.
  • ToF Time of Flight
  • the robot 100 determines that it is possible to use image recognition using LiDAR or an RGB camera instead of a distance measurement function using a ToF sensor.
  • the robot 100 presents an alternative suggestion (step S70) that a recognition service (such as an image recognition function) using LiDAR or an RGB camera can be used instead of a distance measurement function using a ToF sensor.
  • a recognition service such as an image recognition function
  • the robot 100 can obtain functions and the like available to the robot 100, determine whether each action included in the scenario 200 can be executed, and present alternatives if execution is difficult. This allows the user 20 to cause the robot 100 to execute the actions defined in the scenario 200 without rewriting the scenario 200.
  • FIG. 5 is a diagram (3) for explaining the proposal process according to the embodiment.
  • the robot 100 acquires the onboard information of the robot 100 and determines that the robot 100 does not have a microphone and is unable to receive voice input from a person. In this case, the robot 100 determines that it is unable to perform the action of "standby for voice input", which is the subsequent process of step S40.
  • the robot 100 can perform image recognition using LiDAR and an RGB camera, it determines that it can accept input by "human pose recognition," such as a person pointing in the direction of a destination, instead of voice input.
  • the robot 100 proposes performing human body pose estimation instead of voice recognition (step S75).
  • the robot 100 proposes an alternative input means, rather than an input means that uses a sensor or recognizer that the robot 100 does not have. This allows the user 20 to have the robot 100 accept inputs specified in the scenario 200 without having to rewrite the scenario 200.
  • FIG. 4 shows an example of a data table 210 related to the proposal process.
  • Data table 210 is a data table that associates types of actions, alternatives to those actions, hardware (HW) required to execute those alternatives (e.g., sensors, operating mechanisms, etc.), and conditions under which those alternatives are adopted.
  • Data table 210 may be possessed by all robots 100 equipped with robotics PF 135 as a file common to robotics PF 135, or may be attached to a scenario transmitted to robot 100. Alternatively, data table 210 may be stored in a cloud server or the like accessible by robotics PF 135.
  • the robot 100 determines that there is a malfunction in the ToF sensor installed on it.
  • the robot 100 determines the node (action) in the scenario 200 that is affected by the malfunction of the ToF sensor, and refers to the data table 210 to search for alternative actions related to the ToF sensor.
  • the scenario 200 specifies an action of "waiting for a person to come close," and a ToF sensor is normally used for such recognition.
  • the robot 100 searches for a means for performing such recognition without using a ToF sensor.
  • the robot 100 refers to the data table 210 to determine the type of the action, and determines that the type belongs to "environmental recognition.”
  • the robot 100 then refers to the conditions for performing the alternative "person recognition," which are that the robot 100 is equipped with an "RGB camera” and either a "ToF sensor” or a "LiDAR.”
  • the robot 100 determines that there is no problem in performing person recognition, since the robot 100 is equipped with an "RGB camera” and a "LiDAR.” Therefore, the robot 100 can propose "person recognition” as an alternative that does not use a ToF sensor.
  • data table 210 indicates that when the action is "input from a person", "voice recognition", “human pose estimation”, “touch panel input”, etc. may be presented as alternatives.
  • robot 100 determines that a certain action corresponds to "input from a person" ("waiting for voice input” in the example of FIG. 5) and that this action is impossible to take, it refers to data table 210 and searches for alternatives.
  • robot 100 cannot take voice input, other alternatives such as "human pose estimation” and "touch panel input” are presented as alternatives.
  • the robot 100 determines whether the robot 100 is equipped with the sensors necessary to execute the alternative and whether the alternative satisfies the conditions for presenting the alternative. For example, if the robot 100 is not equipped with an RGB camera, a ToF sensor, or LiDAR, the robot 100 does not suggest "human pose estimation” but presents "touch panel input.”
  • the robot 100 presents "human pose estimation” as an alternative to voice input, since the robot 100 is equipped with an RGB camera and LiDAR. Note that if the robot 100 is equipped with a touch panel, the robot 100 may propose "touch panel input” instead of "human pose estimation.”
  • the robot 100 determines that it does not have a voice output function when the action of "saying hello" is incorporated into the scenario, it searches the data table 210 for an alternative output means.
  • the robot 100 can search for "text display” as an alternative to voice output.
  • the robot 100 proposes to the user 20 the alternative of outputting text corresponding to the greeting using the display or projector output provided in the robot 100, instead of greeting through voice output.
  • the proposal process executed by the robot 100 described above is realized, for example, in accordance with the procedure of steps S14, S15, S17, S19, etc. shown in FIG. 2.
  • the robot 100 can propose alternatives for each action when it accepts the scenario 200, and therefore can determine the feasibility of the scenario 200 before it is actually executed, and can update the scenario 200 to make it feasible.
  • Fig. 7 is a diagram showing an example of a user interface in the scenario determination process according to the embodiment.
  • FIG. 7 shows the user interface when the user 20 determines whether the created scenario will work on the robot 100 before executing the scenario on the robot 100.
  • the user interface is a screen displayed on an information processing terminal or the like operated by the user 20.
  • the first screen 220 shown in FIG. 7 is, for example, an example of a scenario editor execution screen, and is a screen used to determine whether a scenario created by the user 20 will actually run without problems when the scenario is loaded into the robot 100.
  • the user 20 can execute the scenario determination process by selecting the start scenario determination button 225 on the first screen 220.
  • the first screen 220 transitions to the second screen 230.
  • the second screen 230 shows that the scenario created by the user 20 is loaded into the robot 100, and the onboard information and other information inside the robot 100 is checked to see if the scenario will operate without any problems.
  • the second screen 230 transitions to the third screen 240.
  • the robot 100 judges that there is a malfunction in one of the sensors used in the scenario.
  • the third screen 240 displays the judgment result by the robot 100. Specifically, the third screen 240 displays a suggestion 245 to the user 20, stating "A sensor malfunction has been discovered. Alternative: Purchase recognition service.”
  • proposal 245 indicates that, due to a sensor malfunction, in order to execute the scenario, it is necessary to acquire a recognizer (recognition service) that the robot 100 does not currently have. Furthermore, proposal 245 includes a preview execution button.
  • the robot 100 acquires the recognition service (e.g., a program related to the recognition process) presented as an alternative via the network.
  • the robot 100 shows the user 20 that the recognition process is executed without problem using the acquired recognition service.
  • the robot 100 shows the user 20 that the acquired recognition service operates without problem by projecting the surrounding environment recognized by the robot 100 (e.g., the surrounding situation captured by a camera) onto a screen, etc.
  • the third screen 240 transitions to the fourth screen 250.
  • a suggestion 255 such as "Do you want to purchase the recognition service?" is displayed.
  • the suggestion 255 is a suggestion that encourages the user 20 to actually purchase the recognition service that the user 20 has confirmed in the preview.
  • the user 20 can execute the created scenario with the robot 100. Note that if the user 20 rejects the alternative suggestion, the robot 100 may display a different alternative suggestion.
  • the user 20 when the user 20 creates a scenario, the user 20 can simulate on the user interface whether the scenario will operate without any problems again, without actually operating the robot 100. This allows the user 20 to quickly and reliably grasp the feasibility of the scenario. Furthermore, the user 20 can operate the scenario on various machines by obtaining the necessary recognition services via the user interface, etc.
  • FIG. 8 is a diagram showing an example of a user interface in a scenario execution process according to an embodiment.
  • FIG. 8 shows the user interface that is displayed when the user 20 causes the robot 100 to execute a scenario that the user 20 has created.
  • the user interface is a screen that is displayed on an information processing terminal or the like operated by the user 20.
  • the fifth screen 260 shown in FIG. 8 is a screen that is displayed when the user 20 loads a scenario created by the user 20 into the robot 100.
  • the user 20 can execute the scenario in the robot 100 by selecting the scenario execution start button 265 on the fifth screen 260.
  • the sixth screen 270 indicates that the scenario created by the user 20 has been loaded into the robot 100, the onboard information inside the robot 100 has been confirmed, and the process to execute the scenario is currently being carried out.
  • the sixth screen 270 transitions to the seventh screen 280.
  • the robot 100 determines that there is a malfunction in one of the sensors while executing the scenario.
  • the seventh screen 280 displays the result of the determination made by the robot 100. Specifically, the seventh screen 280 displays a suggestion 285 to the user 20 stating, "A malfunction has been found in the sensor. Do you want to terminate execution of the scenario?".
  • the robot 100 proposes an alternative plan for not ending the scenario.
  • the seventh screen 280 transitions to the eighth screen 290.
  • the eighth screen 290 displays a suggestion 295 such as "A recognition service is available as an alternative. Would you like to purchase the recognition service?"
  • the suggestion 295 includes content that the robot 100 can propose to the user 20 as an alternative based on the search results, in order to complete the action defined in the scenario without terminating the scenario, by searching for an alternative recognizer or sensor (hardware, etc.).
  • the user 20 can purchase the recognition service from the provider to allow the robot 100 to continue executing the created scenario. If the user 20 rejects the alternative, the robot 100 may display a different alternative.
  • the user 20 can immediately update the scenario through the suggestion process by the robot 100. This allows the user 20 to ensure stable operation of the robot 100, regardless of the compatibility between the scenario and the robot 100.
  • the robot 100 may propose acquiring that service.
  • the robot 100 may propose performing processing using another sensor that can be substituted (e.g., an RGB camera).
  • the robot 100 may automatically update the scenario so that the alternative sensor is automatically used without making a proposal.
  • the robot 100 manages the onboard information of the robot 100, and when a scenario is received, it determines the feasibility of the scenario and performs processing for the scenario to be executed by the robot 100. Furthermore, if it is difficult for the robot 100 to execute the operation, the robot 100 proposes an alternative plan to the user 20. This allows the robot 100 to use scenarios in a general purpose manner, regardless of the mechanisms and functions of each machine, thereby improving the convenience of the user regarding robot control.
  • Fig. 9 is a diagram illustrating an example of the configuration of the robot 100 according to the embodiment.
  • the robot 100 has a communication unit 110, a memory unit 120, and a control unit 130.
  • the robot 100 may also have an input unit (e.g., a keyboard, a touch display, etc.) that accepts various operations from an administrator managing the robot 100 or a user 20, etc., and a display unit (e.g., a liquid crystal display, etc.) that displays various information.
  • an input unit e.g., a keyboard, a touch display, etc.
  • a display unit e.g., a liquid crystal display, etc.
  • the communication unit 110 is realized, for example, by a NIC (Network Interface Card) or a network interface controller.
  • the communication unit 110 is connected to the network N by wire or wirelessly, and transmits and receives information to and from an information terminal or external device used by the user 20 via the network N.
  • the network N is realized, for example, by a wireless communication standard or method such as Bluetooth (registered trademark), the Internet, Wi-Fi (registered trademark), UWB (Ultra Wide Band), or LPWA (Low Power Wide Area).
  • the storage unit 120 is realized, for example, by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.
  • a semiconductor memory element such as a random access memory (RAM) or a flash memory
  • a storage device such as a hard disk or an optical disk.
  • the storage unit 120 stores various information for performing the proposal processing according to the embodiment.
  • the storage unit 120 includes, for example, a sensor information storage unit 121 that stores information about the sensors mounted on the robot 100, a recognizer storage unit 122 that stores information about the recognizer, and a condition information storage unit 123 that stores conditions for the execution of a scenario, alternatives, etc.
  • the sensor information storage unit 121 and the recognizer storage unit 122 correspond to the mounting information, etc. shown in FIG. 1.
  • the condition information storage unit 123 corresponds to the execution information shown in FIG. 1 and the data table 210 shown in FIG. 6.
  • the sensor unit 150 refers to various sensors mounted on the robot 100.
  • examples of the sensor unit 150 include a ToF sensor, LiDAR, a camera, etc.
  • the camera may be in any form, such as a stereo camera, a monocular camera, or a lensless camera.
  • the camera is not limited to a visible light camera such as an RGB camera, but may be a camera with a depth sensor equipped with a ToF sensor, etc.
  • the camera may also be equipped with an AI-equipped image sensor capable of object detection and recognition processing.
  • the sensor unit 150 may also have various sensors other than the LiDAR and the camera.
  • the sensor unit 150 may include a distance measurement system using millimeter wave radar.
  • the sensor unit 150 may also include a depth sensor for acquiring depth data.
  • the sensor unit 150 may also be a sonar that explores the surrounding environment using sound waves.
  • the sensor unit 150 may also include a microphone that collects sounds around the robot 100, an illuminance sensor that detects the illuminance around the robot 100, a humidity sensor that detects the humidity around the robot 100, a geomagnetic sensor that detects the magnetic field at the location of the robot 100, and the like.
  • the mechanism unit 160 refers to a mechanism for operating the robot 100.
  • the mechanism unit 160 includes various mechanisms such as a motor for operating the robot 100 autonomously and wheels operated by the motor.
  • the mechanism unit 160 may also include various mechanisms (such as speakers and LED lamps) for outputting sound and light displays.
  • the mechanism unit 160 may include any known mechanism for moving the robot 100 or for the robot 100 to output some kind of information.
  • control unit 130 is realized by, for example, a CPU, MPU, GPU, etc., executing a program (for example, a proposed program according to the present disclosure) stored inside the robot 100 using a RAM or the like as a working area.
  • the control unit 130 is also a controller, and may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).
  • control unit 130 executes various processes to realize the proposed processing according to the present disclosure. Below, we will explain the processing of each unit other than those already described in FIG. 1.
  • the reception unit 136 receives a scenario for controlling the behavior of the robot 100, which is a scenario that is commonly used among different robots. For example, the reception unit 136 receives a scenario from the user 20, and loads the received scenario into the robot 100, thereby making the robot 100 capable of executing the scenario.
  • the determination unit 137 determines whether the functions or mechanisms installed in the device in which the control unit 130 is installed (meaning the device that executes the scenario and is the subject of judgment of the installed functions, etc.; in this embodiment, this corresponds to the robot 100) satisfy the requirements for executing the acquired scenario.
  • the determination unit 137 determines whether or not the recognizer installed in the robot 100 satisfies the requirements for executing a scenario. Specifically, the determination unit 137 determines that the recognizer does not satisfy at least one of the requirements of image recognition, voice recognition, and distance recognition for executing a scenario. For example, when an action for recognizing a person is set in the scenario, the determination unit 137 determines whether or not the robot 100 is capable of image recognition for recognizing a person. Alternatively, when an action for determining whether or not a person has approached closer than a predetermined distance is set in the scenario, the determination unit 137 determines whether or not the robot 100 is capable of distance recognition for determining the distance from a person.
  • the determination unit 137 may also determine whether or not a sensor for executing a scenario is mounted on the robot 100 as a mechanism. Alternatively, the determination unit 137 may determine whether or not a sensor for executing a scenario is operating normally on the robot 100 as a mechanism.
  • the determination unit 137 also determines whether or not the robot 100 is equipped with an output unit as a mechanism for executing a scenario.
  • An output unit refers to a mechanism for outputting some information to the outside.
  • the determination unit 137 determines whether or not the robot 100 is equipped with an audio output unit (such as a speaker) for executing a scenario.
  • the determination unit 137 may also determine whether or not the robot 100 is equipped with a video output unit (such as a projector or display) as an output unit.
  • the determination unit 137 may also determine whether or not the robot 100 is equipped with a movement mechanism for executing a scenario as a mechanism.
  • the movement mechanism is a mechanism for autonomously moving the robot 100, and refers to, for example, the legs and wheels of the robot 100, as well as power units such as motors for operating them.
  • the execution unit 138 executes the scenario accepted by the acceptance unit 136. Note that the execution unit 138 may appropriately execute a determination process similar to that performed by the determination unit 137 during execution of the scenario, as described in FIG. 8.
  • the suggestion unit 139 suggests alternative means for executing the scenario when the determination unit 137 or the execution unit 138 determines that it is difficult to execute the scenario.
  • the suggestion unit 139 proposes a recognition means to replace the recognizer.
  • the suggestion unit 139 proposes a recognition service that can be acquired via a network as the recognition means.
  • the suggestion unit 139 accesses the server of a business that provides a recognition service, searches for a recognition service that can be used in the scenario, and presents the search results to the user 20.
  • the suggestion unit 139 proposes at least one of image recognition, voice recognition, and distance recognition as an alternative to the recognizer that does not satisfy the requirements. For example, when the suggestion unit 139 determines that image recognition cannot be used in the robot 100, it proposes a recognizer related to voice recognition or distance recognition as an alternative. Note that image recognition, voice recognition, distance recognition, etc. are merely examples, and the suggestion unit 139 may propose any alternative means as long as it is a means that can be used to execute a scenario.
  • the suggestion unit 139 suggests a recognition means to replace the sensor. Furthermore, when a sensor is not operating normally, the suggestion unit 139 may suggest a recognition means to replace the sensor. For example, when there is a malfunction in the RGB camera, the suggestion unit 139 may suggest the use of another sensor such as LiDAR.
  • the suggestion unit 139 may suggest an output means to replace the output unit. Specifically, if the robot 100 is not equipped with an audio output unit (such as a speaker), the suggestion unit 139 suggests a display output means (such as displaying text on a display) to replace the audio output unit.
  • an audio output unit such as a speaker
  • the suggestion unit 139 suggests a display output means (such as displaying text on a display) to replace the audio output unit.
  • the suggestion unit 139 may suggest an output means that replaces the moving mechanism. Specifically, when the robot 100 is not equipped with a moving mechanism, the suggestion unit 139 suggests, as an alternative, an output means that can indicate the destination set by the scenario (for example, pointing to the desired direction using an LED light, etc.).
  • the suggestion unit 139 may update the scenario based on the alternative means. For example, when the user 20 accepts the purchase of a recognition service proposed as an alternative, the suggestion unit 139 automatically updates the scenario so that the scenario is executed using the recognition service.
  • the robot 100 merely shows the functions conceptually, and may take various forms depending on the embodiment.
  • the robot 100 may be configured with two or more devices that are different for each of the above-mentioned functions.
  • the robot 100 may be configured as an edge device and a cloud server that are connected via a network.
  • FIG. 10 is a diagram showing an overview of the proposal process according to the modified example.
  • the robot 100 when the robot 100 performs a scenario determination process or a scenario execution process (step S300), the robot 100 transmits the results to the cloud server 300.
  • the cloud server 300 references the determination results and execution results to generate suitable alternatives, and presents the generated alternatives to the robot 100 (step S310).
  • the robot 100 executes the presented alternative or proposes it to the user 20.
  • the robot 100 can reduce the processing load of the robot 100 by having the cloud server 300 execute the proposal process, which places a high processing load on the robot 100.
  • the robot 100 is not necessarily limited to one that moves autonomously, and may be a smart home appliance such as a smart speaker or a television, or a wearable device such as a smart watch.
  • the suggestion process does not necessarily have to be realized as a rule-based process, such as a process based on alternatives predefined in the data table 210.
  • the robot 100 may learn the results of trials performed by a plurality of users and suggest alternatives based on the learning results.
  • the robot 100 can use historical information of past selections made by multiple users as alternatives to a certain behavior.
  • the robot 100 may propose alternative means for executing a scenario based on the history of alternative means being executed in the past by other robots.
  • the robot 100 may propose multiple alternative means in accordance with a priority order set based on the history of alternative means being executed in the past.
  • the robot 100 may give higher priority to suggesting to the user 20 an alternative that has been selected more frequently by multiple users. Furthermore, the robot 100 may omit proposing an alternative that has rarely been selected in the past, even if the process is stored in the data table 210 as an alternative. In this way, the robot 100 can suggest alternatives that tend to be used by more users, and therefore can give priority to presenting to the user 20 alternatives that are expected to contribute to solving the problem in reality.
  • the robot 100 may propose the acquired alternative to the user 20.
  • alternatives such as a recognizer, used to solve a certain action from the history
  • the robot 100 may propose the acquired alternative to the user 20.
  • the robot 100 can propose a better alternative by acquiring such information.
  • the robot 100 may also propose alternative mechanisms. For example, in the case of guiding a customer, if the robot 100 does not have an autonomous movement mechanism, the robot 100 may propose alternative actions such as outputting light indicating the direction of the destination or outputting movement procedures as audio.
  • the robot 100 may refer to the action corresponding to the alternative and its branch in advance, and may propose only alternatives that have the potential to achieve the objective at the branch beyond. For example, if a scenario includes an action that requires voice output at the branch beyond, the robot 100 may not propose alternatives at that point that would result in voice output at a stage prior to the branch. This allows the robot 100 to avoid proposing meaningless alternatives to the user 20, thereby improving the convenience of the user 20. Note that even in this case, the robot 100 may propose all alternatives to the user 20 with the intention of having the user 20 see what alternatives exist.
  • each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure.
  • the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.
  • the proposed device (robot 100 in the embodiment) according to the present disclosure includes, as a control unit, a reception unit (reception unit 136 in the embodiment) that executes a reception procedure, a determination unit (determination unit 137 in the embodiment) that executes a determination procedure, and a proposal unit (proposal unit 139 in the embodiment) that executes a proposed procedure, and executes the proposed method according to the present disclosure.
  • the reception procedure is a scenario for controlling the behavior of the device, and receives a scenario that is commonly used between different devices.
  • the determination procedure determines whether or not a function or mechanism installed in the device in which the control unit is installed (i.e., the proposed device) satisfies the requirements for executing the accepted scenario.
  • the proposed procedure proposes an alternative means for executing the scenario when it is determined that it is difficult to execute the scenario.
  • the proposed method of the present disclosure reads a scenario common to robots, determines the functions required to execute the scenario, and proposes an alternative if execution is difficult.
  • the proposed method as long as the basic functions that can load the created scenario are common, the scenario can be executed by any robot. Therefore, the proposed method can increase the user's convenience regarding robot control.
  • the determination procedure also determines, as a function, whether or not the recognizer installed in the device in which the control unit is installed satisfies the requirements for executing the scenario.
  • the proposal procedure proposes a recognition means to replace the recognizer determined to not satisfy the requirements.
  • the proposal procedure proposes a recognition service that can be obtained via a network as the recognition means.
  • the proposed method allows users to execute a scenario without relying on the robot's own recognizer, reducing the burden of scenario creation.
  • the determination procedure also determines whether the recognizer meets at least one of the requirements for executing the scenario, namely image recognition, voice recognition, and distance recognition.
  • the proposal procedure proposes at least one of the recognition means, namely image recognition, voice recognition, and distance recognition, to replace the recognizer that does not meet the requirements.
  • the proposed method makes it possible to realize a variety of alternative means, thereby increasing the feasibility of scenarios being executed by robots.
  • the determination procedure determines whether or not a sensor for executing a scenario is installed in the device equipped with the control unit as a mechanism.
  • the proposal procedure proposes a recognition means to replace the sensor when the sensor is not installed in the device equipped with the control unit.
  • the determination procedure determines whether or not a sensor for executing a scenario is operating normally in the device equipped with the control unit as a mechanism.
  • the proposal procedure proposes a recognition means to replace the sensor when the sensor is not operating normally.
  • the proposed method makes it possible to execute a scenario even if there is a malfunction in the robot attempting to execute the scenario.
  • the determination procedure also determines whether or not an output unit for executing a scenario is installed in the device equipped with the control unit as a mechanism.
  • the proposal procedure proposes an output means to replace the output unit when the output unit is not installed in the device equipped with the control unit.
  • the determination procedure determines whether or not an audio output unit for executing a scenario is installed in the device equipped with the control unit.
  • the proposal procedure proposes a display output means to replace the audio output unit when the audio output unit is not installed in the device equipped with the control unit.
  • This proposed method makes it possible to obtain some kind of output from the robot regardless of the type of output means, thereby reducing the possibility of problems occurring, such as confusing customers who come into contact with the robot.
  • the determination procedure also determines whether or not the device equipped with the control unit is equipped with a movement mechanism for executing the scenario as a mechanism.
  • the proposal procedure proposes an output means that replaces the movement mechanism when the device equipped with the control unit does not have a movement mechanism. For example, when the device equipped with the control unit does not have a movement mechanism, the proposal procedure proposes an output means that can indicate the movement destination set by the scenario as an alternative.
  • the proposed method makes it possible to make a robot perform the actions intended by the scenario, regardless of the robot's movement mechanism.
  • the suggestion procedure updates the scenario based on the alternative means.
  • the suggestion procedure may suggest alternative means for executing the scenario based on a history of alternative means being executed in other devices in the past.
  • the suggestion procedure may suggest multiple alternative means in accordance with a priority order set based on a history of alternative means being executed in the past.
  • the proposed method makes it possible to propose alternatives without relying on rules, allowing for flexible response to a variety of situations.
  • FIG. 11 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of a proposed device (robot 100) according to the present disclosure.
  • the computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input/output interface 1600.
  • Each unit of the computer 1000 is connected by a bus 1050.
  • the CPU 1100 operates based on the programs stored in the ROM 1300 or the HDD 1400 and controls each component. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.
  • the ROM 1300 stores boot programs such as the Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.
  • BIOS Basic Input Output System
  • HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs.
  • HDD 1400 is a recording medium that records the proposed program according to the present disclosure, which is an example of program data 1450.
  • the communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet).
  • the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.
  • the input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000.
  • the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input/output interface 1600.
  • the CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input/output interface 1600.
  • the input/output interface 1600 may also function as a media interface that reads programs and the like recorded on a specific recording medium.
  • Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase change rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, and semiconductor memories.
  • optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase change rewritable Disks)
  • magneto-optical recording media such as MOs (Magneto-Optical Disks)
  • tape media magnetic recording media
  • magnetic recording media and semiconductor memories.
  • the CPU 1100 of the computer 1000 executes the proposed program loaded onto the RAM 1200 to realize the functions of the control unit 130, etc.
  • the proposed program according to the present disclosure and data in the storage unit 120 are stored in the HDD 1400.
  • the CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from other devices via the external network 1550.
  • the present technology can also be configured as follows.
  • the control unit a receiving step of receiving a scenario for controlling the behavior of the device, the scenario being used commonly among different devices; a determination step of determining whether or not a function or mechanism installed in a device in which the control unit is installed satisfies a requirement for executing the accepted scenario; a proposal step of proposing an alternative means for executing the scenario when it is determined that the execution of the scenario is difficult;
  • the proposed method includes: (2)
  • the determination procedure includes: As the function, determining whether or not a recognizer installed in a device in which the control unit is installed satisfies requirements for executing the scenario;
  • the proposed procedure comprises: If a recognizer is determined not to satisfy the requirements, an alternative recognition means is proposed to replace the recognizer.
  • the proposed procedure comprises: As the recognition means, a recognition service available via a network is proposed; The proposed method described in (2) above.
  • the determination procedure includes: determining that the recognizer does not satisfy at least one of image recognition, voice recognition, and distance recognition requirements for executing the scenario; The proposed procedure comprises: If a recognizer is determined not to satisfy the requirements, at least one of image recognition, voice recognition, and distance recognition is proposed as an alternative to the recognizer that does not satisfy the requirements.
  • the determination procedure includes: determining whether or not a sensor for executing the scenario is installed in the device in which the control unit is installed; The proposed procedure comprises: When the sensor is not mounted on the device in which the control unit is mounted, a recognition means to replace the sensor is proposed. The proposed method according to any one of (1) to (4).
  • the determination procedure includes: determining whether a sensor for executing the scenario is operating normally in a device in which the control unit is installed; The proposed procedure comprises: When the sensor is not operating normally, an alternative recognition means is proposed. The proposed method according to any one of (1) to (5).
  • the determination procedure includes: determining whether or not an output unit for executing the scenario is installed in the device in which the control unit is installed, as the mechanism;
  • the proposed procedure comprises: When the output unit is not installed in the device in which the control unit is installed, an output means to be substituted for the output unit is proposed.
  • the proposed method according to any one of (1) to (6).
  • the determination procedure includes: determining whether or not a sound output unit for executing the scenario is installed in the device in which the control unit is installed;
  • the proposed procedure comprises: When the audio output unit is not installed in the device in which the control unit is installed, a display output means is proposed to replace the audio output unit.
  • the determination procedure includes: determining whether or not a moving mechanism for executing the scenario is installed in the device in which the control unit is installed, as the mechanism;
  • the proposed procedure comprises: When the moving mechanism is not installed in the device in which the control unit is installed, an output means to replace the moving mechanism is proposed.
  • the proposed procedure comprises: If the device in which the control unit is installed is not equipped with the moving mechanism, an output means capable of indicating the moving destination set by the scenario is proposed as an alternative.
  • the proposed procedure comprises: If the proposed alternative is accepted, updating the scenario based on the proposed alternative; The proposed method according to any one of (1) to (10).
  • the proposed procedure comprises: Suggesting alternative ways to execute the scenario based on a history of alternative ways executed in other devices in the past; The proposed method according to any one of (1) to (11) above. (13) The proposed procedure comprises: If there are multiple alternatives, suggest multiple alternatives in accordance with a priority order set based on a history of the alternatives being implemented in the past; The proposed method according to any one of (1) to (12) above.
  • the proposed device comprises: (15) Computer, a reception unit that receives a scenario for controlling the behavior of the device, the scenario being used commonly among different devices; a determination unit that determines whether a function or mechanism installed in a device having the reception unit satisfies a requirement for executing the received scenario; a suggestion unit that, when it is determined that it is difficult to execute the scenario, suggests an alternative means for executing the scenario;
  • the proposed device comprises: (15) Computer, a reception unit that receives a scenario for controlling the behavior of the device, the scenario being used commonly among different devices; a determination unit that determines whether a function or mechanism installed in a device having the reception unit satisfies a requirement for executing the received scenario; a suggestion unit that, when it is determined that it is difficult to execute the scenario, suggests an alternative means for executing the scenario;
  • a proposal program that functions as a proposal device having the above-mentioned features.
  • Robot 110 Communication unit 120
  • Memory unit 121 Sensor information memory unit 122 Recognizer memory unit 123
  • Control unit 131 Management unit 135 Robotics PF 136 Reception unit 137
  • Determination unit 138 Execution unit 139 Proposal unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Human Computer Interaction (AREA)
  • Manipulator (AREA)
PCT/JP2024/001732 2023-02-01 2024-01-23 提案方法、提案装置および提案プログラム Ceased WO2024162080A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2024574463A JPWO2024162080A1 (https=) 2023-02-01 2024-01-23

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023013807 2023-02-01
JP2023-013807 2023-07-04

Publications (1)

Publication Number Publication Date
WO2024162080A1 true WO2024162080A1 (ja) 2024-08-08

Family

ID=92146538

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2024/001732 Ceased WO2024162080A1 (ja) 2023-02-01 2024-01-23 提案方法、提案装置および提案プログラム

Country Status (2)

Country Link
JP (1) JPWO2024162080A1 (https=)
WO (1) WO2024162080A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009136992A (ja) * 2007-12-10 2009-06-25 Honda Motor Co Ltd ロボット
JP2009281381A (ja) * 2008-05-21 2009-12-03 General Electric Co <Ge> 複合サイクル発電システムの制御装置
CN106936916A (zh) * 2017-03-24 2017-07-07 上海木爷机器人技术有限公司 数据共享方法及装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009136992A (ja) * 2007-12-10 2009-06-25 Honda Motor Co Ltd ロボット
JP2009281381A (ja) * 2008-05-21 2009-12-03 General Electric Co <Ge> 複合サイクル発電システムの制御装置
CN106936916A (zh) * 2017-03-24 2017-07-07 上海木爷机器人技术有限公司 数据共享方法及装置

Also Published As

Publication number Publication date
JPWO2024162080A1 (https=) 2024-08-08

Similar Documents

Publication Publication Date Title
US11670302B2 (en) Voice processing method and electronic device supporting the same
CN113168227B (zh) 执行电子装置的功能的方法以及使用该方法的电子装置
KR102948925B1 (ko) 복수의 지능형 에이전트를 관리하는 전자 장치 및 그의 동작 방법
CN111201539B (zh) 用于确定用户行为的匹配场景的方法、介质和计算机系统
US20190332400A1 (en) System and method for cross-platform sharing of virtual assistants
US10088972B2 (en) Virtual assistant conversations
EP2933796B1 (en) Executing software applications on a robot
JP6728319B2 (ja) 人工知能機器で複数のウェイクワードを利用したサービス提供方法およびそのシステム
WO2016185809A1 (ja) 情報処理装置、情報処理方法およびプログラム
KR102508863B1 (ko) 전자 장치 및 상기 전자 장치로부터 수신된 데이터를 처리하는 서버
CN111866565A (zh) 显示装置及显示装置的控制方法
KR20230111061A (ko) 로봇 및 이의 제어 방법
WO2020195821A1 (ja) 情報処理装置、情報処理方法、情報処理プログラム
CN112384888A (zh) 基于上下文状态的用户界面格式适应
US20250182636A1 (en) Immersive interview with real-time interactive feedback in augmented reality
CN110945455A (zh) 处理用户话语以用于控制外部电子装置的电子装置及其控制方法
WO2020153146A1 (ja) 情報処理装置、及び情報処理方法
CN116775453A (zh) 用于提供具有可切换模型的自主驾驶模拟架构的系统和方法
US10976997B2 (en) Electronic device outputting hints in an offline state for providing service according to user context
JP7176228B2 (ja) 情報処理装置及びプログラム
US12023798B2 (en) Customizing setup features of electronic devices
WO2024162080A1 (ja) 提案方法、提案装置および提案プログラム
US12417767B2 (en) Electronic device and method to control external apparatus
US20210398158A1 (en) Modifying a policy of an input/output device in response to external promotional events
KR102865668B1 (ko) 사용자 발화를 처리하는 전자 장치와 그 동작 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24750031

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024574463

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 24750031

Country of ref document: EP

Kind code of ref document: A1