WO2006120636A2

WO2006120636A2 - Method and device for providing help to a user in using a device by using a robot

Info

Publication number: WO2006120636A2
Application number: PCT/IB2006/051442
Authority: WO
Inventors: Albertus Josephus Nicolaas Breemen; Elmo Marcus Attila Diederiks; Henriette Christine Marie Hoonhout
Original assignee: Koninklijke Philips Electronics N. V.
Priority date: 2005-05-10
Filing date: 2006-05-09
Publication date: 2006-11-16
Also published as: TW200712807A; WO2006120636A3

Abstract

This invention relates to a method and a device for supplying help to a user when using a device, wherein the help is obtained using a robot. When the user requests for such help, the request is processed and subsequently transmitted to a robot. After receiving the help request, the robot approaches the device and helps the user, either by requesting the user directly, or via communicating with the device.

Description

METHOD AND DEVICE FOR PROVIDING HELP TO A USER IN USING A DEVICE BY USING A ROBOT

Field of the invention

The present invention relates to a method and a device for providing help to a user in using a device, wherein said help is provided by a robot. The present invention further relates to a robot adapted to provide help to a user in using said device. Background of the invention

Generally, manufacturers of electronic and other goods provide buyers with user manuals including installation and operation methods, function descriptions, notices, etc., in a booklet or electronic form, along with their products.

For personal computers, the typical help function is included in the software package when it is purchased, where the "Help" function in most cases is a special function along with other main functions provided with the software package. By pressing this "Help" function, the user can e.g. receive animated help via showing characters on the screen, which buttons/menus one has to select in case of a problem etc. On the other hand, many electronic devices have either no display or are connected to a screen/television. An example of such devices is a CD player. The only way for the user to find out how to perform various functions is therefore to go through the manual, which followed the device when it was purchased. However, many users consider such procedures as very tedious and too time demanding, especially since for many high tech devices the manual consists of several hundreds of pages. This often makes the search for help extremely difficult and in some cases even impossible. It can of course happen that the user cannot find the manual when needed.

Object and summary of the invention

It is an object of the present invention to solve these problems by providing an animated help function for users when using such devices. According to one aspect, the present invention relates to a method of providing help to a user in using a device, wherein said help is provided by a robot, the method comprising the steps of:

- receiving an input from said user indicating a need for a help in using said device,

- processing said received input for generating a help request, and

- transmitting said help request to said robot, wherein said robot is adapted to receive said help request and based thereon provide said help.

In that way, the user can be provided with help when using all kinds of devices, such as those having no display screen where the user would otherwise manually have to find out how to solve a particular function. Since today's robots, and especially the future models, possess enormous processing capacity and intelligence a very comfortable help function can be obtained. The robot may be adapted to either ask the user directly about the problem he/she needs to have solved, or be adapted to communicate with the device and request the device for the help the user needs.

The help can e.g. be an animated help comprising pointing towards the button(s) the user should press, or simply illustrating the procedure of performing the function. The device does not even have to be very "intelligent", since, in principle, it only needs to be capable of generating and sending said help request to the robot, wherein subsequently the helping procedure can be based on a direct communication between the user and the robot. Preferably, the help request further comprises an identification of the device, so in that way the robot is aware of what kind of device is requesting for help.

In an embodiment said received input is obtained via push or touch button function comprised in said device, whereby pressing said push or touch button said indication for help is obtained. In that way, the user can request for said help in a very easy and comfortable way.

In an embodiment, said received input comprises an oral input from said user. The user can therefore communicate orally with the device, wherein he/she in a very precise way can formulate the type of problem, and this problem can then be presented to the robot by the device. Furthermore, through such an oral input the user can even request for a particular robot (in case the user possesses more than one robot). This makes the whole communication very user friendly. In an embodiment, said help request comprises the ID number of said device for uniquely identifying said device. In that way, in cases where there is more than one device, the robot can uniquely identify which device sends the request. The request might further include the coordinate position of the device, so that the robot also knows the exact location of the device. The robot could of course have all the IDs of the devices along with their locations pre- stored in a memory.

In an embodiment, the method further comprises transmitting help scripts to said robot, wherein said help scripts are assembled from pre- stored help scripts in said device. In that way, the robot can be provided by the help scripts necessary to help the user, even before the robot has approached the device/user. The help script may be included in the help request, or be sent as a separate document.

In an embodiment, the assembling of said help scripts is based on the user's input or the frequency of the user's attempts to perform a particular function on said device. Therefore, based on the oral communication between the user and the device where the user orally describes the problem, e.g. "I don't know how to record on my VCR", the device can process the user's oral request using e.g. a speech recognizer, and subsequently assemble the relevant help scripts, e.g. those relating to the recording procedure. In case the user has been pressing the "REC" button, but is not aware of the fact that by pressing the "REC" button and the "PLAY" button simultaneously the recording feature is obtained, the device can estimate that the user' s intention is to record. However, in cases where such estimation fails, or the device is not able to estimate what the problem is, the robot might e.g. communicate directly with the user.

In an embodiment, said robot is selected from a number of robots based on at least one selection criteria. This would/could be necessary where the user possesses more than one robot. In that way, only one robot should be selected.

In an embodiment, said selection criteria are based on at least one selection parameter from a group of selection parameters consisting of:

- the coordinate position of said robot towards said device,

- the functional features provided by the robots, - the user's favorable robot, and

- whether or not the robots are busy.

Therefore, the robot can be selected so that waiting time for receiving the help is minimized, where the selection is based on the robot which is closest to the device, or that the provided help is the most comfortable help where the robot has the technical features which are the most favorable (e.g. color display), or that the provided help is very personal where the selected robot is the user's favorable robot. Also, some of the robots might simply not be available because they are solving other tasks. In a further aspect, the present invention relates to a computer readable medium having stored therein instructions for causing a processing unit to execute said method.

According to another aspect the present invention relates to a device adapted to provide help to a user in using said device, wherein said help is provided by a robot, comprising: input means for receiving an input from said user indicating a need for help in using said device,

- a processor for processing said received input and generating a help request, and - a transmitter for transmitting said help request to said robot, wherein said robot is adapted to receive said help request and based thereon provide said help. In an embodiment, said input means for receiving said help request comprises a speech recognizer for receiving an oral input from said user. In that way the device is capable of communicating with the user or the robot. According to yet another aspect, the present invention relates to a robot adapted to provide a help to a user in using a device, comprising:

- a receiver for receiving a help request from said device indicating a need for help in using said device, a processor for processing the received help request, and - a communication means for communicating with said user when providing said help.

Therefore, the processing capacity and the intelligence in such robots can further be utilized in providing animated help functions for the users.

In an embodiment, said processor is, in case of more than one robot being present, adapted to select the robot to provide said animated help function. In such cases, the robots communicate with each other and, e.g. based on the previously mentioned selection criteria, determine which one should help the user in the specific case. These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. Brief description of the drawings

In the following preferred embodiments of the invention will be described referring to the figures, where

figures 1 and 2 show one embodiment according to the present invention where a user, which is using a device, requests for help, figure 3 shows a device according to the present invention, figure 4 shows one embodiment of a robot according to the present invention, figure 5 is a flowchart according to one embodiment of the present invention, where, after starting the device, the user needs help to perform a function on the device, figure 6 is a flowchart according to another embodiment of the present invention, figure 7 is a flowchart seen from the robot point of view.

Description of preferred embodiments

Figure 1 shows one embodiment according to the present invention where a user 102, which is using a device 101, requests for help. The device 101 can be any kind of electronic device, such as CD device or the like, which is typically not connected to a screen/television to show help animations. As illustrated here, the user 102 indicates that he/she needs help in using the device 101 by e.g. pressing a "help- button", whereby a help request 104 is issued or generated and transmitted to a robot 103. In that way the robot 103 is informed that help is needed on this device 101. The help request 104 preferably contains information, which uniquely identifies the device, which can be issued simultaneously when pressing the button. In a preferred embodiment, a help script is extracted from a pre- stored help file within the device 101 and transmitted to the robot 103, either as a separate document or as an additional data file comprised in said help request 104. The robot can be any kind of machine or mechanical device, which possesses high processing capacity and preferably can operate automatically with humanlike skills.

The received input from the user 102 is in one embodiment based on frequency performed by the user of performing particular functions on the device 101, e.g. the user 102 has been trying to record on his/her VCR by pressing the "REC" button a number of times. This frequency is detected by the device 101, which based thereon issues said help request 104. The device 101 might further be adapted to trigger different help scripts relating to the recording procedure. In such a case, the device 101 estimates that the user 102 needs help for recording on the device 101, and based thereon extracts partial help script relating to this subject. In cases where it is impossible to estimate what kind of help the user 102 needs, the robot 103 might be adapted to communicate directly with the user 102 to find out what kind of help is needed by the user.

In one embodiment the help scripts comprise a complete help script for this particular device 101 or, as mentioned previously, a partial help script relating to the particular help that the user 102 needs, e.g. a script relating to recording on the user's VCR. Furthermore, the help request 104 preferably comprises an ID number of the device 101 for uniquely identifying the device along with its coordinate position. The robot 103 may also be preprogrammed in a way that it is already aware of the location of the device 101 with this particular ID number. The help request 104 and help scripts are preferably in a markup language for this particular robot 103, such as for creating physical animations.

In another embodiment the help request 104 comprises only a request to help the user 102. In this embodiment the actual help needed by the user 102 is established orally from the user 102 towards the robot 103, wherein via speech recognizing process the robot 103 processes the user request 102. This could be the case where the device 101 cannot find out what kind of help was needed. In that way, the robot 103 interacts directly with the device 101 by requesting the device 101 for the help script in order to assist the user 102 with the user's help request. The robot 103 may also be adapted to request for said script from a database 105 over a wireless communication channel 106 such as the internet. In one embodiment the robot 103 has the help script for this device 101 already pre-stored so after processing the oral help request from the user the robot 103 can immediately help the user 102 e.g. via animating to the user how to perform the function the user 102 was not able to perform. Such an animation is shown in Fig. 2 where the robot 103 illustrates to the user 102 via pointing or animating and/or orally how to perform the specific function on the device 101 that the user 102 was not able to perform. In another embodiment the user shows the robot some domestic appliance, and then the robot explains how to use that appliance. For instance, the user might need help on a "hair dryer". The robot recognizes the appliance by some kind of a sensor (e.g. a camera) and based thereon helps the user.

In the case where the number of robots is more than one, the robot 103 must be selected from said number of robots based on various selection criteria. The criteria can comprise which one of the robots is closest to the device 101, which of the robots is the most suitable to be used for this particular device (e.g. it can be preferred that the robot comprises a color display), or the user 102 might prefer a particular robot, or his/her favorable robot. Such a selection procedure can either be performed on the device 101 side, or on the robot side. In the former case, the device 101 determines, based on said one or more selection criteria, which one to use. In case the selection criteria comprises which robot is closest to the device 101 the device might transmit a request to all the robots about their coordinate position of the robots, and based on responses from all the roots selects the closest one. The device 101 might also be adapted to receive a request from the user 102, such as an oral request, where the user can instruct the device 101 which robot he/she prefers.

In the latter case where the robots select which one to use, the device 101 transmits a help request 104 to all the robots. By communicating with each other, e.g. by requesting each other for data giving the coordinate position or the location of the surrounding robots along with the identification of the robots, the robots internally determine which one has the shortest distance to the device 101. In one embodiment, the selected robot 103 then transmits data to the device identifying itself towards the device and confirms that this robot is the one who will assist the user 102. Another selection criteria used by the robots to select a robot to help the user 102 is whether or not the robot(s) is/are available, and/or which one of the robots has the help script for this device 101 pre-stored. Now, the device 101 can e.g. transmit a complete or partial help script to this selected robot (assuming the robot does not already have it stored), or the robot might download the script directly from a server (database) 105 over the internet 106 or even from one of the surrounding robots. During or after executing the received help script, the robot 103 can start providing the user with the help needed. If it was not possible to estimate what kind of help the user 102 needs, the robot may be adapted to receive oral information from the user 102 about the help needed, e.g. "I don't know how to record on my VCR", whereby this user input is then processed and used for providing said animated help function for the user.

In one embodiment, if the user 102 is not able to perform the task despite the help provided by the robot, the robot 103 communicates directly with the device, which then carries out the function for the user. Figure 3 shows a device 101, 301 according to the present invention comprising an input means (I_M) 305 for receiving help request from the user 102, a memory (M) 302 having e.g. pre-stored help scripts, a processor (P) 303 which is e.g. adapted to generate said help request 104 based on a command from said user 102, wherein the processor (P) 303 may further be adapted to trigger different help scripts from the memory (M) 302. The command from the user 102 can be an oral command wherein the processor 303 processes the oral command into a request, which is to be transmitted to the particular robot. Finally, the device 101, 301 comprises a transmitter (T) 304 for transmitting said help request and (when needed) said help scripts 104 to said robot 103. The memory 302 is not essential for the present invention, since the robot

103 might, as mentioned previously, obtain all the help scripts form a server (database) 105 over a wireless communication channel 106, e.g. the internet.

Figure 4 shows one embodiment of a robot 103, 401 according to the present invention comprising a receiver (R) 404, a memory (M) 402, a processor (P) 403 and communication means (C_M) 405. The receiver (R) 404 is e.g. adapted for receiving said help request 104 from the device 101, 301. The processor (P) 403 is e.g. adapted to process the various help requests received by the robot or for e.g. determining which robot from said number of robots should be used in helping the user 102. The communication means (C_M) 405 is adapted to provide any kind of help for the user 102 and can e.g. comprise a display for displaying a help function for the user or for displaying an animated help. The communication means (C_M) can further comprise movable parts, e.g. an artificial hand, for creating physical animation for the user, such as to animate for the user how to record on the user' s VCR. The communication means (C_M) 405 can also comprise a speech recognizer for communicating orally with the user 102. In that way the user 102 can, as previously discussed, formulate the problem orally to the robot, i.e. "how do I record on this VCR". Shown is also a database 406 which the robot 103, 401 may access, e.g. in cases where the device does not have help scripts pre-stored.

Figure 5 is a flowchart according to one embodiment of the present invention, where after starting (S) 501 the device, the user needs help to perform a function on the device, e.g. to record or to adjust the timer. This can be indicated by the user by means of e.g. pressing a help button on the device and in that way generating a help request (H_R) 503. Another way of indicating this assistance could be based on the number of attempts to perform said function, e.g. the user has several times pressed the "REC" button on the VCR, whereas the correct way of recording would be to press the "REC" and the "PLAY" buttons simultaneously. However, the attempts made by the user might indicate the user's interest to record. In that way, the device can be adapted to trigger different help scripts (T_H) 505 relating to the recording procedure. It follows that a downloadable help script (T_H) 505 is formulated for the robot, preferably in a markup language for the robots for e.g. creating physical or any other kind of animations for the user. In this embodiment, where more than one robot is in the surrounding of the device, the device further selects the robot (S_R) 507 to be used for helping the user. As discussed previously under Figs. 1-3, the selection may be based on at least one selection parameter indicating which of the robots is closest to the device, which robot has the technical features which are favorable to this particular device (e.g. the robot has a color display, arms etc), or which robot is the user's "favorable" robot (the device could e.g. be adapted to request the user what robot he/she prefers).

The help request (H_R) 503 and help scripts (T_H) 505 are now transmitted by the device (T_H_S) 509 to the selected robot, which then executes the request/script and approaches the user and helps him/her (H_U) 511. The robot might further be adapted to request the user whether he/she needs more help (M_H) 513, if yes the processing steps from 505-511 are repeated, otherwise the helping function provided by the robot ends (E) 515. Figure 6 is a flowchart according to another embodiment of the present invention, where after starting (S) 601 the device, the user needs help to perform a function on the device, where the user indicates this by e.g. pressing a help button, whereby doing so the device receives a help request (H_R) 603. Subsequently, the device triggers different help scripts (T_H) 605, and subsequently transmits (T_H_S) 607 a help script to one or more robots. The difference between this embodiment and the embodiment in Fig. 4 is that, in cases where one or more robots exist in the surroundings of the device, the robots select which one should be used to assist the user. After selection, the robot approaches the user and offers him/her help (H_U) 609. The robot might further be adapted to request the user whether he/she needs more helps (M_H) 611, if yes the processing steps from 605-609 are repeated, otherwise the helping function provided by the robot ends (E) 613.

In still another embodiment, the robot could be adapted to assist the user by communicating directly with the user. In that way, said help button could be used for "calling" the robot, i.e. only transmitting the help request, whereby the device is simultaneously uniquely identified by the robot. In this embodiment, the robot which receives the "call", approaches the device and asks the user what the problem is. The device itself could therefore only be adapted to call the robot, so that the intelligence is more or less on the robot side. Figure 7 is a flowchart seen from the robot point of view, beginning with

(S) 701 the robot receiving a help request (R_H_S) 703 from the device, indicating what device is requesting for help. Also, a help script (H_S) 705 from the device is/may be received, which either can be sent as a separate document or be comprised in the help request. After processing the received documents the robot approaches the device and helps the user (H_U) 707. This help assistance can be in the form of e.g. pointing, animating, displaying etc. as discussed previously. The robot may further be adapted to request the user whether he/she needs more help (M_H) 709. If the answer is yes, the process steps from 703-707 are repeated, otherwise the help is ended (E) 711. It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of other elements or steps than those listed in a claim. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. A method of providing help to a user (102) in using a device (101, 301), wherein said help is provided by a robot (103, 401), the method comprising the steps of: receiving an input from said user (102) indicating a need for a help in using said device (101, 301), - processing said received input for generating a help request (104), and transmitting said help request (104) to said robot (103, 401), wherein said robot is adapted to receive said help request (104) and based thereon provides said help.

2. A method according to claim 1, wherein said received input is obtained via push or touch button functions comprised in said device (101, 301), whereby pressing said push or touch button said indication for help is obtained.

3. A method according to claim 1, wherein said received input comprises an oral input from said user (102).

4. A method according to claim 1, wherein said help request (104) comprises the ID number of said device (101, 301) for uniquely identifying said device.

5. A method according to any of the preceding claims, further comprising transmitting help scripts to said robot, wherein said help scripts are assembled from pre- stored help scripts in said device (101, 301).

6. A method according to claim 5, wherein assembling said help scripts is based on the user' s input or the frequency of the user' s attempts to perform a particular function on said device (101, 301).

7. A method according to any of the preceding claims, wherein said robot (103, 401) is selected from a number of robots based on at least one selection criteria.

8. A method according to claim 7, wherein said selection criteria is based on at least one selection parameter from a group of selection parameters consisting of:

- the coordinate position of said robot (103, 401) towards said device (103,

401),

- the functional features provided by the robots, the user's favorable robot, and whether or not the robots are busy.

9. A computer-readable medium having stored therein instructions for causing a processing unit to execute a method according to claim 1-8.

10. A device (101, 301) adapted to provide help to a user (102) in using said device (101, 301), wherein said help is provided by a robot (103, 401), comprising: input means (305) for receiving an input from said user (102) indicating a need for a help in using said device (101, 301), a processor (303) for processing said received input and generating a help request (104), and - a transmitter (304) for transmitting said help request (104) to said robot

(103, 401), wherein said robot is adapted to receive said help request (104) and based thereon provide said help.

11. A device according to claim 10, wherein said input means (305) for receiving said help request comprises a speech recognizer for receiving an oral input from said user (102).

12. A robot (103, 401) adapted to provide help to a user (102) in using a device (101, 301), comprising: - a receiver (404) for receiving a help request (104) from said device (101,

301) indicating a need for help in using said device (101, 301), a processor (403) for processing the received help request (104), and - a communication means (405) for communicating with said user (102) when providing said help.

13. A robot according to claim 12, wherein said processor (403), in case of more than one robot being present, is adapted to select the robot (103, 401) and to provide said animated help function.