KR20180079824A - Home robot and method for operating the same - Google Patents

Abstract

According to an aspect of the present invention, a method for operating a home robot comprises the following steps of: receiving a voice signal through a plurality of microphones; determining a sound source generating position corresponding to the sound signal received through the microphones; rotating a body of the home robot in correspondence with the determined sound source generating position; acquiring user image information through an image acquisition unit; and rotating the body of the home robot based on the user image information. Therefore, various services can be provided in accordance with voice input.

Description

Home robot and method of operating same

The present invention relates to a home robot and a method of operating a smart home system including the same. And more particularly, to a home robot capable of providing various services according to a user's voice input and an operation method thereof.

Conventionally, home appliances such as a washing machine, an air conditioner, and a vacuum cleaner, which are used in a predetermined space such as a home or an office, individually perform their own functions and operations.

For example, the refrigerator stores food, the washing machine processes the laundry, the air conditioner adjusts the room temperature, and the cooking device performs cooking and the like.

BACKGROUND ART [0002] In recent years, with the development of various communication technologies, a plurality of home appliances are network-connected by wired / wireless communication.

The home appliances constituting the network can transmit data from one device to another, and information of another device can be confirmed from any one of the devices.

In addition, by configuring a network including a smart device such as a mobile terminal, a user can check and control information of home appliances using his / her smart device anytime and anywhere.

The network of such in-home devices is also referred to as a smart home.

The prior art (Application No. 10-2003-0093196) relating to smart home technology constitutes a home network including a washing machine and the like.

The conventional art can communicate with the user himself / herself based on the presence and humidity of the laundry after a certain period of time after the end of washing, and follow-up measures are taken according to the result of the washing. Therefore, damage due to long- .

That is, after the laundry is finished, the user transmits a question message asking whether the laundry is left unattended and whether the follow-up action is to be performed, and if a response is received, the follow-up action is performed according to the response.

As in the related art, the conventional smart home technology remains at a level to provide or control information of a specific device to a user through a portable terminal even when the user is in the home. Further, there has been an inconvenience that the user has to operate the portable terminal every time.

On the other hand, robots have been developed for industrial use and have been part of factory automation. In recent years, medical robots, aerospace robots, and the like have been developed, and home-use robots that can be used in general households are also being developed.

Accordingly, various schemes for increasing the usability of the user by controlling the home appliances constituting the network through the robot have been studied.

Application No. 10-2003-0093196 (filed December 18, 2003)

SUMMARY OF THE INVENTION It is an object of the present invention to provide a home robot and an operation method thereof that can improve user convenience by operating according to voice input.

It is an object of the present invention to provide a home appliance control apparatus and a control method thereof that can control the home appliances constituting a network through a robot so that the user does not need to operate other devices such as a portable terminal and a home appliance, .

It is an object of the present invention to provide a home robot capable of providing various services as well as a home appliance control and an operation method thereof.

According to another aspect of the present invention, there is provided a method of operating a home robot including receiving a voice signal through a plurality of microphones (MIC), inputting a sound signal corresponding to a voice signal received through a plurality of microphones, Determining a generation position, rotating a body of the home robot corresponding to the determined sound source generating position, acquiring user image information through the image acquiring unit, And rotating the body of the robot, thereby providing various services according to the voice input.

According to one aspect of the present invention, there is provided a home robot comprising: a body defining an internal storage space; a support disposed below the body to support the body; A head positioned on the upper side of the body and having an image acquisition unit disposed on the front side thereof, a voice input unit including a plurality of microphones (MIC) receiving audio signals, and a voice input unit corresponding to voice signals received through the plurality of microphones And a control unit for rotating the body corresponding to the determined sound source generating position, wherein the controller rotates the body based on the user image information obtained through the image obtaining unit after the rotation of the body Can be controlled.

According to at least one of the embodiments of the present invention, since the user operates according to the voice input, the user does not need to operate other devices such as a portable terminal, a home appliance, and the like, thereby improving user convenience.

In addition, according to at least one embodiment of the present invention, since the home appliances constituting the network are controlled through the robot, the user does not have to operate other devices such as the portable terminal and the home appliance, Can be increased.

In addition, according to at least one embodiment of the present invention, a home robot capable of providing various services as well as a home appliance control and an operation method thereof can be provided.

Meanwhile, various other effects will be directly or implicitly disclosed in the detailed description according to the embodiment of the present invention to be described later.

1 is a block diagram of a smart home system including a home robot according to an embodiment of the present invention.
2 is a front view showing an appearance of a home robot according to an embodiment of the present invention.
3 is an example of a simplified internal block diagram of a home robot according to an embodiment of the present invention.
4 is a flowchart illustrating an operation method of a home robot according to an embodiment of the present invention.
5 is a flowchart illustrating an operation method of a home robot according to an embodiment of the present invention.
6 is a flowchart illustrating an operation method of a home robot according to an embodiment of the present invention.
7 is a flowchart illustrating an operation method of a smart home system including a home robot according to an embodiment of the present invention.
FIGS. 8 to 18C are views referred to the description of the operation method of the home robot and the smart home system including the same.

1 is a block diagram of a smart home system including a home robot according to an embodiment of the present invention.

Referring to FIG. 1, a home robot system according to an embodiment of the present invention includes a home robot 100 and a communication module to communicate with other devices, a home robot 100, a server 70, And a home appliance 10 that can be connected to the home appliance.

For example, the home appliance 10 may include an air conditioner 11 having a communication module, a robot cleaner 12, a refrigerator 13, a washing machine 14, a cooking appliance 15, and the like.

Meanwhile, the communication module included in the home appliance 10 may be a wi-fi communication module, and the present invention is not limited to the communication method.

Alternatively, the home appliance 10 may include other types of communication modules or a plurality of communication modules. For example, the home appliance 10 may include an NFC module, a zigbee communication module, a Bluetooth communication module, and the like.

The home appliance 10 can be connected to a predetermined server 70 through a wi-fi communication module or the like, and can support smart functions such as remote monitoring and remote control.

The home robot system according to an embodiment of the present invention may include a portable terminal (not shown) such as a smart phone, a tablet PC, and the like.

The user can check information on the home appliances 10 and 20 in the home robot system or control the home appliance 10 through the portable terminal.

On the other hand, it may be inconvenient for a user to use the portable terminal even when the user wants to control the home appliance 10 or confirm predetermined information in the home.

For example, it is more efficient to have a means to control the home appliance 10 in other ways when the user does not know the current location of the mobile terminal or is in another place.

The home robot 100 according to an embodiment of the present invention can receive user's voice input and control the home appliance 10 directly or control the home appliance 10 via the server 70 can do.

Accordingly, the user can control the home appliance 10 without operating other devices other than the home robot 100 arranged in the room, the living room, and the like.

Meanwhile, the home robot system according to an embodiment of the present invention may include a plurality of Internet (IoT) devices (not shown). Accordingly, the home robot system may include the home appliance 10, the home robot 100, and the Internet (IoT) devices.

The home robot system according to the embodiment of the present invention is not limited to the communication system constituting the network.

For example, the home appliance 10, the home robot 100, and the Internet (IoT) devices can be communicatively connected through a wired / wireless router (not shown).

In addition, the devices in the home robot system can configure a mesh topology that is individually communicatively connected.

The home appliance 10 in the home robot system can communicate with the server 70 or the home robot 100 via a wired / wireless router (not shown).

The home appliance 10 in the home robot system can communicate with the server 70 or the home robot 100 via Ethernet.

The home robot system according to an embodiment of the present invention may further include a network device such as a gateway. Alternatively, at least one of the home robots 100 provided in the home may be configured to include the above-described gateway function.

The home appliances 10 included in the home robot system can be network-connected directly between devices or via a gateway.

Further, the home appliance 10 can be network-connected to be able to communicate with the server 70 directly or via a gateway.

Meanwhile, the gateway can communicate with the server 70 or the mobile terminal 50 by Ethernet.

Also, the gateway can communicate with the server 70 or the home robot 100 via the wired / wireless router.

The home appliance 10 can transmit device operation status information, setting value information, and the like to the server 70 and / or the gateway.

The user can confirm information about the home appliance 10 in the home robot system or control the home appliance 10 through the portable terminal or the home robot 100. [

The server 70 and / or the gateway may be connected to the home appliance 100 in response to a specific event occurring in the home appliance 10 in the home robot system or when there is a user command input through the home robot 100 or the like. 10 can be transmitted to each device.

According to an embodiment of the present invention, the gateway may include output means such as a display and an audio output unit.

In this case, the display and the audio output unit may output images and audio based on signals stored in the gateway or received. For example, the music file stored in the gateway may be reproduced and output through the audio output unit.

Also, the display and the audio output unit may output image and audio information related to the operation of the gateway.

The server 70 can store and manage information transmitted from the home appliance 10, the home robot 100, and other devices.

The server 70 may be a server operated by a manufacturer or a company entrusted by a manufacturer of the home appliance.

Information related to the home appliance 10 can be transmitted to the home robot 100 and the home robot 100 can display information related to the home appliance 10.

The home appliance 10 may receive information or receive commands from the home robot 100. At this time, the home appliance 10 may transmit various information to the server 70, and the server 70 may transmit a part or all of the information received from the home appliance 10 to the home robot 100 Lt; / RTI >

The server 70 may process the received information or the received information from the home appliance 10 and transmit the processed information to the home robot 100.

Although FIG. 1 illustrates the case where the server 70 is one, the present invention is not limited thereto, and the system according to the present invention can operate in conjunction with two or more servers.

For example, the server 70 may include a first server for voice recognition and processing, and a second server for providing a home appliance related service such as a home appliance control.

According to an embodiment, the first server and the second server may be composed of three or more servers in which information and functions are dispersed, or may be composed of one integrated server.

For example, the first server for speech recognition and processing may comprise a speech recognition server for recognizing words included in a speech signal and a natural language processing server for recognizing the meaning of a sentence including words included in the speech signal have.

FIG. 2 is a front view showing the appearance of a home robot according to an embodiment of the present invention, and FIG. 3 is an example of a simplified internal block diagram of a home robot according to an embodiment of the present invention.

Referring to FIGS. 2 and 3, the home robot 100 includes main bodies 101 and 102 for forming an outer tube and housing various components therein.

The main bodies 101 and 102 include a body 101 forming a space for accommodating various components constituting the home robot 100 and a support 101 disposed below the body 101 and supporting the body 101. [ (Not shown).

In addition, the home robot 100 may include a head 110 disposed on the upper side of the main bodies 101 and 102. A display 182 capable of displaying an image may be disposed on the front surface of the head 110.

In this specification, the front direction means the + y axis direction, the up and down direction means the z axis direction, and the left and right direction means the x axis direction.

The head 110 may rotate within a predetermined angle range about the x-axis.

Accordingly, when viewed from the front, the head 110 can perform a nodding operation in which the head moves up and down like a person nods his or her head in a vertical direction. For example, the head 110 may perform the home return operation one or more times after rotating within a predetermined range as if a person nods his / her head in the vertical direction.

Meanwhile, depending on the embodiment, at least a part of the front surface on which the display 182 corresponding to the face of the person in the head 100 is disposed may be configured to be nodded.

However, unless specifically described, the operation in which the head 110 is vertically nodded is performed by the display 182 (see FIG. 2) May be replaced with an operation of nodding in the up-and-down direction.

The body 101 may be configured to be rotatable in the left-right direction. That is, the body 101 may be configured to rotate 360 degrees about the z-axis.

In addition, according to the embodiment, the body 101 is also configured to be rotatable within a predetermined angle range about the x-axis, so that it can move as if it nods up and down. In this case, as the body 101 rotates in the up-and-down direction, the head 110 can rotate together with the axis on which the body 101 rotates.

Accordingly, in the present specification, the operation of nodding the head 110 in the up-and-down direction is the same as that in the case where the head 110 itself rotates in the vertical direction when viewed from the front about a predetermined axis, And the head 110 connected to the body 101 is nodded by being rotated together.

Meanwhile, the home robot 100 may include a power supply unit (not shown) connected to the household socket and supplying power to the home robot 100.

Alternatively, the home robot 100 may include a power supply unit (not shown) provided with a rechargeable battery (not shown) to supply power into the home robot 100. According to an embodiment, a power supply unit (not shown) may include a wireless power receiving unit for wirelessly charging the battery.

The home robot 100 may include an image acquisition unit 120 that can capture a predetermined range around the main bodies 101 and 102 and at least the front faces of the main bodies 101 and 102.

The image acquiring unit 120 photographs the surroundings of the main bodies 101 and 102, the external environment, and the like, and may include a camera module. The camera module may include a digital camera. A digital camera includes an image sensor (e.g., a CMOS image sensor) configured with at least one optical lens, and a plurality of photodiodes (e.g., pixels) that are formed by light passing through the optical lens, And a digital signal processor (DSP) that forms an image based on signals output from the photodiodes. The digital signal processor is capable of generating moving images composed of still frames as well as still images.

Several cameras may be installed for each part of the camera for photographing efficiency. Preferably, the image acquisition unit 120 may include a front camera provided on the front surface of the head 110 to acquire images of the front surfaces of the main bodies 101 and 102. However, the number, arrangement, type, and photographing range of the cameras provided in the image obtaining unit 120 are not necessarily limited to these.

The image obtaining unit 120 can photograph the front direction of the home robot 100 and can photograph a user recognition image.

In addition, the image captured and obtained by the image acquisition unit 120 may be stored in the storage unit 130.

Also, the home robot 100 may include a voice input unit 125 for receiving a voice input of a user.

The voice input unit 125 may include a processor for converting the analog voice into digital data or may be connected to the processor to convert the user input voice signal into data to be recognized by the server 70 or the controller 140. [

The voice input unit 125 may include a plurality of microphones to increase the accuracy of user voice input reception and to determine the position of the user.

For example, the voice input unit 125 may include at least two microphones.

A plurality of microphones (MICs) may be disposed at different positions, and an external audio signal including a voice signal may be acquired and processed as an electrical signal.

On the other hand, a microphone, which is an input device, requires at least two sound sources for generating sounds and a user's direction, and the resolution (angle) of direction detection is higher as the distance between the microphones is physically far.

Depending on the embodiment, two microphones may be placed in the head 110.

Further, by including two microphones on the rear surface of the head 110, it is possible to determine the position of the user on the three-dimensional space.

2, the home robot 100 includes a control unit 140 for controlling overall operation, a storage unit 130 for storing various data, and a communication unit 190 for transmitting and receiving data with other devices such as the server 70 .

The home robot 100 may further include a driving unit 160 that rotates the head 110 and the body 101. The driving unit 160 may include a plurality of driving motors (not shown) for rotating and / or moving the body 101 and the head 110.

The control unit 140 controls the overall operation of the home robot 100 by controlling the image acquisition unit 120, the driving unit 160, the display 182, and the like, which constitute the home robot 100.

The storage unit 130 records various types of information required for controlling the home robot 100, and may include a volatile or nonvolatile recording medium. The storage medium stores data that can be read by a microprocessor, and includes a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD- Tape, floppy disk, optical data storage, and the like.

The control unit 140 may transmit the operation state of the home robot 100, user input, etc. to the server 70 or the like through the communication unit 190.

The communication unit 190 includes at least one communication module so that the home robot 100 is connected to the Internet or a predetermined network.

The communication unit 190 is connected to a communication module provided in the home appliance 10 and processes data transmission between the home robot 100 and the home appliance 10. [

Meanwhile, the storage unit 130 may store data for voice recognition, and the control unit 140 may process the voice input signal of the user received through the voice input unit 125 and perform a voice recognition process.

Since a variety of known speech recognition algorithms can be used for the speech recognition process, a detailed description of the speech recognition process will be omitted herein.

On the other hand, the control unit 140 can control the home robot 100 to perform a predetermined operation based on the speech recognition result.

For example, when the command included in the voice signal is a command for controlling the operation of a predetermined home appliance, the controller 140 may control to transmit a control signal based on the command included in the voice signal to the control target home appliance have.

If the command included in the voice signal is a command for controlling the operation of the predetermined home appliance, the control unit 140 may control the body 101 of the home robot to rotate in the direction toward the control target home appliance.

Meanwhile, the speech recognition process may be performed in the server 70 without being performed in the home robot 100 itself.

In this case, the control unit 140 may control the communication unit 190 so that the user input voice signal is transmitted to the server 70.

Alternatively, simple speech recognition may be performed by the home robot 100, and high-dimensional speech recognition such as natural language processing may be performed by the server 70. [

For example, when a keyword voice input including a predetermined keyword is received, the home robot can be switched from a standby state to an operating state. In this case, the home robot 100 performs only a voice recognition process up to whether a keyword voice is input, and voice recognition for a subsequent user voice input can be performed through the server 70. [

Meanwhile, according to an embodiment, the control unit 140 may compare the user image acquired through the image acquisition unit 120 with information stored in the storage unit 130 to determine whether the user is a registered user.

Also, the control unit 140 can control the voice input of the registered user to perform a specific operation only.

On the other hand, a registered user can be set to have different control rights. For example, the parent of the family members can be set to have the comprehensive control authority of the home robot 100 and the home appliance 10. [ In addition, children can set a restriction on the control authority so as not to use a specific device or a specific function. For example, in order to prevent a safety accident, the children are set to have no control authority of the washing machine 14 and the cooking appliance 15, or to set the control authority of some of the functions of the washing machine 14 and the cooking appliance 15 .

The control unit 140 may control the head 110 to be nodded when a user having control rights to the home robot 100 and the home appliance 10 is identified. Accordingly, the user intuitively knows that the home robot 100 has identified himself / herself.

The control unit 140 may control the rotation of the body 101 and / or the head 111 based on the user image information obtained through the image acquisition unit 120.

The control unit 140 may rotate the body 101 in the lateral direction based on the user image information. For example, in a case where the number of faces included in the user image information is one, the body 101 is moved in the left-right direction so that the face included in the user image information is located at the center of the camera of the image obtaining unit 120 It can rotate.

The controller 140 controls the head 111 to rotate up and down so that the display 182 faces the face included in the user image information so that the user can more easily be displayed on the display 182 You can have the information checked.

In addition, there is an advantage that it is advantageous to perform a specific operation such as user identification of the home robot 100 and photographing afterwards.

Accordingly, interaction and communication between the user and the home robot 100 can be facilitated.

If the number of the faces included in the user image information is plural, the controller 140 determines that the average value of the plurality of face positions included in the user image information is greater than the average value of the plurality of face positions included in the user image information, So that the body 101 of the home robot 100 is rotated.

Since the home robot 100 is used in the home, a plurality of family members can use the home robot 100 together. In this case, it may be more natural for the home robot 100 to be directed to the average value of the positions of a plurality of users when there is another person who is adjacent to the talking user, rather than toward one of the family members.

Further, there is an advantage in that it is advantageous to perform a specific operation such as identification of a plurality of users, group photographing for a plurality of users such as a family member, and the like.

On the other hand, according to the embodiment, even if a plurality of users are recognized, they can be set to face the speaker who uttered the voice.

Meanwhile, the control unit 140 may control the body 101 to rotate based on at least one of the number of faces, the position information of the face, and the area information included in the obtained user image information.

The face size of the image acquired by the image acquisition unit 120 varies depending on the distance between the user and the home robot 100. Accordingly, the home robot 100 can control to look toward the optimized position by considering the number of users included in the image acquired by the image acquisition unit 120, as well as the face area information.

On the other hand, the home robot 100 may include the output unit 180 to display predetermined information as an image or output sound.

The output unit 180 may include a display 182 for displaying information corresponding to a user's command input, a processing result corresponding to a user's command input, an operation mode, an operation state, an error state,

The display 182 may be disposed on the front side of the head 110 as described above.

According to an embodiment, the display 182 may be a touch screen having a mutual layer structure with a touch pad. In this case, the display 182 may be used as an input device capable of inputting information by a user's touch in addition to the output device.

The output unit 180 may further include an audio output unit 181 for outputting an audio signal. The sound output unit 181 outputs a notification message such as a warning sound, an operation mode, an operation state, and an error state, information corresponding to a command input by the user, a processing result corresponding to a command input by the user, It can output sound. The sound output section 181 can convert an electric signal from the control section 140 into an audio signal and output it. For this purpose, a speaker or the like may be provided.

Referring to FIG. 2, the sound output unit 181 is disposed on the left and right sides of the head 110, and can output predetermined information as sound.

On the other hand, the appearance and structure of the home robot illustrated in Fig. 2 are illustrative, and the present invention is not limited thereto. For example, the positions and the numbers of the sound input unit 125, the image acquisition unit 120, and the sound output unit 181 may vary according to design specifications, and the rotational direction and angle of each component may be different.

4 is a flowchart illustrating an operation method of a home robot according to an embodiment of the present invention.

Referring to FIG. 4, the home robot 100 according to an embodiment of the present invention can receive a voice signal through a plurality of microphones (MIC) included in the voice input unit 125 (S410). Here, the voice signal may be a signal input after a keyword voice input including a preset keyword is received and the home robot 100 is switched from a standby state to an operating state, or a signal including a predetermined keyword have.

The control unit 140 may determine a sound source position (sound source) corresponding to the sound signal received through the plurality of microphones (S420).

The control unit 140 can determine a sound source location using a time-delay-of-arrival (TDOA) of a signal received by a plurality of microphones.

Meanwhile, the present invention may use another sound source position estimation technique known in addition to the TDOA. For example, various methods such as a method using a steered beam former and a method using high resolution spectrum estimation can be used.

The control unit 140 may calculate a time difference of a voice signal received through the plurality of microphones and may estimate a direction and / or a distance of the sound source corresponding to the voice signal based on the calculated time difference.

Meanwhile, the controller 140 may control the rotation of the body 101 of the home robot 100 according to the determined sound source position (S430).

For example, the control unit 140 may control the body 101 to rotate in the left-right direction so that the home robot 100 faces the identified sound source generating position.

The control unit 140 may acquire a predetermined image by photographing the entire surface of the home robot 100 through the image acquisition unit 120 after rotating the body 101 in the left and right directions, 120) to process user image information (S450).

The control unit 140 can identify the face information in the image acquired through the image acquisition unit 120 using a face recognition technique. Here, the user face information may include at least one of the number of faces in the image acquired through the image acquisition unit 120, the position information of the face, and the area information.

Face recognition algorithms have been available since the 1970s, and a wide variety of solutions are now widely used. For example, it is used in doorway control systems, security systems, real-time surveillance systems, video conferencing, user authentication, file encryption, and distance education.

Accordingly, various techniques are disclosed for face recognition, and various known face recognition algorithms can be used in the present invention. A method of detecting a geometric body feature element, a method of detecting a face by detecting edge information and discriminating the surrounding data, and the like.

For example, if fast recognition speed is required, a face recognition technique using geometric features of a face can be used. This technique is one of the commonly used methods among various face recognition technologies. The technique is to recognize faces of individuals using geometric factors, such as the location, size, and distance between the feature points of the face, such as eyes, nose, and mouth. Here, such geometric features may be included in the elements that are finally left when the resolution of the input image is lowered.

Meanwhile, the control unit 140 may control the body 101 of the home robot 100 to rotate based on the user image information (S460).

In other sound source position estimation techniques known in addition to the TDOA method, the longer the distance between the plurality of microphones becomes, the larger the time difference becomes, and the accuracy of the position estimation becomes higher.

However, the size of the home robot 100 is limited, and a method of increasing the number of microphones to increase the accuracy of position estimation invites an increase in manufacturing cost.

Therefore, according to the present invention, after the home robot 100 performs the first rotation in the direction toward the user according to the sound source position estimation technique, the user recognizes the user's face according to the image recognition technology and performs the second rotation, , And the direction corresponding thereto can be seen.

Accordingly, the user can confirm reliability of the home robot 100 and speech recognition by confirming the moving home robot 100 in response to his / her voice input.

In addition, there is an advantage that it is advantageous to perform a specific operation such as user identification of the home robot 100 and photographing afterwards.

The controller 140 controls the head 110 of the home robot 100 to rotate up and down based on the obtained user image information so that the display 182 of the home robot 100, Can be controlled so that the front surface on which the front surface is disposed faces the user.

Accordingly, the user can more easily confirm the information displayed on the display 182. [

If the number of faces included in the user image information is one, the control unit 140 controls the body 101 to face the face included in the user image information to the center of the camera of the image obtaining unit 120. [ Can be rotated in the left-right direction.

In addition, when the number of faces included in the user image information is plural, the controller 140 may determine that the average value of the plurality of face positions included in the user image information is greater than the average value of the plurality of face positions included in the user image information, So that the body 101 of the home robot 100 is rotated.

Since the home robot 100 is used in the home, a plurality of family members can use the home robot 100 together. Therefore, it may be more natural for the home robot 100 to move according to the number of recognized persons.

Therefore, in an embodiment of the present invention, the user's face information including the number of faces is discriminated from the image captured and acquired by the image obtaining unit 120, thereby performing appropriate rotation according to the number of people, ) To give emotional satisfaction to family members.

The control unit 140 may control the communication unit 190 to transmit the received voice signal S510 to the server 70 and the server 70 may perform a voice recognition process on the received voice signal can do.

In addition, the server 70 may perform an operation corresponding to a command included in the speech recognition result.

For example, when the command included in the voice recognition result is an instruction to operate the washing machine 14 with a specific washing course, the server 70 transmits a control signal to operate the washing machine 14 to perform a specific washing course can do. Also, the server 70 may transmit feedback information including information indicating that the washing machine 14 has been driven to the home robot 100.

Alternatively, in the case where the command included in the voice recognition result is an instruction for requesting specific information, the server 70 may search the corresponding information through the Internet network or may be based on information received from the home appliance 10 Feedback can be made.

Meanwhile, the communication unit 190 may receive feedback information corresponding to a command included in the voice signal from the server 70

The control unit 140 may control the output unit 180 to output the feedback information received from the server 70. [ The output unit 180 may output an image and / or sound based on the feedback information. For example, the display 182 may display the feedback information as a predetermined image, and the sound output unit 181 may output the feedback information as sounds, sound effects, and the like.

Alternatively, the control unit 140 can directly recognize the command included in the received voice signal (S510), and can control to output information based on the recognition result of the voice signal.

Alternatively, after the rotation (S460), the home robot 100 may receive a voice signal including a predetermined command through the voice input unit 125, and the control unit 140 may receive the voice including the predetermined command It is possible to recognize that the command included in the signal is recognized and output information based on the recognition result of the command.

Alternatively, after the rotation (S460), the home robot 100 may receive a voice signal including a predetermined command through the voice input unit 125, and the control unit 140 may receive the voice including the predetermined command It is possible to control the communication unit 190 to transmit the signal to the server 70. [

In this case, when the feedback information corresponding to the predetermined command is received from the server 70 through the communication unit 190, the control unit 140 can control to output the feedback information.

The controller 140 controls the driving unit 160 to nod the head 110 in the up and down direction after receiving the audio signal S410 or after the rotation S460 Can be controlled. Accordingly, the user can intuitively know that the home robot 100 has received his / her voice input.

Meanwhile, the home robot 100 according to an embodiment of the present invention can switch from the standby state to the operating state upon receiving the keyword voice signal corresponding to the preset keyword. It is possible to notify the user that the image displayed on the display 182 of the head 110 has been changed to the operating state.

That is, the image displayed on the display 182 in the standby state can be switched to the image in the operating state.

For example, while the image including specific information is not displayed on the display 182, the display state of the home robot 100 may be expressed as a change in expression by displaying a graphic image similar to a human face or eye .

5 is a flowchart illustrating an operation method of a home robot according to an embodiment of the present invention.

Referring to FIG. 5, the home robot 100 according to an embodiment of the present invention can receive a voice signal through a plurality of microphones of the voice input unit 125 (S510).

If the direction in which the voice signal is received, that is, the sound source generation direction is not the front direction of the home robot 100, the control unit 140 may determine the direction in which the voice signal is received (S520) The body 101 of the home robot 100 may be rotated in the direction in which the voice signal is received (S525).

In addition, the step S525 of rotating the body 101 includes the image acquiring step S440 described with reference to Fig. 4 to the rotation step S460 based on the user image information, And can be rotated toward the user.

The control unit 140 of the home robot 100 may control the driving unit 160 to nod the head 110 in the vertical direction at step S530. Accordingly, the user can intuitively know that the home robot 100 has received his / her voice input.

The control unit 140 may perform a voice recognition process on the voice signal (S540), and may perform control to perform an operation corresponding to a command included in the voice recognition result.

For example, when the command included in the voice signal is a command for controlling the operation of the predetermined home appliance (S550), the controller 140 transmits a control signal based on the command included in the voice signal to the control target home appliance (S560).

Also, the control unit 140 may control the output unit 180 to output information based on the recognition result of the audio signal (S570).

In addition, the control unit 140 acquires the user image through the image acquisition unit 120, and the control unit 140 may determine whether the user has the control right based on the acquired user image (S527).

The control unit 140 can compare the acquired user image with the stored information in the storage unit 130 to determine whether the user has the control right. The control unit 140 determines whether only the registered user's voice input So as to perform an operation.

If the user having the control right to the home robot 100 and the home appliance 10 is identified, the controller 140 controls the head 110 to be nodded in operation S530.

If the identified user is not a user having control authority, the notification information may be output through the output unit 180 (S529).

Meanwhile, the home robot 100 according to an embodiment of the present invention can receive location information about one or more home appliances in the home from the server 70 or the robot cleaner 12 through the communication unit 170 have.

In this case, when the command included in the voice signal is a command for controlling the operation of the predetermined home appliance, the control unit 140 of the home robot 100 rotates the body 101 in the direction toward the control target home appliance .

Unlike the present embodiment in which the speech recognition and the home appliance control are performed in the home robot 100, speech recognition and home appliance control can be performed in the server 70 according to an embodiment.

On the other hand, when the home robot 100 receives the keyword voice signal corresponding to the preset keyword, it can switch from the standby state to the operating state. At this time, it is possible to notify the user that the image displayed on the display 182 provided on the head 110 has been changed to the operating state.

6 is a flowchart illustrating an operation method of a home robot according to an embodiment of the present invention.

Referring to FIG. 6, the red robot may receive a keyword speech signal corresponding to a predetermined keyword (S610).

For example, if 'genie' is set to a word set as a keyword, then the home robot in the standby state may be called 'genie', 'hello genie', 'hi genie' And a keyword voice signal including 'genie'.

Upon receiving the keyword voice signal, the control unit 140 may control the home robot 100 to switch to the operating state.

Here, the operational state of the home robot 100 is a state in which the speech signal received together with the speech input including the keyword, the speech input including the keyword, The voice signal can be transmitted to the server 70 for processing or can be processed by the home robot 100 itself.

The use of the keyword as a trigger signal of speech recognition improves the accuracy of speech recognition and prevents waste of system resources.

On the other hand, when the keyword voice signal corresponding to a predetermined keyword is received, the image displayed on the display 182 provided in the head 110 is changed (S620), and the user can be notified that the keyword voice signal has been switched to the operating state .

Thereafter, as described with reference to Figs. 4 and 5, the home robot 100 can receive a voice signal through the voice input unit 125 (S630). The control unit 140 may control the driving unit 160 to nod the head 110 in the vertical direction at operation S650.

The control unit 140 may control the communication unit 190 to transmit the voice signal to the server 70 in step S660 and the server 70 may perform a voice recognition process on the received voice signal have.

In addition, the server 70 may perform an operation corresponding to a command included in the speech recognition result.

The communication unit 190 may receive the feedback information corresponding to the command included in the voice signal from the server 70 in step S670 and the control unit 140 may receive the feedback information from the server 70, The output unit 180 may be controlled to output the feedback information (S680).

As described above with reference to FIGS. 4 and 5, the present embodiment can also determine the direction in which the voice signal is received (S640), and the direction in which the voice signal is received is the front direction of the home robot 100 , The body 101 of the home robot 100 may be rotated in the direction in which the voice signal is received (S645).

In addition, the step S645 of rotating the body 101 includes the image acquiring step S440 described with reference to Fig. 4 to the rotation step S460 based on the user image information, And can be rotated toward the user.

In addition, the control unit 140 acquires the user image through the image acquisition unit 120, and the control unit 140 can determine whether the user has the control right based on the acquired user image (S647).

The control unit 140 can compare the acquired user image with the stored information in the storage unit 130 to determine whether the user has the control right. The control unit 140 determines whether only the registered user's voice input So as to perform an operation.

If the user having control over the home robot 100 and the home appliance 10 is identified, the controller 140 controls the head 110 to be nodded in operation S630.

If the identified user is not a user having the control authority, the notification information may be output through the output unit 180 (S649).

7 is a flowchart illustrating an operation method of a smart home system including a home robot according to an embodiment of the present invention.

Referring to FIG. 7, when the home robot 100 receives a voice signal (S710)

The home robot 100 can determine the direction in which the voice signal is received, that is, the position at which the sound source is generated, and when the direction in which the voice signal is received is not the front direction of the home robot 100, The body 101 of the home robot 100 may be rotated in the direction in which the robot 100 is received (S720).

At this time, the step S720 of rotating the body 101 includes the image acquiring step S440 described with reference to FIG. 4 and the rotating step S460 based on the user image information, To the user.

The controller 140 controls the head 110 of the home robot 100 to rotate up and down based on the obtained user image information so that the display 182 of the home robot 100, Can be controlled so that the front surface on which the front surface is disposed faces the user.

Thereafter, according to the embodiment, the head 110 of the home robot 100 may be vertically nodded (S730).

Meanwhile, the home robot 100 may transmit the voice signal to the server 70 (S740).

The server 70 may recognize a command included in the voice signal by performing a voice recognition process (operation S750).

If the command included in the voice signal is a command for controlling the operation of the predetermined home appliance 10, the server 70 transmits a control signal based on the command included in the voice signal to the control target home appliance 10 (S750).

According to an embodiment, the home appliance 10 may transmit predetermined data such as an operation state to the server 70 and a response to the received control signal.

Meanwhile, the server 70 can transmit feedback information corresponding to a command included in the voice signal to the home robot 100 (S760), and the home robot 100 transmits the feedback information (S770).

7, a single server 70 is illustrated. However, the present invention is not limited to this, and the system according to the present invention can operate in conjunction with two or more servers.

For example, the server 70 may include a first server for voice recognition and processing, and a second server for providing a home appliance related service such as a home appliance control. According to an embodiment, the first server and the second server may be composed of three or more servers in which information and functions are dispersed, or may be composed of one integrated server.

Meanwhile, the home robot 100 acquires a user image through the image acquisition unit 120, and the controller 140 of the home robot 100 determines whether the user has control authority based on the acquired user image Can be distinguished. The control unit 140 can compare the acquired user image with the stored information in the storage unit 130 to determine whether the user has the control right. The control unit 140 determines whether only the registered user's voice input So as to perform an operation.

The home robot 100 may receive positional information on one or more home appliances in the home from the server 70 or the robot cleaner 12 through the communication unit 170. [

In this case, when the command included in the voice signal is a command for controlling the operation of the predetermined home appliance, the control unit 140 of the home robot 100 rotates the body 101 in the direction toward the control target home appliance .

Also, when an instruction from the server 70 is received and feedback information including information indicating that the control target home appliance has started a specific operation or information indicating that the control target home appliance has performed a specific operation is received , The controller 140 may rotate the body 101 toward the user.

Accordingly, the user intuitively knows that the home appliance to be controlled by the user is being controlled.

On the other hand, when the home robot 100 receives the keyword voice signal corresponding to the preset keyword, it can switch from the standby state to the operating state. It is possible to notify the user that the image displayed on the display 182 of the head 110 has been changed to the operating state.

FIGS. 8 to 18C are views referred to the description of the operation method of the home robot and the smart home system including the same.

8 illustrates a home robot 100 in a standby state.

When the power of the home robot 100 is turned on, the home robot 100 can wait in a standby state. At this time, the display 182 may display a specific image 800 corresponding to a standby state.

9 shows an example in which the red robot 100 rotates in the direction toward the sound source generating position as it receives the keyword sound signal corresponding to the predetermined keyword.

9, the control unit 140 can determine a sound source position (sound source) corresponding to a sound signal received through a plurality of microphones, and the control unit 140 can determine, based on the sound source location, So that the body 101 can be rotated in the lateral direction.

Accordingly, the front face of the home robot 100 can be directed to the user who spoke the keyword.

Also, if 'genie' is set to a word set as a keyword, the home robot in the standby state can switch to the operating state according to the voice signal of the user who speaks the voice including 'genie'.

The use of the keyword as a trigger signal of speech recognition improves the accuracy of speech recognition and prevents waste of system resources.

On the other hand, when a keyword voice signal corresponding to a predetermined keyword is received, the user can notify the user that the image displayed on the display 182 included in the head 110 has been switched to the operating state.

That is, the specific image 800 corresponding to the standby state can be changed to the specific image 900 corresponding to the operation state.

For example, referring to FIG. 9, the specific image 900 corresponding to the operation state may be a sensible image 900 such as an opening eye.

After rotating the body 101 in the left and right direction, the image acquisition unit 120 may acquire a predetermined image by photographing a predetermined range of the front direction of the home robot 100, The user image information can be discriminated by processing the image acquired through the input unit 120. [

The control unit 140 can identify the face information in the image acquired through the image acquisition unit 120 using a face recognition technique. Here, the user face information may include at least one of the number of faces in the image acquired through the image acquisition unit 120, the position information of the face, and the area information.

The control unit 140 may control the rotation of the body 101 and / or the head 111 based on the user image information obtained through the image acquisition unit 120.

10, when the number of the faces 1010 included in the user image information is one, the controller 140 determines whether the face 1010 included in the user image information is included in the image obtained by the image obtaining unit 120 The body 101 can be rotated in the left-right direction so as to be positioned at the center of the camera of the camera.

That is, the control unit 140 controls the body 101 to rotate in the left-right direction so that one object 1010 is positioned in the center of the screen 1000 captured by the image acquisition unit 120 .

The control unit 140 controls the head 111 to rotate up and down so that one object 1010 is positioned at the center of the screen 1000 captured by the image acquisition unit 120 .

11, when the number of the faces 1110, 1120, and 1130 included in the user image information is plural, the controller 140 displays a plurality of faces 1110 and 1120 included in the user image information And 1130 may be controlled to rotate the body 101 of the home robot 100 such that the average value of the position of the robot 100 is located at the center of the camera of the image acquiring unit 120.

That is, the control unit 140 controls the body 101 so that the average positional values of the three objects 1110, 1120, and 1130 are positioned in the center of the screen 1100 captured by the image acquisition unit 120. Direction to rotate.

The controller 140 controls the head 111 so that the positional average value of the three targets 1110, 1120 and 1130 is positioned at the center of the screen 1100 taken by the image obtaining unit 120. [ Direction to rotate.

The average position values of the three objects 1110, 1120, and 1130 may be an average value of relative position coordinates based on a predetermined position in the screen 1100 captured by the image obtaining unit 120.

For example, the X and Y axis coordinates are set with the lower left corner coordinates in the screen 1100 taken by the image obtaining unit 120 as reference points, and the positions of the three objects 1110, 1120, and 1130 in the corresponding coordinate system After the coordinates are mapped, the average value can be calculated.

The home robot 100 according to an embodiment of the present invention determines the sound source generation position and significantly rotates the body 101 in the corresponding direction, At least one small rotation (small rotation) can be performed.

12, in order to control the body 102 in the left and right direction in the first small rotation (S1) based on at least one of the number, position, and area of the face in the image captured by the image acquisition unit 120 can do.

Further, it is possible to control the head 110 to perform the second small rotation (S2) in the vertical direction based on at least one of the number, the position, and the area of the face in the image captured by the image obtaining unit 120. [

If the control unit 140 determines that the small sound S 1 and / or the small sound S 2 in the up and down direction are unnecessary after the sound source generation position is determined and the body 101 is rotated in the left and right direction It is possible to omit one or more small rotations.

Thereafter, the user can provide various services through the home robot 100 by using speech input that speaks a voice including a desired command.

13, the user can input a voice commanding the home robot 100 to show the vegetable storage room (fresh room) of the refrigerator 13, and the home robot 100, upon receipt of the voice signal, The front surface of the head 110 on which the display 110 or the display 182 is disposed can be vertically nodded.

The voice signal received after the time when the voice input is received may be transmitted to the server 70 to be subjected to voice recognition processing or may be processed by the home robot 100 itself.

The home robot 100 receives the feedback information including the image of the inside of the vegetable storage room from the refrigerator 13 or receives the feedback information including the image of the inside of the vegetable storage room of the refrigerator 13 via the server 70 Feedback information can be received.

The home robot 100 may output the received feedback information as video and / or sound.

14, an image 1400 taken inside the vegetable storage room of the refrigerator 13 is displayed on the display 182, and a video image 4200 taken inside the vegetable storage room through the sound output unit 181 is displayed It is possible to output a voice informing that the user is doing.

The home robot 100 may receive the location information of the refrigerator 13 from the server 70 or the robot cleaner 12 through the communication unit 170. [

In this case, the control unit 140 of the home robot 100 may control the body 101 to rotate in the direction toward the refrigerator 13. [

Thereafter, when the feedback information including the image of the inside of the vegetable storage room of the refrigerator 13 is received, the controller 140 may rotate the body 101 back toward the user.

Accordingly, the user can intuitively know that the home robot 100 operates according to his / her voice input.

FIG. 15 illustrates a voice conversation for using various services and services provided by the home robot 100. FIG.

The user can provide various services through the home robot 100 by using speech input that speaks a voice including a desired command.

15, the home robot 100 includes a photograph 1510, a music & dance 1520, a play 1530, a schedule 1540, weather information 1550, a friend registration 1560, Interworking 1570, object recognition 1581, software training 1582, text notification 1583, and gallery 1584 services.

A user who wishes to use the picture taking service 1510 can speak a voice including a keyword 'hello' or the like and a voice requesting a service of taking a picture 1510 such as 'take a picture' .

After the home robot 100 receives the voice including the keyword 'Genie', it can determine the sound source generation position (sound source detection) and rotate in a direction corresponding thereto.

The control unit 140 may control the rotation of the body 101 and / or the head 111 based on the user image information obtained through the image acquisition unit 120.

In addition, the home robot 100 may photograph a photograph by operating a camera of the image obtaining unit 120 when a voice requesting a photographing 1510 service such as 'take a picture' is inputted. In this case, the home robot 100 can output a message to guide photographing, a message to guide the end of photographing, and the like.

Meanwhile, speech recognition of the user's voice input such as 'take a picture' may be performed through the home robot 100 itself or the server 70.

When the server 70 performs the voice recognition process, the home robot 100 transmits a voice signal corresponding to 'take a picture' to the server 70 and receives a recognition result corresponding thereto from the server 70 can do.

The speech recognition may be performed through the home robot 100 itself or the server 70 as described above.

A user who wishes to use the Music & Dance (1520) service can listen to a voice requesting a Music & Dance (1520) service such as " Play Music " I can tell.

After the home robot 100 receives the voice including the keyword 'Genie', it can determine the sound source generation position (sound source detection) and rotate in a direction corresponding thereto.

The control unit 140 may control the rotation of the body 101 and / or the head 111 based on the user image information obtained through the image acquisition unit 120.

When the home robot 100 receives a voice requesting music and dance service 1520 such as 'play exciting music', the home robot 100 transmits one or more music of 'dance' genre stored in the storage unit 130 And output it through the sound output unit 181. [

Further, it is also possible to output a sound for guiding the tune to be reproduced before the reproduction of the predetermined music.

In this case, the home robot 100 can perform motion such as dancing in combination with the motions that can be operated.

A user who wishes to use the service for playing the assimilation 1530 can speak a voice including a keyword 'genie' such as' hellojiney 'and then a voice for requesting a rendition 1530 service such as' have.

After the home robot 100 receives the voice including the keyword 'Genie', it can determine the sound source generation position (sound source detection) and rotate in a direction corresponding thereto.

The control unit 140 may control the rotation of the body 101 and / or the head 111 based on the user image information obtained through the image acquisition unit 120.

The home robot 100 may also reproduce one of the moving pictures stored in the storage unit 130 and output the sound to the sound output unit 1530. [ 181).

It is also possible to output a sound for guiding a moving picture to be reproduced before and during playback of a predetermined moving picture.

On the other hand, if the moving image content stored in the storage unit 130 includes an image, the home robot 100 can reproduce the image on the display 182.

According to the embodiment, when the user does not designate a specific moving image, the home robot 100 may output a message requesting the user to specify the moving image, such as 'Tell a title'. Then, A moving picture can be reproduced.

The user who wants to use the schedule information 1540 and the weather information 1550 service can speak the voice including the keyword 'Jini' such as 'hello jini', and then the schedule information such as 'how about today's schedule' (1540), and weather information (1550).

After the home robot 100 receives the voice including the keyword 'Genie', it can determine the sound source generation position (sound source detection) and rotate in a direction corresponding thereto.

The control unit 140 may control the rotation of the body 101 and / or the head 111 based on the user image information obtained through the image acquisition unit 120.

When the home robot 100 receives a voice requesting a certain guidance 1540 and weather information 1550 service, the home robot 100 can directly search through the Internet network or request and receive related information from the server 70 .

The home robot 100 may provide today's schedule and weather information to the user along with a predetermined graphic and / or voice and sound effects.

The user who wishes to use the friend registration service 1560 speaks a voice including the keyword 'genie' such as 'hellojiney' and then a voice requesting a friend registration service 1560 such as 'register my face' .

After the home robot 100 receives the voice including the keyword 'Genie', it can determine the sound source generation position (sound source detection) and rotate in a direction corresponding thereto.

The control unit 140 may control the rotation of the body 101 and / or the head 111 based on the user image information obtained through the image acquisition unit 120.

In addition, the controller 140 may register a user name, an alias name, and the like in the user image information acquired through the image acquisition unit 120 and store the matching information in the storage unit 130.

In this case, the home robot 100 can request the setting registration name from the user. When the user inputs a specific name, the home robot 100 registers the user corresponding to the specific name and outputs a voice guiding the registration completion.

A user who wishes to use the IoT interworking service 1570 may speak a voice including the keyword 'hello' or the like and then a voice requesting the IoT interworking 1570 service such as 'clean up' .

After the home robot 100 receives the voice including the keyword 'Genie', it can determine the sound source generation position (sound source detection) and rotate in a direction corresponding thereto.

The control unit 140 may control the rotation of the body 101 and / or the head 111 based on the user image information obtained through the image acquisition unit 120.

In addition, the control unit 140 may control the robot cleaner 12 to transmit a control signal corresponding to a command requesting the IoT interworking 1570 service such as 'clean up' to the corresponding IoT device to be controlled have.

The control unit 140 may control the communication unit 190 to transmit the control signal directly to the control target robot cleaner 12 or to the server 70. [

In addition to the robot cleaner 12, other Iot devices such as the home appliance 10, the smart plug, and the dust sensor, such as the tone generator 11, the refrigerator 13, the washing machine 14 and the cooking appliance 15, Can be controlled.

Meanwhile, the home robot 100 may perform a request of a user and output a voice as a result of the execution.

16A to 16E show an example of providing related information and related home appliance control in response to a user's schedule information request.

Referring to FIG. 16A, the user can request to output today's schedule information, and the home robot 100 can perform a nodding operation upon reception of a voice signal.

Referring to FIG. 16B, the home robot 100 can output today's schedule information to the image 1610 and the sound 1620.

Referring to FIG. 16C, based on today's schedule information, the home robot 100 may propose the operation of the apparel management apparatus to the sound 1630. FIG.

On the other hand, as shown in FIG. 16D, if the user agrees to the operation of the apparel management device, the home robot can perform the nodding operation in response to the agreement.

Further, the image displayed on the display 182 can be changed to a screen 1640 corresponding to the user's consent. For example, the screen 1640 corresponding to the user's consent may include a smiling eye image.

Also, as shown in FIG. 16E, it is possible to output the sound 1650 including the auditor corresponding to the user's consent.

On the other hand, the home robot 100 may directly send a control signal to the apparel management apparatus or transmit the control signal to the apparel administration apparatus via the server 70. [

17A to 17D show an example of providing related information and related home appliance control in response to a user's weather information request.

Referring to FIG. 17A, a user can request weather information output, and the home robot 100 can perform a nodding operation upon receipt of a voice signal.

Referring to FIG. 17B, the home robot 100 may output and provide weather information to the image 1710 and the sound 1720.

Referring to FIG. 17C, based on the weather information, the home robot 100 may propose the operation of the washing machine 14 as the sound 1730. FIG.

On the other hand, if the user agrees to the operation of the washing machine 14 as shown in FIG. 17D, the home robot can perform the nodding operation correspondingly.

Further, the image displayed on the display 182 can be changed to a screen 1740 corresponding to the user's consent. For example, the screen 1740 corresponding to the user's consent may include a smile-eye image. Further, it is possible to output sound including an auditor who responds to the user's consent.

Meanwhile, the home robot 100 may send a control signal to the washing machine 14 via the server 70 or transmit a control signal to the washing machine 14 directly.

That is, the home robot 100 not only controls the home appliance according to the user's voice input, but also can suggest home appliance control associated with the user's voice input.

In addition, by communicating with a facial expression and voice corresponding to the user's utterance, convenience and satisfaction of the user can be improved.

18A to 18C show an example in which the home robot 100 photographs a picture in response to a user's photograph photographing request.

Referring to FIG. 18A, the user can request the home robot 100 to take a picture by using a natural voice input such as "Take a picture ".

Referring to FIG. 18B, the home robot 100 may display the image 1810 to be sweep through the camera of the current image acquisition unit 120 on the display 182.

In addition, the home robot 100 can guide the photographing with the sound 1820, and can output the sound of the analog camera similar to that actually photographed by the camera such as 'click'.

Further, the home robot 100 may further display timing information 1811 such as a countdown to the shooting time on the image 1810. [

18C, the home robot 100 may display a photographed image 183 on a display 182 after taking a picture, and output a sound indicating that photographing is completed through the sound output unit 181 .

According to at least one of the embodiments of the present invention, since the user operates according to the voice input, the user does not need to operate other devices such as a portable terminal, a home appliance, and the like, thereby improving user convenience.

In addition, according to at least one embodiment of the present invention, since the home appliances constituting the network are controlled through the robot, the user does not have to operate other devices such as the portable terminal and the home appliance, Can be increased.

In addition, according to at least one embodiment of the present invention, a home robot capable of providing various services as well as a home appliance control and an operation method thereof can be provided.

The configuration and method of the embodiments described above can be applied to a home robot according to the present invention in a limited manner, but the embodiments can be modified so that all or some of the embodiments are selectively combined .

Meanwhile, the operation method of the home robot and the smart home system including the home robot according to the embodiment of the present invention can be implemented as a code that can be read by a processor in a recording medium readable by the processor. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

Home Appliances: 10
Server: 70
Home Robot: 100

Claims (20)

Receiving a voice signal through a plurality of microphones (MIC);
Determining a sound source generation position corresponding to a voice signal received through the plurality of microphones;
Rotating a body of the home robot corresponding to the determined sound source generating position;
Acquiring user image information through an image acquisition unit; And
Rotating the body of the home robot based on the user image information; The method comprising: The method according to claim 1,
And rotating the head of the home robot in a vertical direction based on the obtained user image information. The method according to claim 1,
Wherein the step of rotating the body of the home robot based on the user image information comprises the steps of: when the number of faces included in the user image information is one, And the body of the home robot is rotated so as to come to the home robot. The method according to claim 1,
Wherein the step of rotating the body of the home robot based on the user image information comprises the steps of: when the number of faces included in the user image information is plural, And the body of the home robot is rotated so as to come to the center of the camera of the image acquiring unit. The method according to claim 1,
Wherein the step of rotating the body of the home robot based on the user image information includes the step of rotating the body of the home robot based on at least one of the number of faces included in the obtained user image information, And rotating the body. The method according to claim 1,
Wherein the voice signal includes a predetermined keyword. ≪ RTI ID = 0.0 > 11. < / RTI > The method according to claim 1,
Recognizing a command included in the voice signal; And
And outputting information based on recognition results of the voice signal. The method according to claim 1,
Transmitting the voice signal to a server;
Receiving feedback information corresponding to a command included in the voice signal from the server; And
And outputting the feedback information. The method according to claim 1,
Receiving a voice signal including a predetermined command;
Recognizing a command included in a voice signal including the predetermined command; And
And outputting information based on the recognition result of the command. The method according to claim 1,
Receiving a voice signal including a predetermined command;
Transmitting a voice signal including the predetermined command to a server;
Receiving feedback information corresponding to the predetermined command from the server; And
And outputting the feedback information. A body forming an internal storage space;
A support disposed under the body and supporting the body;
An image acquisition unit capable of capturing an image;
A head positioned above the body and having the image acquisition unit disposed on the front side thereof;
A voice input unit including a plurality of microphones (MIC) through which audio signals are received; And
And a control unit for determining a sound source generating position corresponding to the sound signal received through the plurality of microphones and for rotating the body corresponding to the determined sound source generating position,
Wherein the control unit controls the body to rotate based on the user image information acquired through the image acquisition unit after the rotation of the body. 12. The method of claim 11,
Wherein the control unit controls the head to rotate in a vertical direction based on the obtained user image information. 12. The method of claim 11,
Wherein the control unit controls the body to rotate so that the face included in the user image information is located at the center of the camera of the image obtaining unit when the number of faces included in the user image information is one, . 12. The method of claim 11,
The control unit controls the body to rotate so that an average value of a plurality of face positions included in the user image information is located at the center of the camera of the image obtaining unit when the number of faces included in the user image information is plural Features a home robot. 12. The method of claim 11,
Wherein the control unit controls the body to rotate based on at least one of the number of faces, the position information of the face, and the area information included in the obtained user image information. 12. The method of claim 11,
Wherein the voice signal includes a predetermined keyword. 12. The method of claim 11,
A display for displaying a predetermined image; And
And an acoustic output unit for outputting a predetermined sound,
Wherein the control unit recognizes a command included in the voice signal and controls to output information based on recognition results of the voice signal. 12. The method of claim 11,
A display for displaying a predetermined image;
An audio output unit for outputting a predetermined sound; And
And a communication unit transmitting the voice signal to a server and receiving feedback information corresponding to a command included in the voice signal from the server,
And the control unit controls to output the feedback information. 12. The method of claim 11,
A display for displaying a predetermined image; And
And an acoustic output unit for outputting a predetermined sound,
Wherein the control unit recognizes a command included in a voice signal including the predetermined command when the voice signal including a predetermined command is received through the voice input unit and outputs the information based on the recognition result of the command Wherein the robot is a robot. 12. The method of claim 11,
A display for displaying a predetermined image;
An audio output unit for outputting a predetermined sound; And
And a communication unit for transmitting a voice signal including the predetermined command to the server when the voice signal including the predetermined command is received through the voice input unit and receiving feedback information corresponding to the command from the server, Including,
And the control unit controls to output the feedback information.

Images