KR101671148B1 - Robot cleaner and method for controlling the same - Google Patents

Abstract

The present invention relates to a robot cleaner.
A robot cleaner according to one aspect includes: a casing; A moving device for moving the casing; A camera provided in the casing for acquiring an image including a user's gesture; The method of claim 1, further comprising: extracting an arm image including an arm of a user from the image acquired by the camera, determining an arm angle and a palm direction represented by the gesture of the user in the arm image, And a control unit for controlling the mobile device.

Description

[0001] Robot cleaner and method for controlling same [0002]

The present invention relates to a robot cleaner and a control method thereof.

Generally, a robot cleaner refers to a device for automatically cleaning an area to be cleaned by suctioning foreign matter such as dust from a floor surface while traveling in a self-cleaning area without user's operation.

In recent years, techniques for recognizing a user's gesture or voice command and moving the robot cleaner based on the recognized command have been developed.

Korean Patent Registration No. 1385981 (registration date 2014.04.10), which is a prior art document, discloses a cleaning robot having a gesture recognition function and a control method thereof.

In the case of the prior art document, a technique of performing an optical flow analysis and skin color detection every real time and recognizing a user's gesture is disclosed.

However, according to the prior art document, since the optical flow analysis and the skin color detection are performed in real time, a continuous gesture detection algorithm must be performed internally, so that resources are very large and are not suitable for an embedded system such as a robot cleaner.

In addition, although the user does not take a gesture with the intention of inputting a command, there is a problem that a cleaning lotus senses a user's gesture.

SUMMARY OF THE INVENTION An object of the present invention is to provide a robot cleaner and a control method thereof that can accurately recognize a command intended by a user and execute it.

A robot cleaner according to one aspect includes: a casing; A moving device for moving the casing; A camera provided in the casing for acquiring an image including a user's gesture; And determining an arm angle and an arm direction indicated by the user's gesture in the arm image, determining an intention represented by the determined arm angle, A histogram determining unit for determining a histogram of each of the plurality of cell images; and a control unit for controlling the user based on the histogram of each of the plurality of cell images, Wherein the arm angle determination unit determines an outline of the arm based on the histogram determined through the histogram determination unit and determines that the outline information of the determined arm is a plurality of reference information If it is determined that the pattern matches the pattern of two or more pieces of reference information, The direction of the face of the person is determined in the image acquired by the camera, and the arm angle indicated by the gesture of the user is determined based on the determined face direction.

The robot cleaner may further include a voice recognition device provided in the casing for recognizing a voice command of the user, and the control unit determines that the voice command recognized by the voice recognition device is a gesture recognition command , The camera can be controlled so that the camera can acquire an image including a user's gesture.

In addition, the speech recognition apparatus may determine a point at which a voice command is generated, and the control unit may control the mobile apparatus such that the camera faces the voice command generation point.

The image processing unit may extract an arm image including an arm based on a user's face and a shoulder in an image acquired by the camera.

In addition, the robot cleaner may further include a memory in which reference information for use in arm angle determination is stored, and the controller compares the determined outline information of the arm with the information stored in the memory, The angle can be judged.

In addition, the memory may store a plurality of reference information separately for each arm angle and arm direction indicated by the user's gesture.

In addition, when the controller determines that the arm angle indicated by the user's gesture is the first angle, the control unit may control the mobile device so that the casing moves to one point in the direction in which the arm faces.

The controller may determine the current position and the position of the user, and may control the mobile device to move the casing to a position spaced a predetermined distance from the user's position toward the arm-facing direction.

In addition, when the controller determines that the arm angle indicated by the user's gesture is the second angle, the controller may control the mobile device such that the casing moves to an area located in the user's arm direction.

The robot cleaner may further include a memory in which map information is stored, and the map information includes a plurality of regions to be distinguished, So that the casing can be moved to an area where it is located.

According to another aspect of the present invention, there is provided a method of controlling a robot cleaner, comprising: inputting a voice command in a voice recognition device; Determining whether the input voice command is a gesture recognition command; When the input voice command is a gesture recognition command, the camera acquiring an image including a user's gesture; The control unit may determine an arm angle and an arm direction indicated by the user's gesture in the image acquired by the camera; And controlling the mobile device to determine a desired function of the determined arm angle and to control the mobile device to perform the determined function, wherein the step of determining the arm angle and the arm orientation comprises: Determining an outline of an arm on the basis of each histogram and determining that the outline information of the determined arm coincides with a pattern of two or more pieces of reference information among a plurality of pieces of reference information, And determining an arm angle indicated by the user's gesture based on the determined face direction.

According to the proposed invention, when the voice recognition device of the robot cleaner recognizes the gesture recognition command, the camera is activated, so that unnecessary operation of the camera in the cleaning process or the waiting process of the robot cleaner can be prevented have.

In addition, when the robot cleaner receives the gesture recognition command, the camera can accurately obtain the image including the user's gesture according to the direction change toward the point where the voice command is generated.

In addition, the arithmetic operation for feature extraction for determining the arm angle indicated by the user's gesture is simple and takes a short time.

In addition, even if a camera capable of shooting a monochrome image is used instead of a stereo camera, the cost of the robot cleaner can be reduced because the command indicated by the user's gesture can be determined.

Further, there is an advantage in that an arm angle feature can be extracted from a low-quality image by extracting features for arm angle determination from the arm image of the user acquired by the camera and comparing information stored in the memory.

In addition, since it is possible to extract the arm angle characteristic of the execution for each single image rather than the sequence image (information), the calculation speed is fast and the extraction accuracy of the arm angle characteristic can be improved.

1 is a front view of a robot cleaner according to an embodiment of the present invention;
2 is a block diagram of a robot cleaner according to an embodiment of the present invention.
3 is a flowchart illustrating a method of controlling a robot cleaner according to an embodiment of the present invention.
4 is a flowchart for explaining an arm angle determination algorithm indicated by a user's gesture;
FIGS. 5 and 6 are views showing an image of a user's gesture taken; FIG.
7 is a view showing the movement of the robot cleaner when the arm angle indicated by the user's gesture is 45 degrees;
8 is a view showing the movement of the robot cleaner when the arm angle indicated by the user's gesture is 90 degrees;

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the difference that the embodiments of the present invention are not conclusive.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; may be "connected," "coupled," or "connected. &Quot;

FIG. 1 is a front view of a robot cleaner according to an embodiment of the present invention, and FIG. 2 is a block diagram of a robot cleaner according to an embodiment of the present invention.

Referring to FIGS. 1 and 2, the robot cleaner 1 according to an embodiment of the present invention may include a casing 10 forming an external shape. The casing 10 is not limited, but may be formed in the form of a polygonal shape having a round or rounded edge when viewed from the upper side.

The robot cleaner 1 may further include a moving device 30 for moving the casing 10.

The mobile device 30 may include a plurality of wheels 32 in contact with the bottom surface and a plurality of driving motors 31 for driving the plurality of wheels 32, respectively.

The robot cleaner 1 may further include a suction device 33 for generating a suction force. Although not shown, a suction port is formed in the lower portion of the casing 1, and a dust container in which dust separated from the sucked air is stored may be provided at one side of the casing 10.

The robot cleaner 1 may further include a camera 11 capable of acquiring an image. The camera 11 may acquire a monochrome image. Of course, it is also possible for the camera 11 to acquire a high-quality color image.

The camera 11 may be located, for example, in a front portion of the casing 10. At this time, the camera 11 may be arranged to be inclined upward so that the camera 11 can acquire an image of a user's arm. That is, since the position of the arm of the user is higher than the robot cleaner 1, it is preferable that the camera 11 is inclined upward so that the camera 11 obtains the image of the user's arm.

Of course, it is also possible to obtain the image of the user's arm without the camera 11 being inclined according to the angle of view of the camera 11. [

The robot cleaner 1 includes a control unit 20 for recognizing an arm angle indicated by a user's gesture using an arm angle determination algorithm in an image photographed by the camera 11 and a memory 29 for storing information .

In this specification, the arm angle extracted by the controller 20 is the arm angle indicated by the user's gesture, not the actual angle of the user's arm in the arm image. That is, the arm angle indicated by the user's gesture may be different depending on the relative position of the user and the robot cleaner, and accordingly, the arm angle of the user in the image acquired by the camera may be different from the arm angle indicated by the user's gesture , The present invention judges the arm angle indicated by the user's gesture in the arm image without judging the arm angle of the user in the arm image.

Therefore, the "arm angle" determined by the control section 20 (or arm angle determination section) described below means "arm angle indicated by the user's gesture ".

The controller 20 can control the robot cleaner 1 based on the information determined using the arm angle determination algorithm. That is, the controller 20 can determine the function to be performed by the robot cleaner 1 based on the arm angle determined from the arm image, and control the robot cleaner 1 to operate with the determined function.

The control unit 20 includes an image processing unit 21 for dividing an arm image acquired from the camera 11 into a plurality of cell images, a histogram determination unit 22 for determining a histogram of each of the plurality of cell images, And an arm angle determination unit 23 for determining an arm angle indicated by the user's gesture based on the histogram of each cell image.

The controller 20 further includes a current position determiner 26 for determining the current position of the robot cleaner 1 and a controller for controlling the robot cleaner 1 based on the arm angle information determined by the arm angle determiner 23, 1) for executing the functions of the first embodiment.

The image processing unit 21, the histogram determination unit 22, the arm angle determination unit 23, the current position determination unit 26, and the function execution unit 27 are classified based on their functions. May be configured to operate as one or more software modules to perform functions.

The controller 20 can control the mobile device 30 based on arm angle information determined from an arm image, for example. In this specification, the controller 20 controls the moving device 30 so that the robot cleaner 1 can move to a specific position.

The memory 29 may store reference information for determining the arm angle. Also, the memory 29 may store map information and cleaning history information of an area where the robot cleaner 1 is located. For example, in the cleaning process of the robot cleaner 1, the cleaned area and the non-cleaned area may be separately stored in the memory 29.

The map information may include a plurality of area information classified by a wall or the like. For example, the map information may include a living room (not limited to a living room) separated by a wall or the like and one or more room information. However, the living room and the room are named for convenience of explanation, and map information in which a plurality of areas are divided can be stored in the actual memory.

In addition, the memory 29 may store current position information of the robot cleaner 1.

The robot cleaner 1 may further include a voice recognition device 40. May be provided in the casing (10).

The voice recognition device 40 may further include a voice input unit 41 for inputting voice and a voice analysis unit 42 for analyzing the input voice.

The voice input unit 41 may be a microphone, for example. The voice input unit 41 may include three or more microphones so that the voice recognition unit 40 can identify the point (or direction) at which the voice command is generated. For example, the point at which a voice command is generated is the user's current location.

That is, the distances from the sound source to the respective microphones may be different from each other. Therefore, the time taken for the voice command generated at a specific point to be transmitted to each microphone may be different from the size of the transmitted voice. It is possible to detect the point where the voice command is generated by using the size of the parallax or the sound at which the same voice command is detected in each microphone.

When the command recognized by the voice recognition device 40 is a gesture recognition command, the controller 20 can acquire an image of the user through the camera 11 and determine the arm angle of the user.

For example, if the voice recognition device 40 recognizes a user's gesture recognition command in the course of performing the cleaning of the robot cleaner 1, the controller 20 performs the cleaning of the robot cleaner 1 And activate the camera 11 after switching the direction toward the point where the voice command is generated.

The reason why the camera 11 is activated when the voice recognition device 40 of the robot cleaner 1 recognizes the gesture recognition command is that the camera 11 is activated during the cleaning process or the waiting process of the robot cleaner 1, 11 from operating unnecessarily.

The reason why the robot 11 is activated after the robot cleaner 1 receives the gesture recognition command and changes the direction toward the point where the voice command is generated is that the camera 11 accurately displays the image of the user It is for acquisition.

Of course, when the moving direction of the robot cleaner 1 is directed toward the point where the voice command is generated in the process of cleaning, the robot cleaner 1 does not need to change directions.

Hereinafter, a control method of the robot cleaner will be described.

FIG. 3 is a flowchart illustrating a method of controlling a robot cleaner according to an embodiment of the present invention, and FIG. 4 is a flowchart illustrating an arm angle determination algorithm.

5 (a) to 5 (c) are views showing a case where the user's right arm angle is 45 degrees, and FIGS. 5 (d) f) is a view showing a case where the right arm angle of the user is 90 degrees.

6 (a) to 6 (c) are views showing the case where the left arm angle of the user is 45 degrees, and FIGS. 6 (d) to 6 (f) are views showing the case where the left arm angle of the user is 90 degrees.

Referring to FIGS. 3 to 6, the robot cleaner 1 may perform cleaning by a cleaning start command or may wait for a cleaning start command (S1).

While the robot cleaner 1 performs cleaning or waits, the camera 11 remains in an inactive state.

Then, a voice command of the user can be inputted through the voice recognition device 40 (S2). When a voice is input through the voice input unit 41, the voice analysis unit 42 extracts a command intended by the input voice. The voice analysis unit 42 may transmit the extracted voice command to the controller 20.

The control unit 20 determines whether the voice command input by the voice recognition apparatus 40 is a gesture recognition command (S3).

If it is determined in step S3 that the voice command input by the voice recognition apparatus 40 is not a gesture recognition command, the process returns to step S1. On the other hand, when the voice command input by the voice recognition device 40 is a gesture recognition command, the controller 10 activates the camera 11. [

Then, the camera 11 acquires an image including the user's arm (S4).

As described above, the voice recognition device 40 confirms the voice command generation point, and when the robot cleaner 1 does not face the voice command generation point, specifically, The control unit 20 can control the mobile device 20 such that the camera 11 of the robot cleaner 1 faces the voice command originating point.

As another example, the voice recognition device 40 can control the mobile device 20 so that the robot cleaner 1 is rotated 360 degrees without stopping the voice command generation point. In addition, the camera 11 may acquire an image in a process of rotating the robot cleaner 1 360 degrees while the robot cleaner 1 is stopped. In this case, the camera 11 acquires a plurality of images to be photographed with a time interval, and the control unit 20 can extract an image including a user's arm from the acquired images.

Next, the control unit 20 determines the arm type and the arm angle indicated by the user's gesture in the acquired image (hereinafter referred to as "initial image") (S5).

Specifically, the image processing unit 22 divides the original image into a plurality of cell images, extracts an image including an arm (hereinafter, referred to as an "arm image") from the initial image, Is divided into cell images (S51). At this time, the arm image may include not only an arm but also a face and a shoulder.

For example, the image processing unit 22 may extract a face and a shoulder of a person from the first image acquired by the camera 11, and recognize a human's arm based on the extracted face and shoulder. The image processing unit 22 may extract an arm image from the initial image. At this time, the image processing unit 22 can determine the type of arm included in the arm image acquired based on the extracted face and shoulder, that is, the left arm or the right arm.

The histogram determination unit 22 divides each cell image into a plurality of pixels, and determines a histogram for each pixel (S52).

Next, the arm angle determination unit 23 determines the outline of the arm based on the histogram determined through the histogram determination unit 22 (S53).

Next, the arm angle determination unit 23 compares the determined outline information of the arm with the information stored in the memory 29 (S54). Finally, the arm angle determination unit 23 determines the arm angle indicated by the user's gesture (S55).

In the present embodiment, the outline of the arm is determined based on the histogram determined by the histogram determination unit in the arm image, and the arm angle is determined by comparing the determined outline of the arm and the information stored in the memory 29, There is an advantage that the calculation for feature extraction is simple and takes a little time.

Hereinafter, the difference in image according to the arm type and the arm angle indicated by the user's gesture will be described.

5 (a), the case where the user's gesture indicates that the right arm is positioned at an angle of 45 degrees (which may be referred to as a first angle) to the front of the user, and FIG. 5 If the gesture indicates that the right arm is positioned at an angle of 90 degrees (which may be referred to as the second angle) to the front of the user, the actual angle and direction of the arms in the image acquired by the camera 11 are different.

5 (b), the case where the user's gesture indicates that the right arm is positioned at an angle of 45 degrees in the diagonal direction of the front and right sides of the user, and the case where the user's gesture is the right arm The actual angle and direction of the arms are different in the image acquired by the camera 11, when the camera 11 is positioned at an angle of 90 degrees diagonally to the front and right sides of the user.

5 (c), the gesture of the user indicates that the right arm is positioned at a 45-degree angle to the right side of the user, and the case where the gesture of the user indicates that the right arm is positioned at 90 The actual angle and direction of the arms are different in the image acquired by the camera 11 when the camera 11 is positioned at a certain angle.

6 (a), when the user's gesture indicates that the left arm is positioned at a 45-degree angle to the front of the user, and when the user's gesture indicates that the left arm is positioned at the front The actual angle and direction of the arms in the image acquired by the camera 11 are different.

6 (b), the case where the user's gesture indicates that the left arm is positioned at an angle of 45 degrees in the diagonal direction of the front and left sides of the user, and the case where the user's gesture is the left arm The actual angle and direction of the arm in the image acquired by the camera 11 are different when the camera 11 is positioned at an angle of 90 degrees diagonally to the front and left sides of the user.

6 (c), when the user's gesture indicates that the left arm is positioned at a 45-degree angle to the left side of the user, and when the user's gesture indicates that the left arm is positioned at 90 The actual angle and direction of the arms are different in the image acquired by the camera 11 when the camera 11 is positioned at a certain angle.

In this way, the arm angle indicated by the user's gesture differs from the actual angle of the arm in the image acquired from the camera.

Therefore, in order to accurately determine the arm angle indicated by the gesture of the user from the arm image, the memory 29 is provided with a memory (not shown) for each of FIGS. 5A to 5F and 6A to 6F And reference information corresponding to the gesture is stored. At this time, the reference information is not limited, but may be arm contour information corresponding to each gesture.

For example, since the outline of the arm is different on the arm image for each of the gestures of FIGS. 5A to 5F and FIGS. 6A to 6F and has a certain pattern, only the outline per gesture is stored in the memory You can save it. The information stored in the memory can be reduced. Accordingly, in the arm angle determination process, the comparison process of the contours is simple and takes a little time.

The arm angle determination unit 23 determines whether the arm outline information determined from the image acquired by the camera 11 is the same as any one of the gestures shown in Figs.

That is, the arm angle determination unit 23 determines whether or not the determined arm outline information matches any one of a plurality of pieces of reference information stored separately by the gesture.

5 (b) and 5 (f), when the user's gesture indicates that the right arm is positioned at an angle of 45 degrees in the diagonal direction of the front side and the right side of the user, the camera 11 And an arm contour judged from the image acquired by the camera 11 when the gesture of the user indicates that the right arm is positioned at an angle of 90 degrees to the right side of the user, It is similar to the cuff.

6 (b) and 6 (f), when the user's gesture indicates that the left arm is positioned at an angle of 45 degrees in the diagonal direction of the front and left sides of the user, the camera 11 The arm contour and the arm direction determined from the acquired image and the arm contour and the arm direction judged from the image acquired by the camera 11 when the gesture of the user indicates that the left arm is positioned at 90 degrees to the left side of the user .

Therefore, when the arm angle determination unit 23 determines that the determined arm outline information matches two or more patterns among a plurality of pieces of reference information that are classified and stored according to the gesture, The direction of the face of the person is determined, and finally, the arm angle indicated by the user's gesture is determined.

For example, as shown in Figures 5 (b) and 6 (b), when the user's arm angle is located at an angle of 45 degrees, the user's face generally faces the direction indicated by the user's arm.

That is, the user's gaze is directed toward the 45-degree angle at which the arm is positioned, and the user rotates the neck at a predetermined angle so that the user's gaze is directed toward the 45-degree angle. In this case, the face of the user does not face the front but faces the face while looking diagonally.

On the other hand, when the user's gesture is directed to a right or left 90-degree angle as shown in FIGS. 5 (f) and 6 (f), the user's face is generally in a state in which the user does not face the front face or face down You will see right or left.

As a result, the direction in which the face of the user faces is different according to the arm angle of the user's gesture.

Therefore, in the present invention, the arm angle determination unit 23 determines the direction in which the face of the person faces from the image acquired by the camera 11, and finally determines the arm angle included in the user's gesture. Of course, the memory 29 stores information on the direction of the face of the user for each gesture of the user.

When the arm type and the arm angle indicated by the user's gesture are determined as described above, the controller 20 recognizes the intended function of the arm angle (S6). Then, the control unit 20 controls the robot cleaner 1 to execute the recognized function (S7).

For example, the function executing section 27 can recognize the function intended by the arm angle indicated by the user's gesture, and can control the mobile device 30 so that the robot cleaner 1 executes the recognized function .

7 is a view showing the movement of the robot cleaner when the arm angle indicated by the user's gesture is 45 degrees.

Referring to FIG. 7, the arm angle indicated by the user's gesture may be 45 degrees, for example, to allow the robot cleaner to move to the user's side in one direction of the user's arm.

At this time, the robot cleaner 1 can determine the current position in the current position determination unit 26 and can confirm the voice command generation point by the voice recognition device 40, Lt; / RTI >

For example, the robot cleaner 1 determines its current position and a voice command generation point on the map, determines a point at a certain distance from the user as a target point in a direction indicated by the user's gesture at a voice command generation point , And can move to the determined target point.

A point A in FIG. 7 is a target point when the gesture of the user indicates that the right arm is positioned at a 45-degree angle to the right side of the user as shown in FIG. 5 (c).

The point B in FIG. 7 is a target point in the case where the gesture of the user indicates that the right arm is located at an angle of 45 degrees in the diagonal direction of the front and right sides of the user, as shown in FIG.

The point C in FIG. 7 is a target point when the gesture of the user indicates that the right arm or the left arm is positioned at a 45-degree angle toward the front of the user, as shown in FIGS. 5A and 6A.

The point D in FIG. 7 is a target point when the gesture of the user indicates that the left arm is positioned at an angle of 45 degrees in the diagonal direction of the front and left sides of the user, as shown in FIG.

The point E in FIG. 7 is a target point when the gesture of the user indicates that the left arm is positioned at a 45-degree angle to the left side of the user as shown in (c) of FIG.

8 is a view showing the movement of the robot cleaner when the arm angle indicated by the user's gesture is 90 degrees.

Referring to FIG. 8, the memory 29 may store current position information of the robot cleaner 1 on map information and a map in which a plurality of regions are classified.

When the gesture recognition command is inputted in the speech recognition apparatus 40, the speech recognition apparatus 40 can confirm the point where the speech command is generated.

In the present invention, when the user's arm angle indicated by the user's gesture is 90 degrees, it may mean, for example, a command to move to a region located in the direction in which the user's arm is pointing.

For example, as shown in FIG. 8, the map information of the memory 29 stores a plurality of areas W, X, Y, and Z, and a current position of a voice command generation point and a robot cleaner may be determined .

When the user's gesture indicates that the left arm is positioned at an angle of 90 degrees to the left side of the user, the robot cleaner 1 is located in the X region, It is recognized that it is a Y region present on the right side.

Then, the robot cleaner 1 moves to the Y area. At this time, since the robot cleaner knows the coordinate information of the current position and the coordinate information of the Y area, the robot cleaner 1 can know the coordinates of the center of the Y area and moves to the center of the Y area.

According to the present invention, the arithmetic operation for extracting special features for determining the arm angle indicated by the user's gesture is simple and takes a short time.

In addition, even if a camera capable of shooting a monochrome image is used instead of a stereo camera, the cost of the robot cleaner can be reduced because the command indicated by the user's gesture can be determined.

Further, there is an advantage in that an arm angle feature can be extracted from a low-quality image by extracting features for arm angle determination from the arm image of the user acquired by the camera and comparing information stored in the memory.

In addition, since it is possible to extract the arm angle characteristic of the execution for each single image rather than the sequence image (information), the calculation speed is fast and the extraction accuracy of the arm angle characteristic can be improved.

Meanwhile, the present invention can provide a computer-readable recording medium having recorded thereon a computer program for causing a computer to execute a method for recognizing a gesture.

The recording medium may include a program command, a data file, a data structure, or the like, alone or in combination. The present invention provides a processing system or other electronic device

And may be embodied as a computer program product comprising a machine-accessible medium with built-in instructions that can be used for programming. Such a recording medium may be a transmission medium or a recordable medium for machine-readable information, including magnetic media, optical media, or other suitable media. Such a recording medium includes program instructions for causing an arm angle determination algorithm to be performed. The program instructions may be used to create a general purpose or special purpose processing system programmed with instructions to perform the operations described herein.

In the above embodiment, a configuration for recognizing a gesture is provided in the robot cleaner. Alternatively, a device for recognizing a gesture may be provided in an appliance other than the robot cleaner, and a device for recognizing the gesture It is also possible to use it separately and composed of separate modules.

For example, after the device for recognizing the gesture recognizes the gesture of the user, the robot cleaner may receive the instruction indicated by the gesture from the device for recognizing the gesture, and then perform the instruction indicated by the user's gesture.

1: robot cleaner 11: camera
20: control unit 21: image processing unit
22: Histogram judging unit 22: Arm angle judging unit
26: current position determination unit 27: function execution unit
29: memory 30: mobile device
40: Speech recognition device

Claims (11)

Casing;
A moving device for moving the casing;
A camera provided in the casing for acquiring an image including a user's gesture; And
An arm image including a user's arm is extracted from the image acquired by the camera, an arm angle and an arm direction indicated by the user's gesture are determined in the arm image, an intention represented by the determined arm angle is determined, And a controller
Wherein,
An image processor for dividing the arm image into a plurality of cell images;
A histogram determiner for determining a histogram of each of the plurality of cell images;
And an arm angle determination unit for determining an arm angle indicated by the user's gesture based on each histogram,
The arm angle determination unit may determine,
An outline of the arm is determined based on the histogram determined through the histogram determiner,
Determining a direction in which the face of the person faces in the image acquired by the camera when the determined outline information of the arm matches the pattern of two or more reference information among the plurality of reference information,
And determines the arm angle indicated by the user's gesture based on the determined face direction. The method according to claim 1,
Further comprising a voice recognition device provided in the casing and capable of recognizing a voice command of the user,
Wherein the control unit controls the camera to acquire an image including a user's gesture from the camera when it is determined that the voice command recognized by the voice recognition apparatus is a gesture recognition command. 3. The method of claim 2,
The speech recognition apparatus judges a point where a voice command is generated,
Wherein the control unit controls the mobile device such that the camera faces the voice command generation point. The method according to claim 1,
Wherein the image processing unit extracts an arm image including an arm based on a user's face and a shoulder in an image acquired by the camera. 5. The method of claim 4,
Further comprising a memory in which reference information for use in arm angle determination is stored,
Wherein,
And compares the outline information of the determined arm with the information stored in the memory to determine an arm angle indicated by a user's gesture. 6. The method of claim 5,
Wherein the memory stores a plurality of pieces of reference information separately for each angle and arm direction indicated by a user's gesture. The method according to claim 1,
When the controller determines that the arm angle indicated by the user's gesture is the first angle,
And controls the moving device so that the casing moves to one point in a direction in which the arm is directed. 8. The method of claim 7,
The control unit determines the current position and the position of the user,
And controls the moving device so that the casing moves to a position spaced a certain distance from the position of the user toward the direction of the arm. The method according to claim 1,
When the controller determines that the arm angle indicated by the user's gesture is the second angle,
And controls the moving device such that the casing moves to an area positioned in the user's arm direction. 10. The method of claim 9,
Further comprising a memory in which map information is stored,
Wherein the map information includes a plurality of regions to be divided,
Wherein the control unit controls the moving device such that the casing moves to an area located in an arm direction indicated by a user's gesture among a plurality of areas in a current area. Inputting a voice command in a voice recognition device;
Determining whether the input voice command is a gesture recognition command;
When the input voice command is a gesture recognition command, the camera acquiring an image including a user's gesture;
The control unit may determine an arm angle and an arm direction indicated by the user's gesture in the image acquired by the camera; And
Wherein the control unit includes a step of determining the intended function of the determined arm angle and controlling the mobile device for performing the determined function,
The step of determining the arm angle and the arm direction includes:
Determining an outline of an arm based on a histogram of each of the plurality of cell images,
Determining a direction in which a face of a person faces in an image acquired by the camera when it is determined that the outline information of the determined arm coincides with a pattern of two or more reference information among a plurality of reference information,
And determining an arm angle indicated by a user's gesture based on the determined face direction.

Images