KR20190130196A - Movable robot apparatus - Google Patents

Abstract

The present invention relates to a moving robot device capable of autonomous driving. Provided is the moving robot device comprising: a main body; a driving module for moving the main body; an audio sensing module including a plurality of audio sensors corresponding to different audio division regions around the main body, and generating a plurality of audio signals corresponding to the plurality of audio sensors; and a control module for detecting a voice command signal corresponding to at least one of a plurality of preset commands from the plurality of audio signals on the basis of a speaker-dependent model learned by a voice of a specific user, and supplying a control signal corresponding to the voice command signal to the driving module. Therefore, an objective of the present invention is to provide the moving robot device which can improve an intimacy of a specific user by sensing the specific user.

Description

Mobile Robot Device {MOVABLE ROBOT APPARATUS}

The present invention relates to a mobile robot device capable of autonomous driving.

Robots have been developed for industrial use and have been a part of factory automation. Recently, the application of robots has been further expanded, medical robots, aerospace robots, and the like have been developed, and home robots that can be used in general homes have also been made.

Representative examples of home robots include robot cleaners and intelligent robots.

The robot cleaner is a home appliance that cleans by inhaling dust or foreign matter while driving a certain area by itself. Such a robot cleaner has a rechargeable battery and is equipped with an obstacle sensor that can avoid obstacles while driving and runs on its own to perform a cleaning function.

Intelligent robots are equipped with artificial intelligence for detecting and responding to user's emotions and commands, and provide services such as driving and conversation functions in response to user's emotions and commands.

Such robot cleaners and intelligent robots are generally mobile robot devices equipped with autonomous driving functions.

Regarding the mobile robot device, Korean Patent Laid-Open No. 10-2017-0103556 (published on September 13, 2017) discloses a mobile robot capable of improving avoidance driving performance by detecting terrain or obstacles located in the vicinity.

In addition, Korean Patent Publication No. 10-2017-0107875 (published on September 26, 2017) discloses a robot control system that can draw out more emotional reactions of a user and provide a service more accurately in response to a user's request. do.

According to these prior art documents, there is a problem that the service of the mobile robot device can be provided to everyone who wants to use the mobile robot device. That is, the service target of the mobile robot device is not designated as a specific user. Therefore, there is a problem in that the intimacy of a specific user with respect to the mobile robot device is limited.

In addition, when the service target of the mobile robot device is not designated as a specific user, it is difficult to provide a function of moving based on a specific user, such as following a specific user, moving away from a specific user, and moving around a specific user. There is this.

And, if it is not a biped walking robot, the mobile robot apparatus is arrange | positioned adjacent to the ground. Therefore, the general mobile robot device has a problem that it is difficult to detect a specific user through a face recognition method.

Even if the mobile robot device uses a face recognition method, the user needs to bend to face the camera of the mobile robot device, thereby increasing the inconvenience of the user. In addition, there is a problem that the computational load of the mobile robot device may increase due to a large amount of computation due to face recognition.

An object of the present invention is to provide a mobile robot device that can improve the intimacy of a specific user by sensing a specific user.

An object of the present invention is to provide a mobile robot device that can provide a function to move based on a specific user by detecting a specific user.

An object of the present invention is to provide a mobile robot device capable of detecting a specific user without using a face recognition method.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention, which are not mentioned above, can be understood by the following description, and more clearly by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

The mobile robot device according to an embodiment of the present invention is a control module for detecting a voice command signal corresponding to at least one of a plurality of preset commands from the plurality of audio signals based on a speaker dependent model learned by a specific user's voice. It includes. As a result, since the control signal is generated only by a command input by the voice of the specific user, the intimacy of the specific user with respect to the mobile robot device may be improved.

According to an exemplary embodiment of the present invention, a mobile robot apparatus includes a voice position detector and a front region corresponding to a camera sensing module, which detects an audio splitting region having a loudest volume as a voice position based on a speaker dependent model. And a user location detector for detecting a user location coordinate corresponding to the specific user. As a result, the presence and location of the specific user can be detected, and thus a function of moving based on the specific user can be provided.

A mobile robot device according to an embodiment of the present invention provides a target object model corresponding to a horizontal field of view of a camera sensor module and detects a target object region corresponding to the target object model among the object candidate regions detected in the 3D model. It includes a target object detection unit. Thus, a specific user can be detected without using a face recognition method.

The mobile robot device according to an embodiment of the present invention is a control module for detecting a voice command signal corresponding to at least one of a plurality of preset commands from the plurality of audio signals based on a speaker dependent model learned by a specific user's voice. It includes. As such, the voice of the specific user may be detected by detecting the voice command signal based on the speaker dependent model trained by the specific user's voice. That is, since the control signal is generated only by a command input by the voice of a specific user, the intimacy of the specific user with respect to the mobile robot device may be improved.

When the control module of the mobile robot apparatus according to an embodiment of the present invention detects the voice command signal, the control module detects any one audio signal having the maximum volume level of the voice command signal among the plurality of audio signals, and detects the audio command signal. And a voice position detector for detecting the audio splitting region corresponding to the signal as a voice position. The audio splitting region having the loudest loudness of the voice command signal detected based on the speaker dependent model learned by the voice of the specific user by the voice position detector may be detected as the voice position. The detected audio split region may be estimated to be an area adjacent to a specific user, and thus the position of the specific user may be easily detected.

In addition, the control module of the mobile robot apparatus according to an embodiment of the present invention detects a user position coordinate corresponding to a specific user among the front regions corresponding to the camera sensing module based on the 3D distance image signal by the camera sensing module. The apparatus further includes a user position detector. That is, the control module may detect the presence and the location of a specific user from the 3D distance image signal based on the image recognition technology.

As such, by detecting the presence and location of a specific user, a function of moving based on a specific user may be provided.

In addition, the user position detection unit of the control module of the mobile robot device according to an embodiment of the present invention provides a target object model corresponding to the horizontal field of view of the camera sensor module, the target object model of the object candidate area detected in the three-dimensional model And a target object detector for detecting a target object region corresponding to the target object region. Here, the target object detector prepares a target object model by modeling the shape of the coefficient target corresponding to the horizontal field of view of the camera sensing module, and detects the target object region corresponding to the target object model among the object candidate regions of the 3D model. . As the target object region is detected using the target object model, the presence or absence of an object can be detected even in the form of a human lower body. In this case, since the audio division region in which the voice command signal is the loudest volume is detected, even if the presence or absence of an object is detected by using the lower body shape of a person, the position of a specific user may be estimated as a relatively low error range. . As a result, the target object model does not need to be limited to a specific user's face, so that the specific user can be detected without using a face recognition method. Therefore, an increase in the amount of calculation according to the face recognition method may be prevented, and a posture change of a separate camera or user for face recognition may be excluded.

In addition to the effects described above, the specific effects of the present invention will be described together with the following description of specifics for carrying out the invention.

1 is a view showing an example of a side perspective view of a mobile robot device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of the audio sensing module of FIG. 1.
3 is a diagram illustrating an example of the control module of FIG. 1.
4 is a diagram illustrating an example of the voice command detector of FIG. 3.
5 and 6 illustrate examples of a process of generating a control signal by the control module of FIG. 3.
7 and 8 are views illustrating an example in which the traveling module rotates the main body based on the control signal in the standby state of FIGS. 5 and 6.
9 is a diagram illustrating an example of the user position detector of FIG. 3.
10 is a diagram illustrating an installation example of a camera sensing module.
FIG. 11 is a diagram illustrating an example of distance image information and a target object area in the example of FIG. 10.
FIG. 12 is a diagram illustrating an example of a process of detecting a bottom plane area by the bottom detector of FIG. 9.
FIG. 13 is a diagram illustrating an example of nine points and a reference point of FIG. 12.
FIG. 14 is a diagram illustrating another example of a process of detecting the bottom plane area by the bottom detector of FIG. 9.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

Hereinafter, with reference to the accompanying drawings will be described a mobile robot device according to an embodiment of the present invention.

First, a mobile robot apparatus according to an exemplary embodiment of the present invention will be described with reference to FIGS. 1, 2, and 3.

1 is a view showing an example of a side perspective view of a mobile robot device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of an audio sensing module and a camera sensing module of FIG. 1.

As shown in FIG. 1, the mobile robot apparatus 10 according to an exemplary embodiment of the present invention may include a main body 11 corresponding to an external appearance, a traveling module 12 moving the main body 11, and a main body 11 around the main body 11. An audio sensing module 13 for sensing the sound of the sound, and a control module 14 for supplying a control signal to the driving module 12 based on the audio signal by the audio sensing module (13).

As shown in FIG. 2, the audio sensing modules 131, 132, 133, 134, 135, 136, 137, and 138 (13) of FIG. 1 are provided with different audio divisions ADA1, ADA2, And a plurality of audio sensors 131, 132, 133, 134, 135, 136, 137, and 138 corresponding to ADA3, ADA3, ADA4, ADA5, ADA6, ADA7, and ADA8 (Audio Division Area). Accordingly, the audio sensing module 13 generates a plurality of audio signals corresponding to the plurality of audio sensors 131, 132, 133, 134, 135, 136, 137, and 138.

In addition, a front area FA corresponding to the camera sensing module 15 of FIG. 1 overlaps at least one audio division area ADA1, ADA2, and ADA8 in FIG. 2.

When a plurality of audio signals are input from the audio sensing module 13, the control module 14 corresponds to at least one of a plurality of commands preset from the plurality of audio signals based on a speaker dependent model learned by a specific user's voice. Detect the voice command signal. The control module 14 supplies a control signal corresponding to the voice command signal to the traveling module 12.

Next, the control module of FIG. 1 will be described with reference to FIGS. 3, 4, 5, 6, 7, and 8.

3 is a diagram illustrating an example of the control module of FIG. 1. 4 is a diagram illustrating an example of the voice command detector of FIG. 3. 5 and 6 illustrate examples of a process of generating a control signal by the control module of FIG. 3.

As shown in FIG. 3, the control module 14 of the mobile robot apparatus 10 according to an exemplary embodiment may include a voice command detection unit configured to detect a voice command signal by a specific user's voice from a plurality of audio signals. 141 and a control signal generator 142 for generating a control signal for driving control of the driving module 12. In addition, the control module 14 may further include a voice position detector 143 for detecting any one audio splitting region in which the voice command signal has a maximum volume level. In addition, the control module 14 detects a user position coordinate corresponding to a specific user in the front area (FA of FIG. 2) based on the 3D distance image signal of the camera sensing module 15. It may further include.

The voice command detection unit 141 detects the voice command signal from the plurality of audio signals based on the speaker dependent model learned by the voice of a specific user.

For example, as illustrated in FIG. 4, the voice command detector 141 may include a morpheme analyzer 1411, a recognition error corrector 1412, a part-of-speech filter 1413, and a stopword filter 1414. Can be.

The morpheme analyzer 1411 separates the wavelengths of the plurality of audio signals into band-specific frequencies, converts the wavelength signals separated by band-specific frequencies into the Z plane, and based on a speaker dependent model trained by a specific user's voice. The morpheme is extracted by detecting the effective area from the signal converted to the plane. In addition, the morpheme analyzer 1411 may separate wavelengths into band-specific frequencies by using a fast Fourier transform (FFT) function.

The recognition error correction unit 1412 corrects an error of the extracted morpheme by matching the morpheme extracted through the consonant-based maximum matching method with the morpheme of the pre-stored word.

The part-of-speech filter unit 1413 removes morphemes corresponding to meaningless part-of-speech among the corrected morphemes. The meaningless part of speech here may be a survey, a mother, or the like.

The stopword filter 1414 removes morphemes that are not information related to control among morphemes of meaningful parts of speech. By way of example, when the morpheme of "turn left" is detected, "left" and "turn back" may be detected by the part-of-speech filter part 1413 and the stopword filter part 1414.

As shown in FIG. 5, in the control module 14 of FIG. 3, when a plurality of audio signals are input (S11), the voice command detection unit 141 detects a voice command signal from the plurality of audio signals. (S12).

When the voice command signal is detected (S12), the voice position detection unit (143 in FIG. 3) detects any one audio signal having the maximum volume level of the detected voice command signal among the plurality of audio signals, and detects the detected audio signal. The corresponding audio division region is detected. In this case, the detected audio splitting region is a region in which a voice command signal corresponding to a specific user's voice is detected as the largest volume size, and thus can be estimated as the region closest to the specific user, that is, the voice position. (S13)

When the voice command signal and the voice position are detected as described above (S12 and S13), the control signal generator 142 generates the control signal in the standby state. (S14)

The control signal in the standby state is for indicating that the mobile robot device 10 detects a specific user. For example, the driving module 12 may drive to rotate the main body 11 until the front region (FA in FIG. 2) is disposed in the audio division region detected as the voice position based on the standby control signal. Can be.

In addition, when a voice position is detected (S13), the user position detector 144 detects a user position coordinate corresponding to a specific user in the front area FA based on the 3D distance image signal by the camera sensing module 15. do. Here, the user position detection unit 144 for detecting the user position coordinates will be described in more detail below.

In this case, the control signal generator 142 generates a control signal for driving the driving module 12 based on a command corresponding to the voice command signal. (S16)

In this case, when the detection of the user position coordinates fails, that is, when no person is detected in the front area FA, the control signal generator 142 stops generating the control signal corresponding to the voice command signal. That is, when no person is detected in the front area FA, it is assumed that the command is not a command of a specific user located near the mobile robot apparatus 10, and the generation of the control signal is stopped. As a result, the driving of moving based on a specific user can be stably performed.

Meanwhile, unlike FIG. 5, the control signal generator 142 may generate a control signal in a standby state when a predetermined starting word is detected.

That is, as shown in FIG. 6, when a specific user corresponds to a start word for calling a mobile robot device among a plurality of commands stored in advance (S18), the control signal generator 142 is in a standby state. A control signal can be generated. (S14 ')

Here, the starting word may be a nickname or a title assigned to a mobile robot device by a specific user.

If the voice command signal does not correspond to the starting word (S18), the user position detector 144 corresponds to a specific user in the front area FA based on the 3D distance image signal by the camera sensing module 15. Detect user position coordinates. (S15 ')

In this case, the control signal generator 142 generates a control signal for driving the driving module 12 based on a command corresponding to the voice command signal. (S16)

In this case, when the detection of the user position coordinates fails, that is, when no person is detected in the front area FA, the control signal generator 142 stops generating the control signal corresponding to the voice command signal. (S17)

7 and 8 are views illustrating an example in which the traveling module rotates the main body based on the control signal in the standby state of FIGS. 5 and 6.

As illustrated in FIG. 7, a voice may be input in a state where a specific user 20 is located in a region other than the front area FA of the surroundings of the mobile robot apparatus 10. At this time, when the control module 14 detects the voice command signal and the voice position (ADA4 of FIG. 7), the control module 14 supplies the control signal in the standby state to the driving module 12.

Thus, as shown in FIG. 8, the traveling module 12 rotates the main body 11 in one direction (clockwise in FIG. 8) based on the control signal in the standby state. At this time, the rotation operation is performed until the front area FA reaches the audio division area ADA4 detected as the voice position.

Next, the user position detector 144 of FIG. 3 will be described with reference to FIGS. 9, 10, 11, 12, 13, and 14.

9 is a diagram illustrating an example of the user position detector of FIG. 3. 10 is a diagram illustrating an installation example of a camera sensing module. FIG. 11 is a diagram illustrating an example of distance image information and a target object area in the example of FIG. 10. FIG. 12 is a diagram illustrating an example of a process of detecting a bottom plane area by the bottom detector of FIG. 9. FIG. 13 is a diagram illustrating an example of nine points and a reference point of FIG. 12.

FIG. 14 is a diagram illustrating another example of a process of detecting the bottom plane area by the bottom detector of FIG. 9.

As illustrated in FIG. 9, the user position detector 144 may include a model calculator 1442, a floor detector 1442, an inclination estimator 1443, a model reconstructor 1444, a height map calculator 1445, and the like. An object candidate detector 1446, a target object detector 1447, and a coefficient processor 1548 are included.

First, the camera sensing module 15 includes a range image sensor (not shown) and acquires a 3D distance image signal corresponding to the front area FA. Here, the distance image sensor is a means for sensing the two-dimensional image and the distance of the sensor to each point (point) of the two-dimensional image. For example, the distance image sensor may be at least one of a combination of a distance sensor and a camera sensor, an RGB-D sensor, and a 3D depth camera.

As shown in FIG. 10, the camera sensing module 15 of the mobile robot apparatus 10 according to the exemplary embodiment of the present invention may be installed to have a main axis adjacent to the horizontal plane and oblique to the horizontal plane side.

That is, the camera sensing module 15 is spaced apart from the horizontal plane by a predetermined height (SH; Sensor Height), and has a main axis facing the horizontal plane at a predetermined inclination θ based on a plane horizontal to the horizontal plane. Here, the height SH of the camera sensing module 15 with respect to the horizon may be set smaller than the average value of the waist height of the adult.

Accordingly, the vertical field of view (SVF) of the camera sensing module 15 may correspond to the height SH of the camera sensing module 15 from the horizontal plane. For example, when the height SH of the camera sensing module 15 is 35 cm, the horizontal viewing range SVF is a point spaced about 150 cm from a point spaced about 25 cm from the camera sensing module 15. It may be an area up to. However, this is only an example, and the tilt θ, the height SH and the horizontal viewing range SVF of the camera sensing module 15 may be changed according to the discretion of the installer and the conditions of the front area FA.

Although not shown separately, the camera sensing module 15 may generate a 3D distance image signal based on sensing values obtained from two or more distance image sensors.

In addition, the camera sensing module 15 may perform preprocessing for the distance image information. For example, the preprocessing of the distance image information may include noise reduction, rectification, calibration, color enhancement, color space conversion (CSC), and interpolation (interpolation). ), Camera gain control, and the like.

The model calculator 1442 calculates a 3D model corresponding to the distance image information. Here, the 3D model may be in the form of a set of a plurality of points belonging to the 3D coordinate system. That is, each point of the 3D model may be defined as an X coordinate, a Y coordinate, and a Z coordinate in the 3D coordinate system.

The floor detector 1442 detects a floor plane area of the 3D model.

Here, the bottom plane area may correspond to any one of an area corresponding to the ground plane, a horizontal plane area, and a background area. The floor detector 1442 further calculates an average normal vector of the floor plane region.

For example, as illustrated in blue in FIG. 11, the floor plan region of the 3D model may be detected by the floor detector 1442.

The bottom detection unit 1442 will be described in more detail below.

The slope estimator 1443 estimates the slope of the camera sensing module 15 based on the average normal vector of the floor plane region.

For example, the slope estimator 1443 may derive a plane equation based on the average normal vector of the floor plane area and coordinates of an arbitrary point included in the floor plane area, and estimate the slope by using the plane equation. Here, the arbitrary point included in the floor plan region may be a point selected to derive the floor plan region or an average point of the floor plan region.

The model reconstructor 1444 reconstructs the 3D model based on the estimated slope. As a result, even when the camera sensing module 15 is not installed perpendicular to the floor plane area, it is possible to reduce the overlap when projecting the 3D model to generate a height map.

That is, the model restoring unit 1444 may reconstruct the portion covered by the overlap by correcting the three-dimensional model by the inclination (θ of FIG. 10) of the camera sensing module 15.

The height map calculator 1445 calculates the height map by projecting the restored 3D model in a direction perpendicular to the horizontal plane.

The object candidate detection unit 1446 detects the object candidate area in the reconstructed three-dimensional model based on the calculated height map.

For example, the object candidate detection unit 1446 may detect the object candidate area by removing the floor plane area from the restored 3D model based on the calculated height map.

The target object detector 1447 prepares a target object model according to the horizontal viewing range of the camera sensing module 15, and detects a target object region corresponding to the target object model among the object candidate regions.

For example, as illustrated in FIG. 10, the camera sensing module 15 may be installed to have an oblique main axis on the horizontal plane to face the horizontal plane at a position higher by a predetermined height than the horizontal plane.

In this case, as shown in FIG. 11, the target object model may correspond to the lower body shape of the person standing or walking. In this case, the target object model may be formed of a pair of quadrangles arranged adjacent to each other like the side of the human lower body.

However, this is only an example, and the target object detection unit 1447 may prepare a target object model by modeling a shape of a human lower body located in the horizontal field of view of the camera sensing module 15, that is, the front area FA. . Alternatively, even when the target object is a child or an animal that is not an adult person, the target object model may be prepared by modeling the front area FA.

The user coordinate detector 1424 detects the user position coordinates in the front area FA by tracking the target object area.

For example, when a plurality of target object areas are detected, the user coordinate detector 148 tracks moving objects among the plurality of target object areas, selects the area corresponding to a specific user, and based on the selected area, the user position coordinates. Can be detected. Here, the user location coordinates may be selected from any of the coordinates closest to the ground, the coordinates disposed farthest from the ground, and the midpoint coordinates. However, this is only an example, and the user location coordinates may be selected by any one coordinate included in the selected area.

On the other hand, when a plurality of target object regions are detected, the user coordinate detection unit 1482 tracks an object closest to the main body 11 among the plurality of target object regions, selects the region corresponding to a specific user, and selects the selected region. The user position coordinates can be detected based on this.

Alternatively, when a plurality of target object regions are detected, the user coordinate detection unit 1482 first detects a dynamic object among the plurality of target object regions, and then identifies an object closest to the main body 11 among the dynamic objects corresponding to a specific user. You can also select an area.

As shown in FIG. 12, the floor detection unit 1442 selects a partial region of the 3D model. (S101) Here, the partial area is selected as an area which is likely to be one of a horizontal plane, a horizontal plane, and a background.

The floor detector 1442 selects any nine points disposed at the edges of some areas. (S102) The floor detector 1442 selects an arbitrary reference point disposed in the partial region. (S103)

For example, as shown in FIG. 13, since the normal horizontal plane is disposed below the image, the partial area PA of the 3D model 3D may be selected as the lower 1/3 area of the 3D model 3D. Can be. Here, the nine points 9P may be disposed adjacent to the edge of the partial area PA. In addition, the figure connecting the nine points 9P as one line may be a quadrangle like the partial area PA. The reference point BP may be a center of a figure in which nine points 9P are connected by one line.

Next, as shown in FIG. 12, the floor detector 1442 calculates a normal vector corresponding to nine points based on the reference point. (S104)

For example, in the example of FIG. 13, the normal vector NV may be calculated based on the relationship between the reference point BP and the nine points 9P.

Subsequently, if the angle of the calculated normal vector is less than a predetermined error range (S105), a partial region may be regarded as a bottom plane region. (S106)

That is, when the number of times the angle of the normal vector corresponding to the partial region is less than the error range is greater than or equal to a threshold number of times, in the successive different frames of the distance image information, the floor detection unit 1442 may cover the partial region with the floor plane. Detect by area. (S106)

Specifically, the floor detection unit 13, if the angle of the normal vector corresponding to the partial region is less than the error range (S105), and if the number of derivations of the normal vector whose angle is less than the error range is less than the threshold number (S111), another frame (S112). ) Calculates the normal vectors of some areas. (S104)

Further, the floor detector 1442 may determine the area of the floor plane when the angle of the normal vector corresponding to the partial region is less than the error range (S105), and the number of derivations of the normal vector having the angle less than the error range is greater than or equal to the critical number (S111). Detect by area. That is, when a plurality of normal vectors corresponding to a partial region are derived from a plurality of frames among the distance image information, and a normal number having an angle less than an error range among the plurality of normal vectors is derived above a critical number (S105, S111). In addition, some areas are detected as a floor plane area, and an average value of the derived normal vectors is calculated as an average normal vector of the floor plane area. (S106)

In exemplary embodiments, the threshold number may be five. However, this is only an example, and the threshold number may be specified in a range that can secure the reliability of the floor plan area according to the user's discretion.

The floor detector 1442 deletes the frame when the angle of the normal vector corresponding to the partial region of the plurality of frames included in the distance image information is derived beyond the error range (S105). (S113) Next, the normal vector of the partial region is further calculated in another frame S112. (S104)

At this time, if the number of erased frames is greater than or equal to the threshold number (S114), the floor detector 1442 reduces the size of the partial region to 1/2. (S115) Then, a normal vector is calculated for the reduced partial area. (S104)

As an example, when more than a threshold number of frames in which the angle of the normal vector corresponding to the partial area PA is greater than or equal to the error range occurs, the partial area PA ′ having the size reduced in half may be provided.

On the other hand, the floor detector 1442 may detect the floor plane area in a manner different from that of FIG. 12.

FIG. 14 is a diagram illustrating another example of a process of detecting the bottom plane area by the bottom detector of FIG. 9.

As shown in FIG. 14, the floor detector 1442 divides the three-dimensional model into a plurality of unit regions. (S201)

The floor detector 1442 selects nine points arranged adjacent to the edge of each unit area (S202), and selects an arbitrary reference point disposed in each unit area. Here, the reference point may be a midpoint of a figure in which nine points are connected by one line.

The floor detector 1442 calculates a normal vector corresponding to nine points based on the reference point. In this case, as illustrated in FIG. 13, a normal vector NV corresponding to each unit region may be calculated based on the relationship between the reference point BP and the nine points 9P.

If the angle of the calculated normal vector is less than the predetermined error range (S205), the floor detector 1442 designates each unit area as the floor range. (S206)

On the other hand, if the angle of the calculated normal vector is greater than or equal to a predetermined error range (S205), and there are remaining unit areas for which the normal vector is not calculated (S207), the floor detector 13 changes to any one of the remaining unit areas (S209). ), The process of calculating the normal vector for the changed unit region is resumed. (S202, S203, S204)

Subsequently, when the floor detector 1442 calculates a normal vector for the entire unit area (S207), the floor detection unit 1442 detects the combination of the unit areas designated as the floor range as the floor plane area, and then floors the average value of the normal vectors of the unit areas designated as the floor range. Calculated as the average normal vector of the planar region. (S208)

By doing so, there is an advantage that the amount of calculation for detecting the bottom plane area is reduced compared to the process of FIG. That is, according to the process of FIG. 14, since the normal vector corresponding to the unit area having a relatively small width is calculated and the normal vector is not calculated for each frame, the computation amount may be reduced.

As described above, the mobile robot apparatus according to an embodiment of the present invention detects a voice command signal using a speaker dependent model trained by a specific user's voice from a plurality of audio signals by a plurality of audio sensors. Accordingly, since a specific user may be detected and reacted to only a specific user, the intimacy of the specific user with respect to the mobile robot device may be improved. In addition, a function of driving based on a specific user may be provided.

In addition, the mobile robot device according to an embodiment of the present invention is a three-dimensional distance image signal obtained in the state in which the front area (FA) of the camera sensing module 15 in the voice position based on the control signal in the standby state The presence or absence of a person with respect to the front area FA is detected. As a result, even if the face recognition function requiring a complicated operation is excluded, a specific user can be detected.

The present invention as described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by.

10: mobile robot device 11: main body
12: drive module 13: audio sensing module
14: control module 15: camera sensing module
131, 132, 133, 134, 135, 136, 137, 138: audio sensor
ADA1, ADA2, ADA3, ADA4, ADA5, ADA6, ADA7, ADA8: Audio Division
FA: Frontal Area 3D: Three Dimensional Model
PA: Some Areas 9P: Nine Branches
BP: Reference Point NV: Normal Vector

Claims (12)

In a mobile robot device capable of autonomous driving,
main body;
A traveling module for moving the main body;
An audio sensing module including a plurality of audio sensors corresponding to different audio split regions around the main body, and generating a plurality of audio signals corresponding to the plurality of audio sensors; And
A voice command signal corresponding to at least one of a plurality of preset commands is detected from the plurality of audio signals based on a speaker dependent model trained by a voice of a specific user, and a control signal corresponding to the voice command signal is transmitted to the driving module. Mobile robot device comprising a control module for supplying.
The method of claim 1,
The control module
A voice command detector for detecting the voice command signal from the plurality of audio signals; And
A mobile robot device comprising a control signal generator for generating a control signal for driving control of the drive module.
The method of claim 2,
The control module
When the voice command detection unit detects the voice command signal, the audio command detection unit detects any one of the plurality of audio signals with the maximum volume level of the voice command signal, and determines an audio splitting area corresponding to the detected audio signal. A mobile robot device further comprising a voice position detector for detecting a voice position.
The method of claim 3, wherein
And a camera sensing module configured to generate a three-dimensional distance image signal of a front region, which is one side region around the main body.
The method of claim 4, wherein
When the voice command signal and the voice position are detected, the control signal generator generates a control signal in a standby state.
And the traveling module rotates the main body until the front region is disposed in the audio division region detected by the voice position detection module based on the standby control signal.
The method of claim 4, wherein
When the voice command signal and the voice position are detected, and the voice command signal corresponds to a start word for the specific user to call the mobile robot device among the plurality of commands, the control signal generator waits for the driving module. Transfer the control signal of the state,
And the traveling module rotates the main body until the front region is disposed in the audio division region detected by the voice position detection module based on the standby control signal.
The method of claim 4, wherein
The control module
And a user position detector configured to detect a user position coordinate corresponding to the specific user in the front area based on the 3D distance image signal by the camera sensing module.
The method of claim 7, wherein
And when the detection of the user position coordinate by the user position detector fails, the control signal generator stops generating a control signal corresponding to the detected voice command signal.
The method of claim 7, wherein
The user position detector,
A model calculator configured to calculate a 3D model corresponding to the 3D distance image signal;
A floor detector detecting a floor plane area of the three-dimensional model;
A tilt estimating unit estimating a tilt of the camera sensor module based on an average normal vector of the floor plane regions;
A model reconstruction unit reconstructing the three-dimensional model based on the estimated slope;
A height map calculator for projecting the restored three-dimensional model in a direction perpendicular to a horizontal plane to calculate a height map;
An object candidate detector for detecting an object candidate region in the reconstructed three-dimensional model based on the height map;
A target object detector configured to provide a target object model corresponding to a horizontal field of view of the camera sensor module, and detect a target object region corresponding to the target object model among the object candidate regions; And
And a user coordinate detector for tracking the target object region and detecting the user position coordinates in the front region.
The method of claim 9,
The target object model is a mobile robot device corresponding to the shape of the lower body of the person.
The method of claim 9,
When a plurality of target object regions are detected, the user coordinate detector detects a moving object among the plurality of target object regions, selects the region corresponding to the specific user, and detects the user position coordinates based on the selected region. Moving robot device.
The method of claim 9,
When a plurality of target object regions are detected, the user coordinate detector detects an object closest to the main body among the plurality of target object regions and selects the region corresponding to the specific user, and based on the selected region Mobile robot device for detecting position coordinates.

Images