KR101966718B1 - Apparatus for following user using sound sensor and method thereof - Google Patents

Abstract

The present invention relates to an apparatus and method for following a user using a sound sensor. According to an embodiment of the present invention, the apparatus comprises: a sound sensing unit sensing sound of at least four directions on the basis of a moving object; a path setting unit setting a movement path in a direction having a maximum value by accumulating intensity of the sound for each direction at a predetermined time period; a human body sensing unit sensing a human body when an obstacle is found while the moving object moves along the movement path; and a path correction unit stopping the movement of the moving object when the obstacle is the human body and moving the moving object if the obstacle is an object. Accordingly, the apparatus and method may sense the sound of at least four directions and accumulate the intensity so that the moving object follows a person.

Description

TECHNICAL FIELD [0001] The present invention relates to a user tracking device and a user tracking method using a sound sensor,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a user tracking device and a user tracking method using a sound sensor, and more particularly, to a technique for following a user by using sound intensity.

Recently, robots have been used in various fields due to diversification of scope and purpose of use of robots. Robots can be roughly divided into industrial robots and personal robots. A personal robot is called a human coexistence type humanoid robot having intelligence that can improve the convenience of personal life, and can be classified into a home robot supporting home automation and an entertainment robot for human play.

Korean Patent Registration No. 1643751 (published on June 28, 2016) discloses a robot having an object tracking function. The robot includes a traveling robot body, a traveling driving unit for traveling the robot body, A reception unit provided in the robot body and capable of communicating with the transmission unit so as to be able to receive signal information transmitted from the transmission unit; A controller for controlling the travel driving unit to move the robot body to a point adjacent to the object when the transmitter is out of a predetermined distance or more from the receiver, And a control unit.

However, the above-described conventional technique follows the point adjacent to the object through communication with the signal information of the object to be tracked, and there is a limitation in controlling the movement by the voice command or sound of the user.

SUMMARY OF THE INVENTION The present invention provides a user tracking device using a sound sensor that detects a sound of at least four directions and accumulates the intensity of the sound to follow a person.

The present invention also provides a user tracking device using a sound sensor capable of following a user by recognizing a voice command of a user.

Another object of the present invention is to provide a user tracking device using a sound sensor for resetting a moving path depending on whether a moving object senses a human body.

A user tracking apparatus using a sound sensor according to an exemplary embodiment of the present invention includes a sound sensing unit for sensing sound of at least four directions based on a moving object and a sound sensing unit for accumulating the sound intensity for each direction in a predetermined time period, A human body detecting unit for detecting a human body when an obstacle is detected while the mobile body moves along the movement path; and a control unit for stopping movement of the mobile body when the obstacle is a human body And a path correcting unit for moving the moving object to the bypass path when the obstacle is an object.

The route setting unit may determine whether or not speech is recognized in the sound, and may set the travel route by extracting a predetermined command when the speech is recognized.

The path setting unit may calculate an angle with the sound source using the difference between the sound intensity of the first sound sensor located in the first direction and the sound intensity of the second sound sensor located in the second direction.

In addition, the human body detection unit may determine the human body by acquiring motion information of the obstacle, and may determine the human body by acquiring thermal image information of the obstacle in the absence of motion.

The path correcting unit may reset the movement path by comparing the intensity of the sound accumulated in the first time interval and the intensity of the sound accumulated in the second time interval.

The control unit may further include a gas sensing unit for sensing a change in the concentration of carbon dioxide in the room at a predetermined time period, wherein the path setting unit maintains the sleep mode when the amount of change in the carbon dioxide concentration is equal to or less than a predetermined value, It is possible to switch to the operation mode.

The route setting unit may further include a roughness sensing unit that senses a variation in the illuminance of the room at a predetermined time period, and the route setting unit maintains the sleep mode when the roughness variation is less than a predetermined value, Mode.

The mobile terminal may further include a communication unit for communicating with the user terminal, and the path correcting unit may stop the movement of the mobile unit when receiving the proximity signal from the user terminal.

The sound sensing unit may amplify the sound to increase the voltage of the output signal in a first period of a consecutive time period to a predetermined ratio, and may increase a voltage of the output signal in a second period following the first period, The voltage of the output signal can be adjusted by filtering using an envelope which is reduced to a predetermined ratio.

A user following method of a user following device according to another exemplary embodiment of the present invention includes: a sound sensing step of sensing sound of at least four directions based on a moving object; accumulating the sound intensity in each direction in a predetermined time period A human body detection step of detecting a human body when an obstacle is detected while the mobile body moves along the movement path; and a movement step of moving the mobile body when the obstacle is a human body, And moving the moving object to the detour path if the obstacle is an object.

Accordingly, it is possible to detect a sound of at least four azimuths and accumulate the intensity, so that the mobile body can follow the person.

In addition, the user can recognize the voice command of the user and follow the user.

In addition, the moving path can be reset depending on whether the moving object senses the human body.

1 is a block diagram of a user tracking device using a sound sensor according to an embodiment of the present invention.
FIG. 2 is a flowchart of a user tracking method of a user tracking device using the sound sensor according to FIG. 1;
FIG. 3 is an exemplary view for explaining a moving body of a user following device using the sound sensor according to FIG. 1;
4 is an exemplary diagram for explaining a path setting in the user tracking device using the sound sensor according to FIG.
FIG. 5 is an exemplary diagram for explaining that a sound sensing unit amplifies a voice signal among user tracking devices using the sound sensor according to FIG. 1;

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The terms used are terms selected in consideration of the functions in the embodiments, and the meaning of the terms may vary depending on the user, the intention or the precedent of the operator, and the like. Therefore, the meaning of the terms used in the following embodiments is defined according to the definition when specifically defined in this specification, and unless otherwise defined, it should be interpreted in a sense generally recognized by those skilled in the art.

FIG. 1 is a block diagram of a user tracking device using a sound sensor according to an embodiment of the present invention, and FIG. 2 is a flowchart of a user tracking method of a user tracking device using the sound sensor according to FIG.

1 and 2, a user tracking device 100 using a sound sensor according to an embodiment of the present invention includes a sound sensing unit 110, a path setting unit 120, a human body sensing unit 130, (140).

The sound sensing unit 110 senses sound of at least four directions based on the moving object (S110). Here, the moving body is driven by a wheel using an electric motor. The moving body may be formed in a box shape, but is not limited thereto. For example, the sound sensing unit 110 may be formed on the side surface of the moving body in the directions of north, south, east, and west respectively. The sound sensing unit 110 senses sound using an analog sound sensor. The sound sensing unit 110 senses sounds on the front, rear, left, and right sides of the moving object, and outputs the sound to the path setting unit 120 when a sound having a predetermined intensity or more is sensed.

In addition, the sound sensing unit 110 amplifies the sound to increase the voltage of the output signal in the first period of the consecutive time period to a predetermined ratio. In the second period subsequent to the first period, It is possible to perform filtering using an envelope that is reduced to a set ratio. This is to amplify the sound when the sound is the user's voice signal and accurately analyze the voice signal using the envelope optimized for the user's voice.

Next, the path setting unit 120 accumulates sound intensities for each direction at a predetermined time period and sets a movement path in a direction having a maximum value (S120). In this case, the sound intensity can be set in decibels (dB). The path setting unit 120 sets a movement path for moving the moving body in the front, rear, left, and right directions. The path setting unit 120 can input and store the coordinates of the room in advance. This is to set a route to avoid an obstacle when moving in the room. The path setting unit 120 accumulates sound intensities in each direction at a predetermined time period and compares the maximum values. In this case, the path setting unit 120 measures the intensity of sound using Equation (1) below.

In Equation (1), D represents the intensity of sound, x represents measured sound, and t represents time. t ₁ represents the time at the first time point, and t ₂ represents the time at the second time point. The path setting unit 120 updates and manages the value of the sound intensity for each direction by a predetermined time period. The path setting unit 120 sets the movement path in the direction in which the sound intensity has the maximum value, regenerates the movement path by repeating the process of accumulating the sound intensity again and comparing the maximum value when the moving object is stopped.

In addition, the path setting unit 120 may determine whether a sound in the sound is recognized from the sound sensing unit 110. In this case, when the voice is recognized, the path setting unit 120 extracts a predetermined command and sets a movement path. In other words, the path setting unit 120 firstly determines whether the sound is a human voice or a mechanical sound, and if the sound is a human voice, it is determined whether or not the command is included. This is to allow the moving object to move only when a command for instructing the movement of the moving object is displayed by distinguishing the sound of a television or the like. The command for commanding the movement of the moving object can be varied according to the setting of the user. Accordingly, the moving object can more accurately follow the user by analyzing the voice command of the person.

The path setting unit 120 sets the angle of the sound source with the sound source using the difference between the sound intensity D1 of the first sound sensor located in the first direction and the sound intensity D2 of the second sound sensor located in the second direction. Can be calculated. In this case, the characteristic table indicating the distance according to the sound intensity in each direction is set in advance. The sound intensity D1 in the first direction and the sound intensity D2 in the second direction are applied to the characteristic table and the distance is converted to determine the position of the sound source by triangulation. The path setting unit 120 can set the movement path by calculating the angle with the sound source from the first direction or the second direction.

Next, the human body sensing unit 130 senses the human body when it detects an obstacle while moving along the movement path (S130). The human body detecting unit 130 is for causing a mobile body to follow a person, and can detect the human body using motion detection, contact detection, or the like. For example, if a person's motion is detected by using a Doppler sensor, an infrared sensor, or the like, it can be judged as a person. The human body detecting unit 130 can detect the human body in a complex manner using different kinds of sensors.

In addition, the human body sensing unit 130 can determine the human body by acquiring movement information of the obstacle, and if there is no movement, obtain the thermal image information of the obstacle. The human body sensing unit 130 captures a thermal image of a target object using a thermal imaging camera. In this case, the human body detecting unit 130 can distinguish whether the obstacle is a person or an object by using thermal image information for a preset person. This is because it is difficult to judge whether the obstacle is human or not if motion of the obstacle is not detected. Therefore, the human body sensing unit 130 can discriminate whether it is a human being even in the case of an obstacle without motion using thermal image information.

Next, the path correcting unit 140 stops moving the moving object when the obstacle is a human body, and moves the moving object to the detour path if the obstacle is an object (S140). The path correcting unit 140 determines whether or not the moving object moves. The moving object follows the person. When the moving object is close to the human body, the moving object is stopped and operates in the standby mode. Thereafter, when additional sound is detected, the mode is changed to the operation mode, and the sound is moved along with the sound of the person. The path correcting unit 140 corrects the path so as to bypass the obstacle when the obstacle is an object. This path correction can be corrected through machine learning.

Also, the path correcting unit 140 can reset the travel path by comparing the accumulated sound intensity D1 in the first time interval and the sound intensity D2 accumulated in the second time interval. For example, when the moving object approaches the person based on the intensity of the sound obtained in the first time period, the operation is stopped. Thereafter, the operation mode is switched based on the sound intensity acquired in the second time period. In this way, the path correcting unit 140 determines the moving direction according to the accumulated sound volume in the additional time interval. Therefore, when a person moves, a moving object issues a sound at a changed position, and the moving object senses and follows the moving object.

Meanwhile, the user tracking device 100 using a sound sensor according to an embodiment of the present invention may further include a gas sensing unit 150. The gas sensing unit 150 senses a change in the concentration of carbon dioxide in the room at a predetermined time period. The gas sensing unit 150 senses the presence of a person who has lost a room according to the concentration of carbon dioxide in the room. When a person is indoors, the concentration of carbon dioxide increases with time, and the amount of increase of carbon dioxide according to the indoor area can be confirmed by a preset characteristic table. The gas sensing unit 150 outputs the sensed gas information to the path setting unit 120.

In this case, the path setting unit 120 maintains the sleep mode when the variation amount of the carbon dioxide concentration is equal to or less than the predetermined value, and switches to the operation mode when the variation amount of the carbon dioxide concentration exceeds the preset value. This is to reduce the power consumption by operating in the sleep mode when there is no person, and to be able to operate effectively only when a person is lost. The path setting unit 120 may stop or move the moving object according to the change amount of the carbon dioxide through the gas sensing unit 150, thereby improving the accuracy of the human follow-up by an additional determination other than sound.

Meanwhile, the user tracking device 100 using the sound sensor according to the embodiment of the present invention may further include a roughness sensing unit 160. The roughness sensing unit 160 senses the variation in the illuminance of the room at a predetermined time period. The rough image sensing unit 160 can sense the presence or absence of a person by detecting the presence of ambient light or a change in the interior lighting. Generally, when a person is rewound, when the lighting is turned on or off by using the lighting, it is possible to set whether the illumination difference exceeds the predetermined value to judge whether the user is rewired or not. The roughness sensing unit 160 can detect a change in illuminance when a person is within a predetermined distance. This makes it possible to detect the proximity of an obstacle or the like by using the illuminance change even when the distance measuring sensor malfunctions.

In this case, the path setting unit 120 maintains the sleep mode when the illumination variation amount is equal to or less than the predetermined value, and switches to the operation mode when the illumination variation amount exceeds the preset value. The path setting unit 120 may stop or move the moving object according to the variation in illumination intensity. The path setting unit 120 may switch to the sleep mode to minimize power consumption when the movement of the moving object is stopped.

Meanwhile, the user tracking device 100 using the sound sensor according to the embodiment of the present invention may further include a communication unit 170.

The communication unit 170 communicates with an external user terminal. The communication unit 170 can perform short-range communication such as Bluetooth, or Wi-Fi, 4G, and 5G network communication. The communication unit 170 may share the authentication information through communication with the user terminal. In the case of a user terminal used by a user, a moving object transmits a proximity signal to a moving object in proximity. In this case, when the path correcting unit 140 receives the proximity signal from the user terminal, the path correcting unit 140 can stop the movement of the moving body. For example, it is also possible to mount a wireless charger module on a mobile device. When the user approaches the mobile terminal by approaching the mobile terminal, the mobile terminal transmits the proximity signal while charging the battery. In this case, the mobile unit stops moving and waits. The communication unit 170 can also receive a control signal from the user terminal.

FIG. 3 is an exemplary view for explaining a moving body of a user following device using the sound sensor according to FIG. 1;

Referring to FIG. 3, the moving body 10 may be formed in a box shape. In this case, a plurality of sound sensing units 110 may be formed on the side surface of the moving body 10. The moving body 10 may be formed with a sound sensing unit 110 in a manner such as four orientations, six orientations, and eight orientations. The moving body 10 can move on the ground using an electric motor. Also, the mobile body 10 may be formed in a dragon shape and fly. The moving body 10 may be electrically driven by forming a battery therein. It is also possible that the user follow-up apparatus 100 of the present invention is accommodated in the internal accommodation space of the mobile body 10. A buffer material is formed inside the moving body 10 to prevent the impact.

4 is an exemplary diagram for explaining a path setting in the user tracking device using the sound sensor according to FIG.

Referring to FIG. 4, the user follow-up device 100 can be converted into a distance according to the size of an external sound and set. If there is a maximum size according to the sound size 401 in the first direction and the sound size 402 in the second direction, a weighting rule can be given in the corresponding direction. In this case, the movement path 400 can be set by forming a vector according to the sound volume and combining the sizes of sounds in the first direction and the second direction. The moving body 10 can be moved in the forward, backward, leftward, or rightward direction by adjusting the angle of the moving path 400.

FIG. 5 is an exemplary diagram for explaining that a sound sensing unit amplifies a voice signal among user tracking devices using the sound sensor according to FIG. 1;

Referring to FIG. 5, the sound sensing unit 110 filters an output voltage of a voice signal in a first period a of a continuous time interval using an envelope that amplifies the output voltage of the voice signal at a predetermined ratio. The output voltage V (t) in the _{first period a} can be expressed as: V (t) = x (t) + Vm (0? T <T ₁ ) In this case, x is a positive number, and Vm represents an initial voltage. Therefore, in the first section (a), an envelope filter having a positive slope can be used to reflect the characteristic of the start sound of the speech signal.

Next, in the second section (b) continuous to the first section (a), filtering can be performed using an envelope that reduces the voltage of the output signal to a predetermined ratio. The output voltage V (t) in the second section b can be expressed as: V (t) = - x (t) + V _M (T ₁ ? T <T ₂ ) In this case, x is a positive number, and V _M represents the maximum voltage. Accordingly, in the second period (b), an envelope filter having a negative slope can be used to reflect characteristics after the start of a speech signal.

Next, in the third section (c) continuous to the second section (b), the voltage of the output signal can be kept constant. The output voltage V (t) in the third period (c) can be expressed as " V (t) = a (T ₂ ? T <T ₃ ) In this case, a is a positive number and represents the limiting voltage (V _th ). In the third period (c), the output of the audio signal is limited to the limit voltage ( _Vth ) or less.

Next, in the fourth section (d) continuous to the third section (c), the output signal can be filtered using an envelope that reduces the voltage to a predetermined ratio. The output voltage (V (t)) in the fourth section (d) can be represented as "V (t) = ax ( t≤T 4)". In this case, x is a positive number, a is a positive number, and represents a limiting voltage (V _th ). In the fourth period (d), the output voltage can be limited by using an envelope filter that reduces the output of the audio signal by a predetermined ratio from the limit voltage ( _Vth ). It processes the end of the voice signal and can adjust the x value by reflecting the characteristics of the voice signal.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, Therefore, the present invention should be construed as a description of the claims which are intended to cover obvious variations that can be derived from the described embodiments.

10: Moving object
100: User follower
110: sound detection unit
120: Path setting unit
130: Human body detection unit
140: path correcting unit
150: gas sensing unit
160:
170:
400: Movement path
401: Sound size in the first direction
402: Sound size in the second direction

Claims (10)

A sound detection unit for detecting sound of at least four directions with respect to a moving object;
A path setting unit for setting a movement path in a direction having a maximum value by accumulating the sound intensity for each direction at a predetermined time period;
A human body detecting unit for detecting the human body when the mobile body moves along the movement path and detects an obstacle;
A path correcting unit for stopping movement of the moving object when the obstacle is a human body and moving the moving object to a bypass path when the obstacle is an object; And
And a roughness sensing section for sensing a variation in the illuminance of the room at a predetermined time period,
Wherein the path setting unit comprises:
Wherein the controller is maintained in a sleep mode when the illumination variation amount is less than or equal to a predetermined value, and switches to an operation mode when the illumination variation amount exceeds a preset value. The method according to claim 1,
Wherein the path setting unit comprises:
And a sound sensor for determining whether the sound is recognized in the sound and extracting a predetermined command when the sound is recognized to set the movement path. The method according to claim 1,
Wherein the path setting unit comprises:
And a sound sensor for calculating an angle between the sound intensity of the first sound sensor located in the first direction and the sound intensity of the second sound sensor located in the second direction, . The method according to claim 1,
The human-
And a sound sensor for determining whether the human body is obtained by acquiring motion information of the obstacle and acquiring thermal image information of the obstacle in the absence of motion. The method according to claim 1,
Wherein the path correcting unit comprises:
And a sound sensor for comparing the intensity of the sound accumulated in the first time interval and the intensity of the sound accumulated in the second time interval to reset the movement path. The method according to claim 1,
Further comprising a gas sensing unit for sensing a change in the concentration of carbon dioxide in the room at a predetermined time period,
Wherein the path setting unit comprises:
Wherein the controller is maintained in a sleep mode when the amount of change in the carbon dioxide concentration is equal to or less than a predetermined value and switches to an operation mode when the amount of change in the carbon dioxide concentration exceeds a preset value. delete The method according to claim 1,
Further comprising a communication unit for communicating with a user terminal,
Wherein the path correcting unit comprises:
And a sound sensor for stopping movement of the moving object when the proximity signal is received from the user terminal. The method according to claim 1,
Wherein the sound sensing unit comprises:
Amplifying the sound to increase the voltage of the output signal in a first period of a consecutive time interval and decreasing a voltage of the output signal in a second interval that is continuous to the first interval, For adjusting the voltage of the output signal. A user follow-up method of a user follow-
A sound sensing step of sensing a sound of at least four directions based on the moving object;
Accumulating the sound intensity in each direction at a predetermined time period to set a movement path in a direction having a maximum value;
A human body sensing step of sensing a human body when the mobile body moves along the movement path and detects an obstacle;
Stopping the movement of the moving object when the obstacle is a human body, and moving the moving object to a detour path if the obstacle is an object; And
And an illuminance detection step of sensing a variation in illuminance of the room at a predetermined time period,
The path setting step may include:
Wherein the sleep mode is maintained in a sleep mode when the illumination change amount is equal to or less than a predetermined value, and is switched to an operation mode when the illumination change amount exceeds a preset value.

Images