KR101734398B1 - Smart home service apparatus based on object position and spoken dialogue - Google Patents

Abstract

The present invention relates to a method for a smart home service based on object location and voice conversation. The method comprises the following steps: acquiring motion data of an object; allowing a control unit to obtain the position information of the object from the motion data of the object, to arrange the position information of the object by time zone to obtain motion line information of the object, to generate object determination information by comparing the motion line information with reference information stored in the database, to generate a voice query statement corresponding to the object determination information to transmit the information to a speaker array, and to allow the speaker array to convert the voice query statement into a voice query signal to output the signal; receiving a voice response signal from the object; and controlling a smart home service corresponding to a voice response statement generated from the voice response signal to be provided. According to the present invention, accuracy of the indoor positioning and the voice processing can be improved by integrating an IR-UWB radar technology and the sound quality improvement technique using the beam forming, thereby greatly reducing the possibility of privacy invasion.

Description

FIELD OF THE INVENTION [0001] The present invention relates to a smart home service apparatus based on object location and voice conversation,

The present invention relates to a smart home service device based on object location and voice conversation. More specifically, the present invention is based on the object position and voice conversation, which can improve the accuracy of indoor positioning accuracy and voice processing, and greatly reduce the possibility of invasion of privacy, by combining IR-UWB radar technology and sound quality improvement technique using beam forming And a smart home service apparatus.

Smart home is a crystal of convergence industry that spans the majority of areas related to personal consumption based on internet of things (IoT), and market area is very wide.

Medical health and security services among smart home related industries are important areas, and various technologies are being developed in recent years based on sensor technology.

One of the major constraints on smart home implementation is privacy and security issues, and whether it is possible to provide services with a technology that is acceptable to the user is very important.

Video-based technologies, such as CCTV, have the problem that users are very unhappy in terms of personal privacy protection in the home due to the nature of smart home.

In the case of a single-person household, including the elderly living alone, there is a need for a method that can quickly and accurately detect an outside situation when an emergency occurs, such as a resident's health problem. However, until now, It is not.

Korean Patent Laid-Open Publication No. 10-2008-0092038 (public date: October 15, 2008, name: Ubiquitous home network system and voice service and lighting control method using the same)

It is a technical object of the present invention to provide a smart home service device based on object location and voice conversation that can improve indoor positioning accuracy and voice processing accuracy by converging IR-UWB radar technology and sound quality improvement technique using beam forming do.

Further, the present invention provides a home service control and a security service based on object location and voice conversation in a smart home, thereby improving the quality of a smart home service.

Another object of the present invention is to effectively provide a smart home service while greatly reducing the possibility of privacy invasion by using user location information detected using a radar.

According to the present invention, there is provided a smart home service method based on object location and voice conversation, comprising: a motion data acquiring step of acquiring motion data of an object from a motion data acquiring unit; and a motion data acquiring step of acquiring motion data of the object received from the motion data acquiring unit Acquiring positional information of the object, arranging the positional information of the object in a time zone to obtain the object information of the object, generating the object determination information by comparing the object information with the reference information stored in the database, A voice query step of generating a voice query statement corresponding to the determination information and transmitting the generated voice query message to a speaker array provided in the voice conversation support unit and converting the voice query sentence into a voice query signal and outputting the voice query signal; Microphone array provided in the conversation support section The via includes a voice response receiving step and smart home service provider and controlling such that the smart home service provider to the control unit corresponding to the voice response generated from the audio response signal for receiving a voice response signal from the object.

In the smart home service method based on object location and voice conversation according to the present invention, the motion data obtaining unit includes a plurality of IR-UWB (Impulse Radio-Ultra Wide Band) radars located at distant points from each other.

In the smart home service method based on the object location and voice conversation according to the present invention, after the voice response reception step, the control unit sets the position of the object to a virtual position of a sound source, and performs a beamforming ) Method to improve the sound quality of a voice response signal received from the object through the microphone array.

The smart home service method based on object location and voice communication according to the present invention further includes a sound quality improving step for improving the sound quality of a voice response signal received from the object through the microphone array after the voice response receiving step Wherein the sound quality improving step comprises: a sound source position setting step in which the controller sets the position of the object to a virtual position of the sound source; and a control unit that controls the unit microphones, A directional angle / distance calculation step of calculating unit distances between the microphones and the sound source, and a phase angle calculation step of aligning the phases of unit voice response signals received by the unit microphones using the unit direction angles and the unit distances, Improve the signal-to-noise ratio of the response signal Signal-to-noise ratio is characterized in that it comprises a step improvement.

In a smart home service method based on object location and voice communication according to the present invention, when the object judgment information indicates a normal user in an ordinary situation, in the voice query step, Generates a normal contextual voice query sentence corresponding to the location of the object in terms of time by referring to the reference information, and transmits the generated contextual voice query sentence to the speaker array, and the speaker array converts the normal contextual query sentence into a normal contextual query signal Wherein the control unit converts the normal situation voice response signal received from the microphone array into a normal situation voice response sentence and refers to the reference information stored in the database to correspond to the normal situation voice response sentence Smart It characterized in that the control such that the service is provided.

In a smart home service method based on object location and voice communication according to the present invention, when the object judgment information indicates a critical user in an emergency situation, in the voice query step, the control unit refers to the reference information stored in the database And the speaker array converts the emergency voice query sentence into an emergency situation voice query signal and outputs the emergency voice query message to the speaker array. In the smart home service providing step, the controller transmits the emergency voice query message to the speaker array, When the voice response signal corresponding to the emergency voice query signal is not received from the microphone array, it is determined that the current situation is an emergency and the emergency situation information is transmitted to the external remote device registered in the database do.

In a smart home service method based on object location and voice communication according to the present invention, when the object judgment information indicates a critical user in an emergency situation, in the voice query step, the control unit refers to the reference information stored in the database And the speaker array converts the emergency voice query sentence into an emergency situation voice query signal and outputs the emergency voice query message to the speaker array. In the smart home service providing step, the controller transmits the emergency voice query message to the speaker array, The emergency situation voice response sent from the microphone array is converted into an emergency situation voice response sentence. If the emergency situation voice response sentence includes a content indicating an emergency situation, it is determined that the current situation is an emergency, To an external remote device And the emergency situation information is transmitted.

In a smart home service method based on object location and voice communication according to the present invention, when the object judgment information indicates an intruder, in the voice query step, the control unit refers to the reference information stored in the database, Wherein the controller generates a query message and transmits the query message to the speaker array, and the speaker array converts the invasion situation voice query message into an emergency situation query signal, and outputs the emergency situation query query signal. In the smart home service providing step, When the voice response signal corresponding to the intrusion context voice query signal is not received, it is determined that the current situation is an intrusion situation and the intrusion status information is transmitted to the external remote device registered in the database.

In the smart home service method based on object location and voice conversation according to the present invention, in the smart home service providing step, when the current situation is determined as the emergency situation or the intrusion situation, the control unit operates the image capturing unit The image information of the object is captured so that the image information of the object is included in the emergency situation information or the invasion situation information and is transmitted to the external remote device.

According to the present invention, there is provided an effect of providing a smart home service device based on object location and voice conversation, which can improve indoor positioning precision and voice processing accuracy by converging IR-UWB radar technology and sound quality improvement technique using beam forming have.

Also, by providing home control and security service based on object location and voice conversation in smart home, the quality of smart home service is improved.

Also, by using the detected user location information using the radar, it is possible to effectively provide a smart home service while greatly reducing the possibility of invasion of privacy.

FIG. 1 is a diagram illustrating an example of a device configuration in which a smart home service method based on object location and voice conversation is performed according to an embodiment of the present invention,
2 is a diagram illustrating a smart home service method based on object location and voice conversation according to an embodiment of the present invention,
3 is a diagram showing an example of a configuration of a voice query step in an embodiment of the present invention,
4 is a diagram illustrating an example of a configuration of a sound quality improving step in an embodiment of the present invention,
5 is a view for explaining a directional angle / distance calculating step included in the sound quality improving step in an embodiment of the present invention,
6 is a view for explaining an operation example in a case where object determination information indicates an ordinary user in a normal situation in an embodiment of the present invention,
7 is a diagram for explaining an operation example in a case where object determination information indicates a critical user in an emergency situation in an embodiment of the present invention,
8 is a diagram for explaining an operation example in a case where object determination information indicates an intruder according to an embodiment of the present invention.

It is to be understood that the specific structural or functional description of embodiments of the present invention disclosed herein is for illustrative purposes only and is not intended to limit the scope of the inventive concept But may be embodied in many different forms and is not limited to the embodiments set forth herein.

The embodiments according to the concept of the present invention can make various changes and can take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. It should be understood, however, that it is not intended to limit the embodiments according to the concepts of the present invention to the particular forms disclosed, but includes all modifications, equivalents, or alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms may be named for the purpose of distinguishing one element from another, for example, without departing from the scope of the right according to the concept of the present invention, the first element may be referred to as a second element, The component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like are used to specify that there are features, numbers, steps, operations, elements, parts or combinations thereof described herein, But do not preclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

Before explaining the preferred embodiments of the present invention, the basic principle of the present invention will be described.

The present invention combines object 2D positioning technology using IR-UWB (Impulse Radio-Ultra WideBand) radar with sound quality improvement technology using beamforming, and supports the object position of the open IoT (Internet of Things) Based smart home service.

The basic functions and advantages of the present invention are summarized as follows.

In the present invention, since the sound quality improvement technique based on the user's location information can control and communicate with the device by voice even at a long distance without the user having to carry the terminal unlike the existing methods, It can be effectively applied to the implementation of smart home related services based on the open IOT platform.

In the present invention, since the IR-UWB radar-based 2D positioning technology does not use image information in home monitoring for providing security service, which is an important field of smart home implementation, unlike CCTV, which has a privacy invasion problem, Privacy can be protected.

The IR-UWB radar and the beam quality improvement technique using the beam forming will be described in more detail as follows.

UWB technology, which was born for the purpose of developing high resolution radar for military purposes, has been open to the private sector since the early 2000s and has advantages of low power, broadband characteristics, high precision and permeability.

The IR-UWB radar technology is a technology that emits an impulse signal having a wide frequency characteristic to use a signal received from a person or an object, and has an excellent distance resolution characteristic due to its broadband characteristic and a relatively excellent transmittance . According to this IR-UWB radar technology, unlike other sensors, it is possible to operate normally even in an environment where there is no light or in a field where visibility can not be secured due to deterioration of weather such as snow, rain, Due to the lack of characteristics, it is free from the privacy problem and unlike the method using CCTV, it has a simple hardware structure, so it is possible to design low power and low price, and it is possible to measure precise distance between the radar and target from the received signal through a series of signal processing It is possible to position the target as well as the target when using a plurality of radar devices.

Meanwhile, beamforming, a method of smart antenna, is a technique for localizing the beam in a specific direction. In the field of sound quality improvement, a microphone array is used for sound source position estimation. In addition to the target sound source, There is a problem that it is difficult. The error generated in the sound source localization process is an important factor that degrades the performance of the beamforming based sound quality improvement. Additional operations to reduce this error increase the complexity of the technique.

The present invention fuses a user location detection technique based on an IR-UWB radar technology supporting a high accuracy of several centimeters to several tens of centimeters and a sound quality improvement technique based on beamforming technology to precisely detect an object position in a service area, Improves the quality of the sound received from the object, thereby greatly improving the efficiency of the smart home service.

Unlike video technology such as CCTV, IR-UWB radar technology can detect user's position without using image, minimizing privacy exposures, and achieving precision of several tens of centimeters in position detection. It is possible to accurately grasp the position of a user at risk in a situation.

In addition, by fusing the position detection accuracy of IR-UWB radar technology with beamforming-based sound quality improvement technology, which is one of the techniques for improving voice communication quality, it is possible to quickly and accurately detect the position of a sound source, An improvement result can be obtained. According to the present invention, the position of the sound source can be detected at the 3D level by detecting the user's face position one step further in the IR-UWB radar technology at the 2D position detection level.

By combining the user 2D location information using IR-UWB radar technology and the sound quality improvement technique based on the beam-forming technique based on it, it is possible to improve user convenience and service accuracy by combining with voice-interactive home appliance control and security service in IoT-based smart home service field .

More specifically, the user position detection by IR-UWB radar which supports the accuracy in the range of several centimeters to several tens centimeters enables the user who is moving more than about 1 meter away from the microphone to remove background noise to provide high quality voice call quality . In addition, through the improvement process of the sound quality by the beam-forming technique, it is possible to correct the error that occurs when the user position is obtained by the IR-UWB radar, and more accurate positioning result can be obtained. One of the healthcare items, the activity analysis, enables more accurate analysis. In addition, the user location information by the IR-UWB radar reduces the computation amount of the sound source location process of the sound quality improvement process by beam forming, and even when there are two or more users in the service area, The position can be detected quickly, and this information can be applied to the beamforming sound quality improvement to provide improved voice quality for each user's voice.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a diagram illustrating an example of a device configuration in which a smart home service method based on object location and voice conversation is performed according to an exemplary embodiment of the present invention. Based smart home service.

1 and 2, a smart home service method based on object location and voice conversation according to an exemplary embodiment of the present invention includes a step of acquiring motion data S10, a voice query step S20, a voice response receiving step S30 A sound quality improving step S40, and a smart home service providing step S50.

In the motion data acquisition step (S10), the motion data acquisition unit 10 acquires motion data of the object. For example, the motion data obtaining unit 10 may include two or more Impulse Radio-Ultra Wide Band (IR-UWB) radars installed at distant points from each other. 1 shows an example in which the motion data obtaining unit 10 includes a first IR-UWB radar 110 and a second IR-UWB radar 120. However, the present invention is not limited thereto, May be comprised of three or more IR-UWB radars. Each IR-UWB radar has a transmitting end that emits an impulse signal and a receiving end that receives the impulse signal from the environment or from a person. The sum of the received signals is motion data of the object.

In the voice query step S20, the control unit 30 obtains the position information of the object from the motion data of the object received from the motion data obtaining unit 10, arranges the position information of the object in the time zone, Generates the object judgment information by comparing the movement information with the reference information stored in the database 40, generates a voice query in the form of text corresponding to the object judgment information, and performs TTS (Text To Speech) processing And transmits the voice query sentence data corresponding to the voice query sentence generated through the speaker array 210 to the speaker array 210 provided in the voice chat support unit 20. The speaker array 210 converts voice query sentence data into analog voice And a process of converting it into a query signal and outputting it is performed.

With reference additionally to Fig. 3, an example of the configuration of the voice query step S20 will be described.

3, the voice query step S20 may be configured to include steps S210, S220, S230, S240, and S250.

In step S210, the control unit 30 acquires position information of the object from the motion data of the object received from the motion data obtaining unit 10. [ Hereinafter, an example of a specific configuration in which the control unit 30 acquires the position information of the object will be described, but the configuration for acquiring the object position information is not limited thereto.

First, a process of receiving the motion data, which is a raw signal, from the motion data obtaining unit 10 can be performed by the control unit 30. The impulse signal emitted from the transmitting end of the IR-UWB radar is reflected from objects or people in the environment and is received at the receiving end through various paths. The signal is converted into a digital signal through an ADC (Analog Digital Converter) . The received signal also includes a background noise signal, such as a signal from a clutter, such as a stationary object in an environment that does not want to be detected, and thermal noise present in the environment. Next, a filtering process for suppressing noise components existing in a frequency band other than the frequency band to be used may be performed. Next, a background subtraction process for removing the clutter components other than the target for detection of the target may be performed. Next, a process of compensating for signal attenuation may be performed. As the position of the target is farther away from the IR-UWB radar, the strength of the received signal is greatly attenuated. Therefore, the distance compensation is performed for the received signal in order to detect the target located at a distance, A noise signal which is not desired to be detected during the process may be amplified to cause false detection, so that an appropriate compensation algorithm can be applied to cope with this. Next, Ranging of multi-target process can be performed. From the front-end received signal, an algorithm can be performed for each distance measurement between the radar and the multiple targets. Since there may be multiple paths from a single target, multiple target detection can be prioritized and each distance measurement algorithm based on this can be applied. Next, a Localization of multi-target process can be performed. An algorithm may be performed that collects distance information for multiple targets obtained from a plurality of IR-UWB radar receivers to determine the location of multiple targets. Since the distance information is collected in a non-tagged manner in which the objects among the targets are not distinguished, the object sorting and position location algorithm can be performed through data analysis between a plurality of distance information. Next, a Tracking of multi-target algorithm may be performed to process the position information of the target that is continuously collected for position determination of the more reliable target.

In step S220, the control unit 30 arranges the object position information by time zone to obtain moving line information of the object. The moving information of the object can be obtained by connecting the position information of the object at a specific point in time. Time information about the point of time, position information of the object at the point of time, (40), and the sum of the information stored in the database (40) is the reference information.

In step S230, the control unit 30 compares the copper information of the object with the reference information stored in the database 40 to obtain object determination information. For example, the object judgment information may indicate a party user in an ordinary situation, indicate a party user in an emergency situation, or indicate an illegal intruder who is not a party user.

In step S240, the control unit 30 generates a voice query sentence in a text format corresponding to the object determination information, generates voice query sentence data corresponding to the query sentence generated through the TTS (Text To Speech) process, To the speaker array 210 provided in the support unit 20 is performed.

In step S250, the process of converting the voice query sentence data transmitted from the speaker array 210 from the control unit 30 into an analog voice query signal, i.e., sound is performed.

In the voice response reception step S30, the control unit 30 performs a process of receiving a voice response signal from the object through the microphone array 220 provided in the voice conversation support unit 20. [ For example, this process can be performed in such a manner that the microphone array 220 receives the voice response signal from the object, that is, the voice of the object, and the control unit 30 receives the signal from the microphone array 220 .

In the sound quality improving step S40, a process of improving the sound quality of the voice response signal received from the object through the microphone array 220 is performed by the control unit 30. [

As an example, in the sound quality improving step S40, the controller 30 sets the position of the object to a virtual position of a sound source, applies a virtual position of the sound source to a beamforming method, May be configured to improve the quality of the voice response signal received from the object.

4, the sound quality improving step S40 may include a sound source position setting step S410, a directional angle / distance calculating step S420, and a signal-to-noise ratio improving step S430.

In the sound source position setting step S410, the control unit 30 sets the position of the object to the virtual position of the sound source.

In the directional angle / distance calculation step (S420), the controller 30 calculates the unit distances between the unit microphones constituting the microphone array 220 and the unit distances between the microphones and the sound sources do. 5 is a diagram for explaining a directional angle / distance calculation step (S420) included in the sound quality improving step. 5, the distance between the object, i.e., the sound source, and the unit microphones constituting the microphone array 220 is different. Therefore, the time taken for the sound source signal output from the sound source to reach each unit microphone is different. For example, when a unit microphone located at a distance closest to a sound source is set as a reference microphone, a sound source signal arriving at each of the other unit microphones based on the reference microphone has a time delay of? Ti. In an embodiment of the present invention, for convenience of description, a unit microphone positioned at a distance closest to a sound source is set as a reference microphone, but the present invention is not limited to this, and any one of a plurality of unit microphones may be set as a reference microphone have. When the distance between the sound source and the unit microphones is relatively much larger than the distance between the unit microphones, the paths of the sound source signals output from the sound source to the plurality of unit microphones are parallel to each other. That is, the angles? I between the unit microphones and the reception paths of the sound source signals are all the same.

In the signal-to-noise ratio improving step S430, a process of improving the signal-to-noise ratio of the voice response signal by aligning the phases of the unit voice response signals received by the unit microphones using the unit direction angles and the unit distances is performed. For example, in the step of improving the signal-to-noise ratio (S430), considering the delay time occurring when the sound source signal reaches each unit microphone on the basis of the unit microphone located closest to the sound source, The phase of the sound source signal can be synchronized.

In the smart home service providing step (S50), a process of controlling the controller 30 to provide a smart home service corresponding to the voice response sentence generated from the voice response signal is performed. The specific service provided in the smart home service providing step (S50) may be determined by the object determining information generated in the voice querying step (S20).

Hereinafter, examples of specific operations of the present invention will be described with additional reference to Figs. 6 to 8. Fig.

FIG. 6 is a diagram for explaining an operation example in a case where object determination information indicates an ordinary user in an ordinary situation in an embodiment of the present invention.

6, when the object determination information determined by the controller 30 in the voice query step S20 indicates a normal user in an ordinary situation, in the voice query step S20, 30 generates a contextual voice query sentence having a text format corresponding to the position of the object in the time zone with reference to the reference information stored in the database 40. The control unit 30 transmits the contextual voice query sentence to the normal situation Normal situation voice query sentence data corresponding to the voice query sentence is generated and transmitted to the speaker array 210, and the speaker array 210 converts the normal situation voice query sentence data into the analogous normal type contextual voice query signal and outputs the same .

In the smart home service providing step S50, the control unit 30 converts the normal situation voice response signal of the party user received from the microphone array 220 into the normal situation voice response sentence, To provide a smart home service corresponding to the normal situation voice response sentence.

More specifically, first, through the IR-UWB radars, a party user detects moving to the kitchen around 10 pm.

Next, the control unit 30 inquires of the database 40 about the user's home appliance control activity at the kitchen position at about 10 pm, thereby deriving the 'expected time for the electric cooker's brown rice cooking time 5 hours', which is the expected home appliance control operation.

Next, the control unit 30 generates a sentence 'Would you like to set 5 hours of electric rice cooker brown rice cooking time?' To be queried to a party user, and then outputs it to the speaker array 210 through TTS (Text To Speech) conversion .

Next, the controller 30 stores voice response signals of the user received through the microphone array 220 for each channel, and then performs beamforming sound quality improvement processing using the 2D position information of the user.

Next, the control unit 30 performs voice recognition through STT (Speech To Text) conversion with improved voice quality data, thereby grasping the user's response.

Next, when the determined answer is affirmative, the control unit 30 generates a control signal corresponding to the setting time of the electric rice cooker brown rice cooking time 5 hours, and transmits a control signal to the electric rice cooker through communication between the control unit 30 and the rice cooker And adds the processed control-related information to the database 40. [ If the response is negative, the control unit 30 generates a statement for inquiring the user about a desired home appliance control action, generates a control signal corresponding to the home appliance control action desired by the user through the above- Transmits the control signal to the home appliance through communication with the home appliance, and adds the control related information processed in the database (40).

7 is a diagram for explaining an operation example in a case where object determination information indicates a critical user in an emergency situation in an embodiment of the present invention.

7, when the object judgment information judged by the control unit 30 in the voice query step S20 indicates the party user in the emergency situation, in the voice query step S20, the control unit 30 30 generates an emergency voice query message referring to the reference information stored in the database 40 and transmits the emergency voice query message to the speaker array 210. The speaker array 210 converts the emergency voice query message into an emergency voice query signal And outputs it.

As one example, in the smart home service providing step S50, when the control unit 30 does not receive the voice response signal corresponding to the emergency situation voice query signal from the microphone array 220, it determines that the current situation is an emergency situation It is possible to control the emergency remote situation information to be transmitted to the external remote device 70 registered in the database 40.

As another example, in the smart home service providing step S50, the control unit 30 converts the emergency situation voice response signal received from the microphone array 220 into the emergency situation voice response statement, It is possible to determine that the current situation is an emergency and to control the external remote device 70 registered in the database 40 to transmit the emergency situation information.

In the above two examples, in the smart home service providing step S50, when it is determined that the current situation is an emergency, the control unit 30 operates the image capturing unit 50 to control the image information of the object to be captured , The image information of the object may be included in the emergency situation information and transmitted to the external remote device 70.

8 is a diagram for explaining an operation example in a case where object determination information indicates an intruder according to an embodiment of the present invention.

8, when the object judgment information judged by the control unit 30 indicates the intruder in the voice query step S20, the control unit 30 controls the control unit 30 in the voice query step S20 Generates an invasion context voice query message with reference to the stored reference information, and transmits the invasion context voice query message to the speaker array 210, and the speaker array 210 converts the incoming context query message into an emergency context query signal.

If the control unit 30 does not receive the voice response signal corresponding to the intrusion context voice query signal from the microphone array 220 in the smart home service providing step S50, To the external remote device 70 that is registered in the external remote device 70.

In the smart home service providing step S50, when the current situation is determined as the intrusion state, the control unit 30 controls the image capturing unit 50 to capture the image information of the object, May be included in the emergency situation information to be transmitted to the external remote device 70.

As described above, according to the present invention, a smart home service based on object location and voice conversation, which can improve the accuracy of indoor positioning accuracy and voice processing by combining IR-UWB radar technology and sound quality improvement technique using beam forming There is an effect that the device is provided.

Also, by providing home control and security service based on object location and voice conversation in smart home, the quality of smart home service is improved.

Also, by using the detected user location information using the radar, it is possible to effectively provide a smart home service while greatly reducing the possibility of invasion of privacy.

1: object
10: motion data obtaining unit
20: Voice Conversation Support
30:
40: Database
50:
60: Internal Appliances
70: External remote device
110: first IR-UWB radar
120: second IR-UWB radar
210: speaker array
220: microphone array
S10: Motion data acquisition step
S20: voice query step
S30: Voice response reception step
S40: Steps to improve sound quality
S50: Step of providing smart home service
S410: Sound source position setting step
S420: Direction angle / distance calculation step
S430: Signal-to-noise ratio enhancement step

Claims (9)

A motion data acquisition step of acquiring motion data of the motion data acquisition unit;
The control unit obtains the positional information of the object from the motion data of the object received from the motion data obtaining unit, obtains the motion information of the object by arranging the positional information of the object in the time zone, And generates a voice query sentence corresponding to the object determination information and transmits the generated voice query statement to a speaker array provided in the voice chat support unit. The speaker array transmits the voice query sentence to a voice query signal And outputting the converted voice data;
A voice response reception step of the control unit receiving a voice response signal from the object through the microphone array provided in the voice conversation support unit;
A sound quality improving step of improving the sound quality of a voice response signal received from the object through the microphone array; And
Wherein the control unit controls the smart home service to provide a smart home service corresponding to the voice response sentence generated from the voice response signal having improved sound quality through the sound quality improving step,
The sound quality improving step
Wherein the control unit sets a position of the object as a virtual position of the sound source;
A direction angle / distance calculation step of the unit calculating unit orientations for directing the unit microphones constituting the microphone array to the sound source and unit distances between the unit microphones and the sound source; And
And a signal-to-noise ratio improvement step of improving the signal-to-noise ratio of the voice response signal by aligning the phases of the unit voice response signals received by the unit microphones using the unit direction angles and the unit distances, A method of smart home service based on location and voice conversation. The method according to claim 1,
Wherein the motion data obtaining unit includes a plurality of Impulse Radio-Ultra Wide Band (IR-UWB) radars installed at distant points from each other. delete delete The method according to claim 1,
If the object determination information indicates a normal user in an ordinary situation,
The control unit generates a normal contextual voice query corresponding to the position of the object in time zone with reference to the reference information stored in the database and transmits the generated contextual voice query to the speaker array, Converts the query into a normal situation voice query signal and outputs it,
In the smart home service providing step, the control unit converts the normal situation voice response signal received from the microphone array into a normal situation voice response sentence, refers to the reference information stored in the database, A smart home service based on the object location and voice conversation. The method according to claim 1,
If the object determination information indicates a critical user in an emergency situation,
In the voice query step, the controller generates an emergency voice query message by referring to the reference information stored in the database, and transmits the emergency voice query message to the speaker array. The speaker array transmits the emergency voice query message as an emergency voice query signal And then,
In the smart home service providing step, when the control unit does not receive the voice response signal corresponding to the emergency voice query signal from the microphone array, it determines that the current situation is an emergency, Wherein the control unit controls the emergency situation information to be transmitted to the smart home service based on the object location and voice conversation. The method according to claim 1,
If the object determination information indicates a critical user in an emergency situation,
In the voice query step, the controller generates an emergency voice query message by referring to the reference information stored in the database, and transmits the emergency voice query message to the speaker array. The speaker array transmits the emergency voice query message as an emergency voice query signal And then,
In the smart home service providing step, when the control unit converts the emergency situation voice response signal received from the microphone array into the emergency situation voice response statement and the emergency situation voice response statement includes the content indicating the emergency situation Wherein the control unit determines that the current situation is an emergency and controls the emergency remote situation information to be transmitted to an external remote device registered in the database, based on the object location and voice conversation. The method according to claim 1,
When the object judgment information indicates an intruder,
In the voice query step, the controller refers to the reference information stored in the database to generate an intrusion context voice query message and transmits the intrusion context voice query message to the speaker array. The speaker array transmits the intrusion context voice query message as an emergency context query signal And then,
In the smart home service providing step, when the control unit does not receive a voice response signal corresponding to the intrusion context voice query signal from the microphone array, it determines that the current situation is an intrusion situation, Wherein the control unit controls the invitation situation information to be transmitted to the smart home service based on the object location and voice conversation. delete

Images