KR102645469B1 - Sound Device for Entry Recognition - Google Patents

Abstract

According to the sound device for entrance recognition of the present invention, a sound module including a sound input unit for receiving a sound signal and a sound output unit for outputting a sound signal, a communication module including at least one communication unit connected to a communication network, and a camera at a short distance. In a sound device comprising a short-range module that processes short-range wireless communication with the device and a camera module that acquires image information of a subject, sound data corresponding to a sound signal in the audible frequency band input through the sound input unit is designated. A sound processing unit that checks sound data buffered in a buffer area or buffered in a designated buffer area, and reads the buffered sound data to recognize at least one entrance sound designated for entry/exit recognition among non-verbal sounds or noises that may occur in a designated space. an entrance sound recognition unit that recognizes a specific entrance sound corresponding to at least one entrance sound frequency band and at least one entrance sound pattern, and a designated S (S ≥ 1) number when the entrance sound is recognized. It is provided with a control module including a service control unit that activates at least s (1≤s≤S) service functions related to the entrance sound among the service functions, wherein the control module reads the buffered sound data and selects the user's language. A utterance content recognition unit that recognizes utterances and processes them into natural language to recognize utterance content information corresponding to the user's utterance content, and the user's utterance content is recognized while the entry/exit sound is being recognized, or a certain period of time after recognition of the entry/exit sound ends. a recognition determination unit that checks whether the content of the user's utterance is recognized within a certain period of time; An intelligent processing unit that processes and interprets in relation to service functions, M entry/exit sound frequency band information corresponding to M(M≥1) entry/exit sound frequency bands, and N corresponding to N(N≥1) entry/exit sound patterns. It further includes an information storage unit that stores the dog entrance sound pattern information in a designated storage area, wherein the service control unit communicates with the camera device through a short-range network communication unit, which is the short-range module, or the recognized entrance sound through the cabera module. It includes a function to activate a video service function that takes pictures of the sound source that generated the sound, and a function to activate a specific service function that is related to the entrance sound and corresponds to the content of the recognized user's speech.

Description

Sound Device for Entry Recognition}

The present invention provides a sound module including a sound input unit that receives a sound signal and a sound output unit that outputs a sound signal, a communication module that includes at least one communication unit connected to a communication network, a short-range camera device, and a short-range wireless communication device. In a sound device including a module and a camera module for acquiring image information of a subject, sound data corresponding to a sound signal in the audible frequency band input through the sound input unit is buffered in a designated buffer area or stored in a designated buffer area. A sound processing unit that checks buffered sound data, and at least one entry sound frequency designated to read the buffered sound data and recognize at least one entry sound designated for entry recognition among non-verbal sounds or noises that may occur in a designated space. An entrance sound recognition unit that recognizes a specific entrance sound corresponding to a band and at least one entrance sound pattern, and at least one of the S (S ≥ 1) service functions specified when the entrance sound is recognized, related to the entrance sound. It is provided with a control module including a service control unit that activates s (1≤s≤S) service functions, wherein the control module reads the buffered sound data, recognizes the user's language utterance, and processes the user's language. A utterance content recognition unit that recognizes utterance content information corresponding to the utterance content, and determines whether the user's utterance content is recognized while the entry/exit sound is being recognized or whether the user's utterance content is recognized within a certain time after recognition of the entry/exit sound ends. A recognition determination unit that confirms, and if the content of the user's speech is recognized within a certain period of time during or after the entrance sound is recognized, processing to interpret the recognized content of the user's speech by associating it with s service functions related to the entrance sound an intelligent processing unit that stores M entry/exit sound frequency band information corresponding to M(M≥1) entry/exit sound frequency bands and N entry/exit sound pattern information corresponding to N(N≥1) entry/exit sound patterns in a designated storage area. It further includes an information storage unit for storing information, wherein the service control unit communicates with the camera device through a short-range network communication unit, which is the short-distance module, or captures an image of a sound source that generated the recognized entrance sound through the cabera module. It relates to a sound device for entry/exit recognition that includes a function for activating a service function and a function for activating a specific service function that is related to the entry sound and simultaneously corresponds to the content of the recognized user's speech.

Recently, a stationary sound device that recognizes the user's voice and provides a service corresponding to the recognized voice has been sold. For example, stationary sound devices such as Amazon's 'Echo', SK Telecom's 'NUGU', and KT's 'Giga Genie' are being sold.

However, the conventional stationary sound device combines sound source service and voice recognition technology, and operates mainly as a service that simply provides or reproduces the sound source desired by the user through voice recognition, and has problems that make it difficult to use in other fields.

The purpose of the present invention to solve the above problems is to provide a sound module including a sound input unit that receives a sound signal and a sound output unit that outputs a sound signal, and a communication module that includes at least one communication unit connected to a communication network and a short distance communication module. In the sound device including a camera device, a short-range module that processes short-range wireless communication, and a camera module that acquires image information of a subject, sound data corresponding to a sound signal in the audible frequency band input through the sound input unit. A sound processing unit that buffers sound data in a designated buffer area or checks sound data buffered in a designated buffer area, and reads the buffered sound data to detect at least one entrance sound designated for entry/exit recognition among non-verbal sounds or noises that may occur in the designated space. An entrance sound recognition unit that recognizes a specific entrance sound corresponding to at least one entrance sound frequency band and at least one entrance sound pattern designated to recognize the entrance sound, and a designated S (S ≥ 1) when the entrance sound is recognized. ) of the service functions, and a control module including a service control unit that activates at least s (1≤s≤S) service functions related to the entry/exit sound, wherein the control module reads the buffered sound data and A utterance content recognition unit that recognizes the language utterance and processes it into natural language to recognize utterance content information corresponding to the user's utterance content, and the user's utterance content is recognized while the entry/exit sound is being recognized or after the recognition of the entry/exit sound is completed. a recognition determination unit that checks whether the content of the user's utterance is recognized within a certain time; and, if the content of the user's utterance is recognized within a certain time while or after the entry sound is recognized, the content of the recognized user's utterance is combined with the entry and exit sound. An intelligent processing unit that processes related s service functions to be interpreted, M entry/exit sound frequency band information corresponding to M(M≥1) entry/exit sound frequency bands, and N(N≥1) entry/exit sound patterns. It further includes an information storage unit for storing N entry/exit sound pattern information in a designated storage area, wherein the service control unit communicates with the camera device through a local area network communication unit, which is the local area module, or communicates with the recognized unit through the cabera module. For entrance recognition, including a function to activate a video service function that takes pictures of the sound source that generated the entrance sound, and a function to activate a specific service function that is related to the entrance sound and corresponds to the content of the recognized user's speech. Providing a sound device.

The sound device for entry/exit recognition according to the present invention includes a sound module including a sound input unit for receiving a sound signal and a sound output unit for outputting a sound signal, a communication module including at least one communication unit connected to a communication network, and a camera at a short distance. In a sound device comprising a short-range module that processes short-range wireless communication with the device and a camera module that acquires image information of a subject, sound data corresponding to a sound signal in the audible frequency band input through the sound input unit is designated. A sound processing unit that checks sound data buffered in a buffer area or buffered in a designated buffer area, and reads the buffered sound data to recognize at least one entrance sound designated for entry/exit recognition among non-verbal sounds or noises that may occur in a designated space. an entrance sound recognition unit that recognizes a specific entrance sound corresponding to at least one entrance sound frequency band and at least one entrance sound pattern, and a designated S (S ≥ 1) number when the entrance sound is recognized. It is provided with a control module including a service control unit that activates at least s (1≤s≤S) service functions related to the entrance sound among the service functions, wherein the control module reads the buffered sound data and selects the user's language. A utterance content recognition unit that recognizes utterances and processes them into natural language to recognize utterance content information corresponding to the user's utterance content, and the user's utterance content is recognized while the entry/exit sound is being recognized, or a certain period of time after recognition of the entry/exit sound ends. a recognition determination unit that checks whether the content of the user's utterance is recognized within a certain period of time; An intelligent processing unit that processes and interprets in relation to service functions, M entry/exit sound frequency band information corresponding to M(M≥1) entry/exit sound frequency bands, and N corresponding to N(N≥1) entry/exit sound patterns. It further includes an information storage unit that stores the dog entrance sound pattern information in a designated storage area, wherein the service control unit communicates with the camera device through a short-range network communication unit, which is the short-range module, or the recognized entrance sound through the cabera module. It is characterized by including a function for activating a video service function that takes pictures of the sound source that generated the sound, and a function for activating a specific service function that is related to the entrance sound and corresponds to the content of the recognized user's speech.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

According to the present invention, the entrance sound recognition unit reads the buffered sound data to correspond to at least one entrance sound frequency band among the M entrance sound frequency bands and to at least one entrance sound pattern among the N entrance sound patterns. It may include a function to recognize corresponding entrance and exit sounds.

delete

delete

According to the present invention, the s services are a server-linked service provided in conjunction with an operation server on a communication network through the communication module, a sound-based service provided through the sound module, and a combination of the sound module and the communication module. It may include at least one service among the complex services provided.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

According to the present invention, the complex service recognizes the content of the user's speech within a certain period of time during or after the entrance sound is recognized, and analyzes the content of the user's speech by associating it with the service function related to the entrance sound. It may include services provided through an operation server on a communication network.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

According to the present invention, when at least one entrance sound designated for entry/exit recognition among non-verbal sounds or noises that can occur in a designated space is not recognized, the system operates in a sleep mode that does not respond to the specific situation, and then operates in a sleep mode to recognize the entry/exit sound. When a specific entrance sound is recognized and recognized corresponding to at least one entrance sound frequency band specified and/or at least one entrance sound pattern designated to recognize the entrance sound, the device automatically wakes up from sleep mode and There is an advantage in providing a sound device that activates at least a service function related to the entrance sound or provides a service related to the entrance sound.

Figure 1 is a diagram showing the functional configuration of a sound device for entry/exit recognition according to an implementation method of the present invention.
Figure 2 is a diagram showing the functional configuration of a sound device and an operating server according to another implementation method of the present invention.
Figure 3 illustrates an embodiment of recognizing an entry/exit sound and activating a service related to the entry/exit sound according to an implementation method of the present invention.
Figure 4 shows an embodiment of optimizing (or customizing) the entry/exit sound frequency band for recognizing entry/exit sounds to the entry/exit sound of an entry/exit sound source according to an implementation method of the present invention.
Figure 5 shows an embodiment of optimizing (or customizing) an entry/exit sound pattern for recognizing entry/exit sounds to the entry/exit sound of an entry/exit sound source according to another implementation method of the present invention.
Figure 6 illustrates an embodiment of activating a service related to the entrance sound by linking and recognizing the entrance sound and the content of the speech according to another implementation method of the present invention.

Hereinafter, the operating principle of a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings and description. However, the drawings shown below and the description below are for preferred implementation methods among various methods for effectively explaining the characteristics of the present invention, and the present invention is not limited to the drawings and description below.

That is, the following examples correspond to examples of preferred union forms among the numerous embodiments of the present invention, and examples in which specific configurations (or steps) are omitted in the following examples, or specific configurations (or steps) are omitted. An embodiment of dividing the function implemented in a specific configuration (or step), or an embodiment of integrating the function implemented in two or more configurations (or steps) into one configuration (or step), a specific configuration (or step) It is clearly stated that embodiments that replace the operation sequence of all fall within the scope of the present invention, even if not separately mentioned in the following examples. Therefore, based on the examples below, it is clearly stated that various embodiments corresponding to subsets or complements can be divided retroactively to the filing date of the present invention.

Additionally, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. The terms described below are defined in consideration of the functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the overall content of the present invention.

As a result, the technical idea of the present invention is determined by the claims, and the following examples are a means to efficiently explain the advanced technical idea of the present invention to those skilled in the art to which the present invention pertains. It's just that.

Figure 1 is a diagram showing the functional configuration of a sound device 100 for entry/exit recognition according to an implementation method of the present invention.

In more detail, Figure 1 shows a sound module 105 including a sound input unit 105a that receives a sound signal and a sound output unit 105b that outputs a sound signal, and a communication unit including at least one communication unit connected to a communication network. A sound processing unit ( 140) and read the buffered sound data to correspond to at least one entrance sound frequency band designated to recognize at least one entrance sound designated for entrance recognition among non-verbal sounds or noises that may occur in a designated space, and/or An entrance sound recognition unit 145 that recognizes a specific entrance sound corresponding to at least one entrance sound pattern designated to recognize the entrance sound, and the above among the specified S (S ≥ 1) service functions when the entrance sound is recognized. This shows the functional configuration of a stationary sound device 100 including a control module 130 including a service control unit 150 that activates at least s (1 ≤ s ≤ S) service functions related to entrance and exit sounds. Those of ordinary skill in the art to which the invention pertains will refer to and/or modify Figure 1 to determine various implementation methods for the stationary sound device 100 (e.g., some components may be omitted, subdivided, or A combined implementation method) may be inferred, but the present invention includes all of the above inferred implementation methods, and its technical features are not limited to only the implementation method shown in Figure 1.

Referring to the embodiment of Figure 1, the stationary sound device 100 includes a sound module 105 including a sound input unit 105a that receives a sound signal and a sound output unit 105b that outputs a sound signal. , Equipped with a communication module 120 including at least one communication unit connected to a communication network, and reads sound data corresponding to a sound signal in the audible frequency band input through the sound input unit 105a to generate sound that can be generated in a designated space. Corresponds to at least one entrance sound frequency band designated to recognize at least one entrance sound designated for determining a specific situation among non-verbal sounds or noises, and/or corresponds to at least one entrance sound pattern designated to recognize the entrance sound. When a specific entrance sound is attempted to be recognized and recognized, it is provided with a control module 130 having one or more functional components for activating at least one designated service function related to the entrance sound or providing the designated service function, and operating It is provided with a power module 115 that supplies power. Meanwhile, depending on the implementation method, the stationary sound device 100 includes at least one of a short-distance camera device 185, a short-range module 125 that processes short-range wireless communication, and a camera module 110 that acquires image information of a subject. It may further be provided with modules, and in some cases, may further include an input unit (not shown) including a key button or touch pad that receives user operations, and/or an LED or display that outputs various operating states or information. An output unit (not shown) including a may be further provided. Meanwhile, in the embodiment of Figure 1, the name "module" is only a term for distinguishing and explaining the function and role of the stationary sound device 100 for convenience, and it is clear that it does not itself refer to any independent hardware configuration. I want to make this clear.

The sound module 105 is a general term for sound input/output related components including at least one sound input unit 105a that receives a sound signal and a sound output unit 105b that outputs a sound signal among the components of the stationary sound device 100. Preferably, the sound input unit 105a and the sound output unit 105b are capable of inputting or outputting sound signals in at least an audible frequency band, and the sound input unit 105a or the sound output unit 105b is connected to the control module 130. ) and an interface unit (not shown) that interfaces with.

The sound input unit 105a is a general term for a component that receives a sound signal, and includes one or more microphones that detect air vibration and receive a sound signal converted into an electrical signal, and encode the sound signal according to specified encoding rules. It is comprised of a program (and/or circuit) module that generates sound data. Preferably, the sound input unit 105a receives a sound signal in the audible frequency band (e.g., a sound signal including a voice signal uttered by a user) through the microphone, and inputs a sound signal in the audible frequency band according to a specified encoding rule. Sound data generated by encoding the sound signal can be provided to the control module 130.

According to the implementation method of the present invention, the sound input unit 105a includes one or more directional microphones to detect the direction in which the entrance sound occurs when an entrance sound is recognized in the space where the sound device 100 is mounted. may be implemented, and the present invention is not limited thereto.

According to the method of implementing the present invention, the entrance sound is the sound of unlocking the front door designated to determine the homeowner or family member's return home situation, the sound of pressing a designated number key to open the front door, and the sound designated to determine the homeowner or family member's return home state. It may include at least one noise among the sound of the front door opening or the sound of a doorbell designated to determine the visit status of a visitor or guest.

The sound output unit 105b is a general term for a component that outputs a sound signal. It checks one or more speakers that output a sound signal and sound data to be output through the speaker, and decodes the sound signal that can be output through the speaker ( Decoding) and at the same time, it includes a program (and/or circuit) module that controls the signal output strength and output quality of the sound signal. Preferably, the sound output unit 105b outputs a sound signal generated by decoding sound data requested for sound output through the control module 130 through the speaker, and the signal of the sound signal output is Controlling the output strength to a specified signal strength (e.g., amplifying or reducing the signal strength to a set signal strength to enable reception (or listening) with a signal strength greater than the specified signal reception strength at a distance greater than a specified reference distance, and/or even in the presence of a specified obstruction. Amplify or reduce to a set signal strength to enable reception (or listening) with a signal strength higher than the specified signal reception strength from a specified distance, and/or receive with a signal strength higher than the specified signal reception strength from a specified distance even in an environment where noise exceeds the specified decibel. (or amplify or reduce the signal strength set to enable listening) to output sound and/or control the output quality of the sound signal (e.g., speaker parameters such as Fs. Qts, Vas, etc.) to minimize sound distortion as much as possible. Sound output by blocking or suppressing (e.g., output by controlling the output quality of the sound to block/suppress sound distortion caused by speaker resonance, or, in the case of voice signals, so that audibility (or hearing sensitivity) can be maintained above a certain level) Controls and outputs the output quality of the sound so that the sound components corresponding to the segmental sounds in the voice signal are clearly distinguished, or blocks/suppresses distortion of the frequency or signal strength at which the bit string is encoded when outputting by encoding a bit string in the sound signal. You can control the output quality of the sound so that it can be output.

The communication module 120 is a general term for a component that processes the communication network connection of the stationary sound device 100, and is preferably connected to a designated server (or device) via a wireless network (e.g., mobile communication network, portable Internet, etc.) via a base station. ) and a wireless network communication unit (120a) that communicates by connecting a communication channel with a designated server (or device) via a local area network (e.g., wireless LAN, etc.) via a wireless AP or repeater. It includes at least one communication unit among the local area network communication units 120b, and further includes an interface unit (not shown) that interfaces the communication unit with the control module 130.

The wireless network communication unit 120a is a general term for a communication component that connects the stationary sound device 100 to a wide area wireless network (e.g., mobile communication network, etc.) via a base station, and is an antenna that transmits and receives radio frequency signals in a designated frequency band. , It is composed of at least one RF module, a baseband module, and a signal processing module, and is connected to the control unit 130a to transmit the calculation results corresponding to wireless communication among various calculation results of the control unit 130a through a wireless network. Alternatively, data is received through a wireless network and transmitted to the control unit 130a, and the wireless communication connection, registration, communication, and handoff procedures are performed.

According to the implementation method of the present invention, when the wireless network connected through the wireless network communication unit 120a is a mobile communication network, the wireless network communication unit 120a connects to the mobile communication network and registers the location according to at least one mobile communication standard. , includes a mobile communication configuration that performs at least one of call processing, call connection, data communication, and handoff. Meanwhile, according to the intention of those skilled in the art, the wireless network communication unit 120a connects to a wide area wireless network other than a mobile communication network (e.g., mobile Internet according to IEEE 802.16 related standards, etc.), and performs at least one of location registration, data communication, and handoff. It may include a portable Internet communication configuration, and it is clearly stated that the present invention is not limited to the wireless communication configuration provided by the wireless network communication unit 120a. That is, the wireless network communication unit 120a provides packet communication-based wireless data communication to the stationary sound device 100 via a cell-based base station on the wireless network, regardless of the frequency band of the wireless section or the type or protocol of the communication network. This is the general term for the components that connect to the data network.

The local area network communication unit 120b uses a radio frequency signal as a communication medium within a certain distance (e.g., within 10 m, or within 30 m, etc.) to perform short-range wireless communication with a network connection device (e.g., wireless AP, etc.) provided in the short distance. A general term for communication resources that connect the stationary sound device 100 to a communication network connected to the network connection device through a short-range wireless network that connects sessions, preferably Wi-Fi-based short-range network communication, Bluetooth short-range network communication, or ZigBee-based short-range communication. A short-distance wireless communication session is connected between a network connection device provided in a short distance and the stationary sound device 100 through at least one method of network communication, UWB-based local area network communication, and public wireless communication, and based on this, the stationary sound device ( 100) can be connected to the communication network connected to the network connection device.

According to the implementation method of the present invention, when the short-range network communication unit 120b is implemented as a short-range wireless network that connects a short-range wireless communication session with a network connection device provided within a certain distance from the stationary sound device 100. This is a general term for components that connect the stationary sound device 100 to a data network that provides packet communication-based wireless data communication via a network connection device of the local wireless network. Hereinafter, for convenience, the network communication function of the stationary sound device 100 will be explained mainly on an embodiment using the wireless network communication unit 120a, but the present invention is not limited thereto, and network communication using the local area network communication unit 120b is not limited thereto. We would like to make it clear that this function is also provided.

Meanwhile, according to one example of the method of implementing the present invention, the local area network communication unit 120b communicates with the camera device 185 provided in a short distance of the sound device 100 via the local area wireless network and the local area network-based wireless network. Communication can be handled. When the short-range network communication unit 120b processes wireless communication based on a short-range wireless network with a short-range camera device 185, the stationary sound device 100 and the camera device 185 perform wireless communication based on a short-range wireless network. Based on this, the key value for encrypted communication can be stored or the key value for encrypted communication can be exchanged at the time of connecting wireless communication based on a short-distance wireless network, whereby the stationary sound device 100 and the camera device 185 Can perform encrypted communication using the key value.

The short-range module 125 is a general term for a component that processes short-range wireless communication of the stationary sound device 100, and preferably includes a short-range wireless unit 125a that connects short-range wireless communication of a designated standard. It is provided with an interface unit (not shown) that interfaces (125a) and the control module 130. For example, the short-range wireless unit 125a can process Bluetooth-based short-range wireless communication according to the Bluetooth standard. However, it is clearly stated that the standard for short-range wireless communication of the short-range wireless communication unit 125a is not limited to the Bluetooth standard and may include all short-range wireless communication standards including Wi-Fi, ZigBee, or UWB. Meanwhile, depending on the implementation method, the short-range wireless unit 125a may be the short-range network communication unit 120b.

The short-range wireless unit 125a is a general term for a component that processes short-range wireless communication of a designated standard, and may preferably include a Bluetooth component that processes Bluetooth-based short-range wireless communication according to the Bluetooth standard. However, the short-range wireless communication-related configuration of the short-range wireless unit 125a is not limited to Bluetooth-based short-range wireless communication according to the Bluetooth standard, and short-range wireless communication of various standards other than Bluetooth can be used depending on the intention of those skilled in the art (e.g. , Wi-Fi-based short-range wireless communication according to the Wi-Fi standard, ZigBee-based short-range wireless communication according to the ZigBee standard, and UWB-based short-distance wireless communication according to the UWB standard, as well as short-range wireless communication of all existing or future short-range wireless communication-related standards. available) is clearly stated. Hereinafter, for convenience, the characteristics of the present invention will be described focusing on an embodiment in which the short-range wireless unit 125a processes Bluetooth-based short-range wireless communication, but it is clearly stated that the present invention is not limited thereto.

According to the implementation method of the present invention, the short-range module 125 (or short-range wireless unit 125a) processes short-distance wireless communication of a specified standard with the camera device 185 provided in close range of the sound device 100. You can. When the short-range module 125 (or short-range wireless unit 125a) processes short-range wireless communication with the short-range camera device 185, the stationary sound device 100 and the camera device 185 perform short-range wireless communication. Based on this, the key value for encrypted communication can be stored or the key value for encrypted communication can be exchanged at the time of connecting short-distance wireless communication. As a result, the stationary sound device 100 and the camera device 185 can store the key value for encrypted communication. Encrypted communication can be performed using .

According to the implementation method of the present invention, when the stationary sound device 100 and the app of the wireless terminal used by the user are mapped (e.g., when the designated app is installed on the user's wireless terminal and executed for the first time (or executed before mapping registration), the user Device identification information identifying the stationary sound device 100 in the app of the wireless terminal and identification information of the user wireless terminal (or unique identification information for the app of the user wireless terminal) are registered with the designated operation server 200 to register the stationary sound device. (100) and the app of the user's wireless terminal are matched and registered), the short-range wireless unit 125a connects the app of the matched user's wireless terminal through short-range wireless communication to provide various identification information or information designated to process a specific service. Processing information can be exchanged. For example, information received from the designated operating server 200 as an app of the user's wireless terminal, or input, extracted, or generated through the app of the user's wireless terminal, is transmitted to the mounted sound device 100 through the short-range wireless communication. It is transmitted to the control module 130 and processed (e.g., sound output or transmitted to the operation server 200) and/or received from the designated operation server 200 to the mounted sound device 100, or the mounted sound device ( Information recognized, extracted, or generated in 100) may be transmitted to the app of the user's wireless terminal through the short-distance wireless communication and processed (e.g., output on a screen or transmitted to the operation server 200).

The camera module 110 is a general term for a component that processes camera capture of the stationary sound device 100, and is preferably a camera unit that acquires image information (e.g., image) or image information (e.g., video) of a photographed subject. It is provided with (110a), and is provided with an interface unit (not shown) that interfaces the camera unit 110a and the control module 130. Meanwhile, if it is possible to link with a short-range camera device 185 through the short-range module 125 (or short-range wireless unit 125a) and/or a short-range network wireless unit, the camera module 110 (or camera unit 110a) It can be omitted, and the present invention is not limited thereby.

The camera unit 110a includes one or more lens units that receive optical signals reflected from a subject, an image sensor unit that detects optical signals input through the lens unit and converts them into electrical image signals, and a space between the lens unit and the image sensor unit. It includes an actuator that implements an autofocus function or an image stabilization function, and depending on the implementation method, one or more filter units (e.g., infrared filter, etc.) may be further provided between the lens unit and the image sensor unit. You can. Meanwhile, the camera unit 110a uses an image signal processor (ISP) to process the image information or image information according to a designated procedure to improve the quality of the image information or image information generated through the image sensor unit. More can be provided.

Meanwhile, when the sound device 100 is equipped with the camera module 110, the camera module 110 has a driving unit for changing the shooting direction of the camera unit 110a, and the control module 130 ( Alternatively, the camera module 110 controls the driving unit to change the shooting direction of the camera unit 110a and reads the image information or video information generated through the image sensor unit of the camera unit 110a to produce the sound device 100. It is possible to recognize a sound source (for example, a person or object that generates an entry/exit sound) in the same space as the space in which the device is mounted, and operate the drive unit to capture the sound source through the camera unit 110a. On the other hand, when the sound input unit 105a provided in the sound module 105 includes one or more directional microphones, the control module 130 (or camera module 110) is connected to the sound input unit (105a) of the sound module 105. The direction of the sound of entrance and exit detected through one or more directional microphones provided in 105a) can be confirmed, and the driving unit can be manipulated to photograph the direction of the sound of entrance and exit confirmed through the camera unit 110a.

The control module 130 is a general term for components that control various operations of the stationary sound device 100. Preferably, the operation of generating image data through the camera module 110 and the sensor module through the short-range module 125 A control unit 130a that controls at least one operation or a combination of two or more of the operations of collecting sensing data from the device and communicating with a designated server (e.g., operation server 200) through the communication module 120. It is equipped with a memory unit that stores various information or program codes necessary for the operation of the control unit 130a.

The control unit 130a is a general term for a component that controls the operation of the stationary sound device 100, and is configured to include at least one processor and an execution memory and/or various chips or control circuits. It is connected to each component provided in the stationary sound device 100 through a designated interface (eg, bus (BUS), etc.). According to the present invention, the control unit 130a loads at least one program code provided in the stationary sound device 100 through the processor into the execution memory and operates the result, and operates the result through the bus. It is transmitted to the component part to control the operation of the stationary sound device 100. The functional configuration of the present invention may be implemented in the form of program code and/or in the form of a chip or control circuit. Hereinafter, for convenience, the functional configuration of the present invention may be implemented in the form of program code and stored in a memory unit and then loaded and operated in the form of a program and/or chip. However, the functional components of the present invention implemented in the form of a control circuit will be illustrated and explained within the control unit 130a.

The memory unit 130b is a general term for non-volatile memory corresponding to the storage resources of the stationary sound device 100, and includes at least one program code executed through the control unit 130a and at least one program code used. Store and maintain the dataset. The memory unit basically includes a system program code and a system data set corresponding to the operating system of the stationary sound device 100, a communication program code and a communication data set for processing a wireless communication connection of the stationary sound device 100, and at least one Application program codes and application data sets are stored, and program codes and data sets corresponding to functional configurations for implementing the present invention may also be stored in the memory unit 130b.

The power module 115 is a general term for a component that supplies/applies power necessary for the operation of the stationary sound device 100, and is preferably an external power source (e.g., AC power, DC power rectified AC power, USB power, etc.) It is provided with a power supply unit (not shown) that receives and applies the power, and when the voltage value or current value of the external power source is different from the voltage value or current value required for operation of the stationary sound device 100, the external power supply is supplied to a specified voltage. It is further provided with a power conversion unit (not shown) that converts the value or current value into operating power. Meanwhile, when the external power is supplied through the power supply unit or the external power is converted into operating power through the power conversion unit, the power module 115 charges the external power or operating power and the stationary sound device 100 ) may further include a power charging unit (not shown) that applies the charged charging power when external power is not supplied.

According to the first embodiment of the present invention, the sound device 100 is a dedicated sound produced for an entrance sound recognition-based service that recognizes a designated entrance sound to recognize entrance to a designated space and provides a service related to the entrance sound. It may be device 100.

According to the second embodiment of the present invention, the sound device 100 provides a voice recognition operation mode that provides a well-known voice recognition-based service, and provides a specific setting through a predetermined input unit provided in the sound device 100. It may be a mode switching sound device 100 that switches from the voice recognition operation mode to the entry/exit sound recognition operation mode for the entry/exit sound recognition-based service according to the present invention based on specific voice recognition through input or voice recognition.

According to the third embodiment of the present invention, the sound device 100 is a complex sound device designed to provide a well-known voice recognition-based service and at the same time recognize entry and exit sounds according to the present invention to provide multiple entry and exit sound recognition-based services. It may be (100).

Referring to Figure 1, the control module 130 selects recognition target information to be recognized to initiate or process a specific service among the utterance content information to be recognized through the utterance recognition unit of the control module 130 and acquired through natural language processing. It is provided with an information storage unit 135 that confirms and stores the information in a designated storage area of the memory unit.

When recognition of the content of a user's speech is necessary to determine the initiation or processing of a specific service, the information storage unit 135 provides recognition target information containing the content of the user's speech to be recognized in order to initiate or process the specific service. You can check and store service provision information and mapping for initiation or processing of the specific service.

According to the implementation method of the present invention, the recognition target information to be mapped with the service provision information is designated including the pre-designated utterance content, and/or the user through the sound output unit 105b of the stationary sound device 100. After requesting the content of the utterance to be used as the recognition target information, it can be input through the sound input unit 105a, obtained through natural language processing, and registered/updated. Meanwhile, depending on the implementation method, the recognition target information may be registered/updated through an app of the user's wireless terminal mapped to the stationary sound device 100, but the present invention is not limited thereto.

Referring to Figure 1, the control module 130 buffers sound data corresponding to the sound signal in the audible frequency band input through the sound input unit 105a in a designated buffer area or buffers sound data buffered in the designated buffer area. A sound processing unit 140 that checks, and at least one entry sound frequency band designated to read the buffered sound data and recognize at least one entry sound designated for entry recognition among non-verbal sounds or noises that may occur in the designated space. An entrance sound recognition unit 145 that recognizes a specific entrance sound corresponding to and/or at least one entrance sound pattern designated to recognize the entrance sound, and a designated S (S) when recognizing the entrance sound Among ≥1) service functions, at least s (1≤s≤S) service functions related to entry/exit sounds are activated and/or when entry/exit sounds or user's speech contents are not recognized for more than a specified period of time, the sound device (100) ) and a service control unit 150 that sets the operation mode to sleep mode, and further includes a service provider 180 that provides a service corresponding to the activated service function. Meanwhile, according to another implementation method of the present invention, at least one component of the entry/exit sound recognition unit 145, the service control unit 150, and the service provider 180 is an operating server on a communication network interoperating with the sound device 100. It can be implemented through (200), and if the entry/exit sound recognition unit 145 is implemented in the operation server 200, the sound processing unit 140 transmits the sound data buffered in the designated buffer area to the operation server 200. ), or sound data determined to correspond to entrance and exit sounds from among the buffered sound data may be selectively extracted and provided to the operation server 200. However, for convenience, Figure 1 will explain the features of the present invention focusing on an embodiment in which the entrance sound recognition unit 145, service control unit 150, and service provider 180 are implemented through the sound device 100. . On the other hand, when the entrance sound recognition unit 145, the service control unit 150, and the service provider 180 are implemented through the sound device 100, the entrance sound recognition unit 145, the service control unit 150, and the service provider Study 180 may be implemented in the form of at least one or a combination of two of program code form and chip or control circuit form, and includes the entrance sound recognition unit 145, service control unit 150, and service provider unit 180. The present invention is not limited by the implementation method or the implemented module.

After operating power is applied to the sound device 100 through the power module 115, the sound processing unit 140 generates sound data corresponding to the sound signal in the audible frequency band input through the sound input unit 105a. Continuing (or repeating) the process of buffering in a designated buffer area and/or checking the sound data buffered in the designated buffer area, the entry/exit sound recognition unit 145 reads the buffered sound data and Recognize a specific entry sound corresponding to at least one entry sound frequency band specified to recognize at least one entry sound designated for entry recognition and/or corresponding to at least one entry sound pattern specified to recognize the entry sound. Continue (or repeat) the process. Meanwhile, when the entry/exit sound recognition unit 145 is implemented in the operation server 200, the sound processing unit 140 provides sound data buffered in a designated buffer area to the operation server 200 or provides the buffered sound data to the operation server 200. Sound data determined to correspond to the sound of entering and exiting can be selectively extracted and provided to the operation server 200, but the present invention is not limited thereto.

The service control unit 150 checks whether the entrance sound is not recognized for more than a specified period of time after operating power is applied to the sound device 100 and/or effectively recognizes the entrance sound through the entrance sound recognition unit 145. After that, check whether the entrance sound is not recognized as valid for more than a specified period of time, and/or check whether the contents of the user's utterance are not recognized for more than a specified period of time after operating power is applied to the sound device 100, and/or After the content of the user's utterance is validly recognized according to a designated procedure (e.g., the content of the user's utterance is recognized within a certain period of time during or after the access sound is validly recognized through the access sound recognition unit 145), the user for a specified period of time or more You can check whether the content of the utterance is not recognized. If the entrance sound is not recognized for more than a specified period of time or the content of the user's speech is not recognized for more than a specified period of time, the service control unit 150 changes the operation mode of the sound device 100 (or control module 130). It includes the function of setting to sleep mode. Here, the sleep mode reads the buffered sound data and recognizes the entry/exit sound even if the user's speech content is recognized and wakes up from the sleep mode and/or detects pre-designated specific speech information (e.g., entry/exit sound). In addition, before wake-up from sleep mode by recognizing specific utterance information set to wake up the sound device 100, a service based on the recognized user's utterance content (e.g., a sound source or a signal sound) is recognized. This includes disabling the function that provides sound-based services (outputting sound-based services). Hereinafter, for convenience, the features of the present invention will be described focusing on an embodiment in which the device wakes up from the sleep mode by recognizing an entry/exit sound.

According to the implementation method of the present invention, the sound processing unit 140 buffers sound data corresponding to the sound signal in the audible frequency band input through the sound input unit 105a in a designated buffer area even in the sleep mode. It is desirable to continue (or repeat) the process of checking the sound data buffered in the designated buffer area, and the entry/exit sound recognition unit 145 also reads the buffered sound data in the sleep mode to determine non-verbal language that may occur in the designated space. A specific device corresponding to at least one entry/exit sound frequency band designated for recognizing at least one entry/exit sound among sounds or noises and/or at least one entry/exit sound pattern specified for recognizing the entry/exit sound. It is desirable to continue (or repeat) the process of recognizing entrance and exit sounds. Preferably, when the entrance sound is effectively recognized through the entrance sound recognition unit 145 in the sleep mode, the service control unit 150 wakes up and activates s service functions related to the entrance sound, The service provider 180 may perform a procedure for providing services corresponding to the activated s service functions.

Meanwhile, when two or more service functions related to entrance sounds are activated through the service control unit 150, the service provider 180 performs a procedure for providing services corresponding to the activated service functions in a designated order. And/or a procedure for providing at least one service according to a preset condition (eg, a condition for recognizing a user's utterance) among the activated service functions may be performed.

According to the method of implementing the present invention, the entry/exit sound recognition unit 145 corresponds to the entry/exit sound frequency band of the entry/exit sound generator and/or the unique entry/exit sound of the entry/exit sound generator corresponding to the entry/exit sound pattern of the entry/exit sound generator. It is desirable to recognize, and for this purpose, the control module 130 has a function of learning at least one entry/exit sound frequency band specialized for the entry/exit sound source and/or a function of learning at least one entry/exit sound pattern specific to the entry/exit sound source. It is desirable to have a.

Referring to Figure 1, the control module 130 stores M entry/exit sound frequency band information corresponding to M (M ≥ 1) entry/exit sound frequency bands in a designated storage area and/or stores N (N ≥ 1) entry/exit sound frequency band information in a designated storage area. ) is provided with an information storage unit 135 that stores N entrance sound pattern information corresponding to the entrance sound patterns in a designated storage area, and the entrance sound recognition unit 145 stores the entrance sound frequency band stored in the designated storage area. Entry and exit sounds can be recognized using information and/or entry and exit sound pattern information. Meanwhile, according to another implementation method of the present invention, at least one component of the information storage unit 135 and the entry/exit sound recognition unit 145 is implemented through an operation server 200 on a communication network interoperating with the sound device 100. It can be. However, for convenience, Figure 1 will explain the features of the present invention focusing on an embodiment in which the information storage unit 135 and the entry/exit sound recognition unit 145 are implemented through the sound device 100. Meanwhile, when the information storage unit 135 and the entry/exit sound recognition unit 145 are implemented through the sound device 100, the information storage unit 135 and the entry/exit sound recognition unit 145 are implemented in the form of program code and a chip or control. It may be implemented in at least one of the circuit forms or a combination of the two, and the present invention is not limited by the method of implementing the information storage unit 135 and the entry/exit sound recognition unit 145 or the module implemented.

The information storage unit 135 stores M entry/exit sounds corresponding to M (M≥1) entry/exit sound frequency bands in a designated storage area of the memory unit at the time of manufacturing the control module 130 (or sound device 100). Frequency band information can be stored. In the present invention, among the non-verbal sounds or noises that can occur in a designated space, the entry/exit sound designated for entry/exit recognition has various frequencies (or a wide range of frequencies where the frequency difference between the lowest frequency and the highest frequency is at least 100 Hz or more) depending on the type of sound or the sound source. It can have a range. Accordingly, the present invention divides the frequency band of entrance and exit sounds into M frequency bands (e.g., equal division or experimentally selective division based on a plurality of frequency bands in which entrance and exit sounds are concentrated), and the information storage unit 135 Can store the M entry/exit sound frequency band information corresponding to the divided M entry/exit sound frequency bands in a designated storage area.

The information storage unit 135 stores N entry/exit sound patterns corresponding to N (N≥1) entry/exit sound patterns in a designated storage area of the memory unit at the time of manufacturing the control module 130 (or sound device 100). Information can be stored in a designated storage area. The present invention first generates entry/exit sound pattern information through a qualitative and/or quantitative analysis process for each case of the entry/exit sound pattern confirmed through a sample experiment, and the information storage unit 135 stores the generated entry/exit sound pattern. Information can be stored in a designated storage area.

Meanwhile, according to the implementation method of the present invention, the sound device 100 (or control module 130) can recognize the entrance sound and then transmit the recognized entrance sound information to the designated operation server 200, and the operation The server 200 collects entry/exit sound information from each sound device 100 (or control module 130), and normalizes individual sound characteristics for each sound source from the collected entry/exit sounds. /Or, the entry/exit sound pattern information may be refined or diversified through a qualitative analysis and/or quantitative analysis process for each case, and the refined or diversified entry/exit sound pattern information may be transmitted to each sound device 100 (or control module). (130)). The information storage unit 135 receives refined or diversified entry/exit sound pattern information from the operation server 200 through the above process, and generates N entry/exit sound pattern information including the received entry/exit sound pattern information. It can be saved in a designated storage area.

According to the implementation method of the present invention, M entry/exit sound frequency band information corresponding to M entry/exit sound frequency bands is stored in the designated storage area, and/or N entry/exit sound frequency band information corresponding to N entry/exit sound patterns is stored in the designated storage area. When sound pattern information is stored, the entry/exit sound recognition unit 145 reads the buffered sound data and corresponds to at least one entry/exit sound frequency band among the stored M entry/exit sound frequency bands and/or the stored N entry/exit sound frequency bands. It is desirable to recognize an entry/exit sound corresponding to at least one entry/exit sound pattern among the entry/exit sound patterns.

Meanwhile, the M entry/exit sound frequency band information and/or N entry/exit sound pattern information stored in the designated storage area are not optimized (or customized) for the entry/exit sound of the entry/exit sound source, but rather are in the frequency band and/or sound of the statistical entry/exit sound. This may be information corresponding to a pattern. Therefore, in order to recognize at high speed the entrance sound of an entrance sound source existing in the space where the sound device 100 is installed and/or prevent (or minimize) recognition errors, the stored entrance sound frequency band information and/or entrance sound A process of optimizing (or customizing) the pattern information to the entry/exit sound of the entry/exit sound source is necessary.

Referring to Figure 1, the control module 130 uses the M entry/exit sound frequency band information to determine an entry/exit sound frequency corresponding to an entry/exit sound frequency distribution that has been validly recognized at least i (i≥2) times during a specified period of time. Generate distribution information and/or generate a distribution that uses the N entry and exit sound pattern information to generate entry and exit sound pattern distribution information corresponding to the entry and exit sound pattern distribution that has been validly recognized at least j (j≥2) times during a specified period of time. Unit 155 and statistical processing of the generated entry/exit sound frequency distribution information to optimize (or match) the entry/exit sound of the entry/exit sound source in the space where the sound device 100 is mounted (1≤m≤M). ) determine the m entry/exit sound frequency band information corresponding to the entry/exit sound frequency bands and/or statistically process the generated entry/exit sound pattern distribution information to identify the entry/exit sound source in the space where the sound device 100 is installed. It further includes an information determination unit 160 that determines n entry/exit sound pattern information corresponding to n (1≤n≤N) entry/exit sound patterns optimized (or customized) for entry/exit sounds.

Meanwhile, according to another implementation method of the present invention, at least one component of the distribution generating unit 155 and the information determining unit 160 may be implemented through the operation server 200 on a communication network interoperating with the sound device 100. You can. However, for convenience, Figure 1 will explain the features of the present invention focusing on an embodiment in which the distribution generating unit 155 and the information determining unit 160 are implemented through the sound device 100. Meanwhile, when the distribution generator 155 and the information determination unit 160 are implemented through the sound device 100, the distribution generator 155 and the information determination unit 160 are in the form of a program code and a chip or control circuit. It may be implemented in the form of at least one or a combination of the two, and the present invention is not limited by the method of implementing the distribution generating unit 155 and the information determining unit 160 or the module implemented.

When M entry and exit sound frequency band information corresponding to M entry and exit sound frequency bands is stored in the storage area designated through the information storage unit 135, the distribution generator 155 stores the M entry and exit sound frequency bands stored in the designated storage area. Whenever an entry/exit sound is recognized as valid using the sound frequency band information, the frequency band information of the recognized entry/exit sound can be stored in a designated storage area. The distribution generator 155 generates i entry/exit sound frequency bands that are validly recognized at least i (i ≥ 2) times during a specified period of time based on the entry/exit sound frequency bands that are validly recognized and stored through the M entry/exit sound frequency band information. can be confirmed, and entry/exit sound frequency distribution information corresponding to the distribution of the confirmed i entry/exit sound frequency bands can be generated.

Meanwhile, when entry/exit sound frequency distribution information corresponding to an entry/exit sound frequency distribution that has been validly recognized at least i times during a specified period of time is generated through the distribution generator 155, the information determination unit 160 determines the generated entry/exit sound frequency. The distribution information is processed statistically to optimize (or match) the entrance and exit sounds of the entry and exit sound sources in the space where the sound device 100 is installed to recognize the entry and exit sounds of the entry and exit sound sources at high speed or to prevent (or minimize) recognition errors. In order to do this, m entry/exit sound frequency band information corresponding to m (1≤m≤M) entry/exit sound frequency bands can be determined. Here, the m entry/exit sound frequency bands may be any one of the M entry/exit sound frequency bands, or may include a newly defined entry/exit sound frequency band within the entry/exit sound frequency band range. When the m entry/exit sound frequency band information is determined through the information determination unit 160, the information storage unit 135 sets the determined m entry/exit sound frequency bands as the entry/exit sound frequency band to be read first and stores the designated information. You can store new information in the area or update and store the entry/exit sound frequency band information stored in the storage area. Thereafter, the entry/exit sound recognition unit 145 may read the buffered sound data and first recognize an entry/exit sound corresponding to at least one entry/exit sound frequency band among the m new or newly stored entry/exit sound frequency bands.

Meanwhile, when N entry/exit sound pattern information corresponding to N entry/exit sound patterns is stored in the storage area designated through the information storage unit 135, the distribution generator 155 stores the N entry/exit sound patterns stored in the designated storage area. Whenever an entry/exit sound is validly recognized using pattern information, the Sophie pattern of the validly recognized entry/exit sound can be stored in a designated storage area. The distribution generator 155 identifies j entry/exit sound patterns that have been validly recognized at least j (j≥2) times during a specified period based on the N entry/exit sound pattern information and stored entry/exit sound pattern information. And, entry/exit sound pattern distribution information corresponding to the distribution of the j identified entry/exit sound patterns can be generated.

Meanwhile, when entry/exit sound pattern distribution information corresponding to an entry/exit sound pattern distribution that has been validly recognized at least j times during a specified period of time is generated through the distribution generator 155, the information determination unit 160 determines the generated entry/exit sound pattern. The distribution information is processed statistically to optimize (or match) the entrance and exit sounds of the entry and exit sound sources in the space where the sound device 100 is installed to recognize the entry and exit sounds of the entry and exit sound sources at high speed or to prevent (or minimize) recognition errors. In order to do this, n entry/exit sound pattern information corresponding to n (1≤n≤N) entry/exit sound patterns can be determined. Here, the n entry/exit sound patterns may be any one of the N entry/exit sound patterns, or may be a combination of two or more entry/exit sound patterns, or may include a new crying/spitting pattern. When n entry/exit sound patterns are determined through the information determination unit 160, the information storage unit 135 sets the determined n entry/exit sound patterns as entry/exit sound patterns to be read first and stores them in a designated storage area. The entry/exit sound frequency band information stored in the storage area can be updated and stored. Thereafter, the entry/exit sound recognition unit 145 may read the buffered sound data and first recognize an entry/exit sound corresponding to at least one entry/exit sound pattern among the n entry/exit sound patterns.

Referring to Figure 1, the control module 130 reads the buffered sound data, recognizes the user's language utterance, processes natural language, and recognizes utterance content information corresponding to the user's utterance content. 165) and a recognition determination unit 170 that checks whether the content of the user's utterance is recognized while the entry/exit sound is being recognized or whether the content of the user's utterance is recognized within a certain time after the recognition of the entry/exit sound ends, and the entry/exit sound If the content of the user's speech is recognized within a certain period of time during or after it is recognized, it may be provided with an intelligent processing unit 175 that processes the recognized content of the user's speech to be interpreted by relating it to s service functions related to entrance and exit sounds. there is. Meanwhile, according to another implementation method of the present invention, at least one component of the utterance content recognition unit 165, the recognition determination unit 170, and the intelligent processing unit 175 is an operating server on a communication network interoperating with the sound device 100. It can be implemented through (200), and if the speech content recognition unit 165 is implemented in the operation server 200, the sound processing unit 140 transmits the sound data buffered in the designated buffer area to the operation server 200. ), or sound data determined to correspond to the user's utterance from among the buffered sound data may be selectively extracted and provided to the operation server 200. However, for convenience, Figure 1 will explain the features of the present invention focusing on an embodiment in which the speech content recognition unit 165, recognition determination unit 170, and intelligent processing unit 175 are implemented through the sound device 100. . Meanwhile, when the utterance content recognition unit 165, the recognition determination unit 170, and the intelligent processing unit 175 are implemented through the sound device 100, the utterance content recognition unit 165, the recognition determination unit 170, and the intelligent processing unit 175 are implemented through the sound device 100. The processing unit 175 may be implemented in the form of at least one or a combination of a program code form, a chip, or a control circuit form, and includes the utterance content recognition unit 165, the recognition determination unit 170, and the intelligent processing unit 175. The present invention is not limited by the implementation method or the implemented module.

At the same time, the process of recognizing entrance and exit sounds is performed by reading the buffered sound data through the entrance sound recognition unit 145, and at the same time, the speech content recognition unit 165 reads the buffered sound data and recognizes the user's language utterance. It recognizes and performs natural language processing to recognize speech content information corresponding to the user's speech content. Meanwhile, when the speech content recognition unit 165 is implemented in the operation server 200, the sound processing unit 140 provides sound data buffered in a designated buffer area to the operation server 200 or provides the buffered sound data to the operation server 200. Sound data determined to correspond to the user's utterance (for example, including segmental sounds) may be selectively extracted and provided to the operation server 200, but the present invention is not limited thereto.

According to the implementation method of the present invention, the speech content recognition unit 165 analyzes the buffered sound data through a designated voice recognition algorithm to recognize the voice uttered by the user and converts it into text, as well as morphological analysis, semantic analysis, and conversation. During analysis, utterance content information that recognizes the utterances uttered by the user can be generated through one or more natural language processing procedures.

Meanwhile, according to the implementation method of the present invention, the utterance content recognition unit 165 recognizes the entrance sound by the entrance sound recognition unit 145 and changes the operation mode of the service device from sleep mode through the service control unit 150. It is desirable to perform a function to recognize the contents of the user's speech when woken up. If the operation mode of the sound device 100 is set to sleep mode by the service control unit 150, the utterance content recognition unit 165 may not perform the function of recognizing the user's utterance content, and even if Even if the content of the user's speech is recognized, it can be processed so that it cannot be used for the service.

Meanwhile, when the user's speech content is recognized through the speech content recognition unit 165, the recognition determination unit 170 recognizes the user's speech content while the entrance sound is recognized through the entrance sound recognition unit 145. It is checked whether the user's utterance is recognized or whether the user's utterance is recognized within a certain period of time after recognition of the entry/exit sound is completed through the entry/exit sound recognition unit 145.

If it is confirmed through the recognition determination unit 170 that the user's utterance content has been recognized within a certain period of time during or after the entry/exit sound is recognized, the intelligent processing unit 175 determines that the utterance content recognition unit 165 The content of the user's utterance recognized through is processed to be interpreted by relating it to s service functions related to the entrance sound. As a result, a specific service function corresponding to the utterance content among the s service functions related to the entrance sound can be determined. In this case, the service control unit 150 may activate a specific service function related to the entrance sound and corresponding to the content of the recognized user's speech, and the service provider 180 may activate the specific service function. The provided procedures can be performed according to designated procedures.

If the content of the user's utterance is not recognized within a certain period of time during or after the entrance sound is recognized through the recognition determination unit 170, the intelligent processing unit 175 switches to sleep mode through the service control unit 150. A process of switching to and/or waking up from the sleep mode by the entry/exit sound may be performed.

Meanwhile, according to the implementation method of the present invention, s services related to entrance sounds that can be activated by the service control unit 150 and/or provided through the service provider 180 are provided on the communication network through the communication module 120. A server-linked service provided in conjunction with the operation server 200, a sound-based service provided through the sound module 105, and a complex service provided by using the sound module 105 and the communication module 120 in combination. It may include at least one service.

Meanwhile, among the s services related to the entrance sound, the server interlocking service is a calling service using wireless terminal information, a situation notification service using wireless terminal information, a calling service using an app installed on the wireless terminal, and a calling service using an app installed on the wireless terminal. At least one service among a situation notification service, a video service that provides video information captured through at least one camera to an app installed on a wireless terminal, and a video service that provides video information captured through at least one camera to a designated IPTV device may include.

Meanwhile, among the s services related to the entrance sound, the sound-based service may include a noise output service that outputs a designated service sound at a designated signal output intensity through the sound output unit 105b of the sound module 105, The signal output strength of the sound-based service preferably includes a signal strength that is inversely proportional to the signal input strength of the entrance sound.

Meanwhile, among the s services related to the entrance sound, the composite service recognizes the user's speech content within a certain period of time during or after the entrance sound is recognized and interprets the user's speech content by relating it to the service function related to the entrance sound. Based on the results, services provided through the operation server 200 on the communication network may be included.

Meanwhile, according to the first camera embodiment of the present invention, the sound device 100 is equipped with a short-range camera device 185 and a short-range module 125 for processing short-range wireless communication, and through the short-range module 125, When processing short-distance wireless communication with a short-distance camera device 185, the service control unit 150 recognizes an entrance sound through the entrance sound recognition unit 145 and detects the camera through the short-distance module 125. By communicating with the device 185, it is possible to activate s service functions including an image service function for photographing the sound source that generated the recognized entrance sound through the camera device 185, and the service provider 180 ) communicates with the camera device 185 through the short-distance module 125, receives image information captured through the camera device 185, and provides the image to an app installed on the wireless terminal through the operation server 200. At least one video service may be provided among the services and/or video services provided by a designated IPTV device. Meanwhile, if the camera device 185 is capable of providing the video service, the service provider 180 can process the video service function of the camera device 185 to operate through the short-range module 125.

Alternatively, according to the second camera embodiment of the present invention, wireless communication based on a short-range wireless network is performed with a short-range camera device 185 through the short-range network communication unit 120b of the communication module 120 provided in the sound device 100. In the case of processing, when the service control unit 150 recognizes the entrance sound through the entrance sound recognition unit 145, the service control unit 150 communicates with the camera device 185 through the local area network communication unit 120b to communicate with the camera device ( 185), s service functions including an image service function for photographing the sound source that generated the recognized entrance sound can be activated, and the service provider 180 can activate the service function through the local area network communication unit 120b. By communicating with the camera device 185, video information captured through the camera device 185 is provided, and video services are provided through an app installed on a wireless terminal through the operation server 200 and/or provided through a designated IPTV device. At least one video service among video services may be provided. Meanwhile, if the camera device 185 is capable of providing the video service, the service provider 180 can process the video service function of the camera device 185 to operate through the local area network communication unit 120b.

Or, according to the third camera embodiment of the present invention, if the sound device 100 is equipped with a camera module 110 for acquiring image information of a subject, the service control unit 150 may be configured to include the entrance sound recognition unit ( When an entry sound is recognized through 145), the video service function for filming the sound source that generated the recognized entry sound can be activated through the camera module 110, and the service provider 180 It is possible to provide at least one video service among video services that provide video information captured through the camera module 110 to an app installed on a wireless terminal through the operation server 200 and/or video services provided by a designated IPTV device. there is.

Figure 2 is a diagram showing the functional configuration of the stationary sound device 100 and the operation server 200 according to another implementation method of the present invention.

In more detail, Figure 2 shows an embodiment in which at least some of the functional structures of the control module 130 of the stationary sound device 100 shown in Figure 1 are implemented on the operation server 200. , a person skilled in the art to which the present invention pertains will know how to at least partially modify the embodiments of Figure 1 and Figure 2 by referring to and/or modifying Figure 2 (e.g., Figure 2). Some of the functional configurations implemented on the operation server 200 side are implemented as is on the operation server 200 side, but other parts are implemented in the control module 130 of the stationary sound device 100, or the control module of Figure 1 ( An implementation method of implementing some of the functional configurations implemented in 130) as is in the control module 130 and implementing other parts on the operation server 200, in which some components shown in Figure 1 and Figure 2 are omitted, (or subdivided or combined implementation methods, etc.) may be inferred, but the present invention includes all the inferred implementation methods as scope of rights, and its technical features are based only on the implementation methods shown in Figure 1 and Figure 2. It is not limited.

Referring to Figure 2, the operating server 200 includes a sound receiving unit 205 that receives sound data input through the sound input unit 105a of a designated sound device 100 and temporarily stores it in a designated storage medium, By reading the received sound data, it corresponds to at least one entry sound frequency band designated to recognize at least one entry/exit sound among non-verbal sounds or noises that may occur in a designated space, and/or recognizes the entry/exit sound. an entrance sound recognition unit 145 that recognizes a specific entrance sound corresponding to at least one entrance sound pattern designated for Activate at least s (1≤s≤S) service functions and/or change the operation mode of the sound device 100 to sleep mode when the sound of entry or exit or the content of the user's speech is not recognized for more than a specified period of time. It includes a service control unit 150 for setting, and further includes a service provision unit 180 for providing a service corresponding to the activated service function.

When sound data input through the sound input unit 105a is transmitted from the designated sound device 100, the sound receiving unit 205 receives the sound data transmitted from the sound device 100 and stores it temporarily in the designated storage medium. It can be stored. Meanwhile, when the sound device 100 selectively transmits one or more sound data among sound data corresponding to an entry/exit sound or sound data corresponding to a user's utterance, the sound receiver 205 receives the sound data from the sound device 100. The selectively transmitted sound data can be received and temporarily stored in a designated storage medium.

The entry sound recognition unit 145 reads the received sound data and recognizes at least one entry sound designated for entry recognition among non-verbal sounds or noises that may occur in a designated space. The process of recognizing a specific entrance sound corresponding to at least one entrance sound pattern designated to correspond and/or recognize the entrance sound is continued (or repeated).

The service control unit 150 checks whether the entrance sound is not recognized for more than a specified period of time after operating power is applied to the sound device 100 and/or effectively recognizes the entrance sound through the entrance sound recognition unit 145. After that, check whether the entrance sound is not recognized as valid for more than a specified period of time, and/or check whether the contents of the user's utterance are not recognized for more than a specified period of time after operating power is applied to the sound device 100, and/or After the content of the user's utterance is validly recognized according to a designated procedure (e.g., the content of the user's utterance is recognized within a certain period of time during or after the access sound is validly recognized through the access sound recognition unit 145), the user for a specified period of time or more You can check whether the content of the utterance is not recognized. If the entrance sound is not recognized for more than a specified period of time or the content of the user's speech is not recognized for more than a specified period of time, the service control unit 150 remotely sets the operation mode of the sound device 100 to sleep mode. It can be done including the function.

According to the implementation method of the present invention, the sound device 100 is capable of transmitting the input sound data in a sleep mode, and the sound receiver 205 is configured to operate the sound device 100 in a sleep mode. The sound data is received in this state, and the entry/exit sound recognition unit 145 also reads the received sound data while the operation mode of the sound device 100 is in sleep mode to detect non-verbal sounds or noises that may occur in the designated space. A specific entry sound corresponding to at least one entry sound frequency band specified to recognize at least one entry sound designated for recognition of entry and exit and/or corresponding to at least one entry sound pattern specified to recognize the entry sound. The process of recognition can be continued (or repeated). Preferably, when an entry/exit sound is effectively recognized through the entry/exit sound recognition unit 145 in the sleep mode, the service control unit 150 wakes up the sound device 100 and simultaneously generates s number of entries related to the entry/exit sound. A service function can be activated, and the service provider 180 can perform a procedure to provide a service corresponding to the activated s service functions.

Meanwhile, when two or more service functions related to entrance sounds are activated through the service control unit 150, the service provider 180 performs a procedure for providing services corresponding to the activated service functions in a designated order. And/or a procedure for providing at least one service according to a preset condition (eg, a condition for recognizing a user's utterance) among the activated service functions may be performed.

According to the method of implementing the present invention, the entry/exit sound recognition unit 145 corresponds to the entry/exit sound frequency band of the entry/exit sound generator and/or the unique entry/exit sound of the entry/exit sound generator corresponding to the entry/exit sound pattern of the entry/exit sound generator. It is desirable to recognize, and for this purpose, the operation server 200 has a function of learning at least one entry/exit sound frequency band specialized for the entry/exit sound source and/or a function of learning at least one entry/exit sound pattern specific to the entry/exit sound source. It is desirable to have a.

Referring to Figure 2, the operation server 200 stores M entry/exit sound frequency band information corresponding to M (M ≥ 1) entry/exit sound frequency bands in a designated storage medium and/or N (N ≥ 1). ) is provided with an information storage unit 135 that stores N entrance sound pattern information corresponding to the entrance sound patterns in a designated storage medium, and the entrance sound recognition unit 145 stores the entrance sound frequency band stored in the designated storage medium. Entry and exit sounds can be recognized using information and/or entry and exit sound pattern information.

The information storage unit 135 may store M entry/exit sound frequency band information corresponding to M (M≥1) entry/exit sound frequency bands in a designated storage medium. In the present invention, among the non-verbal sounds or noises that can occur in a designated space, the entry/exit sound designated for entry/exit recognition has various frequencies (or a wide range of frequencies where the frequency difference between the lowest frequency and the highest frequency is at least 100 Hz or more) depending on the type of sound or the sound source. It can have a range. Accordingly, the present invention divides the frequency band of entrance and exit sounds into M frequency bands (e.g., equal division or experimentally selective division based on a plurality of frequency bands in which entrance and exit sounds are concentrated), and the information storage unit 135 Can store the M entry/exit sound frequency band information corresponding to the divided M entry/exit sound frequency bands in a designated storage medium.

The information storage unit 135 may store N entry/exit sound pattern information corresponding to N (N≥1) entry/exit sound patterns in a designated storage medium. The present invention first generates entry/exit sound pattern information through a qualitative and/or quantitative analysis process for each case of the entry/exit sound pattern confirmed through a sample experiment, and the information storage unit 135 stores the generated entry/exit sound pattern. Information can be stored in a designated storage medium.

Meanwhile, according to the implementation method of the present invention, the sound device 100 can recognize the entrance sound and then transmit the recognized entrance sound information to the designated operation server 200, and the intelligent processing unit 175 of the operation server 200 ) collects entry and exit sound information from each sound device 100, and normalizes individual sound characteristics for each sound source from the collected entry and exit sounds, and/or qualitative analysis and/or quantitative analysis for each case. Through the analysis process, entry/exit sound pattern information can be refined or diversified. The information storage unit 135 may receive refined or diversified entry/exit sound pattern information through the above process, and store N entry/exit sound pattern information including the received entry/exit sound pattern information in a designated storage medium. .

According to the implementation method of the present invention, M entry/exit sound frequency band information corresponding to the M entry/exit sound frequency bands is stored in the designated storage medium, and/or N entry/exit sound frequency band information corresponding to the N entry/exit sound patterns are stored in the designated storage medium. When sound pattern information is stored, the entry/exit sound recognition unit 145 reads the received sound data and corresponds to at least one entry/exit sound frequency band among the stored M entry/exit sound frequency bands and/or the stored N entry/exit sound frequency bands. It is desirable to recognize an entry/exit sound corresponding to at least one entry/exit sound pattern among the entry/exit sound patterns.

Meanwhile, the M entry/exit sound frequency band information and/or N entry/exit sound pattern information stored in the designated storage medium are not optimized (or tailored) to the entry/exit sound of a specific sound source, but rather the frequency band and/or sound of statistical entry/exit sound. This may be information corresponding to a pattern. Therefore, in order to recognize at high speed the entrance sound of a specific sound source existing in the space where the sound device 100 is installed and/or prevent (or minimize) recognition errors, the stored entrance sound frequency band information and/or entrance sound A process of optimizing (or customizing) the pattern information to the entrance/exit sound of the specific sound source is required.

Referring to Figure 2, the operation server 200 uses the M entry/exit sound frequency band information to determine an entry/exit sound frequency corresponding to an entry/exit sound frequency distribution that has been validly recognized at least i (i≥2) times during a specified period of time. Generate distribution information and/or generate a distribution that uses the N entry and exit sound pattern information to generate entry and exit sound pattern distribution information corresponding to the entry and exit sound pattern distribution that has been validly recognized at least j (j≥2) times during a specified period of time. Unit 155 and the generated entrance sound frequency distribution information are statistically processed to optimize (or customize) the entrance sound of the entrance sound source in the space where the sound device 100 is mounted, m (1 ≤ m ≤ M ) determine the m entry/exit sound frequency band information corresponding to the entry/exit sound frequency bands and/or statistically process the generated entry/exit sound pattern distribution information to identify the entry/exit sound source in the space where the sound device 100 is installed. It further includes an information determination unit 160 that determines n entry/exit sound pattern information corresponding to n (1≤n≤N) entry/exit sound patterns optimized (or customized) for entry/exit sounds.

When M entry/exit sound frequency band information corresponding to M entry/exit sound frequency bands is stored in the storage medium designated through the information storage unit 135, the distribution generator 155 stores the M entry/exit sound frequency bands stored in the designated storage medium. Using sound frequency band information, identify i entry/exit sound frequency bands that have been validly recognized at least i (i≥2) times during a specified period of time, and enter and exit sound frequency distribution corresponding to the distribution of the identified i entry/exit sound frequency bands. Information can be generated.

Meanwhile, when entry/exit sound frequency distribution information corresponding to an entry/exit sound frequency distribution that has been validly recognized at least i times during a specified period of time is generated through the distribution generator 155, the information determination unit 160 determines the generated entry/exit sound frequency. The distribution information is processed statistically to optimize (or match) the entrance and exit sounds of the entry and exit sound sources in the space where the sound device 100 is installed to recognize the entry and exit sounds of the entry and exit sound sources at high speed or to prevent (or minimize) recognition errors. In order to do this, m entry/exit sound frequency band information corresponding to m (1≤m≤M) entry/exit sound frequency bands can be determined. Here, the m entry/exit sound frequency bands may be any one of the M entry/exit sound frequency bands, or may include a newly defined entry/exit sound frequency band within the entry/exit sound frequency band range. When the m entry/exit sound frequency band information is determined through the information determination unit 160, the information storage unit 135 sets the determined m entry/exit sound frequency bands as the entry/exit sound frequency band to be read first and stores the designated information. It can be newly stored in a medium or the entry/exit sound frequency band information stored in the storage medium can be updated and stored. Thereafter, the entry/exit sound recognition unit 145 may read the received sound data and first recognize an entry/exit sound corresponding to at least one entry/exit sound frequency band among the m new or newly stored entry/exit sound frequency bands.

Meanwhile, when N entry/exit sound pattern information corresponding to N entry/exit sound patterns is stored in a storage medium designated through the information storage unit 135, the distribution generator 155 stores the N entry/exit sound patterns stored in the designated storage medium. Using sound pattern information, identify j entry/exit sound patterns that have been validly recognized at least j (j≥2) times during a designated period of time, and generate entry/exit sound pattern distribution information corresponding to the distribution of the confirmed j entry/exit sound patterns. can do.

Meanwhile, when entry/exit sound pattern distribution information corresponding to an entry/exit sound pattern distribution that has been validly recognized at least j times during a specified period of time is generated through the distribution generator 155, the information determination unit 160 determines the generated entry/exit sound pattern. The distribution information is processed statistically to optimize (or customize) the entry and exit sounds of the entry and exit sound sources in the space where the sound device 100 is installed to recognize the entry and exit sounds of the entry and exit sound sources at high speed or to prevent (or minimize) recognition errors. In order to do this, n entry/exit sound pattern information corresponding to n (1≤n≤N) entry/exit sound patterns can be determined. Here, the n entry/exit sound patterns may be any one of the N entry/exit sound patterns, a combination of two or more entry/exit sound patterns, or may include a new entry/exit sound pattern. When n entry/exit sound patterns are determined through the information determination unit 160, the information storage unit 135 sets the determined n entry/exit sound patterns as entry/exit sound patterns to be read first and stores them in a designated storage medium. The entry/exit sound frequency band information stored in the storage medium can be updated and stored. Thereafter, the entry/exit sound recognition unit 145 may read the received sound data and first recognize an entry/exit sound corresponding to at least one entry/exit sound pattern among the n entry/exit sound patterns.

Referring to Figure 2, the operation server 200 reads the received sound data, recognizes the user's language utterance, processes natural language, and recognizes utterance content information corresponding to the user's utterance content (a utterance content recognition unit ( 165) and a recognition determination unit 170 that checks whether the content of the user's utterance is recognized while the entry/exit sound is being recognized or whether the content of the user's utterance is recognized within a certain time after the recognition of the entry/exit sound ends, and the entry/exit sound If the content of the user's speech is recognized within a certain period of time during or after it is recognized, it may be provided with an intelligent processing unit 175 that processes the recognized content of the user's speech to be interpreted by relating it to s service functions related to entrance and exit sounds. there is.

At the same time as performing the process of recognizing the entrance sound by reading the received sound data through the entrance sound recognition unit 145, the speech content recognition unit 165 reads the received sound data and recognizes the user's language speech. It recognizes and performs natural language processing to recognize speech content information corresponding to the user's speech content.

According to the implementation method of the present invention, the speech content recognition unit 165 analyzes the received sound data through a designated voice recognition algorithm, recognizes the voice uttered by the user, converts it into text, and simultaneously performs morphological analysis, semantic analysis, and conversation. During analysis, utterance content information that recognizes the utterances uttered by the user can be generated through one or more natural language processing procedures.

Meanwhile, according to the implementation method of the present invention, the utterance content recognition unit 165 recognizes the entrance sound by the entrance sound recognition unit 145 and changes the operation mode of the service device from sleep mode through the service control unit 150. It is desirable to perform a function to recognize the contents of the user's speech when woken up. If the operation mode of the sound device 100 is set to sleep mode by the service control unit 150, the utterance content recognition unit 165 may not perform the function of recognizing the user's utterance content, and even if Even if the content of the user's speech is recognized, it can be processed so that it cannot be used for the service.

Meanwhile, when the user's speech content is recognized through the speech content recognition unit 165, the recognition determination unit 170 recognizes the user's speech content while the entrance sound is recognized through the entrance sound recognition unit 145. It is checked whether the user's utterance is recognized or whether the user's utterance is recognized within a certain period of time after recognition of the entry/exit sound is completed through the entry/exit sound recognition unit 145.

If it is confirmed through the recognition determination unit 170 that the user's utterance content has been recognized within a certain period of time during or after the entry/exit sound is recognized, the intelligent processing unit 175 determines that the utterance content recognition unit 165 The content of the user's utterance recognized through is processed to be interpreted by relating it to s service functions related to the entrance sound. As a result, a specific service function corresponding to the utterance content among the s service functions related to the entrance sound can be determined. In this case, the service control unit 150 may activate a specific service function related to the specific sound and corresponding to the content of the recognized user's speech, and the service provider 180 may activate the activated specific service function. The provided procedures can be performed according to designated procedures.

If the content of the user's utterance is not recognized within a certain period of time during or after the entrance sound is recognized through the recognition determination unit 170, the intelligent processing unit 175 switches to sleep mode through the service control unit 150. A process of switching to and/or waking up from the sleep mode by the entry/exit sound may be performed.

Meanwhile, according to the implementation method of the present invention, s services related to entrance sounds that can be activated by the service control unit 150 of the operation server 200 and/or provided through the service provider 180 are provided by the operation server ( It may include at least one service among a server-based service provided through 200) and a complex service provided in conjunction with the sound device 100.

Meanwhile, among the s services related to the entrance sound, the server-based service is a calling service using wireless terminal information, a situation notification service using wireless terminal information, a calling service using an app installed on the wireless terminal, and a calling service using an app installed on the wireless terminal. At least one service among a situation notification service, a video service that provides video information captured through at least one camera to an app installed on a wireless terminal, and a video service that provides video information captured through at least one camera to a designated IPTV device may include.

Meanwhile, among the s services related to the entrance sound, the composite service recognizes the user's speech content within a certain period of time during or after the entrance sound is recognized and interprets the user's speech content by relating it to the service function related to the entrance sound. Based on the results, services provided through the operation server 200 on the communication network may be included.

Figure 3 illustrates an embodiment of recognizing an entrance sound and activating a service related to the entrance sound according to an implementation method of the present invention.

In more detail, Figure 3 shows an embodiment of recognizing entrance sounds using M entrance sound frequency band information and/or N entrance sound pattern information stored in a designated storage area, and activating services related to the recognized entrance sounds. , those of ordinary skill in the technical field to which the present invention pertains can refer to and/or modify Figure 3 to implement various implementation methods (e.g., some steps are omitted) for the entry/exit sound recognition or service activation process. However, the present invention includes all of the above inferred implementation methods, and its technical features are not limited to only the implementation method shown in Figure 3. Meanwhile, the process illustrated in Figure 3 is provided through the sound device 100, the operation server 200, or is provided in a distributed form through the linkage of the sound device 100 and the operation server 200. It is possible, and for convenience, Figure 3 will explain the features of the present invention focusing on the process provided through the sound device 100.

Referring to the embodiment of Figure 3, the sound device 100 stores M entry/exit sound frequency band information corresponding to M entry/exit sound frequency bands for recognizing entry/exit sounds in a designated storage area (300), and/or N entry/exit sound pattern information corresponding to N entry/exit sound patterns for recognizing entry/exit sounds is stored in a designated storage area (300). Afterwards, the sound device 100 receives sound data corresponding to a sound signal in the audible frequency band through the sound input unit 105a, buffers it in a designated buffer area (305), and reads the buffered sound data to read the sound data in the designated space. To a specific entrance sound corresponding to at least one entrance sound frequency band designated to recognize at least one entrance sound designated for determining a specific situation and/or to a specific entrance sound corresponding to at least one entrance sound pattern designated to recognize the entrance sound A procedure for recognizing the corresponding entry/exit sound is performed (310).

According to one embodiment of the present invention, the sound device 100 reads the buffered sound data and corresponds to at least one entry/exit sound frequency band among the stored M entry/exit sound frequency bands and/or the stored N entry/exit sound frequency bands. An entry/exit sound corresponding to at least one entry/exit sound pattern among the entry/exit sound patterns may be recognized (310). According to another embodiment of the present invention, m entry and exit sound frequencies are optimized (or tailored) to the entry and exit sounds of the entry and exit sound sources in the space where the sound device 100 is mounted through the process of Figure 4 and/or 5. When band information is determined and stored in a designated storage area and/or n entry/exit sound pattern information optimized (or tailored) to the entry/exit sound of the specific sound source is determined and stored in a designated storage area, the sound device 100 Reads the buffered sound data and corresponds to at least one entry/exit sound frequency band among the m stored entry/exit sound frequency bands and/or a specific sound corresponding to at least one entry/exit sound pattern among the n stored entry/exit sound patterns. The unique entrance and exit sound of the source can be recognized (310).

If the operation mode of the sound device 100 is sleep mode at the time of recognizing the entrance sound, the sound device 100 may wake up from the sleep mode (315).

Meanwhile, when the entrance sound is recognized and/or the operation mode of the sound device 100 is woken up from sleep mode, the sound device 100 is connected to the entrance sound among the designated S (S ≥ 1) service functions. At least s (1≤s≤S) service functions can be activated (320), and a procedure for providing at least one service function among the activated s service functions can be performed according to a designated procedure (325).

Meanwhile, when the sound data is read and the entrance sound is not recognized, the sound device 100 checks whether the entrance sound is not validly recognized for a specified period of time (330). If the entry/exit sound is not recognized as valid for a specified period of time, the sound device 100 may set the operation mode of the sound device 100 to sleep mode (335).

Figure 4 shows an embodiment of optimizing (or customizing) the entry/exit sound frequency band for recognizing entry/exit sounds to the entry/exit sound of an entry/exit sound source according to an implementation method of the present invention.

In more detail, Figure 4 shows entrance sound frequency distribution information corresponding to the entrance sound frequency distribution that has been validly recognized at least i (i ≥ 2) times during a specified period using M entry and exit sound frequency band information previously stored in the designated storage area. Generate, and statistically process the generated entrance sound frequency distribution information, m (1≤m≤M) optimized (or tailored) to the entrance sound of the entrance sound source in the space where the sound device 100 is mounted. This illustrates an embodiment of determining m entry/exit sound frequency band information corresponding to the entry/exit sound frequency band and storing new/updated information in a designated storage area. Those skilled in the art will understand that this By referring to and/or modifying Figure 4, various implementation methods (e.g., implementation methods in which some steps are omitted or the order is changed) for the process of optimizing (or customizing) the entry/exit sound frequency band may be inferred. The invention includes all of the above-described implementation methods, and its technical features are not limited to only the implementation method shown in Figure 4. Meanwhile, the process illustrated in Figure 4 is provided through the sound device 100, the operation server 200, or is provided in a distributed form through interconnection between the sound device 100 and the operation server 200. It is possible, and for convenience, Figure 4 will explain the features of the present invention focusing on the process provided through the sound device 100.

Referring to the embodiment of Figure 4, the sound device 100 effectively recognizes the entrance sound using M entrance sound frequency band information previously stored in the designated storage area, and then uses the entrance sound used to effectively recognize the entrance sound. Frequency band information is stored and managed in a designated storage area (400).

If i (i ≥ 2) entry/exit sound frequency band information used to effectively recognize entry/exit sounds is stored in the storage area for a specified period of time (405), the sound device 100 stores at least i (i Entry and exit sound frequency distribution information corresponding to the entry and exit sound frequency distribution that has been validly recognized ≥2) times is generated (410), and the generated entry and exit sound frequency distribution information is statistically processed to determine the number of entries in the space where the sound device 100 is installed. After determining m entry/exit sound frequency band information corresponding to m (1≤m≤M) entry/exit sound frequency bands optimized (or tailored) to the entry/exit sound of the entry/exit sound source (415), the determined m entry/exit sound frequencies The band is set as the entry/exit sound frequency band to be read first, and new/updated storage is performed in a designated storage area (420). Afterwards, the sound device 100 recognizes the entry/exit sound using the stored m entry/exit sound frequency band information. You can.

Figure 5 shows an embodiment of optimizing (or customizing) an entry/exit sound pattern for recognizing entry/exit sounds to the entry/exit sound of a specific sound source according to another implementation method of the present invention.

In more detail, Figure 5 shows entrance sound pattern distribution information corresponding to the entrance sound pattern distribution that has been validly recognized at least j (j≥2) times during a specified period of time using N entry sound pattern information previously stored in a designated storage area. n (1≤n≤N) entries optimized (or tailored) to the entry/exit sound of the entry/exit sound source in the space where the sound device 100 is mounted by statistically processing the generated entry/exit sound pattern distribution information. This shows an embodiment of determining n entry/exit sound pattern information corresponding to a sound pattern and storing new/updated information in a designated storage area. Those of ordinary skill in the art to which the present invention pertains will refer to Figure 5. Various implementation methods (e.g., implementation methods in which some steps are omitted or the order is changed) for the process of optimizing (or customizing) the entry/exit sound pattern can be inferred by reference and/or modification, but the present invention does not apply to the above analogy. It includes all possible implementation methods, and its technical features are not limited to only the implementation methods shown in Figure 5. Meanwhile, the process illustrated in Figure 5 is provided through the sound device 100, the operation server 200, or is provided in a distributed form through the linkage of the sound device 100 and the operation server 200. It is possible, and for convenience, Figure 5 will explain the features of the present invention focusing on the process provided through the sound device 100.

Referring to the embodiment of Figure 5, the sound device 100 effectively recognizes an entry/exit sound using N entry/exit sound pattern information previously stored in a designated storage area, and then uses the entry/exit sound pattern used to effectively recognize the entry/exit sound. Information is stored and managed in a designated storage area (500).

If j (j ≥ 2) entry/exit sound frequency band information used to effectively recognize entry/exit sounds is stored in the storage area for a specified period of time (505), the sound device 100 is valid at least j times during the specified period of time. Entry and exit sound pattern distribution information corresponding to the recognized entry and exit sound pattern distribution is generated (510), and the generated entry and exit sound pattern distribution information is statistically processed to allow entry and exit of a specific sound source in the space where the sound device 100 is installed. After determining n entry/exit sound pattern information corresponding to n (1≤n≤N) entry/exit sound patterns optimized (or customized) for sound (515), the determined n entry/exit sound patterns are first read as entry/exit sound patterns. It is set to new/updated and stored in the designated storage area (520). Afterwards, the sound device 100 can recognize entry/exit sounds using the stored n entry/exit sound pattern information.

Figure 6 shows an embodiment of activating a service related to the entrance sound by linking the entrance sound and the content of the speech according to another implementation method of the present invention.

In more detail, Figure 6 shows that when the content of the user's speech is recognized within a certain period of time during or after the entrance sound is recognized, the recognized content of the user's speech is interpreted in relation to s service functions related to the entrance sound and the recognition is performed. This illustrates an embodiment of activating a service related to an entrance sound. Those skilled in the art will refer to and/or modify Figure 6 to perform the process of recognizing the entrance sound or activating the service. Although various implementation methods can be inferred (for example, implementation methods in which some steps are omitted or the order is changed), the present invention includes all the implementation methods inferred above, and the implementation shown in Figure 6 The technical features are not limited to the method alone. Meanwhile, the process illustrated in Figure 6 is provided through the sound device 100, the operation server 200, or is provided in a distributed form through linkage between the sound device 100 and the operation server 200. It is possible, and for convenience, Figure 6 will explain the features of the present invention focusing on the process provided through the sound device 100.

Referring to the embodiment of Figure 6, the sound device 100 reads sound data buffered in a designated buffer area, recognizes the user's language utterance, and processes natural language to recognize speech content information corresponding to the user's utterance content. Perform (600). If the content of the user's speech is not recognized, the sound device 100 checks whether the content of the user's speech is not validly recognized for more than a specified period of time (605). If the content of the user's speech is not recognized as valid for more than a specified period of time, the sound device 100 may set the operation mode of the sound device 100 to sleep mode (610).

Meanwhile, when the content of the user's speech is recognized by reading the sound data, the sound device 100 checks whether the operation mode of the sound device 100 is sleep mode (615). If the device is in sleep mode, it is desirable to perform a wake-up process by recognizing an entry/exit sound.

On the other hand, if the operation mode of the sound device 100 is not sleep mode at the time the user's utterance content is recognized (e.g., while the user's utterance content is being recognized or an entry/exit sound is recognized before a certain time, the sound device 100 (when the sleep mode is woken up), the sound device 100 confirms that the user's utterance is recognized while the entrance sound is recognized and/or detects the user's utterance within a certain time after the end of recognition of the entrance sound. Check whether the content is recognized (620). If the content of the user's speech is not recognized within a certain period of time during or after the entrance sound is recognized, it is desirable to check whether the sound device 100 is in sleep mode or to perform a wake-up process by recognizing the entrance sound. do.

Meanwhile, if the content of the user's speech is recognized within a certain period of time during or after the entrance sound is recognized, the sound device 100 interprets the recognized content of the user's speech by associating it with s service functions related to the entrance sound. (625), and based on this, among the S service functions, a specific service function that is related to the specific sound and corresponds to the content of the recognized user's speech can be activated (630), and the activated specific service function can be activated by a designated service function. The procedure provided can be performed according to the procedure (635).

100: Sound device 105: Sound module
105a: sound input unit 105b: sound output unit
110: Camera module 110a: Camera unit
115: power module 120: communication module
120a: wireless network communication unit 120b: local area network communication unit
125: short-range module 125a: short-range wireless unit
130: control module 130a: control unit
130b: memory unit 135: information storage unit
140: Sound processing unit 145: Entry/exit sound recognition unit
150: Service control unit 155: Distribution generation unit
160: Information decision unit 165: Speech content recognition unit
170: situation determination unit 175: intelligent processing unit
180: Service provider 185: Camera device
200: operation server 205: sound receiver

Claims (23)

A sound module including a sound input unit that receives a sound signal and a sound output unit that outputs a sound signal, a communication module that includes at least one communication unit connected to a communication network, a short-range camera device, a short-range module that processes short-range wireless communication, and a subject. In the sound device including a camera module that acquires image information captured,
a sound processing unit that buffers sound data corresponding to the sound signal in the audible frequency band input through the sound input unit into a designated buffer area or checks sound data buffered in the designated buffer area;
In order to read the buffered sound data and recognize at least one entry sound designated for entry/exit recognition among non-verbal sounds or noises that may occur in a designated space, the buffered sound data corresponds to at least one specified entry/exit sound frequency band and corresponds to at least one entry/exit sound pattern. An entry/exit sound recognition unit that recognizes a corresponding specific entry/exit sound; and
A service control unit that activates at least s (1 ≤ s ≤ S) service functions related to the entrance sound among the specified S (S ≥ 1) service functions when the entrance sound is recognized; and a control module including a. ,
The control module is,
a speech content recognition unit that reads the buffered sound data, recognizes the user's language speech, processes it into natural language, and recognizes speech content information corresponding to the user's speech content;
a recognition determination unit that checks whether the content of the user's utterance is recognized while the entry/exit sound is being recognized or whether the content of the user's utterance is recognized within a certain time after the recognition of the entry/exit sound ends;
When the content of the user's speech is recognized within a certain period of time during or after the entrance sound is recognized, an intelligent processing unit that processes the recognized content of the user's speech to be interpreted in relation to s service functions related to the entrance sound;
Information storage that stores M entry/exit sound frequency band information corresponding to M(M≥1) entry/exit sound frequency bands and N entry/exit sound pattern information corresponding to N(N≥1) entry/exit sound patterns in a designated storage area. It is further provided with:
The service control unit,
A function to activate an image service function that communicates with the camera device through the short-range network communication unit, which is the short-range module, or photographs the sound source that generated the recognized entrance sound through the cabera module;
A sound device for entry recognition that includes a function to activate a specific service function that is related to the entry sound and simultaneously corresponds to the content of the recognized user's speech. delete delete delete delete delete delete The method of claim 1, wherein the entry/exit sound recognition unit:
A function for reading the buffered sound data and recognizing an entry/exit sound corresponding to at least one entry/exit sound frequency band among the M entry/exit sound frequency bands and at least one entry/exit sound pattern among the N entry/exit sound patterns. A sound device for entry/exit recognition, characterized in that it is made by. delete delete delete delete delete delete The method of claim 1, wherein the s services are:
A server interworking service provided in conjunction with an operating server on a communication network through the communication module,
Sound-based services provided through the sound module,
A sound device for entry recognition, characterized in that it includes at least one service among complex services provided by combining the sound module and the communication module.
delete delete delete The method of claim 15, wherein the composite service is:
A service provided through an operation server on a communication network based on the results of recognizing the content of the user's utterance within a certain period of time during or after the entry sound is recognized and interpreting the content of the user's utterance in relation to the service function related to the entry sound. A sound device for entry/exit recognition comprising:
delete delete delete delete