KR102495028B1 - Sound Device with Function of Whistle Sound Recognition - Google Patents

Abstract

According to the sound device having a whistle recognition function of the present invention, a sound module including a sound input unit for receiving a sound signal and a sound output unit for outputting a sound signal, a communication module including at least one communication unit connected to a communication network, and A sound device comprising a short-range camera device, a short-range module for processing short-range wireless communication, and a camera module for acquiring image information of a subject, wherein the sound corresponds to a sound signal of an audible frequency band input through the sound input unit. A sound processing unit that buffers data in a designated buffer area or checks the sound data buffered in the designated buffer area, and reads the buffered sound data to recognize a designated whistling sound among non-verbal sounds or noises that may occur in a designated space to determine a specific situation. A whistle recognition unit for recognizing the whistle sound unique to the whistle sound generator corresponding to the whistle frequency band of at least one designated whistle sound generator and corresponding to the whistle sound pattern of the whistle sound generator and recognizing the whistle sound The camera by using at least s (1≤s≤S) service functions related to the whistle among the S (S≥1) service functions specified in the case and any one or more of the short-distance module, communication module, and camera module. A control module including a service control unit for activating a video service function for photographing a sound generating source that generates the recognized whistling sound through a device, wherein the control module includes M (M≥1) whistle sound frequency bands. An information storage unit for storing M whistle frequency band information corresponding to and N whistle pattern information corresponding to N (N≥1) whistle patterns in a designated storage area, and the M whistle frequency band information generate whistle frequency distribution information corresponding to at least i (i≥2) times validly recognized whistle frequency distribution for a specified period of time, and use the N whistle sound pattern information for a specified period of time at least j ( j≥2) validly recognized whistles Optimize (or customize) the whistle sound of the whistle sound generator in the space where the sound device passes through a distribution generator that generates whistle sound pattern distribution information corresponding to the pattern distribution and statistically processes the generated whistle sound frequency distribution information m whistle frequency band information corresponding to the m (1≤m≤M) frequency bands of whistle sounds generated, statistically processing the generated whistle sound pattern distribution information, By reading the buffered sound data and an information determination unit that determines n whistle sound pattern information corresponding to n (1≤n≤N) whistle sound patterns optimized (or customized) for the whistling sound of the sound generator, A speech contents recognition unit that recognizes speech contents and recognizes speech contents information corresponding to the user's speech contents by recognizing language utterances and processing them in natural language If the user's speech content is recognized within a certain time during or after the whistling sound is recognized, the recognition determination unit confirms whether the user's speech content is recognized within s number related to the whistling sound. It further includes an intelligent processing unit that processes interpretation in association with a service function, and a service providing unit that provides a service corresponding to the activated service function.

Description

Sound device with function of whistle sound recognition {Sound Device with Function of Whistle Sound Recognition}

The present invention is a stationary sound device having a sound module for inputting and outputting sound signals and a communication module for connecting to a communication network, which can be generated in a designated space by reading sound data corresponding to a sound signal of an audible frequency band input through the sound module. Corresponds to at least one whistle frequency band designated for recognizing at least one whistle sound designated for determining a specific reimbursement among non-verbal sounds or noises and/or corresponds to at least one whistle sound pattern designated for recognizing the whistle sound If a specific whistling sound is recognized and recognized, activate at least one designated service function related to the whistling sound among services that can be provided through at least one of the sound module and the communication module (or a combination of the two), or the designated service function To provide a sound device that provides.

Recently, a stationary sound device that recognizes a user's voice and provides a service corresponding to the recognized voice has been sold. For example, stationary sound devices such as Amazon's 'Echo', SK Telecom's 'Who', and KT's 'Giga Genie' are on sale.

However, the conventional stationary sound device is a combination of a sound source service and voice recognition technology, and operates mainly on a service that simply provides or reproduces a sound source desired by a user through voice recognition, and has a problem in that it is difficult to use in other fields.

An object of the present invention for solving the above problems is a sound module including a sound input unit for receiving a sound signal and a sound output unit for outputting a sound signal, and a communication module including at least one communication unit connected to a communication network and a short distance A sound device comprising a short-range module for processing short-range wireless communication with a camera device and a camera module for obtaining image information of a subject, wherein sound data corresponding to a sound signal of an audible frequency band input through the sound input unit A sound processing unit that buffers in a designated buffer area or checks the sound data buffered in the designated buffer area and reads the buffered sound data to recognize a designated whistling sound to determine a specific situation among non-verbal sounds or noises that may occur in a designated space A whistle recognizing unit for recognizing a unique whistling sound of a whistle sound generator corresponding to a whistle frequency band of at least one designated whistle sound generating source and corresponding to a whistling sound pattern of the whistling sound generating source and the whistling sound when recognizing the whistling sound The camera device using at least s (1≤s≤S) service functions related to the whistle among S (S≥1) service functions specified in and any one or more modules of the short-distance module, communication module, and camera module A control module comprising a; service control unit for activating a video service function for photographing a sound generating source that generates the recognized whistling sound through Using an information storage unit that stores corresponding M whistle frequency band information and N whistle pattern information corresponding to N (N≥1) whistle patterns in a designated storage area and the M whistle frequency band information generate whistle frequency distribution information corresponding to the validly recognized whistle frequency distribution at least i (i≥2) times during a designated period of time, and use the N whistle pattern information to generate at least j (j ≥2) times valid recognized whistle pattern A distribution generator for generating whistle pattern distribution information corresponding to the distribution and statistical processing of the generated whistle frequency distribution information to optimize (or customize) the whistle sound of the whistle sound generator in the space where the sound device has passed through m whistle sound frequency band information corresponding to m (1≤m≤M) number of whistle sound frequency bands is determined, the generated whistle pattern distribution information is statistically processed, and the whistle sound in the space where the sound device is installed An information determination unit that determines n whistle pattern information corresponding to n (1≤n≤N) whistle patterns optimized (or customized) for the whistle sound of the source, and the buffered sound data is read to provide the user's language A speech content recognition unit that recognizes speech contents and recognizes speech contents information corresponding to the user's speech contents by recognizing speech and natural language processing; If the user's speech is recognized within a certain period of time during or after the recognition judging unit and the whistling sound are recognized, the recognized user's speech is s services related to the whistling sound It is to provide a sound device equipped with a whistling sound recognition function further comprising an intelligent processing unit processing to interpret in association with a function and a service providing unit providing a service corresponding to the activated service function.

A sound device having a whistle recognition function according to the present invention includes a sound module including a sound input unit for receiving a sound signal and a sound output unit for outputting a sound signal, a communication module including at least one communication unit connected to a communication network, and A sound device comprising a short-range camera device, a short-range module for processing short-range wireless communication, and a camera module for acquiring image information of a subject, wherein the sound corresponds to a sound signal of an audible frequency band input through the sound input unit. A sound processing unit that buffers data in a designated buffer area or checks the sound data buffered in the designated buffer area, and reads the buffered sound data to recognize a designated whistling sound among non-verbal sounds or noises that may occur in a designated space to determine a specific situation. A whistle recognition unit for recognizing the whistle sound unique to the whistle sound generator corresponding to the whistle frequency band of at least one designated whistle sound generator and corresponding to the whistle sound pattern of the whistle sound generator and recognizing the whistle sound The camera by using at least s (1≤s≤S) service functions related to the whistle among the S (S≥1) service functions specified in the case and any one or more of the short-distance module, communication module, and camera module. A control module including a service control unit for activating a video service function for photographing a sound generating source that generates the recognized whistling sound through a device, wherein the control module includes M (M≥1) whistle sound frequency bands. An information storage unit for storing M whistle frequency band information corresponding to and N whistle pattern information corresponding to N (N≥1) whistle patterns in a designated storage area, and the M whistle frequency band information generate whistle frequency distribution information corresponding to at least i (i≥2) times validly recognized whistle frequency distribution for a specified period of time, and use the N whistle sound pattern information for a specified period of time at least j ( j≥2) validly recognized whistles Optimize (or customize) the whistle sound of the whistle sound generator in the space where the sound device passes through a distribution generator that generates whistle sound pattern distribution information corresponding to the pattern distribution and statistically processes the generated whistle sound frequency distribution information m whistle frequency band information corresponding to the m (1≤m≤M) frequency bands of whistle sounds generated, statistically processing the generated whistle sound pattern distribution information, By reading the buffered sound data and an information determination unit that determines n whistle sound pattern information corresponding to n (1≤n≤N) whistle sound patterns optimized (or customized) for the whistling sound of the sound generator, A speech contents recognition unit that recognizes speech contents and recognizes speech contents information corresponding to the user's speech contents by recognizing language utterances and processing them in natural language If the user's speech content is recognized within a certain time during or after the whistling sound is recognized, the recognition determination unit confirms whether the user's speech content is recognized within s number related to the whistling sound. It is characterized in that it further comprises an intelligent processing unit for processing to interpret in association with a service function and a service providing unit for providing a service corresponding to the activated service function.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

According to the present invention, the whistle recognizing unit reads the buffered sound data, corresponds to at least one whistle frequency band among the M whistle frequency bands, and selects at least one whistle pattern among the N whistle patterns. It is characterized in that it comprises a function of recognizing the whistling sound corresponding to.

delete

delete

According to the present invention, the information storage unit further includes a function of setting the determined m whistle frequency bands as firstly read whistle frequency bands and storing new or updated sounds, wherein the whistle recognition unit is configured to set the buffered sound. and a function of first recognizing a whistle sound corresponding to at least one whistle sound frequency band among the m number of whistle sound frequency bands by reading the data.

delete

delete

According to the present invention, the information storage unit further includes a function of setting the determined n whistle sound patterns as whistle patterns to be read first and storing new or updated sounds, and the whistle sound recognition unit stores the buffered sound data and a function of first recognizing a whistle sound corresponding to at least one whistle sound pattern among the n number of whistle sound patterns stored above.

delete

delete

According to the present invention, the service control unit may include a function of activating a specific service function related to the whistling sound and corresponding to the recognized user's utterance.

According to the present invention, the s services are a server interworking service provided in conjunction with an operation server on a communication network through the communication module, a sound-based service provided through the sound module, and a combination of the sound module and the communication module It may include at least one service among complex services provided by

According to the present invention, the server interworking service includes at least one of a paging service using wireless terminal information, a situation notification service using wireless terminal information, a paging service using an app installed in a wireless terminal, and a situation notification service using an app installed in a wireless terminal. It may include at least one of a video service that provides video information captured through a camera of the camera to an app installed in a wireless terminal and a video service that provides video information captured through at least one camera to a designated IPTV device.

According to the present invention, the sound-based service may include a noise output service that outputs a designated service sound at a designated signal output strength through a sound output unit of a sound module.

According to the present invention, the signal output strength may include a signal strength that is inversely proportional to the signal input strength of the whistling sound.

According to the present invention, the complex service recognizes the user's speech within a certain time during or after the whistling sound is recognized, and associates the user's speech with a service function related to the whistling sound. It is characterized by comprising a service provided through an operation server on a communication network.

delete

delete

delete

delete

delete

delete

delete

delete

According to the present invention, when at least one whistle designated for a specific reimbursement determination is not recognized among non-verbal sounds or noises that may occur in a designated space, at least one whistle designated to recognize the whistle while operating in a sleep mode When a specific whistle corresponding to a frequency band and/or corresponding to at least one whistle pattern designated for recognizing the whistle is recognized and automatically wakes up from the sleep mode, at least service related to the whistle is recognized. There is an advantage of providing a sound device that activates a function or provides a service related to the whistling sound.

1 is a diagram showing the functional configuration of a sound device equipped with a whistling sound recognition function according to an embodiment of the present invention.
2 is a diagram showing the functional configuration of a sound device and an operating server according to another embodiment of the present invention.
3 illustrates an embodiment of recognizing a whistle sound and activating a service related to a whistle sound according to an embodiment of the present invention.
4 illustrates an embodiment of optimizing (or customizing) a whistle frequency band for recognizing a whistle to a whistle sound of a whistle sound generator according to an embodiment of the present invention.
5 illustrates an embodiment of optimizing (or customizing) a whistle sound pattern for recognizing a whistle sound to the whistle sound of a whistle sound generator according to another embodiment of the present invention.
FIG. 6 illustrates an embodiment of activating a whistle-related service by linking and recognizing a whistle sound and an utterance according to another embodiment of the present invention.

Hereinafter, the operating principle of the preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings and description. However, the drawings shown below and the description below relate to preferred implementation methods among various methods for effectively explaining the features of the present invention, and the present invention is not limited to only the drawings and description below.

That is, the following examples correspond to examples of preferred union types among numerous embodiments of the present invention, and examples in which specific components (or steps) are omitted or specific configurations (or steps) are omitted in the following embodiments. Embodiments in which functions implemented in are divided into specific configurations (or steps), or embodiments in which functions implemented in two or more configurations (or steps) are integrated into any one configuration (or steps), specific configurations (or steps) It is clearly stated that the embodiments in which the operation sequence of is replaced are all within the scope of the present invention, even if not separately mentioned in the following embodiments. Therefore, it is clearly stated that various examples corresponding to subsets or complements can be divided retroactively from the filing date of the present invention based on the following examples.

In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, the definition should be made based on the contents throughout the present invention.

As a result, the technical spirit of the present invention is determined by the claims, and the following embodiments are a means for efficiently explaining the technical spirit of the present invention to those skilled in the art to which the present invention belongs. only

Figure 1 is a diagram showing the functional configuration of a sound device 100 equipped with a whistling sound recognition function according to an embodiment of the present invention.

In more detail, Figure 1 shows a communication including a sound module 105 including a sound input unit 105a receiving a sound signal and a sound output unit 105b outputting a sound signal and at least one communication unit connected to a communication network. A sound processing unit having a module 120 and buffering sound data corresponding to a sound signal of an audible frequency band input through the sound input unit 105a in a designated buffer area or checking sound data buffered in a designated buffer area ( 140) and at least one whistle frequency band designated to recognize at least one whistle sound designated for determining a specific situation among non-verbal sounds or noises that may occur in a designated space by reading the buffered sound data, and/or Among the whistle recognition unit 145 recognizing a specific whistle sound corresponding to at least one designated whistle pattern to recognize the whistle sound and S (S≥1) service functions designated when the whistle sound is recognized As a functional configuration of a sound device 100 having a control module 130 including a service control unit 150 for activating at least s (1≤s≤S) service functions related to the whistling sound, the present Those skilled in the art to which the invention pertains may refer to and/or modify this Figure 1 in various implementation methods for the sound device 100 (eg, some components are omitted, subdivided, or combined). Implementation method) may be inferred, but the present invention includes all the implementation methods inferred above, and its technical characteristics are not limited only to the implementation method shown in FIG. 1.

Referring to the embodiment of FIG. 1, the sound device 100 includes a sound module 105 including a sound input unit 105a receiving a sound signal and a sound output unit 105b outputting a sound signal; It has a communication module 120 including at least one communication unit connected to a communication network, and reads sound data corresponding to a sound signal of an audible frequency band input through the sound input unit 105a to generate non-verbal information that can occur in a designated space. Corresponding to at least one whistle frequency band designated for recognizing at least one whistle sound designated for determining a specific situation among sounds or noises and/or corresponding to at least one whistle sound pattern designated for recognizing the whistle sound A control module 130 having one or more functional components for activating at least one designated service function related to the whistling sound or providing the designated service function when a specific whistling sound is recognized by attempting to recognize it, and an operating power supply Equipped with a power module 115 for supplying. Meanwhile, according to the implementation method, the sound device 100 includes at least one of the short-distance module 125 for processing short-distance wireless communication with the short-distance camera device 185 and the camera module 110 for acquiring image information of a photographed subject. A module may be further provided, and in some cases, an input unit (not shown) including a key button or a touch pad for receiving a user's operation may be further provided, and/or an LED or display for outputting various operating states or information. An included output unit (not shown) may be further provided. On the other hand, in the embodiment of Figure 1, the name "module" is only a term for explaining the function and role of the sound device 100 for convenience, and it is clearly clarified that it does not refer to any independent hardware configuration. it is put

The sound module 105 is a generic name for components related to sound input/output including one or more of a sound input unit 105a receiving a sound signal and a sound output unit 105b outputting a sound signal among components of the sound device 100. , Preferably, the sound input unit 105a and the sound output unit 105b can input or output sound signals of at least an audible frequency band, and the sound input unit 105a or the sound output unit 105b is used as a control module 130 and an interface unit (not shown) that interfaces with.

The sound input unit 105a is a general term for a component that receives a sound signal, and encodes the sound signal according to one or more microphones and a designated encoding rule for receiving a sound signal that detects air vibration and converts it into an electrical signal. It is configured to include a program (and/or circuit) module that generates sound data. Preferably, the sound input unit 105a receives a sound signal of an audible frequency band (eg, a sound signal including a voice signal uttered by a user) through the microphone, and outputs the input audio frequency band according to a designated encoding rule. Sound data generated by encoding the sound signal of may be provided to the control module 130 .

According to the implementation method of the present invention, the sound input unit (105a) includes one or more directional microphones to detect the direction in which the whistling sound is generated when the whistling sound is recognized in the space where the sound device 100 is mounted. may be implemented, and the present invention is not limited thereby.

The sound output unit 105b is a generic term for a component that outputs a sound signal, and checks one or more speakers that output sound signals and sound data to be output through the speakers and decodes them into the sound signals that can be output through the speakers. It is configured to include a program (and/or circuit) module that controls the signal output strength and output quality of the sound signal while decoding. Preferably, the sound output unit 105b outputs a sound signal generated by decoding the sound data requested to output sound through the control module 130 through the speaker, and the signal of the sound signal output as sound Control the output strength to a specified signal strength (e.g., amplify or reduce the signal strength set to enable receiving (or listening) with a signal strength higher than the specified signal reception strength at a distance greater than a specified reference distance, and/or even in the presence of a specified obstacle) Receive (or listen) at a signal strength higher than the specified signal reception strength at a specified distance, amplify or reduce to a set signal strength, and/or receive at a specified distance at a signal strength higher than the specified signal reception strength even in an environment where noise of a specified decibel or higher exists Sound distortion is minimized by outputting sound by amplifying or reducing (or listening) to a set signal strength) and/or controlling the output quality (eg, speaker parameters such as Fs. Qts, Vas, etc.) of the output sound signal. Sound output by blocking or suppressing (eg, output by controlling the output quality of sound to block / suppress sound distortion caused by resonance of a speaker, or in the case of a voice signal, so that audibility (or hearing sensitivity) can be maintained at a certain level or higher Controls the output quality of the sound so that the sound components corresponding to the articulations on the voice signal are clearly distinguished and outputs it, or blocks/suppresses distortion of the frequency at which the bit string is encoded or signal strength when bit strings are encoded and output in the sound signal. It can be output by controlling the output quality of the sound so that it can be output).

The communication module 120 is a general term for components that process the communication network connection of the sound device 100, and is preferably a designated server (or device) via a wireless network (eg, mobile communication network, mobile Internet, etc.) via a base station. and a wireless network communication unit 120a communicating by connecting a communication channel with a local area network (eg, wireless LAN, etc.) via a wireless AP or repeater to communicate by connecting a designated server (or device) and a communication channel to communicate At least one communication unit among the network communication units 120b is provided, and an interface unit (not shown) for interfacing the communication unit with the control module 130 is further provided.

The wireless network communication unit 120a is a generic term for a communication component that connects the sound device 100 to a wide area wireless network (eg, mobile communication network, etc.) via a base station, and transmits and receives a radio frequency signal of a designated frequency band. An antenna, It is configured to include at least one RF module, a baseband module, and a signal processing module, and is connected to the control unit 130a to transmit operation results corresponding to wireless communication among various operation results of the control unit 130a through a wireless network. Alternatively, data is received through a wireless network and transmitted to the control unit 130a, and at the same time, procedures of connection, registration, communication, and handoff of the wireless communication are performed.

According to the implementation method of the present invention, when the wireless network accessed through the wireless network communication unit 120a is a mobile communication network, the wireless network communication unit 120a accesses and registers a location according to at least one mobile communication standard. , call processing, call connection, data communication, and a mobile communication configuration that performs at least one handoff. On the other hand, according to the intention of those skilled in the art, the wireless network communication unit 120a performs at least one of connection, location registration, data communication, and handoff to a wide area wireless network other than the mobile communication network (eg, portable Internet according to IEEE 802.16 standards). It is clear that the present invention is not limited by the wireless communication configuration provided by the wireless network communication unit 120a. That is, the wireless network communication unit 120a provides wireless data communication based on packet communication via the cell-based base station on the wireless network for the sound device 100 regardless of the frequency band of the wireless section or the type or protocol of the communication network. It is a general term for components connected to the data network.

The local area network communication unit 120b uses a radio frequency signal as a communication medium within a certain distance (eg, within 10 m, or within 30 m, etc.) for short-range wireless communication with a network access device (eg, wireless AP, etc.) provided in a short distance. A general term for communication resources that connect the sound device 100 to a communication network connected by the network access device through a local area wireless network that connects a session, preferably WiFi-based local area network communication, Bluetooth local area network communication, Zigbee-based local area network A short-range wireless communication session is connected between a network connection device provided in a short distance and the sound device 100 through at least one of communication, UWB-based local area network communication, and public wireless communication, and based on this, the sound device 100 is It can be connected to the communication network to which the network access device is connected.

According to the implementation method of the present invention, the local area network communication unit 120b connects a local area wireless communication session with a network connection device provided in a local area within a certain distance from the sound device 100 when a local area wireless network is implemented. It is a general term for components that connect the sound device 100 to a data network that provides wireless data communication based on packet communication via the network access device of the local area network. Hereinafter, for convenience, the network communication function of the sound device 100 will be mainly described using the wireless network communication unit 120a, but the present invention is not limited thereto, and the network communication function using the local area network communication unit 120b. It is also clearly stated that it is provided.

Meanwhile, according to one example of the implementation method of the present invention, the local area network communication unit 120b connects the camera device 185 provided in the local area of the sound device 100 via the local area wireless network and a local area wireless network-based wireless network. communication can be handled. When the local area network communication unit 120b processes wireless communication based on a local area network with the camera device 185 in the local area, the sound device 100 and the camera device 185 perform wireless communication based on the local area wireless network. It is possible to store a key value for encrypted communication or to exchange a key value for encrypted communication at the time of connecting wireless communication based on a local area network, whereby the sound device 100 and the camera device 185 Encrypted communication using key values can be performed.

The short-distance module 125 is a general term for components that process short-range wireless communication of the sound device 100, and includes a short-range wireless unit 125a that connects short-range wireless communication of a predetermined standard, and the short-range wireless unit ( 125a) and an interface unit (not shown) that interfaces the control module 130. For example, the short-range wireless unit 125a may process Bluetooth-based short-range wireless communication according to the Bluetooth standard. However, it is clearly stated that the standard for short-range wireless communication of the short-range wireless unit 125a is not limited to the Bluetooth standard, and may include all short-range wireless communication standards including Wi-Fi, ZigBee, and UWB. Meanwhile, depending on the implementation method, the local area wireless unit 125a may be the local area network communication unit 120b.

The short-distance radio unit 125a is a general term for a component that processes short-range wireless communication of a designated standard, and preferably may include a Bluetooth component that processes Bluetooth-based short-range wireless communication according to the Bluetooth standard. However, the short-range wireless communication-related configuration of the short-range wireless unit 125a is not limited to Bluetooth-based short-range wireless communication according to the Bluetooth standard, and short-range wireless communication of various standards other than Bluetooth may be used according to the intention of a person skilled in the art (e.g., , WiFi-based short-range wireless communication according to the Wi-Fi standard, ZigBee-based short-range wireless communication according to the ZigBee standard, UWB-based short-range wireless communication according to the UWB standard, etc. available) is clearly stated. Hereinafter, for convenience, the characteristics of the present invention will be described mainly in an embodiment in which the short-range wireless unit 125a processes Bluetooth-based short-range wireless communication, but it is clearly stated that the present invention is not limited thereto.

According to the implementation method of the present invention, the short-distance module 125 (or the short-distance wireless unit 125a) processes the short-range wireless communication of a specified standard with the camera device 185 provided in the short-range of the sound device 100. can When the short-distance module 125 (or short-distance wireless unit 125a) processes short-range wireless communication with the short-range camera device 185, the sound device 100 and the camera device 185 operate based on short-range wireless communication. It is possible to store a key value for encrypted communication or to exchange a key value for encrypted communication at the time of connecting short-range wireless communication, whereby the sound device 100 and the camera device 185 use the key value. Encrypted communication can be performed.

According to the implementation method of the present invention, when the sound device 100 and the app of the wireless terminal used by the user are mapped (e.g., when a designated app is installed in the user wireless terminal and executed for the first time (or executed before mapping registration), the user wireless terminal Device identification information for identifying the sound device 100 in the terminal app and identification information of the user's wireless terminal (or unique identification information of the user's wireless terminal app) are registered in the designated operation server 200, and the sound device 100 and the application of the user's wireless terminal are matched and registered), the short-range wireless unit 125a connects the matched app of the user's wireless terminal and short-range wireless communication to provide various identification information or processing information specified to process a specific service. can be exchanged For example, information received as an app of a user's wireless terminal from a designated operation server 200 or input, extracted, or generated through an app of the user's wireless terminal is transmitted through the short-range wireless communication to the stationary sound device 100. It is transmitted to the control module 130 and processed (for example, sound output or transmitted to the operation server 200) and/or received by the sound device 100 from the designated operation server 200 or the sound device 100 for installation. ) The information recognized, extracted, or generated from may be transmitted to the app of the user's wireless terminal through the short-range wireless communication and processed (eg, screen output or transmitted to the operation server 200).

The camera module 110 is a general term for a component that processes camera shooting of the sound device 100, and preferably obtains image information (eg, image) or image information (eg, video) of a photographed subject (eg, video) 110a), and an interface unit (not shown) that interfaces the camera unit 110a and the control module 130. On the other hand, if the short-distance module 125 (or short-range wireless unit 125a) and/or the short-range camera device 185 can be interlocked through the short-range wireless unit, the camera module 110 (or the camera unit 110a) It can be omitted, and the present invention is not limited thereby.

The camera unit 110a includes at least one lens unit that receives an optical signal reflected from a subject, an image sensor unit that detects an optical signal input through the lens unit and converts it into an electrical image signal, and a gap between the lens unit and the image sensor unit. It includes an actuator unit implementing an auto-focus function or an image stabilization function, and according to an implementation method, one or more filter units (eg, an infrared filter, etc.) may be further provided between the lens unit and the image sensor unit. can Meanwhile, the camera unit 110a includes an Image Signal Processor (ISP) that processes the image information or image information according to a designated procedure to improve the quality of the image information or image information generated through the image sensor unit. more can be provided.

Meanwhile, when the camera module 110 is provided in the sound device 100, the camera module 110 includes a driving unit for changing the shooting direction of the camera unit 110a, and the control module 130 ( Alternatively, the camera module 110 controls the driving unit to change the shooting direction of the camera unit 110a and reads image information or image information generated through the image sensor unit of the camera unit 110a to generate the sound device 100. It is possible to recognize a sound generating source (eg, a person or object generating a whistling sound) in the same space as the space where it is mounted, and operate the drive unit to photograph the sound generating source through the camera unit 110a. On the other hand, when one or more directional microphones are included in the sound input unit 105a provided in the sound module 105, the control module 130 (or camera module 110) is the sound input unit of the sound module 105 ( 105a) may check the detected whistle direction through one or more directional microphones, and operate the drive unit to photograph the detected whistle direction through the camera unit 110a.

The control module 130 is a general term for components that control various operations of the sound device 100, and is preferably an operation of generating image data through the camera module 110 and a sensor module through the short-distance module 125. A control unit 130a for controlling at least one operation or a combination of two or more of an operation of collecting sensing data and an operation of communicating with a designated server (eg, operation server 200) through the communication module 120 and a memory unit for storing various information or program codes necessary for the operation of the control unit 130a.

The control unit 130a is a general term for components that control the operation of the sound device 100, and is configured to include at least one processor and execution memory and/or various chips or control circuits, and is configured to include the sound It is connected to each component provided in the device 100 through a designated interface (eg, a bus (BUS), etc.). According to the present invention, the controller 130a loads and calculates at least one program code provided in the sound device 100 into the execution memory through the processor, and transmits the result to at least one component through the bus. and controls the operation of the sound device 100. The functional configuration of the present invention may be implemented in the form of program code and/or in the form of a chip or control circuit. Hereinafter, for convenience, it is implemented in the form of a program code, stored in a memory unit, and loaded and operated in the form of a program and/or chip. However, functional components of the present invention implemented in the form of a control circuit will be illustrated and described in the control unit 130a.

The memory unit is a generic term for non-volatile memory corresponding to storage resources of the sound device 100, and stores at least one program code executed through the control unit 130a and at least one data set used by the program code. and keep it The memory unit basically includes a system program code and system data set corresponding to the operating system of the sound device 100, a communication program code and communication data set for processing wireless communication connection of the sound device 100, and at least one application program. Codes and application data sets are stored, and program codes and data sets corresponding to functional configurations implementing the present invention may also be stored in the memory unit.

The power module 115 is a general term for a component that supplies/applies power necessary for the operation of the sound device 100, and preferably uses an external power source (eg, AC power, DC power obtained by rectifying AC power, USB power, etc.) It is provided with a power supply unit (not shown) for receiving and applying, and when the voltage value or current value of the external power source is different from the voltage value or current value required for the operation of the sound device 100, the external power source is supplied to a specified voltage value or current value. It further includes a power converter (not shown) that converts power into operating power having a current value. Meanwhile, when the external power is supplied through the power supply unit or the external power is converted into operating power through the power conversion unit, the power module 115 charges the external power or operation power and operates the sound device 100. When external power is not supplied, a power charging unit (not shown) may be further provided to apply the charged charging power.

According to the first embodiment of the present invention, the sound device 100 recognizes a designated whistling sound in a designated space and provides a service related to the whistling sound. can be

According to the second embodiment of the present invention, the sound device 100 provides a voice recognition operation mode that provides a well-known voice recognition based service, but sets a specific setting through a predetermined input unit provided in the sound device 100. It may be a mode switching sound device 100 that switches from the voice recognition operation mode to the whistle recognition operation mode for the whistle recognition based service according to the present invention based on specific voice recognition through input or voice recognition.

According to the third embodiment of the present invention, the sound device 100 is a composite sound device designed to provide a well-known voice recognition-based service and at the same time recognize a whistle according to the present invention and provide multiple services based on whistle recognition. (100).

Referring to FIG. 1, the control module 130 selects recognition target information to be recognized in order to initiate or process a specific service among speech content information to be recognized through the speech recognition unit of the control module 130 and acquired through natural language processing. It is provided with an information storage unit 135 that confirms and stores it in a designated storage area of the memory unit.

When it is necessary to recognize the user's utterance content to determine the start or process of a specific service, the information storage unit 135 provides recognition target information including the utterance content to be recognized from the user's utterance in order to initiate or process the specific service. It is possible to check and store service provision information and mapping for initiation or processing of the specific service.

According to the implementation method of the present invention, the recognition target information to be mapped with the service provision information is specified including the previously designated utterance content, and/or is provided to the user through the sound output unit 105b of the sound device 100. After requesting the content of speech to be used as the recognition target information, it may be received through the sound input unit 105a, acquired through natural language processing, and registered/updated. Meanwhile, according to the implementation method, the recognition target information may be registered/updated through an app of a user wireless terminal mapped with the sound device 100, and the present invention is not limited thereto.

Referring to FIG. 1, the control module 130 buffers sound data corresponding to a sound signal of an audible frequency band input through the sound input unit 105a in a designated buffer area or buffers sound data buffered in a designated buffer area. a sound processing unit 140 that checks the buffered sound data and at least one whistle frequency designated to recognize at least one whistle sound designated for determining a specific situation among non-verbal sounds or noises that may occur in a designated space A whistling sound recognizing unit 145 for recognizing a specific whistling sound corresponding to a band and/or corresponding to at least one whistle pattern designated for recognizing the whistling sound, and a designated S when recognizing the whistling sound ( When at least s (1≤s≤S) service functions related to whistles are activated among S≥1) service functions and/or whistles or user's utterances are not recognized for a specified period of time, the sound device ( 100) is provided with a service controller 150 that sets the operation mode to a sleep mode, and a service provider 180 that provides a service corresponding to the activated service function is further provided. Meanwhile, according to another implementation method of the present invention, at least one of the whistling sound recognition unit 145, the service control unit 150, and the service providing unit 180 is an operation server on a communication network that works with the sound device 100. 200, and if the whistling sound recognition unit 145 is implemented in the operation server 200, the sound processing unit 140 transmits sound data buffered in a designated buffer area to the operation server 200. ) Or, among the buffered sound data, sound data determined to correspond to a whistle may be selectively extracted and provided to the operation server 200. However, for convenience, Figure 1 describes the features of the present invention, focusing on an embodiment in which the whistling sound recognition unit 145, the service control unit 150, and the service providing unit 180 are implemented through the sound device 100. . Meanwhile, when the whistling sound recognition unit 145, the service control unit 150, and the service providing unit 180 are implemented through the sound device 100, the whistling sound recognition unit 145, the service control unit 150, and the service The study 180 may be implemented in the form of at least one or a combination of two of a program code form and a chip or control circuit form, and the whistle sound recognition unit 145, the service control unit 150, and the service provider 180 The present invention is not limited by an implementation method or implemented module.

After operating power is applied to the sound device 100 through the power module 115, the sound processing unit 140 generates sound data corresponding to a sound signal of an audible frequency band input through the sound input unit 105a. The process of buffering in the designated buffer area and/or checking the sound data buffered in the designated buffer area is continued (or repeated), and the whistling sound recognition unit 145 reads the buffered sound data in the designated space. Corresponds to at least one whistle sound frequency band designated for recognizing at least one whistle sound designated for determining a specific situation among possible non-verbal sounds or noises and/or at least one whistle sound pattern designated for recognizing the whistle sound Continue (or repeat) the process of recognizing a specific whistling sound corresponding to Meanwhile, when the whistling sound recognition unit 145 is implemented in the operation server 200, the sound processing unit 140 provides sound data buffered in a designated buffer area to the operation server 200 or the buffered sound data Sound data determined to correspond to the middle whistling sound may be selectively extracted and provided to the operating server 200, and the present invention is not limited thereby.

The service controller 150 checks whether a whistling sound is not recognized for a predetermined period of time or longer after operating power is applied to the sound device 100 and/or effectively recognizes a whistling sound through the whistling sound recognizing unit 145. After that, it is checked whether the whistling sound is not recognized for a certain period of time or not, and/or the sound device 100 is powered on, and then it is checked whether the content of the user's utterance is not recognized for a certain period of time or not. After the user's speech content is effectively recognized according to a designated procedure (eg, the user's speech content is recognized during or after the whistling sound is effectively recognized through the whistling sound recognition unit 145), the user's speech content is recognized for a specified period of time or more It can be checked whether the contents of the utterance are not recognized. If the whistling sound is not recognized for a certain period of time or the content of the user's speech is not recognized for more than a certain period of time, the service control unit 150 sets the operation mode of the sound device 100 (or control module 130). It includes a function to set to sleep mode. Here, the sleep mode reads the buffered sound data to wake up from the sleep mode by recognizing a whistle even if the content of the user's utterance is recognized, and/or predetermined specific speech information (e.g., whistle sound). In addition, before waking up from the sleep mode by recognizing specific speech information set to wake up the sound device 100, services based on the recognized user's speech (eg, sound source or signal sound) are recognized. This includes a state in which a function providing a sound-based service) is disabled. Hereinafter, for convenience, features of the present invention will be described focusing on an embodiment in which a whistle is recognized to wake up from a sleep mode.

According to the implementation method of the present invention, the sound processing unit 140 buffers sound data corresponding to the sound signal of the audible frequency band input through the sound input unit 105a in a designated buffer area even in the sleep mode, and/or It is preferable to continue (or repeat) the process of checking the sound data buffered in the designated buffer area, and the whistling sound recognition unit 145 also reads the buffered sound data in the sleep mode to generate non-verbal information that can occur in the designated space. Corresponding to at least one whistle frequency band designated for recognizing at least one whistle sound designated for determining a specific situation among sounds or noises and/or corresponding to at least one whistle sound pattern designated for recognizing the whistle sound It is desirable to continue (or repeat) the process of recognizing a particular whistle. Preferably, in the sleep mode, when a whistling sound is effectively recognized through the whistling sound recognition unit 145, the service control unit 150 wakes up to activate s number of service functions related to the whistling sound. The service providing unit 180 may perform a procedure of providing services corresponding to the activated s number of service functions.

Meanwhile, when two or more service functions related to the whistling sound are activated through the service control unit 150, the service providing unit 180 performs a procedure for providing a service corresponding to the activated service function according to a designated order, or And/or a procedure for providing at least one service according to a predetermined condition (eg, a condition for recognizing user's speech) among the activated service functions may be performed.

According to the implementation method of the present invention, the whistle recognition unit 145 corresponds to the whistle frequency band of the whistle sound generator and/or the unique whistle sound of the whistle sound generator corresponding to the whistle sound pattern of the whistle sound generator. For this purpose, the control module 130 has a function of learning at least one whistle frequency band specific to a whistle sound generating source and/or a function of learning at least one whistling sound pattern specific to a whistling sound generating source. It is desirable to have.

Referring to Figure 1, the control module 130 stores M whistle sound frequency band information corresponding to M (M≥1) whistle sound frequency bands in a designated storage area and/or N (N≥1) whistle sound frequency band information. and an information storage unit 135 for storing N whistle pattern information corresponding to ) number of whistle patterns in a designated storage area, and the whistle sound recognition unit 145 stores whistle sound frequency bands in the designated storage area. A whistle may be recognized using information and/or whistle pattern information. On the other hand, according to another implementation method of the present invention, at least one component of the information storage unit 135 and the whistling sound recognition unit 145 is implemented through the operating server 200 on a communication network that works with the sound device 100. It can be. However, for convenience, the features of the present invention will be described mainly in the embodiment in which the information storage unit 135 and the whistling sound recognition unit 145 are implemented through the sound device 100 in FIG. 1 for convenience. On the other hand, when the information storage unit 135 and the whistle recognition unit 145 are implemented through the sound device 100, the information storage unit 135 and the whistle recognition unit 145 are the program code form and chip or control. It may be implemented in at least one or a combination of two circuit types, and the present invention is not limited by a method of implementing the information storage unit 135 and the whistling sound recognition unit 145 or a module to be implemented.

The information storage unit 135 stores M whistling sounds corresponding to M (M≥1) frequency bands of whistles in a designated storage area of the memory unit at the time of manufacturing the control module 130 (or sound device 100). Frequency band information may be stored. Among the non-verbal sounds or noises that can be generated in a designated space in the present invention, the whistling sound designated for determining a specific situation has a wide range (or a frequency difference of at least 100 Hz or more between the lowest frequency and the highest frequency) depending on the type of sound or sound source. It can have a frequency band. Therefore, the present invention can divide the frequency band of the whistling sound into M frequency bands (eg, equal division or experimentally selective division based on a plurality of frequency bands in which the whistling sound is concentrated), and the information storage unit 135 may store M whistle sound frequency band information corresponding to the divided M whistle sound frequency bands in a designated storage area.

The information storage unit 135 stores N whistle sound patterns corresponding to N (N≥1) whistle sound patterns in a designated storage area of the memory unit at the time of manufacturing the control module 130 (or sound device 100). Information can be stored in a designated storage area. The present invention first generates whistle pattern information through qualitative analysis and/or quantitative analysis of the whistle pattern confirmed through the sample experiment for each case, and the information storage unit 135 generates the whistle pattern. Information can be stored in a designated storage area.

On the other hand, according to the implementation method of the present invention, the sound device 100 (or control module 130) may recognize whistle sound and then transmit the recognized whistle sound information to the designated operation server 200, and the operation The server 200 collects whistle sound information from each sound device 100 (or control module 130), and normalizes individual sound characteristics for each sound source in the collected whistle sound. Normalization process and / Or through the process of qualitative analysis and / or quantitative analysis for each case, the whistle sound pattern information can be refined or diversified, and the refined or diversified whistle pattern information in this way can be provided to each sound device 100 (or control module). (130)). The information storage unit 135 receives refined or diversified whistle sound pattern information from the operation server 200 through the above process, and generates N whistle sound pattern information including the received whistle sound pattern information. It can be stored in the designated storage area.

The information storage unit 135 matches the generated whistle sound pattern information with one or more situation judgment attribute information for determining a specific situation in a designated space at the time when the sound device 100 recognizes the whistle sound, and then matches the designated can be stored in the storage area.

Here, the specific situation may be an emergency situation, a specific person's homecoming situation, a specific person's visit situation, and the like, but is not limited thereto, and various situations may be presented in the course of practice by those skilled in the art.

According to the implementation method of the present invention, M whistle frequency band information corresponding to M whistle frequency bands is stored in the designated storage area, and/or N whistles corresponding to N whistle patterns are stored in the designated storage area. When sound pattern information is stored, the whistle sound recognition unit 145 reads the buffered sound data to correspond to at least one whistle sound frequency band among the stored M whistle sound frequency bands and/or the stored N whistle sound frequency bands. It is preferable to recognize a whistle corresponding to at least one whistle pattern among the whistle sound patterns.

Meanwhile, the M pieces of whistle frequency band information and/or N pieces of whistle pattern information stored in the designated storage area are not optimized (or customized) for the whistle sound of the whistle sound generator, but statistically the frequency band and/or sound of the whistle sound. It will be information corresponding to the pattern. Therefore, in order to recognize the whistle of the whistle sound generator existing in the space where the sound device 100 is mounted at high speed and/or prevent (or minimize) recognition errors, the stored whistle frequency band information and/or the whistle sound A process of optimizing (or customizing) the pattern information to the whistling sound of the whistling sound generating source is required.

Referring to FIG. 1, the control module 130 controls whistle frequencies corresponding to whistle frequency distributions that have been recognized validly at least i (i≥2) times during a specified period of time using the M pieces of whistle frequency band information. Distribution generation that generates distribution information and/or generates whistle pattern distribution information corresponding to the validly recognized whistle pattern distribution at least j (j≥2) times during a specified period of time using the N pieces of whistle pattern information. The unit 155 statistically processes the generated whistle sound frequency distribution information to optimize (or customize) the whistle sound of the whistle sound generator in the space where the sound device 100 passes through m (1≤m≤M m whistle frequency band information corresponding to ) number of whistle frequency bands and/or statistically processing the generated whistle pattern distribution information to determine the whistle sound generating source in the space through which the sound device 100 has passed through An information determination unit 160 for determining n whistle sound pattern information corresponding to n (1≤n≤N) whistle sound patterns optimized (or customized) for whistles is further provided. On the other hand, according to another implementation method of the present invention, at least one component of the distribution generating unit 155 and the information determining unit 160 is implemented through the operation server 200 on a communication network that works with the sound device 100. can However, for convenience, the features of the present invention will be described in FIG. 1 based on an embodiment in which the distribution generating unit 155 and the information determining unit 160 are implemented through the sound device 100. Meanwhile, when the distribution generating unit 155 and the information determining unit 160 are implemented through the sound device 100, the distribution generating unit 155 and the information determining unit 160 are in the form of program codes and chips or control circuits. It may be implemented in the form of at least one or a combination of the two, and the present invention is not limited by a method of implementing the distribution generating unit 155 and the information determining unit 160 or a module to be implemented.

When M whistle sound frequency band information corresponding to the M whistle sound frequency bands is stored in the storage area designated through the information storage unit 135, the distribution generating unit 155 stores the M whistles stored in the designated storage area. Whenever a whistling sound is validly recognized using the sound frequency band information, the validly recognized frequency band information of the whistling sound may be stored in a designated storage area. The distribution generation unit 155 operates i whistle frequency bands that are validly recognized at least i (i≥2) times during a designated period based on the whistle frequency bands that have been validly recognized and stored through the information on the M number of whistle frequency bands. , and whistle frequency distribution information corresponding to the distribution of the i identified whistle frequency bands may be generated.

Meanwhile, when whistle frequency distribution information corresponding to the whistle frequency distribution that has been recognized validly at least i during a specified period of time is generated by the distribution generator 155, the information determination unit 160 determines the generated whistle frequency Distribution information is statistically processed to optimize (or customize) the whistle sound of the whistle source in the space through which the sound device 100 has passed through, thereby recognizing the whistle sound of the whistle source at high speed or preventing (or minimizing) recognition errors It is possible to determine m whistle sound frequency band information corresponding to m (1≤m≤M) whistle sound frequency bands for Here, the m whistle frequency bands may be any one of the M whistle frequency bands, or may include a newly defined whistle frequency band within the whistle frequency band range. When the m whistle frequency band information is determined through the information determination unit 160, the information storage unit 135 sets the determined m whistle frequency bands as whistle frequency bands to be read first and stores the designated information. It may be newly stored in the storage area or updated and stored whistling sound frequency band information stored in the storage area. Thereafter, the whistle recognizing unit 145 reads the buffered sound data and first recognizes a whistle corresponding to at least one whistle frequency band among the new or updated m whistle frequency bands.

Meanwhile, when N whistle sound pattern information corresponding to the N whistle sound patterns is stored in the designated storage area through the information storage unit 135, the distribution generating unit 155 stores the N whistle sound patterns stored in the designated storage area. Whenever a whistling sound is validly recognized using the pattern information, the sophit pattern of the validly recognized whistling sound may be stored in a designated storage area. The distribution generating unit 155 verifies j whistle patterns that have been validly recognized and stored through the N whistle sound pattern information at least j (j≥2) times during a specified period of time based on the stored whistle sound pattern information. and generate whistle pattern distribution information corresponding to the checked distribution of j whistle sound patterns.

Meanwhile, when whistle pattern distribution information corresponding to the whistle sound pattern distribution that has been recognized validly at least j times during a specified period of time is generated by the distribution generating unit 155, the information determining unit 160 determines the generated whistle pattern. Distribution information is statistically processed to optimize (or customize) the whistle sound of the whistle source in the space through which the sound device 100 has passed through, thereby recognizing the whistle sound of the whistle source at high speed or preventing (or minimizing) recognition errors It is possible to determine n whistle sound pattern information corresponding to n (1≤n≤N) whistle sound patterns for Here, the n whistle patterns may be any one of the N whistle patterns, or may include a combination of two or more whistle patterns or a new whistle pattern. When n whistle sound patterns are determined through the information determination unit 160, the information storage unit 135 sets the determined n whistle sound patterns as whistle sound patterns to be read first and newly stores them in a designated storage area. Whistling sound frequency band information stored in the storage area may be updated and stored. Thereafter, the whistle recognition unit 145 may first recognize a whistle corresponding to at least one whistle pattern among the n whistle sound patterns by reading the buffered sound data.

Referring to FIG. 1, the control module 130 reads the buffered sound data, recognizes the user's language utterance, and processes natural language to recognize speech content information corresponding to the user's utterance content. 165), and a recognition determination unit 170 for checking whether the contents of the user's speech are recognized while the whistling sound is being recognized or within a predetermined time after the end of the recognition of the whistling sound, and the recognition determination unit 170, and the whistle sound When the content of the user's speech is recognized within a certain time during or after the recognition of the user's speech, an intelligent processing unit 175 for processing to interpret the recognized user's speech by associating it with s service functions related to the whistle may be provided. there is. On the other hand, according to another implementation method of the present invention, at least one component of the utterance recognition unit 165, the recognition determination unit 170, and the intelligence processing unit 175 is an operation server on a communication network that interworks with the sound device 100. 200. If the utterance recognition unit 165 is implemented in the operation server 200, the sound processing unit 140 transfers sound data buffered to a designated buffer area to the operation server 200. ), or selectively extract sound data determined to correspond to the user's speech among the buffered sound data and provide the sound data to the operation server 200. However, for convenience, Figure 1 describes the features of the present invention, focusing on an embodiment in which the utterance recognition unit 165, the recognition determination unit 170, and the intelligent processing unit 175 are implemented through the sound device 100. . Meanwhile, when the speech contents recognition unit 165, the recognition determination unit 170, and the intelligence processing unit 175 are implemented through the sound device 100, the speech contents recognition unit 165, the recognition determination unit 170 and the intelligence processing unit 175 are implemented. The processing unit 175 may be implemented in the form of at least one or a combination of two of a program code form and a chip or control circuit form, and the utterance recognition unit 165, the recognition determination unit 170, and the intelligence processing unit 175 The present invention is not limited by an implementation method or implemented module.

Whistle recognizing unit 145 reads the buffered sound data and recognizes the whistling sound, and at the same time, the speech content recognizing unit 165 reads the buffered sound data to utter the user's language. and performs natural language processing to recognize speech content information corresponding to the user's speech content. Meanwhile, when the utterance recognition unit 165 is implemented in the operation server 200, the sound processing unit 140 provides sound data buffered in a designated buffer area to the operation server 200 or the buffered sound data It is possible to selectively extract sound data that is determined to correspond to the user's utterance (eg, including segmented sounds) and provide it to the operation server 200, whereby the present invention is not limited.

According to the implementation method of the present invention, the speech content recognition unit 165 analyzes the buffered sound data through a designated speech recognition algorithm, recognizes and converts the speech spoken by the user into text, and at the same time analyzes form, semantics, and dialogue. Speech content information by recognizing the user's utterance content may be generated through one or more natural language processing procedures during the analysis.

Meanwhile, according to the implementation method of the present invention, the utterance recognition unit 165 recognizes the whistling sound so that the operation mode of the service device is changed from the sleep mode through the service control unit 150. It is preferable to perform a function of recognizing the user's utterance in case of waking up. If the operation mode of the sound device 100 is set to the sleep mode by the service control unit 150, the speech content recognition unit 165 may not perform a function of recognizing the user's speech content. Even if the user's speech is recognized, it can be processed so that it is not used for the service.

Meanwhile, when the user's speech content is recognized through the speech content recognition unit 165, the recognition determination unit 170 determines that the user's speech content is recognized while the whistling sound is recognized through the whistling sound recognition unit 145. It is checked whether it is recognized or whether the contents of the user's speech are recognized within a certain time after the whistle recognition is terminated through the whistle recognition unit 145.

If it is confirmed through the recognition determination unit 170 that the contents of the user's speech are recognized during or after the whistling sound is recognized, the intelligence processing unit 175 determines the contents of speech recognizing unit 165 The user's speech contents recognized through the above process are interpreted by relating them to s service functions related to the whistling sound, whereby a specific service function corresponding to the utterance contents among the s service functions related to the whistling sound can be determined. In this case, the service control unit 150 may activate a specific service function related to the specific sound and corresponding to the recognized user's utterance, and the service providing unit 180 may activate the specific service function. The provided procedure can be performed according to the specified procedure.

If the content of the user's speech is not recognized within a certain period of time during or after the whistling sound is recognized through the recognition determination unit 170, the intelligence processing unit 175 enters the sleep mode through the service control unit 150. and/or a process of waking up from the sleep mode by the sound of the whistle may be performed.

On the other hand, according to the implementation method of the present invention, the s number of services related to the whistling sound activated by the service control unit 150 and/or provided through the service providing unit 180 are provided on the communication network through the communication module 120. A server interworking service provided in conjunction with the operation server 200, a sound-based service provided through the sound module 105, and a composite service provided by using the sound module 105 and the communication module 120 in combination. It may include at least one service among them.

On the other hand, among the s services related to the whistling sound, the server interworking service is a paging service using wireless terminal information, a situation notification service using wireless terminal information, a paging service using an app installed on the wireless terminal, and using an app installed on the wireless terminal. At least one of a situation notification service, a video service that provides video information captured through at least one camera to an app installed in a wireless terminal, and a video service that provides video information captured through at least one camera to a designated IPTV device can include

Meanwhile, among the s services related to the whistling sound, the sound-based service may include a noise output service that outputs a designated service sound at a designated signal output strength through the sound output unit 105b of the sound module 105. Preferably, the signal output strength of the sound-based service includes a signal strength that is inversely proportional to the signal input strength of the whistling sound.

On the other hand, among the s services related to the whistle, the composite service recognizes the user's speech within a certain time during or after the whistle is recognized, and analyzes the user's speech by associating the user's speech with a service function related to the whistle Based on one result, the service provided through the operation server 200 on the communication network may be included.

On the other hand, according to the first camera embodiment of the present invention, the sound device 100 is provided with a short-distance module 125 for processing short-range wireless communication with the short-range camera device 185, and through the short-distance module 125 When processing short-range wireless communication with the short-range camera device 185, the service control unit 150 recognizes a whistling sound through the whistling sound recognition unit 145, and the camera device 125 through the short-range module 125. It is possible to activate s number of service functions including an image service function for photographing a sound generating source that generates the recognized whistling sound through the camera device 185 by communicating with the device 185, and the service providing unit 180 ) communicates with the camera device 185 through the short-distance module 125, receives image information captured through the camera device 185, and provides the image to the app installed in the wireless terminal through the operation server 200. At least one video service among service and/or video services provided to a designated IPTV device may be provided. Meanwhile, when the camera device 185 can provide the video service, the service providing unit 180 may process the video service function of the camera device 185 to operate through the short distance module 125 .

Alternatively, according to the second camera embodiment of the present invention, wireless communication based on a local area network with the local camera device 185 is performed through the local area network communication unit 120b of the communication module 120 provided in the sound device 100. In case of processing, the service control unit 150 communicates with the camera device 185 through the local area network communication unit 120b when recognizing the whistling sound through the whistling sound recognition unit 145, and the camera device ( 185), it is possible to activate s number of service functions, including an image service function for photographing the sound generating source that generated the recognized whistling sound, and the service provider 180 communicates through the local area network communication unit 120b. It communicates with the camera device 185 to receive video information captured through the camera device 185 and provides it to an app installed in the wireless terminal through the operation server 200 and / or to a designated IPTV device. At least one video service among video services may be provided. Meanwhile, when the camera device 185 can provide the video service, the service providing unit 180 may process the video service function of the camera device 185 to operate through the local area network communication unit 120b.

Alternatively, according to the third camera embodiment of the present invention, when the sound device 100 is provided with a camera module 110 for acquiring image information of a photographed subject, the service control unit 150 is configured to recognize the whistling sound ( 145), when a whistling sound is recognized, an image service function for photographing a sound generating source that generates the recognized whistling sound may be activated through the camera module 110, and the service providing unit 180 At least one of a video service providing video information captured through the camera module 110 to an app installed in a wireless terminal through the operation server 200 and/or a video service provided to a designated IPTV device may be provided. there is.

Figure 2 is a diagram showing the functional configuration of the sound device 100 and the operation server 200 according to another embodiment of the present invention.

In more detail, Figure 2 shows an embodiment in which at least some of the functional configurations of the control module 130 of the sound device 100 shown in Figure 1 are implemented on the operation server 200 side, Those skilled in the art to which the present invention pertains, an implementation method (eg, this Figure 2 Some of the functional configurations implemented on the operation server 200 side are implemented on the operation server 200 side as they are, but other parts are implemented on the control module 130 of the sound device 100 or the control module 130 of the above drawing 1 A method of implementing some of the functional configurations implemented in the control module 130 as they are, but implementing other portions on the operation server 200 side, some components shown in the above drawing 1 and this drawing 2 are omitted, or subdivided However, the present invention includes all the inferred implementation methods as the scope of rights, and the technical characteristics are not limited only to the implementation methods shown in Figure 1 and this Figure 2. No.

Referring to Figure 2, the operation server 200, the sound receiving unit 205 for receiving the sound data input through the sound input unit 105a of the designated sound device 100 and temporarily storing it in a designated storage medium, Recognizes at least one whistling sound designated for determining a specific situation among non-verbal sounds or noises that may occur in a designated space by reading the received sound data, and corresponds to at least one whistle frequency band designated and/or the whistling sound A whistling sound recognizing unit 145 for recognizing a specific whistling sound corresponding to at least one whistling sound pattern designated to be recognized, and whistling sound and When at least s (1≤s≤S) related service functions are activated and/or a whistling sound or user's speech is not recognized for a specified period of time, the operation mode of the sound device 100 is a sleep mode It has a service control unit 150 set to , and further includes a service providing unit 180 that provides a service corresponding to the activated service function.

When sound data input through the sound input unit 105a is transmitted from the designated sound device 100, the sound receiving unit 205 receives the sound data transmitted from the sound apparatus 100 and temporarily stores the sound data in the designated storage medium. can be stored Meanwhile, when the sound device 100 selectively transmits one or more sound data of sound data corresponding to a whistling sound or sound data corresponding to a user's utterance, the sound receiver 205 transmits from the sound device 100 The selectively transmitted sound data may be received and temporarily stored in a designated storage medium.

The whistle sound recognition unit 145 reads the received sound data and recognizes at least one whistle sound frequency band designated for determining a specific situation among non-verbal sounds or noises that may occur in a designated space. Continue (or repeat) the process of recognizing a specific whistle corresponding to and/or corresponding to at least one whistle pattern designated for recognizing the whistle.

The service controller 150 checks whether a whistling sound is not recognized for a predetermined period of time or longer after operating power is applied to the sound device 100 and/or effectively recognizes a whistling sound through the whistling sound recognizing unit 145. After that, it is checked whether the whistling sound is not recognized for a certain period of time or not, and/or the sound device 100 is powered on, and then it is checked whether the content of the user's utterance is not recognized for a certain period of time or not. After the user's speech content is effectively recognized according to a designated procedure (eg, the user's speech content is recognized during or after the whistling sound is effectively recognized through the whistling sound recognition unit 145), the user's speech content is recognized for a specified period of time or more It can be checked whether the contents of the utterance are not recognized. If the whistling sound is not recognized for more than a certain period of time or the content of the user's utterance is not recognized for more than a certain period of time, the service control unit 150 remotely sets the operation mode of the sound device 100 to a sleep mode. It can be made including the function to do.

According to the implementation method of the present invention, the sound device 100 can transmit the input sound data in a sleep mode, and the sound receiver 205 determines that the operation mode of the sound device 100 is the sleep mode. receiving the sound data in the state, and the whistling sound recognition unit 145 also reads the received sound data in the state that the operation mode of the sound device 100 is the sleep mode, and non-verbal sound or noise that can be generated in the designated space corresponding to at least one whistle frequency band designated for recognizing at least one whistle sound designated for determining a specific situation and/or a specific whistle sound corresponding to at least one whistle sound pattern designated for recognizing the whistle sound. The process of recognizing can be continued (or repeated). Preferably, in the sleep mode, when the whistle sound is effectively recognized through the whistle sound recognition unit 145, the service control unit 150 wakes up the sound device 100 and simultaneously records s number of whistles related to the whistle sound. A service function may be activated, and the service providing unit 180 may perform a procedure of providing a service corresponding to the activated s number of service functions.

Meanwhile, when two or more service functions related to the whistling sound are activated through the service control unit 150, the service providing unit 180 performs a procedure for providing a service corresponding to the activated service function according to a designated order, or And/or a procedure for providing at least one service according to a predetermined condition (eg, a condition for recognizing user's speech) among the activated service functions may be performed.

According to the implementation method of the present invention, the whistle recognition unit 145 corresponds to the whistle frequency band of the whistle sound generator and/or the unique whistle sound of the whistle sound generator corresponding to the whistle sound pattern of the whistle sound generator. It is preferable to recognize, and for this purpose, the operation server 200 has a function of learning at least one whistling sound frequency band specialized for a whistling sound generating source and/or a function of learning at least one whistling sound pattern specific to a whistling sound generating source. It is desirable to have.

Referring to FIG. 2, the operation server 200 stores M whistle frequency band information corresponding to M (M≥1) whistle frequency bands in a designated storage medium and/or N (N≥1) An information storage unit 135 for storing N whistle sound pattern information corresponding to ) number of whistle sound patterns in a designated storage medium, and the whistle sound recognition unit 145 is provided with a whistle sound frequency band stored in the designated storage medium. A whistle may be recognized using information and/or whistle pattern information.

The information storage unit 135 may store M whistle sound frequency band information corresponding to M (M≥1) whistle sound frequency bands in a designated storage medium. Among the non-verbal sounds or noises that can be generated in a designated space in the present invention, the whistling sound designated for determining a specific situation has a wide range (or a frequency difference of at least 100 Hz or more between the lowest frequency and the highest frequency) depending on the type of sound or sound source. It can have a frequency band. Therefore, the present invention can divide the frequency band of the whistling sound into M frequency bands (eg, equal division or experimentally selective division based on a plurality of frequency bands in which the whistling sound is concentrated), and the information storage unit 135 may store M whistle sound frequency band information corresponding to the divided M whistle sound frequency bands in a designated storage medium.

The information storage unit 135 may store N whistle sound pattern information corresponding to N (N≥1) whistle sound patterns in a designated storage medium. The present invention first generates whistle pattern information through qualitative and/or quantitative analysis of the whistle pattern identified through sample experiments for each case, and the information storage unit 135 generates the whistle pattern. Information can be stored in a designated storage medium.

On the other hand, according to the implementation method of the present invention, the sound device 100 can transmit the recognized whistle sound information to the designated operation server 200 after recognizing the whistle sound, and the intelligent processing unit 175 of the operation server 200 ) is a normalization process of collecting whistle sound information from each sound device 100 and normalizing individual sound characteristics for each sound source in the collected whistle sound and/or qualitative analysis and/or quantitative analysis for each case. Whistle sound pattern information can be refined or diversified through the analysis process. The information storage unit 135 may receive refined or diversified whistle sound pattern information through the above process, and store N pieces of whistle sound pattern information including the received whistle sound pattern information in a designated storage medium. .

According to the implementation method of the present invention, M whistle frequency band information corresponding to the M whistle frequency bands is stored in the designated storage medium, and/or N whistles corresponding to the N whistle patterns are stored in the designated storage medium. When sound pattern information is stored, the whistling sound recognition unit 145 reads the received sound data to correspond to at least one whistling sound frequency band among the stored M whistle sound frequency bands and/or the stored N whistle sound frequency bands. It is preferable to recognize a whistle corresponding to at least one whistle pattern among the whistle sound patterns.

Meanwhile, the M whistle frequency band information and/or the N whistle pattern information stored in the designated storage medium are not optimized (or customized) for the whistle sound of the whistle sound generator, but the frequency band and/or sound of the statistical whistle sound. It will be information corresponding to the pattern. Therefore, in order to recognize the whistle of the whistle sound generator existing in the space where the sound device 100 is mounted at high speed and/or prevent (or minimize) recognition errors, the stored whistle frequency band information and/or the whistle sound A process of optimizing (or customizing) the pattern information to the whistling sound of the whistling sound generating source is required.

Referring to FIG. 2, the operating server 200 uses the M number of whistle frequency band information for a designated period of time, at least i (i≥2) times, the whistle frequency corresponding to the recognized whistle frequency distribution. Distribution generation that generates distribution information and/or generates whistle pattern distribution information corresponding to the validly recognized whistle pattern distribution at least j (j≥2) times during a specified period of time using the N pieces of whistle pattern information. The unit 155 statistically processes the generated whistle sound frequency distribution information to optimize (or customize) the whistle sound of the whistle sound generator in the space where the sound device 100 passes through m (1≤m≤M m whistle frequency band information corresponding to ) number of whistle frequency bands and/or statistically processing the generated whistle pattern distribution information to determine the whistle sound generating source in the space through which the sound device 100 has passed through An information determination unit 160 for determining n whistle sound pattern information corresponding to n (1≤n≤N) whistle sound patterns optimized (or customized) for whistles is further provided.

When M whistle frequency band information corresponding to the M whistle frequency bands is stored in the storage medium designated through the information storage unit 135, the distribution generating unit 155 stores the M whistles stored in the designated storage medium. Using the sound frequency band information, i validly recognized whistle frequency bands are identified at least i (i≥2) times during a designated period of time, and whistle frequency distributions corresponding to the identified i whistle frequency band distributions information can be generated.

On the other hand, when whistle frequency distribution information corresponding to the whistle frequency distribution that has been recognized validly at least i during a specified period of time is generated by the distribution generator 155, the information determination unit 160 determines the generated whistle frequency Distribution information is statistically processed to optimize (or customize) the whistle sound of the whistle source in the space through which the sound device 100 has passed through, thereby recognizing the whistle sound of the whistle source at high speed or preventing (or minimizing) recognition errors It is possible to determine m whistle sound frequency band information corresponding to m (1≤m≤M) whistle sound frequency bands. Here, the m whistle frequency bands may be any one of the M whistle frequency bands, or may include a newly defined whistle frequency band within the whistle frequency band range. When the m whistle frequency band information is determined through the information determination unit 160, the information storage unit 135 sets the determined m whistle frequency bands as whistle frequency bands to be read first and stores the designated information. The information may be newly stored in a medium or updated and stored in the whistling sound frequency band information stored in the storage medium. Thereafter, the whistling sound recognizing unit 145 reads the received sound data to first recognize a whistling sound corresponding to at least one whistle sound frequency band among the new or updated stored m whistle sound frequency bands.

Meanwhile, when N whistle sound pattern information corresponding to the N whistle sound patterns is stored in the designated storage medium through the information storage unit 135, the distribution generating unit 155 stores the N whistles stored in the designated storage medium. Using the sound pattern information, at least j (j≥2) times during a designated period of time, validly recognized j whistle patterns are identified, and whistle pattern distribution information corresponding to the distribution of the identified j whistle patterns is generated. can do.

Meanwhile, when whistle pattern distribution information corresponding to the whistle sound pattern distribution that has been recognized validly at least j times during a specified period of time is generated through the distribution generator 155, the information determination unit 160 determines the generated whistle pattern. Distribution information is statistically processed to optimize (or customize) the whistle sound of the whistle source in the space through which the sound device 100 has passed through, thereby recognizing the whistle sound of the whistle source at high speed or preventing (or minimizing) recognition errors It is possible to determine n whistle sound pattern information corresponding to n (1≤n≤N) whistle sound patterns for Here, the n whistle patterns may be any one of the N whistle patterns, or may include a combination of two or more whistle patterns or a new whistle pattern. When n whistle sound patterns are determined through the information determination unit 160, the information storage unit 135 sets the determined n whistle sound patterns as firstly read whistle patterns and newly stores them in a designated storage medium, or Whistling sound frequency band information stored in the storage medium may be updated and stored. Thereafter, the whistle recognition unit 145 may first recognize a whistle corresponding to at least one whistle pattern among the n whistle sound patterns by reading the received sound data.

Referring to FIG. 2, the operation server 200 reads the received sound data, recognizes the user's language utterance, and processes natural language to recognize the utterance content information corresponding to the user's utterance content recognition unit ( 165), and a recognition determination unit 170 for checking whether the contents of the user's speech are recognized while the whistling sound is being recognized or within a predetermined time after the end of the recognition of the whistling sound, and the recognition determination unit 170, and the whistle sound When the content of the user's speech is recognized within a certain time during or after the recognition of the user's speech, an intelligent processing unit 175 for processing to interpret the recognized user's speech by associating it with s service functions related to the whistle may be provided. there is.

While reading the received sound data through the whistling sound recognition unit 145 and performing a process of recognizing the whistling sound, the speech content recognition unit 165 reads the received sound data to utter the user's language. and performs a procedure of recognizing speech content information corresponding to the user's speech content by natural language processing.

According to the implementation method of the present invention, the speech content recognition unit 165 analyzes the received sound data through a designated speech recognition algorithm to recognize and characterize the speech spoken by the user, and at the same time analyze the form, semantics, and dialogue. Speech content information by recognizing the user's utterance content may be generated through one or more natural language processing procedures during the analysis.

Meanwhile, according to the implementation method of the present invention, the utterance recognition unit 165 recognizes the whistling sound so that the operation mode of the service device is changed from the sleep mode through the service control unit 150. It is preferable to perform a function of recognizing the user's utterance in case of waking up. If the operation mode of the sound device 100 is set to the sleep mode by the service control unit 150, the speech content recognition unit 165 may not perform a function of recognizing the user's speech content. Even if the content of the user's speech is recognized, it can be processed so that it is not used for the service.

Meanwhile, when the user's speech content is recognized through the speech content recognition unit 165, the recognition determination unit 170 determines that the user's speech content is recognized while the whistling sound is recognized through the whistling sound recognition unit 145. It is checked whether it is recognized or whether the user's utterance is recognized within a certain time after the whistle recognition is terminated through the whistle recognition unit 145.

If it is confirmed through the recognition determination unit 170 that the contents of the user's speech are recognized during or after the whistling sound is recognized, the intelligence processing unit 175 determines the contents of speech recognizing unit 165 The user's speech content recognized through is processed to be analyzed by relating it to s service functions related to the whistling sound, whereby a specific service function corresponding to the utterance content among the s service functions related to the whistling sound can be determined. In this case, the service control unit 150 may activate a specific service function related to the specific sound and corresponding to the recognized user's utterance, and the service providing unit 180 may activate the specific service function. The provided procedure can be performed according to the specified procedure.

If the content of the user's utterance is not recognized within a certain period of time during or after the whistling sound is recognized through the recognition determination unit 170, the intelligence processing unit 175 enters the sleep mode through the service control unit 150. and/or a process of waking up from the sleep mode by the whistling sound may be performed.

On the other hand, according to the implementation method of the present invention, the service control unit 150 of the operation server 200 activates and / or the s number of services related to the whistling sound that can be provided through the service providing unit 180 is the operation server ( 200) and at least one of a server-based service provided through the sound device 100 and a composite service provided in conjunction with the sound device 100.

On the other hand, among the s services related to the whistling sound, the server-based service is a call service using wireless terminal information, a situation notification service using wireless terminal information, a call service using an app installed on the wireless terminal, and a call service using an app installed on the wireless terminal. At least one of a situation notification service, a video service that provides video information captured through at least one camera to an app installed in a wireless terminal, and a video service that provides video information captured through at least one camera to a designated IPTV device can include

On the other hand, among the s services related to the whistle, the complex service recognizes the user's speech within a certain time during or after the whistle is recognized, and analyzes the user's speech by associating the user's speech with a service function related to the whistle Based on one result, the service provided through the operation server 200 on the communication network may be included.

3 illustrates an embodiment of recognizing a whistle sound and activating a service related to a whistle sound according to an embodiment of the present invention.

In more detail, FIG. 3 shows an embodiment of recognizing a whistle using M pieces of whistle sound frequency band information and/or N pieces of whistle sound pattern information stored in a designated storage area and activating a service related to the recognized whistle sound. , and those of ordinary skill in the art to which the present invention pertains, various implementation methods (eg, some steps are omitted) for the whistle recognition or service activation process by referring to and/or modifying this Figure However, the present invention includes all the inferred implementation methods, and its technical characteristics are not limited only to the implementation method shown in FIG. 3. On the other hand, the process illustrated and described in FIG. 3 is provided through the sound device 100, the operation server 200, or provided in a distributed form through interworking between the sound device 100 and the operation server 200. It is possible, and for convenience, the features of the present invention will be described in FIG. 3 focusing on the process provided through the sound device 100.

Referring to the embodiment of FIG. 3, the sound device 100 stores M whistle frequency band information corresponding to M whistle frequency bands for recognizing whistles in a designated storage area (300) and/or N whistle sound pattern information corresponding to N whistle sound patterns for recognizing whistles is stored in a designated storage area (300). Thereafter, the sound device 100 receives sound data corresponding to the sound signal of the audible frequency band through the sound input unit 105a, buffers it in a designated buffer area (305), reads the buffered sound data, and reads the sound data in the designated space. For recognizing the at least one specified whistle, a whistle corresponding to the specified at least one whistle frequency band and/or a specific whistle corresponding to the specified at least one whistle pattern for recognizing the whistle is selected. A recognition procedure is performed (310).

According to an embodiment of the present invention, the sound device 100 reads the buffered sound data to correspond to at least one whistle frequency band among the stored M whistle frequency bands and/or to the stored N whistle frequency bands. A whistling sound corresponding to at least one of the whistling sound patterns may be recognized (310). According to another embodiment of the present invention, through the process of FIG. 4 and/or FIG. 5, m whistle frequencies optimized (or customized) for the whistle sound of the whistle sound generator in the space where the sound device 100 has passed through When band information is determined and stored in a designated storage area and/or n whistling sound pattern information optimized (or customized) for the whistling sound of the whistling sound generator is determined and stored in a designated storage area, the sound device 100 reads the buffered sound data to correspond to at least one whistle frequency band among the stored m whistle frequency bands and/or a whistle sound corresponding to at least one whistle pattern among the stored n whistle patterns It is possible to recognize the unique whistling sound of the generating source (310).

If the operation mode of the sound device 100 is the sleep mode at the time of recognizing the whistling sound, the sound device 100 may be woken up in the sleep mode (315).

Meanwhile, when the whistling sound is recognized and/or the operation mode of the sound device 100 is woken up from the sleep mode, the sound device 100 among the designated S (S≥1) service functions related to the whistling sound At least s (1≤s≤S) number of service functions may be activated (320), and a procedure for providing at least one service function among the activated s number of service functions according to a specified procedure may be performed (325).

Meanwhile, when the whistling sound is not recognized by reading the sound data, the sound device 100 checks whether the whistling sound is not effectively recognized for a designated period of time (330). If the whistling sound is not validly recognized for a designated period of time, the sound device 100 may set the operation mode of the sound device 100 to a sleep mode (335).

Figure 4 shows an embodiment of optimizing (or customizing) a whistle frequency band for recognizing a whistle sound to the whistle sound of a whistle sound generator according to an embodiment of the present invention.

In more detail, Figure 4 shows whistle frequency distribution information corresponding to the whistle frequency distribution that has been recognized validly at least i (i≥2) times during a specified period of time using M pieces of whistle frequency band information pre-stored in a designated storage area. and m (1≤m≤M) number of optimized (or customized) whistle sounds of the whistle sound generator in the space where the sound device 100 passed through by statistically processing the generated whistle frequency distribution information. It shows an embodiment of determining m pieces of whistle frequency band information corresponding to the whistle sound frequency band and storing them in a designated storage area as new/updated. Various implementation methods (eg, implementation methods in which some steps are omitted or the order is changed) for the process of optimizing (or customizing) the whistle sound frequency band may be inferred by referring to and/or modifying FIG. The invention is made including all the implementation methods inferred above, and its technical characteristics are not limited only to the implementation method shown in FIG. 4. On the other hand, the process illustrated and described in FIG. 4 is provided through the sound device 100, the operation server 200, or provided in a distributed form through interworking between the sound device 100 and the operation server 200. It is possible, and for convenience, the features of the present invention will be described mainly in the process provided through the sound device 100 in FIG. 4 .

Referring to the embodiment of FIG. 4, the sound device 100 effectively recognizes a whistle using M pieces of whistle sound frequency band information pre-stored in a designated storage area, and then the whistle sound used to effectively recognize the whistle sound. The frequency band information is stored in a designated storage area and managed (400).

If i (i≥2) pieces of whistle frequency band information used to validly recognize whistles for a specified period of time are stored in the storage area (405), the sound device 100 provides at least i (i for a specified period of time). ≥ 2) generates whistle frequency distribution information corresponding to the validly recognized whistle frequency distribution (410), statistically processes the generated whistle frequency distribution information, and After determining m whistle frequency band information corresponding to m (1≤m≤M) whistle frequency bands optimized (or customized) for the whistle sound of the whistle sound generator (415), the determined m whistle frequencies The band is first set as a whistle frequency band to be read, and new/updated storage is stored in a designated storage area (420). Then, the sound device 100 recognizes a whistle using the stored m whistle frequency band information. can

5 illustrates an embodiment of optimizing (or customizing) a whistle sound pattern for recognizing a whistle sound to the whistle sound of a whistle sound generator according to another embodiment of the present invention.

In more detail, FIG. 5 shows whistle sound pattern distribution information corresponding to whistle sound pattern distributions that have been recognized validly at least j (j≥2) times during a designated period of time using N pieces of whistle sound pattern information pre-stored in a designated storage area. n (1≤n≤N) number of whistles optimized (or customized) for the whistle sound of the whistle sound generator in the space where the sound device 100 passed through by generating and statistically processing the generated whistle sound pattern distribution information It shows an embodiment of determining n whistle sound pattern information corresponding to a sound pattern and storing it in a new/updated storage area in a designated storage area. Various implementation methods for the process of optimizing (or customizing) the whistling sound pattern (eg, an implementation method in which some steps are omitted or the order is changed) may be inferred by reference and/or modification, but the present invention It is made including all the implementation methods that are, and the technical characteristics are not limited only to the implementation method shown in this Figure 5. On the other hand, the process illustrated and described in this Figure 5 is provided through the sound device 100, provided through the operation server 200, or provided in a distributed form through interworking between the sound device 100 and the operation server 200. It is possible, and for convenience, the features of the present invention will be described in Figure 5 focusing on the process provided through the sound device 100.

Referring to the embodiment of FIG. 5, the sound device 100 effectively recognizes a whistle sound using N pieces of whistle sound pattern information pre-stored in a designated storage area, and then the whistle sound pattern used to effectively recognize the whistle sound. Information is stored and managed in a designated storage area (500).

If j (j≥2) pieces of whistling frequency band information used to validly recognize whistles for a specified period of time are stored in the storage area (505), the sound device 100 is valid at least j times for a specified period of time. Whistle sound pattern distribution information corresponding to the recognized whistle sound pattern distribution is generated (510), and the generated whistle sound pattern distribution information is statistically processed so that the whistle of the whistle sound generating source in the space through which the sound device 100 passes After determining n whistle pattern information corresponding to n (1≤n≤N) whistle patterns optimized (or customized) for sound (515), the determined n whistle patterns are to be read first. is set to new/updated storage in the designated storage area (520), and then the sound device 100 can recognize the whistle sound using the stored n whistle sound pattern information.

6 illustrates an embodiment of activating a service related to a whistling sound by recognizing a whistle sound and an utterance in association with each other according to another embodiment of the present invention.

In more detail, FIG. 6 shows that when the contents of the user's speech are recognized within a certain time during or after the whistling sound is recognized, the recognized user's speech contents are analyzed in association with s service functions related to the whistling sound, and the recognition This shows an embodiment of activating a service related to a whistling sound, and those skilled in the art will refer to and / or modify this Figure 6 to recognize the whistling sound or service activation process Various implementation methods (eg, implementation methods in which some steps are omitted or the order is changed) may be inferred, but the present invention includes all implementation methods inferred above, and the implementation shown in FIG. The technical characteristics are not limited only by the method. On the other hand, the process shown and described in this Figure 6 is provided through the sound device 100, provided through the operation server 200, or provided in a distributed form through interworking between the sound device 100 and the operation server 200. It is possible, and for convenience, the features of the present invention will be described in FIG. 6 focusing on the process provided through the sound device 100.

Referring to the embodiment of FIG. 6, the sound apparatus 100 reads sound data buffered in a designated buffer area, recognizes a user's language utterance, processes it in natural language, and recognizes speech content information corresponding to the user's speech content. is performed (600). If the user's speech content is not recognized, the sound device 100 checks whether the user's speech content is not validly recognized for a specified period of time or longer (605). If the content of the user's utterance is not validly recognized for a specified period of time or longer, the sound device 100 may set the operation mode of the sound device 100 to a sleep mode (610).

Meanwhile, when the user's utterance is recognized by reading the sound data, the sound device 100 determines whether the operation mode of the sound device 100 is a sleep mode (615). If in the sleep mode, it is preferable to perform a process of waking up by recognizing a whistling sound.

On the other hand, if the operation mode of the sound device 100 is not the sleep mode at the time of recognizing the contents of the user's speech (e.g., while the contents of the user's speech are recognized or a whistling sound is recognized before a certain time, the sound apparatus 100 When the sleep mode wakes up), the sound device 100 confirms that the content of the user's speech is recognized while the whistling sound is being recognized, and/or the user's speech is heard within a certain time after the end of the recognition of the whistling sound. It is checked whether the contents are recognized (620). If the content of the user's speech is not recognized within a certain time during or after the whistling sound is recognized, the sound device 100 preferably performs a process of checking whether the sound device 100 is in a sleep mode or waking up by recognizing the whistling sound. do.

On the other hand, when the user's speech content is recognized within a certain period of time during or after the whistling sound is recognized, the sound device 100 associates the recognized user's speech content with s service functions related to the whistling sound and interprets (625), based on this, a specific service function related to the specific sound and corresponding to the recognized user's utterance can be activated (630), and the activated specific service function is designated as The provided procedure may be performed according to the procedure (635).

100: sound device 105: sound module
105a: sound input unit 105b: sound output unit
110: camera module 110a: camera unit
115: power module 120: communication module
120a: wireless network communication unit 120b: local area network communication unit
125: short-range module 125a: short-range wireless unit
130: control module 130a: control unit
130b: memory unit 135: information storage unit
140: sound processing unit 145: whistling sound recognition unit
150: service control unit 155: distribution generation unit
160: information determination unit 165: speech content recognition unit
170: situation judgment unit 175: intelligence processing unit
180: service provider 185: camera device
200: operation server 205: sound receiver

Claims (22)

A sound module including a sound input unit for receiving a sound signal and a sound output unit for outputting a sound signal, a communication module including at least one communication unit connected to a communication network, a short range camera device, and a short range module for processing short range wireless communication and a subject In the sound device having a camera module for obtaining image information photographed,
a sound processing unit which buffers sound data corresponding to the sound signal of the audible frequency band input through the sound input unit in a designated buffer area or checks the sound data buffered in the designated buffer area;
Corresponds to the whistle frequency band of at least one designated whistling sound generating source for recognizing a designated whistling sound for determining a specific situation among non-verbal sounds or noises that may be generated in a designated space by reading the buffered sound data, and of the whistling sound generating source a whistling sound recognizing unit recognizing a unique whistling sound of a whistling sound generating source corresponding to a whistling sound pattern;
When recognizing the whistle, at least s (1≤s≤S) service functions related to the whistle among the designated S (S≥1) service functions and any one or more of the short-distance module, communication module, and camera module A control module including a service control unit for activating an image service function for photographing a sound generating source that generates the recognized whistling sound through the camera device using the module,
The control module,
Information storage for storing M whistle frequency band information corresponding to M (M≥1) whistle sound frequency bands and N whistle pattern information corresponding to N (N≥1) whistle sound patterns in a designated storage area wealth;
generating whistle frequency distribution information corresponding to at least i (i≥2) times validly recognized whistle frequency distribution for a specified period of time using the M pieces of whistle frequency band information; and generating the N pieces of whistle pattern information a distribution generator for generating whistle pattern distribution information corresponding to the validly recognized whistle pattern distribution at least j (j≥2) times during a specified period of time using the information;
Corresponding to m (1≤m≤M) frequency bands of whistles optimized (or customized) for the whistle sound of the whistle sound generator in the space through which the sound device is installed by statistically processing the generated whistle frequency distribution information n (1≤n) optimized (or customized) for the whistle sound of the whistle sound generator in the space through which the sound device is located by determining m pieces of whistle sound frequency band information and statistically processing the generated whistle sound pattern distribution information an information determination unit for determining n whistle sound pattern information corresponding to ≤N) whistle sound patterns;
an utterance content recognition unit that reads the buffered sound data, recognizes a user's language utterance, and recognizes utterance content information corresponding to the user's utterance content by natural language processing;
a recognition determination unit for determining whether the contents of the user's speech are recognized while the whistling sound is being recognized or whether the contents of the user's speech are recognized within a predetermined time after the end of the recognition of the whistling sound;
If the content of the user's speech is recognized during or after the whistling sound is recognized, an intelligent processing unit for interpreting the recognized user's speech content by associating it with s service functions related to the whistling sound; and
A sound device with a whistling sound recognition function, characterized in that it further comprises; a service providing unit providing a service corresponding to the activated service function.
delete delete delete delete delete The method of claim 1, wherein the whistling sound recognition unit,
and a function of reading the buffered sound data and recognizing a whistle corresponding to at least one whistle frequency band among the M whistle frequency bands and corresponding to at least one whistle pattern among the N whistle patterns. A sound device equipped with a whistling sound recognition function, characterized in that made by.
delete According to claim 1,
The information storage unit further includes a function of setting the determined m whistle frequency bands as whistle frequency bands to be read first and storing them as new or updated;
The whistling sound recognizing unit comprises a function of reading the buffered sound data and firstly recognizing a whistling sound corresponding to at least one whistle sound frequency band among the stored m whistle sound frequency bands. A sound device equipped with a recognition function.
delete According to claim 1,
The information storage unit further includes a function of setting the determined n whistle sound patterns as whistle sound patterns to be read first and storing them as new or updated,
wherein the whistling sound recognizing unit includes a function of reading the buffered sound data and firstly recognizing a whistling sound corresponding to at least one whistling sound pattern among the stored n whistling sound patterns. equipped with a sound device.
delete The method of claim 1, wherein the service control unit,
A sound device with a whistle recognition function, characterized in that it comprises a function of activating a specific service function related to the whistle sound and corresponding to the recognized user's utterance.
The method of claim 1, wherein the s number of services,
A server linkage service provided in conjunction with an operation server on a communication network through the communication module;
A sound-based service provided through the sound module;
A sound device with a whistling sound recognition function, characterized in that it includes at least one service among complex services provided by combining the sound module and the communication module.
The method of claim 14, wherein the server interworking service,
Call service using wireless terminal information;
Situation notification service using wireless terminal information;
A call service using an app installed on a wireless terminal;
A situation notification service using an app installed on a wireless terminal,
A video service that provides video information captured by at least one camera to an app installed on a wireless terminal;
A sound device with a whistling sound recognition function, characterized in that it includes at least one service among video services that provide video information captured by at least one camera to a designated IPTV device.
The method of claim 14, wherein the sound-based service,
A sound device with a whistling sound recognition function, characterized in that it comprises a noise output service for outputting a designated service sound at a designated signal output strength through the sound output unit of the sound module.
The method of claim 16, wherein the signal output strength,
A sound device with a whistling sound recognition function, characterized in that it comprises a signal strength inversely proportional to the signal input strength of the whistling sound.
The method of claim 14, wherein the complex service,
The service provided through the operation server on the communication network based on the result of recognizing the contents of the user's speech within a certain period of time during or after the whistling sound is recognized and associating the contents of the user's speech with the service function related to the whistling sound Sound device equipped with a whistling sound recognition function, characterized in that comprising a.

delete delete delete delete

Images