KR102573242B1 - Sound Device for Recognition of Scream Sound - Google Patents

Abstract

According to the sound device for recognizing screams of the present invention, a sound module including a sound input unit receiving a sound signal and a sound output unit outputting a sound signal, and a communication module including at least one communication unit connected to a communication network, and a short-distance A sound device comprising a short-distance module for processing short-range wireless communication with a camera device and a camera module for obtaining image information of a photographed subject, comprising: providing sound data corresponding to a sound signal of an audible frequency band input through the sound input unit; A sound processing unit that checks sound data buffered in a designated buffer area or buffered in a designated buffer area and reads the buffered sound data to recognize designated screams among non-verbal sounds or noises that may occur in a designated space to determine an emergency situation A scream recognition unit for recognizing a scream sound unique to a scream source corresponding to a scream frequency band of at least one scream source designated for and corresponding to a scream pattern of the scream source, and when the scream is recognized At least s (1≤s≤S) service functions related to the screaming among the S (S≥1) service functions specified in the camera device using at least one module among the short-distance module, the communication module, and the camera module A control module including a; service control unit for activating a video service function for photographing a sound generating source that generated the recognized scream through Using an information storage unit that stores corresponding M pieces of screaming sound frequency band information and N screaming sound pattern information corresponding to N (N≥1) screaming sound patterns in a designated storage area and the M screaming sound frequency band information generate scream frequency distribution information corresponding to the scream frequency distribution recognized at least i (i≥2) times during a specified period of time, and use the N pieces of scream pattern information to generate at least j (j A distribution generating unit for generating scream pattern distribution information corresponding to the scream pattern distribution that is recognized validly ≥2) times and a scream sound generating source in the space where the sound device passes through statistical processing of the generated scream frequency distribution information m scream frequency band information corresponding to m (1≤m≤M) scream frequency bands optimized (or customized) for the screams of an information determining unit for determining n pieces of scream pattern information corresponding to n (1≤n≤N) scream patterns optimized (or customized) for the screams of a scream source in a space where a sound device is installed; and A speech content recognition unit that reads the buffered sound data to recognize the user's speech and processes it in natural language to recognize speech content information corresponding to the user's speech, and the user's speech is recognized while the scream is being recognized; A recognition determination unit for confirming whether the user's speech content is recognized within a predetermined time after the end of the recognition of the scream, and when the user's speech is recognized within a predetermined time during or after the scream is recognized, the recognized user It further includes an intelligent processing unit that interprets the utterance by associating it with s number of service functions related to screams, and a service providing unit that provides a service corresponding to the activated service function.

Description

Sound Device for Recognition of Scream Sound}

The present invention is a stationary sound device having a sound module for inputting and outputting sound signals and a communication module for connecting to a communication network, which can be generated in a designated space by reading sound data corresponding to a sound signal of an audible frequency band input through the sound module. Among non-verbal sounds or noises, a specific signal corresponding to at least one screaming frequency band designated for recognizing a designated screaming sound for determining an emergency situation and/or corresponding to at least one screaming pattern designated for recognizing the screaming sound When a scream is attempted and recognized, at least one designated service function related to the scream is activated or the designated service is provided among services that can be provided through at least one of the sound module and the communication module (or a combination of the two). It is to provide a sound device that does.

Recently, a stationary sound device that recognizes a user's voice and provides a service corresponding to the recognized voice has been sold. For example, stationary sound devices such as Amazon's 'Echo', SK Telecom's 'Who', and KT's 'Giga Genie' are on sale.

However, the conventional stationary sound device is a combination of a sound source service and voice recognition technology, and operates mainly on a service that simply provides or reproduces a sound source desired by a user through voice recognition, and has a problem in that it is difficult to use in other fields.

An object of the present invention for solving the above problems is a sound module including a sound input unit for receiving a sound signal and a sound output unit for outputting a sound signal, and a communication module including at least one communication unit connected to a communication network and a short distance A sound device comprising a short-range module for processing short-range wireless communication with a camera device and a camera module for obtaining image information of a subject, wherein sound data corresponding to a sound signal of an audible frequency band input through the sound input unit A sound processing unit that buffers data in a designated buffer area or checks sound data buffered in the designated buffer area and reads the buffered sound data to recognize designated screams among non-verbal sounds or noises that may occur in a designated space to determine an emergency situation. A scream recognition unit for recognizing a scream sound unique to a scream sound generator corresponding to a scream frequency band of at least one scream sound generator designated for the purpose and corresponding to a scream pattern of the scream sound generator and recognizing the scream At least s (1 ≤ s ≤ S) service functions related to the screaming among the S (S ≥ 1) service functions designated in the case of the camera using one or more modules among the short-distance module, the communication module, and the camera module A control module including a; service control unit for activating a video service function for photographing a sound generating source that generates the recognized scream through a device, wherein the control module includes M (M≥1) frequency bands of screams An information storage unit for storing M pieces of screaming sound frequency band information corresponding to N and N screaming sound pattern information corresponding to N (N≥1) screaming sound patterns in a designated storage area and the M screaming sound frequency band information generating scream frequency distribution information corresponding to at least i (i≥2) validly recognized scream frequency distributions for a specified period of time, and using the N pieces of scream pattern information for a specified period of time at least j ( A distribution generating unit that generates scream pattern distribution information corresponding to j≥2) times validly recognized scream pattern distribution, and the generated scream frequency distribution information is statistically processed to generate screams in the space where the sound device passes through. Determining m scream frequency band information corresponding to m (1≤m≤M) scream frequency bands optimized (or customized) for the screaming sound of the source, and statistically processing the generated scream pattern distribution information An information determination unit for determining n pieces of scream pattern information corresponding to n (1≤n≤N) scream patterns optimized (or customized) for the screams of the screaming sound generator in the space where the sound device is placed; and A speech content recognition unit that recognizes the user's speech by reading the buffered sound data and processes natural language to recognize speech content information corresponding to the user's speech; and recognizes the user's speech while the scream is being recognized. or if the user's speech content is recognized within a certain time period during or after the screaming sound is recognized, the recognized user To provide a sound device for recognizing screams further comprising an intelligent processing unit that interprets the contents of speech in association with s number of service functions related to screams and a service provider that provides a service corresponding to the activated service function there is.

A sound device for recognizing screams according to the present invention includes a sound module including a sound input unit for receiving a sound signal and a sound output unit for outputting a sound signal, a communication module including at least one communication unit connected to a communication network, and a short-range A sound device comprising a short-distance module for processing short-range wireless communication with a camera device and a camera module for obtaining image information of a photographed subject, comprising: providing sound data corresponding to a sound signal of an audible frequency band input through the sound input unit; A sound processing unit that checks sound data buffered in a designated buffer area or buffered in a designated buffer area and reads the buffered sound data to recognize designated screams among non-verbal sounds or noises that may occur in a designated space to determine an emergency situation A scream recognition unit for recognizing a scream sound unique to a scream source corresponding to a scream frequency band of at least one scream source designated for and corresponding to a scream pattern of the scream source, and when the scream is recognized At least s (1≤s≤S) service functions related to the screaming among the S (S≥1) service functions specified in the camera device using at least one module among the short-distance module, the communication module, and the camera module A control module including a; service control unit for activating a video service function for photographing a sound generating source that generated the recognized scream through Using an information storage unit that stores corresponding M pieces of screaming sound frequency band information and N screaming sound pattern information corresponding to N (N≥1) screaming sound patterns in a designated storage area and the M screaming sound frequency band information generate scream frequency distribution information corresponding to the scream frequency distribution recognized at least i (i≥2) times during a specified period of time, and use the N pieces of scream pattern information to generate at least j (j A distribution generating unit for generating scream pattern distribution information corresponding to the scream pattern distribution that is recognized validly ≥2) times and a scream sound generating source in the space where the sound device passes through statistical processing of the generated scream frequency distribution information m scream frequency band information corresponding to m (1≤m≤M) scream frequency bands optimized (or customized) for the screams of an information determining unit for determining n pieces of scream pattern information corresponding to n (1≤n≤N) scream patterns optimized (or customized) for the screams of a scream source in a space where a sound device is installed; and A speech content recognition unit that reads the buffered sound data to recognize the user's speech and processes it in natural language to recognize speech content information corresponding to the user's speech, and the user's speech is recognized while the scream is being recognized; A recognition determination unit for confirming whether the user's speech content is recognized within a predetermined time after the end of the recognition of the scream, and when the user's speech is recognized within a predetermined time during or after the scream is recognized, the recognized user It is characterized in that it further comprises an intelligent processing unit for processing to interpret the utterance by associating it with s number of service functions related to screams, and a service providing unit for providing a service corresponding to the activated service function.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

According to the present invention, the scream recognition unit reads the buffered sound data, corresponds to at least one screaming frequency band among the M screaming frequency bands, and selects at least one screaming pattern among the N screaming patterns. It is characterized in that it comprises a function of recognizing the scream corresponding to.

delete

delete

According to the present invention, the information storage unit further includes a function of setting the determined m number of screaming frequency bands as screaming frequency bands to be read first and storing new or updated sounds, and the screaming recognition unit performs the buffered sound. and a function of first recognizing a scream corresponding to at least one screaming frequency band among the m number of screaming frequency bands by reading the data.

delete

delete

According to the present invention, the information storage unit further includes a function of setting the determined n scream patterns as scream patterns to be read first and storing them as new or updated, and the scream recognition unit retrieves the buffered sound data. and a function of first recognizing a scream corresponding to at least one scream pattern among the stored n scream patterns.

delete

delete

delete

delete

According to the present invention, the s services are a server interworking service provided in conjunction with an operation server on a communication network through the communication module, a sound-based service provided through the sound module, and a combination of the sound module and the communication module It may include at least one service among complex services provided by

According to the present invention, the server interworking service includes at least one of a paging service using wireless terminal information, a situation notification service using wireless terminal information, a paging service using an app installed in a wireless terminal, and a situation notification service using an app installed in a wireless terminal. It may include at least one of a video service that provides video information captured through a camera of at least one camera to an app installed in a wireless terminal, and a video service that provides video information captured through at least one camera to a designated IPTV device.

According to the present invention, the sound-based service may include a noise output service for outputting a designated service sound at a designated signal output strength through a sound output unit of a sound module.

According to the present invention, the signal output strength may include a signal strength that is inversely proportional to the signal input strength of the scream.

According to the present invention, the complex service recognizes the user's speech within a certain time during or after the scream is recognized, and associates the user's speech with a service function related to the scream. It is characterized by comprising a service provided through an operation server on a communication network.

delete

delete

delete

delete

delete

delete

delete

delete

According to the present invention, when a scream designated for determining an emergency situation among non-verbal sounds or noises that may occur in a designated space is not recognized, it is operated in a sleep mode that does not respond to the emergency situation, and in order to recognize the scream When a specific scream corresponding to at least one designated scream frequency band and/or corresponding to at least one scream pattern designated for recognizing the scream is recognized and recognized, it is automatically woken up from the sleep mode to It is advantageous to provide a sound device that activates at least a service function related to the scream or provides a service related to the scream.

1 is a diagram showing the functional configuration of a sound device for recognizing screams according to an embodiment of the present invention.
2 is a diagram showing the functional configuration of a sound device and an operating server according to another embodiment of the present invention.
3 illustrates an embodiment of recognizing a scream and activating a service related to the scream according to an embodiment of the present invention.
FIG. 4 illustrates an embodiment of optimizing (or customizing) a screaming sound frequency band for recognizing a screaming sound to the screaming sound of a screaming sound generator according to an embodiment of the present invention.
5 illustrates an embodiment of optimizing (or customizing) a scream sound pattern for recognizing a scream to the scream sound of a scream sound generator according to another embodiment of the present invention.
FIG. 6 illustrates an embodiment of activating a scream-related service by recognizing a scream and the content of an utterance in conjunction with each other according to another embodiment of the present invention.

Hereinafter, the operating principle of the preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings and description. However, the drawings shown below and the description below relate to preferred implementation methods among various methods for effectively explaining the features of the present invention, and the present invention is not limited to only the drawings and description below.

That is, the following examples correspond to examples of preferred union types among numerous embodiments of the present invention, and examples in which specific components (or steps) are omitted or specific configurations (or steps) are omitted in the following embodiments. Embodiments in which functions implemented in are divided into specific configurations (or steps), or embodiments in which functions implemented in two or more configurations (or steps) are integrated into any one configuration (or steps), specific configurations (or steps) It is clearly stated that the embodiments in which the operation sequence of is replaced are all within the scope of the present invention, even if not separately mentioned in the following embodiments. Therefore, it is clearly stated that various examples corresponding to subsets or complements can be divided retroactively from the filing date of the present invention based on the following examples.

In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, the definition should be made based on the contents throughout the present invention.

As a result, the technical spirit of the present invention is determined by the claims, and the following embodiments are a means for efficiently explaining the technical spirit of the present invention to those skilled in the art to which the present invention belongs. only

Figure 1 is a diagram showing the functional configuration of a sound device 100 for recognizing screams according to an embodiment of the present invention.

In more detail, Figure 1 shows a communication including a sound module 105 including a sound input unit 105a receiving a sound signal and a sound output unit 105b outputting a sound signal and at least one communication unit connected to a communication network. A sound processing unit having a module 120 and buffering sound data corresponding to a sound signal of an audible frequency band input through the sound input unit 105a in a designated buffer area or checking sound data buffered in a designated buffer area ( 140) and the buffered sound data are read and correspond to at least one scream frequency band designated to recognize a scream designated for emergency situation determination among non-verbal sounds or noises that may occur in a designated space, and/or the scream A scream recognition unit 145 for recognizing a specific scream corresponding to at least one scream pattern designated for recognizing a sound, and when recognizing the scream, among the designated S (S≥1) service functions, the scream As a functional configuration of a sound device 100 having a control module 130 including a service control unit 150 for activating at least s (1≤s≤S) service functions related to sound, the present invention Those with ordinary knowledge in the relevant technical field may refer to and/or modify this Figure 1 to various implementation methods for the sound device 100 (eg, implementation methods in which some components are omitted, subdivided, or combined). ) can be inferred, but the present invention includes all the implementation methods inferred above, and its technical characteristics are not limited only to the implementation method shown in FIG. 1.

Referring to the embodiment of FIG. 1, the sound device 100 includes a sound module 105 including a sound input unit 105a receiving a sound signal and a sound output unit 105b outputting a sound signal; It has a communication module 120 including at least one communication unit connected to a communication network, and reads sound data corresponding to a sound signal of an audible frequency band input through the sound input unit 105a to generate non-verbal information that can occur in a designated space. A specific scream corresponding to at least one scream frequency band designated for recognizing screams designated for emergency situation determination among sounds or noises and/or corresponding to at least one scream pattern designated for recognizing the screams. A control module 130 having one or more functional components for activating at least one designated service function related to the screaming sound or providing the designated service function when recognizing the sound by attempting to recognize it, and supplying operating power It is provided with a power module 115 that does. Meanwhile, according to the implementation method, the sound device 100 includes at least one of the short-distance module 125 for processing short-distance wireless communication with the short-distance camera device 185 and the camera module 110 for acquiring image information of a photographed subject. A module may be further provided, and in some cases, an input unit (not shown) including a key button or a touch pad for receiving a user's operation may be further provided, and/or an LED or display for outputting various operating states or information. An included output unit (not shown) may be further provided. On the other hand, in the embodiment of Figure 1, the name "module" is only a term for explaining the function and role of the sound device 100 for convenience, and it is clearly clarified that it does not refer to any independent hardware configuration. it is put

The sound module 105 is a generic name for components related to sound input/output including one or more of a sound input unit 105a receiving a sound signal and a sound output unit 105b outputting a sound signal among components of the sound device 100. , Preferably, the sound input unit 105a and the sound output unit 105b can input or output sound signals of at least an audible frequency band, and the sound input unit 105a or the sound output unit 105b is used as a control module 130 and an interface unit (not shown) that interfaces with.

The sound input unit 105a is a general term for a component that receives a sound signal, and encodes the sound signal according to one or more microphones and a designated encoding rule for receiving a sound signal that detects air vibration and converts it into an electrical signal. It is configured to include a program (and/or circuit) module that generates sound data. Preferably, the sound input unit 105a receives a sound signal of an audible frequency band (eg, a sound signal including a voice signal uttered by a user) through the microphone, and outputs the input audio frequency band according to a designated encoding rule. Sound data generated by encoding the sound signal of may be provided to the control module 130 .

According to the implementation method of the present invention, the sound input unit (105a) includes one or more directional microphones to detect the direction in which the scream is generated when the scream is recognized in the space where the sound device 100 is mounted. may be implemented, and the present invention is not limited thereby.

The sound output unit 105b is a generic term for a component that outputs a sound signal, and checks one or more speakers that output sound signals and sound data to be output through the speakers and decodes them into the sound signals that can be output through the speakers. It is configured to include a program (and/or circuit) module that controls the signal output strength and output quality of the sound signal while decoding. Preferably, the sound output unit 105b outputs a sound signal generated by decoding the sound data requested to output sound through the control module 130 through the speaker, and the signal of the sound signal output as sound Control the output strength to a specified signal strength (e.g., amplify or reduce the signal strength set to enable receiving (or listening) with a signal strength higher than the specified signal reception strength at a distance greater than a specified reference distance, and/or even in the presence of a specified obstacle) Receive (or listen) at a signal strength higher than the specified signal reception strength at a specified distance, amplify or reduce to a set signal strength, and/or receive at a specified distance at a signal strength higher than the specified signal reception strength even in an environment where noise of a specified decibel or higher exists Sound distortion is minimized by outputting sound by amplifying or reducing (or listening) to a set signal strength) and/or controlling the output quality (eg, speaker parameters such as Fs. Qts, Vas, etc.) of the output sound signal. Sound output by blocking or suppressing (eg, output by controlling the output quality of sound to block / suppress sound distortion caused by resonance of a speaker, or in the case of a voice signal, so that audibility (or hearing sensitivity) can be maintained at a certain level or higher Controls the output quality of the sound so that the sound components corresponding to the articulations on the voice signal are clearly distinguished and outputs it, or blocks/suppresses distortion of the frequency at which the bit string is encoded or signal strength when bit strings are encoded and output in the sound signal. It can be output by controlling the output quality of the sound so that it can be output).

The communication module 120 is a general term for components that process the communication network connection of the sound device 100, and is preferably a designated server (or device) via a wireless network (eg, mobile communication network, mobile Internet, etc.) via a base station. and a wireless network communication unit 120a communicating by connecting a communication channel with a local area network (eg, wireless LAN, etc.) via a wireless AP or repeater to communicate by connecting a designated server (or device) and a communication channel to communicate At least one communication unit among the network communication units 120b is provided, and an interface unit (not shown) for interfacing the communication unit with the control module 130 is further provided.

The wireless network communication unit 120a is a general term for a communication component that connects the sound device 100 to a wide area wireless network (eg, a mobile communication network) via a base station, and transmits and receives a radio frequency signal of a designated frequency band. An antenna, It is configured to include at least one RF module, a baseband module, and a signal processing module, and is connected to the control unit 130a to transmit operation results corresponding to wireless communication among various operation results of the control unit 130a through a wireless network. Alternatively, data is received through a wireless network and transmitted to the control unit 130a, and at the same time, the wireless communication access, registration, communication, and handoff procedures are performed.

According to the implementation method of the present invention, when the wireless network accessed through the wireless network communication unit 120a is a mobile communication network, the wireless network communication unit 120a accesses and registers a location according to at least one mobile communication standard. , call processing, call connection, data communication, and a mobile communication configuration that performs at least one handoff. On the other hand, according to the intention of those skilled in the art, the wireless network communication unit 120a performs at least one of connection, location registration, data communication, and handoff to a wide area wireless network other than the mobile communication network (eg, portable Internet according to IEEE 802.16 standards). It is clear that the present invention is not limited by the wireless communication configuration provided by the wireless network communication unit 120a. That is, the wireless network communication unit 120a provides wireless data communication based on packet communication via the cell-based base station on the wireless network for the sound device 100 regardless of the frequency band of the wireless section or the type or protocol of the communication network. It is a general term for components connected to the data network.

The local area network communication unit 120b uses a radio frequency signal as a communication medium within a certain distance (eg, within 10 m, or within 30 m, etc.) for short-range wireless communication with a network connection device (eg, wireless AP, etc.) provided in a short distance. A general term for communication resources that connect the sound device 100 to a communication network connected by the network access device through a local area wireless network that connects a session, preferably WiFi-based local area network communication, Bluetooth local area network communication, Zigbee-based local area network A short-range wireless communication session is connected between a network connection device provided in a short distance and the sound device 100 through at least one of communication, UWB-based local area network communication, and public wireless communication, and based on this, the sound device 100 is It can be connected to the communication network to which the network access device is connected.

According to the implementation method of the present invention, the local area network communication unit 120b connects a local area wireless communication session with a network connection device provided in a local area within a certain distance from the sound device 100 when a local area wireless network is implemented. It is a general term for components that connect the sound device 100 to a data network that provides wireless data communication based on packet communication via the network connection device of the local area network. Hereinafter, for convenience, the network communication function of the sound device 100 will be mainly described using the wireless network communication unit 120a, but the present invention is not limited thereto, and the network communication function using the local area network communication unit 120b. It is also clearly stated that it is provided.

Meanwhile, according to one example of the implementation method of the present invention, the local area network communication unit 120b connects the camera device 185 provided in the local area of the sound device 100 via the local area wireless network and a local area wireless network-based wireless network. communication can be handled. When the local area network communication unit 120b processes wireless communication based on a local area network with the camera device 185 in the local area, the sound device 100 and the camera device 185 perform wireless communication based on the local area wireless network. It is possible to store a key value for encrypted communication or to exchange a key value for encrypted communication at the time of connecting wireless communication based on a local area network, whereby the sound device 100 and the camera device 185 Encrypted communication using key values can be performed.

The short-distance module 125 is a general term for components that process short-range wireless communication of the sound device 100, and includes a short-range wireless unit 125a that connects short-range wireless communication of a predetermined standard, and the short-range wireless unit ( 125a) and an interface unit (not shown) that interfaces the control module 130. For example, the short-range wireless unit 125a may process Bluetooth-based short-range wireless communication according to the Bluetooth standard. However, it is clearly stated that the standard for short-range wireless communication of the short-range wireless unit 125a is not limited to the Bluetooth standard, and may include all short-range wireless communication standards including Wi-Fi, ZigBee, and UWB. Meanwhile, depending on the implementation method, the local area wireless unit 125a may be the local area network communication unit 120b.

The short-distance radio unit 125a is a general term for a component that processes short-range wireless communication of a designated standard, and preferably may include a Bluetooth component that processes Bluetooth-based short-range wireless communication according to the Bluetooth standard. However, the short-range wireless communication-related configuration of the short-range wireless unit 125a is not limited to Bluetooth-based short-range wireless communication according to the Bluetooth standard, and short-range wireless communication of various standards other than Bluetooth may be used according to the intention of a person skilled in the art (e.g., , WiFi-based short-range wireless communication according to the Wi-Fi standard, ZigBee-based short-range wireless communication according to the ZigBee standard, UWB-based short-range wireless communication according to the UWB standard, etc. available) is clearly stated. Hereinafter, for convenience, the characteristics of the present invention will be described mainly in an embodiment in which the short-range wireless unit 125a processes Bluetooth-based short-range wireless communication, but it is clearly stated that the present invention is not limited thereto.

According to the implementation method of the present invention, the short-distance module 125 (or short-distance wireless unit 125a) is capable of processing short-range wireless communication of a specified standard with the camera device 185 provided in the short-range of the sound device 100. can When the short-distance module 125 (or short-distance wireless unit 125a) processes short-range wireless communication with the short-range camera device 185, the sound device 100 and the camera device 185 operate based on short-range wireless communication. The key value for encrypted communication can be stored or the key value for encrypted communication can be exchanged at the time of connecting short-range wireless communication, whereby the sound device 100 and the camera device 185 use the key value. Encrypted communication can be performed.

According to the implementation method of the present invention, when the sound device 100 and the app of the wireless terminal used by the user are mapped (e.g., when a designated app is installed in the user wireless terminal and executed for the first time (or executed before mapping registration), the user wireless terminal Device identification information for identifying the sound device 100 in the terminal app and identification information of the user's wireless terminal (or unique identification information of the user's wireless terminal app) are registered in the designated operation server 200, and the sound device 100 and the application of the user's wireless terminal are matched and registered), the short-range wireless unit 125a connects the matched app of the user's wireless terminal and short-range wireless communication to provide various identification information or processing information specified to process a specific service. can be exchanged For example, information received as an app of a user's wireless terminal from a designated operation server 200 or input, extracted, or generated through an app of the user's wireless terminal is transmitted through the short-range wireless communication to the stationary sound device 100. It is transmitted to the control module 130 and processed (for example, sound output or transmitted to the operation server 200) and/or received by the sound device 100 from the designated operation server 200 or the sound device 100 for installation. ) The information recognized, extracted, or generated from may be transmitted to the app of the user's wireless terminal through the short-range wireless communication and processed (eg, screen output or transmitted to the operation server 200).

The camera module 110 is a general term for components that process camera shooting of the sound device 100, and preferably obtains image information (eg, image) or video information (eg, video) of a subject (eg, video) ( 110a), and an interface unit (not shown) that interfaces the camera unit 110a and the control module 130. On the other hand, if the short-distance module 125 (or short-range wireless unit 125a) and/or the short-range camera device 185 can be interlocked through the short-range wireless unit, the camera module 110 (or the camera unit 110a) It can be omitted, and the present invention is not limited thereby.

The camera unit 110a includes at least one lens unit that receives an optical signal reflected from a subject, an image sensor unit that detects an optical signal input through the lens unit and converts the optical signal into an electrical image signal, and a gap between the lens unit and the image sensor unit. It includes an actuator unit that implements an autofocus function or an image stabilization function, and according to an implementation method, one or more filter units (eg, an infrared filter, etc.) may be further provided between the lens unit and the image sensor unit. can Meanwhile, the camera unit 110a includes an image signal processing processor (ISP) that processes the image information or image information according to a designated procedure to improve the quality of the image information or image information generated through the image sensor unit. more can be provided.

Meanwhile, when the camera module 110 is provided in the sound device 100, the camera module 110 includes a driving unit for changing the shooting direction of the camera unit 110a, and the control module 130 ( Alternatively, the camera module 110 controls the driving unit to change the shooting direction of the camera unit 110a and reads image information or image information generated through the image sensor unit of the camera unit 110a to generate the sound device 100. It is possible to recognize a sound generating source (eg, a person or object generating a scream) in the same space as the space in which the sound generating sound is mounted, and operate the drive unit to photograph the sound generating source through the camera unit 110a. On the other hand, when one or more directional microphones are included in the sound input unit 105a provided in the sound module 105, the control module 130 (or camera module 110) is the sound input unit of the sound module 105 ( The direction of the scream sound detected through one or more directional microphones 105a) may be checked, and the direction of the scream sound identified through the camera unit 110a may be photographed by manipulating the drive unit.

The control module 130 is a general term for components that control various operations of the sound device 100, and is preferably an operation of generating image data through the camera module 110 and a sensor module through the short-distance module 125. Equipped with a control unit 130a for controlling at least one operation of an operation of collecting sensing data and an operation of communicating with a designated server (eg, operation server 200) through the communication module 120 or an operation of combining two or more operations and a memory unit for storing various information or program codes necessary for the operation of the control unit 130a.

The control unit 130a is a general term for components that control the operation of the sound device 100, and is configured to include at least one processor and an execution memory and/or various chips or control circuits, and is configured to include the sound It is connected to each component provided in the device 100 through a designated interface (eg, a bus (BUS), etc.). According to the present invention, the controller 130a loads and calculates at least one program code provided in the sound device 100 into the execution memory through the processor, and transmits the result to at least one component through the bus. and controls the operation of the sound device 100. The functional configuration of the present invention may be implemented in the form of program code and/or in the form of a chip or control circuit. Hereinafter, for convenience, it is implemented in the form of a program code, stored in a memory unit, and loaded and operated in the form of a program and/or chip. However, the functional components of the present invention implemented in the form of a control circuit will be illustrated and described in the control unit 130a.

The memory unit is a generic term for non-volatile memory corresponding to storage resources of the sound device 100, and stores at least one program code executed through the control unit 130a and at least one data set used by the program code. and keep it The memory unit basically includes system program codes and system data sets corresponding to the operating system of the sound device 100, communication program codes and communication data sets for processing wireless communication connection of the sound device 100, and at least one application program. Codes and application data sets are stored, and program codes and data sets corresponding to functional configurations implementing the present invention may also be stored in the memory unit.

The power module 115 is a general term for a component that supplies/applies power necessary for the operation of the sound device 100, and preferably uses an external power source (eg, AC power, DC power obtained by rectifying AC power, USB power, etc.) It is provided with a power supply unit (not shown) for receiving and applying, and when the voltage value or current value of the external power source is different from the voltage value or current value required for the operation of the sound device 100, the external power source is supplied to a specified voltage value or current value. It further includes a power converter (not shown) that converts power into operating power having a current value. Meanwhile, when the external power is supplied through the power supply unit or the external power is converted into operating power through the power conversion unit, the power module 115 charges the external power or operation power and operates the sound device 100. When external power is not supplied, a power charging unit (not shown) may be further provided to apply the charged charging power.

According to the first embodiment of the present invention, the sound device 100 is manufactured for a scream recognition-based service that recognizes a designated scream and provides a scream-related service in order to determine an emergency situation in a designated space. It may be a dedicated sound device 100 .

According to the second embodiment of the present invention, the sound device 100 provides a voice recognition operation mode that provides a well-known voice recognition-based service, but sets a specific setting through a predetermined input unit provided in the sound device 100. It may be a mode switching sound device 100 that switches from the voice recognition operation mode to the scream recognition operation mode for the scream recognition based service according to the present invention based on specific voice recognition through input or voice recognition.

According to the third embodiment of the present invention, the sound device 100 is a composite sound device designed to provide a well-known voice recognition-based service and at the same time recognize a scream and provide multiple services based on scream recognition according to the present invention. (100).

Referring to FIG. 1, the control module 130 selects recognition target information to be recognized in order to initiate or process a specific service among speech contents information to be recognized through the speech recognition unit of the control module 130 and acquired through natural language processing. It is provided with an information storage unit 135 that checks and stores it in a designated storage area of the memory unit.

When it is necessary to recognize the user's speech contents in order to determine the start or process of a specific service, the information storage unit 135 provides recognition target information including the speech contents to be recognized from the user's speech in order to initiate or process the specific service. It is possible to check and store service provision information and mapping for initiation or processing of the specific service.

According to the implementation method of the present invention, the recognition target information to be mapped with the service provision information is specified including the previously designated utterance content, and/or is provided to the user through the sound output unit 105b of the sound device 100. After requesting the content of speech to be used as the recognition target information, it may be received through the sound input unit 105a, acquired through natural language processing, and registered/updated. Meanwhile, according to the implementation method, the recognition target information may be registered/updated through an app of a user wireless terminal mapped with the sound device 100, and the present invention is not limited thereto.

Referring to FIG. 1, the control module 130 buffers sound data corresponding to a sound signal of an audible frequency band input through the sound input unit 105a in a designated buffer area or buffers sound data buffered in a designated buffer area. a sound processing unit 140 that checks the buffered sound data and reads the buffered sound data to at least one designated screaming frequency band to recognize a designated screaming sound for determining an emergency situation among non-verbal sounds or noises that may occur in a designated space. A scream recognition unit 145 for recognizing a specific scream corresponding to and/or at least one scream pattern designated for recognizing the scream, and a designated S (S ≥ 1) When at least s (1≤s≤S) service functions related to screams are activated and/or screams or user's utterances are not recognized for a specified period of time, among the service functions, the sound device 100 It has a service control unit 150 that sets the operation mode of the sleep mode to a sleep mode, and further includes a service providing unit 180 that provides a service corresponding to the activated service function. Meanwhile, according to another implementation method of the present invention, at least one of the scream recognition unit 145, the service control unit 150, and the service providing unit 180 is an operating server on a communication network that interworks with the sound device 100. 200, and if the scream recognition unit 145 is implemented in the operation server 200, the sound processing unit 140 transfers sound data buffered to a designated buffer area to the operation server 200. ), or selectively extract sound data determined to correspond to a scream among the buffered sound data and provide the sound data to the operation server 200. However, for convenience, Figure 1 describes the features of the present invention, focusing on an embodiment in which the scream recognition unit 145, the service control unit 150, and the service provider 180 are implemented through the sound device 100. . Meanwhile, when the scream recognition unit 145, the service control unit 150, and the service provider 180 are implemented through the sound device 100, the scream recognition unit 145, the service control unit 150, and the service The study 180 may be implemented in the form of at least one or a combination of two of a program code form and a chip or control circuit form, and the scream recognition unit 145, the service control unit 150, and the service provider 180 The present invention is not limited by an implementation method or implemented module.

After operating power is applied to the sound device 100 through the power module 115, the sound processing unit 140 generates sound data corresponding to a sound signal of an audible frequency band input through the sound input unit 105a. continues (or repeats) the process of buffering in the designated buffer area and/or checking the sound data buffered in the designated buffer area, and the scream recognition unit 145 reads the buffered sound data in the designated space. Recognizing a specific scream corresponding to at least one screaming frequency band designated for recognizing a designated scream for emergency determination and/or corresponding to at least one screaming pattern designated for recognizing the scream Continue (or repeat) the process. Meanwhile, when the scream recognition unit 145 is implemented in the operation server 200, the sound processing unit 140 provides sound data buffered in a designated buffer area to the operation server 200 or the buffered sound data Sound data determined to correspond to the screaming sound may be selectively extracted and provided to the operation server 200, and the present invention is not limited thereby.

The service control unit 150 checks whether a scream is not recognized for a specified period of time or longer after operating power is applied to the sound device 100 and/or effectively recognizes a scream through the scream recognition unit 145. After a specified period of time, it is confirmed that the screaming sound is not recognized effectively, and/or after the operating power is applied to the sound device 100, it is confirmed that the utterances uttered by the user are not recognized for a specified period of time or longer, and/or After the user's speech content is effectively recognized according to a designated procedure (eg, the user's speech content is recognized within a certain time during or after the scream is effectively recognized through the scream recognition unit 145), the user for a specified period of time or more It can be checked whether the contents of the utterance are not recognized. If the screaming sound is not recognized for more than a certain period of time or the contents of the user's speech are not recognized for more than a certain period of time, the service controller 150 sets the operation mode of the sound device 100 (or control module 130). It includes a function to set to sleep mode. Here, the sleep mode reads the buffered sound data to wake up from the sleep mode by recognizing a scream even if the contents of the user's speech are recognized, and/or pre-specified specific speech information (e.g., a scream sound). In addition, before waking up from sleep mode by recognizing specific speech information set to wake up the sound device 100, services based on the recognized user's speech (eg, sound source or signal sound) are recognized. This includes a state in which a function providing a sound-based service) is disabled. Hereinafter, for convenience, features of the present invention will be described focusing on an embodiment in which a scream is recognized to wake up from a sleep mode.

According to the implementation method of the present invention, the sound processing unit 140 buffers sound data corresponding to the sound signal of the audible frequency band input through the sound input unit 105a in a designated buffer area even in the sleep mode, and/or It is preferable to continue (or repeat) the process of checking the sound data buffered in the designated buffer area. A specific scream corresponding to at least one scream frequency band designated for recognizing screams designated for emergency situation determination among sounds or noises and/or corresponding to at least one scream pattern designated for recognizing the screams. It is desirable to continue (or repeat) the process of recognizing sound. Preferably, when a scream is effectively recognized through the scream recognition unit 145 in the sleep mode, the service control unit 150 wakes up to activate s number of service functions related to the scream. The service providing unit 180 may perform a procedure of providing services corresponding to the activated s number of service functions.

Meanwhile, when two or more service functions related to screams are activated through the service control unit 150, the service providing unit 180 performs a procedure for providing services corresponding to the activated service functions according to a designated order, or And/or a procedure for providing at least one service according to a predetermined condition (eg, a condition for recognizing user's speech) among the activated service functions may be performed.

According to the implementation method of the present invention, the scream recognition unit 145 corresponds to the scream frequency band of the scream sound generating source and/or the unique scream sound of the scream sound generating source corresponding to the scream pattern of the scream sound generating source. For this purpose, the control module 130 has a function of learning at least one screaming sound frequency band specific to a screaming sound source and/or a function of learning at least one screaming sound pattern specific to a screaming sound generating source. It is desirable to have.

Referring to Figure 1, the control module 130 stores M screaming frequency band information corresponding to M (M≥1) screaming frequency bands in a designated storage area and/or N (N≥1) and an information storage unit 135 for storing N pieces of scream pattern information corresponding to ) number of scream patterns in a designated storage area, and the scream sound recognition unit 145 uses a scream sound frequency band stored in the designated storage area. A scream may be recognized using information and/or scream pattern information. Meanwhile, according to another implementation method of the present invention, at least one component of the information storage unit 135 and the scream recognition unit 145 is implemented through the operating server 200 on a communication network that works with the sound device 100. It can be. However, for convenience, in Figure 1, the characteristics of the present invention will be described focusing on an embodiment in which the information storage unit 135 and the scream recognition unit 145 are implemented through the sound device 100. On the other hand, when the information storage unit 135 and the scream recognition unit 145 are implemented through the sound device 100, the information storage unit 135 and the scream recognition unit 145 are in the form of program code and chip or control. It may be implemented in at least one or a combination of two circuit types, and the present invention is not limited by a method of implementing the information storage unit 135 and the scream recognition unit 145 or a module to be implemented.

The information storage unit 135 stores M screams corresponding to M (M≥1) scream frequency bands in a designated storage area of the memory unit at the time of manufacturing the control module 130 (or sound device 100). Frequency band information may be stored. Among the non-verbal sounds or noises that can be generated in the designated space in the present invention, the scream designated for emergency determination is a wide range of sounds (or a frequency difference between the lowest frequency and the highest frequency of at least 100 Hz or more) depending on the type of sound or sound source. ) can have a frequency band. Therefore, the present invention can divide the frequency band of the screaming sound into M frequency bands (eg, equal division or selective division based on a plurality of frequency bands in which screams are experimentally concentrated), and the information storage unit 135 may store M pieces of screaming sound frequency band information corresponding to the divided M screaming sound frequency bands in a designated storage area.

The information storage unit 135 stores N scream patterns corresponding to N (N≥1) scream patterns in a designated storage area of the memory unit at the time of manufacturing the control module 130 (or sound device 100). Information can be stored in a designated storage area. First of all, the present invention generates scream pattern information through qualitative and/or quantitative analysis of the scream pattern identified through sample experiments for each case, and the information storage unit 135 stores the generated scream pattern. Information can be stored in a designated storage area.

On the other hand, according to the implementation method of the present invention, the sound device 100 (or control module 130) can transmit the recognized screaming sound information to the designated operation server 200 after recognizing the screaming sound. The server 200 collects scream information from each sound device 100 (or control module 130), and normalizes the individual sound characteristics of each sound source from the collected screams, and / Or through the process of qualitative analysis and / or quantitative analysis for each case, the scream pattern information can be refined or diversified, and thus the refined or diversified scream pattern information can be sent to each sound device 100 (or control module). (130)). The information storage unit 135 receives refined or diversified scream pattern information from the operation server 200 through the above process, and generates N pieces of scream pattern information including the received scream pattern information. It can be stored in the designated storage area.

According to the implementation method of the present invention, M scream frequency band information corresponding to the M scream sound frequency bands is stored in the designated storage area and/or N scream sounds corresponding to the N scream patterns are stored in the designated storage area. When sound pattern information is stored, the scream recognition unit 145 reads the buffered sound data to correspond to at least one screaming frequency band among the stored M screaming frequency bands and/or the stored N screaming frequency bands. It is desirable to recognize a scream corresponding to at least one of the scream patterns.

Meanwhile, the M pieces of scream sound frequency band information and/or N pieces of scream pattern information stored in the designated storage area are not optimized (or customized) for the scream sound of the scream sound generator, but statistical frequency bands and/or sounds of scream sounds. It will be information corresponding to the pattern. Therefore, in order to recognize the screaming sound of the screaming sound source existing in the space where the sound device 100 is mounted at high speed and/or prevent (or minimize) recognition errors, the stored screaming frequency band information and/or screaming sound A process of optimizing (or customizing) the pattern information to the screaming sound of the screaming sound generator is required.

Referring to FIG. 1, the control module 130 controls the scream frequency corresponding to the frequency distribution of screams recognized at least i (i≥2) times during a designated period using the M pieces of scream frequency band information. Distribution generation that generates distribution information and/or generates scream pattern distribution information corresponding to scream pattern distributions that have been validly recognized at least j (j≥2) times during a specified period of time by using the N pieces of scream pattern information. unit 155 and m (1≤m≤M) optimized (or customized) to the screaming sound of the screaming sound generator in the space through which the sound device 100 passed through statistical processing of the generated scream frequency distribution information m screaming frequency band information corresponding to ) number of screaming sound frequency bands and/or statistically processing the generated screaming sound pattern distribution information, An information determination unit 160 for determining n pieces of scream pattern information corresponding to n (1≤n≤N) scream patterns optimized (or customized) for screams is further provided. On the other hand, according to another implementation method of the present invention, at least one component of the distribution generating unit 155 and the information determining unit 160 is implemented through the operation server 200 on a communication network that works with the sound device 100. can However, for convenience, the features of the present invention will be described in FIG. 1 based on an embodiment in which the distribution generating unit 155 and the information determining unit 160 are implemented through the sound device 100. Meanwhile, when the distribution generating unit 155 and the information determining unit 160 are implemented through the sound device 100, the distribution generating unit 155 and the information determining unit 160 are in the form of program codes and chips or control circuits. It may be implemented in the form of at least one or a combination of the two, and the present invention is not limited by a method of implementing the distribution generating unit 155 and the information determining unit 160 or a module to be implemented.

When M pieces of scream sound frequency band information corresponding to the M scream sound frequency bands are stored in the storage area designated by the information storage unit 135, the distribution generating unit 155 stores the M scream sounds stored in the designated storage area. Whenever a scream is validly recognized using the sound frequency band information, the effectively recognized frequency band information of the screaming sound may be stored in a designated storage area. The distribution generating unit 155 is validly recognized through the M screaming frequency band information and i screaming frequency bands that are validly recognized at least i (i≥2) times during a specified period of time based on the stored screaming frequency bands. , and scream frequency distribution information corresponding to the distribution of the identified i scream frequency bands may be generated.

On the other hand, when scream frequency distribution information corresponding to the scream frequency distribution that has been recognized validly at least i during a specified period of time is generated by the distribution generator 155, the information determination unit 160 determines the generated scream frequency Distribution information is statistically processed to optimize (or customize) the screaming sound of the screaming source in the space through which the sound device 100 passed through, thereby recognizing the screaming sound of the screaming source at high speed or preventing (or minimizing) recognition errors It is possible to determine m pieces of screaming sound frequency band information corresponding to m (1≤m≤M) screaming frequency bands. Here, the m screaming sound frequency bands may be any one of the M screaming sound frequency bands, or may include a newly defined screaming sound frequency band within the screaming sound frequency band range. When the m scream frequency band information is determined through the information determination unit 160, the information storage unit 135 sets the determined m scream frequency bands as screaming frequency bands to be read first, and stores the designated information. It can be newly stored in the storage area or updated and stored in the screaming frequency band information stored in the storage area. Thereafter, the scream recognition unit 145 reads the buffered sound data and first recognizes a scream corresponding to at least one scream frequency band among the new or updated m scream frequency bands.

Meanwhile, when N scream pattern information corresponding to the N scream patterns is stored in the storage area designated by the information storage unit 135, the distribution generator 155 stores the N scream sounds stored in the designated storage area. Whenever a scream is effectively recognized using the pattern information, the sophie pattern of the validly recognized scream may be stored in a designated storage area. The distribution generating unit 155 verifies j scream patterns that have been validly recognized and stored through the N pieces of scream pattern information and are validly recognized at least j (j≥2) times during a designated period of time based on the stored scream pattern information. and generate scream pattern distribution information corresponding to the distribution of the identified j scream patterns.

Meanwhile, when scream pattern distribution information corresponding to the validly recognized scream pattern distribution at least j times during a specified period of time is generated through the distribution generating unit 155, the information determination unit 160 determines the generated scream pattern Distribution information is statistically processed to optimize (or customize) the screaming sound of the screaming source in the space through which the sound device 100 passed through, thereby recognizing the screaming sound of the screaming source at high speed or preventing (or minimizing) recognition errors It is possible to determine n pieces of screaming pattern information corresponding to n (1≤n≤N) screaming patterns for processing. Here, the n scream patterns may be any one of the N scream patterns, or may include a combination of two or more scream patterns or a new cry soo pattern. When n scream patterns are determined through the information determination unit 160, the information storage unit 135 sets the determined n scream patterns as scream patterns to be read first and newly stores them in a designated storage area. Scream frequency band information stored in the storage area may be updated and stored. Thereafter, the scream recognition unit 145 may first recognize a scream corresponding to at least one scream pattern among the n scream patterns by reading the buffered sound data.

Referring to FIG. 1, the control module 130 reads the buffered sound data, recognizes the user's language utterance, and processes natural language to recognize speech content information corresponding to the user's utterance content. 165), and a recognition determination unit 170 for confirming whether the contents of the user's speech are recognized while the scream is being recognized or within a certain time after the end of the recognition of the scream, and a recognition determination unit 170; When the content of the user's speech is recognized within a certain time during or after the recognizing, the intelligent processing unit 175 is provided to process the recognized content of the user's speech in association with s number of service functions related to the scream. there is. On the other hand, according to another implementation method of the present invention, at least one component of the utterance recognition unit 165, the recognition determination unit 170, and the intelligence processing unit 175 is an operation server on a communication network that interworks with the sound device 100. 200. If the speech content recognition unit 165 is implemented in the operation server 200, the sound processing unit 140 transfers sound data buffered to a designated buffer area to the operation server 200. ), or selectively extract sound data determined to correspond to the user's speech among the buffered sound data and provide the sound data to the operation server 200. However, for convenience, Figure 1 will explain the features of the present invention based on an embodiment in which the utterance recognition unit 165, the recognition determination unit 170, and the intelligence processing unit 175 are implemented through the sound device 100. . Meanwhile, when the speech contents recognition unit 165, the recognition determination unit 170, and the intelligence processing unit 175 are implemented through the sound device 100, the speech contents recognition unit 165, the recognition determination unit 170 and the intelligence processing unit 175 are implemented. The processing unit 175 may be implemented in the form of at least one or a combination of two of a program code form and a chip or control circuit form, and the utterance recognition unit 165, the recognition determination unit 170, and the intelligence processing unit 175 The present invention is not limited by an implementation method or implemented module.

At the same time as performing a process of recognizing a scream by reading the buffered sound data through the scream recognition unit 145, the speech content recognition unit 165 reads the buffered sound data to utter the user's language and performs a procedure of recognizing speech content information corresponding to the user's speech content by natural language processing. Meanwhile, when the utterance recognition unit 165 is implemented in the operation server 200, the sound processing unit 140 provides sound data buffered in a designated buffer area to the operation server 200 or the buffered sound data It is possible to selectively extract sound data that is determined to correspond to the user's utterance (eg, including segmented sounds) and provide it to the operation server 200, whereby the present invention is not limited.

According to the implementation method of the present invention, the speech content recognition unit 165 analyzes the buffered sound data through a designated speech recognition algorithm, recognizes and converts the speech spoken by the user into text, and simultaneously analyzes form, semantics, and dialogue. Speech content information by recognizing the user's utterance content may be generated through one or more natural language processing procedures during the analysis.

Meanwhile, according to the implementation method of the present invention, the speech content recognition unit 165 recognizes the screaming sound of the screaming sound recognition unit 145 through the service control unit 150 so that the operation mode of the service device is in the sleep mode. It is preferable to perform a function of recognizing the user's utterance in case of waking up. If the operation mode of the sound device 100 is set to the sleep mode by the service control unit 150, the speech content recognition unit 165 may not perform a function of recognizing the user's speech content. Even if the content of the user's speech is recognized, it can be processed so that it is not used for the service.

Meanwhile, when the user's speech content is recognized through the speech content recognition unit 165, the recognition determination unit 170 determines that the user's speech content is recognized while the scream sound is recognized through the scream sound recognition unit 145. It is checked whether or not the user's utterance is recognized within a certain period of time after the recognition of the scream is terminated through the scream recognition unit 145.

If it is confirmed through the recognition determination unit 170 that the content of the user's speech has been recognized during or after the scream is recognized, the intelligence processing unit 175 determines the contents of speech recognition unit 165 The user's speech content recognized through is interpreted in association with s number of service functions related to the screaming sound, whereby a specific service function corresponding to the speech content among the s number of service functions related to the screaming sound can be determined. In this case, the service controller 150 may activate a specific service function related to the scream and corresponding to the recognized user's utterance, and the service provider 180 may activate the specific service function. The provided procedure can be performed according to the specified procedure.

If the content of the user's utterance is not recognized within a certain period of time during or after the scream is recognized through the recognition determination unit 170, the intelligence processing unit 175 enters the sleep mode through the service control unit 150. and/or a process of waking up from the sleep mode by the screaming sound may be performed.

On the other hand, according to the implementation method of the present invention, the s number of services related to screams activated by the service control unit 150 and/or provided through the service providing unit 180 are provided on the communication network through the communication module 120. A server interworking service provided in conjunction with the operating server 200, a sound-based service provided through the sound module 105, and a composite service provided by using the sound module 105 and the communication module 120 in combination. It may include at least one of the services.

On the other hand, among the s services related to the scream, the server linkage service is a call service using wireless terminal information, a situation notification service using wireless terminal information, a call service using an app installed on the wireless terminal, and using an app installed on the wireless terminal. At least one of a situation notification service, a video service that provides video information captured through at least one camera to an app installed in a wireless terminal, and a video service that provides video information captured through at least one camera to a designated IPTV device can include

Meanwhile, among the s number of services related to the scream, the sound-based service may include a noise output service for outputting a designated service sound at a designated signal output strength through the sound output unit 105b of the sound module 105. Preferably, the signal output strength of the sound-based service includes a signal strength that is inversely proportional to the signal input strength of the scream.

On the other hand, among the s services related to the scream, the complex service recognizes the user's speech within a certain time during or after the scream is recognized, and analyzes the user's speech by associating the user's speech with a service function related to the scream Based on one result, the service provided through the operation server 200 on the communication network may be included.

Meanwhile, according to the first camera embodiment of the present invention, the sound device 100 is provided with a short-range module 125 for processing short-range wireless communication with the short-range camera device 185, and through the short-range module 125 When processing short-range wireless communication with the short-range camera device 185, the service control unit 150 recognizes a scream through the scream recognition unit 145, and the camera device 125 through the short-range module 125. By communicating with the device 185, s service functions including an image service function for photographing a sound generating source that generates the recognized scream through the camera device 185 may be activated, and the service providing unit 180 ) communicates with the camera device 185 through the short-distance module 125, receives image information captured through the camera device 185, and provides the image to the app installed in the wireless terminal through the operation server 200. At least one video service among service and/or video services provided to a designated IPTV device may be provided. Meanwhile, when the camera device 185 can provide the video service, the service providing unit 180 may process the video service function of the camera device 185 to operate through the short distance module 125 .

Alternatively, according to the second camera embodiment of the present invention, wireless communication based on a local area network with the camera device 185 is performed through the local area network communication unit 120b of the communication module 120 included in the sound device 100. In case of processing, the service control unit 150 communicates with the camera device 185 through the local area network communication unit 120b when recognizing a scream through the scream recognition unit 145, and the camera device ( 185), it is possible to activate s number of service functions including a video service function for photographing the sound generating source that generated the recognized screaming sound, and the service provider 180 via the local area network communication unit 120b Communicate with the camera device 185 to receive video information captured through the camera device 185 and provide it to an app installed in the wireless terminal through the operation server 200 and / or to a designated IPTV device At least one video service among video services may be provided. Meanwhile, when the camera device 185 can provide the video service, the service providing unit 180 may process the video service function of the camera device 185 to operate through the local area network communication unit 120b.

Alternatively, according to the third camera embodiment of the present invention, when the sound device 100 is provided with a camera module 110 for obtaining image information of a photographed subject, the service control unit 150 is configured to use the scream recognition unit ( 145), when a scream is recognized, a video service function for photographing a sound source generating the recognized scream may be activated through the camera module 110, and the service provider 180 may activate the At least one of a video service providing video information captured by the camera module 110 to an app installed in a wireless terminal through the operation server 200 and/or a video service provided to a designated IPTV device may be provided. there is.

Figure 2 is a diagram showing the functional configuration of the sound device 100 and the operating server 200 according to another embodiment of the present invention.

In more detail, FIG. 2 shows an embodiment in which at least some of the functional configurations of the control module 130 of the sound device 100 shown in FIG. 1 are implemented on the operation server 200 side, Those of ordinary skill in the art to which the present invention pertains, a method of at least partially modifying the embodiments of FIG. 1 and 2 by referring to and/or modifying FIG. 2 (eg, FIG. 2 Some of the functional configurations implemented on the operation server 200 are implemented on the operation server 200 side as they are, but other parts are implemented in the control module 130 of the sound device 100 or the control module 130 of the above drawing 1 An implementation method in which some of the functional configurations implemented in the control module 130 are implemented as they are in the control module 130, but other parts are implemented on the operation server 200 side. However, the present invention includes all the inferred implementation methods as the scope of rights, and the technical characteristics are not limited only to the implementation methods shown in Figure 1 and this Figure 2. No.

Referring to Figure 2, the operation server 200, the sound receiving unit 205 for receiving the sound data input through the sound input unit 105a of the designated sound device 100 and temporarily storing it in a designated storage medium, Reading the received sound data to correspond to at least one screaming frequency band designated for recognizing a designated screaming sound for emergency situation determination among non-verbal sounds or noises that may occur in a designated space, and/or recognizing the screaming sound a scream recognition unit 145 for recognizing a specific scream corresponding to at least one scream pattern designated for the scream, and when recognizing the scream, at least one scream related to the scream among designated S (S≥1) service functions. When s (1 ≤ s ≤ S) number of service functions are activated and / or screams or user's speech are not recognized for a specified period of time or longer, the operation mode of the sound device 100 is set to Sleep mode and a service control unit 150 that provides a service corresponding to the activated service function.

When sound data input through the sound input unit 105a is transmitted from the designated sound device 100, the sound receiving unit 205 receives the sound data transmitted from the sound apparatus 100 and temporarily stores the sound data in the designated storage medium. can be stored Meanwhile, when the sound device 100 selectively transmits one or more sound data among sound data corresponding to a scream or sound data corresponding to a user's utterance, the sound receiver 205 transmits from the sound device 100 The selectively transmitted sound data may be received and temporarily stored in a designated storage medium.

The screaming sound recognition unit 145 reads the received sound data and corresponds to at least one screaming frequency band designated to recognize a designated screaming sound for determining an emergency situation among non-verbal sounds or noises that may occur in a designated space. and/or continues (or repeats) a process of recognizing a specific scream corresponding to at least one scream pattern designated for recognizing the scream.

The service control unit 150 checks whether a scream is not recognized for a specified period of time or longer after operating power is applied to the sound device 100 and/or effectively recognizes a scream through the scream recognition unit 145. After a specified period of time, it is confirmed that the screaming sound is not recognized effectively, and/or after the operating power is applied to the sound device 100, it is confirmed that the utterances uttered by the user are not recognized for a specified period of time or longer, and/or After the user's speech content is effectively recognized according to a designated procedure (eg, the user's speech content is recognized within a certain time during or after the scream is effectively recognized through the scream recognition unit 145), the user for a specified period of time or more It can be checked whether the contents of the utterance are not recognized. If the screaming sound is not recognized for a specified period of time or longer or the user's utterance is not recognized for a specified period of time or longer, the service control unit 150 remotely sets the operation mode of the sound device 100 to a sleep mode. It can be made including the function to do.

According to the implementation method of the present invention, the sound device 100 can transmit the input sound data in a sleep mode, and the sound receiver 205 determines that the operation mode of the sound device 100 is the sleep mode. receiving the sound data in this state, and the scream recognition unit 145 also reads the received sound data in a state where the operation mode of the sound device 100 is the sleep mode, and non-verbal sounds or noises that can be generated in a designated space Recognizes a specific scream corresponding to at least one screaming frequency band designated for recognizing a designated scream and/or corresponding to at least one scream pattern designated for recognizing the scream. The process can be continued (or repeated). Preferably, when a scream is effectively recognized through the scream recognition unit 145 in the sleep mode, the service control unit 150 wakes up the sound device 100 and simultaneously displays s number of sounds related to the scream. A service function may be activated, and the service providing unit 180 may perform a procedure for providing a service corresponding to the activated s number of service functions.

Meanwhile, when two or more service functions related to screams are activated through the service control unit 150, the service providing unit 180 performs a procedure for providing services corresponding to the activated service functions according to a designated order, or And/or a procedure for providing at least one service according to a predetermined condition (eg, a condition for recognizing user's speech) among the activated service functions may be performed.

According to the implementation method of the present invention, the scream recognition unit 145 corresponds to the scream frequency band of the scream sound generator and/or the unique scream sound of the scream source corresponding to the scream pattern of the scream sound generator. It is preferable to recognize, and for this purpose, the operation server 200 has a function of learning at least one screaming sound frequency band specific to a screaming sound source and/or a function of learning at least one screaming sound pattern specific to a screaming sound generating source. It is desirable to have.

Referring to FIG. 2, the operation server 200 stores M pieces of screaming frequency band information corresponding to M (M≥1) screaming frequency bands in a designated storage medium and/or stores N (N≥1) pieces of screaming frequency band information. An information storage unit 135 for storing N pieces of scream sound pattern information corresponding to ) number of scream patterns in a designated storage medium, and the scream sound recognition unit 145 uses a scream sound frequency band stored in the designated storage medium. A scream may be recognized using information and/or scream pattern information.

The information storage unit 135 may store M pieces of screaming sound frequency band information corresponding to M (M≧1) screaming sound frequency bands in a designated storage medium. Among the non-verbal sounds or noises that can be generated in the designated space in the present invention, the scream designated for emergency determination is a wide range of sounds (or a frequency difference between the lowest frequency and the highest frequency of at least 100 Hz or more) depending on the type of sound or sound source. ) can have a frequency band. Therefore, the present invention can divide the frequency band of the screaming sound into M frequency bands (eg, equal division or selective division based on a plurality of frequency bands in which screams are experimentally concentrated), and the information storage unit 135 may store M pieces of screaming sound frequency band information corresponding to the divided M screaming sound frequency bands in a designated storage medium.

The information storage unit 135 may store N pieces of scream pattern information corresponding to N (N≥1) scream patterns in a specified storage medium. First of all, the present invention generates scream pattern information through qualitative and/or quantitative analysis of the scream pattern identified through sample experiments for each case, and the information storage unit 135 stores the generated scream pattern. Information can be stored in a designated storage medium.

On the other hand, according to the implementation method of the present invention, the sound device 100 can transmit the recognized screaming sound information to the designated operation server 200 after recognizing the screaming sound, and the intelligent processing unit 175 of the operating server 200 ) is a normalization process of collecting scream information from each sound device 100 and normalizing individual sound characteristics for each sound source from the collected screams and/or qualitative analysis and/or quantitative analysis for each case. Through the analysis process, the scream pattern information can be refined or diversified. The information storage unit 135 may receive refined or diversified scream pattern information through the above process, and store N pieces of scream pattern information including the received scream pattern information in a designated storage medium. .

According to the implementation method of the present invention, M scream frequency band information corresponding to the M scream sound frequency bands is stored in the designated storage medium, and/or N scream sounds corresponding to the N scream patterns are stored in the designated storage medium. When sound pattern information is stored, the scream recognition unit 145 reads the received sound data to correspond to at least one screaming frequency band among the stored M screaming frequency bands and/or the stored N screaming frequency bands. It is desirable to recognize a scream corresponding to at least one of the scream patterns.

Meanwhile, the M number of screaming sound frequency band information and/or N pieces of screaming pattern information stored in the designated storage medium are not optimized (or customized) for the screaming sound of the screaming source, but statistically the frequency band and/or sound of the screaming sound. It will be information corresponding to the pattern. Therefore, in order to recognize the screaming sound of the screaming sound source existing in the space where the sound device 100 is mounted at high speed and/or prevent (or minimize) recognition errors, the stored screaming frequency band information and/or screaming sound A process of optimizing (or customizing) the pattern information to the screaming sound of the screaming sound generator is required.

Referring to FIG. 2, the operation server 200 uses the M number of screaming sound frequency band information for a specified period of time, at least i (i≥2) times, the screaming sound frequency corresponding to the recognized screaming frequency distribution. Distribution generation that generates distribution information and/or generates scream pattern distribution information corresponding to scream pattern distributions that have been validly recognized at least j (j≥2) times during a specified period of time by using the N pieces of scream pattern information. unit 155 and m (1≤m≤M) optimized (or customized) to the screaming sound of the screaming sound generator in the space through which the sound device 100 passed through statistical processing of the generated scream frequency distribution information m screaming frequency band information corresponding to ) number of screaming sound frequency bands and/or statistically processing the generated screaming sound pattern distribution information, An information determination unit 160 for determining n pieces of scream pattern information corresponding to n (1≤n≤N) scream patterns optimized (or customized) for screams is further provided.

When M pieces of screaming sound frequency band information corresponding to the M screaming sound frequency bands are stored in a storage medium designated through the information storage unit 135, the distribution generating unit 155 determines the M number of screaming sounds stored in the designated storage medium. Using the sound frequency band information, at least i (i≥2) times during a specified period of time, i (i≥2) times are identified as valid scream frequency bands, and a scream frequency distribution corresponding to the distribution of the identified i scream frequency bands. information can be generated.

On the other hand, when scream frequency distribution information corresponding to the scream frequency distribution that has been recognized validly at least i during a specified period of time is generated by the distribution generator 155, the information determination unit 160 determines the generated scream frequency Distribution information is statistically processed to optimize (or customize) the screaming sound of the screaming source in the space through which the sound device 100 passed through, thereby recognizing the screaming sound of the screaming source at high speed or preventing (or minimizing) recognition errors It is possible to determine m pieces of screaming sound frequency band information corresponding to m (1≤m≤M) screaming frequency bands. Here, the m screaming sound frequency bands may be any one of the M screaming sound frequency bands, or may include a newly defined screaming sound frequency band within the screaming sound frequency band range. When the m scream frequency band information is determined through the information determination unit 160, the information storage unit 135 sets the determined m scream frequency bands as screaming frequency bands to be read first, and stores the designated information. It may be newly stored in a medium or the scream sound frequency band information stored in the storage medium may be updated and stored. Thereafter, the scream recognition unit 145 reads the received sound data to first recognize a scream corresponding to at least one scream frequency band among the new or updated m scream frequency bands.

Meanwhile, when N scream pattern information corresponding to the N scream patterns is stored in a designated storage medium through the information storage unit 135, the distribution generating unit 155 determines the N scream patterns stored in the designated storage medium. Using the sound pattern information, at least j (j≥2) times during a specified period of time, validly recognized j scream patterns are identified, and scream pattern distribution information corresponding to the distribution of the identified j scream patterns is generated. can do.

Meanwhile, when scream pattern distribution information corresponding to the validly recognized scream pattern distribution at least j times during a specified period of time is generated through the distribution generating unit 155, the information determination unit 160 determines the generated scream pattern Distribution information is statistically processed to optimize (or customize) the screaming sound of the screaming source in the space through which the sound device 100 passed through, thereby recognizing the screaming sound of the screaming source at high speed or preventing (or minimizing) recognition errors It is possible to determine n pieces of screaming pattern information corresponding to n (1≤n≤N) screaming patterns for processing. Here, the n scream patterns may be any one of the N scream patterns, or may include a combination of two or more scream patterns or a new scream pattern. When n scream patterns are determined through the information determination unit 160, the information storage unit 135 sets the determined n scream patterns as scream patterns to be read first and stores them in a designated storage medium, or Scream frequency band information stored in the storage medium may be updated and stored. Thereafter, the scream recognition unit 145 may first recognize a scream corresponding to at least one scream pattern among the n scream patterns by reading the received sound data.

Referring to FIG. 2, the operation server 200 reads the received sound data, recognizes the user's language utterance, and processes natural language to recognize the utterance content information corresponding to the user's utterance content recognition unit ( 165), and a recognition determination unit 170 for confirming whether the contents of the user's speech are recognized while the scream is being recognized or within a certain time after the end of the recognition of the scream, and a recognition determination unit 170; When the content of the user's speech is recognized within a certain time during or after the recognizing, the intelligent processing unit 175 is provided to process the recognized content of the user's speech in association with s number of service functions related to the scream. there is.

At the same time as performing a process of recognizing a scream by reading the received sound data through the scream recognition unit 145, the speech content recognition unit 165 reads the received sound data to utter the user's language and performs a procedure of recognizing speech content information corresponding to the user's speech content by natural language processing.

According to the implementation method of the present invention, the speech content recognition unit 165 analyzes the received sound data through a designated speech recognition algorithm to recognize and characterize the speech spoken by the user, and at the same time analyze the form, semantics, and dialogue. Speech content information by recognizing the user's utterance content may be generated through one or more natural language processing procedures during the analysis.

Meanwhile, according to the implementation method of the present invention, the speech content recognition unit 165 recognizes the screaming sound of the screaming sound recognition unit 145 through the service control unit 150 so that the operation mode of the service device is in the sleep mode. It is preferable to perform a function of recognizing the user's utterance in case of waking up. If the operation mode of the sound device 100 is set to the sleep mode by the service control unit 150, the speech content recognition unit 165 may not perform a function of recognizing the user's speech content. Even if the content of the user's speech is recognized, it can be processed so that it is not used for the service.

Meanwhile, when the user's speech content is recognized through the speech content recognition unit 165, the recognition determination unit 170 determines that the user's speech content is recognized while the scream sound is recognized through the scream sound recognition unit 145. It is checked whether or not the user's utterance is recognized within a certain period of time after the recognition of the scream is terminated through the scream recognition unit 145.

If it is confirmed through the recognition determination unit 170 that the content of the user's speech has been recognized during or after the scream is recognized, the intelligence processing unit 175 determines the contents of speech recognition unit 165 The user's speech content recognized through is interpreted in association with s number of service functions related to the screaming sound, whereby a specific service function corresponding to the speech content among the s number of service functions related to the screaming sound can be determined. In this case, the service control unit 150 may activate a specific service function related to the specific sound and corresponding to the recognized user's utterance, and the service providing unit 180 may activate the specific service function. The provided procedure can be performed according to the specified procedure.

If the content of the user's utterance is not recognized within a certain period of time during or after the scream is recognized through the recognition determination unit 170, the intelligence processing unit 175 enters the sleep mode through the service control unit 150. and/or a process of waking up from the sleep mode by the screaming sound may be performed.

On the other hand, according to the implementation method of the present invention, the service control unit 150 of the operation server 200 is activated and / or s services related to screams that can be provided through the service providing unit 180 are the operation server ( 200) and at least one of a server-based service provided through the sound device 100 and a composite service provided in conjunction with the sound device 100.

Meanwhile, among the s services related to the scream, the server-based service is a call service using wireless terminal information, a situation notification service using wireless terminal information, a call service using an app installed on the wireless terminal, and a call service using an app installed on the wireless terminal. At least one of a situation notification service, a video service that provides video information captured through at least one camera to an app installed in a wireless terminal, and a video service that provides video information captured through at least one camera to a designated IPTV device can include

On the other hand, among the s services related to the scream, the complex service recognizes the user's speech within a certain time during or after the scream is recognized, and analyzes the user's speech by associating the user's speech with a service function related to the scream Based on one result, the service provided through the operation server 200 on the communication network may be included.

3 illustrates an embodiment of recognizing a scream and activating a service related to the scream according to an embodiment of the present invention.

In more detail, FIG. 3 shows an embodiment of recognizing a scream using M pieces of scream sound frequency band information and/or N pieces of scream pattern information stored in a designated storage area and activating a service related to the recognized scream sound. , and those skilled in the art to which the present invention pertains may refer to and / or modify this figure 3 to various implementation methods for the scream recognition or service activation process (eg, some steps are omitted However, the present invention includes all the inferred implementation methods, and its technical characteristics are not limited only to the implementation method shown in FIG. 3. On the other hand, the process illustrated and described in FIG. 3 is provided through the sound device 100, the operation server 200, or provided in a distributed form through interworking between the sound device 100 and the operation server 200. It is possible, and for convenience, the features of the present invention will be described in FIG. 3 focusing on the process provided through the sound device 100.

Referring to the embodiment of FIG. 3, the sound device 100 stores M pieces of screaming sound frequency band information corresponding to M screaming sound frequency bands for recognizing screams in a designated storage area (300) and/or N pieces of scream pattern information corresponding to the N number of scream patterns for recognizing screams are stored in a designated storage area (300). Thereafter, the sound device 100 receives sound data corresponding to the sound signal of the audible frequency band through the sound input unit 105a, buffers it in a designated buffer area (305), reads the buffered sound data, and reads the sound data in the designated space. Corresponding to at least one screaming frequency band designated for recognizing a screaming sound designated for emergency situation determination and/or corresponding to a specific screaming sound corresponding to at least one screaming pattern designated for recognizing the screaming sound A procedure for recognizing a scream is performed (310).

According to one embodiment of the present invention, the sound device 100 reads the buffered sound data to correspond to at least one screaming frequency band among the stored M screaming frequency bands and/or the stored N screaming frequency bands. A scream corresponding to at least one of the scream patterns may be recognized (310). According to another embodiment of the present invention, through the process of FIG. 4 and/or FIG. 5, m screaming frequencies optimized (or customized) for the screaming sound of the screaming sound generator in the space through which the sound device 100 has passed through When band information is determined and stored in a designated storage area and/or n pieces of scream pattern information optimized (or customized) for the screams of the screaming source are determined and stored in a designated storage area, the sound device 100 reads the buffered sound data to correspond to at least one screaming frequency band among the stored m screaming frequency bands and/or a screaming sound corresponding to at least one screaming pattern among the stored n screaming patterns It is possible to recognize the unique screaming sound of the originator (310).

If the operation mode of the sound device 100 is the sleep mode at the time of recognizing the scream, the sound device 100 may be woken up in the sleep mode (315).

Meanwhile, when the scream is recognized and/or the operation mode of the sound apparatus 100 wakes up from the sleep mode, the sound apparatus 100 is associated with the scream among designated S (S≥1) service functions. At least s (1≤s≤S) number of service functions may be activated (320), and a procedure for providing at least one service function among the activated s number of service functions according to a specified procedure may be performed (325).

Meanwhile, when the sound data is read and the scream is not recognized, the sound device 100 checks whether the scream is not effectively recognized for a designated period of time (330). If the screaming sound is not effectively recognized for a designated period of time, the sound device 100 may set the operation mode of the sound device 100 to a sleep mode (335).

Figure 4 shows an embodiment of optimizing (or customizing) a screaming sound frequency band for recognizing a screaming sound to the screaming sound of a screaming sound generator according to an embodiment of the present invention.

In more detail, FIG. 4 shows scream frequency distribution information corresponding to scream frequency distributions that have been effectively recognized at least i (i≥2) times during a specified period of time using M pieces of scream frequency band information pre-stored in a designated storage area. and m (1≤m≤M) number of optimized (or customized) screams of the screaming sound generator in the space through which the sound device 100 passed through by statistically processing the generated screaming frequency distribution information. It shows an embodiment of determining m pieces of screaming sound frequency band information corresponding to screaming sound frequency bands and storing them in a designated storage area as new/updated. Various implementation methods (eg, implementation methods in which some steps are omitted or the order is changed) for the process of optimizing (or customizing) the screaming sound frequency band may be inferred by referring to and/or modifying FIG. The invention is made including all the implementation methods inferred above, and its technical characteristics are not limited only to the implementation method shown in FIG. 4. On the other hand, the process illustrated and described in FIG. 4 is provided through the sound device 100, the operation server 200, or provided in a distributed form through interworking between the sound device 100 and the operation server 200. It is possible, and for convenience, the features of the present invention will be described mainly in the process provided through the sound device 100 in FIG. 4 .

Referring to the embodiment of FIG. 4, the sound device 100 effectively recognizes screams by using M pieces of scream sound frequency band information pre-stored in a designated storage area, and then the screams used to effectively recognize the screams. The frequency band information is stored in a designated storage area and managed (400).

If i (i≥2) pieces of scream frequency band information used to validly recognize screams for a specified period of time are stored in the storage area (405), the sound apparatus 100 provides at least i (i for a specified period of time). ≥ 2) generates scream frequency distribution information corresponding to the recognized scream frequency distribution (410), statistically processes the generated scream frequency distribution information, and After determining (415) m pieces of screaming sound frequency band information corresponding to m (1≤m≤M) screaming frequency bands optimized (or customized) for the screaming sound of the screaming source, the determined m screaming frequencies The band is first set as the screaming frequency band to be read, and new/updated storage is stored in the designated storage area (420). Then, the sound device 100 recognizes the screaming sound using the stored m screaming frequency band information. can

5 illustrates an embodiment of optimizing (or customizing) a scream pattern for recognizing a scream to the scream of a scream source according to another embodiment of the present invention.

In more detail, FIG. 5 shows scream pattern distribution information corresponding to scream pattern distributions that have been recognized validly at least j (j≥2) times during a specified period of time using N pieces of scream pattern information pre-stored in a designated storage area. n (1≤n≤N) number of screams optimized (or customized) for the screams of the screaming sound generator in the space through which the sound device 100 passed through by generating and statistically processing the generated scream pattern distribution information It shows an embodiment of determining n pieces of screaming sound pattern information corresponding to a sound pattern and storing them in a new/updated storage area in a designated storage area. Various implementation methods (eg, implementation methods in which some steps are omitted or the order is changed) for the process of optimizing (or customizing) the scream pattern may be inferred by reference and/or modification, but the present invention It is made including all the implementation methods that are, and the technical characteristics are not limited only to the implementation method shown in this Figure 5. On the other hand, the process illustrated and described in this Figure 5 is provided through the sound device 100, provided through the operation server 200, or provided in a distributed form through interworking between the sound device 100 and the operation server 200. It is possible, and for convenience, the features of the present invention will be described in Figure 5 focusing on the process provided through the sound device 100.

Referring to the embodiment of FIG. 5, the sound device 100 effectively recognizes screams using N pieces of scream pattern information pre-stored in a designated storage area, and then the scream pattern used to effectively recognize the screams. Information is stored and managed in a designated storage area (500).

If j (j≥2) pieces of scream frequency band information used to validly recognize screams for a specified period of time are stored in the storage area (505), the sound device 100 is valid at least j times for a specified period of time. Scream pattern distribution information corresponding to the recognized scream pattern distribution is generated (510), the generated scream pattern distribution information is statistically processed, and the scream of the scream source in the space through which the sound device 100 passes After determining (515) n pieces of scream pattern information corresponding to n (1≤n≤N) scream patterns optimized for sound (or customized), the determined n scream patterns are to be read first. is set to new/updated storage in the designated storage area (520), and then the sound device 100 can recognize screams by using the stored n pieces of scream pattern information.

FIG. 6 illustrates an embodiment of activating a scream-related service by recognizing a scream and the contents of an utterance in conjunction with each other according to another embodiment of the present invention.

In more detail, FIG. 6 shows that when the contents of the user's speech are recognized within a certain time during or after the scream is recognized, the recognized contents of the user's speech are interpreted in association with s number of service functions related to the scream, and the recognition It shows an embodiment of activating a service related to a scream, and those skilled in the art will refer to and/or modify this Figure 6 to perform the process of recognizing the scream or activating the service. Various implementation methods (eg, implementation methods in which some steps are omitted or the order is changed) may be inferred, but the present invention includes all implementation methods inferred above, and the implementation shown in FIG. The technical characteristics are not limited only by the method. On the other hand, the process shown and described in this Figure 6 is provided through the sound device 100, provided through the operation server 200, or provided in a distributed form through interworking between the sound device 100 and the operation server 200. It is possible, and for convenience, the features of the present invention will be described in FIG. 6 focusing on the process provided through the sound device 100.

Referring to the embodiment of FIG. 6, the sound apparatus 100 reads sound data buffered in a designated buffer area, recognizes a user's language utterance, processes it in natural language, and recognizes speech content information corresponding to the user's speech content. is performed (600). If the user's speech content is not recognized, the sound device 100 checks whether the user's speech content is not validly recognized for a specified period of time or longer (605). If the content of the user's utterance is not validly recognized for a specified period of time or longer, the sound device 100 may set the operation mode of the sound device 100 to a sleep mode (610).

Meanwhile, when the user's utterance is recognized by reading the sound data, the sound device 100 determines whether the operation mode of the sound device 100 is a sleep mode (615). If it is in the sleep mode, it is preferable to perform a process of waking up by recognizing a scream.

On the other hand, if the operation mode of the sound device 100 is not the sleep mode at the time of recognizing the contents of the user's speech (eg, while the contents of the user's speech are recognized or a scream is recognized before a certain time, the sound apparatus 100 When the sleep mode wakes up), the sound device 100 confirms that the content of the user's speech is recognized while the scream is being recognized, and/or the user's speech is heard within a certain time after the end of the recognition of the scream. It is checked whether the contents are recognized (620). If the content of the user's speech is not recognized within a certain time during or after the scream is recognized, it is preferable that the sound device 100 checks whether it is in a sleep mode or wakes up by recognizing the scream. do.

On the other hand, when the user's speech is recognized within a certain time during or after the scream is recognized, the sound device 100 interprets the recognized user's speech by associating it with s service functions related to the scream (625), based on this, a specific service function related to the specific sound and corresponding to the recognized user's utterance can be activated (630), and the activated specific service function is designated as The provided procedure may be performed according to the procedure (635).

100: sound device 105: sound module
105a: sound input unit 105b: sound output unit
110: camera module 110a: camera unit
115: power module 120: communication module
120a: wireless network communication unit 120b: local area network communication unit
125: short-range module 125a: short-range wireless unit
130: control module 130a: control unit
130b: memory unit 135: information storage unit
140: sound processing unit 145: scream recognition unit
150: service control unit 155: distribution generation unit
160: information determination unit 165: speech content recognition unit
170: situation judgment unit 175: intelligence processing unit
180: service provider 185: camera device
200: operation server 205: sound receiver

Claims (22)

A sound module including a sound input unit for receiving a sound signal and a sound output unit for outputting a sound signal, a communication module including at least one communication unit connected to a communication network, a short range camera device, and a short range module for processing short range wireless communication and a subject In the sound device having a camera module for obtaining image information photographed,
a sound processing unit which buffers sound data corresponding to the sound signal of the audible frequency band input through the sound input unit in a designated buffer area or checks the sound data buffered in the designated buffer area;
Corresponds to a screaming sound frequency band of at least one screaming sound generating source designated for recognizing a screaming sound designated for emergency situation determination among nonverbal sounds or noises that may be generated in a designated space by reading the buffered sound data, and the screaming sound generating source a scream recognition unit recognizing a scream sound unique to the scream source corresponding to the scream pattern of the scream; and
When recognizing the scream, at least s (1≤s≤S) service functions related to the scream among the designated S (S≥1) service functions and any one or more of the short-distance module, communication module, and camera module A control module including a service control unit for activating a video service function for photographing a sound generating source that generates the recognized scream through the camera device using the module,
The control module,
Information storage for storing M pieces of screaming sound frequency band information corresponding to M (M≥1) screaming sound frequency bands and N screaming pattern information corresponding to N (N≥1) screaming sound patterns in a designated storage area wealth;
generating scream frequency distribution information corresponding to scream frequency distributions that have been recognized validly at least i (i≥2) times during a designated period of time using the M pieces of scream frequency band information; a distribution generating unit for generating scream pattern distribution information corresponding to at least j (j≥2) times validly recognized scream pattern distributions during a specified period of time using the information;
Corresponding to m (1≤m≤M) frequency bands of screams optimized (or customized) for the screams of the screaming sound generator in the space through which the sound device has passed through statistical processing of the generated scream frequency distribution information n (1≤n) optimized (or customized) for the screaming sound of the screaming source in the space through which the sound device passed through by determining m pieces of screaming sound frequency band information and statistically processing the generated screaming pattern distribution information an information determination unit that determines n pieces of scream pattern information corresponding to ≤N) number of scream patterns;
an utterance content recognition unit that reads the buffered sound data, recognizes a user's language utterance, and recognizes utterance content information corresponding to the user's utterance content by natural language processing;
a recognition determination unit for determining whether the content of the user's speech is recognized while the screaming sound is being recognized or whether the content of the user's speech is recognized within a predetermined time after the end of the recognition of the screaming sound;
If the user's speech content is recognized within a certain period of time during or after the scream is recognized, an intelligent processing unit for interpreting the recognized user's speech content by associating it with s number of service functions related to the scream; and
A sound device for recognizing screams, further comprising: a service providing unit providing a service corresponding to the activated service function.
delete delete delete delete delete The method of claim 1, wherein the scream recognition unit,
and a function of reading the buffered sound data and recognizing a scream corresponding to at least one screaming frequency band among the M screaming frequency bands and corresponding to at least one screaming pattern among the N screaming patterns. A sound device for recognizing screams, characterized in that made by.
delete According to claim 1,
The information storage unit further includes a function of setting the determined m screaming sound frequency bands as screaming sound frequency bands to be read first and storing them as new or updated;
The scream recognition unit comprises a function of reading the buffered sound data and first recognizing a scream corresponding to at least one scream frequency band among the stored m scream frequency bands. A sound device for perception.
delete According to claim 1,
The information storage unit further includes a function of setting the determined n screaming sound patterns as screaming sound patterns to be read first and storing them as new or updated,
The scream recognition unit includes a function of first recognizing a scream corresponding to at least one scream pattern among the stored n scream patterns by reading the buffered sound data. sound device for
delete delete The method of claim 1, wherein the s number of services,
A server linkage service provided in conjunction with an operation server on a communication network through the communication module;
A sound-based service provided through the sound module;
A sound device for recognizing screams, characterized in that it includes at least one service among complex services provided by combining the sound module and the communication module.
The method of claim 14, wherein the server interworking service,
Call service using wireless terminal information;
Situation notification service using wireless terminal information;
A call service using an app installed on a wireless terminal;
A situation notification service using an app installed on a wireless terminal,
A video service that provides video information captured by at least one camera to an app installed on a wireless terminal;
A sound device for recognizing screams, characterized in that it comprises at least one of video services for providing video information captured by at least one camera to a designated IPTV device.
The method of claim 14, wherein the sound-based service,
A sound device for recognizing screams, characterized in that it comprises a noise output service for outputting a designated service sound at a designated signal output strength through a sound output unit of a sound module.
The method of claim 16, wherein the signal output strength,
A sound device for recognizing a scream, characterized in that it comprises a signal intensity inversely proportional to the signal input intensity of the scream.
The method of claim 14, wherein the complex service,
The service provided through the operation server on the communication network based on the result of recognizing the contents of the user's speech within a certain time during or after the scream is recognized and associating the contents of the user's speech with the service function related to the scream A sound device for recognizing a scream, characterized in that it comprises.
delete delete delete delete