KR102529725B1 - Method for Soothing Crying Baby by Using Camera Device - Google Patents

Abstract

According to the method of soothing a crying baby using a camera device of the present invention, a projector having a sound input unit, a sound output unit, and a camera module and projecting a specified shape or image onto a ceiling or wall is provided to communicate with a camera device provided at a designated base. In the method executed through an operation server that does, the audio data input through the sound input unit of the camera device and encoded in a designated manner and the camera module of the camera device photographing a specific baby in the base and encoding in a designated manner Video data is received, stored and managed in a specific storage area of a designated storage medium, and audio data stored in a specific storage area of the storage medium is read and included in a designated baby cry frequency band and preset P (P≥1) The baby crying state is confirmed by recognizing the baby crying sound that maintains the sound matched with at least one of the baby crying patterns within a preset similarity range for a predetermined holding time or longer, and the baby crying state If confirmed, it is checked whether the crying situation response rule information set in response to the crying state of the specific baby captured through the camera device is stored in the designated operation D/B, and the crying situation of the specific baby is stored in the operation D/B. When correspondence rule information is not stored, a visual response measure of projecting a designated projection target among f (f≥1) shapes or i (i≥0) images through the projector unit of the camera device according to a preset common correspondence rule; Preset countermeasures among auditory countermeasures for outputting a designated output target among m (m≥1) pieces of music, s (s≥1) pieces of noise, and v (v≥1) pieces of voice through the sound output unit of the camera device (or a combination of countermeasures) are executed sequentially (or in combination) at regular intervals according to a designated order, and when the operation D/B has stored the specific baby's crying situation handling rule information, the stored crying situation handling rule is Visual countermeasures for projecting a designated projection target among f shapes or i images through the projector unit of the camera device and m music, s noise, and v voices through the sound output unit of the camera device Outputting a designated output target Among the auditory response measures, the response measure (or combination of response measures) set as the highest priority for the specific baby is identified and executed, and while the response measure (or combination of countermeasures) specified through the camera device is being executed, the storage medium reads audio data stored in a specific storage area of , and checks whether the crying state of the specific baby is terminated in response to the executed response action (or combination of response actions) based on the recognized baby cry; When the crying state ends, the execution of the countermeasures (or combination of countermeasures) is terminated within a specified time.

Description

Method for Soothing Crying Baby by Using Camera Device

The present invention has a sound input unit, a sound output unit, and a camera module, and has a projector unit for projecting a designated shape or image onto a ceiling or wall, and is provided with a camera device provided in a designated base. In a method executed through an operating server, Specific storage in a designated storage medium by receiving audio data input through the sound input unit of the camera device and encoded in a designated manner and video data encoded in a designated manner by photographing a specific baby in the base through the camera module of the camera device A first step of storing and managing audio data stored in a specific storage area of the storage medium; A second step of recognizing a baby crying sound that maintains a matched sound within a predetermined similarity range with the baby crying pattern of the baby for more than a predetermined holding time and confirming the baby crying state, and confirming the baby crying state, A third step of checking whether the crying situation response rule information set in correspondence with the crying state of the specific baby captured through the camera device is stored in a designated operating D/B; If situation response rule information is not stored, a visual response measure of projecting a designated projection target among f (f≥1) shapes or i (i≥0) images through the projector unit of the camera device according to a preset common response rule. And a predetermined response among auditory response measures for outputting a designated output target among m (m≥1) pieces of music, s (s≥1) pieces of noise, and v (v≥1) pieces of voice through the sound output unit of the camera device. Measures (or combinations of countermeasures) are executed sequentially (or in combination) for a certain period of time according to the designated order, and if the specific baby's crying situation response rule information is stored in the operation D/B, according to the stored crying situation response rule A visual response measure of projecting a designated projection target among f shapes or i images through the projector unit of the camera device, and outputting a designated output target among m music, s noise, and v voice through the sound output unit of the camera device A fourth step of identifying and executing a response (or combination of response measures) set as the highest priority for the specific baby among the auditory response measures to be performed; while the audio data stored in the specific storage area of the storage medium is read and the crying state of the specific baby is terminated in response to the executed response action (or combination of response actions) based on the recognized baby cry. The present invention relates to a method for soothing a crying baby using a camera device, including a fifth step of confirming and a sixth step of terminating the execution of the countermeasure (or combination of countermeasures) within a specified time when the baby's crying state is over. .

A service that provides video or still images of a baby registered as a member and newborn information (eg, prenatal dream, ultrasound diagnosis picture, etc.) to members through the Internet at a hospital, midwifery center, or postpartum care center where a plurality of web PC cameras are installed is proposed. (Patent Publication No. 2001-0091236 (October 23, 2001)).

However, the conventional service simply provides a video or still image of the baby and newborn information through the Internet, and it is possible to check the video of the baby, but if the baby is crying, directly or by contacting someone else There was only one way to do it.

Recently, a stationary sound device that recognizes a user's voice and provides a service corresponding to the recognized voice has been sold. For example, stationary sound devices such as Amazon's 'Echo', SK Telecom's 'Who', and KT's 'Giga Genie' are on sale.

However, the conventional stationary sound device is a combination of a sound source service and voice recognition technology, and operates mainly on a service that simply provides or reproduces a sound source desired by a user through voice recognition, and has a problem in that it is difficult to use in other fields.

An object of the present invention for solving the above problems is to have a sound input unit, a sound output unit, and a camera module, and a projector unit for projecting a designated shape or image onto a ceiling or wall to communicate with a camera device provided at a designated base. In the method executed through an operation server that does, the audio data input through the sound input unit of the camera device and encoded in a designated manner and the camera module of the camera device photographing a specific baby in the base and encoding in a designated manner A first step of receiving video data and storing and managing it in a specific storage area of a designated storage medium, and reading audio data stored in a specific storage area of the storage medium and included in a designated baby cry frequency band and preset A second step for confirming the baby's crying state by recognizing the baby's crying sound that maintains the sound matched with at least one of the ≥1) baby crying patterns within a preset similarity range for more than a preset holding time Step 3, if the crying state of the baby is confirmed, a third step of checking whether rule information corresponding to the crying situation set in correspondence with the crying state of the specific baby photographed through the camera device is stored in a designated operating D/B; If the specific baby's crying situation response rule information is not stored in the operating D/B, f (f≥1) shapes or i (i≥0) shapes or i (i≥0) shapes are displayed through the projector unit of the camera device according to a preset common response rule. Designated output targets among m (m≥1) pieces of music, s (s≥1) pieces of noise, and v (v≥1) pieces of voices through a visual countermeasure to project a specified projection target among images and a sound output unit of the camera device Among the auditory response measures that output , preset response measures (or combinations of response measures) are sequentially (or combined) executed for a certain period of time according to a designated order, and the specific baby's crying situation response rule information is stored in the operation D/B. When is stored, a visual response measure of projecting a designated projection target among f shapes or i images through the projector unit of the camera device according to the stored crying situation response rule, and m pieces of music and s pieces of music through the sound output unit of the camera device. A fourth step of identifying and executing a response measure (or a combination of response measures) set as the highest priority for the specific baby among auditory response measures for outputting a designated output target among noise and v voices; While the response action (or combination of response actions) is being executed, the audio data stored in the specific storage area of the storage medium is read and the response action (or combination of response actions) executed is responded based on the recognized baby's cry. A fifth step of confirming whether the crying state of the specific baby is ended, and a sixth step of terminating execution of the corresponding action (or combination of corresponding actions) within a specified time when the crying state of the baby is ended. It is to provide a method of soothing a crying baby using a camera device.

A method for soothing a crying baby using a camera device according to the present invention includes a projector having a sound input unit, a sound output unit, and a camera module, and having a projector unit for projecting a designated shape or image onto a ceiling or a wall, and communicating with a camera device provided at a designated base. In the method executed through an operation server that does, the audio data input through the sound input unit of the camera device and encoded in a designated manner and the camera module of the camera device photographing a specific baby in the base and encoding in a designated manner A first step of receiving video data and storing and managing it in a specific storage area of a designated storage medium, and reading audio data stored in a specific storage area of the storage medium to be included in a designated frequency band of a baby's cry and set to P (P ≥ 1) A second step of confirming the baby's crying state by recognizing a baby's crying sound that maintains a matched sound within a preset similarity range with at least one of the baby's crying patterns for a predetermined duration or longer and when the crying state of the baby is confirmed, a third step of checking whether the crying situation response rule information set in correspondence with the crying state of the specific baby photographed through the camera device is stored in a designated operation D/B and the operation D If the specific baby's crying situation response rule information is not stored in /B, one of f (f≥1) shapes or i (i≥0) images is designated through the projector unit of the camera device according to a preset common response rule. Outputting a designated output target among m (m ≥ 1) music, s (s ≥ 1) noise, and v (v ≥ 1) voice through the visual response measure of projecting the projection target and the sound output unit of the camera device Among the auditory response measures, the preset response measures (or combination of response measures) are sequentially executed (or combined) for a certain period of time according to the designated order, and the specific baby's crying situation response rule information is stored in the operation D/B A visual response measure of projecting a designated projection target among f shapes or i images through the projector unit of the camera device according to the stored crying situation response rule, and m pieces of music, s pieces of noise, and v through the sound output unit of the camera device. Among the auditory response measures for outputting the designated output target among the dog's voices, the fourth step of confirming and executing the response measures (or combination of response measures) set as the highest priority for the specific baby and the response measures designated through the camera device (or While the corresponding action combination) is being executed, the audio data stored in the specific storage area of the storage medium is read and the specific baby reacts to the executed corresponding action (or combination of corresponding actions) based on the recognized baby crying. It is characterized by including a fifth step of checking whether the crying state of the baby is ended, and a sixth step of terminating execution of the response action (or combination of response actions) within a designated time when the crying state of the baby is ended.

In the method of soothing a crying baby using a camera device according to the present invention, the first step is to receive moving picture data including the audio data and video data from the camera device and store and manage them in a specific storage area of a designated storage medium. It is characterized by comprising the step of maintaining the process.

In the method for soothing a crying baby using a camera device according to the present invention, the first step is to maintain a process of receiving the audio data and video data from the camera device and storing them in a specific storage area of a designated storage medium. It is characterized by comprising a step.

In the method for soothing a crying baby using a camera device according to the present invention, the first step is to synchronize the times of the audio data and video data respectively received from the camera device and store them in a specific storage area of a designated storage medium. It is characterized in that it further comprises.

In the method of soothing a crying baby using a camera device according to the present invention, in the second step, when the baby's crying sound is recognized, video data received from the camera device in real time and stored in a specific storage area of the storage medium is stored. reading and recognizing more than a specified number of feature points existing in the face region of a specific baby photographed by the camera device and analyzing the geometric relationship between the recognized feature points of the face region of the specific baby to distribute feature points corresponding to the baby's crying state The method further includes checking whether the pattern is matched within a preset similarity range, and when the facial region of the specific baby is recognized, the result of recognizing the crying sound of the baby and the result of recognizing the facial region of the specific baby are combined. It is characterized in that it comprises the step of checking the baby's crying state.

The method for soothing a crying baby using a camera device according to the present invention further includes storing unique information for managing a service using the camera device in a designated operating D/B, wherein the third step comprises: Including the step of confirming whether the crying situation response rule information including the highest priority response action (or combination of response actions) for ending the crying state of a specific baby captured by the device is linked and stored with the unique information characterized by being made.

In the method for soothing a crying baby using a camera device according to the present invention, the third step is to qualify and quantify the characteristics of the baby's crying sound terminated by a specific response measure (or combination of countermeasures). A further step of confirming whether the crying feature information generated by the above is stored in association with the crying situation response rule information including the specific response action (or combination of response actions) as the highest priority response action (or combination of response actions) It is characterized by comprising.

In the method of soothing a crying baby using a camera device according to the present invention, the step of storing guardian wireless terminal information for contacting a guardian of a specific baby photographed through the camera device in a designated operating D/B, wherein the The response measure may further include a notification response measure for sending notification information informing of the crying state of the specific baby to the guardian wireless terminal of the specific baby using the guardian wireless terminal information.

In the method of soothing a crying baby using a camera device according to the present invention, unique identification information for communicating with an app of a guardian's wireless terminal owned by a guardian of a specific baby photographed through the camera device is stored in a designated operating D/B The countermeasures include outputting audio data stored in a specific storage area of the storage medium through a sound output unit of the guardian wireless terminal in conjunction with an app of the guardian wireless terminal based on the unique identification information. It is characterized in that it further comprises a guardian interlocking response measure for processing so that the sound data encoded by the sound input unit of the guardian wireless terminal is output through the sound output unit of the camera device.

In the method for soothing a crying baby using a camera device according to the present invention, the guardian interlocking response action is to interwork with an app of the guardian wireless terminal to transmit image data captured through the camera unit of the guardian wireless terminal to the projector unit of the camera device. It is characterized in that it is made by further comprising a countermeasure for processing so that it is projected and output through.

In the method of soothing a crying baby using a camera device according to the present invention, the crying situation response rule information includes one or more response measures (or combinations of response measures) for each rank in addition to the response measures (or combinations of response measures) of the highest priority. In the fourth step, when the specific baby's crying situation response rule information is stored in the operation D/B, while executing the highest priority response action (or combination of response actions) for a specified time, reading audio data received in real time from the camera device and stored in a specific storage area of the storage medium to confirm whether the crying state of the baby is terminated in response to the executed countermeasure (or combination of countermeasures); and If the baby's crying state does not end within a designated time after executing the countermeasure (or combination of countermeasures), the next-ranked countermeasure (or combination of countermeasures) is executed in real time from the camera device, Further comprising repeating a process of reading audio data stored in a specific storage area of the storage medium and checking whether the crying state of the baby is ended in response to the executed corresponding action (or combination of corresponding actions) up to the last rank. It is characterized in that it is made by.

In the method of soothing a crying baby using a camera device according to the present invention, the fourth step is to read audio data stored in a specific storage area of the storage medium when the countermeasure (or combination of countermeasures) is executed. Recognizing a guardian's soothing voice, which is uttered to relieve the crying state of the specific baby, in addition to the baby's crying sound, and terminating execution of the response action (or combination of response actions) when the soothing voice is recognized. It is characterized by comprising more.

In the method of soothing a crying baby using a camera device according to the present invention, in the fourth step, when the countermeasure (or combination of countermeasures) is executed, video data stored in a specific storage area of the storage medium is read, Recognizing the soothing action of the guardian to relieve the crying state of the specific baby, and terminating the execution of the corresponding action (or combination of corresponding actions) when recognizing the soothing action do.

In the method of soothing a crying baby using a camera device according to the present invention, the fifth step is to read video data stored in a specific storage area of the storage medium while a designated countermeasure is being executed through the camera device, A step of recognizing a designated facial expression change that visually or audibly responds to the executed response action (or a combination of response actions) by analyzing a change pattern between feature points of a specific baby's facial region, When recognized, it is characterized in that it comprises a step of confirming whether the crying state of the specific baby is ended by combining the result of recognizing the crying sound of the baby and the result of recognizing the facial expression change.

In the method of soothing a crying baby using a camera device according to the present invention, when the crying baby state is over, the corresponding action including the type and execution time of the specific response action (or combination of response actions) that ended the crying state. Step 6-1 of constructing and storing history information in the operation D/B and analyzing the history information of a plurality of response measures stored in the operation D/B to take the highest priority response measures to end the crying state of the specific baby (or a combination of corresponding measures) is characterized in that it further comprises a step 6-2 of generating and storing the crying situation response rule information in the operating D/B.

In the method of soothing a crying baby using a camera device according to the present invention, audio data stored in a specific storage area of the storage medium is read and the guardian's voice is ignited to relieve the crying state of the specific baby in addition to the crying sound of the baby. The method may further include recognizing the soothing voice, and step 6-1 may further include omitting the process of storing the corresponding action history information when recognizing the soothing voice.

In the method of soothing a crying baby using a camera device according to the present invention, the step of reading video data stored in a specific storage area of the storage medium and recognizing a guardian's soothing action to relieve the crying state of the specific baby Further, the step 6-1 may further include the step of omitting the process of storing the corresponding action history information when the appeasement action is recognized.

In the method for soothing a crying baby using a camera device according to the present invention, the step 6-2 is to analyze a plurality of response action history information stored in the operation D/B to respond to the end of the crying state of the specific baby. Actions (or combinations of countermeasures) are classified and sorted by the number of terminations so that the countermeasures (or combinations of countermeasures) that terminate the crying state of the specific baby the most are given the highest priority. combination) and storing it in the operation D/B.

In the method of soothing a crying baby using a camera device according to the present invention, step 6-2 includes one or more response measures (or combinations of response measures) for each rank in addition to the response measures (or combination of response measures) of the highest priority. ) It is characterized in that it further comprises the step of generating the crying situation response rule information further comprising.

In the method of soothing a crying baby using a camera device according to the present invention, the response action history information further includes audio data including the crying sound of the baby terminated by each response action (or combination of response actions), wherein the Step 6-2 qualifies and quantifies the characteristics of the baby's crying included in the audio data of the response action history information corresponding to the response action (or combination of response actions) of the highest priority, generating feature information of the highest priority cry terminated by the response action (or combination of response actions) of the highest priority, and the response action (or combination of response actions) of the highest priority and the generated cry of the highest priority It is characterized in that it comprises the step of generating crying situation response rule information including feature information in association with and storing it in the operating D/B.

In the method of soothing a crying baby using a camera device according to the present invention, step 6-2 includes one or more response measures (or combinations of response measures) for each rank in addition to the response measures (or combination of response measures) of the highest priority. ) Qualifying and quantifying the characteristics of the baby's crying included in the audio data of the corresponding action history information to generate crying sound characteristic information for each rank that was terminated by the corresponding action for each rank (or combination of response actions) and generating and storing in the operation D/B rule information corresponding to a crying situation, which further includes response measures (or combinations of corresponding measures) for each rank and characteristic information of crying sounds for each rank. characterized by

According to the present invention, in order to photograph a baby existing at a designated base, it is possible to check whether the baby is crying from a distance through a camera device equipped with a projector provided in the base, and if the baby is crying, through the projector provided in the camera device. There is an advantage in that the baby can stop crying by allowing the baby to see various shapes, images, and videos.

1 is a diagram showing the configuration of a system for soothing a crying baby using a camera device according to an embodiment of the present invention.
2 is a diagram showing a functional configuration of a camera device communicating with an operation server according to an embodiment of the present invention.
Figure 3 is a diagram showing the functional configuration of the guardian wireless terminal and the app that communicates with the operation server according to the implementation method of the present invention.
4 is a diagram illustrating a process of recognizing a crying baby and confirming a crying state of a baby according to the method of the present invention.
5 is a diagram illustrating a process of soothing a crying baby by executing visual response measures and auditory response measures through a camera device according to an embodiment of the present invention.

Hereinafter, the operating principle of the preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings and description. However, the drawings shown below and the description below relate to preferred implementation methods among various methods for effectively explaining the features of the present invention, and the present invention is not limited to only the drawings and description below.

That is, the following examples correspond to examples of preferred union types among numerous embodiments of the present invention, and examples in which specific components (or steps) are omitted or specific configurations (or steps) are omitted in the following embodiments. Embodiments in which functions implemented in are divided into specific configurations (or steps), or embodiments in which functions implemented in two or more configurations (or steps) are integrated into any one configuration (or steps), specific configurations (or steps) It is clearly stated that the embodiments in which the operation sequence of is replaced are all within the scope of the present invention, even if not separately mentioned in the following embodiments. Therefore, it is clearly stated that various examples corresponding to subsets or complements can be divided retroactively from the filing date of the present invention based on the following examples.

In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, the definition should be made based on the contents throughout the present invention.

As a result, the technical spirit of the present invention is determined by the claims, and the following embodiments are a means for efficiently explaining the technical spirit of the present invention to those skilled in the art to which the present invention belongs. only

Figure 1 is a diagram showing the configuration of a system for soothing a crying baby using a camera device according to an embodiment of the present invention.

In more detail, Figure 1 shows a sound input unit, a sound output unit, a camera module, and a projector unit for input through the camera device 100 in the operation server 100 communicating with the camera device 200 provided at a designated base. Receives and stores the audio data and video data of a specific baby in the base or photographed, reads the audio data, recognizes the baby's crying, and when the baby's crying state is confirmed, in response to the confirmed crying state It shows the configuration of a system that executes an auditory countermeasure through the sound output unit of the camera device 200 and/or a visual countermeasure through the projector unit of the camera device 200, which is common in the art to which the present invention belongs. If you have knowledge of, refer to and / or modify this Figure 1 to check the crying state of the baby from the camera device 200, and through the camera device 200, according to the crying state, auditory and / or visual Various implementation methods (for example, implementation methods in which some components are omitted, subdivided, or combined) may be inferred for the system for executing countermeasures, but the present invention includes all implementation methods inferred above. However, the technical characteristics are not limited only to the implementation method shown in FIG. 1.

The system of the present invention is provided with a sound input unit, a sound output unit, and a camera module at a designated base, and a projector unit for projecting a designated shape or image onto a ceiling or wall, and includes audio data input from a specific baby in the base and captured images. A camera device 200 that transmits video data to the operation server 100, a guardian wireless terminal 300 that communicates with the operation server 100 and receives notification information informing of the crying state of the specific baby, When the audio data and video data of a specific baby in the base, which are input or photographed through the camera device 100, are received and stored, and the audio data is read to recognize the baby's crying sound, the baby's crying state is confirmed. In response to the confirmed crying state, an operation server 100 is configured to execute an auditory response through the sound output unit of the camera device 200 and/or a visual response through the projector unit of the camera device 200. It can be done by

The camera device 200 includes a sound input unit, a sound output unit, a camera module, and a projector unit and is provided at a designated base to transmit audio data input from a specific baby in the base and video data taken by the specific baby to the operating server. (100), and interlocks with the operation server 100 to execute auditory response measures through the sound output unit and/or visual response measures through the projector unit according to the crying state of the specific baby. , Preferably, a designated program 255 may be installed and executed to perform at least one device-side function for transmitting the audio data and video data, and for the auditory and visual response measures. For example, the camera device 200 may include various devices having a sound input unit and a sound output unit in which a designated program 255 is installed, a camera module, a communication module, and a projector unit, and the camera device ( 200) will be described in detail with reference to FIG. 2 below.

The guardian wireless terminal 300 communicates with the operation server 100, and is a wireless communication device used by the guardian of the specific baby who wants to receive notification information indicating the crying state of the specific baby from the operation server 100. As a generic term, preferably, a designated app 355 is installed and executed to perform at least one terminal-side function for receiving the notification information from the operation server 100. For example, the guardian wireless terminal 300 may include a guardian's mobile communication terminal, mobile phone, smart phone, tablet PC, etc. in which a designated app 355 is installed, and the function configuration of the guardian wireless terminal 300 It will be described in more detail with reference to Figure 3 below.

The operation server 100 may be implemented in the form of at least one server or a combination of two or more servers, and may be implemented in the form of server software installed on a pre-installed server according to an implementation method. For convenience, a function of receiving and storing audio data and video data of a specific baby in the base that is input or photographed through the camera device 100 according to the present invention through the description or drawings of the embodiments below, and reading the audio data A function of recognizing the baby's crying sound and confirming the baby's crying state, an auditory response through the sound output unit of the camera device 200 in response to the confirmed crying state, and/or a projector of the camera device 200 Some of the functions for executing visual countermeasures through the unit will be described as being implemented through the operation server 100, and some other functions will be described as being implemented through the camera device 200 and the wireless terminal 300, In the present invention, the functions are not clearly divided into device and terminal-side functions and server-side functions, and some of the device and terminal-side functions shown and described in the present invention are included in server-side functions and performed, or conversely, among server-side functions. Some of them may also be included in device and terminal-side functions, and the present invention makes it clear that these embodiments are also included in the scope of rights.

Referring to Figure 1, the operation server 100 includes an information storage unit 105 for storing unique information of the camera device 200 and wireless terminal information of a guardian of a specific baby, and audio from the camera device 200. A data management unit 110 that receives and stores data and video data, a status checking unit 115 that reads the audio data and checks the crying state of the baby, and information on rules for handling crying situations for a specific baby whose crying state has been confirmed. An information confirmation unit 120 that checks, a countermeasure execution unit 125 that checks and executes a countermeasure (or a combination of countermeasures) set for the specific baby, and checks whether the crying state of the specific baby ends. It may be configured to include a countermeasure reaction unit 130 that performs a countermeasure and an execution end unit 135 that ends execution of the countermeasure (or a combination of countermeasures).

The information storage unit 105 may store unique information for managing the service using the camera device in the designated operation D/B 140 .

In addition, the information storage unit 105 may store guardian wireless terminal information for contacting the guardian of a specific baby photographed through the camera device 200 in the designated operation D/B 105 .

In addition, the information storage unit 105 transmits unique identification information for communication with the app of the guardian's wireless terminal 300 owned by the guardian of a specific baby captured through the camera device 200 to the designated operation D/B (140). ) can be stored.

The data management unit 110 captures audio data input through the sound input unit of the camera device 200 and encoded in a designated manner, and a specific baby in the base through the camera module of the camera device 200, and captures the audio data in a designated manner. Encoded video data may be received and stored and managed in a specific storage area of the designated storage medium 145 .

According to the implementation method of the present invention, the data management unit 110 receives moving picture data including the audio data and video data from the camera device 200 and stores and manages the video data in a specific storage area of the designated storage medium 145. process can be maintained.

In addition, according to the implementation method of the present invention, the data management unit 110 receives the audio data and video data from the camera device 200 and stores them in a specific storage area of the designated storage medium 145 and manages the process. The data management unit 110 may synchronize the times of the audio data and the video data respectively received from the camera device 200 and store them in a specific storage area of the designated storage medium 145 .

The state checking unit 115 stores audio data and video data in a specific storage area of the storage medium 145 through the data management unit 110, and then the audio data stored in the specific storage area of the storage medium 145. Preset maintenance of a sound matched with at least one baby crying pattern among preset P (P≥1) baby crying patterns included in a designated baby crying frequency band by reading data and within a preset similarity range It is possible to check the baby's crying state by recognizing the baby's crying sound that lasts for more than an hour.

According to the implementation method of the present invention, when the state checking unit 115 recognizes the crying sound of the baby, video data received from the camera device 200 in real time and stored in a specific storage area of the storage medium 145 is stored. by reading and recognizing more than a specified number of feature points existing in the face region of a specific baby photographed by the camera device, and analyzing the geometric relationship between the recognized feature points of the face region of the specific baby to analyze the feature point distribution pattern corresponding to the baby's crying state. It is possible to check whether the match is within a pre-set similarity range, and when the facial region of the specific baby is recognized, the crying state of the baby can be confirmed by combining the result of recognizing the crying sound of the baby with the result of recognizing the facial region of the specific baby. can

When the information checking unit 120 confirms the crying state of a specific baby through the status checking unit 115, the crying situation response rule set in response to the crying state of the specific baby captured through the camera device 200 It can be checked whether the information is stored in the designated operating D/B 140.

Here, the crying situation response rule information may further include one or more response measures (or combinations of response measures) for each rank in addition to the response measures (or combination of response measures) of the highest priority.

According to the implementation method of the present invention, the information checking unit 120, when unique information for managing a service using the camera device 200 is stored in the designated operating D/B 140, the camera device It can be checked whether the crying situation response rule information including the highest priority response action (or combination of response actions) for ending the crying state of a specific baby photographed through (200) is stored in association with the unique information.

In addition, according to the implementation method of the present invention, the information check unit 120 qualifies and quantifies the characteristics of the baby's cry terminated by a specific response measure (or a combination of response measures) to generate It may be confirmed whether the crying feature information is stored in association with the crying situation response rule information including the specific response action (or combination of response actions) as the highest priority response action (or combination of response actions).

According to the present invention, the countermeasure is performed when the guardian wireless terminal information for contacting the guardian of a specific baby photographed through the camera device 200 is stored in the designated operation D/B 140, the guardian wireless terminal A notification response measure for sending notification information notifying the crying state of the specific baby to the wireless terminal 300 of the guardian of the specific baby using the information may be further included.

In addition, according to the present invention, the countermeasure is an operation D/B (140) designated with unique identification information for communicating with the app of the guardian's wireless terminal (300) possessed by the guardian of the specific baby photographed through the camera device (200). ), in conjunction with the app of the guardian wireless terminal 300 based on the unique identification information, the audio data stored in the specific storage area of the storage medium 145 is transferred to the guardian wireless terminal 300. It may further include a guardian interlocking countermeasure for outputting through the sound output unit or processing the encoded sound data input through the sound input unit of the guardian wireless terminal 300 to be output through the sound output unit of the camera device 200. there is.

Here, the guardian interlocking response measures interwork with the app of the guardian wireless terminal 300 to project image data captured by the camera unit of the guardian wireless terminal 300 through the projector unit of the camera device 200 and output the image data. countermeasures may be included.

If it is confirmed through the information checking unit 120 that the corresponding rule information for handling the crying situation of the specific baby is not stored in the operating D/B 140, the countermeasure execution unit 125 executes a predetermined common response rule. Accordingly, a visual response measure of projecting a designated projection target among f (f≥1) shapes or i (i≥0) images through the projector unit of the camera device 200 and m (m Among the auditory countermeasures that output the designated output targets among ≥1) pieces of music, s (s≥1) pieces of noise, and v (v≥1) pieces of voices, preset countermeasures (or combinations of countermeasures) are performed in a designated order. It can be executed sequentially (or in combination) for a certain amount of time according to the

In addition, the countermeasure execution unit 125, when the specific baby's crying situation response rule information is stored in the operation D/B 140 through the information check unit 120, determines the information according to the stored crying situation response rule. A visual response measure of projecting a designated projection target among f shapes or i images through the projector unit of the camera device 200 and m music, s noise, and v voices through the sound output unit of the camera device 200 Among auditory response measures for outputting a designated output target, a response measure set as the highest priority for the specific baby (or a combination of response measures) may be identified and executed.

According to the implementation method of the present invention, the countermeasure execution unit 125 performs one or more countermeasures (or combinations of countermeasures) for each rank in addition to the countermeasures (or combinations of countermeasures) of the highest priority in the crying situation response rule information. ), and when the specific baby's crying situation response rule information is stored in the operating D/B 140, the camera device executes the highest priority response action (or combination of response actions) for a specified time. Audio data received in real time from 200 and stored in a specific storage area of the storage medium 145 may be read to confirm whether the crying state of the baby is terminated in response to the executed response action (or combination of response actions). .

Then, the countermeasure execution unit 125 executes the countermeasures (or combination of countermeasures) of the designated rank and if the crying state of the baby does not end within a designated time, the countermeasure of the next rank (or combination of countermeasures) is executed. ) for a specified time, read audio data received in real time from the camera device 200 and stored in a specific storage area of the storage medium 145 in response to the executed countermeasure (or a combination of countermeasures) The process of checking whether the crying baby is finished may be repeated up to the last rank.

Meanwhile, according to the implementation method of the present invention, the countermeasure execution unit 125 reads audio data stored in a specific storage area of the storage medium 145 when the countermeasure (or combination of countermeasures) is executed. In addition to the crying sound of the baby, a guardian's soothing voice uttered to relieve the crying state of the specific baby may be recognized, and when the soothing voice is recognized, execution of the response action (or combination of response actions) may be terminated. can

In addition, according to the implementation method of the present invention, the countermeasure execution unit 125 reads video data stored in a specific storage area of the storage medium 145 when the countermeasure (or combination of countermeasures) is executed. It is possible to recognize a guardian's soothing action to relieve the crying state of the specific baby, and when the soothing action is recognized, execution of the corresponding action (or a combination of corresponding actions) may be terminated.

The countermeasure response unit 130 confirms and executes a countermeasure (or a combination of countermeasures) set for a specific baby through the countermeasure execution unit 125, and then executes the countermeasure specified through the camera device 200. While the (or combination of countermeasures) is being executed, audio data stored in a specific storage area of the storage medium 145 is read, and the executed countermeasure (or combination of countermeasures) is determined based on the recognized baby's cry. In response, it is possible to determine whether the crying state of the specific baby is ended.

According to the implementation method of the present invention, the countermeasure reaction unit 130 reads video data stored in a specific storage area of the storage medium 145 while a designated countermeasure is being executed through the camera device 200. It is possible to recognize a designated facial expression change that responds visually or aurally to the executed response action (or a combination of response actions) by analyzing a change pattern between feature points of the recognized facial region of the specific baby, and recognize the designated expression change. In one case, it is possible to determine whether the crying state of the specific baby is ended by combining the result of recognizing the crying sound of the baby and the result of recognizing the facial expression change.

The execution termination unit 135 may terminate execution of the response action (or combination of response actions) within a specified time when it is confirmed through the response action response unit 130 that the crying state of the specific baby is ended. there is.

According to the implementation method of the present invention, the execution termination unit 135, when the crying baby state ends, responds including the type and execution time of a specific response action (or combination of response actions) that ended the crying state. The action history information is configured and stored in the operation D/B (140), and a plurality of response action history information stored in the operation D/B (140) is analyzed to determine the highest priority for terminating the baby's crying state of the specific baby. Crying situation response rule information including response measures (or a combination of response measures) may be generated and stored in the operation D/B 140 .

Here, the execution termination unit 135 reads the audio data stored in the specific storage area of the storage medium 145, and the soothing voice of the guardian uttered to relieve the crying state of the specific baby in addition to the crying sound of the baby. can be recognized, and when the soothing voice is recognized, the process of storing the corresponding action history information can be omitted.

According to the implementation method of the present invention, the execution termination unit 135 reads the video data stored in the specific storage area of the storage medium 145 and executes the guardian's soothing operation to relieve the crying state of the specific baby. If the appeasement operation is recognized, the process of storing the corresponding action history information may be omitted.

According to the implementation method of the present invention, the execution termination unit 135 analyzes a plurality of corresponding action history information stored in the operation D/B 140 to take a corresponding action (or response action) that terminates the crying state of the specific baby. Combinations of actions) are classified and sorted by the number of terminations, and the response action (or combination of response actions) that terminates the crying state of the specific baby the most is included as the highest priority response action (or combination of response actions). Crying situation response rule information may be generated and stored in the operation D/B (140).

According to the implementation method of the present invention, the execution termination unit 135 further includes one or more response measures (or combination of response measures) for each priority in addition to the response action (or combination of response actions) of the highest priority. Correspondence rule information can be created.

According to the implementation method of the present invention, the response action history information may further include audio data including a baby's cry terminated by each response action (or combination of response actions), and the execution end unit 135 Qualifies and quantifies the characteristics of the baby's crying included in the audio data of the response action history information corresponding to the response action (or combination of response actions) of the highest priority, and the response action of the highest priority ( Or a combination of response measures) to generate the highest priority crying characteristic information, and link the highest priority response action (or combination of response actions) with the generated highest priority crying characteristic information. Situation response rule information may be generated and stored in the operation D/B 140 .

According to the implementation method of the present invention, the execution termination unit 135 records a response action history corresponding to one or more corresponding response actions (or combinations of response actions) for each priority, in addition to the response action (or combination of response actions) of the highest priority. Characteristics of baby crying included in the audio data of the information can be qualitatively and quantified to generate information on crying sound characteristics for each rank, which is terminated by a corresponding action for each rank (or a combination of countermeasures), and a response for each rank Crying situation response rule information further including measures (or a combination of countermeasures) and the generated characteristic information of crying sounds for each rank may be generated and stored in the operation D/B 140 .

Figure 2 is a diagram showing the functional configuration of a camera device communicating with an operation server according to an embodiment of the present invention.

In more detail, drawing 2 is characterized by having a sound input unit, a sound output unit, and a camera module, and a projector unit for projecting a designated shape or image onto a ceiling or wall, provided at a designated base, and input through the sound input unit. audio data encoded in a designated manner and a specific baby in the base are photographed through the camera module, video data encoded in a designated manner is transmitted to the operating server 100, and the specific baby is transmitted through the operating server 100 When the crying state of is confirmed, it shows the functional configuration of the camera device 200 to stop the crying of the specific baby by projecting a designated projection target through the projector unit, which is common in the technical field to which the present invention belongs. Anyone with knowledge of this will be able to infer various implementation methods for the function of the camera device 200 by referring to and/or modifying this Figure 2, but the present invention is made including all the implementation methods inferred above. However, the technical characteristics are not limited only to the implementation method shown in FIG. 2.

Referring to FIG. 2, the camera device 200 includes a control unit 205, a memory unit 240, a camera module 210, a driving module 215, a sound input unit 220, and a sound output unit 225 and a short distance. A network communication unit 230, a wired network communication unit 235, and a projector unit 245 may be provided, and a power supply unit 250 including a battery for power supply may be provided.

The control unit 205 is a general term for components that control the operation of the camera device 200, and includes at least one processor and an execution memory, and each component provided in the camera device 200 and a bus ( BUS) can be connected. According to the present invention, the control unit 205 loads and calculates at least one program code provided in the camera device 200 into the execution memory through the processor, and transmits the result to at least one configuration through the bus. It is possible to control the operation of the camera device 200 by transferring the information to the unit. Hereinafter, for convenience, a configuration corresponding to the program 250 of the camera device 200 implemented in the form of program code will be illustrated and described in the controller 205.

The memory unit 240 is a generic term for non-volatile memory corresponding to storage resources of the camera device 200, and includes at least one program code executed by the control unit 205 and at least one program code used. Datasets are saved and maintained. The memory unit 240 basically stores system program codes and system data sets corresponding to the operating system of the camera device 200, communication program codes and communication data sets for processing the wireless communication connection of the wireless terminal 300, and at least One application program code and one application data set are stored, and the program code and data set corresponding to the program 250 of the camera device 200 are also stored in the memory unit 240 .

The camera module 210 may include an optical unit, a charge coupled device (CCD), and a module driving the same, and may obtain bitmap data input to the CCD through the optical unit. The bitmap data may include both image data and video data (or moving picture data) of a still image. In addition, the camera module 210 may further include an actuator and/or various filters (eg, an IR filter) for an auto focus function and the like.

The driving module 215 may drive the camera of the camera device 200 to move up/down/left/right.

The sound input unit 220 may be composed of a microphone for receiving a sound signal and a module for driving the same, and audio data obtained by encoding a sound signal input through the microphone according to a designated encoding rule is transmitted to the control unit 205. ) can be transmitted. According to the implementation method of the present invention, the sound input unit 220 may receive a sound signal (eg, a sound of crying, a sound of laughter, etc.) input from a specific baby at a base equipped with the camera device 200 .

The sound output unit 225 may be composed of a speaker that outputs a sound signal and a driving module that drives the same, and decodes sound data corresponding to sound output among various calculation results of the controller 205 to It can be output through the speaker. According to the implementation method of the present invention, the sound output unit 225 interworks with the operating server 100, m (m≥1) music, s (s≥1) noise, v (v≥1) Among auditory response measures for outputting a designated output target among the dog's voices, a preset response measure (or a combination of response measures) may be output.

The local area network communication unit 230 connects a communication session by using a radio frequency signal as a communication medium within a certain distance (eg, 10 m) and connects the camera device 200 to a communication network based on this. , Preferably, the camera device 200 may be connected to a communication network through at least one of Wi-Fi communication, Bluetooth communication, public wireless communication, and UWB. According to the present invention, the local area network communication unit 230 may connect the camera device 200 to a data network providing short-range wireless data communication based on packet communication through a wireless AP.

The wired network communication unit 235 is characterized by providing a wired communication means for performing the wired communication function of the camera device 200, and a network communication unit providing communication with the operation server 100 through a wired network and/or It may include a cable communication unit that provides communication with an external communication device through a predetermined cable communication port (eg, RS-232c, USB (Universal Serial Bus), etc.), a communication protocol for providing communication in software, and It may include a communication application.

The projector unit 245 may project a designated shape or image onto a ceiling or wall. According to the implementation method of the present invention, the projector unit 245 interlocks with the operation server 100 to project a designated projection target among f (f≥1) shapes or i (i≥0) images onto the ceiling or wall. can project

Referring to FIG. 2, the program 250 of the camera device 200 includes an audio data transmission unit 260 that transmits audio data to the operation server 100 and video data to the operation server 100. A video data transmission unit 265, a video data transmission unit 270 for transmitting video data to the operating server 100, a visual response unit 275 for projecting a designated projection target through a projector unit, and a sound output unit It may be configured to include an auditory countermeasure unit 280 that outputs a designated output target through.

The audio data transmission unit 260 may transmit audio data input through the sound input unit and encoded in a designated manner to the operating server 100 .

The video data transmitter 265 may transmit video data encoded in a designated manner to the operating server 100 by photographing a specific baby in the base through a camera module.

The video data transmitter 270 may transmit video data including the audio data and video data to the operation server 100 .

When the crying state of a specific baby is confirmed through the audio data, video data, and video data in the operation server 100, the visual countermeasure unit 275 performs f (f ≥ 1) shapes or i (i ≥ 0 ) images, a designated projection target can be projected.

The auditory countermeasure unit 280, when the operating server 100 confirms the crying state of a specific baby through the audio data, video data, and moving picture data, m (m≥1) pieces of music, s (s≥ 1) Among the noises and v(v≥1) voices, a designated output target can be output.

Figure 3 is a diagram showing the functional configuration of the guardian wireless terminal and the app that communicates with the operating server according to the implementation method of the present invention.

In more detail, this drawing 3 receives notification information notifying the crying state of a specific baby photographed through the camera device 200 from the operation server 100, or interlocks with the operation server 100 to operate the operation server 100 The function of the wireless terminal 300 to receive and output the audio data stored in, or provide the encoded sound data input through the sound input unit to the camera device 200 through the operation server 100 to output, It shows the configuration of the app 355 installed in the wireless terminal 300, and those skilled in the art to which the present invention belongs may refer to and / or modify this figure 3 to the wireless terminal 300 ), but the present invention includes all the implementation methods inferred above, and its technical characteristics are not limited only to the implementation method shown in FIG. 3. Preferably, the wireless terminal 300 of FIG. 3 may include at least one of various mobile communication terminals, various mobile phones, various smart phones, and various tablet PCs.

Referring to FIG. 3, the wireless terminal 300 includes a control unit 305, a memory unit 350, a screen output unit 310, a user input unit 315, a sound input unit 320, a sound output unit 325, It includes a camera unit 330, a wireless network communication unit 335, a local area network communication unit 340, and a USIM reader unit 345, and may include a battery for supplying power.

The control unit 305 is a general term for components that control the operation of the wireless terminal 300, and may include at least one processor and an execution memory, and each component provided in the wireless terminal 300 and It can be connected through a bus (BUS). According to the present invention, the control unit 305 loads and calculates at least one program code provided in the wireless terminal 300 into the execution memory through the processor, and calculates the result to at least one configuration through the bus. It is possible to control the operation of the wireless terminal 300 by transferring the information to the unit. Hereinafter, for convenience, a configuration corresponding to the app 355 of the present invention implemented in the form of program code will be illustrated and described in the control unit 305.

The memory unit 350 is a generic term for non-volatile memory corresponding to storage resources of the wireless terminal 300, and includes at least one program code executed through the control unit 305 and at least one program code used. Datasets can be saved and maintained. The memory unit 350 basically stores system program codes and system data sets corresponding to the operating system of the wireless terminal 300, communication program codes and communication data sets for processing the wireless communication connection of the wireless terminal 300, and at least One application program code and application data set can be stored, and the program code and data set corresponding to the app 355 of the present invention can also be stored in the memory unit 350.

The screen output unit 310 includes a screen output device (for example, a liquid crystal display (LCD), etc.) and an output module driving the same, and is interlocked with the control unit 305 to obtain various calculation results of the control unit 305. An operation result corresponding to the screen output may be output to the screen output device.

The user input unit 315 includes one or more user input devices (eg, a button, a keypad, a touch pad, a touch screen linked to the screen output unit 310, etc.) and an input module that drives them, and the control unit 305 ) to input commands for instructing various operations of the control unit 305 or data required for operation of the control unit 305 may be input.

The sound input unit 320 may be composed of a microphone for receiving a sound signal and a driving module for driving the same, and encodes the sound signal input through the microphone according to a designated encoding rule to the control unit 305. can be conveyed

The sound output unit 325 may be composed of a speaker that outputs a sound signal and a driving module that drives the same, and decodes sound data corresponding to sound output among various calculation results of the controller 305 to It can be output through the speaker.

The camera unit 330 includes a camera lens, an image sensor, and a camera module that drives the same, and includes actuators and/or various filters (eg, IR filters) for an auto-focus function or an image stabilization function. It is made by further including Preferably, the camera unit 330 receives an optical signal for a subject within a designated angle of view among subjects in the viewing direction through a camera lens, digitizes it through an image sensor, generates image data, and provides the image data to the control unit 305. , An auto-focus function or an image stabilization function may be provided through the actuator, and the quality of the image data may be corrected through at least one filter.

The wireless network communication unit 335 and the local area network communication unit 340 are generic names for communication resources for connecting the wireless terminal 300 to a designated communication network. Preferably, the wireless terminal 300 may include a wireless network communication unit 335 as a basic communication resource and may include one or more local area network communication units 340 .

The wireless network communication unit 335 is a general term for communication resources that connect the wireless terminal 300 to a wireless communication network via a base station, and includes an antenna, an RF module, a baseband module, and a signal for transmitting and receiving radio frequency signals of a specific frequency band. It is configured to include at least one processing module, and is connected to the control unit 305 to transmit an operation result corresponding to wireless communication among various operation results of the control unit 305 through a wireless communication network or to receive data through a wireless communication network. and transmits it to the control unit 305, and at the same time performs the wireless communication access, registration, communication, and handoff procedures. According to the present invention, the wireless network communication unit 330 can connect the wireless terminal 300 to a communication network including a communication channel and a data channel via an exchange, and in some cases packet communication without going through the exchange. It can be connected to a data network that provides based wireless network data communication (eg, Internet).

According to the implementation method of the present invention, the wireless network communication unit 335 performs at least one of access to a mobile communication network, location registration, call processing, call connection, data communication, and handoff according to the CDMA/WCDMA/LTE standard. contains the composition Meanwhile, according to the intention of those skilled in the art, the wireless network communication unit 335 may further include a portable Internet communication configuration for performing at least one of access to the portable Internet, location registration, data communication, and handoff according to IEEE 802.16 related standards. It is clearly stated that the present invention is not limited by the wireless communication configuration provided by the wireless network communication unit 335. That is, the wireless network communication unit 335 is a general term for a component that accesses a wireless communication network through a cell-based base station regardless of a frequency band of a radio section or a type or protocol of a communication network.

The local area network communication unit 340 connects a communication session using a radio frequency signal as a communication medium within a certain distance (eg, 10 m) and connects the wireless terminal 300 to a communication network based on this. , Preferably, the wireless terminal 300 may be connected to a communication network through at least one of Wi-Fi communication, Bluetooth communication, public wireless communication, and UWB. According to the present invention, the local area network communication unit 340 connects the wireless terminal 300 to a data network providing short-range wireless data communication based on packet communication through a wireless AP.

The USIM reader unit 345 is a general term for a configuration that exchanges at least one data set with a universal subscriber identity module mounted on or detachable from the wireless terminal 300 based on the ISO/IEC 7816 standard. As, the data set is exchanged in a half-duplex communication method through an APDU (Application Protocol Data Unit).

The USIM is a SIM type card equipped with an IC chip conforming to the ISO/IEC 7816 standard, an input/output interface including at least one contact point connected to the USIM reader unit 345, and at least one program code for the IC chip. and an IC chip memory for storing a data set, and an IC chip memory connected to the input/output interface and calculating program codes for the IC chip according to at least one command transmitted from the wireless terminal 300 or extracting (or processing) the data set and a processor that transmits data to the input/output interface.

The controller 305 of the wireless terminal 300 performs the function of the wireless terminal 300 according to the present invention from an app providing server (eg, Apple's App Store, etc.) designated through a data network to which the communication resources are accessible. A process of downloading and installing the app 355 for the application may be performed. Meanwhile, when the app 355 downloaded and installed in the wireless terminal 300 is executed for the first time after installation (or executed before registering information), the combination of the wireless terminal 300 and the app 355 (or the app 355 installed) Create medium identification information (eg, device token, UUID, etc.) that identifies the wireless terminal 300 or the app 355 installed on the wireless terminal 300 as a unique medium, or through the designated operation server 100 A procedure for receiving an assignment may be performed, and a procedure for registering medium identification information for uniquely identifying the combination of the wireless terminal 300 and the app 355 as a unique medium in the designated operation server 100 may be performed.

Meanwhile, the app 355 of the wireless terminal 300 performs a procedure of registering wireless terminal information (eg, phone number, IMSI, IMEI, etc.) uniquely identifying the wireless terminal 300 in the designated operation server 100. In this case, the operation server 100 may store and manage the medium identification information and the wireless terminal information in association with each other. On the other hand, the app 355 of the wireless terminal 300 is unique identification information (eg, resident registration number, unique ID (or member ID), name, name, date of birth, etc.), and in this case, the operating server 100 can store and manage the medium identification information and the unique identification information of the user in connection with each other.

Referring to Figure 3, the app 355 of the wireless terminal 300, the notification information receiving unit 360 for receiving notification information from the operation server 100, and processing the audio data received from the operation server 100 It will be composed of an audio data processing unit 365 that transmits sound data to the operation server 100, a sound data processing unit 370 that transmits sound data to the operation server 100, and an image data processing unit 375 that transmits image data to the operation server 100. can

The notification information receiving unit 360, when the operating server 100 confirms the crying state of a specific baby photographed through a camera device provided at a specific base, the crying state of the specific baby from the operating server 100 It is possible to receive notification information notifying.

The audio data processing unit 365 may receive audio data stored in a specific storage area of the storage medium 145 of the operation server 100 and output the received audio data through the sound output unit.

The sound data processing unit 370 may transmit encoded sound data received from the sound input unit to the operation server 100 so as to be output through the sound output unit of the camera device.

The image data processing unit 375 transmits the image data photographed through the camera unit to the operating server 100 and may project and output the image data through the projector unit of the camera device 200 .

4 is a diagram illustrating a process of recognizing a crying baby and confirming a crying state of a baby according to the method of the present invention.

In more detail, this drawing 4 is provided with a sound input unit, a sound output unit, and a camera module, and has a projector unit for projecting a designated shape or image onto a ceiling or wall, and communicates with a camera device 200 provided at a designated base. 100) receives and stores audio data and video data of a specific baby in the base that are input or photographed through the camera device 100, and recognizes the baby's crying by reading the audio data to determine the baby's crying state As shown in the process of confirming, if you have ordinary knowledge in the art to which the present invention belongs, refer to and / or modify this Figure 4 to recognize the baby's crying sound and confirm the crying state of the baby. Various implementation methods (eg, implementation methods in which some steps are omitted or the order is changed) may be inferred, but the present invention includes all implementation methods inferred above, and only the implementation method shown in FIG. 4 Its technical features are not limited.

Referring to Figure 4, an operation server 100 having a sound input unit, a sound output unit, and a camera module, and having a projector unit for projecting a designated shape or image onto a ceiling or wall and communicating with a camera device 200 provided at a designated base. ), stores the unique information for managing the service using the camera device in the designated operation D/B (140) (400).

In addition, the operation server 100 may store guardian wireless terminal information for contacting the guardian of a specific baby photographed through the camera device 200 in the designated operation D/B 105, and the camera device 200 ) Unique identification information for communicating with the app of the guardian wireless terminal 300 possessed by the guardian of a specific baby photographed through may be stored in the designated operating D/B 140.

The camera device 200 photographs a specific baby in the base through a camera module to generate video data encoded in a designated manner (405), and generates audio data input through a sound input unit and encoded in a designated manner (410). ), and transmits the generated video data and audio data to the operation server 100 (415).

The operating server 100 captures audio data input through the sound input unit of the camera device 200 and encoded in a designated way, and a specific baby in the base through the camera module of the camera device 200 and uses a designated method. The encoded video data is received and stored and managed in a specific storage area of the designated storage medium 145 (420).

According to the implementation method of the present invention, the operation server 100 receives moving picture data including the audio data and video data from the camera device 200 and stores and manages the video data in a specific storage area of the designated storage medium 145. process can be maintained.

In addition, according to the implementation method of the present invention, the operation server 100 receives the audio data and video data from the camera device 200 and stores them in a specific storage area of the designated storage medium 145 and manages the process. The data management unit 110 may synchronize the times of the audio data and the video data respectively received from the camera device 200 and store them in a specific storage area of the designated storage medium 145 .

After the audio data and video data are stored in a specific storage area of the designated storage medium 145, the operation server 100 reads the audio data stored in the specific storage area of the storage medium 145 (425), Maintaining a matched sound within a preset similarity range with at least one baby crying pattern among the preset P (P≥1) baby crying patterns included in the designated baby crying frequency band for more than a preset holding time By recognizing the baby's crying sound, the baby's crying state is checked (430).

According to the implementation method of the present invention, when the operation server 100 recognizes the crying sound of the baby, it reads video data received from the camera device 200 in real time and stored in a specific storage area of the storage medium 145. to recognize more than a specified number of feature points existing in the facial region of a specific baby photographed through the camera device, analyze the geometric relationship between feature points of the recognized specific baby's facial region, and analyze the feature point distribution pattern corresponding to the baby's crying state and Matching within a preset similarity range can be confirmed, and when the facial region of the specific baby is recognized, the crying state of the baby can be confirmed by combining the result of recognizing the crying sound of the baby with the result of recognizing the facial region of the specific baby. there is.

5 is a diagram illustrating a process of soothing a crying baby by executing visual response measures and auditory response measures through a camera device according to an embodiment of the present invention.

In more detail, FIG. 5 shows the operation server 100 after reading the audio data received from the camera device 100 through the process of FIG. 4 to recognize the baby's crying sound and confirming the baby's crying state. In response to the confirmed crying state, auditory response measures through the sound output unit of the camera device 200 and/or visual response measures through the projector unit of the camera device 200 are executed to treat the baby whose crying state has been confirmed. As an illustration of a soothing process, those skilled in the art will refer to and/or modify Figure 5 to execute visual and auditory countermeasures through the camera device 200. Therefore, it will be possible to infer various implementation methods (eg, implementation methods in which some steps are omitted or the order is changed) for the process of soothing a crying baby, but the present invention includes all the implementation methods inferred above. The technical characteristics are not limited only to the implementation method shown in FIG. 5.

Referring to FIG. 5, after the crying state of the baby is confirmed through the process of FIG. 4, the operation server 100 sets a crying situation corresponding to the crying state of a specific baby photographed through the camera device 200. It is checked whether correspondence rule information is stored in the designated operation D/B (140) (500). Here, the crying situation response rule information may further include one or more response measures (or combinations of response measures) for each rank in addition to the response measures (or combination of response measures) of the highest priority.

According to the implementation method of the present invention, the operation server 100, when unique information for managing the service using the camera device 200 is stored in the designated operation D/B 140, the camera device ( 200), it can be checked whether the crying situation response rule information including the highest priority response action (or combination of response actions) for ending the crying state of a specific baby captured is stored in association with the unique information.

In addition, according to the implementation method of the present invention, the operation server 100 qualifies and quantifies the characteristics of the baby's cry terminated by a specific response measure (or a combination of response measures) to generate crying It may be confirmed whether the sound feature information is stored in association with the crying situation response rule information including the specific response action (or combination of response actions) as the highest priority response action (or combination of response actions).

According to the present invention, the countermeasure is performed when the guardian wireless terminal information for contacting the guardian of a specific baby photographed through the camera device 200 is stored in the designated operation D/B 140, the guardian wireless terminal A notification response measure for sending notification information notifying the crying state of the specific baby to the wireless terminal 300 of the guardian of the specific baby using the information may be further included.

In addition, according to the present invention, the countermeasure is an operation D/B (140) designated with unique identification information for communicating with the app of the guardian's wireless terminal (300) possessed by the guardian of the specific baby photographed through the camera device (200). ), in conjunction with the app of the guardian wireless terminal 300 based on the unique identification information, the audio data stored in the specific storage area of the storage medium 145 is transferred to the guardian wireless terminal 300. It may further include a guardian interlocking countermeasure for outputting through the sound output unit or processing the encoded sound data input through the sound input unit of the guardian wireless terminal 300 to be output through the sound output unit of the camera device 200. there is. Here, the guardian interlocking response measures interwork with the app of the guardian wireless terminal 300 to project image data captured by the camera unit of the guardian wireless terminal 300 through the projector unit of the camera device 200 and output the image data. countermeasures may be included.

When the operation D/B 140 stores the rule information corresponding to the crying situation of the specific baby, the operation server 100 performs f shapes through the projector unit of the camera device 200 according to the stored rules corresponding to the crying situation of the baby. Confirm a visual response measure for projecting a designated projection target among i or i images and an auditory response measure for outputting a designated output target among m music, s noise, and v voice through the sound output unit of the camera device 200 In step 505, the response action (or combination of response actions) set as the highest priority for the specific baby among the confirmed response actions is checked and executed (510).

On the other hand, if it is confirmed that the specific baby's crying situation response rule information is not stored in the operation D/B 140, the operation server 100 operates the projector of the camera device 200 according to a preset common response rule. Visual countermeasures for projecting a designated projection target among f (f≥1) shapes or i (i≥0) images through the unit and m (m≥1) pieces of music, s (s) through the sound output unit of the camera device Among auditory countermeasures that output designated output targets among ≥1) noises and v(v≥1) voices, preset countermeasures (or combinations of countermeasures) are sequentially (or combined) at regular intervals in a designated order. Run (515).

According to the implementation method of the present invention, the operation server 100 sets one or more response measures (or combinations of response measures) for each priority in addition to the response measures (or combinations of response measures) of the highest priority in the crying situation response rule information. In addition, when the operation D/B 140 stores the specific baby's crying situation response rule information, the camera device 200 executes the highest priority response measure (or combination of response measures) for a specified time. ) and reads audio data received in real time and stored in a specific storage area of the storage medium 145 to determine whether the crying state of the baby is terminated in response to the executed countermeasure (or a combination of countermeasures).

In addition, the operation server 100 executes the corresponding action (or combination of response actions) of the next rank when the crying state of the baby does not end within a specified time after executing the corresponding action (or combination of response actions) of the next rank. While executing for a specified time, audio data received in real time from the camera device 200 and stored in a specific storage area of the storage medium 145 is read, and the baby responds to the executed countermeasure (or a combination of countermeasures). The process of checking whether the crying state ends may be repeated up to the last rank.

On the other hand, according to the implementation method of the present invention, the operating server 100 reads audio data stored in a specific storage area of the storage medium 145 when the corresponding action (or combination of corresponding actions) is executed, and the baby In addition to the crying sound, a guardian's soothing voice uttered to relieve the crying state of the specific baby may be recognized, and when the soothing voice is recognized, the execution of the response action (or combination of response actions) may be terminated. .

In addition, according to the implementation method of the present invention, the operation server 100 reads video data stored in a specific storage area of the storage medium 145 when the corresponding action (or combination of corresponding actions) is executed, and the specific It is possible to recognize the guardian's soothing action to relieve the baby's crying state, and when the soothing action is recognized, execution of the corresponding action (or combination of corresponding actions) may be terminated.

After the countermeasure (or combination of countermeasures) set for the specific baby is confirmed and executed, while the operation server 100 executes the designated countermeasure (or combination of countermeasures) through the camera device 200. When the audio data stored in the specific storage area of the storage medium 145 is read and the baby's cry is recognized (520), the executed countermeasure (or combination of countermeasures) based on the recognized baby's cry In response, it is confirmed whether the baby crying state of the specific baby ends (525).

According to the implementation method of the present invention, the operation server 100 reads video data stored in a specific storage area of the storage medium 145 while a designated countermeasure is being executed through the camera device 200 and recognizes the video data. It is possible to recognize a designated facial expression change that responds visually or audibly to the executed response action (or a combination of response actions) by analyzing the change pattern between feature points of the facial region of a specific baby, and when the designated change in expression is recognized , It is possible to determine whether the crying state of the specific baby is ended by combining the result of recognizing the crying sound of the baby and the result of recognizing the facial expression change.

When it is confirmed that the crying state of the specific baby is ended, the operation server 100 ends the execution of the response action (or combination of response actions) within a specified time (530).

According to the implementation method of the present invention, the operation server 100, when the crying baby state ends, the response action including the type and execution time of the specific response action (or combination of response actions) that ended the crying state. The history information is configured and stored in the operation D/B (140), and a plurality of response action history information stored in the operation D/B (140) is analyzed to respond with the highest priority for terminating the crying state of the specific baby. Crying situation response rule information including measures (or a combination of countermeasures) may be generated and stored in the operation D/B 140 .

In addition, the operating server 100 reads the audio data stored in the specific storage area of the storage medium 145 and listens to the soothing voice of the guardian uttered to relieve the crying state of the specific baby in addition to the crying sound of the baby. If the soothing voice is recognized, the process of storing the corresponding action history information may be omitted.

According to the implementation method of the present invention, the operating server 100 reads video data stored in a specific storage area of the storage medium 145 and recognizes a guardian's soothing action to relieve the crying state of the specific baby. If the appeasement operation is recognized, the process of storing the corresponding action history information may be omitted.

According to the implementation method of the present invention, the operation server 100 analyzes a plurality of response action history information stored in the operation D/B 140 to take a response action (or response action) that terminates the crying state of the specific baby. Combinations) are classified and sorted on the basis of the number of terminations, and the response action (or combination of response actions) that terminates the baby crying state of the specific baby the most is included as the highest priority response action (or combination of response actions) Crying situation response rule information may be generated and stored in the operation D/B 140 .

According to the implementation method of the present invention, the operation server 100 responds to a crying situation further including one or more corresponding measures (or combinations of countermeasures) for each priority in addition to the corresponding measures (or combinations of countermeasures) of the highest priority. Rule information can be created.

According to the implementation method of the present invention, the response action history information may further include audio data including a baby's cry terminated by each response action (or a combination of response actions), and the operation server 100 The characteristics of the baby's crying included in the audio data of the response action history information corresponding to the response action (or combination of response actions) of the highest priority are qualified and quantified to ensure that the response action of the highest priority (or Crying situation including generating characteristic information of the crying sound of the highest priority, which is terminated by the combination of response measures), and linking the response action of the highest priority (or combination of response measures) with the generated characteristic information of the highest priority crying sound Corresponding rule information may be generated and stored in the operation D/B 140 .

According to the implementation method of the present invention, the operation server 100 provides response action history information corresponding to one or more response measures (or combinations of response actions) for each priority in addition to the response action (or combination of response actions) of the highest priority. Characteristics of crying sounds included in the audio data of can be qualitatively and quantified to generate information on characteristics of crying sounds for each ranking level terminated by response measures (or combinations of countermeasures) for each ranking level, and corresponding measures for each ranking level. Crying situation response rule information further including (or a combination of countermeasures) and the generated crying sound characteristic information for each rank may be generated and stored in the operation D/B 140 .

100: operation server 105: information storage unit
110: data management unit 115: status check unit
120: information confirmation unit 125: response action execution unit
130: countermeasure reaction unit 135: execution end unit
140: operation D/B 145: storage medium
200: camera device 300: user wireless terminal

Claims (21)

A method executed through an operation server having a sound input unit, a sound output unit, and a camera module and having a projector unit for projecting a designated shape or image onto a ceiling or wall and communicating with a camera device provided at a designated base,
Specific storage in a designated storage medium by receiving audio data input through the sound input unit of the camera device and encoded in a designated manner and video data encoded in a designated manner by photographing a specific baby in the base through the camera module of the camera device A first step of managing storage in an area;
Audio data stored in a specific storage area of the storage medium is read and included in a designated baby crying frequency band and at least one baby crying pattern among preset P (P≥1) baby crying patterns and a preset similarity range A second step of confirming the baby's crying state by recognizing the baby's crying sound that maintains the matched sound within a predetermined holding time or longer;
If the crying state of the baby is confirmed, a third step of checking whether the crying situation response rule information set in correspondence with the crying state of the specific baby photographed through the camera device is stored in a designated operating D/B;
If the specific baby's crying situation response rule information is not stored in the operating D/B, f (f≥1) shapes or i (i≥0) shapes or i (i≥0) shapes are displayed through the projector unit of the camera device according to a preset common response rule. Designated output targets among m (m≥1) pieces of music, s (s≥1) pieces of noise, and v (v≥1) pieces of voices through a visual countermeasure to project a specified projection target among images and a sound output unit of the camera device Among the auditory response measures that output , preset response measures (or combinations of response measures) are sequentially (or combined) executed for a certain period of time according to a designated order, and the specific baby's crying situation response rule information is stored in the operation D/B. When is stored, a visual response measure of projecting a designated projection target among f shapes or i images through the projector unit of the camera device according to the stored crying situation response rule, and m pieces of music and s pieces of music through the sound output unit of the camera device. A fourth step of identifying and executing a response measure (or a combination of response measures) set as the highest priority for the specific baby among auditory response measures for outputting a designated output target among noise and v voices;
While the corresponding countermeasure (or combination of countermeasures) specified through the camera device is being executed, the corresponding countermeasure (or countermeasure) executed based on the recognized baby's cry by reading the audio data stored in the specific storage area of the storage medium. a fifth step of confirming whether the crying state of the specific baby is terminated in response to a combination of measures; and
A sixth step of terminating the execution of the response measures (or a combination of countermeasures) within a specified time when the baby's crying state ends.
The method of claim 1, wherein the first step,
and maintaining a process of receiving moving picture data including the audio data and video data from the camera device and storing and managing them in a specific storage area of a designated storage medium. .
The method of claim 1, wherein the first step,
and maintaining a process of receiving the audio data and video data from the camera device and storing and managing them in a specific storage area of a designated storage medium.
The method of claim 3, wherein the first step,
The method of soothing a crying baby using a camera device, characterized in that it further comprises the step of synchronizing the times of the audio data and video data received from the camera device and storing them in a specific storage area of a designated storage medium.
The method of claim 1, wherein the second step,
If the baby's cry is recognized,
reading video data received in real time from the camera device and stored in a specific storage area of the storage medium, and recognizing a predetermined number or more feature points existing in a facial region of a specific baby photographed through the camera device; and
Further comprising the step of analyzing the geometric relationship between the feature points of the recognized facial region of the specific baby and confirming that the feature point distribution pattern corresponding to the baby's crying state is matched within a preset similarity range,
When the facial area of the specific baby is recognized,
The method of soothing a crying baby using a camera device comprising the step of confirming the crying state of the baby by combining the result of recognizing the crying sound of the baby with the result of recognizing the facial region of the specific baby.
According to claim 1,
Storing unique information for managing services using the camera device in a designated operating D/B;
The third step,
Checking whether the crying situation response rule information including the highest priority response action (or combination of response actions) for ending the crying state of a specific baby captured through the camera device is stored in association with the unique information A method of soothing a crying baby using a camera device, comprising:
The method of claim 1 or 6, wherein the third step,
Crying feature information generated by qualifying and quantifying the characteristics of a baby's cry terminated by a specific response action (or combination of response actions) gives the specific response action (or combination of response actions) priority The method of soothing a crying baby using a camera device, characterized in that it further comprises a step of checking whether the response measures (or combinations of response measures) in the order are stored in association with the crying situation response rule information.
According to claim 1,
Further comprising the step of storing guardian wireless terminal information for contacting the guardian of a specific baby photographed through the camera device in a designated operating D/B,
The response measure may further include a notification response measure for sending notification information informing of the baby crying state of the specific baby to the guardian wireless terminal of the specific baby using the guardian wireless terminal information. How to soothe a crying baby.
According to claim 1,
Further comprising storing unique identification information for communication with an app of a guardian wireless terminal possessed by a guardian of a specific baby photographed through the camera device in a designated operating D/B,
The countermeasure is to process audio data stored in a specific storage area of the storage medium in conjunction with an app of the guardian wireless terminal based on the unique identification information to be output through the sound output unit of the guardian wireless terminal, or the guardian wireless terminal. The method of calming a crying baby using a camera device, characterized in that it further comprises a guardian interlocking countermeasure for processing the encoded sound data input through the sound input unit of the terminal to be output through the sound output unit of the camera device.
The method of claim 9, wherein the guardian interlocking countermeasures,
In conjunction with the app of the guardian wireless terminal, the image data captured through the camera unit of the guardian wireless terminal is projected through the projector unit of the camera device and processed to be output. How to soothe a crying baby.
According to claim 1,
The crying situation response rule information further includes one or more response measures (or combination of response actions) for each priority in addition to the response action (or combination of response actions) of the highest priority,
In the fourth step,
When the operation D/B stores rule information corresponding to the crying situation of the specific baby,
The executed countermeasure (or countermeasure) is performed by reading the audio data received from the camera device in real time and stored in a specific storage area of the storage medium while executing the countermeasure (or combination of countermeasures) of the highest priority for a specified time. Combination of) confirming whether the crying state of the baby is terminated in response to; and
If the baby's crying state does not end within a specified time after executing the countermeasures (or combination of countermeasures) of the designated rank, while executing the countermeasures (or combination of countermeasures) of the next rank for the designated period of time, the camera device Repeating a process of reading audio data received in real time and stored in a specific storage area of the storage medium and confirming whether the crying state of the baby is terminated in response to the executed response action (or combination of response actions) up to the last rank. A method of soothing a crying baby using a camera device, characterized in that it further comprises.
The method of claim 1, wherein the fourth step,
Upon implementation of the above countermeasures (or combination of countermeasures);
reading audio data stored in a specific storage area of the storage medium and recognizing a soothing voice of a guardian uttered to relieve the crying state of the specific baby in addition to the crying sound of the baby; and
The method of soothing a crying baby using a camera device according to claim 1 , further comprising terminating the execution of the corresponding measures (or a combination of corresponding measures) when the soothing voice is recognized.
The method of claim 1, wherein the fourth step,
Upon implementation of the above countermeasures (or combination of countermeasures);
reading video data stored in a specific storage area of the storage medium and recognizing a guardian's soothing action to relieve the crying state of the specific baby; and
The method of soothing a crying baby using a camera device according to claim 1 , further comprising the step of terminating the execution of the response action (or combination of response actions) when the soothing action is recognized.
The method of claim 1, wherein the fifth step,
While the corresponding action specified through the camera device is being executed, video data stored in a specific storage area of the storage medium is read and a pattern of change between feature points of the recognized facial region of a specific baby is analyzed, and the corresponding action (or corresponding action) is executed. Further comprising recognizing a specified facial expression change that visually or auditorily responds to a combination of actions),
When the above-specified facial expression change is recognized,
The method of soothing a crying baby using a camera device comprising the step of confirming whether the crying state of the specific baby is ended by combining the result of recognizing the crying sound of the baby and the result of recognizing the change in facial expression.
According to claim 1,
When the baby's crying state is terminated, the 6th-constructing and storing the corresponding action history information including the type and execution time of the specific response action (or combination of response actions) that terminated the crying baby state and storing it in the operation D/B. Level 1; and
By analyzing the plurality of response action history information stored in the operation D/B, the crying situation response rule information including the highest priority response action (or combination of response actions) for ending the crying state of the specific baby is generated. The method of soothing a crying baby using a camera device, characterized in that it further comprises a 6-2 step of storing in the operating D/B.
According to claim 15,
Further comprising the step of reading audio data stored in a specific storage area of the storage medium and recognizing the soothing voice of the guardian uttered to relieve the crying state of the specific baby in addition to the crying sound of the baby,
The method of soothing a crying baby using a camera device, characterized in that step 6-1 further comprises omitting the process of storing the corresponding action history information when the soothing voice is recognized.
According to claim 15,
Reading video data stored in a specific storage area of the storage medium to recognize a guardian's soothing action to relieve the crying state of the specific baby;
The step 6-1 further comprises the step of omitting the process of storing the corresponding action history information when the soothing action is recognized.
The method of claim 15, wherein the 6-2 step,
A plurality of response action history information stored in the operating D/B is analyzed to classify the response action (or combination of response actions) that terminated the baby's crying state of the specific baby, sorted based on the number of terminations, and the specific baby's baby Creating and storing the crying situation response rule information including the response action (or combination of response actions) that terminates the crying state the most as the highest priority response action (or combination of response actions) and storing it in the operating D/B A method of soothing a crying baby using a camera device, comprising:
The method of claim 18, wherein the 6-2 step,
In addition to the response action (or combination of response actions) of the highest priority, generating crying situation response rule information further comprising one or more response actions (or combination of response actions) for each priority. A method to soothe a crying baby using a camera device.
According to claim 15,
The response action history information further includes audio data including a crying sound of a baby terminated by each response action (or combination of response actions);
In the 6-2 step,
The characteristics of the baby's crying included in the audio data of the response action history information corresponding to the response action (or combination of response actions) of the highest priority are qualified and quantified to ensure that the response action of the highest priority (or generating the highest-priority cry feature information terminated by a combination of countermeasures; and
generating crying situation response rule information including linking the highest priority response action (or a combination of response actions) with the generated crying sound characteristic information of the highest priority order and storing it in the operating D/B; A method of soothing a crying baby using a camera device, characterized in that.
The method of claim 20, wherein the 6-2 step,
Qualification and quantification of the characteristics of the baby's crying included in the audio data of the response action history information corresponding to one or more response actions (or combinations of response actions) for each priority in addition to the response action (or combination of response actions) of the highest priority above. generating cry feature information for each rank that is terminated by a corresponding measure for each rank (or a combination of corresponding measures); and
generating and storing in the operation D/B rule information corresponding to a crying situation, which further includes response measures (or combinations of corresponding measures) for each rank and characteristic information of crying sounds for each rank; A method of soothing a crying baby using a camera device characterized in that.

Images