KR20210072202A - Method for Outputting Infants Emotional Condition - Google Patents

Abstract

According to the present invention, a method for outputting the emotional state of an infant is carried out through an operating server communicating with a camera device provided at a designated base and having a sound input unit and a camera module. The method generates expression-based emotional state determination information for determining an emotion state value of a baby for each designated emotional item corresponding to a baby expression through a designated artificial intelligence-based learning process to store the expression-based emotional state determination information in a designated storage area, photographs a specific baby within the base through the camera module of the camera device to receive video data encoded by a designated method and audio data inputted through the sound input unit of the camera device to be encoded by a designated method to link, store, and manage the video data and the audio data in a specific storage area of a designated storage medium, performs a procedure for reading designated video data through the artificial intelligence-based expression-based emotional state determination information to determine baby emotion state values for n1 (1<=n1<=N1) emotional items corresponding to baby expressions of the video data based on N1 (1<=N1<=N) baby expression data distinguishable through baby expressions among designated N (N>=2) baby emotion items, uses the calculated baby emotion state values for the n1 emotional items to generate baby emotion state information for designated n (1<=n<=N) emotional items to be displayed on the video data, and provides transmission data including video data displaying the generated baby emotion state information for the n emotional items in a designated area on the video data for an IPTV or an application of a user terminal.

Description

{Method for Outputting Infants Emotional Condition}

The present invention provides one or more data acquired through the camera device when transmitting data including video data of a specific baby in the base through a camera device provided at a designated base is provided to an app or IPTV of a designated user's wireless terminal. Based on the basis, baby emotional state information for each designated emotional item for the specific baby is generated, and the baby emotional state information for each of the generated emotional items is displayed and provided in a designated area on the video data of the transmission data.

A service that provides video or still images and newborn information (e.g., taemong, ultrasound diagnostic photos, etc.) of a baby who has been registered as a member in a hospital or midwifery or postpartum cooking center where multiple web PC cameras are conventionally installed to members through the Internet is proposed. (Patent Publication No. 2001-0091236 (October 23, 2001)).

However, the conventional service simply provides a baby's video or still image and newborn information through the Internet. It has technically difficult problems.

It is an object of the present invention to solve the above problems, after storing emotional state determination information for determining the baby emotional state value for each designated emotional item of the baby, and then using the camera module of the camera device, a specific baby in the base By performing a procedure of receiving acquired data including video data obtained by photographing and storing it in a specific storage area of a specified storage medium, and reading one or more data including specified video data through the emotional state determination information, Determines the baby emotional state value for each specified emotional item for the baby, generates baby emotional state information for each specified emotional item using the calculated one or more baby emotional state information, and generates the generated baby emotional state information in a specified predetermined area on the video data. An object of the present invention is to provide a method for outputting an infant's emotional state by providing transmission data including video data indicating baby emotional state information for each n emotional item to an app or IPTV of a user's wireless terminal.

The method for outputting the infant's emotional state according to the present invention comprises a sound input unit and a camera module and is executed through an operation server that communicates with a camera device provided at a designated base. A first step of generating expression-based emotional state determination information for determining the baby emotional state value for each designated emotional item corresponding to the baby's expression through the first step and storing it in a designated storage area, and a specific baby in the base through the camera module of the camera device A second step of receiving video data encoded in a specified method by photographing and audio data encoded in a specified method input through the sound input unit of the camera device, and storing and managing in a specific storage area of a specified storage medium in association with the artificial intelligence; N1 (1≤N1≤N) baby emotion items that can be distinguished through baby expression among N (N≥2) specified baby emotion items by performing a procedure of reading specified video data through expression-based emotional state identification information Based on the third step of determining the baby emotional state value for each n1 (1≤n1≤N1) emotional item corresponding to the baby expression in the video data, and the calculated baby emotional state value for each of the n1 emotional items. A fourth step of generating baby emotional state information for each of n (1≤n≤N) specified emotional items that can be displayed on the video data, and the generated baby emotional state information for each of the n emotional items in a specified area on the video data and a fifth step of providing transmission data including the displayed video data to an app or IPTV of the user's wireless terminal.

In the method for outputting an infant's emotional state according to the present invention, the first step is e corresponding to a designated baby facial region among E (E≥2) baby expressions designated by performing a designated AI-based learning process. (1≤e≤E) baby expressions are determined, and N1 (1≤N1≤N) baby emotion items that can be distinguished through baby expressions among the specified N (N≥2) baby emotion items are determined above. n1 (1≤n1≤N1) baby emotion items having a matching relationship with e baby expressions are identified, and babies by n1 emotion items corresponding to the correlation between the identified e baby expressions and n1 baby emotion items and generating expression-based emotional state determination information for determining the emotional state value and storing it in a designated storage area.

In the method for outputting an infant's emotional state according to the present invention, the first step comprises: N (N≥2) specified while corresponding to the specified e(1≤e≤E) baby expressions among the specified E(E≥2) baby expressions ) among the baby emotion items, a plurality of video information corresponding to a scene including N1 (1≤N1≤N) baby emotion items distinguishable through baby expressions and a baby face region having a specified correlation are input, and the input Setting the baby emotional state value for each of n1 (1≤n1≤N1) emotion items for the plurality of video information as an output variable and the geometry between the specified feature points of the baby's facial region included in the inputted plurality of video information By reading the relationship and recognizing the pattern, P(P≥2) scene pattern characteristic information corresponding to the baby emotional state value for each n1 emotion item corresponding to the baby expression in the baby's facial area is extracted as input variables (Feature Vectors) Baby emotion for each n1 emotion item corresponding to the baby's expression in the baby's face area among a plurality of scenes including the baby's face area by learning more than a specified number of the input and output variables through the setting step and a designated artificial intelligence algorithm and generating expression-based emotional state determination information for determining the state value and storing it in a designated storage area.

In the method for outputting an infant's emotional state according to the present invention, the first step includes: N (N≥2) specified while corresponding to the specified e(1≤e≤E) baby expressions among the specified E(E≥2) baby expressions ) of baby emotion items, N1 (1≤N1≤N) baby emotion items that can be distinguished through baby expressions and a specified number of baby face areas of a plurality of video information corresponding to a scene including a baby face area with a specified correlation P(P≥2) scenes extracted as pattern characteristics of n1 (1≤n1≤N1) emotional state values for each emotion item corresponding to the baby's expression in the baby's facial area by reading the geometric relationship between the feature points and recognizing the pattern The step of receiving pattern characteristic information as input variables (Feature Vectors), setting the baby emotional state value for each n1 emotion item for the video information from which the P scene pattern characteristic information is extracted, as an output variable, and the specified artificial intelligence algorithm Expression-based emotion for determining the baby emotional state value for each n1 emotion item corresponding to the baby's expression in the baby's facial area among a plurality of scenes including the baby's facial area by learning more than a specified number of the input and output variables through and generating and storing the state determination information in a designated storage area.

In the infant emotional state output method according to the present invention, the output variables include e pieces of baby expression information corresponding to the baby's facial region included in the video information, and n1 babies corresponding to the baby's facial expressions in the baby's facial area. It characterized in that it further comprises at least one piece of information of the emotion item.

In the method for outputting an infant's emotional state according to the present invention, the second step includes receiving the moving picture data including the video data and the audio data from the camera device and storing and managing it in a specific storage area of a designated storage medium. It is characterized in that it is done.

In the method for outputting an infant's emotional state according to the present invention, the second step comprises receiving the video data and the audio data from the camera device, respectively, and storing and managing them in a specific storage area of a designated storage medium. do it with

In the method for outputting an infant's emotional state according to the present invention, the second step further comprises the step of synchronizing the time of the video data and the audio data received from the camera device and storing them in a specific storage area of a designated storage medium. characterized by being made.

In the method for outputting an infant's emotional state according to the present invention, the transmission data may further include audio data stored in a specific storage area of the storage medium.

In the method for outputting an infant's emotional state according to the present invention, the designated video data includes at least one of video data periodically selected for each designated playback time unit and video data periodically selected for each designated number of playback frames. characterized by being made.

In the method for outputting an infant's emotional state according to the present invention, the method further comprising the step of confirming whether transmission data including video data stored in a specific storage area of the storage medium is provided by an App or IPTV of a designated user's wireless terminal do it with

In the method for outputting an infant's emotional state according to the present invention, the third step is to read the video data specified through the artificial intelligence-based expression-based emotional state determination information to perform a procedure of reading the video data among the E specified baby expressions Determine e baby facial expressions corresponding to the baby's facial area, and determine n1 baby emotion items that have a matching relationship with the e determined baby expression items among N1 baby emotion items that can be distinguished through the baby expression, and confirm the and determining the baby emotional state value for each n1 emotion item corresponding to the correlation between the e baby expressions and the n1 baby emotion items.

In the method for outputting an infant's emotional state according to the present invention, the fourth step is characterized in that it further comprises the step of storing the generated baby emotional state information for each of the n emotional items in association with the video data.

In the method for outputting an infant's emotional state according to the present invention, the fourth step is linking the generated baby emotional state information for each of the n emotional items with a time value (or timestamp) for synchronizing the video data with a designated playback time point It is characterized in that it is made including the step of storing.

In the method for outputting an infant's emotional state according to the present invention, the first step is a cry for determining a baby's emotional state value for each designated emotional item corresponding to a baby's cry through a designated artificial intelligence-based learning process. It characterized in that it further comprises the step of generating the base emotional state determination information and storing it in a designated storage area.

In the method for outputting an infant's emotional state according to the present invention, the first step is to perform a designated AI-based learning process to c corresponding to a designated baby cry among designated C (C≥2) baby cry sounds. (1≤c≤C) baby crying sounds are identified, and based on N2 (1≤N2≤N) baby emotion items that can be distinguished through baby crying among the specified N (N≥2) baby emotion items, n2 (1≤n2≤N2) baby emotion items having a matching relationship with the identified c baby cry sounds are identified, and n2 items corresponding to the correlation between the identified c baby cry sounds and n2 baby emotion items are identified. It is characterized in that it comprises the step of generating cry-based emotional state determination information for determining the baby emotional state value for each emotion item and storing it in a designated storage area.

In the method for outputting the infant's emotional state according to the present invention, the first step is to correspond to the specified c (1≤c≤C) baby cries among the specified C (C≥2) baby cries and specify N(N) Among the ≥2) baby emotion items, N2 (1≤N2≤N) baby emotion items that can be distinguished through baby cry sounds and a plurality of audio information corresponding to a baby crying with a specified correlation are input, and the input plurality setting the baby emotional state values for each n2 (1≤n2≤N2) emotion items for the audio information of The step of extracting Q (Q≥2) sound pattern characteristic information corresponding to the emotional state value of the baby for each emotional item and setting it as an input variable (Feature Vectors), and outputting more than a specified number of the input variables and output through a designated artificial intelligence algorithm Creating and storing cry-based emotional state determination information for determining the baby emotional state value for each n2 emotional items corresponding to the baby crying sound included in the specified audio data by learning the variable and storing it in the designated storage area; including; characterized by being made.

In the method for outputting the infant's emotional state according to the present invention, the first step is to correspond to the specified c (1≤c≤C) baby cries among the specified C (C≥2) baby cries and specify N(N) Among ≥2) baby emotion items, N2 (1≤N2≤N) baby emotion items that can be distinguished through baby cry sounds and a plurality of audio information corresponding to a baby crying with a specified correlation are read and pattern recognized Q (Q≥2) sound pattern characteristic information extracted as pattern characteristics of baby emotional state values for each n2 (1≤n2≤N2) emotion items corresponding to the cry are input as input variables (Feature Vectors), Setting the baby emotional state value for each n2 emotion items for the audio information from which Q sound pattern characteristic information is extracted as an output variable, and learning more than a specified number of the input and output variables through a specified artificial intelligence algorithm to learn the specified audio and generating cry-based emotional state determination information for determining the baby emotional state value for each n2 emotional items corresponding to the baby crying sound included in the data and storing it in a designated storage area.

In the infant emotional state output method according to the present invention, the output variable includes c pieces of baby cry information corresponding to the baby crying included in the audio information, and n2 pieces of baby emotion item information corresponding to the baby crying sound. It is characterized in that it further comprises at least one piece of information.

In the method for outputting an infant's emotional state according to the present invention, the third step is to perform a procedure of reading the specified audio data through the artificial intelligence-based cry-based emotional state determination information to perform the specified N (N≥2) Based on N2 (1≤N2≤N) baby emotion items that can be distinguished through baby crying among baby emotion items, baby emotions for each n2 (1≤n2≤N2) emotion items corresponding to the crying sound of the audio data It characterized in that it further comprises the step of determining the state value.

In the method for outputting an infant's emotional state according to the present invention, the third step is to perform a procedure of reading designated audio data through the artificial intelligence-based cry-based emotional state determination information to perform a procedure of reading out of the designated C baby cry sounds. Determine c baby cry sounds corresponding to baby crying on audio data, and determine n2 baby emotion items that have a matching relationship with the determined c baby cry sounds among N2 baby emotion items that can be distinguished through baby cry and determining the baby emotional state value for each n2 emotion items corresponding to the correlation between the identified c baby cry sounds and the n2 baby emotion items.

In the method for outputting an infant's emotional state according to the present invention, the fourth step further includes the step of generating baby emotional state information for each of the n designated emotional items by additionally using the calculated baby emotional state values for each of the n2 emotional items. It is characterized in that it comprises.

In the method for outputting an infant's emotional state according to the present invention, among the sensing data sensed through a sensor module provided in or linked to the camera device, S (S≥2) pieces of sensing data of a designated sensing target related to the emotional expression of a baby Matching relationship information between the sensing value pattern information and N3 (1≤N3≤N) baby emotion items that can be distinguished based on the sensing value pattern of the specified sensing target among the N designated baby emotion items, and the S sensed value pattern information and Further comprising the step of registering and storing sensing-based emotional state determination information including the sensing-based correlation numerical relationship information obtained by quantifying the correlation between N3 baby emotion items in the operation D/B, wherein the second step is the camera The method further comprises the step of receiving the sensed data sensed through a sensor module provided in or associated with the device, and storing and managing the data in a specific storage area of a designated storage medium in association with the video data, wherein the third step includes the video data and Recognizes k (k≥1) sensed value recognition information for the sensing data of a specified sensing target associated with the emotional expression of the specific baby by reading the associated sensing data, and the recognition based on the sensing-based emotional state determination information Check the k sensed value recognition information and s (1≤s≤S) sensed value pattern information matched with more than a preset standard similarity, and n3(1≤n3) ≤N3) baby emotion items are determined, and the correlation between the s sensed value pattern information and n3 baby emotion items is based on a matching ratio between the recognized k sensed value recognition information and the s sensed value pattern information Further comprising the step of calculating a baby emotional state value for each n3 emotion items obtained by digitizing the baby emotional state for each n3 emotion items corresponding to the recognized k sensed value recognition information by reading the numerical relationship, the fourth step is to generate baby emotional state information for each of the n designated emotion items by additionally using the calculated baby emotional state values for each of the n3 emotion items. It is characterized in that it further comprises a step.

In the method for outputting an infant's emotional state according to the present invention, the sensor module includes a wearable sensor module that is worn on a baby's body to sense a designated sensing target, and a radar that senses the baby's designated sensing target using a designated electromagnetic wave. (Radar) characterized in that it comprises at least one sensor module among the sensor modules.

In the method for outputting an infant's emotional state according to the present invention, the sensing target comprises at least one of a baby's heartbeat, a baby's body temperature, a baby's ambient humidity, and whether or not the baby is breathing.

In the method for outputting an infant's emotional state according to the present invention, the S sensed value pattern information includes the range information of the sensing value characteristics constituting each sensing value pattern for the sensing data of a designated sensing target related to the emotional expression of the baby. It is characterized in that it comprises.

In the method for outputting an infant's emotional state according to the present invention, the sensed value characteristic includes at least one or a combination of two or more of the sensing value change rule characteristic, the sensed value change characteristic, the sensed value change rate characteristic, and the sensed value change periodicity characteristic. It is characterized in that it comprises.

In the method for outputting the infant's emotional state according to the present invention, the sensing-based correlation numerical relationship information is proportional to the matching ratio between the sensing value recognition information recognized through the sensing data of a specified sensing target and the pre-registered sensing value pattern information The correlation calculated through the relational expression is also characterized in that it includes numerical information.

In the method for outputting an infant's emotional state according to the present invention, the sensing-based correlation numerical relationship information includes: each matching ratio interval between sensing value recognition information recognized through sensing data of a specified sensing target and pre-registered sensing value pattern information It is characterized in that the star correlation also includes numerical information.

In the infant emotional state output method according to the present invention, the baby emotional state information for each of the n emotional items comprises n emotional item information and numerical information of the baby's emotional state for each emotional item in the form of text. do.

In the method for outputting an infant's emotional state according to the present invention, the baby emotional state information for each of the n emotional items comprises n emotional item information and numerical information of the baby's emotional state for each emotional item in the form of a graph. do.

According to the present invention, when transmitting data including video data of a specific baby in the base through a camera device provided at a designated base is provided to an app or IPTV of a designated user wireless terminal, one or more acquired through the camera device By generating baby emotional state information for each specified emotional item for the specific baby based on the data, and displaying and providing the baby emotional state information for each of the generated emotional items in a specified area on the video data of the transmission data, A user who watches the transmission data through an app on the terminal or IPTV is a baby's video screen corresponding to the video data of the transmission data or the baby's sound corresponding to the audio data of the transmission data. It has the advantage of being able to grasp the status in real time.

1 is a diagram showing the configuration of a system for displaying a baby's emotional state according to an embodiment of the present invention.
2 is a diagram illustrating an information storage/registration process according to an embodiment of the present invention.
3 is a diagram illustrating storage and management of the acquired data acquired through the camera device 180 of a designated base according to an embodiment of the present invention and stored in a designated storage medium.
4 is a diagram illustrating transmission data including video data indicating baby emotional state information for each designated emotion item according to the first embodiment of the present invention.
5 is a diagram illustrating transmission data including video data indicating baby emotional state information for each designated emotion item according to a second embodiment of the present invention.
6 is a diagram illustrating transmission data including video data indicating baby emotional state information for each designated emotion item according to a third embodiment of the present invention.

Hereinafter, the principle of operation of the preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings and description. However, the drawings shown below and the following description are for preferred implementation methods among various methods for effectively explaining the features of the present invention, and the present invention is not limited only to the following drawings and description.

That is, the following embodiment corresponds to an embodiment of a preferred union type among numerous embodiments of the present invention, and an embodiment in which a specific configuration (or step) is omitted in the following embodiment, or a specific configuration (or step) An embodiment in which the implemented function is divided into a specific configuration (or step), or an embodiment in which the function implemented in two or more configurations (or step) is integrated into any one configuration (or step), of a specific configuration (or step) Embodiments in which the order of operation is replaced, etc. are clearly stated to be within the scope of the present invention, even if not separately mentioned in the following embodiments. Therefore, it is clearly specified that various embodiments corresponding to a subset or a complement can be divided retroactively from the filing date of the present invention based on the following examples.

In addition, in the following description of the present invention, if it is determined that a detailed description of a related well-known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. And the terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of the user or operator. Therefore, the definition should be made based on the content throughout the present invention.

As a result, the technical spirit of the present invention is determined by the claims, and the following examples are one means for efficiently explaining the technical spirit of the present invention to those of ordinary skill in the art to which the present invention belongs. only

1 is a diagram showing the configuration of a system for displaying a baby's emotional state according to an embodiment of the present invention.

In more detail, FIG. 1 shows that transmission data including video data of an infant including a specific baby in the base through the camera device 180 provided at the designated base is transmitted through the designated user's wireless terminal 170's app or IPTV. When provided to (175), baby emotional state information for each designated emotional item for the specific baby is generated based on one or more data acquired through the camera device 180, and is stored in a designated area on the video data of the transmission data. As a diagram showing the configuration of a system for displaying and providing the baby emotional state information for each emotion item generated above, those skilled in the art to which the present invention pertains may refer to and/or modify FIG. Various implementation methods for the configuration of the system (eg, implementation methods in which some components are omitted, subdivided, or combined) may be inferred, but the present invention includes all the inferred implementation methods, and this drawing The technical characteristics are not limited only to the implementation method shown in 1 .

The system of the present invention includes a camera device 180 that is provided at a designated base to photograph a subject and can be connected to a communication network, and an image signal obtained by photographing a specific baby in the base through the camera module of the camera device 180. A function of receiving acquired data including video data encoded in a specified manner and storing it in a specific storage area of a specified storage medium, and transmitting transmission data including video data stored in a specific storage area of the storage medium to the specified user wireless terminal 170 ) of the app or IPTV 175, a function of generating baby emotional state information for each specified emotional item for the specific baby based on one or more data acquired through the camera device 180, the video and an operation server 100 that performs one or more of the functions of displaying the baby emotional state information for each emotion item generated in a designated area on the data. The operation server 100 may be implemented in the form of an independent server, a combination of two or more servers, and at least one or a combination of two or more of software executed through a pre-equipped server, and implementing the operation server 100 The present invention is not limited by the embodiment or the name of the operation server 100 .

The camera device 180 is a generic term for a terminal or device provided at a designated base and equipped with a camera for photographing a subject. Preferably, it is provided in a postpartum care center or a space in which a baby in a home resides, so that a specific baby existing in the space is captured. It may include a terminal or device equipped with a camera for photographing.

According to the embodiment of the present invention, the camera device 180 encodes a video signal obtained through a camera module for photographing a designated subject in a base, a communication module connected to a designated communication network and the camera module in a designated manner. and a control module for performing a procedure for generating acquired data including data and transmitting it to a designated operation server 100 through the communication module.

The camera module includes at least one lens unit that receives an optical signal reflected from a subject, an image sensor unit that detects an optical signal input through the lens unit and converts it into an electrical image signal, and automatically between the lens unit and the image sensor unit. It is made by including an actuator unit (Actuator) for implementing a specified function operation including a focus function or a zoom function. Meanwhile, according to an implementation method, the camera module may further include a filter unit including an infrared filter for photographing a subject even at night between the lens unit and the image sensor unit. Meanwhile, the camera module includes a PCB for providing the image signal generated through the image sensor unit to the control module. Meanwhile, according to the implementation method, the camera module may further include an image signal processing processor (ISP) that processes the image signal according to a specified procedure to improve the quality of the image signal generated through the image sensor unit. .

The communication module is a generic term for modules connecting to a packet-based communication network, and preferably includes at least one or both of a communication module connecting to a communication network using a wire such as a UTP cable and a communication module connecting to a communication network through a wireless LAN. may include Meanwhile, according to the implementation method, the communication module may further (or replace) a communication module for connecting to a communication network through a mobile communication network or a separately designated wireless communication in addition to the wireless LAN, and the present invention is not limited thereto.

The control module may generate video data by encoding an image signal generated through the camera module in a specified manner. Meanwhile, according to an implementation method, the video data may be generated through the camera module and transmitted to the control module. Meanwhile, the video data may further include a time value (or timestamp) of a time point at which the image signal is captured.

The control module may perform a procedure of generating acquired data including the video data and transmitting it to the designated operation server 100 through the communication module. Meanwhile, according to an implementation method, the acquired data may further include audio data in addition to the video data and/or may further include sensing data sensed through a designated sensor module.

According to the implementation method of the present invention, the camera device 180 further includes a sound input unit for receiving a sound signal in the base, in this case, the control module encodes the sound signal obtained through the sound input unit in a specified manner. A procedure of generating acquired data further including audio data and transmitting it to a designated operation server 100 through the communication module may be performed.

The sound input unit is configured to include a microphone that converts air vibration into a sound signal in the form of an electrical signal, and an interface unit for interfacing it with a control module, and the control module converts the sound signal obtained through the sound input unit into a designated and specified method It can be encoded to generate audio data. Meanwhile, when the sound input unit is implemented in the form of a separate module according to the implementation method, the audio data may be generated through the camera module through the sound input unit and transmitted to the control module. Meanwhile, the audio data may further include a time value (or timestamp) of a time point at which the sound signal is input.

When the camera device 180 is provided with the sound input unit to generate audio data obtained by encoding a sound signal obtained through the sound input unit, the control module generates acquired data including the video data and audio data, A procedure of transmitting to the designated operation server 100 through the communication module may be performed.

According to an embodiment of the present invention, the acquired data may be generated in the form of moving picture data (or multimedia data) including the video data and audio data in a specified data format.

According to another embodiment of the present invention, the acquired data includes the video data and the audio data in each designated data format, but the time value (or timestamp) included in the video data and the audio data is used to determine the viewpoint. Each data type that can be synchronized can be created.

According to the embodiment of the present invention, the camera device 180 may further include a sound output unit for outputting a specified sound signal with a specified signal strength. In this case, the control module outputs the sound signal through the sound output unit according to a specified procedure. Audio data corresponding to the sound signal to be output through the sound output unit from the operation server 100 designated through the communication module and/or to generate audio data corresponding to the sound signal to be generated and/or extracted from the designated storage area and/or through the communication module A process of receiving and outputting a sound signal corresponding to the generated/extracted/received audio data through the sound output unit may be performed.

The sound output unit is configured to include a speaker for outputting a sound signal in the form of an electrical signal obtained by decoding audio data and an interface unit for interfacing the same with a control module, wherein the control module generates audio data to be output through the sound output unit, and / or extracts from a specified storage area and / or receives from the specified operation server 100 through the communication module, and receives a sound signal corresponding to the audio data through the sound output unit with a specified signal strength (eg, the camera device 180) ), the sound signal output through the speaker can be output with the signal strength set so that it is delivered at the specified sound level to the baby spaced within or outside the specified distance).

According to the embodiment of the present invention, the camera device 180 may include or be associated with a sensor module for sensing a designated sensing target of the baby through a designated sensor, and the control module may be configured to detect A procedure of generating acquired data further including sensing data encoded by a sensing signal in a specified manner and transmitting it to the specified operation server 100 through the communication module may be performed. Preferably, the baby's designated sensing target sensed through the sensor module may include at least one of a baby's heartbeat, baby's body temperature, baby's ambient humidity, and baby's apnea.

The sensor module may include a wearable sensor module that is worn on a designated part of the baby's body and senses a designated sensing target of the baby, in this case, the control module performs short-distance communication (e.g., Bluetooth, etc.) Through connection with the wearable sensor module, a sensing signal corresponding to a sensed value sensed through the sensor module may be received, or sensing data obtained by encoding the sensing signal in a specified manner through the sensor module may be received.

Meanwhile, the sensor module may include a radar sensor module for sensing a designated sensing target of the baby using a designated electromagnetic wave, in this case, the control module is a sensing signal corresponding to the sensing value sensed through the radar sensor module can be checked and encoded in a specified way to generate sensing data. Meanwhile, the radar sensor module may generate sensing data encoded in a specified manner and provide the sensing data to the control module. In this case, the control module may receive the sensing data from the radar sensor module.

According to the implementation method of the present invention, the sensed data includes a sensed value sensed by a sensing target designated through the sensor module, and preferably a time value (or timestamp) at the point in time when the sensed value is sensed through the sensor module. ) may be further included.

When the camera device 180 is provided with or linked to the sensor module to generate sensing data that senses a designated sensing target of a baby, the control module adds the sensing data to the acquired data and is designated through the communication module. A procedure for transmitting to the operation server 100 may be performed. Preferably, the sensing data may be included in the acquired data in a form included in a specified data format, and a time value (or timestamp) included in the video data and the sensing data may be generated in a data format capable of synchronizing a viewpoint. can

According to the embodiment of the present invention, the camera device 180 further includes a driving module for driving the photographing direction of the camera module within a specified angular range in up/down/left/right directions, wherein the control module includes the camera A process of recognizing an object corresponding to a baby (or a baby's face) or a cradle or a blanket on which the baby is lying may be attempted by reading video data captured through the module. If the object is not recognized, the control module repeats the process of recognizing the object while the shooting direction of the camera module is spaced apart by a specified angular range in a specified direction through the driving module until the entire angular range is reached can do. If the object is recognized through the recognition process, the camera device 180 controls the photographing direction so that the recognized facial region of the baby exists within a designated region on the video data, and movement of the baby's facial region In response to the movement or tracking the facial area of the baby, it is possible to control the photographing direction of the camera module so that it exists within a designated area on the video data. Meanwhile, the process of controlling the shooting direction of the camera module or recognizing the object through the driving module may be executed through the server, and the present invention is not limited thereto.

According to the embodiment of the present invention, the camera device 180 may further include a projector unit for projecting a specified shape or image to the ceiling or wall through a lamp, in which case the control module is configured to control the projector unit according to a specified procedure The lamp may be turned on/off, or the control module may receive control information for turning on/off the lamp of the projector unit from the designated operation server 100 through the communication module, and transmit the control information. Based on this, the lamp of the projector unit may be turned on/off. On the other hand, when a shape or image to be projected by selecting any one of a plurality of films through the projector unit can be selected, the control module selects a film of the projector unit and/or turns on/off the lamp according to a specified procedure. Alternatively, the control module may receive control information for selecting the film of the projector unit and/or turning on/off the lamp from the designated operation server 100 through the communication module, and receiving the control information It is possible to select a film of the projector unit and/or to turn on/off the lamp on the basis. On the other hand, if an electronic shape or an electronic image can be projected through the projector unit, the control module extracts the electronic shape or electronic image stored in a designated storage area according to a specified procedure and processes it as a shape or image to be projected through the projector unit, and/ Alternatively, the lamp may be turned on/off, or the control module receives the electronic shape or electronic image stored from the designated operation server 100 through the communication module and/or control information for turning the lamp on/off and may process the received electronic shape or electronic image as a shape or image to be projected through the projector unit and/or turn on/off the lamp based on the control information.

The user wireless terminal 170 is a generic term for a wireless communication device used by a user to receive transmission data including video data captured through the camera device 180, and is a mobile phone, a smartphone, and a tablet PC used by the user. It may include various wireless communication devices, such as.

According to the embodiment of the present invention, the user wireless terminal 170 registers information specified in the operation server 100, and includes video data captured from the operation server 100 through the camera device 180. An application is installed and executed to receive transmission data. Hereinafter, even if there is no separate mention, the description of the function implemented by using the user wireless terminal 170 as a subject or object is to clarify that the function is implemented through the app.

The operation server 100 is a generic name of servers capable of communicating with the camera device 180 provided at the designated base and communicating with the app of the user wireless terminal 170 that has installed and executed the designated app, and is the camera of the camera device 180 . A function of photographing a specific baby in the base through a module, receiving acquired data including video data encoded in a specified manner, and storing it in a specific storage area of a specified storage medium, and video data stored in a specific storage area of the storage medium It performs at least one of the functions of providing the transmitted data including the transmitted data to the app of the designated user wireless terminal 170 or the IPTV 175, and transmits one or more data acquired through the camera device 180 according to the present invention. At least one function of a function of generating baby emotional state information for each specified emotional item for the specific baby based on the basis, and a function of displaying the baby emotional state information for each of the generated emotional items in a specified area on the video data. carry out

According to an embodiment of the present invention, the transmission data includes, in addition to video data stored in a specific storage area of the storage medium, audio data stored in a specific storage area of the storage medium in association with the video data, or the video data It may further include audio data of video data.

Referring to FIG. 1 , the operation server 100 stores and manages emotional state determination information for determining a baby emotional state value for each specified emotional item for a baby photographed through the camera device 180 in a designated storage area. and an information management unit 105 that

The information management unit 105 generates artificial intelligence-based expression-based emotional state determination information for determining the baby emotional state value for each designated emotional item corresponding to the baby's expression through a designated artificial intelligence-based learning process. It may be stored in the storage area, and/or receive pattern recognition-based expression-based emotional state determination information from a designated operating terminal and stored in the designated storage area.

According to the implementation method of the present invention, when the information management unit 105 generates artificial intelligence-based expression-based emotional state determination information, a designated designated artificial intelligence-based supervised learning process corresponding to the baby's expression Artificial intelligence-based expression-based emotional state determination information for determining the baby emotional state value for each emotion item may be generated and stored in a designated storage area.

According to the implementation method of the present invention, the information management unit 105 performs a specified AI-based learning process, and e(1≤e≤e) corresponding to the specified baby facial region among the specified E (E≥2) baby expressions. E) Determining the baby expressions, and based on the N1 (1≤N1≤N) baby emotion items that can be distinguished through the baby expression among the designated N (N≥2) baby emotion items, the e number of baby expressions and Determine n1 (1≤n1≤N1) baby emotion items having a matching relationship, and determine the baby emotional state value for each n1 emotion item corresponding to the correlation between the identified e baby expressions and n1 baby emotion items It is possible to generate facial expression-based emotional state determination information based on artificial intelligence for the purpose of doing so and store it in a designated storage area.

According to the embodiment of the present invention, the N baby emotion items are 'joy (喜)', 'anger (怒)', 'sad (哀)', 'pleasure (樂)', 'love (愛)', Among the basic emotions such as 'hate' and 'lust', one or more emotions (e.g., 'joy', 'anger', 'sadness', It may include an emotional item corresponding to 'pleasure', etc.), and in addition to the basic emotions, various emotions that the baby may feel (eg, 'irritability', 'loneliness', 'stupid', 'displeasure', 'fear', etc.) ), and/or an emotion in which at least one of the basic emotions is embodied according to the baby's situation (e.g., among 'greed', 'drowsiness' corresponding to sleep desire or 'hunger' corresponding to appetite, etc.) ) may further include an emotion item corresponding to the.

Meanwhile, according to an embodiment of generating the artificial intelligence-based expression-based emotional state determination information, the information management unit 105 may include a specified e(1≤2) among E(E≥2) baby expressions specified from a specified database or operating terminal. The facial area of the baby with the specified correlation with N1 (1≤N1≤N) baby emotion items that can be distinguished through the baby expression among the N (N≥2) specified baby emotion items corresponding to e≤E) baby expressions. Receive big data including a plurality of video information corresponding to the scene including the input, and output variable (Labels) the baby emotional state value for each n1 (1≤n1≤N1) emotion items for the inputted plurality of video information After setting to , according to a specified procedure, geometrical relationships between specified feature points of the baby's facial area included in the input plurality of video information are read and pattern recognized, and n1 emotion items corresponding to the baby's facial expression in the baby's facial area are identified. P(P≥2) pieces of scene pattern characteristic information corresponding to the baby's emotional state value are extracted, the extracted P pieces of scene pattern characteristic information are set as input variables (Feature Vectors), and more than a specified number of Expression-based emotion based on artificial intelligence for learning the input variable and the output variable to determine the baby emotional state value for each n1 emotion item corresponding to the baby expression in the baby face area among a plurality of scenes including the baby face area Status determination information can be generated. Meanwhile, the output variable may further include at least one of e pieces of baby expression information corresponding to the baby facial region included in the video information and n1 pieces of baby emotion item information corresponding to the baby expression of the baby face area. can

Alternatively, the information management unit 105 may be configured to correspond to the specified e(1≤e≤E) baby expressions among E(E≥2) baby expressions specified from a specified database or operating terminal, and N(N≥2) specified babies Among the emotion items, a plurality of video information corresponding to a scene including N1 (1≤N1≤N) baby emotion items that can be distinguished through a baby's expression and a baby's facial area with a specified correlation are read according to a specified procedure and pattern recognition big data including P (P≥2) scene pattern characteristic information extracted as pattern characteristics of baby emotional state values for each n1 (1≤n1≤N1) emotion items corresponding to the baby's facial expression in the baby's facial area After receiving as input variables (Feature Vectors) and setting the baby emotional state values for each n1 emotion item for the video information from which the P scene pattern characteristic information is extracted, as output variables (Labels), through the designated artificial intelligence algorithm An artificial intelligence-based method for determining the baby emotional state value for each n1 emotion item corresponding to the baby expression in the baby's facial area among a plurality of scenes including the baby's facial area by learning more than a specified number of the input and output variables Expression-based emotional state determination information may be generated. Meanwhile, the output variable may further include at least one of e pieces of baby expression information corresponding to the baby facial region included in the video information and n1 pieces of baby emotion item information corresponding to the baby expression of the baby face area. can

According to the implementation method of the present invention, when the information management unit 105 generates the expression-based emotional state determination information based on pattern recognition, the information management unit 105 is F (F≥2) present in the baby's facial region. E(E≥2) baby expression pattern information corresponding to the geometric relationship between the keypoints related to the baby's emotional expression among the ten feature points and N1( Expression-based emotion including matching relationship information between 1≤N1≤N) baby emotion items and quantifying the correlation between the E baby expression pattern information and N1 baby emotion items. Register and save the status determination information in the designated operation D/B.

According to the implementation method of the present invention, the information management unit 105 receives E pieces of baby expression pattern information corresponding to the geometric relationship between the feature points related to the emotional expression of the baby among F feature points existing in the baby's facial area from the designated operation terminal. registration, matching relationship information matching the E pieces of baby expression pattern information with N1 baby emotion items that can be distinguished based on baby expression patterns among the designated N baby emotion items, and the E pieces of baby expression pattern information and N1 Expression-based correlation, which quantifies the correlation between baby emotional items, can be registered and stored in a designated operation D/B by registering numerical relationship information. For example, the information management unit 105 includes the E baby expression pattern information and N1 baby emotion items as row or column titles, and stores the expression-based correlation numerical relationship information into a cell corresponding to each row or column. It is possible to register and store the expression-based emotional state determination information in the form of a table included as the information of the designated operation D/B.

According to an embodiment of the present invention, the E pieces of baby expression pattern information may include geometric relationship information between each feature point corresponding to each baby expression pattern associated with the baby's emotional expression. Preferably, the geometric relationship between the feature points may include at least one of a distance relationship between each feature point, an angular relationship between each feature point, and a combination relationship between a distance relationship between each feature point and an angular relationship. For example, the geometric relationship information between the feature points may include at least one of a distance value between each feature point based on a specified reference feature point distance value, an angle value between each feature point, and a combination of a distance value between each feature point and an angle value. can

Meanwhile, according to another embodiment of the present invention, the E pieces of baby expression pattern information may include range information of a geometric relationship between each feature point corresponding to each baby expression pattern associated with the baby's emotional expression. Preferably, the geometric relationship between the feature points may include at least one of a distance relationship between each feature point, an angular relationship between each feature point, and a combination relationship between a distance relationship between each feature point and an angular relationship. For example, the range information of the geometric relationship between the feature points may include a numerical range of distances between each feature points based on a specified reference numerical distance between feature points, a numerical range of angles between each feature, and a combination of a numerical range of distances between feature points and an angular numerical range It may include more than one numerical range.

According to an embodiment of the present invention, the expression-based correlation numerical relationship information is a correlation value calculated through a relational expression proportional to the matching ratio between the baby expression recognition information that recognizes the baby's expression and the registered baby expression pattern information. may contain information. For example, the expression-based correlation numerical relationship information may be calculated as smaller numerical information as the matching ratio between the baby expression recognition information and the registered baby expression pattern information is smaller, and the larger the matching ratio is, the larger the numerical information is calculated. can be

On the other hand, according to another embodiment of the present invention, the facial expression-based correlation numerical relationship information is the correlation numerical information for each matching ratio section between the baby expression recognition information that recognizes the baby's expression and the registered baby expression pattern information. may include For example, the expression-based correlation numerical relationship information can be calculated as '0' or numerical information of a specified minimum value when the matching ratio between the baby expression recognition information and the registered baby expression pattern information is a preset minimum ratio section, , when the matching ratio is a preset maximum ratio section, it may be calculated as numerical information of a specified maximum value.

On the other hand, according to the implementation method of the present invention, after the expression-based emotional state determination information is registered and stored in the operation D/B through a designated operation terminal, when the designated baby expression recognition information is recognized through the following information recognition unit 130 , the information management unit 105 reads video data received from the camera device 180 and stored in a specific storage area of a designated storage medium after the baby expression recognition information is recognized to recognize an object corresponding to the baby's guardian At the same time, by reading the audio data linked with the video data, the voice related to the baby's emotional recognition among the voices uttered by the baby's guardian to the baby (eg, 'My baby is hungry', 'My baby's poop?', 'My baby is sleepy', etc.) can be performed. If the baby's expression recognition information is recognized and then the parent's voice related to the baby's emotion recognition is recognized, the information management unit 105 patterns the recognized baby's expression recognition information and responds to the baby's emotion recognition related voice at the same time. By repeating the process of identifying a baby emotion item based on voice recognition, and learning a combination of the baby expression pattern information patterned with the recognized baby expression recognition information and the identified voice recognition-based baby emotion item, the expression-based Big data-based accuracy and reliability of emotional state identification information can be continuously improved.

Meanwhile, according to an extended embodiment of the present invention, the information management unit 105 recognizes a baby cry based on the audio data obtained through the sound input unit of the camera device 180 in addition to the expression-based emotional state determination information. A sensor module for registering and storing cry-based emotional state determination information for determining the emotional state of the operation D/B, and/or sensing a designated sensing target related to the emotional expression of the baby in the camera device 180 is provided Sensing-based emotional state determination for determining the baby's emotional state using the sensing data when sensing data obtained by sensing a designated sensing target of the baby is received from the camera device 180 through the sensor module Information can be registered and stored in operation D/B.

Meanwhile, according to another embodiment of the present invention, the information management unit 105 operates the cry-based emotional state determination information and/or sensing-based emotional state determination information instead of (or replaces) the expression-based emotional state determination information. It can be registered and stored in D/B, and the present invention is not limited thereto. Hereinafter, for convenience, the present invention through an embodiment in which expression-based emotional state determination information is used as basic emotional state determination information and the cry-based emotional state determination information and/or sensing-based emotional state determination information is used as secondary emotional state determination information will be described, but the technical features of the present invention are not limited thereto, and the cry-based emotional state determination information and/or sensing-based emotional state determination information is also basic according to the embodiments of the present invention and the description of the claims. It is clearly stated that it can be used as emotional state identification information.

According to an extension/other embodiment of the present invention, the information management unit 105 is artificial intelligence for determining the baby emotional state value for each designated emotional item corresponding to the baby cry through a designated artificial intelligence-based learning process. Cry-based emotional state determination information may be generated and stored in a designated storage area, and/or pattern recognition-based cry-based emotional state determination information may be registered from a designated operating terminal and stored in a designated storage area.

According to the implementation method of the present invention, when the information management unit 105 generates artificial intelligence-based cry-based emotional state determination information, it responds to a baby cry through a designated artificial intelligence-based supervised learning process. Artificial intelligence-based cry-based emotional state determination information for determining the baby emotional state value for each specified emotional item can be generated and stored in a designated storage area.

According to the implementation method of the present invention, the information management unit 105 performs a designated AI-based learning process, and c(1≤c≤C) baby cries among the designated C (C≥2) baby cries. The sound is determined, and the matching relationship with the c baby cry sounds determined above based on N2 (1≤N2≤N) baby emotion items that can be distinguished through baby cry among the specified N (N≥2) baby emotion items To determine n2 (1≤n2≤N2) baby emotion items with For artificial intelligence-based cry-based emotional state determination information can be created and stored in a designated storage area.

On the other hand, according to an embodiment of generating the artificial intelligence-based cry-based emotional state determination information, the information management unit 105 is a designated c (C ≥ 2) among C (C≥2) baby cries designated from a designated database or operating terminal. Corresponding to 1≤c≤C) baby crying sounds, it has a specified correlation with N2 (1≤N2≤N) baby emotion items that can be distinguished through baby cry among the designated N(N≥2) baby emotion items. Receives big data including a plurality of audio information corresponding to a baby's cry, and sets the baby emotional state value for each n2 (1≤n2≤N2) emotion items for the inputted plurality of audio information as output variables (Labels) After setting, according to a specified procedure, the plurality of input audio information is read and pattern recognized, and P (P≥2) scene pattern characteristics corresponding to the baby emotional state value for each n2 emotional items corresponding to the baby crying sound Extract information, set the extracted P scene pattern characteristic information as input variables (Feature Vectors), and learn more than a specified number of input and output variables through a specified artificial intelligence algorithm to learn baby cries included in the specified audio data It is possible to generate artificial intelligence-based cry-based emotional state determination information for determining the baby emotional state value for each n2 emotional items corresponding to the sound. Meanwhile, the output variable may further include at least one of c pieces of baby cry information corresponding to the baby crying and n2 pieces of baby emotion item information corresponding to the baby cry included in the audio information.

Alternatively, the information management unit 105 corresponds to the specified c (1≤c≤C) baby cries among the C (C≥2) specified baby cries from the specified database or operating terminal, and the designated N (N≥2) Among the baby emotion items, N2 (1≤N2≤N) baby emotion items that can be distinguished through the cry of a baby and a plurality of audio information corresponding to a baby crying with a specified correlation are read according to a specified procedure and pattern recognized Big data containing P(P≥2) scene pattern characteristic information extracted as pattern characteristics of baby emotional state values for each n2 (1≤n2≤N2) emotion items corresponding to the baby’s cry is used as input variables (Feature Vectors). ) and set the baby emotional state values for each n2 emotion items for the audio information from which the P scene pattern characteristic information is extracted as output variables (Labels), and then input more than a specified number through a specified artificial intelligence algorithm. By learning the variable and the output variable, it is possible to generate artificial intelligence-based cry-based emotional state determination information for determining the baby emotional state value for each n2 emotional items corresponding to the baby cry included in the specified audio data. Meanwhile, the output variable may further include at least one of c pieces of baby cry information corresponding to the baby crying and n2 pieces of baby emotion item information corresponding to the baby cry included in the audio information.

According to the implementation method of the present invention, when the information management unit 105 generates pattern recognition-based cry-based emotional state determination information, C (C≥2) pieces of information related to the baby's emotional expression among the baby's cry patterns It includes matching relationship information between baby cry pattern information and N2 (1≤N2≤N) baby emotion items that can be distinguished based on a baby cry pattern among N designated baby emotion items, and the C baby cry pattern information It is possible to register and store cry-based emotional state identification information including numerical relationship information on the cry-based correlation level, which quantifies the correlation between the and N2 baby emotion items, in the operation D/B.

According to the implementation method of the present invention, the information management unit 105 registers C pieces of baby cry pattern information related to the emotional expression of a baby among baby cry patterns from a designated operating terminal, and the C pieces of baby cry pattern information to register matching relationship information matching N2 baby emotion items that can be distinguished based on baby cry patterns among the N designated baby emotion items, and calculate the correlation between the C baby cry pattern information and N2 baby emotion items. Numericalized cry-based correlation can also be registered and stored in a designated operation D/B by receiving numerical relationship information. For example, the information management unit 105 includes C baby cry pattern information and N2 baby emotion items as row or column titles, and converts the cry based correlation numerical relationship information into cell information corresponding to each row or column. It is possible to register and store cry-based emotional state determination information in the operation D/B in the form of a table including

According to an embodiment of the present invention, the C pieces of baby cry pattern information may include cry feature information constituting each baby cry pattern related to the emotional expression of the baby. Preferably, the cry characteristics include a frequency band characteristic, an arrangement relational characteristic of a frequency band characteristic, a periodicity characteristic of a frequency band characteristic, a sound intensity characteristic, an arrangement relational characteristic of a sound intensity characteristic, a periodicity characteristic of a sound intensity characteristic, a frequency band characteristic, and It may include one or more characteristics of a combination characteristic of a sound intensity characteristic, a characteristic of an arrangement relationship between a frequency band characteristic and a sound intensity characteristic, and a periodicity characteristic of a frequency band characteristic and a sound intensity characteristic.

Meanwhile, according to another embodiment of the present invention, the C pieces of baby cry pattern information may include information on the range of cry characteristics constituting each baby cry pattern related to the emotional expression of the baby. Preferably, the cry characteristics include a frequency band characteristic, an arrangement relational characteristic of a frequency band characteristic, a periodicity characteristic of a frequency band characteristic, a sound intensity characteristic, an arrangement relational characteristic of a sound intensity characteristic, a periodicity characteristic of a sound intensity characteristic, a frequency band characteristic, and It may include one or more characteristics of a combination characteristic of a sound intensity characteristic, a characteristic of an arrangement relationship between a frequency band characteristic and a sound intensity characteristic, and a periodicity characteristic of a frequency band characteristic and a sound intensity characteristic. For example, the range information of the crying sound feature may include an allowable range of a frequency band for each feature and/or an allowable range of a sound intensity for each feature.

According to an embodiment of the present invention, the cries-based correlation numerical relationship information is calculated through a relational expression proportional to the matching ratio between the baby cry recognition information that recognizes the baby's cry and the registered baby cry pattern information. The correlation may also include numerical information. For example, the cry-based correlation numerical relationship information may be calculated as smaller numerical information as the matching ratio between the baby cry recognition information and the registered baby cry pattern information is smaller, and the larger the matching ratio is, the larger the numerical relationship information. information can be calculated.

On the other hand, according to another embodiment of the present invention, the correlation information based on the cry is the correlation between each matching ratio section between the baby cry recognition information that recognizes the baby's cry and the registered baby cry pattern information. It may also include numerical information. For example, the cry-based correlation numerical relationship information is calculated as '0' or numerical information of a specified minimum value when the matching ratio between the baby cry recognition information and the registered baby cry pattern information is a preset minimum ratio section may be, and when the matching ratio is a preset maximum ratio section, it may be calculated as numerical information of a designated maximum value.

On the other hand, according to the implementation method of the present invention, after the cry-based emotional state determination information is registered and stored in the operation D/B through a designated operation terminal, the designated baby cry recognition information is recognized through the following information recognition unit 130 In this case, the information management unit 105 reads the video data received from the camera device 180 and stored in a specific storage area of a designated storage medium after the baby cry recognition information is recognized to correspond to the baby's guardian. At the same time as an object is recognized, audio data associated with the video data is read, and among the voices uttered by the baby's guardian to the baby, voices related to the baby's emotional recognition (eg, 'My baby is hungry', 'Our baby poop? ?', 'My baby is sleepy', etc.) can be performed. If the voice related to the baby emotion recognition of the guardian is recognized after the baby cry recognition information is recognized, the information management unit 105 patterns the recognized baby cry recognition information and the voice related to the baby emotion recognition Repeat the process of identifying the baby emotion item corresponding to the voice recognition and learning the combination of the baby cry pattern information obtained by patterning the recognized baby cry recognition information and the identified voice recognition-based baby emotion item By doing so, it is possible to continuously improve the big data-based accuracy and reliability of the cry-based emotional state determination information.

Meanwhile, according to an extension/other embodiment of the present invention, the information management unit 105 senses a designated sensing target associated with the emotional expression of a baby among sensing data sensed through a sensor module provided in or linked to the camera device 180 . Matching relationship between S (S≥2) sensing value pattern information for data and N3 (1≤N3≤N) baby emotion items that can be distinguished based on the sensing value pattern of the specified sensing target among the N specified baby emotion items It is possible to register and store sensing-based emotional state determination information including sensing-based correlation numerical relationship information, which includes information and quantifies the correlation between the S sensed value pattern information and N3 baby emotion items, in the operation D/B.

According to the embodiment of the present invention, the sensor module is a wearable sensor module that is worn on a baby's body and senses a designated sensing target, and a radar sensor module that senses a designated sensing target of the baby using a designated electromagnetic wave. may include at least one sensor module among, and the designated sensing target may include at least one of a baby's heartbeat, a baby's body temperature, a baby's ambient humidity, and whether the baby is apnea.

According to the embodiment of the present invention, the information management unit 105 senses a designated sensing target associated with the emotional expression of a baby among sensing data sensed through a sensor module provided in or linked to the camera device 180 from a designated operating terminal. Matching relationship information between N3 baby emotion items that can register S sensing value pattern information for data, and distinguish the S sensing value pattern information based on the sensing value pattern of a specified sensing target among the N specified baby emotion items can be registered, the sensing-based correlation numerical relationship information obtained by quantifying the correlation between the S sensed value pattern information and the N3 baby emotion items can be registered and stored in a designated operation D/B. For example, the information management unit 105 includes the S sensed value pattern information and N3 baby emotion items as row or column titles, and includes sensing-based correlation numerical relationship information as cell information corresponding to each row or column. It is possible to register and store sensing-based emotional state determination information in the operation D/B in the form of a table.

According to an embodiment of the present invention, the S sensing value pattern information may include sensing value characteristic information constituting each sensing value pattern for sensing data of a designated sensing target related to the emotional expression of a baby. Preferably, the sensed value characteristic may include at least one or a combination of two or more of a change rule characteristic of a sensed value, a characteristic of a change amount of a sensed value, a characteristic of a change rate of a sensed value, and a characteristic of a periodicity of a change of a sensed value.

Meanwhile, according to another embodiment of the present invention, the S sensed value pattern information includes range information of sensing value features constituting each sensing value pattern for sensing data of a designated sensing target related to the emotional expression of the baby. can Preferably, the sensed value characteristic may include at least one or a combination of two or more of a change rule characteristic of a sensed value, a characteristic of a change amount of a sensed value, a characteristic of a change rate of a sensed value, and a characteristic of a periodicity of a change of a sensed value. For example, the range information of the sensing value feature may include an allowable range of a change rule for each feature, an allowable range of a change amount of a sensing value for each feature, and an allowable range of a sensing value change rate for each feature.

According to an embodiment of the present invention, the sensing-based correlation numerical relationship information is calculated through a relational expression proportional to a matching ratio between sensing value recognition information recognized through sensing data of a specified sensing target and pre-registered sensing value pattern information The correlation may also include numerical information. For example, the sensing-based correlation numerical relationship information may be calculated as small numerical information as the matching ratio between the sensing value recognition information and the previously registered sensing value pattern information is smaller, and the larger the matching ratio is, the larger the numerical information is calculated. can be

Meanwhile, according to another embodiment of the present invention, the sensing-based correlation numerical relationship information is a correlation for each matching ratio section between sensing value recognition information recognized through sensing data of a specified sensing target and pre-registered sensing value pattern information. It may also include numerical information. For example, the sensing-based correlation numerical relationship information may be calculated as '0' or numerical information of a specified minimum value when the matching ratio between the sensing value recognition information and the registered sensing value pattern information is a preset minimum ratio section, , when the matching ratio is a preset maximum ratio section, it may be calculated as numerical information of a specified maximum value.

On the other hand, according to the implementation method of the present invention, after the sensing-based emotional state determination information is registered and stored in the operation D/B through a designated operation terminal, when the specified sensing value recognition information is recognized through the following information recognition unit 130 , the information management unit 105 reads video data received from the camera device 180 and stored in a specific storage area of a designated storage medium after the sensing value recognition information is recognized to recognize an object corresponding to the baby's guardian At the same time, by reading the audio data linked with the video data, the voice related to the baby's emotional recognition among the voices uttered by the baby's guardian to the baby (eg, 'My baby is hungry', 'My baby poop? A procedure for recognizing 'My baby is sleepy', etc.) can be performed. If, after the sensing value recognition information is recognized, the voice related to the baby emotion recognition of the guardian is recognized, the information management unit 105 patterns the recognized sensing value recognition information and responds to the voice related to the baby emotion recognition at the same time The sensing by repeating the process of identifying a baby emotion item based on voice recognition, and learning a combination of the baby cry pattern information obtained by patterning the recognized sensing value recognition information and the identified voice recognition-based baby emotion item. It is possible to continuously improve the accuracy and reliability of big data based emotional state identification information.

Referring to Figure 1, the operation server 100, a target to be provided with transmission data including video data of a specific baby in the base through the camera device 180 provided at the designated base, and the camera device ( 180) and includes an information registration unit 110 that receives the mapping information and stores it in a designated management D/B.

A user who purchases or receives the camera device 180 enters subscription application information through a user terminal or a paper-type subscription application (eg, an application for combining the service using the camera device 180 and the IPTV 175 ); When an application for combining the service using the camera device 180 and the user wireless terminal 170, etc.) is prepared, the information registration unit 110 corresponds to the subscription application information or the subscription application through the user terminal or a designated registration terminal. User information and camera device 180 information uniquely identifying the camera device 180 provided/supplied to the user may be registered and stored in the management D/B. Meanwhile, the information registration unit 110 provides ID/PW information for providing transmission data including video data captured through the camera device 180 to the IPTV 175 used by the user through the user terminal or a designated registration terminal. can be registered and stored in the management D/B in association with the user information and/or camera device 180 information. On the other hand, according to the embodiment of the present invention, the ID/PW information is to provide transmission data including video data photographed through the camera device 180 to a wired terminal or a wireless terminal 170 used by the user through the web. may be used, and the present invention may also include such an embodiment within the scope of its rights.

On the other hand, when the user installs the designated app in the user wireless terminal 170 used by the user and operates for the first time (or before subscription), the information registration unit 110 provides information about the user from the app of the user wireless terminal 170. User information and camera device 180 information uniquely identifying the camera device 180 provided/supplied to the user may be registered and stored in the management D/B. On the other hand, the information registration unit 110 is unique to the app of the user wireless terminal 170 in order to provide transmission data including video data captured through the camera device 180 as the app of the user wireless terminal 170 . It is possible to generate unique identification information to identify or receive from the app of the user wireless terminal 170 and store it in the management D/B in connection with the user information and/or camera device 180 information, and generate the unique identification information In one case, it may be provided as an app of the user wireless terminal 170 and stored in a designated storage area. Meanwhile, the information registration unit 110 provides an ID/PW for providing transmission data including video data captured through the camera device 180 from the app of the user wireless terminal 170 to the IPTV 175 used by the user. Information may be registered and stored in the management D/B in association with the user information and/or camera device 180 information. On the other hand, according to the embodiment of the present invention, the ID/PW information is to provide transmission data including video data photographed through the camera device 180 to a wired terminal or a wireless terminal 170 used by the user through the web. may be used, and the present invention may also include such an embodiment within the scope of its rights.

On the other hand, according to the embodiment of the present invention, the app of the user wireless terminal 170 is unique information (eg, of the other wireless terminal 170 to be provided with transmission data including the video data captured through the camera device 180) , mobile phone number, etc.) can be input/registered, and in this case, the information registration unit 110 stores the unique information of the other wireless terminal 170 in the management D/B, and the unique information of the other wireless terminal 170 is stored in the management D/B. Inducing the installation of the designated app to the other wireless terminal 170 through the information and/or checking the installed and executed app in the other wireless terminal 170, and uniquely identifying the app of the other wireless terminal 170 By generating unique identification information or storing it in the management D/B in conjunction with the registered camera device 180 information and being provided from the app of the other wireless terminal 170, in addition to the app of the user wireless terminal 170, the It is possible to process so that transmission data including video data captured through the camera device 180 can be provided through an app of another wireless terminal 170 that is selected through an app of the user wireless terminal 170 .

Referring to FIG. 1 , the operation server 100 is obtained through a communication management unit 115 that manages a communication connection with a camera device 180 provided at a designated base, and a camera module of the camera device 180 , a data receiving unit 120 for receiving acquired data including video data encoded in a specified manner; and a data storage unit 125 for storing and managing the received acquired data in a specific storage area of a specified storage medium.

When power is input to the camera device 180 provided at the designated base, the control module of the camera device 180 exchanges one or more designated information with the designated operation server 100 through the communications module according to a designated procedure to establish a communication connection. request, and the communication management unit 115 authenticates the validity of the camera device 180 based on the exchange of the specified information and establishes a communication connection with the camera device 180 .

When communication is established with the camera device 180 , the communication management unit 115 dynamically generates disposable address information for receiving the acquired data acquired through the camera device 180 and provides it to the camera device 180 . In this case, the control module of the camera device 180 receives the one-time address information, sets the one-time address information as a destination address to which the acquired data is to be transmitted, and then sets the acquired acquired data to the one-time address. can be sent to the destination address Here, the one-time address information may include address information composed of dynamically generated parameter values so that the address domain information is the same but the parameter values after the domain are not duplicated.

According to the embodiment of the present invention, the communication management unit 115 may periodically dynamically generate new one-time address information for the camera device 180 and provide it to the camera device 180. In this case, the camera device The control module of 180 receives the new one-time address information, interrupts the process of transmitting the acquired data to the destination address corresponding to the previous one-time address, and at the same time transmits the previous transmission information of transmitting the acquired data to the previous one-time address. After confirming, and setting the new one-time address information as a new destination address to which the acquired data is to be subsequently transmitted, the acquired data transmitted to the previous one-time address based on the previous transmission information is transferred to the new destination address corresponding to the new one-time address It can then be transmitted, and the above process can be repeated periodically. Hereinafter, it is specified that the camera device 180 can transmit the acquired data to the one-time address even if there is no separate mention. Meanwhile, according to another embodiment of the present invention, the process of generating or applying the one-time address information of the communication management unit 115 may be omitted, and the present invention is not limited thereto.

According to the embodiment of the present invention, when the control module of the camera device 180 transmits acquired data including video data that is acquired through the camera module and encoded in a specified manner, the data receiver 120 is Receives acquisition data including video data acquired and encoded in a specified manner through can manage

On the other hand, according to the implementation method of the present invention, the control module of the camera device 180 is obtained through the camera module and encoded in a specified manner video data and audio data obtained through the sound input unit and encoded in the specified manner. When the acquired data is transmitted, the data receiving unit 120 receives acquired data including video data obtained through the camera module and encoded in a specified manner and audio data obtained through the sound input unit and encoded in a specified manner, and , the data storage unit 125 associates and stores the video data and audio data of the received acquired data in a specific storage area of a designated storage medium and manages it for a certain period of time.

According to the embodiment of the present invention, the control module of the camera device 180 may transmit acquired data in the form of moving picture data including the video data and audio data, and in this case, the data receiving unit 120 is the camera device The acquired data in the form of moving picture data is received from 180, and the data storage unit 125 may store and manage the received moving picture data in a specific storage area of a designated storage medium.

On the other hand, according to the embodiment of the present invention, the control module of the camera device 180 may transmit the acquired data including the video data and the audio data in the form of respective data, in this case, the data receiving unit 120 Receives acquired data including the video data and audio data from the camera device 180, respectively, and the data storage unit 125 associates the video data and audio data respectively received from the camera device 180, It can be stored and managed in a specific storage area of a designated storage medium. Preferably, the data storage unit 125 synchronizes the times of the video data and the audio data based on the time values (or timestamps) of the video data and the audio data, and stores them in a specific storage area of a designated storage medium. .

On the other hand, according to the embodiment of the present invention, the control module of the camera device 180 is obtained through the camera module and encoded in a specified manner, in addition to video data and/or audio data obtained through the sound input unit and encoded in a specified manner. , when the acquired data that is sensed through the specified sensor module and further includes sensing data encoded in a specified manner is transmitted, the data receiving unit 120 receives the acquired data including the sensing data, and the data storage unit 125 ) can be managed for a certain period of time by linking and storing the video data and/or audio data of the received acquired data and the sensing data in a specific storage area of a designated storage medium. Preferably, the data storage unit 125 is configured to store the video data and/or audio data based on a time value (or timestamp) of the video data and/or audio data and a time value (or timestamp) of the sensing data. The time of the sensor may be synchronized with the time of the sensed data and stored in a specific storage area of a designated storage medium.

According to the first baby emotional state generation embodiment of the present invention, the operation server 100 includes video data stored in a specific storage area of the storage medium as an app or IPTV 175 of a designated user wireless terminal 170. Baby emotion for each emotion item according to a specified procedure based on the video data included in the transmission data and/or data associated with the video data (eg, audio data and/or sensing data, etc.) at the time of providing the transmission data By generating the status information, it is possible to process it to be displayed in real time in a designated area on the video data of the transmission data provided to the app of the user wireless terminal 170 or the IPTV 175 .

Meanwhile, according to the second baby emotional state generation embodiment of the present invention, the operation server 100 stores the acquired data received from the camera device 180 in a specific storage area of a designated storage medium, Based on the acquired data stored in the storage area, baby emotional state information for each emotion item is generated according to a specified procedure and stored in association with the acquired data. In this case, the video data stored in a specific storage area of the storage medium can be stored. When provided to the app or IPTV 175 of the designated user's wireless terminal 170, including the baby emotional state information for each emotion item associated with the video data of the transmitted data, to be displayed in a designated area on the video data of the transmitted data can be processed

Referring to Figure 1, the operation server 100, when determining the baby emotional state value for each specified emotional item of the baby through the artificial intelligence-based emotional state determination information, through the artificial intelligence-based emotional state determination information an emotion state determination unit 140 that determines a baby emotional state value for each specified emotion item by performing a procedure of reading video data and/or audio data obtained through the camera device 180, and the calculated one or more babies and an information generating unit 145 for generating baby emotional state information for each designated emotional item by using the emotional state value.

When the information management unit 105 generates and stores and manages artificial intelligence-based expression-based emotional state determination information, the emotional state determination unit 140 provides a video designated through the artificial intelligence-based expression-based emotional state determination information. Based on the N1 (1≤N1≤N) baby emotion items that can be distinguished through the baby expression among the specified N (N≥2) baby emotion items by performing a data reading procedure, the video data corresponding to the baby expression The baby emotional state value for each n1 (1≤n1≤N1) emotion items is determined.

According to the implementation method of the present invention, the emotional state determination unit 140 performs a procedure of reading specified video data through an artificial intelligence algorithm corresponding to the expression-based emotional state determination information based on the artificial intelligence, and thus designated E babies. Among the facial expressions, e baby facial expressions corresponding to the baby facial region on the video data are determined, and n1 baby emotion items having a matching relationship with the e determined baby expression items among the N1 baby emotion items that can be distinguished through the baby expression are selected. and determine the baby emotional state value for each n1 emotion items corresponding to the correlation between the identified e baby expressions and the n1 baby emotion items.

The information generating unit 145 generates baby emotional state information for each of n (1≤n≤N) designated emotion items that can be displayed on video data by using the determined baby emotional state values for each of the n1 emotion items. Meanwhile, when the baby emotional state information for each of the n emotion items is generated after the acquired data is stored in a specific storage area of the storage medium according to the second baby emotional state generation embodiment of the present invention, the information generating unit 145 may store the generated baby emotional state information for each of the n emotion items in a specific storage area of the storage medium in association with the video data.

Referring to Figure 1, when the operation server 100 determines the baby's emotional state value for each designated emotional item through the pattern recognition-based emotional state determination information, it is acquired through the camera device 180 of the designated base. The information recognition unit 130 for recognizing recognition information of a designated recognition target for determining the baby's emotion by reading at least one of the acquired data, and the recognized recognition information and An information determining unit 135 that identifies pattern information of a designated recognition target matched with a set standard similarity or higher and determines one or more baby emotion items having a matching relationship with the identified pattern information, and a matching ratio between the recognition information and the pattern information An emotional state determination unit that reads the correlation numerical relationship between the pattern information and the baby emotion items based on , and determines the baby emotional state value for each emotion item that digitizes the baby emotional state for one or more emotion items corresponding to the recognition information ( 140), and an information generating unit 145 for generating baby emotional state information for each designated emotional item by using the calculated one or more baby emotional state values.

The information recognition unit 130 reads the designated video data and recognizes f (1≤f≤F) feature points for a specific baby's facial region within the base, and among the recognized f feature points, associated with the baby's emotional expression Recognize i (i≥1) pieces of baby expression recognition information corresponding to geometric relationships between feature points.

According to the embodiment of the present invention, the designated video data may include at least one of video data periodically selected for each designated playback time unit and video data periodically selected for each designated number of playback frames.

In the case of the first baby emotional state generation embodiment of the present invention, the information recognition unit 130 is an app of the user wireless terminal 170 corresponding to the unique identification information stored in the management D/B or stored in the management D/B. It is checked whether transmission data including video data stored in a specific storage area of the storage medium is provided to the IPTV 175 identified/authenticated through ID/PW information. If the transmission data is provided by the app of the user wireless terminal 170 or the IPTV 175, the information recognition unit 130 reads the designated video data among the video data of the transmission data and reads a specific baby in the base. While recognizing the f feature points for the facial region of , i baby expression recognition information corresponding to the geometric relationship between the feature points related to the emotional expression of the baby among the recognized f feature points may be recognized.

Meanwhile, in the case of the second baby emotional state generation embodiment of the present invention, the information recognition unit 130 stores the acquired data received from the camera device 180 in a specific storage area of the storage medium and then stores the specific storage medium in the storage medium. Reading the designated video data among the video data stored in the area, while recognizing the f feature points for the facial region of a specific baby in the base, i number corresponding to the geometric relationship between the feature points related to the emotional expression of the baby among the recognized f feature points It can recognize baby's facial expression recognition information.

When i baby facial expression recognition information is recognized through the information recognition unit 130, the information determining unit 135 is configured to recognize the i baby facial expressions based on the facial expression-based emotional state determination information stored in the operation D/B. Check the facial expression pattern information of e (1≤e≤E) babies matching the recognition information and the preset standard similarity. If e pieces of baby expression pattern information matched with the recognized i baby expression recognition information and a preset level of similarity or more are identified, the information determining unit 135 has a matching relationship with the identified e baby expression pattern information. n1 (1≤n1≤N1) baby emotion items are identified.

When the n1 baby emotion items corresponding to the i baby expression recognition information are determined through the information determining unit 135, the emotional state determining unit 140 determines the i baby expression recognition information and the e babies. Based on the matching ratio between the expression pattern information, the correlation numerical relationship between the e baby expression pattern information and n1 baby emotion items is read, and the baby emotional state for each n1 emotion items corresponding to the recognized i baby expression recognition information is read. Determines the baby emotional state value for each n1 emotional item that digitizes , and the information generating unit 145 uses the determined baby emotional state value for each n1 emotional item to display a designated n (1≤1) that can be displayed on the video data. Generates baby emotional state information for each n≤N) emotional items.

Meanwhile, when the baby emotional state information for each of the n emotion items is generated after the acquired data is stored in a specific storage area of the storage medium according to the second baby emotional state generation embodiment of the present invention, the information generating unit 145 may store the generated baby emotional state information for each of the n emotion items in a specific storage area of the storage medium in association with the video data.

On the other hand, when the information management unit 105 generates and stores and manages artificial intelligence-based cry-based emotional state determination information, the emotional state determination unit 140 collects the artificial intelligence-based cry-based emotional state determination information. Based on the N2 (1≤N2≤N) baby emotion items that can be distinguished through the cry of a baby among the N (N≥2) specified baby emotion items by performing a procedure of reading the specified audio data through the It is possible to determine the baby emotional state value for each n2 (1≤n2≤N2) emotion items corresponding to the cry.

According to the implementation method of the present invention, the emotional state determination unit 140 performs a procedure of reading the specified audio data through an artificial intelligence algorithm corresponding to the artificial intelligence-based cry-based emotional state determination information, and thus the designated C pieces Among the baby cries, c baby cries corresponding to the baby cries on the audio data are determined, and n 2 baby cries have a matching relationship with the determined c baby cries among the N2 baby emotion items that can be distinguished through the baby cries. It is possible to determine the baby emotion items, and determine the baby emotional state values for each n2 emotion items corresponding to the correlation between the identified c baby cry sounds and the n2 baby emotion items.

On the other hand, when pattern recognition-based cry-based emotional state determination information is stored in the operation D/B through the information management unit 105, the information recognition unit 130 provides audio data of a predetermined section associated with the video data. It is possible to recognize j (j≥1) pieces of baby cry recognition information for the specific baby by reading .

When j baby cry recognition information is recognized through the information recognizing unit 130, the information determining unit 135 is configured to recognize the j recognized cries based on the crying sound-based emotional state determination information stored in the operation D/B. Check the c (1≤c≤C) baby cry pattern information that matches the baby cry recognition information and the preset standard similarity level or more. If the recognized j pieces of baby cry recognition information and the c pieces of baby cry pattern information matching the preset reference similarity or more are identified, the information determining unit 135 matches the identified c baby cry pattern information Identify n2 (1≤n2≤N2) baby emotion items that have a relationship.

When the n2 baby emotion items corresponding to the j baby cry recognition information are determined through the information determining unit 135, the emotional state determining unit 140 determines the recognized j number of baby cry recognition information and the c Based on the matching ratio between the baby cry pattern information, n2 emotions corresponding to the recognized j pieces of baby cry recognition information by reading the correlation numerical relationship between the c baby cry pattern information and n2 baby emotion items The baby emotional state values for each n2 emotional items obtained by digitizing the baby emotional states for each item are determined, and the information generating unit 145 uses (additionally) the determined n baby emotional state values for each of the n2 emotional items. It is possible to generate baby emotional state information for each emotion item.

According to the embodiment of the present invention, when the baby emotional state values for each of the n1 emotion items have been previously calculated, the information generating unit 145 is configured to include the calculated baby emotional state values for each of the n1 emotion items and the calculated n2 emotions. Baby emotional state information for each of the n designated emotion items may be generated by combining the baby emotional state values for each emotion item. Preferably, the baby emotional state information for each n emotion items is a union relationship of 'baby emotional state values for n1 emotion items ∩ baby emotional state values for n2 emotion items', or a subset relationship of the union. .

On the other hand, when generating baby emotional state information for each n emotion items by combining the baby emotional state value for each n1 emotion item and the baby emotional state value for each n2 emotion item, the information generating unit 145 generates the n1 emotion items Among the n2 emotion items, the specified statistical calculation for each baby's emotional state value (for example, the calculation of average value) or the correlation between the emotion item and the expression or cry is assigned to the designated recognition target for overlapping emotion items. Through weighted statistical calculation, baby emotional state information of the overlapping emotion items may be generated.

Meanwhile, the sensing-based emotional state determination information is stored in the operation D/B through the information management unit 105, and video data and/or audio data are provided in a specific storage area of the storage medium and the camera device 180 or when the sensed data sensed through the associated sensor module is stored in association, the information recognition unit 130 reads the sensing data associated with the video data ??/or audio data to read the sensing data associated with the emotional expression of the specific baby. It is possible to recognize k (k≥1) pieces of sensing value recognition information for the sensing data of the sensing target.

On the other hand, when the k sensed value recognition information is recognized through the information recognition unit 130, the information determining unit 135 performs the k sensed sensing values based on the sensing-based emotional state determination information stored in the operation D/B. Check the value recognition information and s (1≤s≤S) sensed value pattern information that matches the preset reference similarity or more. If the recognized k sensed value recognition information and the s sensed value pattern information matched with more than a preset reference similarity are matched, the information determining unit 135 has a matching relationship with the identified s sensed value pattern information n3 (1≤n3≤N3) baby emotion items are identified.

When the n3 baby emotion items corresponding to the k sensed value recognition information are determined through the information determining unit 135 , the emotional state determining unit 140 sets the k sensed value recognition information and the s sensing values. Based on the matching ratio between the value pattern information, the correlation numerical relationship between the s sensed value pattern information and the n3 baby emotion items is read, and the baby emotional state for each n3 emotion items corresponding to the recognized k sensed value recognition information. Determines the baby emotional state value for each n3 emotion items digitized, and the information generating unit 145 uses (additionally) the baby emotional state value for each of the n3 emotion items determined to determine the baby emotion for each of the n emotion items. You can create status information.

According to the implementation method of the present invention, when the baby emotional state values for each of the n1 emotion items have been previously calculated, the information generator 145 is configured to generate the baby emotional state values for each of the calculated n1 emotion items and the k sensed values. Baby emotional state information for each of the n designated emotion items may be generated by combining the baby emotional state values for each n3 emotion items calculated based on the recognition information. Preferably, the baby emotional state information for each n emotion items is a union relationship of 'baby emotional state values for n1 emotion items ∩ baby emotional state values for n3 emotion items', or a subset relationship of the union. .

On the other hand, when generating baby emotional state information for each n emotion items by combining the baby emotional state value for each n1 emotion item and the baby emotional state value for each n3 emotion item, the information generating unit 145 generates the n1 emotion items and n3 emotion items, designated according to a specified statistical calculation for each baby’s emotional state value (for example, an average value calculation calculation) or a correlation between the emotion item and the facial expression or the sensing value of the specified sensing target for overlapping emotion items It is possible to generate baby emotional state information of the overlapping emotion item through statistical calculation in which a specified weight is given to a recognition target.

Meanwhile, according to the embodiment of the present invention, when the baby emotional state value for each n1 emotion item and the baby emotional state value for each n2 emotion items for the j baby cry recognition information are pre-calculated, the information generating unit 145 ) is the calculated baby emotional state value for each n1 emotion item, the calculated baby emotional state value for each n2 emotion item, and the baby emotional state value for each n3 emotion item calculated based on the k sensed value recognition information By combining, it is possible to generate baby emotional state information for each of the n specified emotional items. Preferably, the baby emotional state information for each of the n emotion items is a union relationship of 'Baby emotional state value for each n1 emotion item ∩ Baby emotional state value for each n2 emotion item ∩ Baby emotional state value for each n3 emotion item', Alternatively, it may be a subset relation of the union.

Meanwhile, when generating baby emotional state information for each n emotion item by combining the baby emotional state value for each n1 emotion item, the baby emotional state value for each n2 emotion item, and the baby emotional state value for each n3 emotion item, the information The generating unit 145 is configured to perform a specified statistical calculation (eg, average value calculation) for each baby's emotional state value with respect to the overlapping emotion items among the n1 emotion items, n2 emotion items, and n3 emotion items or corresponding emotion items. Baby emotional state information of the overlapping emotion items may be generated through statistical calculation in which a specified weight is given to a specified recognition target according to a correlation between an expression and a cry or a sensing value of a specified sensing target.

Referring to Figure 1, the operation server 100, the information application unit 150 for displaying the baby emotional state information for each of the generated n emotion items in a specified area on the video data, including the video data and a data providing unit 155 that provides transmission data to the app of the user wireless terminal 170 or to the IPTV 175 .

The baby emotional state information for each n emotion items generated by the information generating unit 145 includes n emotion item information and numerical information of the baby's emotional state for each emotion item in text form. It may include emotional state information. Alternatively, the baby emotional state information for each of the n emotion items generated through the information generating unit 145 is a graph-type emotion item that includes n emotion item information and numerical information of the baby’s emotional state for each emotion item in a graph form. Baby emotional state information may be included.

The information application unit 150 is the n emotion items generated through the information generation unit 145 in a specified area on the video data of the transmission data to be provided to the app or IPTV 175 of the specified user wireless terminal 170 . Separate baby emotional state information (eg, baby emotional state information for each emotional item in text form, or baby emotional state information for each emotional item in graph form) may be displayed.

The data providing unit 155 provides transmission data including video data in which the baby emotional state information for each of the n emotion items is displayed in a designated area to the app or IPTV 175 of the designated user wireless terminal 170, The app or IPTV 175 of the user wireless terminal 170 may receive and output the transmission data. Preferably, the transmission data may further include audio data in addition to video data in which the emotional state information of the baby for each of the n emotion items is displayed in a designated area, and the present invention is not limited thereto.

2 is a diagram illustrating an information storage/registration process according to an embodiment of the present invention.

In more detail, FIG. 2 shows a designated recognition target for generating baby emotional state information for each designated emotional item (eg, a baby expression, a baby cry, a sensing value sensing a specified sensing line of Aging through a grounded sensor module, etc.) Storing individual emotional state determination information or mapping the camera device 180 with a target to be provided with transmission data including video data of a specific baby in the base through the camera device 180 provided at the designated base As shows a process of registering information and storing it in a designated management D/B, those of ordinary skill in the art to which the present invention pertains may refer to and/or modify this figure 2 to various implementation methods for the above process (For example, an implementation method in which some steps are omitted or the order is changed) may be inferred, but the present invention includes all implementation methods inferred above, and the technical characteristics of the implementation method shown in FIG. This is not limited.

Referring to Figure 2, the operation server 100 recognizes at least one designated recognition target (eg, baby expression, baby cry, and the specified sensing dialogue of Aging through a sensor module) to recognize in order to determine the emotional state of the baby. The sensed value, etc.) is stored and managed in a designated storage area or operation D/B for each emotional state determination information (200). According to the embodiment of the present invention, the emotional state determination information for each recognition target may include artificial intelligence-based emotional state determination information and/or pattern recognition-based emotional state determination information.

Referring to FIG. 2 , the operation server 100 includes a target and the camera device 180 to receive transmission data including video data of a specific baby in the base through the camera device 180 provided at the designated base. ) to map the user information about the user from the app of the user wireless terminal 170, the camera device 180 information uniquely identifying the camera device 180 provided/supplied to the user, the user wireless terminal 170 The unique identification information for uniquely identifying the app of the user and the ID/PW information set by the user can be requested for mapping registration according to a specified procedure (205), and the operation server 100 is the app of the user wireless terminal 170. The user information, camera device 180 information, unique identification information and ID/PW information are received from the user according to a specified procedure (210), and the user information, camera device 180 information, unique identification information, and ID/PW information are received (210). Information and the like are linked and stored in the designated management Dd/B (215). Meanwhile, according to the implementation method, the user information, camera device 180 information, ID/PW information, etc. are a user terminal used by a user who purchases or is supplied with the camera device 180 or a designated registered terminal that has received the user's subscription application can be registered through

3 is a diagram illustrating storage and management of the acquired data acquired through the camera device 180 of a designated base according to an embodiment of the present invention and stored in a designated storage medium.

In more detail, this figure 3 shows that the acquired data including the video data acquired through the camera module of the camera device 180 provided at the designated base is transmitted to the operation server 100 and stored in a specific storage area of the designated author media. As an illustration of the management process, those of ordinary skill in the art to which the present invention pertains, refer to and/or modify FIG. 3 and various implementation methods for the process (eg, some steps are omitted, or implementation method in which the order is changed), but the present invention is made including all the inferred implementation methods, and the technical characteristics are not limited only to the implementation method shown in FIG. 3 .

Referring to FIG. 3, when the power supply for operation is confirmed (300), the camera device 180 exchanges one or more specified information with a designated operation server 100 through a communication module according to a designated procedure to establish a communication connection. When requesting (305), the operation server 100 authenticates the validity of the camera device 180 by exchanging specified information with the camera device 180 (310), communication connection with the camera device 180 and (315), and correspondingly, communication with the operation server 100 is connected to the camera device 180 (315).

Meanwhile, the operation server 100 dynamically generates ( 320 ) one-time address information for receiving the acquired data acquired through the camera device 180 , and provides the generated one-time address information to the camera device 180 . and (325), the camera device 180 may receive the one-time address information from the operation server 100 (330), and set the received one-time address information as a destination address to transmit the acquired data (335). ), the process of generating the one-time address information or setting the destination address may be repeated periodically (eg, every 30 seconds).

Meanwhile, the camera device 180 may acquire video data of a baby in a designated base through the camera module ( 340a ), and transmit the acquired data including the video data to a designated destination address ( 345 ).

Alternatively, the camera device 180 acquires video data of a baby in a designated base through the camera module (340a), and after acquiring audio data in the base through the sound input unit (340b), the video data and the acquired data including audio data may be transmitted to a designated destination address ( 345 ).

Alternatively, the camera device 180 acquires video data of a baby in a designated base through the camera module (340a), and after acquiring sensing data sensing a designated sensing target of the baby through the sensor module (340c) ), the acquired data including the video data and the sensing data may be transmitted to a designated destination address ( 345 ).

Alternatively, the camera device 180 acquires video data of a baby in a designated base through the camera module (340a), acquires audio data in the base through the sound input unit (340b), and uses the sensor module After acquiring the sensing data for sensing the baby's designated sensing target ( 340c ), the acquired data including the video data, the audio data, and the sensing data may be transmitted to the designated destination address ( 345 ).

The operation server 100 receives the acquired data including at least video data from the camera device 180 (350), and stores the received acquired data (or each data included in the acquired data) of a designated storage medium. It is stored and managed in a specific storage area (355).

4 is a diagram illustrating transmission data including video data indicating baby emotional state information for each designated emotion item according to the first embodiment of the present invention.

In more detail, FIG. 4 is a video in which baby emotional state information for each of the n emotion items is generated by reading the designated video data, and then the baby emotional state information for each of the n emotion items is displayed in a designated area on the video data. As showing a process of providing transmission data including data to the app of the user wireless terminal 170 or the IPTV 175, those skilled in the art to which the present invention pertains, refer to FIG. 4 and Various implementation methods for the above process (eg, an implementation method in which some steps are omitted or the order is changed) may be inferred with / or modified, but the present invention includes all the inferred implementation methods, The technical characteristics are not limited only to the implementation method shown in FIG. 4 .

Referring to FIG. 4, when artificial intelligence-based expression-based emotional state determination information is generated and stored through the process of FIG. 2, video data stored in a specific storage area of the storage medium designated through the process of FIG. 3 is included. video data acquired through the camera device 180 in a specific storage area of a storage medium designated through the process of FIG. 3 or at the time of providing transmission data to the designated user's wireless terminal 170's app or IPTV 175 After the acquired data including a is stored, the operation server 100 performs a procedure of reading the video data specified through the artificial intelligence-based expression-based emotional state determination information to perform n1 pieces corresponding to the baby expression of the video data. A baby emotional state value for each emotion item is determined (a00).

On the other hand, when the expression-based emotional state determination information based on pattern recognition is generated and stored through the process of FIG. 2, transmission data including video data stored in a specific storage area of the storage medium designated through the process of FIG. 3 is designated. Acquisition data including the video data acquired through the camera device 180 in a specific storage area of a storage medium designated through the process of FIG. 3 or at the time provided by the app of the user wireless terminal 170 or the IPTV 175 After is stored, the operation server 100 reads the designated video data and recognizes f feature points for a specific baby's facial region in the base, and geometrical relationship between the feature points related to the baby's emotional expression among the recognized f feature points Recognizes i pieces of baby expression recognition information corresponding to ( 400 ).

If the facial expression recognition information of i babies is recognized based on the video data, the operation server 100 determines the recognized i based on the expression-based emotional state determination information stored in the operation D/B through the process of FIG. 2 . The e number of baby expression pattern information matched with the baby expression recognition information and a preset level of similarity or more is checked (405), and n1 baby emotion items having a matching relationship with the e baby expression pattern information are determined (410).

If n1 baby emotion items corresponding to the recognized i baby expression recognition information are determined, the operation server 100 determines a matching ratio between the recognized i baby expression recognition information and the e baby expression pattern information. Based on the basis, n1 emotion items obtained by reading the numerical relationship of correlation between the e baby expression pattern information and n1 baby emotion items to quantify the baby emotional state for each n1 emotion items corresponding to the recognized i baby expression recognition information A star baby emotional state value is determined (415).

If the baby emotional state values for each of the n1 emotional items are calculated, the operation server 100 uses the determined baby emotional state values for each of the n1 emotional items to display the baby for each of the n designated emotional items on the video data. Generates emotional state information (420).

If the baby emotional state information for each of the n emotion items is generated, the operation server 100 is an app of the user wireless terminal 170 corresponding to the unique identification information stored in the management D/B through the process of FIG. 2 or the Display the baby emotional state information for each of the n emotion items in a designated area on the video data of the transmission data to be provided to the IPTV 175 identified/authenticated through the ID/PW information stored in the management D/B (425) ), and provides transmission data including video data displaying baby emotional state information for each of the n emotion items to the app or IPTV 175 of the user wireless terminal 170 (430), and the user wireless terminal 170 ) app or IPTV 175 receives and outputs transmission data including video data indicating baby emotional state information for each of the n emotion items ( 435 ). Advantageously, the transmission data may further include audio data associated with the video data.

5 is a diagram illustrating transmission data including video data indicating baby emotional state information for each designated emotional item according to the second embodiment of the present invention.

In more detail, FIG. 5 shows baby emotional state information for each n emotion items specified by reading the audio data associated with the specified video data and using (additional) the recognized baby cry recognition information, and then a specified schedule on the video data. It shows the process of providing transmission data including video data displaying the baby emotional state information for each of the n emotion items generated in the area to the app of the user wireless terminal 170 or the IPTV 175, and the present invention is Those of ordinary skill in the art can infer various implementation methods (eg, an implementation method in which some steps are omitted or the order is changed) for the above process by referring to and/or modifying this figure 5. However, the present invention is made including all the implementation methods inferred above, and the technical characteristics are not limited only to the implementation method shown in FIG. 5 .

Referring to FIG. 5, when artificial intelligence-based cry-based emotional state determination information is generated and stored through the process of FIG. 2, video data stored in a specific storage area of the storage medium designated through the process of FIG. A video acquired through the camera device 180 in a specific storage area of a storage medium designated through the process of FIG. 3 or at a point in time when the transmitted data including the transmitted data is provided to the app or IPTV 175 of the designated user wireless terminal 170 After the acquired data including the data is stored, or in conjunction with the baby emotional state value calculation process for each n1 emotion items based on the baby expression recognition shown in FIG. 4, the operation server 100 is the artificial intelligence-based cry Through the sound-based emotional state determination information, a procedure of reading audio data associated with specified video data is performed to determine the baby emotional state value for each n2 emotional items corresponding to the baby cry of the audio data (b00) .

On the other hand, when the expression-based emotional state determination information based on pattern recognition is generated and stored through the process of FIG. 2, transmission data including video data stored in a specific storage area of the storage medium designated through the process of FIG. 3 is designated. Acquisition data including the video data acquired through the camera device 180 in a specific storage area of a storage medium designated through the process of FIG. 3 or at the time provided by the app of the user wireless terminal 170 or the IPTV 175 After is stored, or in conjunction with the baby emotional state value calculation process for each n1 emotion item based on baby expression recognition shown in FIG. 4 , the operation server 100 provides audio data of a certain section associated with the specified video data. is read to recognize j number of baby cry recognition information for the specific baby ( 500 ).

If j baby cry recognition information is recognized based on the audio data, the operation server 100 recognizes the recognition based on the crying sound-based emotional state determination information stored in the operation D/B through the process of FIG. 2 . Check the c baby cry pattern information matched with the j number of baby cry recognition information and the preset standard similarity or more (505), and n2 baby emotion items having a matching relationship with the identified c baby cry pattern information is determined (510).

If n2 baby emotion items corresponding to the recognized j pieces of baby cry recognition information are determined, the operation server 100 is configured to perform a difference between the recognized j baby cry recognition information and the c baby cry pattern information. Based on the matching ratio, the correlation numerical relationship between the c baby cry pattern information and n2 baby emotion items is read, and the baby emotional state for each n2 emotion items corresponding to the recognized j baby cry recognition information is digitized. A baby emotional state value for each n2 emotion items is determined (515).

If the baby emotional state values for each of the n2 emotional items are calculated, the operation server 100 uses the determined baby emotional state values for each of the n2 emotional items to display the baby for each of the n designated emotional items on the video data. Emotional state information may be generated ( 520 ).

On the other hand, when the baby emotional state value for each n1 emotion item is calculated through the process of FIG. 4 at a point in time before, during, or after the baby emotional state value for each n2 emotion item is calculated, the operation server 100 is By combining the determined baby emotional state values for each of the n1 emotional items and the n2 emotional state values for each of the n2 emotional items, it is possible to generate baby emotional state information for each of the n designated emotional items that can be displayed on the video data (520). .

If the baby emotional state information for each of the n emotion items is generated, the operation server 100 is an app of the user wireless terminal 170 corresponding to the unique identification information stored in the management D/B through the process of FIG. 2 or the Display the baby emotional state information for each of the n emotion items in a designated area on the video data of the transmission data to be provided to the IPTV 175 identified/authenticated through the ID/PW information stored in the management D/B (525) ), and provides transmission data including video data displaying baby emotional state information for each of the n emotion items to an app or IPTV 175 of the user wireless terminal 170 (530), and the user wireless terminal 170 ) app or IPTV 175 receives and outputs transmission data including video data indicating baby emotional state information for each of the n emotion items ( 535 ). Advantageously, the transmission data may further include audio data associated with the video data.

6 is a diagram illustrating transmission data including video data indicating baby emotional state information for each designated emotion item according to the third embodiment of the present invention.

In more detail, FIG. 6 is a video after generating baby emotional state information for each n emotion item specified by reading the sensing data associated with the specified video data and/or audio data and using (additional) the recognized sensing value recognition information. As showing the process of providing transmission data including video data displaying the baby emotional state information for each of the n emotion items generated in a designated area on the data to the app of the user wireless terminal 170 or the IPTV 175 , those of ordinary skill in the art to which the present invention pertains, refer to and/or modify this figure 6 to see various implementation methods for the above process (eg, an implementation method in which some steps are omitted or the order is changed) Although it can be inferred, the present invention is made including all of the inferred implementation methods, and the technical characteristics are not limited only to the implementation method illustrated in FIG.

Referring to FIG. 6, a time point at which transmission data including video data stored in a specific storage area of a storage medium designated through the process of FIG. 3 is provided to an app or IPTV 175 of a designated user wireless terminal 170, or After the acquired data including the video data acquired through the camera device 180 is stored in a specific storage area of the storage medium designated through the process of FIG. 3, or n1 emotions based on the baby's expression recognition shown in FIG. In conjunction with the baby emotional state value calculation process for each item, or in conjunction with the baby emotional state value calculation process for each n2 emotional items based on baby cry recognition shown in Fig. 5, the operation server 100 provides the designated video data and/or read sensing data associated with audio data to recognize k sensed value recognition information for sensing data of a specified sensing target associated with the emotional expression of the specific baby ( 600 ).

If k sensed value recognition information is recognized based on the sensing data, the operation server 100 determines the k recognized based on the sensing-based emotional state determination information stored in the operation D/B through the process of FIG. 2 . Check the s sensed value recognition information and the s sensed value pattern information matched with more than a preset reference similarity (605), and determine n3 baby emotion items having a matching relationship with the identified s sensed value pattern information (610) ).

If n3 baby emotion items corresponding to the recognized k sensed value recognition information are determined, the operation server 100 determines a matching ratio between the recognized k sensed value recognition information and the s sensed value pattern information. Based on the s sensed value pattern information and the correlation numerical relationship between the n3 baby emotion items, the n3 emotion items obtained by digitizing the baby emotional state for each of the n3 emotion items corresponding to the recognized k sensed value recognition information A star baby emotional state value is determined (615).

If the baby emotional state values for each of the n3 emotional items are calculated, the operation server 100 uses the determined baby emotional state values for each of the n3 emotional items to display the baby for each of the n designated emotional items on the video data. Emotional state information may be generated ( 620 ).

On the other hand, when the baby emotional state value for each n1 emotion item is calculated through the process of FIG. 4 at a point in time before, during, or after the baby emotional state value for each n3 emotion item is calculated, the operation server 100 is By combining the determined baby emotional state value for each n1 emotion item and the baby emotional state value for each n3 emotion item, it is possible to generate baby emotional state information for each n designated emotion item displayable on the video data (620). .

Alternatively, at a point in time before, during, or after the baby emotional state values for each of the n3 emotional items are calculated, n1 baby emotional state values for each of the n1 emotional items are calculated through the process of FIG. 4, and n2 values are obtained through the process of FIG. When the baby emotional state value for each emotion item is calculated, the operation server 100 determines the baby emotional state value for each of the n1 emotion items, the baby emotional state value for each n2 emotion item, and the baby emotion for each of the n3 emotion items. By combining the state values, it is possible to generate baby emotional state information for each of n designated emotional items that can be displayed on the video data ( 620 ).

If the baby emotional state information for each of the n emotion items is generated, the operation server 100 is an app of the user wireless terminal 170 corresponding to the unique identification information stored in the management D/B through the process of FIG. 2 or the Display the baby emotional state information for each of the n emotion items in a designated area on the video data of the transmission data to be provided to the IPTV 175 identified/authenticated through the ID/PW information stored in the management D/B (625) ), and provides transmission data including video data displaying baby emotional state information for each of the n emotion items to an app or IPTV 175 of the user wireless terminal 170 (630), and the user wireless terminal 170 ) or IPTV 175 receives and outputs transmission data including video data indicating baby emotional state information for each of the n emotion items ( 635 ). Advantageously, the transmission data may further include audio data associated with the video data.

100: operation server 105: information management unit
110: information register 115: communication management unit
120: data receiving unit 125: data storage unit
130: information recognition unit 135: information discrimination unit
140: emotional state determination unit 145: information generation unit
150 information application unit 155: data provision unit
170: wireless terminal 175: IPTV
180: camera device

Claims (31)

In the method executed through an operation server having a sound input unit and a camera module to communicate with a camera device provided in a designated base,
A first step of generating expression-based emotional state determination information for determining the baby emotional state value for each designated emotional item corresponding to the baby's expression through a designated artificial intelligence-based learning process and storing it in a designated storage area;
Video data encoded in a specified manner by photographing a specific baby in the base through the camera module of the camera device and audio data encoded in a specified manner input through the sound input unit of the camera device are received and stored in a specified storage medium a second step of linked storage and management in the area;
N1 (1≤N1≤N) items that can be distinguished through a baby expression among N (N≥2) specified baby emotion items by performing a procedure of reading specified video data through the expression-based emotional state determination information based on the artificial intelligence a third step of determining a baby emotional state value for each n1 (1≤n1≤N1) emotion items corresponding to the baby expression in the video data based on the baby emotion item;
a fourth step of generating baby emotional state information for each of n (1≤n≤N) designated emotional items that can be displayed on video data by using the calculated baby emotional state values for each of the n1 emotional items; and
A fifth step of providing transmission data including video data displaying the baby emotional state information for each of the n emotion items generated in a specified area on the video data to an app or IPTV of the user's wireless terminal; output method.
According to claim 1, wherein the first step,
By performing a specified AI-based learning process, e(1≤e≤E) baby expressions corresponding to the specified baby facial area are determined among the specified E(E≥2) baby expressions, and N(N Among ≥2) baby emotion items, n1 (1≤n1≤N1) items having a matching relationship with the e number of baby expressions determined above based on N1 (1≤N1≤N) baby emotion items that can be distinguished through baby expressions Determining baby emotion items, generating expression-based emotional state determination information for determining baby emotional state values for each n1 emotion item corresponding to the correlation between the identified e baby expressions and n1 baby emotion items, and storing the specified A method of outputting an infant's emotional state, characterized in that it comprises the step of storing in the area.
According to claim 1, wherein the first step,
N1 (1≤N1≤) that can be distinguished through the baby expression among the specified N(N≥2) baby emotion items while corresponding to the specified e(1≤e≤E) baby expressions among the specified E(E≥2) baby expressions Receive a plurality of video information corresponding to a scene including N) baby emotion items and a baby facial region having a specified correlation, and n1 (1≤n1≤N1) emotion items for the input plurality of video information setting a star baby emotional state value as an output variable;
By reading the geometric relationship between the specified feature points of the baby's facial area included in the plurality of input video information and recognizing the pattern, P(P() corresponding to the baby's emotional state value for each n1 emotion item corresponding to the baby's expression in the baby's facial area. extracting P≥2) pieces of scene pattern characteristic information and setting them as input variables (Feature Vectors);
Through a designated artificial intelligence algorithm, more than a specified number of the input and output variables are learned to determine the baby's emotional state value for each n1 emotion item corresponding to the baby's expression in the baby's facial area among a plurality of scenes including the baby's facial area Generating expression-based emotional state determination information for the purpose and storing the information in a designated storage area; Infants and infants emotional state output method, characterized in that it comprises a.
According to claim 1, wherein the first step,
N1 (1≤N1≤) that can be distinguished through the baby expression among the specified N(N≥2) baby emotion items while corresponding to the specified e(1≤e≤E) baby expressions among the specified E(E≥2) baby expressions By reading and pattern recognition geometrical relationships between designated feature points of a baby's facial area of a plurality of video information corresponding to a scene including N) baby emotion items and a baby's facial area having a specified correlation, the baby's facial expression in the baby's facial area P (P≥2) scene pattern characteristic information extracted as pattern characteristics of baby emotional state values for each corresponding n1 (1≤n1≤N1) emotion items are input as input variables (Feature Vectors), and the P scenes setting a baby emotional state value for each n1 emotional item with respect to the video information from which the pattern characteristic information is extracted as an output variable;
Through a designated artificial intelligence algorithm, more than a specified number of the input and output variables are learned to determine the baby's emotional state value for each n1 emotion item corresponding to the baby's expression in the baby's facial area among a plurality of scenes including the baby's facial area Generating expression-based emotional state determination information for the purpose and storing the information in a designated storage area; Infants and infants emotional state output method, characterized in that it comprises a.
5. The method of claim 3 or 4, wherein the output variable is
e pieces of baby expression information corresponding to the baby facial region included in the video information;
and at least one piece of information among n1 pieces of baby emotion item information corresponding to the baby's expression in the baby's facial area.
According to claim 1, wherein the second step,
and receiving the moving picture data including the video data and the audio data from the camera device and storing and managing them in a specific storage area of a designated storage medium.
According to claim 1, wherein the second step,
and receiving the video data and audio data from the camera device, respectively, and storing and managing them in a specific storage area of a designated storage medium.
The method of claim 7, wherein the second step comprises:
The method for outputting an infant's emotional state, characterized in that it further comprises the step of synchronizing the time of the video data and the audio data received from the camera device and storing the synchronized time in a specific storage area of a designated storage medium.
According to claim 1, wherein the transmission data,
The method for outputting an infant's emotional state, characterized in that it further comprises audio data stored in a specific storage area of the storage medium.
The method according to claim 1, wherein the designated video data comprises:
Video data that is periodically screened by a specified unit of playing time;
A method for outputting an infant's emotional state, comprising at least one of video data periodically selected for each specified number of reproduction frames.
The method of claim 1,
The method for outputting the emotional state of infants and toddlers, characterized in that it further comprises the step of confirming whether transmission data including video data stored in a specific storage area of the storage medium is provided by an App or IPTV of a designated user's wireless terminal.
According to claim 1, wherein the third step,
By performing a procedure of reading the video data specified through the expression-based emotional state determination information based on the artificial intelligence, the e baby expressions corresponding to the baby face area on the video data are determined among the E baby expressions specified, and the baby expressions are determined. Among the N1 baby emotion items that can be distinguished through the n1 baby emotion items, n1 baby emotion items having a matching relationship with the identified e baby expressions are determined, and n1 corresponding to the correlation between the identified e baby expressions and n1 baby emotion items. A method for outputting an infant's emotional state, comprising the step of determining a baby's emotional state value for each emotional item.
According to claim 1, wherein the fourth step,
and storing the baby emotional state information for each of the generated n emotional items in association with the video data.
14. The method of claim 13, wherein the fourth step comprises:
and storing the generated baby emotional state information for each of the n emotional items in association with a time value (or timestamp) for synchronizing with a specified playback time of the video data.
According to claim 1, wherein the first step,
It further includes the step of generating cry-based emotional state determination information to determine the baby emotional state value for each designated emotional item corresponding to the baby's cry through a designated artificial intelligence-based learning process and storing it in a designated storage area A method for outputting an infant's emotional state, characterized in that by
16. The method of claim 15, wherein the first step comprises:
Performs a designated AI-based learning process to determine c(1≤c≤C) baby cries corresponding to the designated baby cry among the designated C (C≥2) baby cries, and determines the designated N( Based on N2 (1≤N2≤N) baby emotion items that can be distinguished through baby cry among N≥2) baby emotion items, n2 (1≤n2≤N) having a matching relationship with the c baby cry sounds determined above Cry-based emotional state determination information for determining N2) baby emotion items and determining baby emotional state values for each n2 emotion items corresponding to the correlation between the identified c baby cry sounds and n2 baby emotion items A method for outputting an infant's emotional state, characterized in that it comprises the step of generating and storing in a designated storage area.
16. The method of claim 15, wherein the first step comprises:
N2(1) distinguishable through baby cry among N(N≥2) specified baby emotion items while corresponding to specified c(1≤c≤C) baby cries out of C(C≥2) specified baby cries ≤N2≤N) baby emotion items and a plurality of audio information corresponding to a crying baby having a specified correlation are received, and n2 (1≤n2≤N2) baby emotion items for the inputted audio information are received. setting the emotional state value as an output variable;
By reading out the plurality of input audio information and pattern recognition, Q (Q≥2) pieces of sound pattern characteristic information corresponding to the baby emotional state values for each n2 emotion items corresponding to the crying sound are extracted and input variables (Features) Vectors);
Cry-based emotional state determination information for determining the baby emotional state value for each n2 emotional items corresponding to the baby cry included in the specified audio data by learning more than a specified number of the input and output variables through a specified artificial intelligence algorithm The method for outputting the emotional state of infants and toddlers, characterized in that it comprises; generating and storing in a designated storage area.
16. The method of claim 15, wherein the first step comprises:
N2(1) distinguishable through baby cry among N(N≥2) specified baby emotion items while corresponding to specified c(1≤c≤C) baby cries out of C(C≥2) specified baby cries ≤N2≤N) baby emotion items and n2 (1≤n2≤N2) baby emotion items corresponding to the baby crying sound by reading a plurality of audio information corresponding to the crying baby with a specified correlation and recognizing the pattern Q (Q≥2) sound pattern characteristic information extracted as the pattern characteristic of the emotional state value is input as an input variable (Feature Vectors), and the Q sound pattern characteristic information is extracted by n2 emotional items for the extracted audio information. setting the baby emotional state value as an output variable;
Cry-based emotional state determination information for determining the baby emotional state value for each n2 emotional items corresponding to the baby cry included in the specified audio data by learning more than a specified number of the input and output variables through a specified artificial intelligence algorithm The method for outputting the emotional state of infants and toddlers, characterized in that it comprises; generating and storing in a designated storage area.
The method of claim 17 or 18, wherein the output variable is
c baby cry information corresponding to the baby cry included in the audio information;
The method for outputting an infant's emotional state, characterized in that it further includes at least one piece of information among n2 pieces of baby emotion item information corresponding to the baby's cry.
16. The method of claim 15, wherein the third step,
N2 (1≤N2≤N) distinguishable through baby cry among N (N≥2) specified baby emotion items by performing a procedure of reading specified audio data through the AI-based cry-based emotional state determination information ) Baby emotional state, characterized in that it further comprises the step of determining the baby emotional state value for each n2 (1≤n2≤N2) emotion items corresponding to the baby cry of the audio data based on the number of baby emotion items output method.
The method of claim 20, wherein the third step comprises:
By performing a procedure of reading designated audio data through the artificial intelligence-based cry-based emotional state determination information, c baby cries corresponding to baby cries on the audio data are determined among C designated baby cries, Among the N2 baby emotion items that can be distinguished through a cry, n2 baby emotion items having a matching relationship with the identified c baby cry are determined, and correlation between the identified c baby cry and n2 baby emotion items The method for outputting an infant's emotional state, characterized in that it comprises the step of determining the baby's emotional state value for each n2 emotional items corresponding to the figure.
22. The method of claim 20 or 21, wherein the fourth step comprises:
and generating baby emotional state information for each of the n designated emotional items by additionally using the calculated baby emotional state values for each of the n2 emotional items.
The method of claim 1,
The first step is
Among the sensing data sensed through the sensor module provided in or linked to the camera device, S (S≥2) sensing value pattern information for the sensing data of the specified sensing target related to the emotional expression of the baby and the N specified baby emotion items Sensing that quantifies the matching relationship information between N3 (1≤N3≤N) baby emotion items and the correlation between the S sensed value pattern information and N3 baby emotion items that can be distinguished based on the sensing value pattern of the specified sensing target Further comprising the step of registering and storing sensing-based emotional state determination information including numerical relationship information based on correlation in operation D/B,
The second step is
Further comprising the step of receiving sensing data sensed through a sensor module provided in or linked to the camera device, and storing and managing the video data in a specific storage area of a designated storage medium in association with the method,
The third step is
Reading the sensing data associated with the video data to recognize k (k≥1) sensing value recognition information for the sensing data of a specified sensing target associated with the emotional expression of the specific baby,
Based on the sensing-based emotional state determination information, s (1≤s≤S) sensed value pattern information matched with the recognized k sensed value recognition information and a preset standard similarity or higher is checked, and the identified s sensed values Identify n3 (1≤n3≤N3) baby emotion items that have a matching relationship with pattern information,
Based on the matching ratio between the recognized k sensed value recognition information and the s sensed value pattern information, the correlation numerical relationship between the s sensed value pattern information and n3 baby emotion items is read, and the recognized k senses Further comprising the step of calculating the baby emotional state value for each n3 emotion items that digitize the baby emotional state for each n3 emotion items corresponding to the value recognition information,
The fourth step is
and generating baby emotional state information for each of the n designated emotional items by additionally using the calculated baby emotional state values for each of the n3 emotional items.
The method of claim 23, wherein the sensor module,
A wearable sensor module that is worn on the baby's body and senses a specified sensing target,
A method for outputting an infant's emotional state, comprising at least one sensor module among radar sensor modules for sensing a designated sensing target of a baby by using a designated electromagnetic wave.
The method of claim 23 or 20, wherein the sensing target,
A method for outputting an infant's emotional state, characterized in that it includes at least one of a baby's heartbeat, a baby's body temperature, a baby's ambient humidity, and whether the baby is apnea.
The method of claim 23, wherein the S sensed value pattern information comprises:
A method for outputting an infant's emotional state, characterized in that it includes range information of a sensing value characteristic constituting each sensing value pattern for sensing data of a specified sensing target related to the emotional expression of a baby.
The method of claim 26, wherein the sensed value characteristic,
A method for outputting an infant's emotional state, comprising at least one or a combination of two or more of the sensing value change rule feature, the sensed value change feature, the sensed value change rate feature, and the sensed value change periodic feature.
The method of claim 23, wherein the sensing-based correlation numerical relationship information,
Infant emotional state output method, characterized in that it includes the correlation numerical information calculated through the relational expression proportional to the matching ratio between the sensing value recognition information recognized through the sensing data of the specified sensing target and the previously registered sensing value pattern information .
The method of claim 23, wherein the sensing-based correlation numerical relationship information,
A method for outputting an infant's emotional state, characterized in that it includes numerical information on correlations for each matching ratio section between sensing value recognition information recognized through sensing data of a specified sensing target and pre-registered sensing value pattern information.
According to claim 1, wherein the baby emotional state information for each of the n emotion items,
A method for outputting an infant's emotional state, characterized in that it includes n emotional item information and numerical information of the baby's emotional state for each emotional item in the form of text.
According to claim 1, wherein the baby emotional state information for each of the n emotion items,
A method for outputting an infant's emotional state, characterized in that it includes n emotion item information and numerical information of the baby's emotional state for each emotion item in the form of a graph.

Images