KR102288860B1 - Apparatus for Intelligent Infant Observation and Driving Method Thereof - Google Patents

Abstract

The present invention relates to an intelligent infant observation apparatus and a method of driving the apparatus. An intelligent infant observation apparatus according to an embodiment of the present invention may include a communication interface part for receiving the photographed image of an arbitrary space from a photographing device installed in the arbitrary space where children are active, and a control part for analyzing the received photographed image and provides an analysis result related to the activity of the children to terminal devices possessed by their parents.

Description

Intelligent infant observation device and driving method thereof {Apparatus for Intelligent Infant Observation and Driving Method Thereof}

The present invention relates to an intelligent infant observation device and a method of driving the device, and more particularly, a terminal device for parents and teachers by determining the movement path, behavior pattern, abnormal situation, etc. of children who are active in a specific space, such as a kindergarten. It relates to an intelligent infant observation device that provides a notification in the form of a video or the like, and a driving method of the device.

In the case of CCTV installed in daycare centers and kindergartens, there is no separate function other than the action of recording and storing only a specific direction. Since many accidents have recently occurred in daycare centers and kindergartens, teachers and parents are also very interested in children's behavior in the hospital. To this end, teachers have to take pictures of school activities with a camera and send them to the parents, and parents have no other choice but to ask the children or call the teachers because they have no information other than the photos the teacher sends.

In this case, there is a problem that teachers spend a lot of time on other tasks rather than direct education and management of children, and there is a problem that parents have no choice but to rely on information provided by teachers.

Korea Patent Publication No. 1997-0004858 (Jan. 29, 1997) Korean Patent Publication No. 2002-0032059 (2002.05.03) Korean Patent Application Laid-Open No. 2001-0088476 (September 28, 2001) Korean Patent Application Laid-Open No. 10-2004-0033933 (2004.04.28) Korean Patent Application Laid-Open No. 10-2004-0092735 (2004.11.04) Korean Patent Publication No. 10-2019-0030508 (2019.03.29) Korea Patent Publication No. 10-1409500 (2014.06.12)

An embodiment of the present invention provides an intelligent infant observation device that provides a notification in the form of a video to the terminal device of parents and teachers by determining the movement path, behavior pattern, abnormal situation, etc. of children who are active in a specific space, such as a kindergarten, and An object of the present invention is to provide a method of driving the device.

The intelligent infant observation apparatus according to an embodiment of the present invention includes a communication interface unit for receiving a photographed image of an arbitrary space from a photographing device installed in an arbitrary space where children are active, and a child's activity by analyzing the received photographed image. and a control unit that provides the analysis result related to the parental device to a terminal device possessed by the parent.

The control unit may determine the activity state of the children included in the captured image, and provide a captured photo or an edited video for an arbitrary section of the captured image to a terminal device owned by the parents of the children based on the determination result. .

The control unit may determine, as the activity state, an action radius and a favorite object to be active in the arbitrary space, and further provide characteristic information of the child's development to the terminal device based on the determination result.

The controller may selectively transmit notification information to the terminal device of the teacher and the parent of the child when it is determined as a result of the determination that the child leaves the monitoring area of the photographing device and does not return even after a specified time.

The controller may detect the body temperature of the child by operating the thermal camera as the photographing device when the abnormal condition of the child is determined as the determination result.

When transmitting the edited video to the terminal device, the control unit may transmit the contents of children's conversations by inserting them as subtitles.

When inserting the subtitles, the controller may insert the subtitles using a language dictionary for each child generated based on big data on languages that can be spoken for each age of the child.

In addition, the driving method of the intelligent infant observation device according to an embodiment of the present invention comprises the steps of: receiving, by a communication interface unit, a photographed image of an arbitrary space from a photographing device installed in an arbitrary space where children are active, and a control unit, the receiving and analyzing one captured image and providing an analysis result related to the activity of the child to a terminal device possessed by the parent.

The providing step is to determine the activity state of the children included in the captured image, and based on the determination result, provide a captured photo or edited video for an arbitrary section of the captured image to a terminal device possessed by the parents of the children. can

The providing may further provide characteristic information of the child's development to the terminal device based on the determination result by determining an action radius and a favorite object, etc. to be active in the arbitrary space as the activity state.

In the providing step, when it is determined as a result of the determination that the child leaves the monitoring area of the photographing device and does not return even after a specified time, notification information may be selectively transmitted to the terminal device of the child's teacher and parent.

The driving method may further include detecting the body temperature of the child by operating a thermal camera as the photographing device when the abnormal situation of the child is determined as the determination result.

In the providing step, when the edited video is transmitted to the terminal device, the contents of children's conversations may be inserted as subtitles and transmitted.

In the step of inserting the subtitles, the subtitles may be inserted using a language dictionary for each child, which is generated based on big data on languages that can be spoken by each age of the child when inserting the subtitles.

According to an embodiment of the present invention, first, the action radius of each child is tracked within a plurality of fixed CCTVs (at this time, when disappearing from the CCTV, the time is also checked and the time in the CCTV blind area is also checked), and the object adjacent to the child (e.g., books, toys, friends, teachers, meals, etc.) to analyze the time, etc., capture each child's photo for a certain amount of time and send it to parents It will be possible to provide characteristics of children's development by analyzing objects, etc., and to transmit real-time captured photos to each parent so that activities in the hospital can be easily checked.

In addition, an embodiment of the present invention provides an alarm to parents and teachers when a child stays in the bathroom or alone for a long time in a blind spot, and can easily check the children's body temperature, so that the teacher and parents can quickly grasp the children's activities in the hospital can

Furthermore, in an embodiment of the present invention, for example, by displaying a voice with subtitles for a specific situation occurring in a kindergarten, the child can deliver activity, participation, etc. to parents through actual play, collaboration, class, etc. It can make it possible to grasp accurate information about activities in kindergarten.

1 is a view showing an intelligent infant observation system according to an embodiment of the present invention;
2 is a block diagram illustrating the detailed structure of the intelligent infant observation device of FIG. 1;
3 is a flowchart showing the driving process of the intelligent infant observation device of FIG. 1, and
4 is a flowchart of another driving process of the intelligent infant observation device of FIG. 1 .

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a view showing an intelligent infant observation system according to an embodiment of the present invention.

As shown in FIG. 1 , the intelligent infant observation system 90 according to an embodiment of the present invention includes a photographing device 100 , a teacher terminal device 105 , a parent terminal device 110 , and an intelligent infant observation device 130 . includes some or all of

The photographing device 100 is installed in a space where children are active, for example, in an arbitrary space such as a daycare center or kindergarten. Of course, the photographing apparatus 100 may be installed in a home to observe infants and nanny. The photographing apparatus 100 includes a general CCTV (Closed Circuit Television) camera or an IP (Internet Protocol) camera as a surveillance camera. In addition, the photographing apparatus 100 may include a Pan-Tilt-Zoom (PTZ) camera capable of pan, tilt, and zoom operations as well as a fixed camera.

For example, the photographing device 100 is preferably installed in a plurality of spaces divided by partitions such as walls, ie, for each classroom or for each class, such as a nursery school or a kindergarten. Therefore, it is preferable that a plurality of photographing devices 100 are installed in a specific nursery school or kindergarten building. From this point of view, in the embodiment of the present invention, unnecessary data is filtered out by first analyzing the captured image for each building, for example, only the data required by the intelligent infant observation device 130 of FIG. 1 can be selected and transmitted. there is. To this end, the embodiment of the present invention may further include an imaging device such as an edge device. Of course, the edge device may be a specific computer provided in kindergarten. However, it may also be a compact device on the market. When the edge device is additionally installed as described above, it is preferable that the edge device receives the images of the plurality of photographing devices 100 , and the edge device interworks with the intelligent infant observation device 130 .

The photographing apparatus 100 may detect the illumination of an arbitrary space of a nursery school or kindergarten by including an illumination sensor, and may control ON and OFF the photographing operation based on the detection result. Therefore, in the final state when all the children return home, the photographing operation may be stopped. Of course, it may be possible to operate with a motion sensor instead of an illuminance sensor. When a movement of a person or the like is sensed in an arbitrary space, the photographing operation may be started and the photographing operation may be stopped when the movement is not detected.

For example, the photographing devices 100 installed in various nurseries and kindergartens across the country may be centrally controlled by the intelligent infant observation device 130 . Therefore, the sensing data by the above sensor is provided to the intelligent infant observation device 130, and based on the analysis result of the sensing data, the imaging device 100 of a specific building may be controlled on/off. Of course, current time information may be considered for on-off control. Usually, in buildings such as nurseries and kindergartens, children's admission and discharge are carried out within a specified time range. Accordingly, even if motion is detected in an arbitrary space at other times, it may be ignored and the photographing apparatus 100 may not operate. Of course, in relation to this precise operation, the intelligent infant observation device 130 can automatically adjust the time range or control the operation of the photographing device 100 based on big data of a specific building or space, and Since the photographing apparatus 100 may operate accordingly, the embodiment of the present invention will not be limited to any one operation.

In general, the photographing apparatus 100 may photograph an arbitrary space, and may be installed so that there is no blind spot in the corresponding space. For example, the photographing apparatus 100 may be installed in a hallway in addition to an arbitrary space, or may be installed in a space such as a bathroom. Accordingly, the photographing apparatus 100 may transmit the photographed image to the intelligent infant observation apparatus 130 by photographing every move of the children.

Above all, the photographing apparatus 100 according to an embodiment of the present invention may further include a thermal camera or a thermal imager. Accordingly, for example, when there is a request for operation of the thermal camera from the intelligent infant observation device 130 , it may operate to provide the captured image to the intelligent infant observation device 130 . The thermal camera selectively operates to check the body temperature in a specific condition, for example, when an abnormal situation different from the normal behavior of a child is determined.

The teacher terminal device 105 or the parent terminal device 110 may include a smart phone or the like. For example, the intelligent infant observation device 130 can operate as a platform, so that the application (hereinafter, the app) can be installed by downloading it to the teacher terminal device 105 or the parent terminal device 110, and through this In the case of a teacher, it may be possible to observe the child's condition by periodically accessing the data such as videos or notifications related to the child they manage, and in the case of the parent, or by accessing the website under their own interest. there is. Of course, the teacher terminal device 105 or the parent terminal device 110 is preferably a smart phone, but since it may further include a tablet PC, a wearable device such as a domestic S company's Galaxy * Gear, and a desktop computer or a laptop computer, etc. , it will not be particularly limited to any one device.

The teacher's terminal device 105 registers identification information (eg, device ID) or phone number for his/her smartphone when using the service. You can be notified of an alarm that notifies you. In addition, the intelligent infant observation device 130 determines the abnormal situation of the child based on the photographed image, and as a result of measuring the child's body temperature through a thermal camera as the photographing device 100, when fever is detected, it is 105) can be notified. Of course, this notification can be made in various ways according to the intention of the system designer, but the teacher terminal device 105 can receive various information related to the protection of the children while the children stay in the class. Of course, such notification may be limited when it is determined that leaving work has been accomplished through the teacher's own location registration.

The parent terminal device 110 may receive periodic notifications or videos through an app service provided by the intelligent infant observation device 130 of FIG. 1 rather than accessing a homepage operated by a nursery school or the like. Of course, it is possible to receive notifications, images, and videos with edited time sections because of the limited time from when your child goes to school to discharge. Typically, the parent terminal device 110 captures a picture of their child for a certain period of time and confirms the transmitted image, and the radius of activity in the garden and a favorite object such as a book, a toy, a friend, a teacher, an object such as a meal It is possible to receive information related to the developmental characteristics of the child analyzed based on the

For example, when it is determined that their child is very interested in books and particularly prefers books in a specific field, the parent can transmit pre-stored characteristic information, ie, education data, necessary for parenting or education. In addition, the parent terminal device 110 allows you to check the activities in the hospital for your child, and, of course, it may be possible to check in real time by accessing a specific time, and if you stay in the blind spot for a long time, for example, you do not get along well with other children. In this case, it is determined as an abnormal situation, and the parent terminal device 110 may be notified of its state.

As such, the parent terminal device 110 may enable real-time location confirmation and monitoring of my child. It enables the parent terminal device 110 to check the behavioral patterns of infants or children (eg, intimacy with friends and teachers, utilization of books and teaching materials, etc.), and the path taken in kindergarten can be identified. Furthermore, it is possible to check the infant's abnormal situation (eg, body temperature) It can also be linked to a conversation. In other words, when parents receive a notification regarding their child, they can suggest appropriate responses to the teacher through a conversational channel such as an SMS message or chat. For example, when a notification is simultaneously transmitted to the terminal devices 105 and 110 of the teacher and the parent, that is, in such a situation, a conversation channel such as chatting is automatically created or opened on the screen so that the conversation can take place immediately. .

The communication network 120 includes both wired and wireless communication networks. For example, a wired/wireless Internet network may be used or interlocked as the communication network 120 . Here, the wired network includes an Internet network such as a cable network or a public telephone network (PSTN), and the wireless communication network includes CDMA, WCDMA, GSM, Evolved Packet Core (EPC), Long Term Evolution (LTE), Wibro network, etc. is meant to include Of course, the communication network 120 according to the embodiment of the present invention is not limited thereto, and may be used, for example, in a cloud computing network under a cloud computing environment, a 5G network, and the like. For example, if the communication network 120 is a wired communication network, the access point in the communication network 120 may connect to a switching center of a telephone company, etc., but in the case of a wireless communication network, it accesses the SGSN or GGSN (Gateway GPRS Support Node) operated by the communication company to transmit data. or by accessing various repeaters such as a Base Transmissive Station (BTS), NodeB, and e-NodeB to process data.

The communication network 120 may include an access point (AP). Here, the access point includes a small base station such as a femto or pico base station that is often installed in a building. Femto or pico base stations are classified according to the classification of small base stations, according to the maximum number of the photographing device 100, the teacher terminal device 105, and the like can be connected. Of course, the access point may include a short-distance communication module for performing short-distance communication such as Zigbee and Wi-Fi with the photographing device 100 and the like. The access point may use TCP/IP or Real-Time Streaming Protocol (RTSP) for wireless communication. Here, short-range communication may be performed in various standards such as Bluetooth, Zigbee, infrared, radio frequency (RF) such as ultra high frequency (UHF) and very high frequency (VHF), and ultra wideband communication (UWB) in addition to Wi-Fi. Accordingly, the access point extracts the location of the data packet, designates the best communication path for the extracted location, and forwards the data packet to the next device, for example, the intelligent infant observation device 130 along the designated communication path. The access point may share several lines in a general network environment, and includes, for example, a router, a repeater, and a repeater.

The intelligent infant observation device 130 may operate, for example, as a platform for providing infant observation services, and may be a server. For systematic management of data, the DB 130a may be included or linked. The intelligent infant observation device 130 is protected by the teacher by downloading the app to the teacher terminal device 105 or the parent terminal device 110 and installing it, or in the case of a parent, situation data related to their child in real time or periodically It is possible to receive To this end, the intelligent infant observation device 130 receives a photographed image from a photographing device 100 installed in an arbitrary space such as a nursery school or an edge device installed in a building of a nursery school to perform a primary image processing operation. Analyzes, recognizes faces of children in the captured image, and through this, real-time location check can be performed, and it can operate intelligently by identifying movement routes, analyzing and classifying behavior patterns, and judging abnormal situations. Here, the term "intelligent" may include enabling appropriate information to be automatically notified to the teacher terminal device 105 or the parent terminal device 110 in a specific situation as a result of image analysis.

Typically, the intelligent infant observation device 130 recognizes the faces of children in the captured image and confirms the real-time location based on this. Through the child's face recognition, metadata for a specific child is generated based on this, and the generated metadata can be analyzed to determine and predict children's behavior. Of course, when a specific behavior is predicted from a specific child, information about this may be transmitted to the teacher terminal device 105 . This can be predicted through data collection and analysis over a long period of time, and when programs such as artificial intelligence deep learning are applied, it is not difficult to predict what a specific child will do in a specific situation, so how much action will it take? any may be possible Above all, the intelligent infant observation device 130 may determine the time on a day when the child's behavior pattern is different from usual. For example, if there was active behavior on a particular day, compare today's behavioral analysis with the previous day's behavior. Therefore, if the child's behavior is judged to be the same as usual, the child's behavior is not judged abnormal. However, when the behavior is different from that of the previous day, it is possible to classify the behavior pattern and register data on the situation in which the behavior was exhibited.

In this process, the intelligent infant observation device 130 is a photographing device 100 installed in an arbitrary space and operates a thermal camera to receive and analyze the child's thermal image to check the child's body temperature. When a fever is detected as a result of the inspection, it can be notified to the terminal devices 105 and 110 carried by the teacher and the parent, and data related to these situations can be systematically classified and stored in the DB 130a of FIG. 1 . . In this way, in the case of a situation that the teacher and the parent need to know at the same time (eg, an emergency, etc.), the intelligent infant observation device 130 uses an automatic pop-up on the screen of the terminal device so that the teacher and the parent can be directly connected to a conversation. You can create or open a chat channel.

In addition, the intelligent infant observation device 130 may perform an editing function when transmitting a video to the parent terminal device 110 . In general, in the case of CCTV, a microphone (hereinafter referred to as a microphone) is not installed, so the voice cannot be heard. Of course, since it is possible to provide a service by installing a microphone in the photographing apparatus 100, the embodiment of the present invention will not be limited to any one form. However, it is explained on the premise that the microphone is not installed in consideration of the cost aspect, etc., and the intelligent infant observation device 130 generates voice contents based on the children's face recognition and mouth shape and processes them as subtitles to process the parent terminal device 110 ) can be provided. Of course, initially, a microphone can be used to check the language proficiency of each child, that is, to acquire spoken language data, and through a module such as voice recognition, it acquires and generates language data, such as vocabulary or sentences, that the child can speak. so you may be able to use it.

In other words, when an image according to a specific situation (eg, education, conversation between friends, etc.) occurring in the kindergarten and a voice received from the microphone are received, the intelligent infant observation device 130 as a server displays the faces of each child in the incoming image. After checking, the children's voices are processed as subtitles so that the video is edited for a certain period of time, and the server can deliver the edited video to the parents. The reason for processing with subtitles is that when multiple children are talking, the voice of a specific child cannot be confirmed, so the server checks the correct voice through the face and lips of a specific child, matches them, and displays the names and voices of the children as subtitles. In this way, the activity and participation level of a specific child can be communicated to parents through actual play, collaboration, and class, so that parents can grasp accurate information through their child's kindergarten activities.

It is not easy to estimate the audio content based on the shape of the children's mouths in the recorded video. In other words, the range of languages spoken by children may vary. Therefore, the intelligent infant observation device 130 may collect big data in relation to languages that children of 5, 6, and 7 years of age can speak in advance and store it in the DB 130a. In addition, since each child has a different language proficiency, a language dictionary, that is, a library customized for each child, may be created, and then the voice contents may be estimated based on this. For example, a voice library for child A may be generated by analyzing a captured image according to time change, and subtitles are generated based on the child's voice library when estimating the voice contents. In addition, by learning this process through deep learning such as big data generation and artificial intelligence, estimating the voice of a specific child in relation to the shape of the lips can increase the accuracy by that much.

More specifically, the language proficiency of child A, that is, the number of languages spoken may be 100. On the other hand, the language proficiency of child B, that is, the number of languages spoken can be 50. As such, the intelligent infant observation device 130 may generate a corpus (or language) dictionary specialized for each child, and of course, such a corpus dictionary may be generated based on the big data related to the age-specific language collected in advance. And by increasing the data of children A and B according to time change, big data related to the language proficiency of a specific child can also be generated. Through this process, it is possible to increase the accuracy related to the estimation of the children's voice contents, to generate the voice contents as subtitles based on this, and to provide a moving picture to the parent terminal device 110 .

In addition, the intelligent infant observation device 130 according to an embodiment of the present invention performs natural language processing through the voice recognition module of the microphone, but if it is difficult to distinguish a specific vocabulary, the voice content is estimated based on the shape of the child's mouth through the image The operation may be performed in various forms, such as Accordingly, the embodiment of the present invention will not specifically limit the format in which the caption information is generated.

As mentioned above, the intelligent early childhood observation device 130 may provide useful educational information to the parent terminal device 110 based on information related to a behavioral pattern of a child in a nursery school. Of course, in order to provide such educational information, related data may be collected and stored from educational institutions in advance, and then provided based on this. For example, the behavioral pattern of a child is analyzed based on intimacy with friends and teachers, and the use of books and teaching materials in the recorded video. And, it is possible to provide educational information, such as a recommendation to conduct an essay education to a child exhibiting such a behavior pattern, or to perform an education such as music or art. As described above, the intelligent infant observation device 130 according to an embodiment of the present invention analyzes the state of children and notifies the parents of the state information, but also provides educational information related to the educational direction of a specific child based on the behavioral patterns of children. It will be possible to expand platform services by providing them together.

FIG. 2 is a block diagram illustrating a detailed structure of the intelligent infant observation device of FIG. 1 .

As shown in FIG. 2 , the intelligent infant observation device 130 of FIG. 1 according to an embodiment of the present invention includes a communication interface unit 200 , a control unit 210 , an intelligent infant observation unit 220 , and a storage unit 230 . includes some or all of

Here, "including some or all" means that some components such as the storage unit 230 are omitted to configure the intelligent infant observation device 130, or components such as the intelligent infant observation unit 220 are included in the control unit 210 ), which means that it can be configured by being integrated with other components, such as, and will be described as including all in order to help a sufficient understanding of the invention.

The communication interface 200 communicates with each of the plurality of photographing devices 100 , the teacher terminal device 105 and the parent terminal device 110 via the communication network 120 of FIG. 1 . In the process of performing communication, operations such as modulation/demodulation, encoding/decoding, muxing/demuxing, etc. may be performed, which are obvious to those skilled in the art, and thus further description will be omitted. Above all, the communication interface unit 200 downloads and installs the app according to the request of the teacher terminal device 105 and the parent terminal device 110 in order to provide a service, tentatively called intelligent infant observation service, according to an embodiment of the present invention. can do.

The communication interface unit 200 receives the captured image from the plurality of photographing devices 100 or selectively from the thermal imaging camera and transmits the received image to the controller 210 . In addition, the communication interface unit 200 under the control of the control unit 210 notifies the child-related video or notification to the teacher terminal device 105 or the parent terminal device 110 according to the analysis result of the received captured image. Of course, the teacher terminal device 105 receives a notification related to children managed in a specific space of the daycare center or kindergarten, and the parent terminal device 110 receives a video or notification related to their child.

The communication interface unit 200 performs communication with the photographing device 100, for example, a CCTV or IP camera, according to the control of the control unit 210, and also selectively selects a thermal imaging camera in a specific situation according to the control of the control unit 210. can make it work For example, a first captured image of a CCTV or IP camera and a second captured image of a video camera selectively operated at a specific time may be contrasted with each other. Through this, for example, when a fever is detected in a child at a specific position in the second captured image, it is possible to quickly detect who the child is based on the first captured image. The communication interface unit 200 may participate in this operation.

The controller 210 is in charge of overall control operations of the communication interface 200, the intelligent infant observation unit 220, and the storage unit 230 of FIG. 2 constituting the intelligent infant observation device 130 of FIG. 1 . When the captured image is received from the communication interface unit 200, the controller 210 matches the identification information of the photographing device 100, stores the captured image in the storage 230, and then the intelligent infant observation unit 220 for image analysis. ) can be provided.

In addition, the control unit 210 controls the communication interface unit 200 to provide various data to the teacher terminal device 105 and the parent terminal device 110 according to the request of the intelligent infant observation unit 220 . For example, the control unit 210 may provide the child development characteristic information corresponding to the analysis result of the captured image to the parent at the request of the intelligent infant observation unit 220 . As a result of analyzing the behavior of a specific child, the child's parent terminal device 110 by extracting the characteristics of the child's development matching it based on the radius of activity in the hospital and favorite objects, that is, books, toys, friends, teachers, meals, etc. It can be helpful in raising or educating one's own children by providing them as

The intelligent infant observation unit 220 analyzes the captured image to observe the behavior (or activity) in a hospital, such as a daycare center, and operates intelligently based on the observation result. In other words, the intelligent infant observation unit 220 can track the action radius for each child and check the time when it disappears from the monitoring area. In addition, it is possible to identify books, toys, friends, etc. that are close to the child and analyze the time. These observations can be used to provide information about child development characteristics to the parents of a particular child. In addition, the intelligent infant observation unit 220 may notify the teacher or parent of this when the child stays in the blind spot for a long time.

In addition, the intelligent infant observation unit 220 may analyze a captured image of a thermal imaging camera received in a specific situation, or perform a contrast operation with a captured image provided from a CCTV or IP camera. In other words, when heat is detected in a child at a specific location in the image taken by the thermal imaging camera, it can be confirmed who the child is in the location through the captured image such as CCTV. In other words, by recognizing the face of the child at the location and notifying the teacher or parent that a specific child has a fever based on the recognition, intelligent operation can be performed in a specific situation. Of course, the intelligent infant observation unit 220 operates a thermal imaging camera based on this when it is determined that a specific child exhibits a behavior different from the previous one, that is, as a result of the analysis of the image, and measures the children's body temperature through this. Since it can be made, the embodiment of the present invention will not be particularly limited to any one form.

Above all, the intelligent infant observation unit 220 according to an embodiment of the present invention can enable the parent terminal device 110 to view the children's activities in a real-time video in real time. Of course, since the computational processing load is large, it is possible to provide an edited video at a specific point in time. It is also possible to do anything. Of course, events here can include incidents, accidents and situations that parents may like.

In addition, the intelligent infant observation unit 220 transmits caption information including caption data so that captions can be displayed on the screen when the video is provided to the parent terminal device 110 , and the captions are processed on the screen. Of course, this can be seen on the premise that audio is not acquired from a camera installed in an arbitrary space. However, it will not be particularly limited thereto. The intelligent infant observation unit 220 creates a corpus dictionary for each child based on the language-related big data collected in advance for each age of the children, and based on this, creates and provides subtitles based on the shape of the lips of a specific child in the image it may be possible to For example, in the case of whole sentences, that is, natural language processing, after recognizing voice data acquired from a microphone through a voice recognition module, it may be possible to judge difficult words or sentences based on the shape of lips in the image. Of course, since I have already explained enough about this, I will replace it with those contents.

The storage unit 230 may temporarily store various data processed under the control of the control unit 210 and then output it. The captured image may be stored and then output under the control of the controller 210, and after the data for generating a specified format in the intelligent infant observation unit 220 is stored, it may be systematically classified and stored in FIG. 1 .

In addition to the above, the communication interface unit 200, the control unit 210, the intelligent infant observation unit 220, and the storage unit 230 of FIG. 2 may perform various operations, and other details have been sufficiently described above, so the contents I want to replace them with

On the other hand, as another embodiment of the present invention, the control unit 210 may include a CPU and a memory, and may be formed as a single chip. The CPU includes a control circuit, an arithmetic unit (ALU), an instruction interpreter and a registry, and the memory may include a RAM. The control circuit performs a control operation, the operation unit performs an operation operation of binary bit information, and the instruction interpreter performs an operation of converting a high-level language into a machine language and a machine language into a high-level language, including an interpreter or a compiler. , the registry may be involved in software data storage. According to the above configuration, for example, at the beginning of the operation of the intelligent infant observation device 130, the program stored in the intelligent infant observation unit 220 is copied, loaded into a memory, that is, a RAM, and then executed to increase the data operation processing speed. can be increased quickly.

3 is a flowchart illustrating a driving process of the intelligent infant observation device of FIG. 1 .

Referring to FIG. 3 together with FIG. 1 for convenience of explanation, the intelligent infant observation device 130 according to an embodiment of the present invention is a photographing device ( 100) receives a photographed image (or a first photographed image) of an arbitrary space (S300). Of course, in this process, when the thermal imaging cameras are simultaneously operated, the thermal imaging camera may also receive the captured image (or the second captured image). Here, the thermal imaging camera may be selectively operated in certain situations. Here, the specific situation may mean a predetermined time interval or may mean a time when it is determined that a specific child exhibits a behavior different from usual or exhibits an abnormal behavior based on a captured image received from the CCTV.

In addition, the intelligent infant observation device 130 analyzes the received captured image and provides the analysis result related to the activity of the child to the terminal device 110 possessed by the parent (S320). Here, the analysis result determines the activity state (eg, location and behavior, etc.) of the children included in the captured image, and based on the determination result, an arbitrary time or It may include a captured photo or an edited video for the section.

In this process, the intelligent infant observation device 130 may transmit caption information together when the edited video is provided, and the caption information or caption data includes children's voice contents. Above all, a corpus dictionary customized for each child can be used to estimate the children's accurate speech content, and this corpus dictionary can be used by collecting big data related to the language of each age group in advance. In other words, if 100 pieces of data related to the language spoken by 7-year-olds were collected as big data, at this time, 7-year-old child A could speak 70, while child B could speak 50. Therefore, it can be estimated that the corresponding language is spoken based on the shape of the child's lips through the analysis of the captured image, and the accuracy of the estimation result can be further improved when an operation such as artificial intelligence deep learning is performed in the process. .

Also, child A speaks 70 languages, but can also speak a new language by learning a new language. In this case, it is recognized that a new language is spoken based on the shape of the lips in the image, and the new language is extracted from big data and brought to the corpus dictionary of child A. i.e. include In this way, it is possible to increase the accuracy of the prediction through learning, thereby increasing the accuracy of the audio content of the subtitle information.

In addition to the above, the intelligent infant observation apparatus 130 according to an embodiment of the present invention can perform various operations, and other detailed information has been sufficiently described above, so it will be replaced with the contents.

4 is a flowchart of another driving process of the intelligent infant observation device of FIG. 1 .

Referring to FIG. 4 together with FIG. 1 for convenience of explanation, the intelligent infant observation apparatus 130 according to an embodiment of the present invention receives a video from the photographing apparatus 100 of FIG. 1 ( S400 ). In general, the captured image receives video and audio data, but in the embodiment of the present invention, only video data can be received due to the characteristics of CCTV. Depending on the performance of CCTV, it can transmit 30 or 60 unit video frames per second.

The intelligent infant observation device 130 stores the video data in order in the video frame buffer, that is, the storage unit, and then retrieves and analyzes the image (S410, S420).

In addition, the intelligent infant observation device 130 may detect and classify an object (eg, a person object of a child or teacher, an object object such as a book, etc.) through image analysis and track it (S430, S440). Tracking may be performed based on a result of human face recognition or a feature point of an object.

The intelligent infant observation device 130 recognizes a face from the detected human object, and performs real-time location tracking of a specific object, that is, children based on the recognition result (S450, S455). Of course, the location tracking here can be utilized to calculate the child's action radius. If the action radius is wide, it may be judged to be an active child, and if there are many friends around it, it may be judged to be sociable. Also, if you immerse yourself in a book, you can judge that you like it.

The image analysis result as described above may be generated as metadata for a specific child, and the intelligent infant observation device 130 may perform behavioral analysis of children through statistics of such generated and accumulated metadata (S460). , S470).

As mentioned earlier, these statistical data can be used to determine the children's movement path, for example, whether the movement path leaves the filming area, for example, leaves a random space to go to the bathroom, and then returns. In addition, it is possible to identify and classify objects around the child, such as books or toys, and classify the child's behavioral patterns based on the judgment result (S481 to S485). Since this has been sufficiently explained above, we will replace it with those contents.

In addition, the intelligent infant observation device 130 may detect the child's heat by operating the thermal imaging camera when an abnormal situation different from usual is determined as a result of analyzing the child's behavior pattern in the captured image (S487) . The intelligent infant observation device 130 can perform thermal detection of all children by periodically operating the thermal imaging camera, but when an abnormality is found in the behavioral pattern of a specific child, it operates the thermal imaging camera at the time Thermal sensing may be performed, and since it may be variously designed according to the intention of a system designer, the embodiment of the present invention will not be limited to any one form.

In addition to the above, the intelligent infant observation apparatus 130 according to an embodiment of the present invention can perform various operations, and other detailed information has been sufficiently described above, so it will be replaced with the contents.

On the other hand, even though it has been described that all components constituting an embodiment of the present invention operate by being combined or combined into one, the present invention is not necessarily limited to this embodiment. That is, within the scope of the object of the present invention, all the components may operate by selectively combining one or more. In addition, although all of the components may be implemented as one independent hardware, some or all of the components are selectively combined to perform some or all functions of the combined components in one or a plurality of hardware program modules It may be implemented as a computer program having Codes and code segments constituting the computer program can be easily deduced by those skilled in the art of the present invention. Such a computer program is stored in a computer-readable non-transitory computer readable media, read and executed by the computer, thereby implementing an embodiment of the present invention.

Here, the non-transitory readable recording medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, memory, etc. . Specifically, the above-described programs may be provided by being stored in a non-transitory readable recording medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and it is common in the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Various modifications may be made by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: photographing device 105: teacher terminal device
110: parent terminal device 120: communication network
130: intelligent infant observation device 200: communication interface unit
210: control unit 220: intelligent infant observation unit
230: storage

Claims (8)

a communication interface unit for receiving a photographed image of an arbitrary space from a photographing device installed in an arbitrary space where children are active; and
Containing; including a;
The control unit determines the activity state of children included in the captured image, and provides a captured photo or edited video for an arbitrary section of the captured image to a terminal device carried by parents of children based on the determination result,
The control unit detects the body temperature of the child by operating the thermal camera as the photographing device when the abnormal situation of the child is determined as the determination result,
The controller is an intelligent infant observation device for inserting subtitles by using a language dictionary for each child generated based on big data on languages that can be spoken by each age of the child when inserting the subtitles. delete According to claim 1,
The control unit is an intelligent infant observation device for further providing characteristic information of the child's development to the terminal device based on the determination result by determining an action radius and a favorite object to be active in the arbitrary space as the activity state. According to claim 1,
The control unit, as a result of the determination, when it is determined that the child leaves the monitoring area of the photographing device and does not return even after a specified time, the intelligent infant observation device for selectively transmitting notification information to the terminal device of the child's teacher and parents. delete According to claim 1,
The control unit, an intelligent infant observation device for transmitting the edited video to the terminal device by inserting the contents of the children's conversations as subtitles. delete Receiving, by a communication interface unit, a photographed image of an arbitrary space from a photographing device installed in an arbitrary space where children are active; and
The control unit analyzes the captured image and provides an analysis result related to the activity of the child to the terminal device possessed by the parent;
determining, by the controller, the activity state of children included in the captured image, and providing a captured photo or edited video for an arbitrary section of the captured image to a terminal device carried by parents of children based on the determination result;
detecting, by the controller, the temperature of the child by operating the thermal camera as the photographing device when the abnormal situation of the child is determined as a result of the determination; and
inserting, by the control unit, a subtitle using a language dictionary for each child generated based on big data on languages that can be spoken by a child's age when inserting the subtitle;
Driving method of the intelligent infant observation device further comprising.

Images