KR101248323B1 - Method and Apparatus for providing Ubiquitous Smart Parenting and Customized Education Service, and Recording medium thereof - Google Patents

Abstract

Monitoring the child's behavior patterns, generating text messages to be educated according to the observed behavior patterns, converting the text messages into voice messages, and converting the voice messages Disclosed is a ubiquitous intelligent parenting and customized educational service method comprising the steps of modifying prosody.

Description

Method and Apparatus for providing Ubiquitous Smart Parenting and Customized Education Service, and Recording medium

The present invention relates to a method and apparatus for Ubiquitous Smart Parenting And Customized Education (U-SPACE) that can take care of a child when the child is left alone, such as a dual-parent parent.

Today, with the advancement of robot technology, it is easy to see the different kinds of robots that help humans in our daily lives, enabled by the ubiquitous computing environment. These robots are particularly useful for people who need ongoing care, and assistant robots have emerged to help work in dangerous factory environments. In addition, mobile robots have been developed to assist the elderly by providing information about daily activities such as weather, TV programs, and hospital treatment schedules.

In this context as well, nowadays high interest in the ubiquitous computing environment has been concentrated in children's daily life. The ubiquitous intelligent child care service and the customized education service (U-SPACE service) proposed in the present specification are ubiquitous systems for caring for and caring for children who spend a lot of time living alone at home, such as door-key children. In addition, this system is designed with RFID technology to protect children from physical hazards, and finds ways to provide multimedia contents suitable for children through natural language processing techniques.

The present invention proposes a ubiquitous intelligent parenting service and a customized education service method, apparatus and recording medium.

In particular, the present invention provides a customized guidance message to protect children from various dangers occurring in the house, which detects and warns of dangerous objects and situations in consideration of children's location information and behavior patterns. In addition, the present invention provides a ubiquitous intelligent parenting service and a customized education service method, apparatus, and a recording medium that provide a voice message so that the guide message can be felt by a voice expressing human emotion by an emotional voice generation system.

In order to achieve the above object, an embodiment of the ubiquitous intelligent parenting and customized education service method comprises the steps of monitoring a child's behavior pattern; Generating a text message to be educated in accordance with the observed behavioral pattern; Converting the text message into a voice message; And modifying the prosody of the converted voice message according to the sentence type category of the generated text message.

On the other hand, one embodiment of the ubiquitous intelligent parenting and customized education service device to achieve the above object is a sensor for monitoring the behavior pattern of the child; A message generator for generating a text message to be educated according to the observed behavior pattern; A TTS converter for converting the text message into a voice message; And an emotional prosody modification module that modifies the prosody of the converted voice message according to the sentence type category of the generated text message. To provide.

Meanwhile, in order to achieve the above object, an embodiment of the ubiquitous intelligent parenting and customized education service device includes at least one touch sensor for interacting with a child; An RFID tag identifying the child's identity information, location information, and repetitive behavior patterns; And a communication module for receiving an emotional rhyme corrected voice message according to the repetitive behavior pattern from an external server through the wireless Internet, a PDA having a speaker and a touch screen for outputting the received emotional rhyme corrected voice message. Provide customized educational service devices.

Furthermore, the present invention provides a recording medium on which a program for executing the ubiquitous intelligent parenting and customized education service method is recorded.

According to the ubiquitous intelligent parenting service and the customized education service method, the message may be customized by the user according to the child's location information and the child's repetitive behavior pattern, and the text guide message may be provided with the emotional rhyme correction system of the present invention. By voiced by a commercial text-to-speech engine (TTS engine), messages can be modified into voice messages that express anger, fear, happiness, sadness, and neutral emotional states to more effectively warn children of danger and lead to safe behavior. .

In addition, the emotional rhyme correction system can play a role of a parent who reads fairy tales to the child as part of the home schooling service.

FIG. 1 is a diagram illustrating various fields in which ubiquitous intelligent parenting service and customized education service (U-SPACE service) may be utilized.
2 is a flowchart illustrating a ubiquitous intelligent parenting service and a customized education service method.
3 is a time table illustrating a time window of an attention span used to detect repetitive behavior patterns of a child.
4 is an exemplary diagram illustrating a process of generating a modified guide message from an initial guide message through a language type analyzer and a language sentence type identification system.
FIG. 5 is a diagram illustrating pitch, pause, duration, and intensity, which are components that play a major role in emotional speech generation.
FIG. 6 is a reference table showing the K-ToBI labeling system, accent verse tones, and intonation passage boundary tones used in the pitch contour analysis method.
FIG. 7 is a table showing numerical values of the influence of the parameters of the prosody structure and numerical values of the pattern of the tonal curve in each emotional state.
FIG. 8 is a functional block diagram illustrating a configuration of an emotional rhyme correction system operating in a process of outputting an emotional voice message from an input text message.
9 is a diagram illustrating a scenario example of a bear-shaped prototype and a dangerous situation in which the ubiquitous intelligent parenting service and the customized education service device of the present invention are installed.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected to or connected to the other component, It should be understood that an element may be "connected," "coupled," or "connected."

FIG. 1 is a diagram illustrating various fields in which ubiquitous intelligent parenting service and customized education service (U-SPACE service) may be utilized.

For children with no parents or guardians, U-SPACE in a ubiquitous computing environment can play many roles in many areas. For example, you can prevent accidents by alerting you to dangerous situations that are not recognized by young children. You can also teach them how to use household appliances that your child is not familiar with. It can provide content that can be used for various infant therapies, and can also manage the child's schedule.

Hereinafter, according to an embodiment of the present invention, the intelligent parenting service and the customized education service will be described in relation to the role of protecting and caring for the child in the utilization area of the U-SPACE.

2 is a flowchart illustrating a ubiquitous intelligent parenting service and a customized education service method.

Referring to FIG. 2, the ubiquitous intelligent parenting service and the customized education service method according to the present invention include monitoring 210 a behavioral pattern of a child and generating a text message to be educated according to the observed behavioral pattern 220. And converting the text message into the voice message 230 and modifying the prosody of the converted voice message according to the sentence type category of the text message.

Looking specifically step by step, the present invention is first to monitor the behavioral pattern of the child (210), for example, in the case of a five-year-old child according to several reports have been studied to be easy to be associated with various accidents, so the present specification Primarily targets five-year-old children to identify repetitive and potentially dangerous behavioral patterns of children against dangerous objects such as hot water dispensers in water purifiers, stoves, wall sockets, and glass windows. At this time, the action that occurs repeatedly at least two or more times in a predetermined time window is detected. The temporal range of the time window mentioned above will be described with reference to FIG. 3.

FIG. 3 is a time table illustrating a time window of an attention span 310 used to detect repetitive behavior patterns of a child.

 Attention span is the time during which a person can concentrate on a task without being distracted. In general, for young children, the normal attention span is considered to be three to five minutes per year, depending on the age of the child. Thus, for a five year old child, 15 to 25 minutes is the time window set as the attention period.

In FIG. 3, when the child's predetermined behavior (eg, touching the gas range) is referred to as the first action 320, if the child's behavior is touched again within the attention period 310, this is repeated. 330 is detected, and in this case, when the action of touching the gas range outside the caution period 310 is performed, a separate sanction is required as compared to the new action 340. As will be described below, in this case, the text message notified to the child is modified to a stronger sentence type sentence type, and the emotional state of the voice in the voice message is also modified to be expressed in an angry tone.

Referring back to FIG. 2, the location information of the child may be monitored together with the monitoring of the behavior pattern. In other words, in order to detect a dangerous situation that has occurred in a child, it is necessary to monitor where the child is doing what.

Next, a text message is generated according to the monitored behavior pattern (220). For example, if the child repeatedly touches the gas stove continuously within a certain time, the child generates and provides a text message of warning, but instead of providing the existing message as it is, After making the corrections, you provide a message that is more context-sensitive. Regarding such a message modification operation, it will be described in more detail with reference to FIG. 4.

4 is an exemplary diagram illustrating a process of generating a modified guide message from an initial guide message through a language type analyzer and a language sentence type identification system.

According to one embodiment of the present invention, a predetermined number (eg, 45) of sample messages are first collected from a group of professionals having a job of caring for a child, and then appropriately selected from among the sample messages according to dangerous target objects detected in the vicinity of the child. Select the guidance message. Here, the detailed expression and sentence type of the child's name, location information, etc. will be modified later.

Referring to FIG. 4, there is shown a process of generating a message when the withdrawal shows a behavior pattern of touching the gas range.

First, a sentence 410 suitable for the observed behavior pattern is first selected from previously collected sample sentences as an initial guide message. Then, the child's name and location information are corrected through a Korean morphology analyzer 420, and the sentence type is corrected through a Korean sentence type identification system 430. .

The language sentence type identification system 430 classifies categories of sentence types into one of declarative, imperative, suggestive, interrogative, and exclamatory according to sentence ending and punctuation. Can be.

In FIG. 4, Alice was withdrawn, the outlet was changed to a gas range, and the sentence type was changed from recommended to imperative due to the continued repetitive behavior of the withdrawal. This process is performed by the system for detecting behavior patterns (440).

Accordingly, the modification guide message finally displayed due to the repetitive behavior of the withdrawal in FIG. 4 is generated and provided as a sentence 450 of the statement type.

Referring back to FIG. 2, when the modified guide message is generated, it is converted into a voice message through the TTS system (230). In this case, the TTS system may use a system that is used universally.

Finally, step 240 of modifying the prosody of the voice message according to the sentence type category is performed.

In linguistics, rhyme is a term encompassing tonality, intonation, accent, rhythm, and sound field. Rhymes refer to the features of sound that make a difference in meaning in addition to the characteristics of phonemes, which are each segmented sound. That is, the rhyme may vary depending on the emotional state of the person.

In step 240 of modifying the prosody of the voice message, the prosody of the voice message may be modified by at least one of a change in the duration and pause length of the sound, the intensity of the sound, and the height of the sound.

FIG. 5 illustrates pitch 510, pause 520, duration 530, and intensity 540 as components that play a major role in emotional speech generation. ).

That is, in modifying the rhythm of the voice message to generate an emotional (emotional, emotional) voice message, by modifying the key components shown in Figure 5, it is possible to express emotions of different emotional states together in the message despite the same message. have.

FIG. 6 shows K-ToBI labeling system, Accentual Phrase Tones, and Intonation Phrase Boundary Tones used in the pitch contour analysis method. This is a reference table.

In this specification, a K-ToBI labeling system using a speech corpus distributed by the Speech Information Technology and Industry Promotion Center for analysis of the emotional rhyme structure for Korean characters is presented. Used.

For reference, K-ToBI (Korean Tones and Break Indices) is a rhythm notation that applies to Korean. Rhyme here is an intonation that expresses one's feelings when a person speaks, stress that appears in a stressed part of a word or sentence, a break that stops for a moment, or stops for a moment while waiting for the other's attention. Pause, etc. Since these rhymes do not have pronounced pronunciations like consonants or vowels, ToBI standardizes these rhymes.

For example, we might pronounce words or sentences strongly at very high pitches when feelings are full, or weakly pronounce them at very low pitches when grass is dead.

In acoustic / phonology, the intonation changes can be seen by the frequency of the vocal cords called pitch. Strongly pronounced portions appear at high pitch values, and weakly pronounced portions appear at low pitch values. These values, which can be seen as pitch values or curves, are constantly changing within a sentence or dialogue and it is difficult to see clearly what type the measured values represent.

Therefore, according to the convention of ToBI, the pitch is marked with a symbol such as H for high pitches and L for low pitches, * for special accents, and% for accent boundaries. When these conventions are combined, H + L can be briefly represented by strong pronunciation followed by weak pronunciation, and L + H shows the change from weak pronunciation to strong pronunciation. When H +! H *, you will see that you are descending from a high place to a stressed low syllable, and! Represents a step down.

FIG. 7 is a table showing numerical values of the influence of the parameters of the prosody structure and numerical values of the pattern of the tonal curve in each emotional state.

In FIG. 7, the emotional states are divided into four categories: anger (anger), fear (fear, fear), happiness, sadness, and tonality in the structure of general rhyme for each emotional state. Effects of pitch, intensity, speech rate, and pause length are shown in FIG. 7A.

On the other hand, as shown in the thick boxed figure in Fig. 7 (b) in which neutrality is calculated as a standard for pitch contour patterns, it is as high as HL% in the case of anger. The pattern that changes from pitch to low tones is dominant (see Figure 6 (c)), the pitch goes up as high as H% in the case of fear, and the high to low pitch as LH% in the case of happiness. The pattern changes to pitch, and in the case of sadness, the low pitch as L% is the dominant pattern.

In FIG. 7 (b), such a value may be interpreted as a pattern in which an intonation tone having a value of two or more according to the residual adjusted by the calculation is dominant for the emotional state.

8 is a functional block diagram illustrating a configuration of an emotional rhyme correction system that is operated in the process of outputting an emotional voice message from an input text message.

Referring to FIG. 8, first, an input text message 810 is vocalized, where the text message has a repeated behavior such as, for example, the child continuously touches the gas range within a certain time. According to the sentence type (sentence type) is a modified text message.

First, the input text message 810 is voiced through the TTS system 820, and after the completion of the basic voice work, the voice message with the rhyme modified is finally output through the emotional rhythm correction system 840.

That is, the voice message which is in a neutral emotional state via the TTS system 820 is converted into a pitch contour mapping function module 841 and a general prosody modification function in the emotional rhythm correction system 840. After passing through 842, an emotional voice message 850 is finally generated that expresses an emotional state according to the sentence type category of the text message.

The pitch contour mapping function module 841 includes a sentence ending syllable correction unit reflecting different intonations of a pitch contour correction and sentence type composed of a linear function, a sine function, and an exponential function. and a sentence final syllable modification module, wherein the general prosody modification function 842 modifies the emotional speech by modifying at least one or more of the overall tonality, strength, and speed of the speech message. You can output

9 is a diagram illustrating a scenario example of a bear-shaped prototype and a dangerous situation in which the ubiquitous intelligent parenting service and the customized education service device of the present invention are installed.

That is, referring to FIG. 9 (a), as a ubiquitous intelligent parenting and customized education service device, at least one or more touch sensors for interaction with a child, an RFID tag for identifying the child's identity information, location information, and repetitive behavior patterns, and an external server Appears from a doll-shaped service robot including a communication module capable of receiving an emotional rhyme corrected voice message over the wireless Internet, a speaker for outputting the received emotional rhyme corrected voice message, and a PDA equipped with a touch screen. have.

As shown in the figure, the appearance of the service robot can be a model of a teddy bear, in order to provide a safe and comfortable feeling for the children.

On the other hand, RFID (Radio-Frequency Identification) technology, which identifies the child's identity information, location information, and repetitive behavior pattern, is a technology that recognizes information over a long distance by using radio waves, which includes an RFID tag and an RFID reader (receiver). need.

The tag consists of an antenna and an integrated circuit, which records information in the integrated circuit and sends the information to the reader via the antenna. This information is used to identify the tagged object. For example, it functions like a bar code, and what makes RFID different from a bar code system is that it uses radio waves instead of light. This allows tags to be read over long distances, not just at short distances like barcode readers, and even to receive information through objects between transmitters and receivers.

RFID can be categorized as a power to be used again. RFID that reads and communicates information on the chip only by the power of the reader is called passive RFID. Semi-passive RFID has a built-in battery in the tag. It uses the power to read the information and the power of the reader to communicate. Finally, active RFID uses the power of tags to both read the information on the chip and communicate that information.

On the other hand, RFID is also classified into the frequency of radio waves used for communication instead of power. RFID using low frequency is called Low-Frequency Identification (LFID) and uses 120 ~ 140 khz radio waves. High-Frequency Identification (HFID) uses 13.56 Mhz, and UHFID (Ultra High-Frequency Identification), which uses higher frequency, uses radio waves in the 868-956 Mhz band.

Referring to FIG. 9 (b), FIG. 9 (b) is an example of a scenario representing a dangerous situation, showing when a young child wants to operate a dispenser of hot water of a water purifier. 9 (b) shows a graph of the frequency and intensity of the event reported by the force sensor. In this case, the automatically generated customized message is vocalized by the emotional voice message generation system described above, and then wirelessly transmitted to the PDA installed in the U-SPACE service robot and provided to the child.

According to the ubiquitous intelligent parenting service and the customized education service method of the present invention described above, it is possible to provide a user-customized message according to the child's location information and the child's repetitive behavior pattern, wherein the text guide message is emotional By rhyming by a rhyme correction system and a commercial text-to-speech engine, a voice message expressing the human emotional state can be modified to more effectively warn children of danger.

The present invention includes a recording medium on which a program that implements a ubiquitous intelligent parenting and customized education service method is recorded.

Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage, and the like, and also in the form of carrier waves (for example, transmission over a network such as the Internet). It also includes implementations. The computer readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner.

In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. In other words, within the scope of the present invention, all of the components may be selectively operated in combination with one or more. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. Codes and code segments constituting the computer program may be easily inferred by those skilled in the art. Such a computer program may be stored in a computer readable storage medium and read and executed by a computer, thereby implementing embodiments of the present invention. As the storage medium of the computer program, a magnetic recording medium, an optical recording medium, a carrier wave medium, or the like may be included.

It is also to be understood that the terms such as " comprises, "" comprising," or "having ", as used herein, mean that a component can be implanted unless specifically stated to the contrary. But should be construed as including other elements. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

Claims (15)

In the ubiquitous intelligent parenting and customized education service method of the ubiquitous intelligent parenting and customized education service device,
The education service device
Monitoring the child's behavioral patterns and locations;
Generating a text message to be educated in accordance with the observed behavioral pattern;
Converting the text message into a voice message; And
Modifying the prosody of the converted voice message according to the sentence type category of the generated text message,
Generating the text message
Selecting a sentence from one of sample sentences collected in advance according to the observed behavior pattern;
Changing at least one of name and location information that is a component of the selection sentence in accordance with the name of the child and the observed position; And
Identifying the number of repetition of the behavior from the observed behavior pattern, and accordingly modifying the sentence type of the selected sentence to generate the text; ubiquitous intelligent parenting and customized education service method comprising a.
The method of claim 1,
Monitoring the behavior pattern,
Ubiquitous intelligent parenting and customized education service method, characterized in that for detecting a plurality of times the behavior repeatedly occurring within a predetermined time window (time window).
The method of claim 2,
Monitoring the behavior pattern,
Ubiquitous intelligent parenting and customized education service method characterized in that for monitoring the location information of the child with the behavior pattern.
The method of claim 3, wherein
The time window (time window),
Ubiquitous intelligent parenting and tailored education service method, characterized in that for determining the range of time according to the attention span (attention span) according to the age of the child.
delete The method of claim 1,
The sentence type category is any one of description, command, recommendation, question and admiration ubiquitous intelligent parenting and customized education service method.
The method of claim 1,
Modifying the prosody of the voice message,
Ubiquitous intelligent parenting and customized education service method, characterized in that the rhythm of the voice message according to the emotional state by modifying at least one or more of the duration and pause length of the sound, the intensity of the sound, the height of the sound.
The method of claim 7, wherein
Modifying the rhyme of the voice message,
Ubiquitous according to the K-ToBI (Korean Tones and Break Indices) protocol, and determining the dominant pattern for any one of the plurality of emotional states according to the residual adjusted by the operation. Intelligent parenting and tailored educational service method.
The method of claim 8,
The plurality of emotional states are any one of anger, fear, happiness, sadness and neutral ubiquitous intelligent parenting and customized education service method.
The method of claim 9,
Modifying the rhyme of the voice message,
Modifying a pitch contour of the voice message; And
Ubiquitous intelligent parenting and personalized education service method further comprising the step of reflecting different accents of the sentence type through a sentence final syllable modification function according to the sentence type.
11. The method of claim 10,
Modifying the rhyme of the voice message,
Ubiquitous intelligent parenting and customized education, characterized in that for modifying at least one or more of the overall tone, strength, speed of the voice message by modifying the rhyme of the voice message to output an emotional speech (emotional speech) Service method.
In the ubiquitous intelligent parenting and customized education service device,
Sensors to monitor child's behavioral patterns and locations;
A message generator for generating a text message to be educated according to the behavior pattern observed through the sensor;
A TTS converter for converting the text message generated by the message generator into a voice message; And
And an emotional prosody modification module that modifies the prosody of the converted voice message according to the sentence type category of the generated text message converted by the TTS conversion unit.
The message generating unit
After selecting one of the pre-collected sample sentences according to the observed behavior pattern, and changing at least one of the name and location information that is a component of the selection sentence according to the child's name and the observed position, and the observed behavior Ubiquitous intelligent parenting and customized education service device characterized in that the number of repetition of the action is determined from a pattern, and the sentence type of the selected sentence is modified accordingly to finally generate the text.
13. The method of claim 12,
The emotional rhyme correction unit,
A tonal curve mapping function having a pitch contour correction composed of a linear function, a sine function, and an exponential function and a sentence final syllable modification module reflecting different intonations of sentence type; And
And a general prosody modification function module for modifying the rhythm of the voice message to output an emotional speech by modifying at least one or more of the overall tone, strength, and speed of the voice message. Featuring ubiquitous intelligent parenting and customized educational service devices.
delete A recording medium on which a program that implements the ubiquitous intelligent parenting and customized education service method according to any one of claims 1 to 4 and 6 to 11 is recorded.

Images