KR102106905B1 - IoT doll system for controlling smart contents - Google Patents

Abstract

The present invention is a smart of a smart device that detects the movement of a doll coupled with an IoT device, manipulates virtual content displayed on the smart device, and provides feedback in the form of vibration or sound from the doll according to the situation occurring in the smart content. It relates to an IoT toy system that controls content, and includes an IoT toy that wirelessly communicates with a smart device, and a smart device that executes an application that includes smart content interworking with the IoT toy.

Description

IoT toy system for controlling smart contents {IoT doll system for controlling smart contents}

The present invention relates to an IoT toy system that controls the smart content of a smart device, and more specifically, to detect the movement of a toy that an IoT device is combined to manipulate virtual smart content displayed on the smart device and generate it from the smart content. It relates to an IoT toy system that controls smart content that provides feedback in the form of vibrations or sounds from toys according to the situation.

Recently, as the spread of smart devices such as smartphones and tablet PCs has expanded, opportunities for infants and toddlers to access such smart devices have increased, and accordingly, attempts to utilize smart devices as means of play or education for infants and toddlers are increasing. .

Nevertheless, it is still common for parents to prefer their children to play with traditional toys, such as dolls, robots, or blocks, which smart devices can offer a variety of play or educational means suitable for infants and toddlers through a variety of applications. Although there is an advantage, it is because exposure to electromagnetic waves for a long time may adversely affect a child, and there is a fear of over-immersion in content.

Accordingly, in recent years, research on so-called smart toys that link traditional toys with smart devices has been actively conducted.

Such a smart toy may provide a new play means that traditional toys such as dolls and robots could not provide, but the conventional smart toy merely displays some preset content through a smart device in response to manipulation of the traditional toy. Therefore, the children are easily fed up, and there is a problem that various contents that a smart device can provide cannot be properly utilized because only the one-way control between a traditional toy and a smart device is controlled.

Therefore, by controlling the smart content of the application executed by the smart device freely according to the operation of the traditional toy, and providing feedback according to the situation occurring in the smart content to the user through the traditional toy, various contents that the smart device can provide are efficiently It is required to have a technology for smart toys that can be utilized.

Republic of Korea Patent Publication No. 10-2017-0073047 (2017.06.28)

The present invention has been devised to solve the above problems, and an object of the present invention is to provide an IoT toy system that controls smart contents of a smart device according to the movement of the toy by interlocking the toy with the smart device using the Internet of Things (IoT). Is done.

In addition, the present invention aims to provide feedback to the user through toys according to a situation occurring on the smart content, not merely to manipulate the smart content.

The technical problem to be achieved by the present invention is not limited to the technical problem mentioned above, and various technical problems can be derived within a range obvious to a person skilled in the art from the contents to be described below.

IoT toy system for controlling smart content according to an embodiment of the present invention,

IoT toys that communicate wirelessly with smart devices; And a smart device that executes an application that includes smart content interworking with the IoT toy.

According to an embodiment, the IoT toy includes a doll part in which an IoT device is coupled inside or outside; And an IoT device unit including a wireless communication module in wireless communication with the smart device.

According to an embodiment, the IoT device unit may include a sensor unit that senses movement of an IoT toy and generates sensor data; And a feedback providing unit providing feedback to the user.

According to an embodiment, the sensor unit may include at least one or more of an acceleration sensor, a gyro sensor, or a vibration sensor.

According to an embodiment, the feedback providing unit may include a vibration motor and / or a speaker.

According to an embodiment, the smart device may control the smart content by receiving the sensor data from an IoT toy.

According to an embodiment, the smart device may filter the received sensor data using a deep neural network (DNN).

According to an embodiment, the deep neural network (DNN) may be a convolutional neural network (CNN) or a recurrent neural network (RNN) structure.

According to an embodiment, the smart device may include an application execution unit that executes an application including smart content; A display unit displaying the smart content; And a communication unit wirelessly communicating with the wireless communication module of the IoT toy.

According to an embodiment, the application execution unit may generate predetermined feedback information according to a situation occurring in the smart content.

According to an embodiment, the feedback information may include vibration information or sound information.

According to an embodiment, the vibration information may be generated when a character hits an obstacle or senses danger.

According to an embodiment, the sound information may include a greeting corresponding to the start or end of the smart content or a description of a specific target included in the content.

According to an embodiment, the feedback information may be generated when a certain time has elapsed after the execution of the smart content.

According to an embodiment, the vibration information and the sound information may be generated by varying the number of times, intensity, or size according to a situation that occurred in the smart content or a time elapsed after the smart content is executed.

According to an embodiment, the smart device may transmit the generated feedback information to the IoT toy, and the IoT toy may provide feedback to the user according to the received feedback information.

The present invention provides education and play means in which a doll, which is a traditional play equipment, and a smart device are interlocked, thereby preventing children from being overly immersed in the smart device.

In addition, the present invention can provide a new education and play means to children by allowing the smart contents to be freely manipulated according to the movement of the doll by linking the doll with the smart device using the Internet of Things (IoT).

In addition, the present invention can provide education and play means for children to have fun without feeling bored in a short period of time by providing feedback to the user through the doll according to the situation occurring in the smart content by manipulation of the doll. .

1 is a view showing an IoT toy and a smart device according to an embodiment of the present invention.
2 is a view showing smart content displayed on a display of a smart device according to an embodiment of the present invention.
3 is a diagram illustrating a structure of a deep neural network (DNN) for noise removal according to an embodiment of the present invention.
4 is a view showing a process of removing noise of sensor data using a deep neural network (DNN) according to an embodiment of the present invention.
5 is a diagram showing the structure of a deep neural network (DNN) for classifying motions according to an embodiment of the present invention.
6 is a flowchart illustrating a smart device control through an IoT toy and a feedback process according to one embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the embodiments allow the publication of the present invention to be complete, and general knowledge in the technical field to which the present invention pertains. It is provided to fully inform the holder of the scope of the invention, and the invention is only defined by the scope of the claims. The same reference numerals refer to the same components throughout the specification.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings commonly understood by those skilled in the art to which the present invention pertains. In addition, terms defined in the commonly used dictionary are not ideally or excessively interpreted unless specifically defined. The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, the singular form also includes the plural form unless otherwise specified in the phrase.

In the drawings, parts not related to the description are omitted in order to clearly describe the present invention, and the same reference numerals are attached to the same or similar elements throughout the specification. And, when a part is said to “include” a certain component, this means that other components may be further included instead of excluding other components, unless otherwise stated. In addition, "part" described in the specification means a unit or block that performs a specific function.

1 is a view showing an IoT toy 100 and a smart device 200 according to an embodiment of the present invention.

Referring to FIG. 1, an IoT toy system for controlling smart contents according to an embodiment of the present invention includes an IoT toy 100 wirelessly communicating with a smart device; And a smart device 200 that executes an application that includes smart content interworking with the IoT toy 100.

The IoT toy 100 includes a doll part 110 to which an IoT device is coupled, either internally or externally; And it may include an IoT device unit 120 including a wireless communication module 121 for wireless communication with a smart device.

The doll unit 110 may be coupled with an IoT device to control the smart content of the smart device 200 according to its movement. The doll unit 110 is a component that children directly contact and manipulate in using the IoT toy system that controls smart contents according to the present invention, and the outside is made of a soft texture material so that children can play safely. It is preferably made of a stuffed toy stuffed with cotton or vinyl inside. The external shape of the doll unit 110 may be an animal type or a human type character familiar to children, but is not limited to a specific shape and may be formed of various types of characters. A space for coupling the IoT device may be formed inside or outside the doll part 110.

The IoT device unit 120 is coupled to the inside or outside of the doll, detects the movement of the doll, and wirelessly communicates with the smart device 200 to transmit sensor data to the smart device 200 or feedback information from the smart device 200 Can receive.

To this end, the IoT device unit 120 according to an embodiment of the present invention includes: a wireless communication module 121 for wireless communication with a smart device; Sensor unit 122 for detecting the movement of the doll; It may include a feedback providing unit 123 for providing feedback to the user. In addition, although not shown in the drawing, the IoT device unit 120 according to an embodiment may further include a power supply unit for supplying power to the IoT device and a control unit controlling the IoT devices.

The wireless communication module 121 may wirelessly communicate with the smart device 200 at a short distance to transmit sensor data to the smart device 200 or receive feedback information from the smart device 200. The wireless communication module 121 may wirelessly communicate with the smart device 200 using Wi-Fi, Bluetooth, ZigBee, and broadband short-range communication. At this time, when the wireless communication module 121 controls the smart device by wirelessly communicating with any smart device according to the manipulation of the IoT toy, security-related problems may occur and the smart device desired by the user can be efficiently Since it is difficult to control, the wireless communication module 121 according to an embodiment may control only the specific smart device 200 by performing wireless communication only with the specific smart device 200.

The sensor unit 122 may detect the movement of the doll unit 110 to generate sensor data. To this end, the sensor unit 122 according to an embodiment of the present invention may include at least one of an acceleration sensor, a gyro sensor, or a vibration sensor.

The feedback providing unit 123 provides feedback to the user through the IoT toy 100 according to feedback information received from the smart device 200. The feedback information may be vibration level or sound information. To this end, the feedback providing unit 123 according to an embodiment of the present invention may provide feedback to the user in the form of vibration or sound, including at least one of a vibration motor or a speaker.

The wireless communication module 121, the sensor unit 122, and the feedback providing unit 123 of the IoT device unit are integrally coupled to the inside or outside of the doll unit 110 or formed inside or outside the doll unit 110. It can be formed to be detachable to the space.

Meanwhile, the smart device 200 according to an embodiment of the present invention includes an application execution unit 210 for executing an application including smart content; A display unit 220 for displaying the smart content; And it may include a communication unit 230 for wireless communication with the wireless communication module of the IoT toy.

The application execution unit 210 executes an application including smart content controlled according to the manipulation of the IoT toy. The smart content includes characters having a shape similar to an IoT toy operated by a user, so that the character is preferably controlled in response to the movement of the IoT toy, but is not limited thereto, and includes characters of various shapes or contents of various kinds. You can.

According to an embodiment, the application execution unit 210 may generate feedback information according to a situation occurring in smart content. The feedback information may include vibration information or sound information. The vibration information may be generated in response to a case where a character hits an obstacle or senses a danger as the smart content is controlled. At this time, the vibration information may be generated by varying the intensity and number of times depending on the situation occurring on the smart content. The sound information may include a greeting corresponding to the start or end of the smart content or a description of a specific object included in the content. The sound information may be generated in a foreign language as well as Korean.

In addition, the vibration information and sound information may be generated even when a certain time has elapsed after the execution of the smart content, thereby preventing over-immersion of children. The application execution unit 210 may generate feedback information by varying the number of vibrations, the intensity, or the sound volume according to the elapsed time after the smart content is executed.

The display 220 displays the smart content so that the user can visually recognize it.

The communication unit 230 wirelessly communicates with the wireless communication module 121 of the IoT toy 100 to receive sensor data from the wireless communication module 121, or to receive feedback information generated by the application execution unit 210 in the wireless communication module ( 121).

Meanwhile, the smart device 200 according to an embodiment may provide visual or sound information to the user separately from the IoT toy through the display unit 220 or a speaker provided inside the smart device.

As the smart device 200, a device such as a PC, a laptop, a smart phone, a tablet PC, or a smart TV may be used.

2 is a diagram showing smart content displayed on the display unit 220 of the smart device 200 according to an embodiment of the present invention.

Referring to FIG. 2, a rabbit-shaped character (hereinafter, character A) whose movement is controlled according to manipulation of an IoT toy and a dog-shaped character (hereinafter, character B) included in smart content are illustrated.

Character A is controlled based on the sensor data of the movement of the IoT toy according to the user's manipulation. For example, according to the movement of the IoT toy, a specific part of the smart content may be moved, or before, after, left and right, or an operation such as jumping may be performed.

When an event related to the character B included in the smart content occurs according to the movement or operation of the character A, the application execution unit 210 may generate feedback information and transmit it to the IoT toy 100. The feedback information may be vibration information indicating that a specific event has occurred or sound information for a description related to character B. The IoT toy 100 may provide feedback to the user in the form of vibration or sound according to the received feedback information.

In FIG. 2, the character B in the shape of a puppy is described as an example, but this is only an example for explanation, and it is needless to say that it may be a character or an object of any shape.

Meanwhile, since the RAW data generated by the sensor unit 122 and transmitted to the smart device 200 includes a lot of noise, a process of removing noise from the sensor data on the time axis may be necessary.

Accordingly, the IoT toy system for controlling smart content according to an embodiment of the present invention may filter sensor data using a deep neural network (DNN). The deep neural network (DNN) refers to a system or network that builds one or more layers in one or more computers to perform judgment based on a plurality of data.

3 is a diagram illustrating a structure of a deep neural network (DNN) for noise removal according to an embodiment of the present invention.

Referring to FIG. 3, a deep neural network (DNN) may include an input layer, one or more hidden layers, and an output layer. In the input layer, data for learning is input, and a result value calculated through the hidden layer and the result layer is compared with a ground truth, and the weight value is updated in reverse. After all the training is over, you can enter the information that requires prediction to get the result value.

According to an embodiment, an array between t-n and t + m based on the current frame time t may be input to an input layer of a deep neural network (DNN) for noise reduction. Here, n and m mean the number of frames. In the result layer, a single data at the current time t is output in response to the input value of the input layer. That is, when an array of sensor data between a certain time before and after the current frame time t is input to the input layer, a result value at the current time t is output correspondingly and replaced with data at the current time t. Noise can be removed. The calculation using the deep neural network (DNN) may be performed for each sensor, or values of all channels may be simultaneously input to obtain respective values.

The deep neural network (DNN) may be a convolutional neural network (CNN), a recurrent neural network (RNN), or an artificial neural network having a similar structure.

4 is a view showing a process of removing noise of sensor data using a deep neural network (DNN) according to an embodiment of the present invention.

4 (a) and 4 (b), among the arrangement of RAW data shown in FIG. 4 (a), an arrangement of data corresponding to a frame time from t-5 to t + 2 is performed through the deep neural network ( DNN) may be input to an input layer. Thereafter, as shown in FIG. 4 (c), a single result value corresponding to the current frame time t is output for data input to the input layer using a deep neural network (DNN), and FIG. 4 (d) By replacing the sensor data at the current frame time t using the output value as shown above, it is possible to obtain an array of sensor data from which noise has been removed.

Thus, by filtering the sensor data received by the smart device 200 to remove noise, it is possible to provide more stable and high-quality results in various situations and environments.

5 is a diagram showing the structure of a deep neural network (DNN) for classifying motions according to an embodiment of the present invention.

Referring to FIG. 5, the smart device 200 according to an embodiment of the present invention may recognize and classify the motion of a doll according to sensor data received using a deep neural network (DNN). The operation of the doll may be an operation such as forward, left and right rotation, jump or drawing, but is not limited thereto, and may include various operations.

The smart device 200 according to an embodiment inputs sensor data for each received channel into an input layer of a deep neural network (DNN), and learns in an initial hidden layer. By combining with, it is possible to classify what kind of action it is finally and output it in the output layer. The sensor data for each channel is data generated by the sensor unit 122 and may include X, Y, and Z rotation values and vibration values for a time axis.

6 is a flowchart illustrating a smart device control through an IoT toy and a feedback process according to one embodiment of the present invention.

Referring to FIG. 6, first, the application execution unit 210 of the smart device 200 executes an application including smart content, and connects the IoT toy 100 and the smart device 200 using wireless communication.

When the wireless communication module 121 included in the IoT toy 100 wirelessly communicates with any smart device located at a short distance to control the smart device, security-related problems may occur, and the smart device desired by the user may be generated. Since it is difficult to efficiently control, the wireless communication module 121 according to an embodiment of the present invention can control only the specific smart device 200 by performing wireless communication only with the specific smart device 200. To this end, the present invention specifies a smart device to be controlled by the IoT toy and performs a process of connecting the IoT toy and the smart device by wireless communication. In FIG. 6, a wireless communication connection is performed after an application is executed, but an application of the smart device 200 may be executed after the wireless communication connection is performed.

In this way, after the application of the smart device 200 is executed, and the wireless communication of the IoT toy 100 and the smart device 200 is connected, the sensor unit 122 of the IoT toy 100 is an IoT toy according to the user's operation. Generates sensor data by detecting the movement of the, and transmits it to the smart device 200 through the wireless communication module 121.

The smart device 200 removes the noise of the sensor data received using the deep neural network (DNN), and controls the smart content of the application using the filtered data.

The smart device 200 generates feedback information according to a situation occurring in the smart content and transmits it to the IoT toy. The feedback information may include vibration information or sound information. The vibration information may be generated in response to a case in which a character hits an obstacle or senses a danger as the smart content is controlled. At this time, the vibration information may be generated by varying the intensity and number of times depending on the situation occurring on the smart content. The sound information may include a greeting corresponding to the start or end of the smart content or a description of a specific object. The sound information may be generated in Korean or foreign languages.

In addition, the vibration information and sound information may be generated even when a certain time has elapsed after the execution of the smart content, thereby preventing over-immersion of children. At this time, the application execution unit 210 may generate feedback information by varying the number of vibrations, the intensity, or the sound volume according to the elapsed time after the smart content is executed.

The IoT toy provides feedback to the user in the form of vibration or sound according to the received feedback information.

As such, those skilled in the art to which the present invention pertains will appreciate that the present invention may be implemented in other specific forms without changing its technical spirit or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be construed that all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts are included in the scope of the present invention. do.

100: IoT toys 110: doll part
120: IoT device unit 121: wireless communication module
122: sensor unit 123: feedback providing unit
200: smart device 210: application execution unit
220: display unit 230: communication unit

Claims (10)

IoT toys that communicate wirelessly with smart devices; And
Includes; smart device for executing an application that includes the smart content interlocked with the IoT toys;
The IoT toy includes a doll part in which an IoT device is coupled to the inside or outside; and an IoT device part including a wireless communication module wirelessly communicating with the smart device.
The smart device may include an application execution unit that executes an application including smart content; a display unit that displays the smart content; And a communication unit for wireless communication with the wireless communication module of the IoT toy.
The IoT device unit includes a sensor unit that senses the movement of the IoT toy and generates sensor data; and a feedback providing unit that provides feedback to the user.
The smart device controls the smart content by receiving the sensor data from the IoT toy, but in order to filter the received sensor data using a deep neural network (DNN), among the arrangement of RAW data of the received sensor, t-5 The data array corresponding to the frame time from t to 2 is input to the input layer of the deep neural network (DNN), and then the data input to the input layer (input layer) using the deep neural network (DNN). Controlling smart content characterized by obtaining sensor data from which noise has been removed by outputting a single result value corresponding to the current frame time t, and replacing the sensor data at the current frame time t using the output result value IoT toy system.
delete delete According to claim 1,
The sensor unit comprises at least one of an acceleration sensor, a gyro sensor, a vibration sensor, IoT toy system for controlling smart content.
According to claim 1,
The feedback providing unit comprises a vibration motor and / or speaker, IoT toy system for controlling the smart content.
delete delete delete According to claim 1,
The application execution unit IoT toy system for controlling the smart content, characterized in that for generating a predetermined feedback information according to the situation occurring in the smart content.
The method of claim 9,
The smart device transmits the generated feedback information to the IoT toy,
The IoT toy provides feedback in the form of vibration or sound to the user according to the received feedback information,
The vibration information and sound information may be generated even if a certain time has elapsed after the execution of the smart content, IoT toy system for controlling the smart content.

Images