KR20220105967A - AR interactive education system based on paper toy - Google Patents

Abstract

The present invention relates to an AR interactive education system based on a paper toy, which consists of: a teaching tool made by manually assembling component puzzles; and a smartphone displaying the teaching tool in augmented reality on a screen. A material of the teaching tool is wood, which is a paper material. According to the present invention, the component puzzles are provided to assemble an object by utilizing a wood material such as balsa wood, the texture of a 3D model is applied in real time using coloring and AR technology, and the object can be controlled using Bluetooth communication through scratch coding, and thus the AR interactive education system has a remarkable effect that can be used for creative education.

Description

AR interactive education system based on paper toy

The present invention relates to a paper toy-based AR interactive education system, and more particularly, by using art, craft, AR (augmented reality technology), and IOT technology, creative education for infants and young children, dementia in the elderly, prevention of mental illness, and developmentally disabled children It is about a paper toy-based AR interactive education system that can be used as education for children.

In the era of the 4th industrial revolution, the education field is rapidly changing from the existing e-learning industry to the Edu-Tech industry due to the rapid development of ICT technology. The industry is replacing the demand of the e-learning industry.

And as an example of prior patent technology, in the registered patent publication No. 10-2145748,

A coloring book in which at least one functional image is developed around the main image as the main image as the main image as the main image;

It is linked with the coloring book through wireless communication, and when the coloring book is recognized at a predetermined interval, the function is expressed in a dynamic state from a statically developed functional diagram in the main drawing of the coloring book, and the main image and functional formula It includes a smart device in which images are implemented by interlocking with each other, and the smart device is a photographing module that recognizes an image designed in a coloring book, and extracts the image recognized by the photographing module, When the image extracted by the image extraction module and image extraction module is selected to extract all colored areas, the main image and the functional image are combined and implemented in a dynamic state, or the main image and the functional image independently realize movement. , the background includes an augmented reality implementation module that projects the reality as it is and outputs an object on the actual background, and the functional image implemented in the smart device is automatically combined with the main image or selectively combined with the main image by the user's selection a specific function can be performed, the functional image is moved to the position of the main image, and a specific function is realized in a state where the functional image and the main image are combined. When positioned, the image is recognized and the first text related to the subject and content corresponding to the recognized image is output, and when all the developed functional images realize the function, the display of the smart device includes the conclusion to be conveyed on the page The implementation of the animation is completed as the second text, which is the summarized text, is output, and the coloring book is given a unique identification code for each page so that the smart device identifies each page, and the unique identification code is a unique patterned form. Formed and affixed or printed as a sticker on a coloring book

A system for providing a coloring book implementation service based on augmented reality including

In addition, the Registration Patent Publication No. 10-2005437 includes a plurality of first images 11 having a color, and a QR code 15 to recognize information on the first image 11 .

a coloring book 10 to do; An application 21 that includes the information of the first image 11, recognizes the QR code 15 using a provided camera, and provides augmented reality content corresponding to the first image 11 by interworking This is configured, and after acquiring the first image 11 through the camera, a smart terminal 20 that provides the corresponding augmented reality content;

In the augmented reality-based coloring book system consisting of One or a plurality of second images 13 having a white background are configured and included in the information of the QR code 15 , and the application 21 includes the first image 11 and the second image 13 . , after recognizing any one of the images of the third image 14 colored by the user and outputting the corresponding augmented reality content, the smart terminal 20 at the output final position deviates from the position, A direction indication guide portion (21a) for indicating the direction of the corresponding position,

For psychological analysis of children, the color psychological analysis guide 21b, which is a standard for psychological analysis, and any one image among the first image 11, the second image 13, and the third image 14 are recognized. An augmented reality content providing unit 21c for outputting augmented reality content corresponding thereto, analyzing the color after recognizing the third image 14, and providing augmented reality content coated with a color corresponding thereto; 3 After recognizing the image 14, an analysis unit 21d that grasps the color distribution map and compares and analyzes it with the color psychological analysis guide 21b, and the psychological state of the user is analyzed by the analysis unit 21d A storage unit 21e that stores the data (d) so that it can be viewed in the form of an illustrated book including the corresponding date and time, and the user selects the data (d) stored in the storage unit 21e, including the corresponding date and time and a psychological analysis system using an augmented reality-based coloring book for children, characterized in that the psychological state result providing unit 21f that provides the result (p) in the form of text or image so as to check the psychological state is disclosed.

In addition, Laid-Open Patent Publication No. 10-2020-0077049 discloses an image receiving unit for receiving an image of an object photographed from the photographing unit;

an image processing unit that extracts feature points of the object from the image, identifies the object by comparing it with reference feature points stored in advance, and creates an augmented reality image by superimposing information on the object on the image; and

An apparatus for providing augmented reality of a markerless method, including an image output unit for outputting the augmented reality image to a display unit, is disclosed.

In addition, Korean Patent Laid-Open Publication No. 10-2017-0134277 discloses: 1) executing a service app in a user terminal, selecting an image to be colored from the server, and downloading;

2) coloring the downloaded image with a specific pattern using a tool provided by the service app;

3) printing the colored image;

4) Execute the service app and scan the printed image of the colored image to recognize the marker and

extracting color information;

5) generating colored augmented reality content based on an image colored by a user by coloring the augmented reality content provided from the server according to the extracted color information; and

7) Displaying the color augmented reality content on the display unit of the user terminal; Augmented reality service method for coloring game, characterized in that it includes. However, the prior arts still had problems in that it was difficult to apply the texture of the 3D model in real time by using AR technology, and it was inconvenient to control the object, so that it was difficult to utilize it for creative education.

Therefore, the present invention has been devised to solve the above problems, and provides it as a part puzzle so that any object can be assembled using a wood material such as balsa wood, and 3D using the coloring and AR technology This is to provide a paper toy-based AR interactive education system that can be used for creative education by applying the texture of the model in real time and providing control over the object through Bluetooth communication through scratch coding.

The present invention relates to a paper toy-based AR interactive education system, and a teaching aid manufactured by manually assembling part puzzles;

It is characterized in that it consists of a smartphone that displays the teaching aid in augmented reality on the screen.

The material of the parish is characterized in that the paper material, Wood (Wood).

Therefore, the present invention provides as a part puzzle so that any object can be assembled using a wood material such as balsa wood, coloring, applying the texture of the 3D model in real time using AR technology, and scratch coding By providing a way to control objects through Bluetooth communication, there is a remarkable effect that can be used for creative education.

1 is an embodiment screen to which the AR coloring technology of the present invention is applied.
Figure 2 is an AR coloring technology vehicle teaching application configuration diagram of the present invention
Figure 3 is a picture of the assembled vehicle teaching aid of the present invention;
Figure 4 is a vehicle assembly picture of the present invention
5 is a picture of 4DC motor, Bluetooth RC car, and Arduino board parts of the present invention
6 is a picture of a dedicated board for 4WD support of the present invention
7 is a photo of an embodiment of controlling the teaching material through Bluetooth communication of the present invention

The present invention relates to a paper toy-based AR interactive education system, and a teaching aid manufactured by manually assembling part puzzles;

It is characterized in that it consists of a smartphone that displays the teaching aid in augmented reality on the screen.

The material of the parish is characterized in that the paper material, Wood (Wood).

In addition, the teaching aid is installed with a controller, the controller is a dedicated board device comprising an input/output unit, a Bluetooth communication unit, a motor drive unit, and a control unit, characterized in that the smart phone and Bluetooth communication is possible.

In addition, the smartphone app is characterized in that it consists of an image receiving unit, an image processing unit, an augmented reality realization unit, and a control unit.

The present invention will be described in detail with reference to the accompanying drawings as follows.

The present invention provides a part puzzle so that any object (ex. car) can be assembled by using a wood material such as balsa wood, and the texture of the 3D model is applied in real time by using coloring and AR technology. It is an invention related to a creative teaching aid & mobile application that can be utilized for creative education by providing a way to control objects through Bluetooth communication through scratch coding.

The AR coloring technology applied to the present invention is a technology that creates your own 3D model by generating the texture of the 3D model in real time using the AR coloring technology.

The present invention is a Bluetooth communication APP through a four-wheel drive dedicated board. By providing a dedicated board in which I/O devices, Bluetooth, and motor drive are integrated, it is easy to expand functions when operating a vehicle module and applying other sensors through Bluetooth. .

And since it supports a web-based block coding language, it can be produced with Arduino control and Android apps.

By providing the coding education platform Codenori based on the block-type programming language Scratch, it is possible to implement the function to control the teaching aids, and the program developed in the block coding language is automatically generated as an Android executable file Apk for mobile devices. can control the parish.

An application that provides a parish implementation service is installed on the smartphone, and the parish is linked through wireless communication, so that the parish is recognized when the parish is photographed at a predetermined interval. The smartphone recognizes the colored teaching materials so that the statically developed image can be expressed in a dynamic form.

In the process of recognizing the teaching aid by the camera, text related to the subject and content corresponding to the recognized image may be output on the screen of the smartphone. This means that text is output when you bring your smartphone to the parish.

In other words, the teaching aid is given a unique identification code for each part so that the smartphone can identify each page. Soon, when the unique identification code is recognized through the smartphone, the text corresponding to the corresponding photographed image is output on the smartphone. The unique identification code may be formed in a unique patterned form for each part and may be attached or printed on the part in a sticker manner. The text consists of a phrase related to the content and topic to be delivered to the user on a specific page.

The image receiving unit of the present invention receives the image from the camera and converts it into a format that the image processing unit can process. The image receiver converts an analog signal into a digital signal.

The image processing unit processes the image captured by the camera through the image receiving unit. When a specific image is acquired by the camera, the image processing unit calculates the area to be extracted from the acquired image and outputs the extracted area on the screen of the smartphone. The image processing unit can extract both the uncolored area and the colored area of the teaching surface do.

The image processing unit can identify the teaching aid even if there is no marker, which is a unique identification code determined on the object, in the process of identifying the teaching aid. The image processing unit may use a markerless type of identifying an object by extracting a feature point that is a standard for identifying a teaching aid from an image taken by the teaching tool and comparing it with a pre-stored reference key point. At this time, the image processing unit includes a keypoint extraction unit that extracts the keypoints of the teaching tool from the image using a keypoint extraction algorithm, and a matching unit that compares the keypoints of the teaching tool with the reference keypoints stored in advance in the keypoint DB to identify what the teaching tool is.

The key point extraction unit uses a key point extraction algorithm. The key point extraction algorithm extracts the key points of the teaching aid from the image taken. The feature point extraction algorithm may use the same algorithm as the algorithm used to obtain the reference feature point. The keypoint extraction unit may extract a plurality of different types of keypoints for one teaching tool by using a plurality of different keypoint extraction algorithms. That is, the key point extraction algorithm extracts a point in an image where the brightness or saturation of pixels is different as a key point, or divides the image into a plurality of areas and extracts a key point for each area.

The feature point storage unit stores information about the reference feature points and parishes of various parishes. The reference feature point is the feature point of the precinct acquired in advance through the feature point acquisition algorithm. Since the appearance of the object appearing in the image may be displayed differently depending on the direction in which one teaching aid was shot or the type of camera, a plurality of different reference feature points may be mapped to one teaching aid and stored in the storage unit.

The matching unit identifies the object by comparing the key points of the object extracted by the key point extraction unit with the reference key points stored in advance in the storage unit. The matching unit reads the reference characteristic points of various objects stored in the storage unit, compares them with the characteristic points of the teaching aid, and determines whether a similar reference characteristic point exists. When the matching unit finds similar reference feature points, it reads information related to the object from the storage unit and outputs it to the augmented reality realization unit. The image processing unit includes a filtering, correction, and color unit.

The augmented reality implementation unit is implemented in a dynamic state when the image extracted by the image processing unit is selected. The augmented reality implementation unit outputs the area colored by the user as a three-dimensional shape based on augmented reality, and the background is projected as it is in reality so that the colored object teachings are simultaneously output on the actual background.

The augmented reality implementation unit creates an augmented reality image by superimposing the information on the identified parish on the image. Information about the parish may include the name and type of the parish. The augmented reality implementation unit generates an augmented reality image by arranging information on the image so that the information overlaps the extracted feature point and the information does not cover the object. The augmented reality implementation unit may output the generated augmented reality image to the image output unit.

On the other hand, in the present invention, the storage unit stores the objects colored by the user, and also stores the objects colored by other users. When a signal is input through an input unit such as a touch screen or a touch key, the control unit displays the object through the image processing unit of the user's smartphone who desires to use the object created by the other user through the storage unit. To implement this, the smartphone app can access the web server or the cloud and the communication unit to receive other people's objects from the database in real time or upload their own objects.

And as another embodiment, the present invention has a matching function, and the control unit does not illuminate the image that the user selects to use as an augmented reality marker to use as a medium for calling an object, and shoots and stores the image or goes out of place. Even so, the control unit continuously augments and displays the object initially stored in the storage unit by the user through the matching unit at a place different from the first illuminated by the smartphone or at an arbitrary place. At this time, the first saved object is displayed by connecting to the web server or the cloud and the communication unit.

As the teaching materials of the present invention, the vehicle manufacturing and operation sequence are described as follows. Steps to make vehicle parts using wood material

a coloring step of painting the vehicle parts with color;

an assembly step of assembling a vehicle;

Displaying an AR image by illuminating the smartphone to the vehicle; includes. The coloring step and the assembling step can be performed in reverse order to color the outside surface after assembly. And the present invention has a check and feedback function on the completeness of coloring in a paper toy. Considering the difficult part to color in the section, the function to automatically paint in connection with the existing color, and when a less colored section occurs, a message is displayed in the image recognition function to induce the coloring to continue.

Therefore, the present invention provides as a part puzzle so that any object can be assembled using a wood material such as balsa wood, coloring, applying the texture of the 3D model in real time using AR technology, and scratch coding By providing a way to control objects through Bluetooth communication, there is a remarkable effect that can be utilized for creative education.

Claims (3)

teaching aids made by assembling parts puzzles by hand;
Paper Toy-based AR interactive education system, characterized in that it consists of a smartphone that displays the teaching aid in augmented reality on the screen According to claim 1, wherein the material of the teaching material is a paper toy-based AR interactive education system, characterized in that the paper material Wood (Wood) According to claim 2, wherein the teaching aid is a controller is installed, the controller is a dedicated board device comprising an input/output unit, a Bluetooth communication unit, a motor drive unit, and a control unit, and Bluetooth communication with a smartphone is possible, and the smartphone app is an image Paper Toy-based AR interactive education system, characterized in that it consists of a receiver, an image processing unit, an augmented reality realization unit, and a control unit

Images