KR20220100271A - A coding training system enables module assembly through augmented reality - Google Patents

Abstract

The present invention relates to a coding training system which comprises: a module assembly in which a plurality of modules independently performing specific functions are assembled to be configured; a user terminal which recognizes a module combination of being actually coupled with a camera during module assembly, and displays a recognized real object and a virtual object generated based on a target object for the real object; and a server which generates a source code based on pre-stored topologies for the real and virtual objects.

Description

A system for coding education that enables module assembly through AR {A CODING TRAINING SYSTEM ENABLES MODULE ASSEMBLY THROUGH AUGMENTED REALITY}

The present invention relates to a coding education system capable of assembling a module through AR, and by combining modules capable of topology recognition through a wireless device such as a tablet PC, a smartphone, etc. into AR (Augmented Reality), a source code is easily generated It relates to a coding education system capable of assembling a module through AR, which can increase interest in coding at the same time.

Recently, various module-based production tools for education, hobbies, research, production, etc. have been proposed. The modules included in these manufacturing tools can each perform a specific function, and are provided to be connected to each other to form a module assembly. In this case, each module may be provided to be electrically connected to each other to transmit and receive energy, signals, data, and the like.

These module assemblies are used by various people such as researchers who are experts in software or hardware, students, and the general public.

However, it may be difficult for the general public or students except for experts to write and use software suitable for assembling each module to operate properly and driving the assembled module assembly.

In addition, since there was no system for coding education that could learn coding by assembling modules anytime, anywhere, development of a system that provides a service for learning coding in a fun way regardless of time and place to activate more active coding education I need this.

[Related technical literature]

1. User participatory e-book system and operation method using module (Patent Application No. 10-2020-0004579)

The present invention has been devised to respond to the above technical problem, and an object of the present invention is to substantially supplement various problems caused by limitations and disadvantages in the prior art, and the topology of modules photographed with a tablet PC AR (Augmented Reality) provides the number of cases for module combination by recognizing , and by assembling modules based on this, the user's interest can be amplified and the source code can be generated in an easier way. It is to provide a coding education system that can be assembled.

A coding education system according to an embodiment of the present invention includes: a module assembly configured by assembling a plurality of modules independently performing a specific function; a user terminal that recognizes the module assembly actually coupled in the module assembly with a camera, and displays a recognized real object and a virtual object generated based on a target object for the real object; and a server that generates source codes based on pre-stored topologies for real objects and virtual objects.

In addition, the recommended source code, which is the recommended source code, may be set to be editable as some of the module combinations recognized by the camera are replaced with modules of the same type.

Also, when the module assembly coupled based on the recommended source code does not operate normally, the learning unit may determine that an error has occurred and learn the cause of the error.

In addition, the server may include a recommended assembling information generating unit that provides recommended assembly information for a module that can be combined based on at least one target object, but is provided in a module preset by the user.

In addition, the server may include an augmented reality graphic display unit for AR (Augmented Reality) conversion of a two-dimensional image taken from the camera so that the real object and the target object are displayed as a three-dimensional image.

In addition, the server provides the source code for the overall module combination structure in which the real object and the virtual object are combined, and may include a code information generator that separately displays and provides the source code so that the source code corresponding to the virtual object can be checked. have.

Also, the virtual object may be a virtual image generated by dragging and dropping a module displayed as recommended assembly information on the display unit of the user terminal.

In another embodiment of the present invention, the method of operating a coding education system includes: receiving a module combined image photographed from a camera of a user terminal using a module information receiving unit; recognizing the target object from the module combined image using the target object control unit; providing recommended assembling information of various cases that can be combined based on the target object by using the recommended assembling information generating unit; and receiving virtual assembly information based on the recommended assembly information and determining final assembly information.

A coding education system according to another embodiment of the present invention includes: a module assembly configured by assembling a plurality of modules independently performing a specific function; a user terminal for photographing the module combination actually coupled in the module assembly with a camera, and displaying a source code generated based on the captured module combination; and a server for recommending and generating a source code by comparing a decision pattern including topology information about module combination captured from the camera and module attachment order information with a pre-stored decision pattern.

The solution of the problem according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present specification.

According to the embodiments of the present invention, there are at least the following effects.

According to the present invention, when the topology of modules photographed through a wireless device including a camera module is recognized, the number of possible cases is provided while assembling the modules in AR (Augmented Reality), thereby enhancing the user's thinking ability and problem-solving ability can help you improve.

According to the present invention, it is possible to help increase the user's interest in coding by recognizing the topology of modules that can be combined as the next module and providing the corresponding source code together.

According to the present invention, the entire process of assembling a module is indicated as source code, but the source code corresponding to the currently assembled module is separately indicated, thereby helping to improve the understanding of coding.

According to the present invention, it is possible to simultaneously provide a user's fun and learning effect for the content by outputting visual information, auditory information, etc. in the process of content through module combination.

According to the present invention, by augmenting various contents for coding education, it is possible to inspire interest with interactive contents and help to more easily understand the concepts and algorithms for coding.

According to the present invention, as coding education can be performed only with a wireless device and a module kit, the utilization of products for untact education can be increased.

The effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present specification.

1 is a perspective view showing a state in which a module assembly according to an embodiment of the present invention is assembled.
FIG. 2 is a plan view illustrating an internal structure of some modules of the module assembly of FIG. 1 .
3 is a schematic diagram of a system for teaching coding according to an embodiment of the present invention.
4 is a block diagram illustrating a server according to an embodiment of the present invention.
5 is a flowchart for explaining the operation process of the coding education system according to an embodiment of the present invention.
6 is an exemplary view of a screen on which recommended assembly information is displayed according to an embodiment of the present invention.
7 is an exemplary view of a screen on which augmented reality graphics are displayed according to an embodiment of the present invention.
8 is an exemplary view of a screen on which a source code is displayed together with recommended assembly information according to an embodiment of the present invention.

Advantages of the invention, and methods of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims.

The shapes, sizes, proportions, angles, numbers, etc. disclosed in the drawings for explaining the embodiments of the present invention are illustrative and the present invention is not limited to the illustrated matters. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. When 'including', 'having', 'consisting', etc. mentioned in this specification are used, other parts may be added unless 'only' is used. When a component is expressed in the singular, the case in which the plural is included is included unless otherwise explicitly stated.

In interpreting the components, it is interpreted as including an error range even if there is no separate explicit description.

Each feature of the various embodiments of the present invention may be partially or wholly combined or combined with each other, and as those skilled in the art will fully understand, technically various interlocking and driving are possible, and each embodiment may be independently implemented with respect to each other, It may be possible to implement together in a related relationship.

For clarity of interpretation of the present specification, terms used herein will be defined below.

As used herein, the term “user” refers to a user who uses the coding education system of the present invention, and is characterized by being composed of at least one person. In this specification, a user may be referred to as a participant, a player, or a user.

Hereinafter, a module that is hardware-coupled in the coding education system according to an embodiment of the present invention will be described with reference to FIGS. 1 to 2 .

1 is a perspective view showing a state in which a module assembly according to an embodiment of the present invention is assembled. FIG. 2 is a plan view showing the internal structure of some modules of the module assembly of FIG. 1 .

The module assembly 300 may be defined as a set (10, 20, 30, 40, 50) of one or more modules that can be assembled with each other or a structure in which they are assembled, and is not limited by the purpose, type, form, number of modules, etc. does not For example, the module assembly 300 may be part of a research kit used by a researcher to fabricate a device for performing a specific purpose.

The plurality of modules (10, 20, 30, 40, 50) transmit and receive signals, data, or electrical energy (hereinafter, also referred to as “electrical signals”) represented as changes in voltage or current with other modules or external devices. It can be defined as an object configured to be able to These modules are provided with a central processing unit (CPU), a memory 316, a power supply, or the like, which can be operated under the control of other modules, such as a sensing means, a processing means and a driving means, etc. Each may be independently driven.

In addition, each of the modules 10 , 20 , 30 , 40 , and 50 may be configured to independently perform a specific function or to perform a specific function by interaction with other modules. When the modules include a central processing unit (CPU), firmware may be installed for each module.

As described above, each of the modules 10, 20, 30, 40, and 50 is largely divided into an input module, an output module, and a setup module as shown in Table 1 below. can

<Table 1>

The input module is an infrared module that can receive an infrared signal from a remote control, etc., a gyroscope module that can detect the angle and acceleration of the X, Y, and Z axes, and the handle of the module. Dial module that can measure rotation angle or rotation speed using rotation, detects button press, detects click, double click, and pressed state, and toggles on/off /Off) button module to keep the state, environment module to measure temperature, humidity, illuminance, etc., microphone to detect ambient sound intensity (dB) and frequency It may be composed of a module (Mic module) and an ultrasonic module (Ultrasonic module) capable of sensing a distance.

In addition, the output module includes an LED module, a speaker module, and a display module that displays a picture drawn by a user, characters, or module information on the screen, which can be visually displayed according to color change, A motor controller module that transmits electric signals to the motor module to perform rotational motion by setting speed, angle, and torque, or a motor module that converts electrical signals received from the motor controller into rotational motion ( Motor; MDP-14), etc.

In addition, the setup module is a battery module used to supply power to other modules and a network module that can connect the module with a PC, smartphone or module through Bluetooth, Wi-Fi or USB connection. (Network module) and the like.

In addition, each of the modules 10 , 20 , 30 , 40 , and 50 may be connected to an external driving device 80 by a cable 70 . In this case, the module assembly 300 may be a device for selectively operating a motor by receiving a signal from a remote control or a smart phone. The configuration of the module assembly 300 as described above is only one example, and each module 10, 20, 30, 40, 50 can perform any function independently or through interworking with other modules. may be provided for.

The modules 10 , 20 , 30 , 40 , and 50 may be three-dimensional in the shape of a polygonal column having a plurality of side surfaces to be in surface contact with other neighboring modules. Here, the surface contact does not mean that all areas of the side are all in contact, but only a part of the side is in contact so that the side of one module and the side of the other module face each other and are understood to include a part in contact should be Referring to FIG. 4 , each of the modules 10 , 20 , 30 , 40 , and 50 is illustrated as having a square-shaped plane as an example. That is, the modules in this embodiment will be described as an example of a rectangular parallelepiped having four sides. Meanwhile, as another embodiment, the modules 10 , 20 , 30 , 40 , and 50 may be formed in a polygonal prism shape such as an equilateral triangle, a rectangle, or a regular pentagon in plan view, and in particular, may be formed in a regular polygonal shape. Some modules may have different three-dimensional shapes. In addition, some of the modules may have various three-dimensional shapes such as horns and polyhedrons.

The module 10 includes a housing 1 forming an external appearance, a terminal 2 exposed on the side surface of the housing 1 to transmit or receive an electric signal to another module connected thereto, and optionally to the outside of the housing. A pin installation part 105 provided with a pin protruding to (3 in FIG. 1 or 106 in FIG. 2) and a pin receiving part (in FIG. 1 (4) or FIG. 2 ( in FIG. 103)) may be included.

Meanwhile, one side of the first module 10 may include a serial port capable of communicating with an external device. For example, the serial port may use a port of various standards capable of performing wired serial communication, such as Universal Serial Bus (USB), USB-C type, IEEE 1394, and Thunderbolt.

The housing 1 is a case formed in the shape of a rectangular parallelepiped having a square plane, and protects internal components. The housing 1 may be provided in a form in which the upper case 1a and the lower case 1b are coupled as shown in FIG. 1 . As for the method of configuring the housing 1, the upper case 1a and the lower case 1b may be integrally formed, or may be assembled by being divided into more parts, if necessary. In addition, the lower case 1b may further include a frame forming an external and internal structure, a substrate 102 provided inside the frame, and a functional unit 104 installed on the substrate 102 . In more detail, the functional unit 104 for implementing the function of the first module 10 may be mounted on the substrate 102 , and may be fixedly installed in the center of the inner space of the frame. The functional unit 104 may include, for example, a microprocessor 180 , which may be provided to control the first module 10 if the first module 10 is driven by an independent firmware.

As another example, when the first module 10 is an infrared sensor module, the function unit 104 is configured with an analog digital converter (ADC) and other modules or external hardware necessary to process the infrared sensor and values sensed from the infrared sensor. It may include necessary devices such as an interface necessary for communication, for example, a communication interface such as I2C or UART, or USB. As such, in the modules 10 , 20 , 30 , 40 , and 50 of the present invention, the type of the module may be determined according to the function of the above-described functional unit 104 .

As another example, if the functional unit 104 includes a microprocessor 180 and a memory 190 to be described later, and includes an OS or firmware capable of controlling other units, it may be a main module. .

Alternatively, when the functional unit 104 can transmit the sensing value of the sensor to another module or an external device, it may be a sensor module.

Alternatively, the functional unit 104 receives an electrical signal from another module, including various wired and wireless communication devices such as NB-IOT, LTE, LoRa, WiFi, Bluetooth, USB, and cable modem, and receives an electrical signal from another external device through the above-described wired/wireless communication device. If it can transmit an electrical signal to the communication module can be.

Alternatively, when the functional unit 104 includes various actuators such as a motor and actuator control circuits to enable driving, it may be a driving module. Other more detailed structures and combinations of the module of the present invention are described in detail in Korean Patent No. 10-1761596, which is incorporated by reference in the present patent specification.

The terminal 2 may transmit an electrical signal or the like to or receive from another module connected thereto. For example, the terminal 2 receives an electrical signal from the board 102 provided in the housing 1 and receives the electrical signal from the other module in contact with the terminal. It can be transmitted through the terminal. The terminal 2 may have a plurality of contact points or connection pins, which may have various shapes according to a method of transmitting an electric signal or the like, a standardized standard, and the like.

Such a terminal 2 includes a pin ((3) in FIG. 1 or (106) in FIG. 2), a pin installation part 105, and a pin receiving part ((4) in FIG. 1 or (103) in FIG. 2) and It may be arranged on one side of the housing 1 to form a set. Specifically, the terminal 2 may be disposed between a pin ((3) in FIG. 1 or (106) in FIG. 2) and a pin receiving part ((4) in FIG. 1 or (103) in FIG. 2), It may be in contact with a terminal disposed between the pin of another module and the pin receiving portion. In this embodiment, the terminal 2 is provided on all sides of the housing 1 as an example, but depending on the embodiment, there may be a side on which the terminal 2 is not formed.

The frame is a structure constituting a part or all of the housing 1 , and forms an outer shape of a part or all of the housing 1 , and may provide a space and a structure for installing various parts therein. In this embodiment, the frame is described as an example forming the lower case 1b of the housing 1, but the scope of the present invention is not limited thereto. In addition, in this embodiment, the frame is formed in a rectangular shape and has four corners.

Hereinafter, a system for coding education according to an embodiment of the present invention 1000 and a server constituting the same will be described with reference to FIGS. 3 to 4 .

3 is a schematic diagram of a system for teaching coding according to an embodiment of the present invention. 4 is a block diagram for explaining the configuration of a server according to an embodiment of the present invention.

Referring to FIG. 3 , the coding education system 1000 includes a server 100 , a user terminal 200 , and a module assembly 300 .

The system for coding education 1000 is the number of cases that can be assembled so that the modules can be assembled in AR (Augmented Reality) when the topology of the modules photographed through a wireless device including a camera is recognized in order to maximize users' interest in coding. It is a system that provides various versions of content as it is provided. Specifically, the coding education system 1000 is characterized in that the server 100 and the user terminal 200 are connected through a network 900 such as the Internet, an intranet, or a broadband communication network. In this case, the server 100 or the user terminal 200 may provide an authoring tool capable of writing, recommending, and compiling (or interpreting) the source code of a program capable of driving the module assembly 300 . An authoring tool capable of writing, recommending, and compiling (or interpreting) such source code is described in detail in Korean Patent Application Nos. 10-2018-0114166 and 10-2018-0114171 previously invented by the present applicant. and the above two patents are incorporated by reference.

In more detail, the server 100 captures and recognizes the module coupling state and recognizes augmented reality graphics (hereinafter, also referred to as 'AR graphics') or virtual reality graphics (hereinafter also referred to as 'VR graphics') corresponding to the recognized target object. ) is provided together with the source code, or the recommended assembly information that can be combined based on the recognized target object is provided in various cases. As shown in FIG. 4 , the server 100 includes a module information receiving unit 110 , a target object control unit 120 , a recommended assembly information generating unit 130 , an augmented reality graphic display unit 140 , and a code information generating unit 150 . ), a learning unit 160 , a communication control unit 170 , a microprocessor 180 and a memory 190 .

The module information receiving unit 110 is configured to receive information about the modules that the user has before assembling the module. For example, information on modules (hereinafter, also referred to as 'module information') includes at least one of a module ID, a body message, a module category, function information, and meta information. Here, the module ID may be a module-specific identifier such as a universally unique identifier (UUID) or a temporary ID given when combining by the main module. In addition, the module ID may be a means for confirming the operation state of the module by checking the packet between the modules of the present invention. For example, since the modules of the present invention have a module for sending a message and a module for receiving a message, the module ID includes a unique number (Source module information (for sender)) containing information of the message sending module that sends a message and a message. It may be composed of a unique number (Target module information (for receiver)) containing information of the receiving message receiving module. The operation state determination process using the module ID will be described together with a learning unit to be described later.

In addition, the module category is a concept of classifying modules having similar functions into a higher level concept based on a function of performing a plurality of modules in order to assist in finding replaceable modules. For example, modules for inputting electrical signals such as sensors, keys, and microphones include 'inputs', actuators, displays, LEDs, etc. are 'outputs', or microprocessors 180, which can execute programs ported by the user. For a module, a network module, a battery module, or a weight display module, 'setup' may be an example of a module category.

In addition, the function information indicates detailed functions of the module, and for example, a control module, an infrared sensor module, a switch module, a power module, and a network module may be detailed functions. In this case, the function of the module may vary according to the configuration of the functional unit 104 in each module.

In addition, the meta information is other information of the module, and for example, may be a specific device name such as 'fan', or may be related search words related to a fan such as 'wind' or 'shiwon'.

In addition, the driving information is a driver for driving each module. Such driving information may be stored by the corresponding module and provided to the control module when the module is connected, or may be downloaded from the user terminal 200 when the control module provides module information to the user terminal 200 . In addition, the firmware is software for operating the module assembly 300 , and may be created as an executable file after compiling by writing a source code through the user terminal 200 . Alternatively, when the control module provides assembly information to the user terminal 200, a source code is recommended from the user terminal 200, and the recommended source code (hereinafter, also referred to as 'recommended source code') is compiled into an executable file ( firmware) may be downloaded from the user terminal 200 .

The target object control unit 120 is a component of hardware and/or software capable of detecting a topology (hereinafter, also referred to as a 'connection structure of the module assembly 300') of the module assembly 300 that is module-coupled to each other. , it performs the function of determining the target object in a number of cases by executing a protocol that detects the topology. In more detail, the target object control unit 120 is based on module topology information including module unique information, connection order (hereinafter also referred to as 'module attachment order information'), lower connection module information, module category information, and step information. to determine the topology.

Here, the module-specific information 1048 - 1 indicates module IDs of functional modules included in and connected to the module assembly. In this case, the module ID may be an identifier unique to the module, such as a Universally Unique IDentifier (UUID), or a temporary ID given by the control module when connecting.

The connection order indicates the order of accessing the main module. When a module group that is a plurality of modules already coupled to the main module is combined, the same common connection rank may be assigned to the module group.

The sub-connected module information indicates module-specific information about a module with a lower level than the corresponding module. Basically, each module stores information on modules connected to the corresponding module, and when connected to the main module, can provide the main module with information on the lower connected modules.

The module category information is information that classifies the module according to the function of the module. For example, the input module is a module for collecting input electrical signals, and includes various sensors such as a microphone, a temperature sensor, a gyroscope sensor, an infrared sensor, and an ultrasonic sensor, and key input such as a dial, a button, and a keyboard. The output module includes an actuator such as a motor, a light emitting diode (LED), a liquid crystal display device, and a speaker. The communication module includes a network module such as LTE and Bluetooth. The main module is a module capable of executing an application programmed in the user terminal 200 and includes a microprocessor. Based on such category information, the server may determine whether a module is a replaceable module or an addable module when recommending a module.

The step information means a distance from the main module, and may be determined by counting modules connected on the shortest path from the main module to the corresponding module.

In addition, the target object controller 120 may determine the connection topology of the module according to the sensed topology and write it to the memory 190 . On the other hand, a method for detecting and determining a topology based on the module topology information of the target object control unit 120 is described in detail in Korean Patent Application No. 10-2018-0114171 previously invented by the present applicant, and the above patent is incorporated by reference.

The recommended assembly information generating unit 130 is configured to provide information on a module that can be combined next to modules in an already combined state based on the topology sensed by the target object control unit 120 . Here, the recommended assembling information is recommended assembling information for modules that can be combined as virtual objects, and is generated from among the modules that are included in the combinable modules and set in advance by the user (pre-set that the user has) do it with However, according to an embodiment, a module not included in a preset module may be provided as recommended assembly information by the user, which may require a purchase for a new module, thereby stimulating the user's interest by performing more various module combinations. There is an effect that can make it happen.

In addition, the images of the virtual objects provided in the number of possible combinations are converted into AR (Augmented Reality) or VR (Virtual Reality) by the augmented reality graphic display unit 140 and displayed in 3D form. The augmented reality graphic display unit 140 is configured to display a real object recognized through the camera of the user terminal and a virtual object that can be combined with the real object on the screen of the user terminal 200 . In other words, the augmented reality graphic display unit 140 may display the position of a module that can be coupled next to the combinable side of the real object as a virtual object like a real object. In this case, the virtual objects are sequentially displayed based on the module topology information, and a plurality of sequentially displayed virtual objects may be displayed on the screen of the user terminal 200 .

The code information generating unit 150 is configured to generate a source code based on module assembly information. In more detail, the code information generating unit 150 is configured to generate a source code corresponding to a topology pre-stored in the server 100 . In other words, the code information generating unit 150 generates source codes for the modules when assembling the modules based on the source codes pre-stored in the memory 190 of the server 100 . In this case, the code information generating unit 150 may simultaneously generate source codes for assemblable virtual objects. Accordingly, a plurality of source codes may be displayed on the screen of the user terminal 200 according to the recommended assembly information generated by the recommended assembly information generating unit 130 in various cases.

Also, the code information generator 150 may recommend and provide a source code having a high similarity among pre-stored source codes according to module attachment order information and topology information. Specifically, based on the target object, which is topology information reflected up to module attachment order information, it is possible to search for and recommend a source code having the same information or similar information in a pre-stored source code. At this time, the source code with the same topology and module attachment order information in the pre-stored source code is given the highest priority in the search, and the one with the same topology but different module attachment order information is taken in the next order, and the topology and module attachment order information are all in the next order. In other cases, the recommendation source code may be provided by setting the recommendation priority, such as taking the next order.

Here, the recommended source codes may further include category information, each component module, summary information, VR execution video, topology thumbnail image information of the completed module, and author information in addition to the code. Here, VR execution video is virtually sound input (using microphone), light input (using camera), proximity (using camera), sound output (using speaker), light output (LED light source), and the slope of the geoid plane. Detection (using a gyroscope), direction detection (using geomagnetic sensor), east, west, north, and south directions can be used.

On the other hand, the recommended source code is characterized in that it is editable by substituting some modules of the same type. For example, as the recommended source code can be edited, the recommended source code that inputs infrared is changed to the source code that can input the microphone, or the source code that notifies the speaker module with sound by flashing the light from the lamp module. can be substituted. Accordingly, in the module replacement process, 'variables or module addresses attached to the module before replacement' according to the replacement as the same module can be automatically converted into the 'module address of the module after replacement' with the help of AI.

In addition, when the code information generating unit 150 according to another embodiment captures a predetermined pattern including module category information with a camera, the patterns displayed on the upper part of each module (eg, bartode, QR code, specific color, The left exterior of the housing) and the overall arrangement pattern of the patterns may be recognized and the source code of the category stored as the corresponding pattern among the source codes pre-stored in the server 100 may be recommended. That is, when the module category is included in the overall listing pattern, recommendation codes having the same category among the previously stored source codes when the entire pattern is photographed may be provided. In this case, the execution of the source codes may be displayed as a VR image.

The learning unit 160 is configured to learn whether a module corresponding to the virtual assembly information input by the user among the recommended assembly information generated by the recommended assembly information generation unit 130 operates so that the module assemblies can operate normally. . The learning unit 160 may determine whether an error exists by examining packet data based on an error checking method, and may normally operate module combinations by learning the cause of the error.

In more detail, the learning unit 160 may determine the type of error generated by examining the packet data. At this time, the cause of the error can be identified by analyzing the type of error that occurred. By learning this, the occurrence of errors can be minimized. Here, the method of inspecting packet data is a method using the communication packet structure of the present invention in which a message is exchanged between a sender and a receiver. For example, when the recevier normally receives the sender's message, it is possible to check whether the recevier is operating normally by sending a reply that the recevier has successfully received the sender again. In this case, the sender may mean an input module and the receiver may mean an output module. In other words, since the communication packet structure of the present invention has a structure in which a reply to the transmitted message is always provided when the sender transmits a message, it is possible to check whether the operation is being performed normally by checking the packet between modules.

Accordingly, the learning unit 160 may detect a defective state of the module. In more detail, when the value of the data measured by the input module exceeds the threshold value set by the user, it is determined as normal and a message is transmitted from the user to the predetermined output module so that the predetermined output module can perform the operation. However, if the output module does not respond due to a module itself defect, a module defect error message is sent to the user terminal 200 as it is determined that the output module is defective because a acknowledgment message (hereinafter, also referred to as a response) is not received from the output module. Module fault error message may be displayed on the screen by sending.

Also, when there is no assembly information that can be combined based on the target object, the learning unit 160 may determine that there is no module to be used and transmit a notification. In other words, since a packet address containing module information for a target module to which a message is to be sent is assigned to the communication packet (hereinafter, also referred to as a 'MODI packet') of the present invention, the message is transmitted even though the target module does not exist. In this case, a transmission error message can be displayed.

In addition, the learning unit 160 may check whether the operation state of the module is abnormal. Specifically, it is possible to determine whether a module operation is abnormal by utilizing a module information packet that can confirm whether a module operation is being performed normally among the communication packets of the present invention. For example, if it is assumed that the motor is operating with the motor module, which is one of the output modules, set to 30% speed output, a communication packet requesting module information is transmitted while the motor is operating to set the output value of the motor. By checking whether the operation is well performed while maintaining this 30%, it is possible to check whether there is an abnormality in the operation state.

Also, the learning unit 160 may check whether the module operates normally by using a virtual module that determines whether the source code written by the user is abnormal. Specifically, it is possible to verify whether the entire code operates without a problem before inserting the entire code into the actual module by generating a communication packet using a virtual module assuming that the source code written by the user is executed. That is, the learning unit 160 verifies the source code in the background of the coding education system 1000 in which the virtual module exists by using the characteristics that the standard of the communication packet is determined and the types of commands that can be performed of the module are determined. It is possible to quickly and easily determine whether the source code is operating abnormally.

The communication control unit 170 serves to control communication when the module 300 and the server 100 or the user terminal 200 communicate with each other through wireless communication. Specifically, the communication control unit 315 may control communication with an external device and communication between modules. Communication with an external device may include various wired/wireless communications such as NB-IOT, LTE, LoRa, WiFi, Bluetooth, USB, and cable modem. Meanwhile, communication between modules may be performed using various internal communication means such as UART, I2C, LIN, and CAN. The communication control unit 315 may include the above-described various wired/wireless communication means or communication means between modules, or may communicate in a manner that controls the separate network module when connected to a separate network module.

The microprocessor 180 includes a module information receiving unit 110 , a target object control unit 120 , a recommended assembly information generating unit 130 , an augmented reality graphic display unit 140 , a code information generating unit 150 , and a learning unit 160 ). When at least a part of the software module is implemented as a software module, the software module is executed.

The memory 190 includes a software module that can be executed in the microprocessor 180 and various types of information necessary for the operation of the module assembly 300 and various pieces of information related to assembly information of a connected module, for example, the memory 190 . may include module information, driving information, assembly information, and firmware. Also, the memory 190 may store source codes for modules constituting the module assembly. In this case, in the memory 190 , not only the completed source code that runs without an error is stored, but also the incomplete source code being edited (replaced) may be stored. Also, the memory 190 may store a genre of content displayed on the user terminal 200 , author information of the content, keywords, content information, and the like.

Meanwhile, in the present invention, the screen of the user terminal 200 may also be referred to as a display unit, and the display unit connects content information, quest performance information, event elements, etc. It is a configuration that performs output.

Referring to FIG. 3 , the user terminal 200 is configured to display content received from the server 100 and a quest corresponding to the mode module 320 selected by the user. Specifically, the user terminal 200 receives assembly information from the module assembly 300 connected through a network, and provides content corresponding to the assembly information. For example, the user terminal 200 according to the embodiment of the present invention is based on a tablet PC (Tablet PC). However, the type of the user terminal 200 is not limited thereto, and a 2D display, a 3D display, a head-mounted display (HMD), a smart Google (Smart) provided together with a camera, a speaker, a gyroscope, a geomagnetic sensor, etc. google) may be any one of them. Referring to FIG. 1 , although it is shown that there are a plurality of user terminals 200 , the user terminals 200 according to the user's play mode (eg, when there is one player, when there are two players, or more). It may be one, or it may be more than one. In the present invention, there may be one player, two or more players. However, when there is one user terminal 200 and there are two or more players, the screen area of the user terminal 200 may be divided by the number of players to display content.

In addition, the user terminal 200 may provide an authoring tool capable of writing, recommending, and compiling (or interpreting) the source code of a program capable of driving the module assembly 300 . An authoring tool capable of writing, recommending, and compiling (or interpreting) such source code is described in detail in Korean Patent Application Nos. 10-2018-0114166 and 10-2018-0114171 previously invented by the present applicant. and the above two patents are incorporated by reference.

The module assembly 300 is an educational kit that allows users, such as students and teachers, to understand the operating principle of an electronic device more interestingly while assembling the modules, and is configured to be connected to the user terminal 200 by wire or wireless. The module assembly 300 may be connected to the user terminal 200 by wire/wireless and may be connected to the user terminal 200 through a network such as the Internet, an intranet, or a broadband communication network. Alternatively, it may be connected to the user terminal 200 through wired/wireless communication, in particular, a serial port connection. In addition, the network may include wired short-range communication means such as USB, RS-222 and IEEE1394, wireless short-range communication means such as WiFi and Bluetooth, and Internet connection.

Hereinafter, an operation process of the coding education system according to an embodiment of the present invention will be described with reference to FIGS. 5 to 8 .

5 is a flowchart for explaining the operation process of the coding education system according to an embodiment of the present invention. 6 is an exemplary view of a screen on which recommended assembly information is displayed according to an embodiment of the present invention. 7 is an exemplary view of a screen on which augmented reality graphics are displayed according to an embodiment of the present invention. 8 is an exemplary view of a screen on which a source code is displayed together with recommended assembly information according to an embodiment of the present invention. Each configuration in FIGS. 6 to 8 has the same role as the configuration described above, and thus a redundant description thereof will be omitted.

In step S501 , the module information receiving unit 110 of the server 100 receives the module combined image photographed from the camera of the user terminal 200 . Referring to FIG. 6 , when the user clicks the camera button 603 displayed on the display unit of the user terminal 200 to photograph the actually coupled module combination 601 through a camera mounted on the back, the display unit displays a real object. It is characterized in that the virtual object position information 602 in the form of AR is displayed together with the image of , while blinking in a dotted line form. In FIG. 6 , it is assumed that the virtual object 602 is displayed as the virtual object 602 in two cases (CASE 1 and 2), but the combination type, the combination position, and the number of combinations of the virtual object are not limited thereto.

Then, in step S502, the target object is recognized from the module combined image using the target object control unit 120 of the server 100.

Then, in step S503, the augmented reality graphic display unit 140 of the server 100 provides the recommended assembly information in the form of an image that can be combined based on the target object in the form of AR. Referring to FIG. 7 , as shown in FIG. 6 , the recommended assembly information 702 that can be combined based on the target object for the real object recognized as the module combination actually coupled through the camera is photographed is displayed in AR form. At this time, when the user drags and drops the combinable modules shown in the recommended assembly information 702 to assemble them based on the previously recommended virtual object location information 602, the AR-shaped virtual object 704 is created. may be displayed at that location. At this time, the modules that can be dragged and dropped are generated by RF ID tagging, photographing, reading barcodes, or in-app payment for a specific virtual module with an actual module. Meanwhile, although the components displayed on the display in FIG. 7 are illustrated as being 2D, it is preferable to understand that they are all displayed as a 3D image (3-dimensional image) converted into AR.

At this time, as in step S504, the code information generating unit 150 of the server 100 simultaneously provides a pre-stored source code matching the target object. Referring to FIG. 8 , it can be seen that an image of a real object and an image of a virtual object are displayed in the object display area 800 of the user terminal 200, and source information 801A and 801B for each are displayed at the same time. can 801A denotes a source code corresponding to the first virtual object 704, and 801 denotes the entire source code for module combining up to the first virtual object 704 in the real object. In addition, 802B denotes a source code corresponding to the second virtual object 803, and 802 denotes the entire source code for module combining up to the second virtual object 803 in the real object.

In step S505, the virtual assembly information input by the user among a plurality of recommended assembly information displayed on the user terminal 200 is received. At this time, like step S506, the learning unit 160 of the server 100 determines whether the module is operating. At this time, when the module combinations normally operate normally, the received virtual assembly information is determined as the final assembly information as in step S507. However, if it does not operate normally, it is determined that an error has occurred, and the error is checked as in step S508 to learn the cause of the error. Since the error checking method is the same as the function performed by the learning unit 160 above, a redundant description will be omitted.

Thereafter, it is determined whether there is an error in step S509, and if it is determined that there is an error, step S508 is repeated, and if it operates normally, step S507 is performed.

As described above, the present invention provides the number of cases that can be assembled while allowing the modules to be assembled in AR (Augmented Reality) when the topology of the modules photographed through the wireless device including the camera module is recognized, thereby enhancing the user's thinking ability and problem solving ability. can help you improve.

According to the present invention, it is possible to help increase the user's interest in coding by recognizing the topology of modules that can be combined as the next module and providing the corresponding source code together.

According to the present invention, the entire process of assembling a module is indicated as source code, but the source code corresponding to the currently assembled module is separately indicated, thereby helping to improve the understanding of coding.

According to the present invention, it is possible to simultaneously provide a user's fun and learning effect for the content by outputting visual information, auditory information, etc. in the process of content through module combination.

According to the present invention, by augmenting various contents for coding education, it is possible to inspire interest with interactive contents and help to more easily understand the concepts and algorithms for coding.

According to the present invention, as coding education can be performed only with a wireless device and a module kit, the utilization of products for untact education can be increased.

Each block in the accompanying block diagram and combinations of steps in the flowchart are composed of firmware, software, or hardware. may be performed by an algorithm or computer program instructions. These algorithms or computer program instructions may be embodied in a processor of a general purpose computer, special purpose computer, or other programmable digital signal processing device, such that the instructions are executed by the processor of the computer or other programmable data processing equipment. These will create a means for performing the functions described in each block of the block diagram or in each step of the flowchart.

These algorithms or computer program instructions may also be stored in a computer-usable or computer-readable memory 316, which may direct a computer or other programmable data processing equipment to implement a function in a particular manner, and thus the computer-usable Alternatively, the instructions stored in the computer readable memory 316 may produce an article of manufacture containing instruction means for performing a function described in each block in the block diagram or in each step in the flowchart. The computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to create a computer or other programmable data processing equipment. It is also possible that instructions for performing the processing equipment provide steps for performing the functions described in each block of the block diagram and each step of the flowchart.

Further, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function(s). It should also be noted that it is also possible for the functions recited in blocks or steps to occur out of order in some alternative embodiments. For example, it is possible that two blocks or steps shown one after another may in fact be performed substantially simultaneously, or that the blocks or steps may sometimes be performed in reverse order depending on the corresponding function.

In addition, the system according to the present invention can be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored. The computer-readable recording medium includes a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, etc.) and an optical readable medium (eg, a CD-ROM, a DVD, etc.). In addition, the computer-readable recording medium is distributed in a network-connected computer system so that the computer-readable code can be stored and executed in a distributed manner.

1: Housing 2: Terminal
3, 106: pin 4, 103: pin receiving part
101: corner 102: board
104: function unit 105: pin mounting part
1000: Server for coding training
100: server 200: user terminal
300: module assembly

Claims (14)

a module assembly in which a plurality of modules independently performing specific functions are assembled and configured;
a user terminal that recognizes a module combination actually coupled among the module assembly with a camera, and displays a recognized real object and a virtual object generated based on a target object for the real object; and
A system for coding education comprising a server for recommending and generating source codes based on pre-stored topologies for the real object and the virtual object. According to claim 1,
The recommended source code, which is the recommended source code, is set to be editable as some of the module combinations recognized by the camera are replaced with modules of the same type. 3. The method of claim 2,
When the module assembly coupled based on the recommended source code does not operate normally, it is determined that an error has occurred and further comprising a learning unit for learning the cause of the error. According to claim 1,
The server provides at least one recommended assembly information for a module that can be combined based on the target object, but includes a recommended assembly information generator that is provided in a module preset by the user, Coding education system. According to claim 1,
The server comprises an augmented reality graphic display unit for AR (Augmented Reality) conversion of the two-dimensional image taken from the camera so that the real object and the target object are displayed as a three-dimensional image, the coding education system. According to claim 1,
The server provides a source code for the overall module combination structure in which the real object and the virtual object are combined, and includes a code information generator that separately displays and provides the source code so that the source code corresponding to the virtual object can be checked A system for teaching coding. 5. The method of claim 4,
The virtual object is a virtual object generated by dragging and dropping the module displayed as the recommended assembly information on the display unit of the user terminal, the coding education system. Receiving the module combined image taken from the camera of the user terminal using the module information receiving unit;
recognizing the target object from the module combined image using the target object control unit;
providing recommended assembling information of various cases that can be combined based on the target object by using the recommended assembling information generating unit; and
Receiving virtual assembly information based on the recommended assembly information and determining final assembly information; Containing, the operating method of the coding education system. 9. The method of claim 8,
The target object is determined based on module topology information including module unique information, connection ranking, lower connection module information, module category information, and step information, the operating method of the coding education system. 9. The method of claim 8,
AR (Augmented) a two-dimensional image so that the virtual object selected by the user among the recommended assembly information generated based on the target object and the real object for the module combination displayed on the user terminal using the augmented reality graphic display is displayed as a three-dimensional image Reality) further comprising the step of transforming, the operating method of the coding education system. 9. The method of claim 8,
The step of providing the recommended assembly information further comprises the step of providing source code information in real time based on the topology information for the recommended assembly information, the operating method of the coding education system. 9. The method of claim 8,
The user terminal is a wireless communication terminal including a tablet PC and a smart phone, the operating method of the coding education system. 9. The method of claim 8,
The module combination includes at least one of an input module and an output module,
The input module is at least one of an environment module, a button module, a gyroscope module, a microphone module, an infrared module, a dial module, and an ultrasonic module,
The output module is at least one of an LED module, a speaker module, a display module, a motor controller module, and a motor module, the operating method of the coding education system. a module assembly in which a plurality of modules independently performing specific functions are assembled and configured;
a user terminal for photographing the actually combined module combination of the module assembly with a camera, and displaying a source code generated based on the photographed module combination; and
Comprising a server for recommending and generating the source code by comparing a decision pattern including topology information and module attachment order information about the module combination photographed from the camera with a pre-stored decision pattern.

Images