KR102007491B1 - Method for providing coding training using virtual robot - Google Patents

Abstract

A purpose of the present invention is to provide a coding education method using augmented reality. More specifically, the present invention relates to a coding education providing method using a virtual robot as a method performed by a computing device including a camera. The coding education providing method using a virtual robot includes: a first step of obtaining coding education content; a second step of selecting one or more points included in the coding education content; a third step of obtaining an image photographed by a camera; a fourth step of recognizing the photographed image and matching one or more objects included in the photographed image with the one or more points; a fifth step of displaying the photographed image and the matched points; a sixth step of displaying a virtual robot on the photographed image; a seventh step of displaying one or more instruction blocks individually including one or more instructions to control the motion of the virtual robot; an eighth step of assigning the one or more instruction blocks to the one or more points based on a user input for the displayed instruction blocks; a ninth step of controlling the virtual robot depending on an order in which the one or more instruction blocks are arranged wherein the virtual robot moves according to the one or more points and performs an instruction corresponding to an instruction block assigned to each point; and a tenth step of displaying the motion of the virtual robot depending on the control result.

Description

Method for providing coding training using virtual robot}

The present invention relates to a method of providing coding education using a virtual robot.

As interest in coding education increases, applications and services are provided for beginners, children, and teenagers to provide coding learning by providing a block-type set of instructions rather than directly dealing with a programming language. In addition, a learning service using a robot operating according to coding is provided to visually show a coding result and to induce interest.

Augmented Reality is a technology that overlays virtual objects in the real world. It is also called Mixed Reality (MR) because it combines the virtual world with additional information in real time into one image.

Patent Publication No. 10-2018-0013487, filed Feb. 7, 2018

The problem to be solved by the present invention is to provide a coding education method using augmented reality.

Problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention for solving the above problems, a method performed by a computing device including a camera, the first step of obtaining coding education content, selecting one or more points included in the coding education content A second step, a third step of acquiring a captured image from the camera, a fourth step of recognizing the captured image, and matching one or more objects and the one or more points included in the captured image, the captured image One or more instructions each including a fifth step of displaying an image and the matched one or more points, a sixth step of displaying a virtual robot on the photographed image, and one or more instructions for controlling the movement of the virtual robot A seventh step of displaying a block, the one or more based on user input for the displayed one or more instruction blocks An eighth step of placing the instruction block at the one or more points, controlling the virtual robot according to the order in which the one or more instruction blocks are arranged, wherein the virtual robot moves along the one or more points, and each point And a tenth step of performing a command corresponding to the instruction block disposed in the step, and displaying a movement of the virtual robot according to the control result.

In addition, the first step may include an eleventh step of obtaining a mission included in the coding education content and a twelfth step of displaying the mission, and the ninth step may include: achieving the mission according to the control result A thirteenth step of determining whether or not, and a fourteenth step of displaying feedback according to whether the mission is achieved.

The thirteenth step may include a fifteenth step of determining an interaction between the one or more objects and the virtual robot, and a sixteenth step of determining whether the mission is achieved at each of the one or more points based on the interaction. The fourteenth step may include a seventeenth step of displaying feedback based on whether the mission is achieved at each of the one or more points.

In addition, the fifteenth step may include an eighteenth step of determining one or more contacts between the one or more objects and the virtual robot, and determining the one or more objects corresponding to each of the one or more contacts and the site of the virtual robot. A nineteenth step and a twentieth step of determining the interaction based on the position, the movement of the one or more contacts and the portion corresponding to each of the one or more contacts.

The twenty-second step may include: twenty-first step of dividing a first contact included in the one or more contacts into a second contact point and a third contact point; and twenty-second step of tracking movement of the second contact point and the third contact point; When the distance between the second contact point and the third contact point is greater than or equal to a predetermined reference value, determining a portion of the at least one object and the virtual robot corresponding to each of the second contact point and the third contact point; And a twenty-fourth step of determining the interaction based on the position, the movement of the second contact point and the third contact point, and a portion corresponding to the second contact point and the third contact point, respectively.

The eleventh step may include: twenty-five step of determining a type of the one or more objects; and generating a movement path of the virtual robot based on the position and type of the one or more objects included in the captured image. In step 26, obtaining a first mission for allowing the virtual robot to move along the movement path; and performing a second mission for allowing the virtual robot to perform a predetermined interaction with at least one of the one or more objects. It may include the 28 th step of obtaining.

In addition, the eleventh step may include: twenty-ninth step of generating and displaying one or more virtual objects on the movement path; thirty thirty step of determining a type of the one or more virtual objects; and location of the one or more virtual objects. And a thirty-one step of updating the movement route based on a type, a thirty-second stage of acquiring a third mission for causing the virtual robot to move along the updated movement route, and the virtual robot is configured to perform the at least one virtual operation. The method may include a thirty-third step of obtaining a fourth mission for performing a predetermined interaction with at least one of the objects.

The eighth step may further include a thirty-fourth step of disposing at least a portion of the one or more instruction blocks on the virtual robot, and the ninth step may correspond to the instruction block disposed on the virtual robot. A step 35 of performing a command; a step 36 of performing a command corresponding to the command block disposed at the one or more points; and a command corresponding to the command block disposed of the virtual robot; The method may further include a thirty-seventh step of stopping the operation and displaying the feedback when a command corresponding to the instruction block disposed at the arrived point collides.

Other specific details of the invention are included in the detailed description and drawings.

According to the disclosed embodiment, there is an effect of providing coding education using a virtual robot without an actual robot using augmented reality.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a flowchart illustrating a coding education method using augmented reality according to an exemplary embodiment.
2 is a diagram illustrating a method of obtaining coding education content and a virtual robot control method corresponding thereto according to an embodiment.
3 is a flowchart illustrating a method of determining whether a mission is accomplished according to an exemplary embodiment.
4 is a flowchart illustrating a method of determining an interaction according to an exemplary embodiment.
5 is a flowchart illustrating a method of determining a contact point and an interaction according to an exemplary embodiment.
6 is a flowchart illustrating an example of obtaining a mission.
7 is a flowchart illustrating an example of obtaining a mission based on a virtual object.
8 is a flowchart illustrating a method of arranging an instruction block according to an exemplary embodiment.
9 is a block diagram of a computing device according to an embodiment.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be embodied in various different forms, and the present embodiments only make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs. It is provided to fully inform the skilled worker of the scope of the invention, which is defined only by the scope of the claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used herein, "comprises" and / or "comprising" does not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like elements throughout, and "and / or" includes each and all combinations of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are of course not limited by these terms. These terms are only used to distinguish one component from another. Therefore, of course, the first component mentioned below may be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms used in the present specification (including technical and scientific terms) may be used in a sense that can be commonly understood by those skilled in the art. In addition, terms that are defined in a commonly used dictionary are not ideally or excessively interpreted unless they are specifically defined clearly.

As used herein, the term "part" or "module" refers to a hardware component such as software, FPGA, or ASIC, and the "part" or "module" plays certain roles. However, "part" or "module" is not meant to be limited to software or hardware. The “unit” or “module” may be configured to be in an addressable storage medium or may be configured to play one or more processors. Thus, as an example, a "part" or "module" may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Functions provided within components and "parts" or "modules" may be combined into smaller numbers of components and "parts" or "modules" or into additional components and "parts" or "modules". Can be further separated.

The spatially relative terms " below ", " beneath ", " lower ", " above ", " upper " It can be used to easily describe a component's correlation with other components. Spatially relative terms are to be understood as including terms in different directions of components in use or operation in addition to the directions shown in the figures. For example, when flipping a component shown in the drawing, a component described as "below" or "beneath" of another component may be placed "above" the other component. Can be. Thus, the exemplary term "below" can encompass both an orientation of above and below. Components may be oriented in other directions as well, so spatially relative terms may be interpreted according to orientation.

The numbers (e.g., first step, second step, etc.) described together with each step described in the present specification are for distinguishing each step, and not for specifying the execution order or prosecutorial relationship of each step.

As used herein, the term “robot” or “virtual robot” refers to a hardware device or a virtual device configured in software for performing a command according to a control, and is not intended to limit a specific kind.

In the present specification, the computing device refers to any kind of hardware device including at least one processor, and may be understood as a meaning encompassing a software configuration that operates on the hardware device according to an embodiment. For example, a computing device may be understood as including, but not limited to, a smartphone, a tablet PC, a computer, and a user client and an application running on each device.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Each step described in connection with FIGS. 1 to 8 is described as being performed by a computing device, but the subject of each step is not limited thereto, and at least some of the steps may be performed in different devices according to embodiments. It may be.

In an embodiment disclosed below, the computing device includes a camera, photographs a surrounding image using the camera, and displays an augmented reality image in which a virtual robot is coupled to the photographed image.

The computing device obtains a command for controlling the virtual robot according to the user input, controls the virtual robot according to the obtained command, and displays the control result.

The virtual robot interacts with the objects included in the captured image and the virtual objects included in the augmented reality image under control, and the computing device provides feedback based on the result of the interaction.

1 is a flowchart illustrating a coding education method using augmented reality according to an exemplary embodiment.

In step S101, the computing device performs a first step of acquiring coding education content.

In one embodiment, the computing device obtains coding education content from a server via a network. In another embodiment, the computing device may obtain the coding training content stored in the memory and may also generate the coding training content based on a predetermined rule.

In step S102, the computing device performs a second step of selecting one or more points included in the coding education content.

In one embodiment, the coding education content may be for writing instructions that cause a virtual robot to interact with and move one or more real or virtual objects.

For example, the coding education content may be for writing a command for the virtual robot to move to a specific path avoiding one or more obstacles. In addition, the coding education content may be for writing a command for the virtual robot to grab and transport a specific object on the movement path.

Therefore, one or more points included in the coding education content means one or more objects to be interacted with the virtual robot. For example, the one or more objects may include obstacles to be avoided by the virtual robot, objects to be transported by the virtual robot, and the like.

In operation S103, the computing device performs a third operation of acquiring an image captured by the camera.

In operation S104, the computing device recognizes the photographed image, and matches one or more objects and one or more points included in the photographed image.

For example, one or more objects may mean a specific object included in the captured image, and according to an embodiment, a specific position of a wall or a floor may be recognized as an object according to the selection of a computing device.

The computing device matches one or more points included in the coding education content with respect to one or more objects included in the captured image. For example, a specific object included in the photographed image may be matched as an obstacle included in coding education content, and a command may be written so that the virtual robot avoids the object.

Similarly, for a particular object included in the captured image, a virtual robot may write a command to move the object. In this case, the virtual robot cannot move around with the real object, so that the computing device can move the virtual robot and the virtual robot and the real object (specifically, the photographed real that is displayed on the screen of the computing device). Based on the positional relationship with the object), it may be determined that the virtual object is caught by the virtual robot, and that the object moves with the virtual robot as the virtual robot moves. According to an embodiment, the computing device removes the actual object moved by the virtual robot from the screen, and the image of the virtual object corresponding to the actual object or the point matched with the actual object is moved by the virtual robot. May be displayed, but is not limited thereto.

A method of determining the positional relationship between the virtual robot and the actual object by the computing device will be described later in detail.

In operation S105, the computing device performs a fifth operation of displaying the captured image and the matched one or more points.

For example, the computing device displays a photographed image, and displays a point (or a virtual object corresponding to the point) included in the coding education content at a location where at least one object recognized in the photographed image is photographed. The point matched with each object may be displayed by replacing the object at the position of the actual object, or may be displayed overlapping with the object.

In operation S106, the computing device performs a sixth operation of displaying a virtual robot on the captured image. The size and shape of the virtual robot is not limited. According to an embodiment, the virtual robot may include a tool for interacting with the virtual object (eg, tongs for holding and moving the virtual object).

In operation S107, the computing device may perform a seventh operation of displaying one or more instruction blocks each including one or more instructions for controlling the movement of the virtual robot.

For example, each instruction block may include instructions (eg, forward, backward, left turn, right turn, etc.) to move the virtual robot in a specific direction.

In addition, each instruction block may include instructions (eg, grabbing an object, colliding with or avoiding an object, pushing an object, and the like) for interacting with an object and a virtual robot.

In addition, each instruction block may include instructions that are performed based on a particular condition (eg, to stop, evade, or grab an object when the object is recognized).

In step S108, the computing device performs an eighth step of placing the one or more instruction blocks at the one or more points based on user input for the displayed one or more instruction blocks.

For example, the computing device receives a touch input for selecting and dragging and dropping each of one or more command blocks displayed from a user through a touch screen device, and moving each command block to a position corresponding to the received user input to display the touch input. can do. Each instruction block can be moved to each point according to the user's choice and arranged. Each point is included in the coding training content as described above. It can be understood as meaning including points that need to be performed.

For example, each point could mean an obstacle to be avoided by a virtual robot, a passage to pass through, a door to open and pass through, a wall to change direction, etc. Matches one or more objects included. For example, each point may be matched to correspond to an actual wall, a specific position of the floor, an object located on the floor, etc. included in the image captured by the camera, and the command block may be matched to each point according to a user input. Can be deployed. When a plurality of instruction blocks are arranged at one point, they may be arranged according to the order in which they are arranged, and the order may be changed according to a user input.

In step S109, the computing device controls the virtual robot according to the order in which the one or more instruction blocks are arranged, wherein the virtual robot moves along the one or more points and corresponds to the instruction blocks disposed at each point. Perform a ninth step.

For example, the virtual robot moves along a point included in the coding education content, and when arriving at each point, performs a command corresponding to the instruction block disposed at each point.

For example, if a virtual robot arrives at a point where an obstacle is placed, and a command block including a command to turn right and go straight to the point and a command to turn left to go straight is arranged in order, then turn right and go straight. Turn left and go straight ahead to avoid obstacles.

As another example, the virtual robot may move straight after grabbing an object disposed at a point when a command block including an instruction to grab an object disposed at a point at a specific point and an instruction block including a command to go straight are arranged in order.

As another example, the virtual robot may turn right without going into the passage when the instruction block is arranged to turn right and go straight without entering the passage at a point corresponding to the passage that may go straight.

In operation S110, the computing device performs a tenth operation of displaying the movement of the virtual robot according to the control result. That is, the computing device displays the movement and interaction of the virtual robot according to the above control.

2 is a diagram illustrating a method of obtaining coding education content and a virtual robot control method corresponding thereto according to an embodiment.

In operation S101, the computing device performs an eleventh operation S111 of obtaining a mission included in the coding education content.

For example, a mission included in coding education content may include a mission to move a virtual robot from one location to another, a mission to move a virtual robot along a specific path, and a virtual robot to move a specific object. It may include a mission, but is not limited thereto.

In addition, the computing device performs a twelfth step S112 of displaying the mission.

In step S109 described above, the computing device performs a thirteenth step S113 of determining whether the mission is achieved according to the control result.

In one embodiment, each mission includes one or more points according to the mission content, and the computing device matches each point with the captured image while the virtual robot moves through augmented reality within the captured image. In addition, the interaction between the virtual robot and each point (that is, parts of the captured image matched with each point) may be determined to determine whether the mission is achieved.

In addition, the computing device performs a fourteenth step S114 of displaying feedback based on whether or not the mission is achieved.

For example, the computing device determines whether the mission has been achieved after the operation of the virtual robot is completed, and generates feedback based on where the mission has been achieved or where it failed and what is the problem. Can provide.

In addition, the computing device may determine the movement of the virtual robot in real time, and provide feedback by determining whether each operation is in accordance with the mission or out of the mission.

According to an embodiment, the computing device may determine in advance whether the mission is successful based on the arrangement of the instruction blocks disposed at each point, and in this case, provide the feedback requesting to change the arrangement of the instruction blocks.

3 is a flowchart illustrating a method of determining whether a mission is accomplished according to an exemplary embodiment.

In operation S113, the computing device performs a fifteenth operation S115 of determining an interaction between the one or more objects and the virtual robot.

For example, the computing device may determine the interaction based on the positional relationship between the virtual robot and the virtual image corresponding to the object included in the captured image or the point matched thereto.

In one embodiment, the virtual robot may include an ultrasonic sensor. In addition, a command according to the distance detected by the ultrasonic sensor may be arranged in the virtual robot. In this case, the computing device determines the distance and angle between the object and the virtual robot, and calculates the virtual output of the ultrasonic sensor therefrom. The computing device may control the virtual robot based on the calculated virtual output. In addition, the computing device may determine the interaction based on the contact point between the one or more objects and the virtual robot, which will be described later.

In addition, the computing device performs a sixteenth step S116 of determining whether the mission is achieved at each of the one or more points based on the interaction.

For example, according to the coding education content, a mission may be assigned to a virtual point so that the virtual robot may avoid the collision without colliding with the point, and correspondingly, the computing device may not be able to reach the point. You might have placed instruction blocks that change the direction of to avoid them. In this case, the virtual robot will be avoided without colliding with the point under the control of the computing device, and the computing device may determine that the mission of the point has been achieved.

In addition, in step S114 described above, the computing device performs a seventeenth step S117 of displaying feedback based on whether or not the mission is achieved at each of the one or more points.

For example, when the mission at each point is achieved, the computing device may provide feedback about whether the mission is achieved by differently expressing the color of the point, but is not limited thereto.

4 is a flowchart illustrating a method of determining an interaction according to an exemplary embodiment.

In operation S115 described above, the computing device performs an eighteenth operation S118 of determining one or more contact points between the one or more objects and the virtual robot.

For example, the computing device analyzes an augmented reality image to determine a contact point between one or more objects and a virtual robot. For example, the contact point may mean a portion in which an object and a virtual robot contact or overlap each other in the augmented reality image.

In addition, the computing device performs a nineteenth step S119 of determining the one or more objects corresponding to each of the one or more contacts and the part of the virtual robot.

For example, the computing device may determine, for each of the one or more contacts, which part of the object and the virtual robot the contact corresponds to. For example, if the object is a box to be carried by a virtual robot, the contacts may correspond to a portion of the box and a portion of the tongs used to transport the box in the virtual robot.

In addition, the computing device performs a twentieth step (S120) of determining the interaction based on the position, the movement of the one or more contacts, and a portion corresponding to each of the one or more contacts.

For example, when the first contact point and the second contact point are determined, the first contact point may correspond to one point of the carrying object box and the second contact point may correspond to another point. Similarly, the first contact may correspond to one part of the forceps included in the virtual robot, and the second contact may correspond to the other part of the forceps. In this case, the computing device may determine that the virtual robot grabbed the carrying object box using forceps. In addition, in this state, when the positions of the first contact point and the second contact point move, the computing device may determine that the virtual robot lifts or moves the carrying object box according to the moving direction of the contact point.

5 is a flowchart illustrating a method of determining a contact point and an interaction according to an exemplary embodiment.

In operation S120, the computing device performs a twenty-first operation S121 of dividing a first contact included in the one or more contacts into a second contact and a third contact.

For example, even when one contact is determined, the computing device may preliminarily divide the contact into two or more contacts, and determine that each contact is in the same position.

In addition, the computing device performs a twenty-second step S122 of tracking the movements of the second contact point and the third contact point.

Also, when the distance between the second contact point and the third contact point is greater than or equal to a predetermined reference value, the computing device may determine a portion of the one or more objects and the virtual robot corresponding to each of the second contact point and the third contact point. The twenty third step S123 of determining is performed.

In an embodiment, the parts of the object or the virtual robot corresponding to the contact point may be separated and separated from each other. For example, an object can be separated into two or more. There may be an embodiment in which the objects are stacked in blocks in which they can be separated, or are cut by interaction with a virtual robot. In addition, the forceps of the virtual robot can be switched to the open state in the closed state. In this case, when the tip of the forceps corresponds to the contact point, the contact point may be separated into two or more as the forceps are opened.

In addition, the computing device performs a twenty-fourth step S124 of determining the interaction based on the position, the movement of the second contact point and the third contact point, and a portion corresponding to the second contact point and the third contact point, respectively. .

When the contact point is separated into two or more contacts, the computing device may determine an interaction between the object and the virtual robot by re-determining an area of the object and the virtual robot corresponding to each of the separated contact points.

In one embodiment, the object and the virtual robot may come into contact and move away. In this case, the existing contact may be separated into the object-side contact point and the virtual robot-side contact point, and the computing device may determine the interaction between the object and the virtual robot by re-determining an object and a portion corresponding to each contact point.

That is, the computing device may determine the generation of the contact point, and then track the movements in which the contact point disappears or separated into two or more, thereby determining the interaction between the virtual robot and the object.

6 is a flowchart illustrating an example of obtaining a mission.

In the above-described step S111, the computing device performs a twenty-fifth step S125 of determining the type of the one or more objects.

For example, the computing device may determine whether each object is an object to be carried by a virtual robot, an object to be avoided by the virtual robot, an object through which the virtual robot can pass, and the like.

In addition, the computing device performs a 26 th step (S126) of generating a movement path of the virtual robot based on the position and type of the one or more objects included in the captured image.

For example, the computing device may generate a path to which the virtual robot should move based on a result of determining the type of each object, and the path may determine an optimal path or a range of a path to which the virtual robot can move. It may include, but is not limited thereto.

In addition, the computing device performs a twenty-seventh step (S127) of acquiring a first mission for causing the virtual robot to move along the movement path.

The range of the movement path may be set differently according to the difficulty level, and a mission for allowing the virtual robot to move within a predetermined range may be obtained, and the movement path of the same range may be analyzed by topological analysis of the movement path of the virtual robot. If it is determined that the mission is determined to have achieved the mission may be obtained.

In addition, the computing device performs a twenty-eighth step (S128) of acquiring a second mission for allowing the virtual robot to perform a predetermined interaction with at least one of the one or more objects.

For example, the mission may include, but is not limited to, a mission that causes the virtual robot to avoid, collide, or transport a specific object.

7 is a flowchart illustrating an example of obtaining a mission based on a virtual object.

In operation S111, the computing device performs operation 29 (S129) of generating and displaying one or more virtual objects on the movement path.

For example, the virtual object is an object not included in the captured image and may be displayed using augmented reality at a point of the captured image.

In addition, the computing device performs a thirtieth step S130 of determining the type of the one or more virtual objects.

For example, the type of virtual object may include, but is not limited to, an object to be carried by a virtual robot, an object to be avoided by the virtual robot, an object through which the virtual robot can pass, and the like.

In addition, the computing device performs a thirty-first step S131 of updating the movement path based on the location and type of the one or more virtual objects.

For example, if the added virtual object is an obstacle to be avoided by the virtual robot, the movement path may be updated to move to avoid it.

In addition, the computing device performs a thirty-second step (S132) of acquiring a third mission for causing the virtual robot to move along the updated movement path.

In addition, the computing device performs a thirty-third step (S133) of acquiring a fourth mission for allowing the virtual robot to perform a predetermined interaction with at least one of the one or more virtual objects.

For example, the fourth mission may include a mission for a virtual robot to carry a virtual object, make a specific shape using a plurality of virtual objects, or stack a plurality of virtual objects such as blocks. can do.

8 is a flowchart illustrating a method of arranging an instruction block according to an exemplary embodiment.

In operation S108, the computing device performs a thirty-fourth operation S134 of disposing at least some of the one or more instruction blocks on the virtual robot.

In one embodiment, the computing device may not only place at least some of the instruction blocks on the points obtained from the coding education content based on the user input, but also on the virtual robot. That is, when a drag and drop input for a command block is received from a user, the command block is disposed at a position where a virtual robot is disposed, and a plurality of command blocks are disposed on the virtual robot, according to the arrangement order. You can sort the block of instructions. The order in which the command blocks are arranged may be changed according to user input.

For example, the command block disposed in the virtual robot may be an operation performed by the virtual robot irrespective of the point, and the operation using the virtual sensor equipment for the virtual robot to recognize and operate each object or point. Can include them.

For example, the command block disposed in the virtual robot may include a command to advance the virtual robot and a command to stop once the obstacle is recognized.

In the above-described step S109, the computing device performs a thirty-fifth step S135 of performing a command corresponding to the instruction block disposed in the virtual robot.

In addition, when the computing device arrives at the one or more points, the computing device performs a 36 th step S136 of performing a command corresponding to the instruction block disposed at the arrived point.

For example, the virtual robot may basically go straight according to the command block disposed in the virtual robot, and may stop once the obstacle is recognized.

Thereafter, according to the instruction block disposed at the point corresponding to the obstacle, an operation of lifting or avoiding the obstacle may be performed.

In addition, when the command corresponding to the instruction block arranged in the virtual robot collides with the instruction corresponding to the instruction block arranged in the arrived point, the computing device stops the operation and displays the feedback (S137). ).

For example, the instruction block disposed in the virtual robot includes an instruction to collide with an obstacle when it encounters an obstacle, and the instruction block arranged at a point corresponding to the obstacle includes an instruction to avoid the obstacle. If so, a crash occurs. In this case, in order to prevent an error, the computing device may stop the operation of the virtual robot and display the feedback.

In addition, the priority of the instruction block may be set according to an embodiment. For example, if a command according to an instruction block arranged in a virtual robot collides with a command according to an instruction block arranged in each point, the instruction according to the instruction block arranged in the virtual robot takes precedence or vice versa. It can be set to.

In addition, the priority may be set differently according to the type of the instruction (for example, the avoiding instruction may be given priority over the instruction colliding with the obstacle), and the priority may be set for each instruction block. For example, each instruction block may be sequentially numbered, and if different instruction blocks collide with each other, the instructions of the higher numbered instruction block may be followed.

In one embodiment, when the operation of the virtual robot is stopped and the feedback is displayed according to step S137, one of the conflicting command blocks is selected according to the user's selection, and the virtual robot is controlled according to the selected command block. It may be.

In addition, the user may perform debugging when the operation of the virtual robot is stopped and change the arrangement of the command block.

9 is a block diagram of a computing device according to an embodiment.

The processor 102 may include a connection passage (eg, a bus, etc.) that transmits and receives signals with one or more cores (not shown) and graphics processor (not shown) and / or other components. .

The processor 102 according to one embodiment executes one or more instructions stored in the memory 104 to perform the method described in connection with FIGS. 1-8.

For example, the processor 102 executes one or more instructions stored in a memory to acquire a coding education content; a second step of selecting one or more points included in the coding education content; A third step of acquiring an image, a fourth step of recognizing the photographed image and matching one or more objects and the one or more points included in the photographed image, the photographed image and the matched one or more points A fifth step of displaying, a sixth step of displaying a virtual robot on the photographed image, a seventh step of displaying one or more command blocks each including one or more commands for controlling the movement of the virtual robot, and Based on the user input for the displayed one or more instruction blocks, the one or more instruction blocks may be deleted. In an eighth step of disposing at points on the control panel, the virtual robot is controlled according to the order in which the one or more command blocks are arranged, wherein the virtual robot moves along the one or more points, and the command block is disposed at each point. A ninth step of performing a command corresponding to the second step and a tenth step of displaying the movement of the virtual robot according to the control result.

Meanwhile, the processor 102 may include random access memory (RAM) and read-only memory (ROM) for temporarily and / or permanently storing a signal (or data) processed in the processor 102. , Not shown) may be further included. In addition, the processor 102 may be implemented in the form of a system on chip (SoC) including at least one of a graphic processor, a RAM, and a ROM.

The memory 104 may store programs (one or more instructions) for processing and controlling the processor 102. Programs stored in the memory 104 may be divided into a plurality of modules according to functions.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, in a software module executed by hardware, or by a combination thereof. Software modules may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art.

The components of the present invention may be implemented in a program (or an application) and stored in a medium to be executed in combination with a computer which is hardware. The components of the present invention may be implemented as software programming or software elements, and likewise, embodiments may include various algorithms implemented in combinations of data structures, processes, routines or other programming constructs, such as C, C ++. It may be implemented in a programming or scripting language such as Java, an assembler, or the like. The functional aspects may be implemented with an algorithm running on one or more processors.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

100: computing device
102: processor
104: memory

Claims (1)

A method performed by a computing device comprising a camera,
Obtaining a coding education content;
Selecting at least one point included in the coding education content;
A third step of acquiring an image captured by the camera;
Recognizing the captured image and matching the at least one point with at least one object included in the captured image;
A fifth step of displaying the captured image and the matched one or more points;
Displaying a virtual robot on the captured image;
Displaying at least one instruction block each including at least one instruction for controlling movement of the virtual robot;
Placing the at least one instruction block at the at least one point based on user input to the displayed at least one instruction block;
Controlling the virtual robot according to the order in which the one or more instruction blocks are arranged, wherein the virtual robot moves along the one or more points and performs a command corresponding to the instruction block disposed at each point; Step 9; And
A tenth step of displaying a movement of the virtual robot according to the control result; Including,
The first step,
An eleventh step of obtaining a mission included in the coding education content; And
A twelfth step of displaying the mission; Including,
The ninth step,
Determining whether the mission is achieved according to the control result; And
Displaying a feedback according to whether the mission is accomplished; Including,
The thirteenth step,
Determining an interaction between the one or more objects and the virtual robot; And
Determining whether the mission is achieved at each of the one or more points based on the interaction; Including,
The fourteenth step,
Displaying a feedback according to whether the mission is achieved at each of the one or more points; Including,
The fifteenth step,
An eighteenth step of determining one or more contact points between the one or more objects and the virtual robot;
Determining a location of the at least one object and the virtual robot corresponding to each of the at least one contact point; And
Determining the interaction based on the position, the movement of the one or more contacts, and a portion corresponding to each of the one or more contacts; Including,
The 20th step,
Dividing a first contact included in the at least one contact into a second contact and a third contact;
A twenty-second step of tracking movement of the second contact point and the third contact point;
If the distance between the second contact point and the third contact point is greater than a predetermined reference value, determining a portion of the at least one object and the virtual robot corresponding to each of the second contact point and the third contact point; ; And
Determining an interaction based on a position, a movement of the second contact point and the third contact point, and a portion corresponding to the second contact point and the third contact point, respectively; Including,
The 24th step,
Determining an operation and deformation of the virtual robot based on a part of the virtual robot corresponding to each of the second contact point and the third contact point, and the position and movement of the second contact point and the third contact point; And
Determining movements and deformations of the one or more objects based on portions of the one or more objects corresponding to each of the second and third contacts, and positions and movements of the second and third contacts; Including,
The eleventh step,
Determining a type of the at least one object;
Generating a movement path of the virtual robot based on the position and type of the one or more objects included in the captured image;
Obtaining a first mission for allowing the virtual robot to move along the movement path; And
Obtaining a second mission for causing the virtual robot to perform a predetermined interaction with at least one of the one or more objects; Including, coding education providing method using a virtual robot.

Images