KR102396390B1 - Method and terminal unit for providing 3d assembling puzzle based on augmented reality - Google Patents

Abstract

Disclosed are a method and a terminal for providing an augmented reality-based three-dimensional assembling puzzle. In this case, as a method for a terminal operated by at least one processor to provide an augmented reality (AR)-based three-dimensional assembly puzzle, the step of outputting a real image obtained through a camera installed in the terminal to a screen; When a touch input is generated on the screen, visualizing a grid in which a plurality of cells having geometric shapes are formed and a plurality of three-dimensional blocks having shapes corresponding to the plurality of cells; moving the generated three-dimensional block according to the touch input and arranging it in one of the plurality of cells; Determining whether the mission is completed, and when it is determined that the mission is completed, visualizing a predefined three-dimensional model corresponding to the completed body to which the three-dimensional blocks are assembled, wherein the disposing includes: The three-dimensional block can be placed in the single cell by using a snap function.

Description

A method and terminal for providing a three-dimensional assembly puzzle based on augmented reality {METHOD AND TERMINAL UNIT FOR PROVIDING 3D ASSEMBLING PUZZLE BASED ON AUGMENTED REALITY}

The present invention relates to a method and a terminal for providing a three-dimensional assembling puzzle based on augmented reality.

With the start of smart education and the evolution of various educational methods using technology, teaching methods are inevitably changed, and students are also focusing on and getting used to classes using the changed technology.

The 'experiential storytelling service', where infants and children can experience like a protagonist in a fairy tale in augmented reality, and the experience of observing the universe by going to outer space that cannot be visited directly, or experiencing the language and culture of a country without going abroad. The number of educational cases using augmented reality technology, such as the 'experiential learning service' that exists, is on the rise.

One problem to be solved by the present invention is to provide a method and a terminal for providing a three-dimensional assembling puzzle in the form of a puzzle game based on augmented reality (AR).

According to one feature of the present invention, there is provided a method in which a terminal operated by at least one processor provides an augmented reality (AR)-based 3D assembly puzzle, a real image obtained through a camera installed in the terminal. outputting on a screen, when a touch input is generated on the screen, visualizing a grid in which a plurality of cells having geometric shapes are formed and a plurality of three-dimensional blocks having shapes corresponding to the plurality of cells; moving the three-dimensional block in which the user's touch input has occurred among the three-dimensional blocks according to the touch input and arranging the three-dimensional block in one of the plurality of cells; Determining whether the mission that satisfies the set final target position value is completed, and when it is determined that the previous mission is completed, visualizing a predefined 3D model corresponding to the completed body in which the three-dimensional blocks are assembled, , in the arranging step, the three-dimensional block is arranged in the one cell using a snap function.

If the current position value of the three-dimensional block according to the movement displacement of the touch input does not reach any one of the plurality of cells, the snap function is based on the current position value and the position value of each of the plurality of cells. may be a function of calculating a distance value between the three-dimensional block and the plurality of cells, and automatically arranging the three-dimensional block as a cell having the smallest distance value.

In the arranging step, when the current position value according to the movement displacement of the touch input with respect to the first three-dimensional block in which the touch input has occurred is a column in which the second three-dimensional block is already arranged, the first three-dimensional block is The second three-dimensional block may be disposed in a cell, and the second three-dimensional block may be moved to a previous position of the first three-dimensional block.

In the arranging step, when the first three-dimensional block in which a touch input is generated overlaps with a second three-dimensional block already disposed at a specific point when the first three-dimensional block is moved according to the movement displacement of the touch input, the first three-dimensional block is placed in the second three-dimensional block. The first three-dimensional block is moved while floating to the upper part of the second three-dimensional block by the height of the block, or the second three-dimensional block is made transparent when the first three-dimensional block and the second three-dimensional block overlap. The visual effect passing through the two-dimensional block can be visualized on the screen.

The touch input includes starting a touch of a three-dimensional block, ending a touch releasing a touch from the three-dimensional block, moving the touch to a specific point on the screen while touching the three-dimensional block, and clicking a button provided in the three-dimensional block to determine the three-dimensional block. including a touch rotation that moves by an angle;

In the arranging step, the three-dimensional block is arranged in a specific cell on the grid through successive operations of the touch start, the touch end, the touch movement, and the touch rotation,

The completion of the mission is determined by determining whether the position value and rotation value of the three-dimensional block in which the touch input has occurred satisfy preset values;

As the final target position value for each three-dimensional block, the preset value may be a position value of a corresponding specific cell and a rotation value of the three-dimensional block in the specific cell.

In the three-dimensional block, a three-dimensional object including an event point is disposed, and after the step of visualizing the predefined three-dimensional object, when a user touch input for selecting the event point occurs, the event point is set The method may further include outputting a visual effect or an auditory effect.

According to another feature of the present invention, the terminal is a camera, a touch screen, a communication device that transmits and receives data through a network with a server device, and a memory for loading a program that provides a three-dimensional assembly puzzle based on Augmented Reality (AR) and a processor executing the program, wherein the processor outputs the real image acquired through the camera on the screen of the touch screen, and when a touch input signal is transmitted from the touch screen, a plurality of geometric shapes A grid in which cells are formed and a plurality of three-dimensional blocks having shapes corresponding to the plurality of cells are visualized on the screen of the touch screen, and a three-dimensional block in which a user's touch input has occurred among the plurality of three-dimensional blocks is applied to the touch input. If the current position value of the three-dimensional block according to the movement displacement of the touch input does not reach any one of the plurality of cells, the current position value and the The distance value between the three-dimensional block and the plurality of cells is calculated based on the position value of each of the plurality of cells, the three-dimensional block is automatically arranged in the cell with the smallest distance value, and the position value of the arranged three-dimensional block When the final target position value set in advance for each of the plurality of three-dimensional blocks is satisfied, a predefined three-dimensional body in which the three-dimensional blocks are assembled is visualized.

The terminal further includes an Inertial Measurement Unit (IMU), and the processor searches for a horizontal plane in the real image based on feature points extracted from the real image received from the camera and measurement data transmitted from the IMU, and the horizontal plane When a touch input occurs on the screen, the grid and the plurality of three-dimensional blocks are visualized, the touch starts to touch the three-dimensional block, the touch ends to release the touch from the three-dimensional block, and the three-dimensional block is touched up to a specific point on the screen The three-dimensional block is placed in a specific cell on the grid through continuous operations of moving touch movement and touch rotation of moving the three-dimensional block by a predetermined angle by clicking a button provided on the three-dimensional block, and the three-dimensional block in which the touch input is generated When the position value and rotation value of the block satisfy the position value of a specific cell previously set for each three-dimensional block and the rotation value of the three-dimensional block in the specific cell, the predefined 3D finished body may be visualized.

According to an embodiment of the present invention, by providing a three-dimensional assembling puzzle in the form of a puzzle game that is more interesting and immersive beyond the existing two-dimensional multimedia learning content, the culture, art, memorial and symbol of a specific country determined according to the scenario By providing various learning contents such as , animal and plant environment, climate, etc., it is possible to motivate learning and increase learning performance, not just visual pleasure.

1 is a system configuration diagram for providing a three-dimensional assembling puzzle based on augmented reality (AR) according to an embodiment of the present invention.
2 is a flowchart illustrating a method of providing a 3D assembling puzzle based on augmented reality according to an embodiment of the present invention.
3 is a flowchart illustrating a method of providing a 3D assembling puzzle based on augmented reality according to another embodiment of the present invention.
4 is an exemplary diagram of an initial augmented reality screen according to an embodiment of the present invention.
5 is an exemplary view of a screen in which a three-dimensional block and a grid are visualized according to an embodiment of the present invention.
6 and 7 are exemplary views for explaining a process of disposing a three-dimensional block according to an embodiment of the present invention.
8 is an exemplary view showing a complete assembly of three-dimensional blocks according to an embodiment of the present invention.
9 is an exemplary diagram of a three-dimensional model corresponding to an assembled body according to an embodiment of the present invention.
10 and 11 are exemplary views of a screen in which a three-dimensional block and a grid are visualized according to another embodiment of the present invention.
12 is an exemplary view for explaining a process of disposing a three-dimensional block according to another embodiment of the present invention.
13 is an exemplary diagram of three-dimensional blocks according to another embodiment of the present invention.
14 is an exemplary diagram of a 3D model in which the three-dimensional blocks of FIG. 13 are assembled.
15 is an exemplary diagram of three-dimensional blocks according to another embodiment of the present invention.
16 is an exemplary diagram of a three-dimensional model in which the assembly of the three-dimensional blocks of FIG. 15 is completed.
17 is a block diagram illustrating a hardware configuration of a terminal according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily carry out the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

In addition, terms such as “…unit”, “…group”, “…module”, etc. described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. can

The devices described in the present invention are composed of hardware including at least one processor, a memory device, a communication device, and the like, and a program to be executed in combination with the hardware is stored in a designated place. The hardware has the configuration and capability to implement the method of the present invention. The program includes instructions for implementing the method of operation of the present invention described with reference to the drawings, and is combined with hardware such as a processor and a memory device to execute the present invention.

1 is a system configuration diagram for providing a three-dimensional assembling puzzle based on augmented reality (AR) according to an embodiment of the present invention.

Referring to FIG. 1 , the terminal 100 accesses the application server 300 through the network 200 .

The terminal 100 is a mobile device possessed by a user, and refers to a device that transmits and receives various data according to a user's key manipulation or command and provides an augmented reality service. According to the embodiment, the terminal 100 is a tablet PC (Tablet PC), a laptop (Laptop), a personal computer (PC, Personal Computer), a smart phone (Smart Phone), a personal information terminal (PDA, Personal Digital Assistant), mobile It may include a mobile communication terminal and a notebook (NoteBook).

The terminal 100 includes a camera 101 , an Inertial Measurement Unit (IMU) 103 , a touch screen 105 , and a 3D assembly puzzle application 107 .

The camera 101 starts operating from the execution point of the three-dimensional assembling puzzle application 107 to photograph the surrounding environment of the terminal 100, and transmits the real image of the captured surrounding environment to the three-dimensional assembling puzzle application 107. do. In this case, the delivered real image may be a plurality of captured images delivered for a specified unit time according to a shooting speed (eg, pictures per second (PPS), frames per second (FPS)). That is, the real image taken in the video mode, not the photo mode, is transmitted to the three-dimensional assembling puzzle application 107 .

The IMU 103 may also be referred to as an inertial measurer. The IMU 103 may measure the change rate and displacement amount for posture change and position movement of the terminal 100 . The IMU 103 may be a combination of sensors configured to detect changes in the position and orientation of the terminal 100 based on inertial acceleration. For example, the combination of sensors may include accelerometers and gyroscopes. Since the operating principle of the IMU 103 is known, a detailed description thereof will be omitted.

The touch screen 105 functions as an input unit and a display unit. The touch screen 105 outputs the augmented reality screen output by the three-dimensional assembling puzzle application 107 . The touch screen 105 detects a user's touch input on the augmented reality screen and transmits it to the three-dimensional assembling puzzle application 107 .

The three-dimensional assembling puzzle application 107 operates in conjunction with the application server 300 . The 3D assembling puzzle application 107 is commonly referred to as an 'App.'

According to one embodiment, the 3D assembly puzzle application 107 may transmit a user's touch input signal to the application server 300 and control the output according to the control signal determined by the application server 300 . That is, the control subject for providing the augmented reality-based 3D assembly puzzle may be the application server 300 .

According to another embodiment, the 3D assembling puzzle application 107 serves as a provider of the augmented reality-based 3D assembling puzzle, and may operate in a state connected to the application server 300 .

The three-dimensional assembling puzzle application 107 provides a three-dimensional assembling puzzle based on augmented reality. The three-dimensional assembling puzzle application 107 outputs an augmented reality screen in which the three-dimensional assembling puzzle is visualized at the point touched by the user on the real image screen captured by the camera 101 .

The three-dimensional assembly puzzle is divided into a plurality of three-dimensional blocks, and each three-dimensional block is assembled to form a three-dimensional complete body having a specific shape.

Each three-dimensional block may have the same shape or different shapes. The shape of the three-dimensional body to which each three-dimensional block is assembled may be different from the shape of each three-dimensional block.

Each three-dimensional block may be implemented as one of a three-dimensional topography, a facility related to a specific area, and various movable three-dimensional contents, but this is only an example and is not limited thereto.

The three-dimensional assembly puzzle application 107 is a three-dimensional complete object and a variety of three-dimensional content such as famous architectural monuments of major cities, national food, natural beauty, culture and artworks, climate change, etc. that match the three-dimensional complete object. and audio explanations can be provided. Therefore, the three-dimensional assembling puzzle application 107 can provide three-dimensional assembling puzzle content in the form of a puzzle game that is more interesting and immersive than the two-dimensional multimedia learning content, and the culture and art of a specific country, souvenirs and symbols, animals and plants By providing various learning contents such as environment and climate, it is possible not only to enjoy visual pleasure, but also to motivate learning and increase learning performance.

The three-dimensional assembling puzzle application 107 may provide various three-dimensional assembling puzzles using augmented reality technology that can detect a horizontal plane without a separate puzzle learning aid or tracking marker.

The three-dimensional assembling puzzle may include, but is not limited to, a three-dimensional structure puzzle such as a building, a structure, or a plant, and a puzzle implementing a character.

The three-dimensional assembling puzzle application 107 includes an augmented reality (AR) engine 109 and a three-dimensional puzzle providing unit 111 . The augmented reality (AR) engine 109 and the 3D puzzle providing unit 111 are divided into specific functions or specific operations performed by the 3D assembling puzzle application 107 .

The augmented reality (AR) engine 109 performs various functions for implementing augmented reality in the terminal 100 . The three-dimensional puzzle providing unit 111 performs various functions for providing a three-dimensional assembling puzzle based on the augmented reality in conjunction with the augmented reality (AR) engine 109 .

The touch input transmitted from the touch screen 105 is provided to the augmented reality (AR) engine 109 and the 3D puzzle providing unit 111 . The augmented reality (AR) engine 109 and the 3D puzzle providing unit 111 may interpret the touch input to calculate the movement displacement. In this case, the 3D puzzle providing unit 111 may receive the movement displacement calculated by the augmented reality (AR) engine 109 .

The 3D puzzle providing unit 111 operates according to a predetermined scenario, and when the movement displacement is transmitted, it may control the augmented reality (AR) engine 109 to implement the corresponding augmented reality operation.

The augmented reality (AR) engine 109 detects a horizontal plane in the real image screen based on the real image transmitted from the camera 101 and the measurement data transmitted from the IMU 103, and derives three-dimensional coordinates of the horizontal plane. It may be provided to the dimension puzzle providing unit 111 . The three-dimensional puzzle providing unit 111 may control the augmented reality (AR) engine 109 to determine an object or object to be implemented on the augmented reality screen based on the three-dimensional coordinates, determine the object or action of the object, and implement it. there is. In this way, the augmented reality (AR) engine 109 performs an operation to implement the augmented reality screen, and the 3D puzzle providing unit 111 controls the augmented reality (AR) engine 109 in the embodiment of the present invention. Performs a series of control operations to provide a three-dimensional assembly puzzle according to the.

The augmented reality (AR) engine 109 may be implemented using Google's ARCore or Apple's ARKit.

The augmented reality (AR) engine 109 detects a horizontal plane in a real image by utilizing the feature points used in motion tracking technology of ARCore or ARKit. At this time, the augmented reality (AR) engine 109 extracts key points from the real image transmitted from the camera 101. Since the method of extracting the key points uses motion tracking technology of ARCore or ARKit, a detailed description will be omitted.

The augmented reality (AR) engine 109 determines the position and direction of the terminal 100 according to the movement of the terminal 100 based on the measurement data transmitted from the IMU 103 . At this time, the specific principle of determining the position and direction of the terminal 100 based on the measurement data received from the IMU 103 corresponds to a known technology, and thus a detailed description thereof will be omitted.

The augmented reality (AR) engine 109 detects a horizontal plane in the real image based on the feature points extracted from the real image and the location and direction of the terminal 100 . At this time, since the specific principle of detecting the horizontal plane follows the implementation principle of ARCore or ARKit, a detailed description will be omitted.

The augmented reality (AR) engine 109 may perform the operations shown in Table 1.

division explanation Object Recognition Recognizes objects in images through RGB color distribution and density analysis on the screen Object Detection Find out where an object is Object Tracking Check the position change of the object

The augmented reality (AR) engine 109 measures the size of the floor surface in the real image seen through the camera 101 through object recognition, selects the location of the floor surface through object detection, and the floor surface through object tracking change the position of

The augmented reality (AR) engine 109 recognizes objects (measures the size of the floor), object detection (selects the location of the floor), and tracks objects (floor position change) to recognize a floor object, that is, a horizontal plane, and detect a singularity constituting the horizontal plane.

The real image screen acquired through the camera 101 is composed of numerous pixels, and the shape of an object can be defined by extracting pixels having similar RGB colors and concentrations at regular distance intervals based on the three-dimensional axis of the user's point of view.

The position of the object can be detected through the change of the camera viewpoint by the user, and a pixel with a similar height value (z-axis height value) is defined as a singularity. By storing the information (3D axis information, color information, location information) of the singularity derived in this way, the correlation and continuity of the singularity are checked through the 3D axis change amount and object tracking when the camera is moved to another viewpoint to check the floor surface. Recognize.

The horizontal plane sensed by the augmented reality (AR) engine 109 becomes the basic base on which the three-dimensional blocks are placed.

The horizontal plane sensed by the augmented reality (AR) engine 109 is a region having a certain range on the screen output by the touch screen 105 , and the region is expressed by three-dimensional coordinates (x, y, z). When the user touches the horizontal plane on the screen, the augmented reality (AR) engine 109 receives the touch input from the touch screen 105, and derives a location (three-dimensional coordinates) of the point where the touch input occurs on the horizontal plane. there is.

The 3D puzzle providing unit 111 visualizes a grid and three-dimensional blocks at a location close to a location where the touch input derived from the augmented reality (AR) engine 109 is generated.

The 3D puzzle providing unit 111 randomly arranges the three-dimensional blocks at a point close to the point where the grid is visualized through a random function.

Here, the grid guides the positions where the three-dimensional blocks are to be placed, and as shown in FIG. 5 to be described later, the grid P15 may have a two-dimensional shape of a finished body. For example, the shape of the grid may be various geometric shapes such as a square shape, a hexagon shape, and the like.

The three-dimensional puzzle providing unit 111 outputs the real image acquired through the camera 101 on the screen of the touch screen 105 . When a touch input occurs on the screen output to the touch screen 105 , the 3D puzzle providing unit 111 drives the augmented reality (AR) engine 109 to generate a grid and a plurality of squares having geometric shapes. A plurality of three-dimensional blocks having a shape corresponding to a cell of are visualized on the touch screen 105 .

The three-dimensional puzzle providing unit 111 moves the three-dimensional block generated by the user's touch input among the plurality of three-dimensional blocks according to the touch input, and arranges the three-dimensional block in one of the plurality of cells. At this time, it is determined whether the mission in which the position value of the arranged three-dimensional block satisfies the final target position value set in advance for each of the plurality of three-dimensional blocks is completed.

When it is determined that the mission is completed, the 3D puzzle providing unit 111 visualizes a predefined 3D model corresponding to the completed body in which the three-dimensional blocks are assembled.

The 3D puzzle providing unit 111 interprets a touch input transmitted from the touch screen 105 to calculate a movement displacement, and determines a user action based on this.

A user action is an action for assembling a three-dimensional block, and can be broadly defined as shown in Table 2 below.

Action Classification Action Description Touch Start (Touch_Begin) Selected by touching a solid block Touch End (Touch_End) Release the touch on the screen when the three-dimensional block is at the position you want to place (at this time, the snap function works as soon as the touch is terminated, and the three-dimensional block is automatically placed at the nearest point in the grid) Touch_Move Move to a specific location while touching a three-dimensional block touch rotation As a rotation button, it can be formed at a specific point on the screen or on a visualized three-dimensional block. Touching the rotate button rotates the three-dimensional block at a predefined rotation angle (eg, one touch rotates 15 degrees, two touches 45 Rotate degrees, touch 3 times to rotate 90 degrees)

As shown in Table 2, the three-dimensional puzzle providing unit 111 starts a touch to touch a three-dimensional block, ends a touch to release a touch from the three-dimensional block, moves a touch to a specific point on the screen while touching the three-dimensional block, and moves a three-dimensional block The augmented reality (AR) engine 109 is controlled to place the three-dimensional block in a specific cell on the grid through a continuous operation of touch rotation to move the three-dimensional block by a predetermined angle by clicking a button provided in the .

The three-dimensional blocks are set so that the user can directly move and rotate them. In this case, the rotation angle of each three-dimensional block is preset to a specific angle. For example, the rotation angle may be 15 degrees, 45 degrees, or 90 degrees. When a user's touch input for rotating a three-dimensional block is transmitted from the touch screen 105, the 3D puzzle providing unit 111 calculates a movement displacement by interpreting the touch input, and the rotation angle according to the movement displacement exceeds 15 degrees. Alternatively, even within 15 degrees, the three-dimensional block can be rotated only by 15 degrees to be visualized on the screen of the touch screen 105 .

The 3D puzzle providing unit 111 may induce the three-dimensional blocks to be arranged in one cell by using a snap function. Specifically, when the current position value of the three-dimensional block according to the movement displacement of the touch input does not reach any one of the plurality of cells, the 3D puzzle providing unit 111 provides the current position value and the position value of each of the plurality of cells. By calculating the distance value between the three-dimensional block and the plurality of cells based on the , a snap function can be implemented that automatically arranges the three-dimensional block to the cell with the smallest distance value.

That is, a snap function is provided according to the distance between the position where the three-dimensional block has moved and the positions of each of the grid cells. According to the snap function, even if the three-dimensional block does not exactly reach the position of the grid column according to the touch input, the three-dimensional block can be automatically placed in the grid column closest to the current position of the three-dimensional block moved according to the touch input.

The three-dimensional puzzle providing unit 111 stores each position information of the generated grid cells in the List function. Then, the position of the active object and the reach value of each grid cell stored in the List function are calculated in real time, and the position of the grid cell with the smallest distance value is determined as the snapped position. However, the snap function is executed when a touch input corresponding to the end of the touch in Table 1 occurs, and is not executed when the distance between the movement displacement value of the three-dimensional block and the grid cell exceeds a certain distance value.

The 3D puzzle providing unit 111 may process an object according to a touch input event as follows. That is, when the current position value according to the movement displacement of the touch input with respect to the first three-dimensional block in which the touch input has occurred is a column in which the second three-dimensional block is already arranged, the three-dimensional puzzle providing unit 111 is configured to provide the first three-dimensional block. The block may be placed in a cell in which the second three-dimensional block is disposed, and the second three-dimensional block may be moved to a previous position of the first three-dimensional block. That is, the second three-dimensional block is moved to the position where the first three-dimensional block is initially arranged, and the first three-dimensional block is arranged in the grid column where the second three-dimensional block is arranged.

In addition, when the 3D puzzle providing unit 111 moves the first three-dimensional block in which the touch input has occurred according to the movement displacement of the touch input and overlaps with the second three-dimensional block already disposed at a specific point, the first three-dimensional block is generated. The height of the two-dimensional block can be moved while floating on the upper part of the second three-dimensional block. Alternatively, the three-dimensional puzzle providing unit 111 makes the second three-dimensional block transparent at the point in time when the first three-dimensional block and the second three-dimensional block overlap, thereby displaying a visual effect in which the first three-dimensional block passes through the second three-dimensional block on the screen. The augmented reality (AR) engine 109 may be controlled to make it visible.

The three-dimensional puzzle providing unit 111 determines whether the mission for the three-dimensional block is completed whenever a touch end event for the three-dimensional block occurs. The 3D puzzle providing unit 111 may determine whether all three-dimensional blocks have reached a target value or are still in progress, and may check mission completion.

The three-dimensional puzzle providing unit 111 may store dictionary information about the three-dimensional block as shown in Table 3.

item explanation Texture Texture image applied to 3D shape (mountain, water, earth, etc.) Mission completion information Target position value and rotation value: Position value and rotation value of x, y, z value corresponding to the predefined final target (grid cell)
Position and rotation values before activation a) When the block is first placed by a random function, the position and rotation values are stored b) When the active object moves to another puzzle piece located on the grid, it intersects with each other (event occurs) and the position relative to the newly moved position and update the rotation value
c) When the active object moves to a place where there is no puzzle piece, only the position and rotation values for the newly moved position are updated without an event.
Event point information -Various events (voice provision, animation implementation, image output, etc.) can be provided depending on the puzzle scenario - Event occurrence occurs by directly touching a point located on the block
-Event point information has the following information
a) Relative coordinates on the block to the position of the point
b) parameters for the nature of the event to be raised;

In Table 3, the activation object refers to a three-dimensional block in which a touch input is generated.

The 3D puzzle providing unit 111 may determine whether the position value and rotation value of the three-dimensional block in which the touch input has occurred satisfy preset values. Here, the preset value means mission completion information in Table 2, and as the final target position value for each three-dimensional block, the position value of the corresponding specific cell and the rotation value of the three-dimensional block in the specific cell are respectively set.

In the three-dimensional block, a three-dimensional object including an event point may be disposed. After the three-dimensional puzzle providing unit 111 visualizes the three-dimensional complete body assembled with the three-dimensional blocks, when a touch input for selecting an event point on the three-dimensional block occurs, the visual or auditory effect set at the event point is output. Augmented reality (AR) engine 109 may be controlled.

In this case, the 3D puzzle providing unit 111 may provide various effects according to the puzzle scenario, for example, effects such as voice provision, animation implementation, image output, and the like. The event is generated by the user directly touching the event point located on the three-dimensional block. For example, there is a 3D puzzle set in a desert, and each three-dimensional block may have 3D objects such as a cactus, a camel, and a pyramid arranged respectively. Event points are planted in the 3D object, and when the user selects a camel, the 3D puzzle providing unit 111 checks the nature of the event, and an augmented reality (AR) engine to output a voice description or a camel image for the corresponding camel animal. (109) can be controlled.

The three-dimensional puzzle providing unit 111 may store the position and rotation values when the three-dimensional block is first placed by a random function. The three-dimensional puzzle providing unit 111 intersects each other when the active three-dimensional block moves to a place with another three-dimensional block located on the grid (event occurs) and updates the position and rotation values for the newly moved position. can When the three-dimensional block in which the touch input has occurred moves to a place where there are no other three-dimensional blocks, the three-dimensional puzzle providing unit 111 may newly update only the position and rotation values of the newly moved position without an event.

Based on the configuration described above, a series of processes for providing an augmented reality-based 3D assembling puzzle will be described.

2 is a flowchart illustrating a method of providing an augmented reality-based 3D assembling puzzle according to an embodiment of the present invention, which will be described in connection with the configuration of FIG. 1 .

Referring to FIG. 2 , the 3D puzzle providing unit 111 outputs a real-time image screen acquired through the camera 101 to the touch screen 105 ( S101 ).

The three-dimensional puzzle providing unit 111 detects a horizontal plane based on the image acquired from the camera 101 and the sensor data acquired from the IMU 103 (S103).

When a user touch input is sensed in the horizontal plane area (S105), the 3D puzzle providing unit 111 includes 3D stereoscopic blocks and The augmented reality screen in which the grid is visualized is output (S107).

When a series of touch input operations for selecting, moving, ending, and rotating the three-dimensional block as shown in Table 1 is detected ( S109 ), the three-dimensional puzzle providing unit 111 places the three-dimensional three-dimensional block on the grid according to the touch input operation. Arranged in a plurality of cells (S111).

The three-dimensional puzzle providing unit 111 determines whether the mission is complete (S113), and if the mission is not completed, starts again from step S109. On the other hand, when the mission is completed, an augmented reality screen in which the three-dimensional model corresponding to the finished body to which the three-dimensional blocks are assembled is visualized (S115).

3 is a flowchart illustrating a method of providing an augmented reality-based 3D assembly puzzle according to another embodiment of the present invention, which will be described in connection with the configuration of FIG. 1 .

Referring to FIG. 3 , the three-dimensional puzzle providing unit 111 detects a touch input for selecting an event point disposed on the three-dimensional three-dimensional block in a state in which the three-dimensional three-dimensional block is visualized on the real-time image screen (S201) (S203). Then, the multimedia content matching the event point is output (S205). Here, the multimedia content may provide a visual effect or an auditory effect, and may be as described in Table 2.

Embodiments to which the contents described above with reference to FIGS. 1 to 3 are applied will be described.

4 is an exemplary view of an initial augmented reality screen according to an embodiment of the present invention, FIG. 5 is an exemplary view of a screen in which a three-dimensional block and a grid are visualized according to an embodiment of the present invention, FIGS. 6 and 7 are It is an exemplary view for explaining a process of arranging a three-dimensional block according to an embodiment of the present invention, Fig. 8 is an exemplary view showing a complete assembly of three-dimensional blocks according to an embodiment of the present invention, and Fig. 9 is a diagram of the present invention It is an exemplary diagram of a three-dimensional model corresponding to an assembled body according to an embodiment.

When the user runs the 3D assembling puzzle application 107 and drives the camera, a horizontal plane in which a plurality of circles are arranged in a uniform arrangement may be displayed on the real image screen P11 as shown in FIG. 4 .

When the user touches a specific point on the horizontal plane, a plurality of three-dimensional blocks P13 and a grid P15 are displayed at the touched point or a point close to the touched point as shown in FIG. 5 . At this time, a plurality of cells are formed in the grid P15 .

6 and 7, when the user touches and selects the three-dimensional block P17, moves it to one of a plurality of cells on the grid P15, rotates it, and ends, the three-dimensional block ( P17) is placed. The three-dimensional block P17 is rotated and then disposed so that the rotation value according to the specified scenario, that is, the assembly can be completed.

When the mission is completed after all the three-dimensional blocks P13 are arranged on the grid P15, the finished body P19 assembled with the three-dimensional blocks P13 as shown in FIG. 8 is visualized, and then the finished body as shown in FIG. A three-dimensional model P21 corresponding to is visualized. After the 3D model P21 is visualized, the operation of the 3D assembly puzzle application 107 may be initialized.

Hereinafter, since the embodiment of FIGS. 10 to 16 is another embodiment of FIGS. 4 to 9 , a duplicate description will be omitted and only another embodiment will be described.

10 and 11 are exemplary views of a screen in which a three-dimensional block and a grid are visualized according to another embodiment of the present invention, and FIG. 12 is an exemplary view for explaining a process of disposing a three-dimensional block according to another embodiment of the present invention. , FIG. 13 is an exemplary diagram of three-dimensional blocks according to another embodiment of the present invention, FIG. 14 is an exemplary diagram of a three-dimensional model in which the assembly of the three-dimensional blocks of FIG. 13 is completed, and FIG. 15 is another embodiment of the present invention It is an exemplary diagram of three-dimensional blocks according to an example, and FIG. 16 is an exemplary diagram of a three-dimensional model in which the assembly of the three-dimensional blocks of FIG. 15 is completed.

Referring to FIG. 10 , the grid P23 may be displayed in a grid pattern. Referring to FIG. 11 , three-dimensional blocks P25 having a hexahedral shape may be disposed next to a grid P23 of a grid pattern. Referring to FIG. 12 , a three-dimensional block may be arranged in a cell of the grid P23 through a series of operations in which the user selects the three-dimensional block and moves, rotates, and terminates the three-dimensional block.

At this time, the three-dimensional blocks are three-dimensional blocks P31 having an arbitrary shape as shown in FIG. 13, and if they are arranged on a grid of a grid pattern, a palace model P33 as shown in FIG. 14 can be completed. In this case, if event points provided in the three-dimensional blocks P31 of FIG. 13 are clicked on, contents related to the palace model P33 may be output.

In addition, the three-dimensional block is three-dimensional blocks P35 having an arbitrary shape as shown in FIG. 15 , and when these are arranged on a grid of a grid pattern, a pyramid model P37 can be completed as shown in FIG. 16 . In this case, if event points provided in the three-dimensional blocks P35 of FIG. 15 are clicked on, contents related to pyramid history, Egyptian history, and the like may be output.

Meanwhile, FIG. 17 is a block diagram illustrating a hardware configuration of a terminal according to an embodiment of the present invention.

Referring to FIG. 17 , the terminal 400 includes a processor 401 , a memory 403 , a touch screen 405 , an IMU 407 , a camera 409 , a network interface 411 and a storage 413 , and , they may communicate via bus 415 .

The processor 401 may be configured to implement the methods described based on FIGS. 1 to 16 . The processor 401 may be a central processing unit (CPU) or a semiconductor device that executes instructions stored in the memory 403 or the storage 413 .

Here, the processor 401 may be configured to perform the function of the 3D puzzle application 107 according to the above embodiment.

The memory 403 is connected to the processor 401 and stores various information related to the operation of the processor 401 .

The memory 403 may store instructions to be executed by the processor 401 or may temporarily store instructions by loading instructions from the storage 413 . The processor 401 may execute instructions stored in or loaded from the memory 403 .

In an embodiment of the present invention, the memory 403 may be located inside or outside the processor 401 , and the memory 403 may be connected to the processor 401 through various known means.

The network interface 411 is connected to the network 200 and is configured to transmit and receive signals. In particular, the network interface 411 is configured to provide data received from the network 200 to the processor 401 .

It should be understood that the systems, apparatuses, and methods disclosed in the embodiments of the present invention may be implemented in other ways. For example, the disclosed device embodiments are by way of example only. For example, the division of processing units in a system is merely a logical division of functions, and may be other divisions during actual implementation. For example, multiple devices or components may be combined or integrated into other systems, or some functions may be ignored or not performed. In addition, the devices configured to perform the method according to the embodiment of the present invention may be integrated into one device, each of the devices may exist alone physically, or two or more devices may be integrated into one device. An apparatus configured to perform the method described in the above embodiments may be implemented in the form of hardware or may be implemented in the form of a software functional unit.

The embodiment of the present invention described above is not implemented only through the apparatus and method, and may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims are also provided. is within the scope of the right.

Claims (8)

As a method of providing a three-dimensional assembling puzzle based on augmented reality (AR) in a terminal operated by at least one processor,
outputting the real image acquired through the camera installed in the terminal on the screen;
visualizing a grid in which a plurality of cells having geometric shapes are formed and a plurality of three-dimensional blocks having shapes corresponding to the plurality of cells when a touch input is generated on the screen;
moving a three-dimensional block in which a user's touch input has occurred among the plurality of three-dimensional blocks according to the touch input and arranging the three-dimensional block in one of the plurality of cells;
determining whether a mission in which the position value of the arranged three-dimensional block satisfies a preset final target position value for each of the plurality of three-dimensional blocks is completed; and
When it is determined that the mission is completed, visualizing a predefined three-dimensional model corresponding to the completed body in which the three-dimensional blocks are assembled,
The arranging step is
Place the three-dimensional block in the one cell using a snap function,
When the current position value according to the movement displacement of the touch input with respect to the first three-dimensional block in which the touch input has occurred is a column in which the second three-dimensional block is already disposed among the plurality of cells, the first three-dimensional block is the second three-dimensional block A method of placing in a placed cell and moving the second solid block to a previous position of the first solid block. In claim 1,
The snap function is
When the current position value of the three-dimensional block according to the movement displacement of the touch input does not reach any one of the plurality of cells, the three-dimensional block and A function of calculating a distance value between the plurality of cells and automatically arranging the three-dimensional block in a cell having the smallest distance value. delete In claim 1,
The arranging step is
When the first three-dimensional block in which a touch input has occurred is moved according to the movement displacement of the touch input and overlaps with a second three-dimensional block already disposed at a specific point, the first three-dimensional block is divided into the second three-dimensional block by the height of the second three-dimensional block. It moves while floating on the upper part of the two-dimensional block, or makes the second three-dimensional block transparent when the first three-dimensional block and the second three-dimensional block overlap, so that the first three-dimensional block passes through the second three-dimensional block. A method of visualizing a visual effect on the screen. In claim 1,
The touch input is
Touch start by touching a three-dimensional block, end touch to release a touch from a three-dimensional block, touch movement that moves to a specific point on the screen while touching the three-dimensional block, and touch that moves the three-dimensional block by a predetermined angle by clicking a button provided on the three-dimensional block including rotation,
The arranging step is
Placing the three-dimensional block in a specific cell on the grid through successive operations of the touch start, the touch end, the touch movement, and the touch rotation,
Whether the mission has been completed,
determining whether the position value and rotation value of the three-dimensional block in which the touch input has occurred satisfy preset values;
The preset value is
The method of claim 1 , wherein a position value of a specific cell and a rotation value of the stereoscopic block in the specific cell are set as the final target position value for each three-dimensional block. In claim 1,
The three-dimensional block is
A three-dimensional object including event points is placed,
After the step of visualizing the predefined three-dimensional finished body,
When a user touch input for selecting the event point is generated, outputting a visual effect or an auditory effect set at the event point
A method further comprising: camera,
touch screen,
A communication device that transmits and receives data through a server device and a network;
A memory for loading a program that provides a three-dimensional assembly puzzle based on Augmented Reality (AR), and
a processor executing the program;
The processor is
The real image acquired through the camera is output on the screen of the touch screen, and when a touch input signal is transmitted from the touch screen, a grid having a plurality of geometrical squares formed therein and a shape corresponding to the plurality of squares Visualize a plurality of three-dimensional blocks on the screen of the touch screen,
Among the plurality of three-dimensional blocks, the three-dimensional block generated by the user's touch input is moved according to the touch input and disposed in one of the plurality of cells, wherein the current position value of the three-dimensional block according to the movement displacement of the touch input is If it does not reach any one of the plurality of cells, the distance value between the three-dimensional block and the plurality of cells is calculated based on the current position value and the position values of each of the plurality of cells, and the distance value is the most Automatically arranges the three-dimensional block in the smallest number of spaces,
When the position value of the arranged three-dimensional block satisfies the final target position value set in advance for each of the plurality of three-dimensional blocks, a predefined three-dimensional body in which the three-dimensional blocks are assembled is visualized;
When the current position value according to the movement displacement of the touch input with respect to the first three-dimensional block in which the touch input has occurred is a column in which the second three-dimensional block is already disposed among the plurality of cells, the first three-dimensional block is the second three-dimensional block A terminal that is placed in an arranged cell and moves the second three-dimensional block to a previous position of the first three-dimensional block. In claim 7,
Further comprising an Inertial Measurement Unit (IMU),
The processor is
A horizontal plane is searched in the real image based on the feature points extracted from the real image received from the camera and the measurement data transmitted from the IMU, and when a touch input is generated on the horizontal plane, the grid and the plurality of three-dimensional blocks are visualized, ,
The three-dimensional block is determined by starting a touch of touching the three-dimensional block, ending the touch releasing the touch from the three-dimensional block, moving the touch to a specific point on the screen while touching the three-dimensional block, and clicking a button provided on the three-dimensional block Placing the three-dimensional block in a specific cell on the grid through a continuous operation of touch rotation that moves by an angle,
When the position value and rotation value of the three-dimensional block in which the touch input has occurred satisfy the position value of a specific cell previously set for each three-dimensional block and the rotation value of the three-dimensional block in the specific cell, the predefined 3D finished body is visualized to do, terminal.

Images