KR20230126187A - Device and method for providing contents based on augmented reality - Google Patents

Abstract

An embodiment of the present invention provides an apparatus and method for implementing content through augmented reality when a plurality of markers around a miniature are recognized. The method includes the steps of: generating data for a miniature structure; recognizing an image of at least one marker; recognizing a shooting direction; matching the shooting direction and the data; and outputting content.

Description

Apparatus and method for implementing contents using augmented reality {DEVICE AND METHOD FOR PROVIDING CONTENTS BASED ON AUGMENTED REALITY}

The present invention relates to an apparatus and method for generating a miniature, which is the background of content implemented in augmented reality, as modeled data, and enabling the modeled data to be implemented as augmented reality through a device. will be.

The contents described below are only described for the purpose of providing background information related to an embodiment of the present invention, and the contents described do not naturally constitute prior art.

Augmentation Reality is a technology that superimposes virtual objects on the real world that users see with their eyes. It combines the virtual world with additional information in real time into the real world and shows it as a single image. ) is also called

Augmented reality seems to be similar to virtual reality, but it is clearly distinguished depending on whether the subject is virtual or real. Specifically, the difference between virtual reality and augmented reality may be determined depending on whether or not the place where the protagonist is located is a virtual new space.

A limitation of such augmented reality technology is that a server is used so that a plurality of users can simultaneously view augmented reality content that one user is viewing, but development costs for this increase.

In addition, only virtual characters implemented in augmented reality are implemented in augmented reality. For example, when the virtual character is a tiger, only the tiger is implemented in augmented reality through the device, and the video captured by the device is not related to the tiger. Therefore, there is a limit in that a user using the service may come into contact with the virtual character with a sense of being separated from surrounding objects.

The foregoing background art is technical information that the inventor possessed for derivation of the present invention or acquired during the derivation process of the present invention, and cannot necessarily be said to be known art disclosed to the general public prior to filing the present invention.

An embodiment of the present invention is to provide an apparatus and method for implementing content using augmented reality that, when implementing content through augmented reality through a device, enables a device to implement a miniature image related to the content together with the content.

In addition, in the embodiment of the present invention, after generating miniature modeled data that is output along with the content, the content is implemented on the generated modeled data so that the content and the miniature can be reproduced even in an area where the miniature and the content interfere. An object of the present invention is to provide an apparatus and method for implementing content using augmented reality capable of minimizing independent output or unnatural output of content.

In addition, in an embodiment of the present invention, when a plurality of markers including content information are disposed around a miniature and a single device recognizes a first marker and a second marker among the plurality of markers at different times, first recognized It is an object of the present invention to provide an apparatus and method for implementing content using augmented reality that enables content to be output based on a point in time at which a first marker is recognized.

Unlike this, in an embodiment of the present invention, when a plurality of markers including content information are arranged around a miniature, and a plurality of devices recognize a first marker and a second marker among the plurality of markers at different times, each device To provide a content implementation device and method using augmented reality that allows the same content to be output at different times.

The object of the present invention is not limited to the above-mentioned tasks, and other objects and advantages of the present invention not mentioned above can be understood by the following description and will be more clearly understood by the embodiments of the present invention. It will also be seen that the objects and advantages of the present invention may be realized by means of the instrumentalities and combinations indicated in the claims.

A method of implementing content using augmented reality according to an embodiment of the present invention generates modeled data for a miniature structure that becomes a background of content implemented in augmented reality through a device, and creates a modeled data on the miniature and around the miniature. After recognizing an image of at least one marker including the content information, the device recognizes a photographing direction in which the miniature is photographed based on a preset front of the miniature, and determines a photographing direction and a direction corresponding to the photographing direction. After matching the modeled data for the miniature structure in a parallel direction, when the modeled data and the miniature structure match, a process of outputting the content through the device may be performed.

In an embodiment, the process of recognizing an image includes recognizing a first marker among the at least one marker, and using a second marker at a position different from that of the first marker based on a first viewpoint at which the first marker is recognized. It may be performed as a process of recognizing at a second time point later than the first time point. At this time, the content may be output based on the first viewpoint.

In an embodiment, when recognizing an image, the first marker is recognized through a first device adjacent to a first marker among the at least one marker, and a second marker at a position different from the first marker is detected through a second device. It can be recognized at a different time point from the first point of view. In this case, the first device and the second device may output the same content at different times.

In an embodiment, the content is plural, and the first content output time may be set at a first time after a predetermined time from the output time of outputting the content, and the first content output time is the first content output time, When the first device recognizes the first marker, the first content can be reproduced.

In an embodiment, the modeled data may be generated to correspond to the miniature appearance.

In an embodiment, when it is determined that the modeled data and the miniature structure information do not match, the miniature image re-recognition notification may be notified.

In an embodiment, the content may be output when the miniature and the marker are simultaneously recognized.

In an embodiment, when the movement of the device is detected, the content may be moved along the movement direction of the device.

A content implementation device according to an embodiment of the present invention includes a processor and a memory electrically connected to the processor and storing at least one code executed by the processor, wherein the memory is to be executed by the processor. When the processor generates modeled data for a miniature structure that becomes a content background implemented in augmented reality through a device, and displays the miniature and at least one marker disposed around the miniature, including the content information, as an image After recognition, after recognizing a photographing direction in which the device photographs the miniature based on the front of the preset miniature, the modeled data for the miniature structure in a direction parallel to the photographing direction and the direction corresponding to the photographing direction When the miniature structures match, codes that cause the contents to be output through the device may be stored.

Other aspects, features, and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

An embodiment of the present invention implements augmented reality using a device, and content may be realized in the implemented augmented reality. In this case, the content may be implemented at a location adjacent to a miniature, which is an actual object used as a background screen of the content (eg, around the miniature, inside the miniature, etc.). Here you can place markers around the miniature or on the miniature itself to bring content to life. When the placed marker is recognized through the device, content may be output around the miniature based on content information stored in the marker. As a result, content implemented through augmented reality can be implemented more realistically without being separated from surrounding objects.

Also, the content can be output without interfering with the miniatures. Specifically, modeled data of a miniature may be generated in order to output a video of a miniature along with content from a device. It may be pre-stored in a device and a server, and when the device recognizes a miniature image (image) matching the stored modeling data, modeling data corresponding to the miniature may be output. At this time, the content output through the marker is recognized as the background behind the modeling data, for example, when the content and the miniature interfere, such as when the content should be output behind a miniature wall, either the content or/and the miniature is independently displayed. Make it output so that the content can be output naturally.

Meanwhile, a plurality of markers including content information may be arranged around the miniature. In this case, when a first marker and a second marker adjacent to the first marker are recognized among a plurality of markers through one device, the content can be output based on the previously recognized first marker. That is, even if a plurality of markers are recognized at different points in time, content can be output without overlapping.

Alternatively, a plurality of markers may be recognized by different devices. That is, when the first device recognizes a first marker among the markers and the second device recognizes a second marker, the first device and the second device can view the same content even at different viewpoints. That is, the same content information may be included in each of a plurality of markers, and even if different devices recognize markers at different positions at different times, the same content may be implemented in each device.

In this case, when each device recognizes the marker at different points of time, the start point of outputting the content may be set to be the same. That is, even if the first time point at which the first device recognizes the first marker is earlier than the second time point at which the second device recognizes the second marker, the second device sets the second device to the second device regardless of the time point at which content is output to the first device. By outputting the content based on the point in time at which the marker is recognized, a plurality of users can use the same content even if the content is implemented through different devices.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram schematically illustrating a communication environment of a device for implementing content through augmented reality according to an embodiment of the present invention.
2 is a diagram illustrating a miniature modeling process for implementing content through augmented reality according to an embodiment of the present invention.
3 and 4 are diagrams illustrating an example in which a plurality of markers are arranged around a miniature according to an embodiment of the present invention.
5 is a diagram illustrating an example of inputting content information output using augmented reality to each of a plurality of markers according to an embodiment of the present invention.
6 is a diagram illustrating conditions for outputting content through a plurality of markers according to an embodiment of the present invention.
7 is a block diagram showing the configuration of a device that implements content through augmented reality according to an embodiment of the present invention.
8 is a block diagram showing the configuration of a server connected to a device for implementing content through augmented reality according to an embodiment of the present invention.
9 is a flowchart illustrating a process of implementing content through augmented reality according to an embodiment of the present invention.

Hereinafter, the embodiments invented in this specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, when it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification in describing the embodiments disclosed herein, the detailed descriptions thereof will be omitted. In addition, the accompanying drawings are only for making it easy to understand the embodiments invented in this specification, and the technical ideas invented in this specification are not limited by the accompanying drawings, and are included in the spirit and technical scope of the present invention. It should be understood to include all modifications, equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Expressions in the singular number include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Hereinafter, with reference to the drawings, a device for implementing content through augmented reality according to the present invention will be described in detail.

1 is a diagram schematically illustrating a communication environment of a device for implementing content through augmented reality according to an embodiment of the present invention, and FIG. 2 illustrates a miniature modeling process for implementing content through augmented reality according to an embodiment of the present invention. It is an illustrated drawing.

Referring to the drawing, in an environment 10 implementing an apparatus for implementing content through augmented reality, a miniature 200 including a marker 100, a device 50, and a server 300 may communicate with each other through a network. .

Briefly looking at the environment 10 implementing content through augmented reality, first, a user who wants to use content may use the device 50 to recognize the miniature 200 and the marker 100 as images. When the marker 100 and the miniature 200 are image-recognized, related content may be implemented in the device 50 according to content information pre-stored in the marker 100 . At this time, the implemented content is content implemented through augmented reality, and may be any one of content related to the miniature 200 information and a virtual character. do it with

Augmentation reality, in which content is implemented, is a technology that superimposes virtual objects on the real world seen by the user.

The marker 100 refers to an arbitrary object easily recognized by computer vision technology, and mainly uses a unique color or pattern on a black background, a geometric shape, or a three-dimensional object. For example, it may be any one of the forms of inputting an identification code consisting of a recently used QR code, a letter, a number, or a combination thereof. Unlike this, the marker 100 can also be implemented in an image such as a photograph, and the marker 100 of the embodiment of the present invention will be described as an example consisting of a unique color, pattern or geometric shape on a black background, but the marker 100 The present invention is not limited by the form of.

As shown in FIGS. 3 to 5 to be described below, the marker 100 may be printed on the miniature 200 so that the miniature 200 and the marker 100 may be integrated. ) may be separately arranged around the periphery. The markers 100 separately disposed around the miniature 200 may be formed on one of the front and side surfaces of the miniature 200, but may be disposed on all of the front, side, and rear surfaces of the miniature 200. . In the following description, an example in which the plurality of markers 100 are disposed on all of the front, side, and rear surfaces of the miniature 200 will be described.

The miniature 200 may be a real object disposed adjacent to the marker 100 . For example, the miniature 200 may be an object that expresses the surrounding environment of the content implemented through the marker 100. When the specifically implemented content is an animal, the miniature 200 may be a tree, grass, mini forest, and the like. and, when the content is a game character, it may be a space representing a game environment.

The device 50 includes a photographing unit (55_refer to FIG. 7) capable of recognizing the marker 100 and the miniature 200, and the marker 100 and the miniature 200 recognized by the photographing unit 55 are captured. It is a device capable of generating content based on information about. Such a device 50 is, for example, an HMD display 50A capable of displaying based on MR (Mixed Reality) or AR/VR, a portable terminal 50B to display through any one application, and an AR image in the form of glasses. It may be any one of the provided smart glasses 50C.

In other words, the device 50 is a device that generates a virtual space image for a virtual character called through the marker 100 . Such a device 50 can recognize the image of the marker 100 and the miniature 200 at the same time, and the device 50 includes the miniature 200 and the miniature 200 around the marker 100, which is information included in the content ( 400) may be output.

In addition, the device 50 may include an application, program, etc. in which content can be implemented in augmented reality. ) and at the same time, when the related application is operated or the marker 100 and the miniature 200 are taken after the application is operated, the content 400 can be implemented in augmented reality as an example.

Here, the content 400 output through the device 50 may be output at a location adjacent to the miniature 200, which is a real space. In addition, the content 400 may move along with the movement direction of the device 50 and may move along with the gaze of the user using the device 50 (actual gaze toward the device).

As described above, the miniature 200 may be an actual object related to the content 400 . Meanwhile, since the content 400 is output around the miniature 200, a user using augmented reality can access content that is in harmony with the surrounding environment.

Referring to FIG. 1 again, looking at the process of implementing content using augmented reality, the user can image the marker 100 around the miniature 200 using the device 50 . In this case, the marker 100 may be disposed around the tree-shaped miniature 200 . Also, the user may recognize the marker 100 and the miniature 200 at the same time.

After that, information stored in the marker 100 may be output through the device 50 . Assuming that the information stored in the marker 100 is, for example, an animal, animal content may be output around the miniature 200 on the device 50 . That is, a virtual character is created around the miniature 200, which is a real space, so that a user who encounters content through augmented reality can check more lively content.

Meanwhile, when the contents 400 and the miniature 200 are output together, the contents 400 and the miniature 200 may be interfered with (refer to (d) in FIG. 2_refer to the virtual character output behind the miniature). At this time, the contents 400 are output to match the arrangement of the miniatures 200 .

To this end, a modeled miniature 2000 may be generated in order to output an image of the miniature 200 from the device 50 together with the content. Specifically, referring to FIG. 2 , modeled data identical to the appearance of the miniature 200 may be generated, and the generated modeled data may be previously stored in the device 50 or/and the server 300 . Thereafter, when the device 50 recognizes the miniature 200 as an image (image), modeled data information corresponding to the miniature 200 located at a predetermined location is stored in the memory of the device 50 or/and the server 300. can be output from That is, the user recognizes the miniature 200 as an image, but information output from the device 50 outputs a modeled miniature (2000_modeled data), not the miniature 200 itself.

In this case, the content 400 may be recognized as a background background among all information output to the device 50 . Illustratively, when the content 400 and the miniature 200 interfere, such as when the content 400 needs to be located behind a wall of the miniature 200, the modeled miniature 2000 takes precedence over the device 50. After being output, the content 400 is covered by the modeled miniature 2000 so that the user can see the content 400 as being located behind the wall of the miniature 200 .

Referring back to FIG. 2, the process of generating the modeled miniature 2000 specifically recognizes the miniature 200 that may be the background of the content 400 (see (a) in FIG. 2), 3D graphics that can correspond to the miniature 200 can be generated (refer to (b) of FIG. 2). That is, modeled data obtained by applying a transparent shader (wall) to the miniature 200 based on the 3D graphics of the miniature 200 may be generated.

Thereafter, the marker 100 may be placed around the miniature 200 or the miniature 200 itself, and the 3D graphically modeled miniature 2000 may be placed on the marker 100 (Fig. 3(c)). reference). In this case, the device 50 may extract the modeled miniature 2000 by extracting coordinates centered on the marker 100 .

That is, when the device 50 recognizes the marker 100 , the modeled miniature 2000 may be extracted from the device 50 . At this time, the modeled miniature 2000 extracted from the device 50 is a shape made of a transparent shader. That is, what is shown in the device 50 is the modeled miniature 2000, not the actual miniature 200. At this time, the device 50 moves the device 50 so that the extracted modeled miniature 2000 and the actual miniature 200 correspond so that the actual miniature 200 can be placed in the frame of the modeled miniature 2000. . Due to this, the user may derive the same result as viewing the miniature 200 through the device 50 .

Referring back to the drawing, the marker 100 may be disposed on either the front or side of the miniature 200 . For example, when the marker 100 is located in front of the miniature 200 and the device 50 recognizes the miniature 200 from the front of the miniature 200, the modeled miniature 2000 stored in the device 50 1: 1 correspondence between and the actual miniature 200 can be easily achieved.

Unlike this, when the user recognizes an image (video) of the miniature 200, the image (video) can be recognized from the side of the miniature 200 (left and right directions of the directional axis in FIG. 4). That is, the device 50 is recognized in terms of the modeled miniature 2000 . In this case, when the modeled miniature 2000 stored in the device 50 is a front side, miniature 200 information output from the device 50 may be information directed to the front side (front and rear directions of the direction axis of FIG. 4 ). . Therefore, the miniature displayed on the device 50 and the modeled miniature 2000 do not match. To this end, even when the image of the miniature 200 is recognized from the side, the modeled miniature 2000 matching information on the modeled miniature 2000 to the side can be output to the device 50 .

In addition, when recognizing the miniature 200 from the side, the contents 400 can be output based on the side direction of the modeled miniature 2000 . As described above, even if the sides of the miniature 200 are recognized, the modeled miniature 2000 may automatically output 3D data corresponding to the sides of the miniature 200 . Accordingly, an operation of moving the content 400 output around the miniature 200 along the side of the miniature 200 may also be performed. As a result, contents output from any direction of the miniature by the user can be output according to the shape, shape, etc. of the miniature according to stored conditions regardless of the direction of the miniature.

As described above, in an embodiment of the present invention, a plurality of markers 100 may be disposed around the miniature 200 . For example, the marker 100 will be described as an example in which the markers 100 are respectively disposed on the front, rear, and both sides of the miniature 200 . The plurality of markers 100 arranged around the miniature 200 include the same information about the output content. That is, even if only one of the plurality of markers 100 is recognized, the content included in the marker 100 may be output.

A detailed description in this regard will be described with reference to FIGS. 3 to 5 below.

Meanwhile, the device 50 may be movable. If the content 400 is attempted to be output, when the movement of the device 50 is detected, the content 400 may be moved along the moving direction of the device 50 to enable more realistic content output.

The server 300 may be a database providing information on the marker 100 and the miniature 200 . In addition, the server 300 may include a web server or application server that the user can control by voice or touch using an application installed in the device 50 or a web browser.

In addition, the server 300 may receive a command for recognizing the marker 100 and the miniature 200 for outputting the content 400 shown by the device 50, and the device 50 may perform the command in response to the received command. Actions can be saved. Here, the operation that the device 50 can perform may mean, for example, that the outputted content 400 is output to move around a miniature or that an environment corresponding to a virtual character is output together with the virtual character.

3 and 4 are diagrams illustrating an example in which a plurality of markers are arranged around a miniature according to an embodiment of the present invention, and FIG. 5 is output using augmented reality to each of a plurality of markers according to an embodiment of the present invention. FIG. 6 is a diagram showing an example of inputting content information to be used, and FIG. 6 is a diagram showing conditions for outputting content through a plurality of markers according to an embodiment of the present invention.

Referring to the drawing, a plurality of markers 100 may be arranged around a miniature 200 for implementing content through augmented reality according to an embodiment of the present invention.

As shown in FIG. 3, markers 100 may be placed on each of the front, rear, and side surfaces of the miniature 200 centered on the miniature 200. Alternatively, when the miniature 200 is spherical (round shape) as shown in FIG. 4 A plurality of markers 100 may be disposed along the circumference of the miniature 200 .

The same content information may be included in the plurality of markers 100 arranged in this way. The same content information may mean that, for example, the same content can be output even if one device recognizes the first marker 100A or the second marker 100B. Alternatively, when each different device recognizes a different marker, it may mean that content output to each device is the same.

To this end, the same content information should be input to each of the plurality of markers 100 . Specifically, referring to FIG. 5 , the first marker 100A may be placed at an arbitrary position (an embodiment of the present invention: in front of the miniature 200) centered on the miniature 200. The first marker 100A to be placed may store modeled data 200 for the miniature 200. Therefore, when the device 50 recognizes the first marker 100A as an image, the content 400 is output around the miniature 200. can

Similarly, the second marker 100B may be disposed at a location different from the first marker 100A centered on the miniature 200 (an embodiment of the present invention: one side of the miniature 200). Modeled data 200 for the miniature 200 identical to that of the first marker 100A may also be stored in the disposed second marker 100B. In this way, the third marker 100C and the fourth marker 100D including the modeled data 200 for the miniature 200 may be disposed on the other surface of the miniature 200 . In this way, the plurality of markers 100 are arranged along the periphery of the miniature 200 so that the contents 400 can be output around the miniature 200 even when the markers are recognized as images on any surface of the miniature 200 .

At this time, there is only one device 50 that recognizes the plurality of markers 100, and one device 50 can recognize the plurality of markers 100 almost simultaneously. For example, after the device 50 recognizes the first marker 100A, it may recognize a second marker 100B adjacent to the first marker 100A. In this case, a command for outputting content may be stored in the device 50 based on the first recognized marker. That is, since the first marker 100A is first recognized, content information stored in the second marker 100B recognized later is not output, thereby preventing overlapping content from being output.

Unlike this, the number of devices 50 recognizing the plurality of markers 100 may be plural. That is, different devices at different locations may recognize different markers 100 . At this time, since the same content information is included in the plurality of markers 100, the same content may be output to different devices. For example, even when the first device and the second device recognize the first marker 100A and the second marker 100B, the same content information is included in the first marker 100A and the second marker 100B. Therefore, the same content can be output to the first and second devices.

At this time, the first device recognizes the second marker 100B disposed on the side of the miniature 200, and the second device recognizes the fourth marker 100D disposed opposite the second marker 100B. In this case, it can be said that the first and second devices recognize the miniature 200 from different sides. In this case, even if the side of the miniature 200 is recognized, 3D data corresponding to the side of the modeled miniature 2000 is outputted from each other. Accordingly, contents output to each device may be output around the miniature 200 according to stored conditions regardless of the direction of the miniature.

Meanwhile, the content stored in the plurality of markers 100 may be a voice service explaining the miniatures 200 . Specifically, the content may be a plurality of events in which voice services for explaining miniatures are divided by time.

In general, since the start time of a voice service can be set in advance, when several devices recognize a plurality of markers 100 at different times, when an arbitrary device fails to meet the voice service start time and cannot use the voice service may occur. That is, if the voice service is provided only when several devices simultaneously recognize the marker in accordance with the voice service start time, the first device uses the voice service according to the start time, while the second device does not use the voice service. It may happen that you cannot.

To this end, as shown in FIG. 6, the content is a voice service and may include a plurality of events set based on time. For example, the content (voice service) start time is set in advance, and the first time at which the first device 50 recognizes the first marker 100A is not the content (voice service) start time, but a predetermined time from the start time. This may be past time. In this case, the first content output to the first device may be output when a predetermined time has elapsed. That is, it is possible to listen to a voice service at a position where time has elapsed without listening to a voice service that has been started from the beginning.

Based on these characteristics, referring to FIG. 6 , event A (voice service start), when the first device recognizes the first marker and the first device recognizes the first marker is the start time of directing at which content is output. It may be a starting point. Alternatively, the second device may recognize the second marker at a point in time when event B (midway of voice service) may start after a predetermined time. In this case, the second device may not output the content from the start, but may output the content for event B after a predetermined time has elapsed.

These events can be set to restart when the content output ends, so even if multiple devices output about different events (e.g., event C, event E, etc.) Since the content for is output, each device can use the entire content.

For example, as shown in FIG. 4, when an exhibit (miniature) is very large and requires explanation about the exhibit, even if the viewpoint of recognizing the surroundings of the miniature or the marker displayed on the miniature at each location is different, each device You will be able to check the description of the miniature as a whole.

7 is a block diagram showing the configuration of a device for implementing content through augmented reality according to an embodiment of the present invention, and FIG. 8 is a communication connection with a device for implementing content through augmented reality according to an embodiment of the present invention. It is a block diagram showing the configuration of the server.

Referring to the drawing, the device 50 includes a communication unit 52, a setting unit 53, a photographing unit 55, a modeling generation unit 56, a matching unit 57, a memory 58, and a processor 59. can be configured to include

The communication unit 52 of the device 50 is configured to receive related images from the server 300 when the marker 100 and the miniature 200 are recognized through the device 50 .

Also, the communication unit 52 may serve to receive a content display request signal from a user who manipulates the device 50 . Furthermore, the communication unit 52 may be a device including hardware and software necessary for transmitting and receiving signals such as control signals or data signals with other network devices through wired or wireless connections. In this embodiment, another electronic device may be a device capable of transmitting information about the recognized marker 100 and the miniature 200 to the device 50 .

The setting unit 53 is a component capable of setting modeled data 200 for the miniature 200 to a plurality of markers 100 disposed around the miniature 200 . In addition, the setting unit 53 is also a component capable of setting a playback time of a plurality of contents so that the first content is output at a first time after a predetermined time has elapsed from the time of outputting the contents.

The photographing unit 55 is a component for image recognition of the marker 100 and the miniature 200 . To this end, the photographing unit 55 may be a camera included in the device 50 . The photographing unit 55 may collect information about the marker 100 and the miniature 200 through image and video capture.

The analysis unit 54 may analyze information included in the captured marker 100 when the photographing unit 55 captures the marker 100 . When the marker 100 information is analyzed through the analyzer 54, the content 400 may be output together with the modeled miniature 2000 generated by the modeling generator 56 to be described later.

The modeling generator 56 is a component that models the miniature 200 based on information of the miniature 200 photographed by the photographing unit 55 or actual information of the miniature 200 pre-stored in the server 300 .

As described above, when the contents 400 and the miniature 200 are output together, the miniature 200 may be modeled in order to prevent the contents 400 and the miniature 200 from being mixed and output.

Specifically, the modeling generator 56 matches the image of the miniature 200 captured by the photographing unit 55 with the actual information of the miniature 200 previously stored in the server 300 to model the appearance of the miniature 200 and the corresponding one. A customized miniature (2000) can be created.

For example, the modeling generation unit may generate modeling using a fake depth occlusion technique in a low-end mobile or similar AR device without a depth sensor.

As described above, the modeling generating unit 56 allows the recognized external appearance of the miniature 200 to be recognized as a transparent shader. To this end, if a shader marker is set on the modeled miniature 200 obtained by converting the miniature 200 into 3D data, and the actual miniature 200 is placed based on the set shader marker, the modeled miniature (2000) and the actual miniature ( 200) may be arranged 1:1.

Thereafter, when the marker 100 is placed at the center, when the actual miniature 200 arranged 1:1 is placed on the modeled miniature 2000, augmented reality may be produced through the device 50. That is, the content 400 of the augmented reality is directed around the actual miniature 200, and there is an effect that the content can be covered by walls or obstacles of the miniature 200.

The matching unit 57 may match whether information on the modeled miniature 2000 generated by the modeling generator 56 and the miniature 200 photographed by the photographing unit 55 match. Also, the matching unit 57 may match whether information of the marker 100 photographed and analyzed by the photographing unit 55 matches image information of the photographed miniature 200 .

Specifically, the marker 100 includes animal information, and when the marker 100 is photographed, the animal is implemented in augmented reality, whereas the miniature 200 is the case of an arch, the information of the marker 100 and Image information of the miniature 200 may not match. For example, matching whether the information of the marker 100 analyzed by the analysis unit 54 is a content related to soldiers and battles, and whether the photographed miniature 200 is a miniature such as a tree or a forest where an animal is to be output. It can be confirmed through part 57. This information can be checked by the matching unit 57, and if the content information included in the marker 100 and the information of the miniature 200 photographed by the photographing unit 55 do not match, the user is informed through the device 50. can inform

In other words, if the contents information included in the marker 100 and the information of the miniature 200 photographed by the photographing unit 55 do not match, the user is notified that the information of the marker 100 or the miniature 200 is incorrect, The marker 100 or the miniature 200 to be recognized can be checked or re-recognized so that the user can use more accurate content.

The memory 58 may store information of the marker 100 and information of the modeled miniature 2000 . Specifically, the memory 58 outputs a modeled miniature 2000 when the marker 100 and the miniature 200 are photographed by the photographing unit 55, and the content 400 represented by the marker 100 is modeled Information for enabling the miniature 2000 to be output to the device 50 may be included.

That is, when the memory 58 determines that the information of the marker 100 analyzed by the matching unit 57 and the miniature 200 are related, the content 400 indicated by the marker 100 is output to the device 50. Instructions to do so may be stored in memory 58 .

Furthermore, the memory 58 is a configuration for recording various information necessary for the operation of the server 300 in addition to commands for implementing content through augmented reality in the device 50, and may include a volatile or non-volatile recording medium. . The recording medium stores data that can be read by the processor 59, and includes a hard disk drive (HDD), solid state disk (SSD), silicon disk drive (SDD), ROM, RAM, CD-ROM, magnetic tape, It may include a floppy disk, an optical data storage device, and the like.

The processor 59 controls a command signal capable of executing an operation for implementing content in the device 50 according to the information of the analyzed marker 100 and the relationship between the marker 100 and the miniature 200. can This processor 59, as a kind of central processing unit, can provide various functions for selecting content displayed by the device 50 or controlling the movement of content by driving control software loaded in the memory 58. .

Here, the processor 59 (processor) may include all types of devices capable of processing data. Here, a 'processor' may refer to a data processing device embedded in hardware having a physically structured circuit to perform functions expressed by codes or instructions included in a program, for example. As an example of such a data processing device built into hardware, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated (ASIC) circuit), field programmable gate array (FPGA), etc., but the scope of the present invention is not limited thereto.

Referring back to the drawings, the server 300 according to an embodiment of the present invention provides information such as markers 100 and miniatures 200 for content implementation using augmented reality to the device 50 so that the device 50 ) may be a configuration for storing information capable of implementing augmented reality.

The server 300 may perform the same function as the processor 59 of the device 50 as a separately configured device outside the device 50 or a configuration mounted inside the device 50 .

In addition, the server 300 may communicate with at least one device 50 and may derive a result by analyzing data on behalf of the device 50 (or also). Here, the meaning of helping other devices may mean distribution of computing power through distributed processing.

Moreover, the server 300 may be implemented as a single server, a plurality of server sets, a cloud server, or a combination thereof. That is, a plurality of servers 300 may be configured to configure a learning device set (or cloud server).

To this end, the server 300 may include a communication unit 320, a receiver 330, a processing unit 340, a memory 350, a processor 360, and the like.

The communication unit 320 may correspond to a configuration encompassing the communication unit 52 of the device 50 . That is, data can be transmitted and received with other devices through wired/wireless communication or an interface.

Memory 350 and processor 360 may correspond with memory 58 and processor 59 of device 50 .

The receiving unit 330 may receive information on the marker 100 and the miniature 200 through the device 50, and the processing unit 340 may implement the received information on the marker 100 and the miniature 200 as augmented reality. commands can be created.

In this way, when the receiver 330 receives information on the marker 100 and the miniature 200 through the device 50, content that can be implemented as augmented reality is generated, and information about the generated content is transmitted to the device 50. can be sent to The information on the content transmitted to the device 50 may be a virtual character implemented while moving around the miniature 200 , or may be an image associated with the miniature 200 .

On the other hand, in an embodiment of the present invention, the server 300 will be described as an example of a configuration included in the device 50, and otherwise, the device 50 and the server 300 may be separate configurations am.

9 is a flowchart illustrating a process of implementing content through augmented reality according to an embodiment of the present invention.

Referring to the drawing, first, modeled data identical to the appearance of the miniature 200 may be generated (S110).

The generated modeled data may be previously stored in the device 50 or/and the server 300 . Thereafter, when the device 50 recognizes the miniature 200 as an image, modeled data information corresponding to the image-recognized miniature 200 may be output from the memory of the device 50 or/and the server 300 .

Specifically, a miniature 200 that can be a background of augmented reality can be created, and 3D graphics that can correspond to the created miniature 200 can be created.

Thereafter, the marker 100 may be placed around the miniature 200 and the 3D graphic modeled miniature 2000 may be placed on the marker 100 (S120). In this case, the device 50 may extract the modeled miniature 2000 by extracting coordinates centered on the marker 100 .

In the process, the miniature 200 and the modeled miniature 2000 may be matched (S130). That is, since what is shown in the device 50 is not the actual miniature 200 but the modeled miniature 2000, the device 50 should correspond between the extracted modeled miniature 2000 and the actual miniature 200.

At this time, a plurality of markers 100 may be disposed around the miniature 200 . For example, the markers 100 may be disposed on the front, rear, and both sides of the miniature 200, respectively. The plurality of markers 100 arranged include the same information about content. That is, even if any one marker among the plurality of markers 100 is recognized, the content included in the marker 100 may be output. The same content information may mean that, for example, the same content can be output even if one device recognizes the first marker 100A or the second marker 100B. Alternatively, when each different device recognizes a different marker, it may mean that content output to each device is the same. To this end, the same content information may be input to each of the plurality of markers 100 .

In this way, when the miniature 200 and the modeled miniature 2000 are matched, and any one of the plurality of markers 100 is recognized, the content 400 stored in the marker 100 can be output (S140, S150).

At this time, there is only one device 50 that recognizes the plurality of markers 100, and one device 50 can recognize the plurality of markers 100 almost simultaneously. For example, after the device 50 recognizes the first marker 100A, it may recognize a second marker 100B adjacent to the first marker 100A. In this case, a command for outputting content may be stored in the device 50 based on the first recognized marker. That is, since the first marker 100A is first recognized, content information stored in the second marker 100B recognized later is not output, thereby preventing overlapping content from being output.

Unlike this, the number of devices 50 recognizing the plurality of markers 100 may be plural. That is, different devices at different locations may recognize different markers 100 . At this time, since the same content information is included in the plurality of markers 100, the same content may be output to different devices. For example, even when the first device and the second device recognize the first marker 100A and the second marker 100B, the same content information is included in the first marker 100A and the second marker 100B. Therefore, the same content can be output to the first and second devices.

At this time, the first device recognizes the second marker 100B disposed on the side of the miniature 200, and the second device recognizes the fourth marker 100D disposed opposite the second marker 100B. In this case, it can be said that the first and second devices recognize the miniature 200 from different sides. In this case, even if the side of the miniature 200 is recognized, 3D data corresponding to the side of the modeled miniature 2000 is outputted from each other. Accordingly, contents output to each device may be output around the miniature 200 according to stored conditions regardless of the direction of the miniature.

On the other hand, as described above, what is shown on the device 50 is the modeled miniature 2000, not the actual miniature 200, and since the extracted modeled miniature 2000 and the actual miniature 200 are matched, the user can use the device It is like viewing miniature (200) through (50).

Even if the content 400 is output behind one side of the miniature 200, the miniature 200 may be output in advance so that the content 400 may be output as it is on the back side of the miniature 200.

In this way, an example may implement augmented reality using a device, and content may be realized in the implemented augmented reality. In this case, the content may be implemented at a location adjacent to a miniature, which is an actual object used as a background screen of the content (eg, around the miniature, inside the miniature, etc.). Here you can place markers around the miniature or on the miniature itself to bring content to life. When the placed marker is recognized through the device, content may be output around the miniature based on content information stored in the marker. As a result, content implemented through augmented reality can be implemented more realistically without being separated from surrounding objects.

Also, the content can be output without interfering with the miniatures. Specifically, modeled data of a miniature may be generated in order to output a video of a miniature along with content from a device. It may be pre-stored in a device and a server, and when the device recognizes a miniature image (image) matching the stored modeling data, modeling data corresponding to the miniature may be output. At this time, the content output through the marker is recognized as the background behind the modeling data, for example, when the content and the miniature interfere, such as when the content should be output behind a miniature wall, either the content or/and the miniature is independently displayed. Make it output so that the content can be output naturally.

Meanwhile, a plurality of markers including content information may be arranged around the miniature. In this case, when a first marker and a second marker adjacent to the first marker are recognized among a plurality of markers through one device, the content can be output based on the previously recognized first marker. That is, even if a plurality of markers are recognized at different points in time, content can be output without overlapping.

Alternatively, a plurality of markers may be recognized by different devices. That is, when the first device recognizes a first marker among the markers and the second device recognizes a second marker, the first device and the second device can view the same content even at different viewpoints. That is, the same content information may be included in each of a plurality of markers, and even if different devices recognize markers at different positions at different times, the same content may be implemented in each device.

In this case, when each device recognizes the marker at different points of time, the start point of outputting the content may be set to be the same. That is, even if the first time point at which the first device recognizes the first marker is earlier than the second time point at which the second device recognizes the second marker, the second device sets the second device to the second device regardless of the time point at which content is output to the first device. By outputting the content based on the point in time at which the marker is recognized, a plurality of users can use the same content even if the content is implemented through different devices.

The description of the embodiments of the present invention described above is for illustrative purposes, and those skilled in the art can easily modify them into other specific forms without changing the technical spirit or essential features of the present invention. you will understand that Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof should be construed as being included in the scope of the present invention.

Claims (13)

Generating modeled data for a miniature structure that becomes a content background implemented in augmented reality through a device;
recognizing an image of the miniature and at least one marker disposed around the miniature and including the content information;
Recognizing a photographing direction in which the device photographs the miniature based on a preset front of the miniature;
matching the modeled data for the miniature structure in a direction parallel to the direction corresponding to the photographing direction; and
When the modeled data and the miniature structure match, outputting the content through the device,
How content is implemented.
According to claim 1,
The step of recognizing the image is,
recognizing a first marker of the at least one marker; and
Recognizing a second marker at a position different from the first marker at a second time point later than the first time point based on the first time point at which the first marker was recognized;
The step of outputting the content is,
Outputting the content based on the first viewpoint,
How content is implemented.
According to claim 1,
The step of recognizing the image is,
Recognizing a first marker of the at least one marker through a first device adjacent to a first marker;
Recognizing a second marker at a different location from the first marker at a different time point than the first point of view through a second device,
The step of outputting the content is,
The first device and the second device output the same content at different times,
How content is implemented.
According to claim 3,
The content is plural,
Setting a first content output time point at a first time after a predetermined time has elapsed from the output time point of outputting the content;
The step of outputting the content is,
When the first time point is the first content output time point, the first device reproduces the first content when recognizing the first marker.
How content is implemented.
According to claim 1,
The generating step is
Generating the modeled data to correspond to the miniature appearance,
How content is implemented.
According to claim 1,
The matching step is
Including guiding a miniature image re-recognition notification when it is determined that the modeled data and the miniature structure information do not match.
How content is implemented.
According to claim 1,
The outputting step is
The content is output when the miniature and the marker are simultaneously recognized.
How content is implemented.
According to claim 1,
Further comprising the step of detecting the movement of the device,
The outputting step is
When the movement of the device is detected, moving the content along the movement direction of the device,
How content is implemented.
processor; and
A memory electrically connected to the processor and storing at least one code executed by the processor;
When the memory is executed by the processor, the processor
After creating modeled data for a miniature structure that is a background of content implemented in augmented reality through a device, and disposed around the miniature and the miniature, at least one marker including the content information is image-recognized, After recognizing a photographing direction in which the device photographs the miniature based on the front of the miniature that is set in advance, the modeled data for the miniature structure in a direction parallel to the photographing direction and the miniature structure If matched, storing codes that cause the content to be output through the device,
Content Implementation Device.
According to claim 9,
When the memory is executed through the processor, the processor,
A first marker among the at least one marker is recognized, and a second marker at a position different from the first marker is recognized at a second time point later than the first time point based on the first time point at which the first marker is recognized. In the case of doing so, storing codes that cause the content to be output based on the first point of view,
Content Implementation Device.
According to claim 9,
When the memory is executed through the processor, the processor,
The first marker is recognized through a first device adjacent to the first marker among the at least one marker, and a second marker at a position different from the first marker is provided through a second device, a viewpoint different from the first viewpoint. In case of recognizing, the first device and the second device store codes that cause the same content to be output at different times,
Content Implementation Device.
According to claim 11,
When the memory is executed through the processor, the processor,
The content is plural, and a first content output time point is set at a first time after a predetermined time has elapsed from the output time point of outputting the content. When the first time point is the first content output time point, the first device storing codes that cause the first content to be reproduced when the first marker is recognized;
Content Implementation Device.
According to claim 9,
The content is a plurality of events divided by time for the miniature,
Content Implementation Device.

Images