KR102218095B1 - Augmented Reality Implementation Method Using Person and Plane Recognition, and Medium Storing Instructions to Execute the method Thereof - Google Patents

Abstract

The present invention relates to an augmented reality implementation method and a device therefor. More specifically, the present invention relates to the method for implementing augmented reality and the computer-readable storage medium, wherein when a user wants to photograph an arbitrary object, preferably a person, with a device equipped with a photographing means, a plane among the space in which the target person is located is recognized, and, by having the virtual character appear on the plane and pose, allows the target person and the virtual character appear as if they were standing at the same place.

Description

Augmented Reality Implementation Method Using Person and Plane Recognition, and Medium Storing Instructions to Execute the method Thereof}

The present invention relates to a method of implementing an augmented reality and an apparatus therefor, and more specifically, a device equipped with a photographing means to recognize an arbitrary object, preferably a person, among the spaces in which the target person is located, , An augmented reality implementation method and a computer-readable storage medium in which a target person and a virtual character appear as if they are standing on the same plane by making a pose after appearing on a corresponding plane.

Portable devices equipped with photographing means are now very easy to access in everyday life, and in particular, taking pictures and taking videos using a smartphone with the explosive increase in the use of smartphones can be seen very easily.

In recent years, moving away from the level of simply photographing the target object included in the camera frame, especially people, by applying various effects within the frame, users can take pictures or videos more interestingly, or create photos and videos of more diverse contents. Many ideas have been proposed to make this possible.

As one of these ideas, conventionally, a service method has been proposed in which a picture can be taken while the target person and the virtual character exist in one space by further displaying a virtual character around the person to be photographed. . However, in the conventional method, since the virtual character could be displayed only at a predetermined position within the frame of the camera, there have been inconvenient points such as the target person moving directly or the person taking the picture must move to change the shooting angle.

An object of the present invention is to enable the virtual character to appear and position naturally in connection with a target object, preferably a spatial position with a target person, when a photo shoot accompanied by a virtual character is to be taken.

The present invention was derived by focusing on such a problem, and invented to provide additional technical elements that can not be easily invented by those of ordinary skill in the art, as well as being able to solve the technical problems of salpin above. Became.

Korean Patent Registration No. 10-0916706 (announced on September 11, 2009)

An object of the present invention is to identify a space in which the target person is located from an image so that the virtual character can be properly positioned around the target person to be photographed.

In addition, an object of the present invention is to render a virtual character at a location where a target person is photographed without a separate manipulation.

In addition, an object of the present invention is to provide an effect such that a virtual character actually appears and moves in the identified space, so that a user can feel more enjoyable or feel immersive like a real situation.

In addition, an object of the present invention is to enable a plurality of users to share in various forms by converting a result after shooting into an output in various formats.

Meanwhile, the technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, a method for implementing an augmented representation service according to the present invention includes: obtaining an image including a target person; Receiving a user input for selecting a virtual object; Identifying a plane in which the target person is located in the image; Loading a virtual object selected by the user input; Displaying the loaded virtual object on a plane identified in the image; And generating image data or image data from an image including the virtual object and the target person.

In addition, in the method of implementing the augmented reality service, the step of identifying a plane in which the target person is located may include identifying an object with the largest plane area among the objects located at the bottom of the image as a target plane. It can be characterized.

In addition, the method of implementing the augmented reality service may further include identifying the target person in the image before the step of identifying a target plane in which the target person is located in the image.

In this case, the step of identifying the target person may include identifying a lower body region of the target person.

In addition, the step of identifying the target plane on which the target person is located may include identifying an object having the largest plane area among the objects in contact with the lower body area of the identified target person as the target plane. I can.

In addition, in the method of implementing the augmented reality service, displaying the virtual object on the target plane may include displaying a predefined animation for the virtual object. In this case, before the step of displaying the virtual object on the target plane, determining a target position at which the virtual object will be located on the target plane; may further include. In addition, the determining of a target position in which the virtual object will be located may include an interval between the target person in the image, a size of an area overlapping the target person when the virtual object is located, or the virtual object It may be characterized in that it is determined based on at least one of a ratio of an area overlapping with the target person compared to the target person area.

In addition, in the method of implementing the augmented reality service, the step of identifying the target person may include identifying a lower body region, an upper body region, and a face region of the target person. In this case, displaying the virtual object on the target plane comprises displaying the virtual object in a predefined pose corresponding to a lower body region, an upper body region, or a face region of the identified target person. can do.

Meanwhile, a computer-readable storage medium according to another embodiment of the present invention stores instructions for executing a method of implementing an augmented reality service, but the method of implementing the augmented reality service includes an image including a target person. Obtaining a; Identifying a target plane in which the target person is located in the image; Receiving a user input for selecting a virtual object; Loading a virtual object selected by the user input; Displaying the loaded virtual object on the target plane; And generating image data or image data from an image including the virtual object and a target person.

According to the present invention, a space in which a target person is located can be identified from an image, and a virtual character can be used to create a natural movement in the space, thereby displaying a realistic virtual character to the user. There is.

In addition, according to the present invention, by allowing the virtual character to be rendered at an appropriate shooting position around the target person in the previously identified space, there is an effect of enabling easy shooting even if the photographer or the target person does not specifically manipulate or move.

The effects of the present invention are not limited to the above-mentioned effects, and other effects that are not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a simplified diagram illustrating a process in which a target person is photographed together with a virtual character by a method of implementing an augmented reality service according to the present invention.
2 is a sequence diagram illustrating a method of implementing an augmented reality service according to the present invention.
3 and 4 are diagrams for explaining a process of identifying a plane on which a target person is located.
5 is a diagram illustrating a virtually generated line in which a target person is located when it is difficult to identify a plane.
6 is a sequence diagram illustrating a virtual character displayed in a frame.
7 and 8 illustrate locations where virtual characters are displayed.
9 shows an embodiment for improving image quality when a target person and a virtual character are photographed (captured).
10 is for explaining a process of converting a target person and a virtual character into data of various formats after being photographed.

Details of the object and the technical configuration of the present invention and the effects of the operation thereof will be more clearly understood by the following detailed description based on the accompanying drawings in the specification of the present invention. An embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

The embodiments disclosed herein should not be construed or used as limiting the scope of the present invention. It is natural to those skilled in the art that the description including the embodiments of the present specification has various applications. Accordingly, any of the embodiments described in the detailed description of the present invention are illustrative for better describing the present invention, and it is not intended that the scope of the present invention be limited to the embodiments.

The functional blocks shown in the drawings and described below are only examples of possible implementations. In other implementations, other functional blocks may be used without departing from the spirit and scope of the detailed description. Further, while one or more functional blocks of the present invention are represented as individual blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.

In addition, the expression that includes certain elements is an expression of "open type" and simply refers to the existence of the corresponding elements, and should not be understood as excluding additional elements.

Furthermore, when a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to the other component, but other components may exist in the middle. do.

FIG. 1 is a brief illustration of a process of using an augmented reality service according to the present invention, and FIG. 1 (a) shows a state in which a target person is photographed using a photographing device 100, and FIG. 1 (b) ) Shows a virtual character next to the target person, showing that the target person and the virtual character actually shoulder to shoulder in the same space and take a pose to take a picture. For reference, in this detailed description, in particular, a virtual character among virtual objects that can be displayed next to a target person will be described as a representative example. However, those displayed next to the target person include not only virtual characters but also other virtual objects. It will be appreciated that objects (stars, cars, plants, etc.) can all be included.

Specifically, (a) of FIG. 1 shows a state in which the target person 10 is included in the frame F of the photographing apparatus 100, and in this case, the frame F is the photographing apparatus 100 ) Can be understood as referring to a screen displayed on the display. In addition, in (b) of FIG. 1, the virtual character 20 is located next to the target person 10, in particular, the virtual character 20 is positioned so that it appears to be standing on the plane where the target person 10 is located. It shows a state in which a picture is taken or a video is taken in a state in which the target person 10 and the virtual character 20 are realized as if they are together in the same space.

As shown in FIG. 1, the present invention relates to a target object, preferably, a position where a target person stands when a target person is to be photographed, a plane 200 on which the target person stands, a head, an upper body, and a lower body of the target person. The object of the present invention is to allow a user who uses the service to experience a new experience by appearing or displaying a virtual character in the frame F of the photographing apparatus 100 after recognizing the like.

Before describing the present invention in earnest, a brief description will be given of the photographing apparatus 100 referred to in this detailed description. The photographing device 100 is understood to include all devices capable of photographing a target object as a still image (photo) or a video, and the photographing device 100 includes a central processing unit (CPU) and a memory for calculation. Note that you understand that you are doing. The central processing unit may also be called a controller, a microcontroller, a microprocessor, a microcomputer, or the like. In addition, the central processing unit may be implemented by hardware, firmware, software, or a combination thereof. When implemented using hardware, an application specific integrated circuit (ASIC) or a digital signal processor (DSP) , DSPD (digital signal processing device), PLD (programmable logic device), FPGA (field programmable gate array), etc., when implemented using firmware or software, modules, procedures, or functions that perform the above functions or operations Firmware or software may be configured to include. In addition, the memory is ROM (Read Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, SRAM (Static RAM), It may be implemented as a hard disk drive (HDD) or a solid state drive (SSD). In addition, the photographing apparatus 100 may further include a communication module capable of connecting to an external terminal or an external server through a network in addition to a central processing unit and a memory. As an easy example, the photographing device 100 in this detailed description may be a portable device such as a smart phone, a tablet PC, and a laptop computer.

For reference, in the following detailed description, a smartphone or a tablet PC will be described as a representative example of the photographing device 100 in order to aid understanding of the invention.

2 is a sequence diagram illustrating a method of implementing an augmented reality according to the present invention.

Referring to FIG. 2, the first step in the augmented reality implementation method is a step (S100) of obtaining an image including a target person. The target person means a person to be photographed, and in this case, the target person may be one or more.

For reference, although the term target person is used in this detailed description, it is understood that the subject of photographing is not necessarily limited to a person, and the term target object may be used instead in a comprehensive sense. For example, a target object that can be photographed by the photographing apparatus 100 may include animals such as cats and dogs, and living things such as plants, in addition to people, and may also include inanimate objects such as food, dolls, and furniture.

Meanwhile, the meaning of acquiring the image including the target person in step S100 means that the frame among the images displayed through the display of the photographing apparatus 100 in the process of the photographer composes the target person to be included in the frame F. It may mean that you can acquire some or all of (F). In other words, among the series of steps to be described later, steps to identify the floor surface on which the target person is located by analyzing the acquired image or to identify the face/upper/lower body of the target person are also included.Step S100 It can be understood as a step of acquiring an image for the same image analysis. The actual implementation of step S100 may be made in a manner in which a part or all of the frame F among the real-time images captured by the photographing device 100 is temporarily or permanently stored in a storage means such as a memory, and the photographing device From the perspective of the user operating 100, the step S100 may be perceived as a screen displayed in real time while composing a composition for photographing a target person.

After step S100, a selection input for which virtual character among the plurality of virtual characters to appear next to the target person may be received from the user. (S101) In step S101, a selection menu screen may be provided so that a user (photographer) can directly select a desired virtual character from several text lists or icon lists through the photographing device 100, and the photographing device 100 May receive a selection input from a user. In this case, there is no limit to the method of selection input, such as touch input, input through button manipulation, and motion input of a smartphone. On the other hand, in this detailed description, it is described that step S101 is performed immediately after step S100, but the step of selecting a virtual character does not have to exist immediately after step S100, and before the step of loading a virtual character (S104) to be described later. It should be understood that any point in time can exist at any point in time.

After receiving the selection input for the virtual character from the user, the step (S102) of identifying the target person included in the frame (F) may be performed. Step S102 may include the steps of discriminating and identifying the face, upper body, and lower body of the target person, and the step of identifying the face of the target person includes, for example, the basic shape of the face and components that can be included in the face (for example, : If the eyes, nose, mouth, etc.) are determined by image analysis, the corresponding area can be implemented as an algorithm that identifies the face, and the upper body and lower body also satisfy a series of conditions according to a predefined algorithm. It can be identified by the upper body or lower body of the target person.

In another way, the photographing device 100, for example, a smartphone is owned by a specific user, and the number of persons to be photographed by the smartphone may be limited to a few. Face information, upper body information, or lower body information about a person is pre-stored and then compared with objects in the current frame F, an object with high similarity can be identified as a specific person. For example, a limited number of people or targets, such as A herself, A's family, A's dog, A's friend B, etc., may be the main target for shooting by a smartphone owned by User A. Information such as face, upper body, lower body shape), color (color of clothes, color of dog, etc.) may be stored in the storage medium (memory) of the smartphone in advance, and objects are included by the smartphone. When an image is acquired, which of the objects is a target person or a target object can be identified by comparing it with the stored information.

On the other hand, in step S102, in particular, the process of identifying the lower body of the target person may be essentially included, because it is necessary to easily find the plane where the target person is located, that is, the floor surface. A methodology for identifying the plane on which the target person is located will be described later. For reference, there may be various algorithms used to identify the lower body of a target person. For example, learning images of people standing or sitting, especially learning the shape of people's lower body, and using the algorithm obtained as a result. Accordingly, an algorithm for extracting a region having a shape similar to a lower body of a person among regions in which the target person is displayed in the frame F of the current photographing apparatus 100 may be used.

Next, step S103 is a step of identifying a plane on which the target person is located. In order to display the virtual character naturally, determine which side the target person is supporting, such as which side of the frame (F) the target person is standing on, or on which side the target person is lying in contact with the body. The identification process is indispensable, but this step can be understood as a step of identifying the plane where the target person is located. In this detailed description, the plane in which the target person is located will be referred to as a'target plane' for convenience.

The step of identifying the target plane may be implemented by various methodologies. Hereinafter, several embodiments will be described.

First of all, one of the frames F of the image acquired in step S100 above may be considered an embodiment in which a target plane is identified with the largest surface area among arbitrary objects located at the bottom. For reference, in the case of this first embodiment, it may be performed without step S102, which is the step of identifying the target person. Specifically, in the first embodiment above, after simply dividing the frame (F) of the acquired image into top and bottom, and after identifying the outlines of the objects existing at the bottom, the width of the inner surface of the outline has the largest value. Can be identified by the target plane. For example, assuming that an arbitrary frame F is composed of 1280 X 800 pixels, an area of 1280 X 400 to the bottom of the frame F may be defined as the lower area of the image. An object recognized as having the largest area among several objects included in the lower area can be identified as a target plane.

3 shows an arbitrary frame (F) of an image including a target person, and it can be seen that a plurality of objects 200 are included in the frame (F) in addition to the target person. Among these, in the first embodiment of identifying the target plane, it is not necessary to distinguish and identify the target person's face (300A), upper body (300B), and lower body (300C), but the target person is also within the frame (F). It can only be treated as an existing object. For reference, in the method of identifying each object in the frame (F), if the difference between the values between adjacent pixels exceeds the reference value, the algorithm that defines it as a boundary is repeated to extract the outlines of arbitrary objects, thereby extracting each object. A method of identifying may be used, but is not limited thereto.

It has been described above that the target plane identification according to the first embodiment can identify the object recognized as having the largest area among the objects existing at the bottom of the image as the target plane. Referring to FIG. 3, the lower part of the image ( It can be identified that objects such as 200A, 300B, and 300C exist in an area below the middle of the frame in the vertical direction), of which 200A having the widest area can be identified as a target plane. At this time, the upper and lower divisions of the image include a region including from the lowest pixel in the vertical direction of the frame F to the pixel located at the middle point of the frame F as described above, and the frame F The remaining upper part of) can be divided into the upper part. As described above, in the case of the first embodiment, the outline of each object is extracted from the objects existing at the bottom of the frame (F), the area within each outline is calculated, and the object with the largest value is identified as the target plane. In this embodiment, a person can be performed even if the target person is not necessarily identified. For reference, the meaning that there is no need to identify the target person in the first embodiment means that it is not necessary to identify the target person as a'person', and of course the process of identifying the target person as a single'object' Understand the need.

The object plane identification method according to the second embodiment is based on the assumption that the lower body region of the target person is first identified, and any one of the objects in contact with the lower body region of the target person or objects in contact with the lower body region is a target plane. It is a way to identify as. When describing the previous step S102, it was explained that the face, upper body, and lower body of the target person included in the frame (F) can be identified, and in particular, the process of identifying the lower body can be essentially included.In the second embodiment, In the method of identifying the target plane according to the method, the lower body region of the target person identified above is used.

4, the lower body area 300C of the target person identified in the previous step by the photographing apparatus 100 is indicated by a thick line, and the object area 200A in contact with the upper bold line is indicated by hatching. It can be confirmed that there is. That is, in the second embodiment, an object in contact with the lower body area of the target person identified by line may be preferentially identified as the target plane.

Meanwhile, in the case of an image other than the embodiment shown in FIG. 4, there may be a case where only one object region does not contact the lower body region. For example, in the case of an image acquired outdoors, other objects such as rocks, trees, and ponds placed around the target person may be included. Accordingly, in the frame F, a plurality of objects may correspond to the lower body area of the target person. You can be in touch at the same time. In this case, the method of identifying the target plane includes (i) a method of identifying the object having the widest area among the plurality of objects as the target plane, and (ii) the object having the longest contact with the outline of the lower body area. There may be a method of identifying the target plane as the target plane, or (iii) a method of identifying an object in contact with the lowermost end of the target person's lower body region as the target plane. (iii) Regarding the method, in FIG. 4, the lowermost end of the lower body region is indicated by the reference numeral 300D (the lowermost end may be plural), and at this time, the object in contact with any one of the lowermost ends or all of the lowermost ends are in contact. An object that exists can be identified as a target plane.

On the other hand, during the process of performing step S103, there may be a case in which the target plane cannot be identified. In this case, as shown in FIG. 5, it is in contact with the lowermost end of the lower body area of the target person, but the base of the frame F and the By arbitrarily defining a parallel reference line 210A, the reference line may be recognized as one target plane. For example, when it is difficult to determine which is the target plane because the area of the lower body area and the area of the objects that are in contact with each other are not distinguishable from each other, and in other image analysis, it is difficult to extract a boundary line to identify objects. In many cases, such as difficult cases, it may not be possible to identify the target plane.In this case, instead of discriminating the target plane from within the frame (F), an arbitrary reference line is drawn within the frame (F). You can define a new one and display a virtual character above this baseline.

Meanwhile, although FIG. 5 illustrates an embodiment in which the reference line 210A is arbitrarily defined within the frame F, it is understood that the reference line is not necessarily defined only within the frame F. For example, the image photographing the target person may include only the face and the upper body without including the lower body of the target person. Even in this case, the photographing apparatus 100 may perform an operation outside the frame F. Also, a virtual reference line can be arbitrarily defined. In order to define a virtual reference line, information on the size of the face area, the size of the upper body area, and the ratio of the general person included in the frame (F) may be used, and the photographing apparatus 100 may use the size of the face area. Alternatively, the location of the virtual lower body region may be estimated by matching the size of the upper body region to the ratio of the general person, and accordingly, a reference line in contact with the lower body of the virtual lower body region may be defined.

Meanwhile, in addition to the above-described methods, there may also be a method for identifying a target plane by extracting one or more feature points from an image acquired by the photographing device 100 and tracking the extracted feature points within a continuous image. have. Specifically, it has been described in step S100 that an arbitrary image can be obtained while the user is using the photographing device 100 to compose a photographing composition for a target person. In this case, the photographing device 100 At least one or more feature points may be extracted or defined from an image, and a distance to which the feature points move from consecutive images may be calculated. Meanwhile, the photographing apparatus 100 may itself include an inertial measurement unit, and by using the inertial sensor, the movement distance, rotational accuracy, and posture of the photographing apparatus 100 Angle, etc.), and the target plane can be recognized by analyzing the relationship between the frames in the continuous image using the information obtained by the inertial sensor and the movement distance of the feature point calculated previously. Of course, you can calculate the shape, size, and distance of the target plane.

On the other hand, it is understood that there may be a wider variety of methods for recognizing the target plane in addition to the previously introduced methods.

We looked at step S103, which is the step of identifying the abnormal target plane.

Referring to FIG. 2 again, after step S103, a step of loading a virtual character (S104) may follow. In this step, in order to implement the virtual character selected by the user in step S101 in the frame F, a virtual character previously stored in the photographing device 100 or a virtual character stored in a server connected to the photographing device 100 by a network This is the step of loading. Data for a plurality of virtual characters, for example, image data for a virtual character, animation data for a virtual character, etc. may be stored in advance in a storage medium (memory) of the photographing device 100, such as The data may be downloaded from a server connected to an external network. Alternatively, the data on the virtual characters may be stored only in the server, and whenever there is a request from the photographing device 100, that is, when there is a user's virtual character selection input, It may be downloaded to the photographing device 100 and temporarily stored.

After step S104, a step (S105) of displaying the selected virtual character on the previously identified target plane may be performed. 6 shows an embodiment in which the virtual character walks from the left side of the frame F to a predetermined position next to the target person in order. In step S105, as shown in FIG. 6, various animation data for displaying the virtual character in the frame F may be used, and even if it is not animation data, image data and image data are displayed so that the virtual character is displayed at a predetermined position. Various effects may be utilized.

Meanwhile, in step S105, an operation may be performed as to which position in the frame F to display the virtual character, that is, at which position to display the virtual character next to the target person. In the previous step, if there was a process of identifying the target plane, which is the space where the virtual character can be located within the frame (F), in this step, the location information on exactly where the virtual character is to be displayed among the target planes above is identified. will be. In this detailed description, the location where the virtual character is displayed will be expressed in terms of'target location 400' for convenience. 7 shows an embodiment in which the target position 400 is arbitrarily displayed on the side of the target person.

In the operation of determining the target location 400, for example, (i) the distance between the target person, (ii) the size of the area overlapping the target person when the virtual object is located, or (iii) the virtual object At least one criterion of a ratio of an area overlapping the target person to the target person area may be considered.

As an example of (i), the center point of the virtual character (assuming the existence of a randomly defined center point) is based on the center point of the previously identified target character (this also assumes the existence of a randomly defined center point). The distance may be determined so as to be included within a set range, or the distance may be determined so that the side portion of the virtual character shape is spaced apart by a preset range from the side portion of the target person.

Examples of (ii) and (iii) will be described with reference to FIG. 8. In FIG. 8, an overlapped area 410 between a target person and a virtual character is shown, and the photographing device 100 may determine the target location 400 so that the area of the overlapping area 410 does not exceed a preset value. Alternatively, the target location 400 may be determined so that the area ratio of the overlapping area 410 does not exceed a preset value when compared with the area area of the target person. The examples in (ii) and (iii) are all intended to prevent the problem of obscuring the target person by overlapping the virtual character too much with the target person.

Meanwhile, in step S105, not only the target position 400 in which the virtual character will be located is determined, but also a pose of the virtual character may be determined. Implementation of the pose of the virtual character may include, for example, a method in which a virtual character image of a preset pose is randomly displayed in the frame (F), or a method of displaying the pose of the virtual character in the same manner as the pose of the target person identified previously. Can be implemented.

After step S105, a step S106 of generating new image data or new image data while the virtual character is displayed in the frame F is performed. In simple terms, step S106 is a step in which an actual photographing is performed by the photographing device 100 or a step of capturing a state in which the virtual character is displayed in the frame F. In this step, image data or image data that makes the virtual character and the target person appear to exist in one space are finally generated.

Meanwhile, when the size of the camera image is adjusted to match the device screen size in step S106, there may be a problem in that image quality is distorted. More specifically, although the resolution of the original camera image and the screen resolution of the device are usually different, a problem arises in that the resolution decreases in the process of changing the camera resolution in order to fit the camera image on the entire screen of the device. ), the image data or video data is of poor quality, so the user cannot help but obtain a high-quality result.

In the present invention, step S106 is introduced, including a series of steps to be described later in order to solve this problem. 9A to 9D are processes sequentially shown in order to understand a method for improving image quality. In step S106, the first original image (image) of the photographing device and the original image (image) of the photographing device It starts by loading the virtual character rendering image scaled to the same size as ). (Fig. 9(a)) The original image (video) of the photographing device means an image (video) in which the target person is being photographed as it is expressed or at an arbitrary point in time, and the virtual character rendering image is also shown in Fig. 9(a). As shown, it means an image in which only a virtual character is implemented (at this time, the size of the virtual character is adjusted to the same size as the original image (image)).

After the original image (video) and the virtual character rendering image are loaded, the step of separating the virtual character part and the background part from the virtual character rendering image as shown in FIG. 9(b), and the virtual character part is excluded The conversion step to a binarized image in which the background of is inverted may be further performed.

After the conversion step to the binarized image is performed, as shown in Fig. 9(c), the pixel data of the previously obtained binarized image is used as masking in the original image (image), resulting in the original image (image). It is possible to obtain a first modified image (video) in which the virtual character part is excluded from

After obtaining the first modified image (video) in which the virtual character part is excluded from the original image (video), as shown in FIG. 9(d), the first modified image (video) is shown in FIG. 9(a). ), it is possible to obtain a captured image with improved image quality distortion problem by performing the inter-pixel ADD operation of the virtual character rendering image of ). On the other hand, the image obtained in this way will be recognized as a photograph or image in which the target person and the virtual character appear to exist in one space from the perspective of the user.

Meanwhile, referring to FIG. 10, the photographing device 100 generates an image or image in which a virtual character is added to an image photographed through a lens in a new format 500, thereby allowing the user to access the image data or image. Data can be created. Image data format can include various things including PNG, GIF, JPG, BMP, TIF, APNG, etc.Video data format can include various things including AVI, MPG, WMV, FLV, MOV, DAT, MP4, etc. have.

Meanwhile, the image data or image data generated by the step S106 may be transmitted to other users by using an export function or a sharing function on an application.

A method of implementing the above augmented reality service and a computer-readable storage medium storing instructions for executing the method have been described. Meanwhile, the present invention is not limited to the specific embodiments and application examples described above, and various modifications are implemented by those of ordinary skill in the art without departing from the gist of the present invention claimed in the claims. Of course, these modifications should not be understood to be distinguished from the technical spirit or prospect of the present invention.

Meanwhile, the method described herein may be implemented in hardware or a combination of hardware and software suitable for a specific application. Here, the hardware includes both general-purpose computer devices such as personal computers and mobile communication terminals, as well as enterprise-specific computer devices, and computer devices include memory, microprocessors, microcontrollers, digital signal processors, application integrated circuits, programmable gate arrays, and programmable array organizations. It may be implemented as a device including, or a combination thereof.

In addition, the computer software, instructions, codes, etc. described above can be stored or accessed by a readable device, wherein the readable device includes a computer component, RAM or ROM having digital information used for computing over a period of time. Solid state storage, permanent storage such as optical disk, large storage storage such as hard disk, tape, drum, etc., optical storage such as CD or DVD, flash memory, floppy disk, magnetic tape, paper tape, standalone RAM disk, removable from computer It may include memory such as mass storage and dynamic memory, static memory, variable storage, and network-attached storage such as the cloud. On the other hand, the commands and codes here are information-oriented languages such as SQL and dBase, system languages such as C, Objective C, C++, and assembly, architectural languages such as Java, NET, and application languages such as PHP, Ruby, Perl, and Python. All of the same languages are included, but the present invention is not limited thereto, and all languages widely known to those of ordinary skill in the art to which the proposed invention belongs may be included.

In addition, the "computer-readable medium" described herein includes any medium that contributes to providing instructions to a processor for program execution. Specifically, it includes, but is not limited to, a nonvolatile medium such as an information storage device, an optical disk, a magnetic disk, and a volatile medium such as a dynamic memory, and a transmission medium such as a coaxial cable, a copper wire, and an optical fiber for transmitting information.

100 shooting devices
10 target people 20 virtual characters
200A, 200B, 200C, 200D objects (background, floor)
210A baseline
300A, 300B, 300C, 300D object (target person)
200 flat
400 target locations
410 overlap area
500 format

Claims (12)

In the method of implementing an augmented reality service,
Obtaining an image including a target person;
Receiving a user input for selecting a virtual object;
Identifying a plane in which the target person is located in the image;
Loading a virtual object selected by the user input;
Displaying the loaded virtual object on a plane identified in the image;
Generating image data or image data from an image including the virtual object and a target person;
Including,
The step of identifying the plane on which the target person is located,
The object having the largest plane area among the objects located at the bottom of the image is identified as a target plane,
How to implement augmented reality service.
delete delete delete delete The method of claim 1,
Displaying the virtual object on the target plane,
Characterized in that to display a predefined animation for the virtual object,
How to implement augmented reality service.
The method of claim 6,
Before the step of displaying the virtual object on the target plane,
Determining a target position in which the virtual object will be located on the target plane;
It characterized in that it further comprises,
How to implement augmented reality service.
The method of claim 7,
The step of determining a target location where the virtual object will be located,
At least one of a distance between the target person in the image, a size of an area overlapping with the target person when the virtual object is located, or a ratio of an area overlapping the virtual object and the target person to the target person area It characterized in that it is determined on the basis of,
How to implement augmented reality service.
The method of claim 1,
The step of identifying the target person,
Characterized in that to identify the lower body region, the upper body region, and the face region of the target person,
How to implement augmented reality service.
The method of claim 9,
Displaying the virtual object on the target plane,
Displaying the virtual object in a predefined pose corresponding to a lower body region, an upper body region, or a face region of the identified target person,
How to implement augmented reality service.
A computer-readable storage medium storing instructions for executing a method of implementing an augmented reality service,
The method of implementing the augmented reality service,
Obtaining an image including a target person;
Identifying a target plane in which the target person is located in the image;
Receiving a user input for selecting a virtual object;
Loading a virtual object selected by the user input;
Displaying the loaded virtual object on the target plane;
Generating image data or image data from an image including the virtual object and a target person;
Including,
The step of identifying the plane in which the target person is located,
The object having the largest plane area among the objects located at the bottom of the image is identified as a target plane,
Computer-readable storage media. In the method of implementing an augmented reality service,
Obtaining an image including a target person;
Receiving a user input for selecting a virtual object;
Identifying the target person in the image;
Identifying a plane in which the target person is located in the image;
Loading a virtual object selected by the user input;
Displaying the loaded virtual object on a plane identified in the image;
Generating image data or image data from an image including the virtual object and a target person;
Including,
The step of identifying the target person includes identifying a lower body region of the target person,
The step of identifying the target plane on which the target person is located comprises identifying, as the target plane, an object having the largest plane area among the objects in contact with the lower body area of the identified target person,
How to implement augmented reality service.

Images