KR20110010666A - Providing method of augmented reality and personal contents in terminal with camera - Google Patents

Abstract

PURPOSE: A method for providing an augmented reality and personalized contents in a terminal with a camera is provided to offer personalized contents with 3D contents. CONSTITUTION: A code generating unit(300) extracts specific point information from the image(100). The code generation unit generates a unique code to a personalized contents(120). If a user lights a target object or an image and code(200) through a camera mounted device(400), the device recognizes the code. A contents providing server(600) receives specific feature information, 3D contents(110), and the personalized contents(120).

Description

Providing Method of Augmented Reality and Personal Contents in Terminal with Camera}

The present invention relates to a method for providing augmented reality and personalized content corresponding to a code in a camera built-in terminal.

Augmented Reality (AR) is a modified form of Virtual Reality (VR) that refers to a user interface technology that combines computer-generated virtual reality with real-life images viewed by the user.

However, the existing augmented reality technology, using the method of augmenting the corresponding content on the marker by comparing the image pattern of the pre-recorded marker as shown in Figure 1, or the feature point of the image (markerless) without the marker as shown in Figure 2 Use information to express 3D information directly on a picture or object.

When using a marker, the implementation is easy and the recognition speed is fast, but there are disadvantages in that a separate marker is prepared and aesthetic expression is poor.

The markerless method of solving this problem does not require a separate marker and can use a picture or an object as it is, but it must be separately informed that augmented reality is applied because the user does not know whether a user uses augmented reality. When the number of images to be compared is large due to a large amount of data calculations for comparing the data, it is difficult to guarantee real-time.

In addition, since only one 3D content is matched to one image in both the marker method and the markerless method, it is difficult to recycle the 3D content, and it is difficult and expensive to implement the 3D content for the individual to use.

On the other hand, when using a two-dimensional barcode as shown in Figure 3 for a personalized service separately from augmented reality, it is possible to include a lot of information, such as numbers, text, URL, etc. in the two-dimensional barcode, code generation is easy and recognition speed is high Although it is widely used, it is difficult to express three-dimensional information on a small two-dimensional bar code, and it is difficult to express content by looking only at a code, and aesthetic expression is inferior.

In view of the above-described problems of the prior art, the present invention combines code and markerless augmented reality so that not only one content is connected to one image, but different 3D content and personalized content according to the code. To provide to the user in the camera built-in terminal, such as a smartphone.

In order to achieve the above technical problem, the present invention, in a method for providing augmented reality and personalized content in the camera built-in terminal,

Illuminating the object itself on which the image 100 and the code 200 are displayed or the screen on which the object is displayed to the camera built-in terminal 400; An application of the camera built-in terminal 400 recognizing the code 200 and transmitting the code value to the content providing server 600; The content providing server 600 transmits the feature point information of the image 100 corresponding to the code value, the 3D content information 110, and the menu information connected to the personalized content 120 to the camera built-in terminal 400. Doing; An application of the camera built-in terminal 400 displaying three-dimensional content information 110 and menu information on a screen of the camera built-in terminal 400 in the image 100 using the feature point information; And receiving the personalized content 120 corresponding to the menu selected by the user from the content providing server 600 and displaying the personalized content 120 on the screen.

Using the present invention as described above, by combining the code and markerless augmented reality, it becomes possible to provide a service that provides personalized content with the three-dimensional content in the smartphone.

In more detail, when an individual registers his or her own personalized content such as a picture, video, voice, or text together with the image to be used, the code is automatically generated, and the personalized content according to the code is expressed on the image for personalization. Services become available.

In addition, augmented reality is realized by combining personal content with pre-implemented three-dimensional content, so that the reusability of the already produced three-dimensional content can be improved and an individual can use augmented reality easily and inexpensively without paying a high cost. do.

In addition, since personalized content and three-dimensional content that matches the image are stored in the server, the user can easily update without changing the information by changing only the information stored in the server on the web or mobile phone.

In addition, in the case of directly recognizing an image, if the number of images increases, the search takes a long time. However, when used with a code as in the present invention, since the image is recognized at the moment of recognizing the code, the calculation amount is significantly reduced. Real time guaranteed.

In addition, since the two-dimensional bar code is already widely used, the user recognizes that a code recognition program that is generally used should be used at the moment of recognizing the code even if the user does not know whether the augmented reality is applied or not. Even if the program is recognized as a guide through the use of the augmented reality program through a URL stored in the code, it is easy to determine whether augmented reality is applied.

In addition, authorized users and passwords can be managed on the server, preventing the provision of content to unwanted users, thereby protecting personal privacy.

1 illustrates marker-based augmented reality in the prior art.
2 illustrates markerless augmented reality in the prior art.
Figure 3 shows a two-dimensional bar code used in the prior art.
4 is an overall configuration diagram of the present invention.
5 is a detailed configuration diagram of the code generation unit of the present invention.
Figure 6 shows an example of applying a smart phone to which the present invention is applied to a wedding invitation.
Figure 7 shows a flow chart for providing augmented reality and personalized content by applying the smart phone to which the present invention is applied to a wedding invitation.
8 illustrates various arrangements and combining methods of images and codes in the present invention.
Figure 9 illustrates various code types, arrangements and forms in the present invention.
Figure 10 shows a seal or sticker or stamp to which the present invention is applied.
Figure 11 shows a business card to which the present invention is applied.

First, the overall configuration of the present invention will be described with reference to FIG.

Images (100) using photographs of users, caricatures, images of stars or merchandise, and icons using characters for paper or electronic objects (wedding invitations, cards, letters, business cards, product brochures, signs, books, etc.) The code 200 (two-dimensional code such as a barcode or QR code, a custom code, etc.) is printed (paper type) or attached (electronic form), which code 200 is present on the image 100. Or adjacent to the image 100.

This code 100 is a code generated when the personalized content described below is registered in the content providing server 600.

The code generation unit 300 provides the content providing server 600 with the image 100, the 3D content 110, and the personalized content 120 (picture, audio, video, text, etc.) to be used by the user. And extracts feature point information from the image 100 to be used by the user, and generates unique codes (barcode two-dimensional barcodes, custom codes, etc.) for the personalized content 120 provided by the user. This unique code can be a unique serial number (eg 0987654321) or a URL (http: // (address) / code = 0987654321).

In this case, the user inputs the image 100 to the code generation unit 300 and selects the 3D content 110 that is manufactured to synthesize the input image 100 and the personalized content 120. In this case, instead of selecting the 3D content 110, the user may be prepared in advance for the image 100 and provided to the user.

That is, when the user inputs the personalized content 120 to the code generation unit 300 (in this case, the user may select which size and location of the personalized content to be displayed, etc.), and generate the code. The unit 300 generates a unique code 200 for the personalized content 200, and the generated code 200 is attached to the image 100.

In addition, in order to protect user privacy, the code generation unit 300 allows a person who knows the password to register the password, or register a list of available user telephone numbers so that only the terminal of the registered phone number can be used. You can do that.

The camera-mounted device 400 refers to a terminal with a built-in camera, and various forms such as a portable terminal with a built-in camera (eg, a smartphone), a PC connected with a camera, a tablet PC with a built-in camera, a goggle connected with a camera, etc. It is possible.

When the user illuminates the object itself or the image 100 and the code 200 on the screen on which the object is displayed by using the camera-mounted device 400, the code 200 is recognized and transmitted to the content providing server 600. The content providing server 600 receives the feature point information of the image corresponding to the code, the 3D content 110, and the personalized content 120, and displays the 3D image on the screen of the camera-mounted device 400 on the image 100. Content 110 and personalized content 120 are displayed.

The communication network 500 uses a mobile communication network, the Internet, a wireless LAN, and the like, transmits an image and personalized content input by a user to the content providing server 600, and communicates with the camera-mounted device 400.

The content providing server 600 receives the feature point information of the image 100, the 3D content information 110, and the personalized content information 120 together with the code 200 generated by the code generation unit 300, and stores them in a database. Information stored in the 700 and the camera-mounted device 400 recognizes the code 200 and makes a request to the content providing server 600, the information corresponding to the code 200 in the database 700 (the feature point of the image). Information, 3D content information, and personalized content information) is extracted and transmitted to the camera-mounted device 400.

In this case, the personalized content 120 stored in the content providing server 600 may be modified by the user accessing the content providing server 600 through the web, and after selecting an image stored in the mobile phone (smartphone) Select the personalized content (photo, audio, video, text, etc.) stored on your phone, such as how to upload an image (# 4200 upload) on your phone, and enter the identifier (#), service name (MP) and code number (43567) When inputted together and transmitted, it is also possible to automatically modify the pre-stored personalized content 120 corresponding to the service name (MP) and code number 43567 automatically with the personalized content 120 stored in the mobile phone.

The database 700 includes unique code information, feature point information of an image corresponding to the code, three-dimensional content information, information (display position, size, etc.) related to the three-dimensional content information being displayed on the image, and personalization. Information about the content (user's name, age, photo, video, voice data, greetings, etc.).

Then, access rights related information is stored so that only users with access rights have access.

Next, a detailed configuration of the code generation unit 300 will be described with reference to FIG. 5.

Since the application software recognizes which image 100 is already used through the code value, the feature point extractor 310 extracts the feature point from the image 100 to be used by the user.

In addition, through the content input unit 320, the user inputs (or selects) the 3D content 110 to be displayed on the image 100, and the position and height of the 3D content 110 on the image 100. Specifies how much to display in.

The input 3D content 110 is produced in a format in which a display unit that can be directly linked to or displayed with the personalized content 120 input by the user is designated. For example, when a user touches a balloon in the 3D content 110, a video (personalized content) input by the user is played, or the user's face (personalized content) is mapped and displayed on the balloon itself.

The personalized content collecting unit 330 receives the personalized content 120 (photo, video, voice data, text information) of the user from the user, receives pre-produced content, or is provided with a camera, a microphone, or the like. Recorded and input from the device.

In addition, the code output unit 340 generates a unique code value corresponding to the image 100, the three-dimensional content 110 and the personalized content 120, the barcode corresponding to the code value, the two-dimensional barcode, After converting the custom code into a printable image, the user downloads the code image and places it on or around the image 100 to be used.

Now, with reference to the example of FIG. 6 and the flowchart shown in FIG. 7 applied to a wedding invitation in the form of a paper for a method of providing personalized content in a smartphone by combining code and markerless augmented reality according to the present invention. This will be described.

In the wedding invitation to which the present invention is applied, the marriage-related content (image 100) (the ribbon pattern and the letter of the wedding invitation in the first drawing of FIG. 6) is present at the top, and the three-dimensional content 110 related to the wedding invitation at the bottom is personalized. A code 200 representing the content 120 is printed.

The sender of the wedding invitation registers the 3D content 110 and the personalized content 120 corresponding to the code to the content providing server 600 in advance.

In this case, when the code generation unit 300 provides the user with a set of three-dimensional content 110 for each use, and allows the user to select the specific three-dimensional content 110 according to the use, the same three for the specific use (for example, a wedding invitation). Using dimensional content 110 (e.g., the dancing character in the first drawing of FIG. 6), and personalized content 120 is distinguished according to the code, the user does not need to create three-dimensional content 110 and the 3D content The recyclability of the 110 is increased.

Thus, when the user who received the wedding invitation is driving the application according to the present invention on the smartphone (camera-mounted device 400) to illuminate the wedding invitation through the camera of the smartphone, the application code at the bottom of the captured image ( 200) (S1).

Now, when the application of the smartphone transmits the recognized code 200 to the server 600 (S2), the server 600, the feature point information corresponding to the code value, the three-dimensional content (dance characters), and personalized The menu information (four symbols) connected to the content 120 is transmitted to the smartphone (S3). In this case, when it is desirable to transmit the personalized content 120 to the smartphone at once (for example, when the data size of the personalized content is small, high speed data transmission is possible even if the data size is large and the storage capacity of the smartphone is sufficient). In this step (S3) it is also possible to have the personalized content 120 delivered to the smartphone.

The application of the smartphone uses the transmitted feature point information to match the feature point in the current camera image 100 to display the three-dimensional content 120 (a dancing character) at an appropriate position and orientation, and to connect a menu connected with personalized content ( 4 symbols) (refer to the first drawing of FIG. 6) (S4).

Now, when the user touches (selects) a related image symbol to view the bride's picture among the four symbols, the smartphone receives the bride's picture related data, which is personalized content, from the server 600 and displays the bride's picture on the screen. Displaying a picture (see the second drawing of FIG. 6), and touching the map symbol to view the location of the wedding hall, the smartphone receives data related to the wedding hall location map, which is personalized content, from the server 600 and then related to the wedding hall on the screen. The map is displayed (see the third drawing of FIG. 6) (S5).

That is, according to the present invention, if the code 200 is transmitted to the content providing server 600 through the wedding invitation that the camera of the smartphone shines, the content providing server 600 may provide the feature point information corresponding to the code value, Three-dimensional content (a dancing character) and a menu connected with personalized content (four symbols) are delivered to the smartphone, and the three-dimensional content (a dancing character) is displayed on the screen of the smartphone together with the image 100 and personalized. The menu connected to the content is displayed on the screen so that the personalized content is displayed on the screen according to the user's menu selection, and it is possible to combine the code and the markerless augmented reality to provide personalized content in the smartphone.

On the other hand, in the wedding invitation to which the present invention is applied, the marriage-related content (image 100) (the ribbon pattern and the words Wedding Invitation in the first drawing of Figure 6) is printed on the top, the wedding invitation-related three-dimensional at the bottom Although the description has been made of a case where the image and the code are arranged separately from each other in such a manner that the code 200 representing the content 110 and the personalized content 120 is printed, the image and the code may have various arrangements and combinations. As such, this will be described with reference to FIG. 8.

As shown in the upper part of the first drawing of FIG. 8, the color QR code may be formed separately from the image, but the color QR code may be synthesized inside the image as shown in the lower part of the first drawing.

In addition, as shown in the second drawing of FIG. 8, a code may be derived by inserting a watermark for a code into an image to illuminate the image with an application such as a Scantag application.

And, unlike the third upper part of FIG. 8, if the icon is used as the image as the lower part, the user has a high affinity for the icon and the user can easily recognize the content of the content.

Next, the type, arrangement, and form of the code will be described with reference to FIG.

The top and bottom views of Fig. 9 are different in that the image is in the form of a graphic image or a photograph, and the type, arrangement and shape of the codes are the same.

Columns 1-2 are QR codes placed on the top and left sides. Columns 2-6 use the entire service name (MPOP) and code (98762) as one code. Column 5 is the service name and code. By overlapping the above and below, the service name and code of one of the upper and lower parts can be recognized by the rest even if they are damaged.

Now, various business methods to which the present invention is applied will be described.

In the case of a wedding invitation printing service or wedding album production service, the service provider produces a set of images 100 (e.g. icons) and augmented reality three-dimensional content 110 (e.g. dancing characters) for each service concept. Saved in the content providing server 600, the user connects to the content providing server 600 to select the desired from the image 100 and the three-dimensional content 110, the user's personalized content 120 (e.g., Providing a personalized code 200 by providing voice, music, dictionary, video, information, etc., printing the selected image 100 and the generated code 200 on a paper-type wedding invitation, or an electronic wedding invitation (eg : E-mail invitation).

Now, the recipient who has received this paper wedding invitation or electronic wedding invitation illuminates the smartphone with the image 100 and the code 200 on the monitor screen on which the paper wedding invitation or electronic wedding invitation is displayed. Recognizes and transmits the code 200 to the content providing server 600, and the content providing server 600 displays the dancing character, which is the three-dimensional content 110 corresponding to the code, on the image 100 in augmented reality. When the menu for the personalized content 120 is displayed, the personalized content 120 corresponding to the menu selection of the receiver is displayed on the screen.

In this case, depending on the selection menu, the wedding ceremony-related photos such as the bride's photo and the map of the wedding ceremony location may be displayed on the screen. It is also possible to display the money account account information for the payment or to be associated with the money micropayment service of the money gift.

In addition, if necessary, the service user (groom, bride) may set a period in which the personalized content 120 is provided to the content providing server 600, and a menu and personalized information displayed only on a specific smart phone (recognized by a specific mobile phone number). You can also add content 120.

You can extend the wedding service to send photos taken by the bride and groom's friends, comment on the pictures, or upload a picture taken on a honeymoon later, and then only notify those who have seen your wedding invitation content. You can also perform the function of.

In addition, as shown in FIG. 10, the present invention is applied to a Christmas seal / TB tube seal, a star or a product / company promotion sticker, a stamp, and accesses a website to store personalized content such as personal preference music and content in advance, Make sure to include images and codes in the form of paper or electronic seals, stickers or stamps, and the buyers of these paper or electronic seals / stickers / stamps may be paper or electronic letters or cards. Or other objects (diaries, notebooks, devices, blogs, etc.) that allow the recipient to illuminate paper, electronically shaped seals, stickers, or stamps on their smartphones so that the corresponding three-dimensional and personalized content is displayed. It is also possible.

At this time, for example, A seal / sticker / stamp may be associated with content A, and B seal / sticker / stamp may be associated with content B. However, if A seal / sticker / stamp and B seal / sticker / stamp are placed together, C content will be It is possible to make it linked, and to make the entire seal / sticker / stamp set reflect the D content.

If the present invention is applied to a business card and a business card in the form of a paper or electronic form printed with an icon (image) and a code (bar code) is printed on the bottom of the corporate logo as shown in Fig. 11 (of course, the corporate logo itself may be an image. Corporate logos and icons can be images), paper business cards on smartphones, or electronic business card screens on smartphones, to provide the person's photo or greeting. You can call or text a message, or display promotional content (publicity photos, mobile homepage connections, directions, solutions and product introductions, etc.) from related companies.

In addition, when the present invention is applied to a movie ticket or a tour guide booklet in paper form or electronic form, a film ticket or light guide in paper form is applied to a smartphone by applying an appropriate image and code to the movie ticket or tour guide booklet. If you shine a booklet or screen a movie ticket or tourist guide booklet in electronic form with a smartphone, personalized content for the promotion of the movie or tourist attraction is displayed on the screen. At this time, by entering a unique number (eg # MPOP98762) with the corresponding code in the smart phone, it is also possible to send their content (eg photo) to participate in the event.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

For example, in the above description, a method of using a link for playing a photo or a video when a predetermined operation (such as a touch of a specific area) is performed on the 3D content is described. It is also possible to combine and display the user's content in a specific area of the three-dimensional content by fusion.

In the above description, when the user inputs the personalized content 120 to the code generator 300, the code generator 300 generates the code 200 for the personalized content 200. In addition, a user or a company or an association may issue an image 100 and a code 200 and cause the personalized 3D content 110 and the personalized content 120 to be linked to each other. It is also possible for a user to edit and use the 3D content 110 and the personalized content 120 from the 3D content 110, the personalized content 120, and the code 200 set provided by the content providing server. It is also possible for 600 to provide a set of images 100 and codes 200 and to allow a user to select three-dimensional content 110 and personalized content 120.

100: image
200: code
300: code generation unit
400: device with camera
500: communication network
600: content providing server
700: database

Claims (15)

In the method of providing augmented reality and personalized content in the camera built-in terminal,
Illuminating the object itself on which the image 100 and the code 200 are displayed or the screen on which the object is displayed to the camera built-in terminal 400;
An application of the camera built-in terminal 400 recognizing the code 200 and transmitting the code value to the content providing server 600;
The content providing server 600 transmits the feature point information of the image 100 corresponding to the code value, the 3D content information 110, and the menu information connected to the personalized content 120 to the camera built-in terminal 400. Making;
An application of the camera built-in terminal 400 displaying three-dimensional content information 110 and menu information on a screen of the camera built-in terminal 400 in the image 100 using the feature point information;
And augmented reality in the camera-embedded terminal, characterized in that the camera-integrated terminal 400 receives the personalized content 120 corresponding to the menu selected by the user from the contents providing server 600 and displays it on the screen. How to Provide Personalized Content. The method of claim 1,
The camera built-in terminal 400 is a smartphone, characterized in that for providing augmented reality and personalized content in the camera built-in terminal. The method of claim 1,
The code value is
Method for providing augmented reality and personalized content in a camera built-in terminal, characterized in that for providing the image (100), code (200), three-dimensional content (110) and personalized content (120) at the user side. The method of claim 1,
The code value is
Method for providing augmented reality and personalized content in a camera built-in terminal, characterized in that the generated by inputting the personalized content 120 provided by the user to the code generation unit (300). The method of claim 1,
The three-dimensional content information 110,
In the camera built-in terminal, characterized in that the user inputs the image 100 to the code generation unit 300, and the user selects that the input image 100 and the personalized content 120 can be synthesized How to provide augmented reality and personalized content. According to claim 1,
The personalized content (120), the user is connected to the content providing server 600 through the web, characterized in that to modify, a method for providing augmented reality and personalized content in the camera built-in terminal. According to claim 1,
The personalized content 120, after the user selects the personalized content stored in the camera built-in terminal 400, and enters the service name and code number with an identifier to be transmitted automatically modified, characterized in that the camera built-in terminal How to provide augmented reality and personalized content in. According to claim 1,
The image 100 and the code 200 is formed by separating the code from the image, a method for providing augmented reality and personalized content in a built-in camera terminal. According to claim 1,
The image (100) and the code (200) is formed by synthesizing a color QR code inside the image, a method for providing augmented reality and personalized content in a camera built-in terminal. According to claim 1,
The image (100) and the code (200) is formed by inserting a watermark for the code in the image, a method for providing augmented reality and personalized content in a built-in camera terminal. According to claim 1,
The image 100 is an icon, a method for providing augmented reality and personalized content in a built-in camera terminal. According to claim 1,
The object is a wedding invitation in paper or electronic form,
The personalized content 120 is at least one of the wedding ceremony-related photos, the wedding ceremony location map, the bride and groom's voice, outdoor shooting video, celebration money account information for the celebration of the celebrations, mobile payments associated with celebrations How to provide augmented reality and personalized content in. According to claim 1,
The object is an augmented reality in the camera built-in terminal, characterized in that the paper form or the electronic form of the seal or sticker or stamp attached to the paper form or the electronic form of the image 100 and the code 200 is present How to Provide Personalized Content. According to claim 1,
The object is a paper or electronic form of a business card, characterized in that for providing augmented reality and personalized content in the camera built-in terminal. In the method of providing augmented reality and personalized content in the camera built-in terminal,
Illuminating the object itself on which the image 100 and the code 200 are displayed or the screen on which the object is displayed to the camera built-in terminal 400;
An application of the camera built-in terminal 400 recognizing the code 200 and transmitting the code value to the content providing server 600;
The content providing server 600 stores the feature point information of the image 100 corresponding to the code value, the 3D content information 110, the personalized content 120, and menu information connected to the personalized content 120. Transmitting to the camera built-in terminal 400;
An application of the camera built-in terminal 400 displaying three-dimensional content 110 and menu information on a screen of the camera built-in terminal 400 in the image 100 using the feature point information;
And displaying the personalized content 120 corresponding to the menu selected by the user on the screen by the camera built-in terminal 400. The method of providing augmented reality and personalized content in the built-in camera terminal.

Images