JP2014219822A - Content display device, content display method, program, and content display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a content display device capable of displaying various types of contents while discriminating similar AR markers or AR markers of different colors.SOLUTION: Upon imaging a marker including a digital watermark of by a camera of a terminal 3 with a camera, a digital watermark code is extracted from an image currently imaged. A database 7 is then searched to extract a content 9 associated to the digital watermark code. The content 9 is displayed in a display part 36 of the terminal 3 with a camera, according to a positional state of an AR marker image.

Description

  The present invention relates to an AR technology for displaying content on a mobile terminal or the like, and more particularly to an AR technology for displaying content using an AR marker and a digital watermark.

  Conventionally, AR (Augmented Reality) technology for reading a marker printed on a card or the like and displaying a three-dimensional image or the like associated with the marker information on a display unit of a portable terminal has been developed, and a card game (For example, Non-Patent Document 1 and Patent Document 1).

Non-Patent Document 1 describes a method for detecting a three-dimensional position of a marker used in the AR technology.
Patent Document 1 discloses a system that provides a soccer game by reading a marker printed on the back of a game card.

  Furthermore, recently, software for displaying a three-dimensional image associated with an AR marker photographed by a camera of a mobile terminal at a position based on the three-dimensional position of the AR marker on a display screen of the mobile terminal is provided.

Hirokazu Kato et al., “Augmented Reality System Based on Marker Tracking and Its Calibration”, Transactions of the Virtual Reality Society of Japan, Vol. 4, no. 4, 1999

JP 2002-301264 A

However, there are some problems with the conventional AR technology.
FIG. 5 is a diagram showing a display example of the conventional AR technique and its problems.

  As shown in FIG. 5A, in the AR technique, for example, when the AR marker 2a is photographed with a camera, the AR display content 4a associated with the AR marker 2a is displayed on the photographed marker.

However, in the conventional AR technology, as shown in FIG. 5 (b), there is a problem that when displaying different AR display contents 4b, it is necessary to prepare AR markers 2b of different images.
That is, it is necessary to prepare the AR marker and the content so that the displayed content and the AR marker are associated with each other in a one-to-one relationship, and in order to display many types of content, the number of AR markers is required. There is a problem that it becomes complicated.

Moreover, the AR marker 2c of FIG.5 (c) has shown the case where the design is the same as the AR marker 2a of Fig.5 (a), and a color is different.
In such a case, in the conventional AR technology, only the symbol is used for the discrimination of the marker, so that the AR display content 4c to be displayed is the same as the AR display content 4a in FIG. There is a problem that display content of different colors (content 6c to be displayed) cannot be displayed depending on the color of the image.

  FIG. 5D and FIG. 5E show an example in which a part of the AR markers 2d and 2e are different. That is, the design of “Mr. Daikoku” on both AR markers 2d and 2e is the same, but the direction of the “fish” on the side is upward in the AR marker 2d in FIG. 5D, and the AR marker 2e in FIG. Then, it is a case where the design of the AR marker is partially different, such as downward.

In the conventional AR technology, since a real-time image recognition technology is used in a portable terminal or the like, there is a problem that such a fine difference between AR markers cannot be recognized.
Therefore, for example, the display content 8d should be displayed in the AR marker 2d in FIG. 5D (“fish” is upward), and the display content 8e is displayed in the AR marker 2e in FIG. 5E (“fish” is downward). In spite of the correct display content, the AR marker cannot be recognized correctly, and the display contents 4d and 4e are confused and displayed.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a content display device that can display various contents by identifying similar AR markers or different color AR markers. It is to be.

In order to achieve the above-mentioned object, the first invention is a content display device that displays information by reading information from a marker printed by superimposing a marker image and a digital watermark, the digital watermark code and Storage means for linking and storing content, photographing means for photographing the marker, code extracting means for identifying a digital watermark of the photographed marker and extracting a code, and the content corresponding to the extracted code Content acquisition means for acquiring image data, position recognition means for recognizing the position of the marker image of the photographed marker, and display means for displaying the acquired content at the recognized position. It is a display device.
According to the first aspect, since the content to be displayed corresponding to the marker image is the content corresponding to the digital watermark code printed superimposed on the marker image, various contents can be used regardless of the marker image. Can be displayed flexibly.

The marker according to the first aspect of the invention is preferably the same marker image in which the different digital watermarks are superimposed and printed.
Thereby, different contents can be displayed even in the same marker image.

The marker according to the first aspect of the invention is preferably a marker image of the same design and a different color, on which the digital watermarks that are different for each color are superimposed and printed.
Thus, different contents can be displayed even with marker images of the same design and different colors.

The content of the first invention is a three-dimensional image, the position recognition means further recognizes a change from a reference coordinate system of the marker image, and the display means changes the change to the recognized position. It is desirable that the three-dimensional image is deformed and displayed according to the above.
Thereby, a three-dimensional image can be flexibly displayed on the marker image.

According to a second aspect of the present invention, a computer having storage means for reading information from a marker printed by superimposing a marker image and a digital watermark and displaying the content, and storing the digital watermark code and the content in association with each other is performed. A content display method, a photographing step for photographing the marker, a code extracting step for identifying a digital watermark of the photographed marker and extracting a code, and content acquisition for obtaining the content corresponding to the extracted code A content display method comprising: a step; a position recognition step for recognizing a position of a marker image of the photographed marker; and a display step for displaying the acquired content at the recognized position.
According to the second invention, since the content to be displayed corresponding to the marker image is the content corresponding to the digital watermark code printed superimposed on the marker image, various contents can be used regardless of the marker image. Can be displayed flexibly.

A third invention is a program for causing a computer to function as a content display device that displays information by reading information from a marker printed with a marker image and a digital watermark superimposed thereon, and Storage means for associating and storing the code and content, photographing means for photographing the marker, code extraction means for identifying the digital watermark of the photographed marker and extracting the code, and the content corresponding to the extracted code Is a program for functioning as a content acquisition means for acquiring the image, a position recognition means for recognizing the position of the marker image of the photographed marker, and a display means for displaying the acquired content at the recognized position.
By installing the third invention in a computer such as a portable terminal, the first invention can be obtained and the method according to the second invention can be executed.

  The fourth invention is a content display system in which a content display device that reads information from a marker printed with a marker image and a digital watermark superimposed and displays the content, and a database server are connected via a network. The database server comprises storage means for associating and storing a digital watermark code and the content, and the content display device identifies a photographing means for photographing the marker and a digital watermark for the photographed marker. Code extraction means for extracting the code, content acquisition means for acquiring the content corresponding to the extracted code from the database server, position recognition means for recognizing the position of the marker image of the photographed marker, The acquired content is displayed at the recognized position. And it shows means, a content display system comprising: a.

  As described above, the content display device of the present invention can display a plurality of different display contents by printing different digital watermarks even for the same marker image. Markers and marker images of different colors can be identified by a digital watermark code.

  According to the present invention, it is possible to provide a content display device that can display various contents by identifying similar AR markers or different color AR markers.

The figure which shows the structural example of the content display apparatus 1 which concerns on embodiment of this invention. The block diagram which shows the hardware structural example of the terminal 3 with a camera The flowchart which shows the flow of a process of the content display apparatus 1. The figure which shows the example of a display of the content display apparatus 1 The figure which shows the example of a display of the conventional AR technique, and its problem

  Hereinafter, a preferred embodiment of a content display device 1 according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a diagram showing a configuration example of a content display device 1 according to an embodiment of the present invention.
The content display device 1 includes, for example, a camera-equipped terminal 3, a digital watermarked marker 5, a database 7, and the like.

The digital watermarked marker 5 is, for example, a marker printed by superimposing a digital watermark on an AR marker disclosed in Non-Patent Document 1, Patent Document 1, or the like.
As shown in FIG. 1, the digital watermarked marker 5 is invisible or difficult to see from a frame (black frame portion) for detecting the three-dimensional position of the marker and a pattern such as characters inside the marker. It consists of a digital watermark (not shown).
In the example of FIG. 1, there is a frame for detecting a three-dimensional position, but this frame is not provided and only a picture may be used. In that case, a three-dimensional position is detected from the pattern.

  The digital watermark uses, for example, Digimarc Corporation's digital watermark technology (for example, JP-T-2002-511585, JP-A-2005-328528), for example, a 100-bit code (including an error code code) is subliminalized. This is realized by encoding and printing while slightly changing the brightness (or reflectance) of a plurality of areas provided in the digital watermarked marker 5.

The camera-equipped terminal 3 is, for example, a terminal such as a tablet terminal, a smartphone, or a camera-equipped mobile phone, but may be a computer such as a personal computer provided with a camera function.
By using the camera function, the digital watermarked marker 5 is imaged and taken into the camera-equipped terminal 3.

  The storage unit of the camera-equipped terminal 3 stores an identification program for identifying the digital watermarked marker 5 and a display program for displaying the display content 9 associated with the identification result. The display content 9 is displayed on the display 3.

In the database 7, an electronic watermark identification code and display content are associated with each other and stored in advance. The pattern of the AR marker may be stored in the database 7.
The database 7 may be built in the camera-equipped terminal 3 or may be installed in an external database server or the like and connected to the camera-equipped terminal 3 via a wireless or wired network.

  As shown in FIGS. 1A and 1B, when the digital watermark code of the digital watermarked marker 5 is identified by the camera-equipped terminal 3, it is stored in the database 7 in the display unit of the camera-equipped terminal 3. The display content 9 associated with the digital watermark code is displayed.

  As shown in FIG. 1 (a) and FIG. 1 (b), even when the digital watermarked marker 5 has the same AR marker pattern as shown in FIG. Different display contents 9a and 9b are displayed.

FIG. 2 is a block diagram illustrating a hardware configuration example of the camera-equipped terminal 3.
The hardware configuration in FIG. 2 is an example, and different configurations can be adopted. The camera-equipped terminal 3 can be configured by, for example, a personal computer.

  The camera-equipped terminal 3 includes a control unit 31, a storage unit 32, a media input / output unit 33, a communication control unit 34, an input unit 35, a display unit 36, a peripheral device I / F unit 37, and the like connected via a bus 38. The A camera 39 is connected to the peripheral device I / F unit 37.

  The control unit 31 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

  The CPU calls a program stored in the storage unit 12, ROM, recording medium, etc. to a work memory area on the RAM, executes it, drives and controls each device connected via the bus 38, and processes the control device 15. Execute. The ROM is a non-volatile memory and permanently stores a boot program for the camera-equipped terminal 3, a program such as BIOS, data, and the like. The RAM is a volatile memory, and temporarily stores a program, data, and the like loaded from the storage unit 32, ROM, recording medium, and the like, and includes a work area used by the control unit 31 for performing various processes.

  The storage unit 32 is an HDD (hard disk drive) or the like, and stores a program executed by the control unit 31, data necessary for program execution, an OS (operating system), and the like. With regard to the program, a control program corresponding to an OS (operating system) and application programs such as an identification process of the digital watermarked marker 5 and a display content display process executed by the content display device 1 described later are stored. Each of these program codes is read by the control unit 31 as necessary, transferred to the RAM, read by the CPU, and executed as various means.

The media input / output unit 33 (drive device) inputs / outputs data, for example, media such as a CD drive (-ROM, -R, -RW, etc.), DVD drive (-ROM, -R, -RW, etc.) Has input / output devices.
The communication control unit 34 includes a communication control device, a communication port, and the like, and performs communication control with other devices via a network. The network may be wired or wireless.

The input unit 35 inputs data, and includes an input device such as a touch panel, a keyboard, a pointing device such as a mouse, and a numeric keypad.
The display unit 36 is a display device such as a liquid crystal panel.

The peripheral device I / F (interface) unit 37 is a port for connecting peripheral devices. For example, a peripheral device such as a camera 39 is connected, and data is transmitted / received via the peripheral device I / F (interface) unit 37. The peripheral device I / F unit 37 is configured by USB, IEEE 1394, RS-232C, or the like, and the connection form with the peripheral device may be wired or wireless.
In FIG. 2, the camera 39 is connected to the peripheral device I / F unit 37, but may be directly connected to the bus 38.
The bus 38 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

  The database 7 may be built in the storage unit 32 of the camera-equipped terminal 3, or the database 7 is installed in an external database server or the like, and is stored in the database 7 via the communication control unit 34 of the camera-equipped terminal 3. You may make it connect.

  When the database 7 is provided in an external database server, the hardware configuration example of the database server is almost the same as the configuration example excluding the camera 39 of the camera-equipped terminal 3, and thus detailed description is omitted. To do.

FIG. 3 is a flowchart showing the flow of processing of the content display device 1.
First, when a content display program stored in the camera-equipped terminal 3 is activated, the control unit 31 of the camera-equipped terminal 3 operates the function of the camera 39, and the digital watermarked marker 5 is imaged (step 101). ).

Next, the control unit 31 of the camera-equipped terminal 3 processes the captured image and recognizes a digital watermark code (step 102).
As a method for recognizing a digital watermark code from a captured image, for example, a digital watermark code decoding method described in JP-A-2005-328628 may be used.

Next, the control unit 31 of the camera-equipped terminal 3 searches the database 7 and extracts content associated with the digital watermark code (step 103). The content is a two-dimensional or three-dimensional image, and is stored in the database 7 in association with a digital watermark code.
In addition to images, the content may be a URL (Uniform Resource Locator) of a site such as a moving image or a homepage.

Next, the control unit 31 of the camera-equipped terminal 3 recognizes the AR marker on the captured image (step 104). Here, a change in the shape of the captured image of the AR marker depending on the imaging direction is recognized, and the position status of the AR marker is grasped by the method of Non-Patent Document 1, for example.
That is, in the case of the digital watermarked marker 5 having the AR marker with the frame as shown in FIG. 1, if the image is taken from directly above by the camera 39, the frame of the AR marker becomes a square, but the image is taken from diagonally above. For example, based on the change to a shape such as a rectangle or a parallelogram, for example, the change of the AR marker from the reference coordinate system is recognized, and the position status of the AR marker is grasped.

  Next, the control unit 31 of the camera-equipped terminal 3 displays the content extracted in step 103 on the display unit 36 according to the position status of the AR marker extracted in step 104 (step 105). At the time of display, the control unit 31 of the camera-equipped terminal 3 appropriately modifies the two-dimensional or three-dimensional image that is the content, such as rotation and enlargement / reduction, according to the position situation.

FIG. 4 is a diagram illustrating a content display example by the content display device 1 according to the embodiment of the present invention.
As shown in FIGS. 4A and 4B, even if the AR marker is the same, different display contents are displayed by printing different digital watermarks.
That is, in FIG. 4A, the content display device 1 captures and processes the digital watermark-containing marker 5a on which the digital watermark a is printed, for example, thereby displaying, for example, a conical image content 9a. In FIG. 4B, the content 9b of the rectangular parallelepiped image is displayed by processing the digital watermarked marker 5b in which the digital watermark b is printed on the same AR marker as in FIG. 4A.

In this way, different display contents 9 can be displayed by the digital watermarked marker 5 in which different digital watermarks are printed on the same AR marker.
This has the advantage that it is not necessary to create an AR marker for each content, and has the advantage that the AR marker is unified and the user can easily recognize that it is a marker. However, there is an advantage that a plurality of different contents 9 can be displayed due to a difference in digital watermark.

FIG. 4C shows a case of a marker 5c with a digital watermark obtained by printing a digital watermark c on an AR marker of a different color with the same pattern as that in FIG.
In the conventional AR technology, since the color of the AR marker is not identified, even if the colors are different, it is determined to be the same as the AR marker of FIG. 4A and the same display content 9a is displayed. By printing (digital watermarked marker 5c), different display contents 9c can be displayed.
That is, it is possible to display contents of the same shape and different colors for each color of the AR marker.

As described above, by using the digital watermarked marker 5 that is a combination of the AR marker and the digital watermark, it becomes possible to display a plurality of contents for the same AR marker pattern. It becomes possible to display the content of the color corresponding to the color of the marker.
Furthermore, as shown in FIGS. 5D and 5E, it is possible to display different contents in which minute differences in the pattern of the AR marker are distinguished.

In the above description, the storage unit 32 of the camera-equipped terminal 3 is provided with a program such as extraction processing of a digital watermark code. It is also possible to have the program and the database 7.
In this case, the captured image of the digital watermarked marker 5 captured by the camera-equipped terminal 3 is sent to the server, the server extracts the digital watermark code, searches the database 7, and extracts the content associated with the digital watermark code, The content may be sent to the camera-equipped terminal 3 so that the camera-equipped terminal 3 displays the content on the display unit 36.

  In the above description, the AR marker, which is a pattern of a digital watermarked marker, is used only for extracting the position situation (steps 104 and 105 in FIG. 3), but it is linked to the combination of the AR marker pattern and the digital watermark code. The displayed content may be displayed.

  In this case, the content is stored in the database 7 in association with the combination of the AR marker pattern and the digital watermark code, and after the digital watermark code is recognized in step 102 in FIG. 3, the AR marker recognition process in step 104 in FIG. After that, the search process of the database 7 in step 103 of FIG. 3 is performed to extract the content.

  As described above, the content display device 1 according to the embodiment of the present invention uses the watermarked marker 5 combining the AR marker and the digital watermark to print different digital watermarks with the unified AR marker. This makes it possible to display a variety of different contents. In addition, it is possible to distinguish different colors of AR markers and minute differences in patterns by printing different digital watermarks and display different contents.

  The preferred embodiments of the content display device 1 have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood.

1 ……… Content display device 2 ………… AR marker 3 ……… Camera-equipped terminal 4 ……… AR display content 5 ……… Marker with digital watermark 7 ……… Database 9 ……… Display content

Claims (7)

A content display device that displays information by reading information from a marker printed with a marker image and a digital watermark superimposed,
Storage means for storing the electronic watermark code and the content in association with each other;
Photographing means for photographing the marker;
Code extraction means for identifying the digital watermark of the photographed marker and extracting the code;
Content acquisition means for acquiring the content corresponding to the extracted code;
A position recognition means for recognizing the position of the marker image of the photographed marker;
Display means for displaying the acquired content at the recognized position;
A content display device comprising: The content display apparatus according to claim 1, wherein the marker includes the same marker image printed with the different digital watermarks superimposed. The content according to claim 1, wherein the marker is a marker image of the same design and a different color, and the digital watermark that is different for each color is superimposed and printed. Display device. The content is a three-dimensional image,
The position recognition means further recognizes a change from the reference coordinate system of the marker image,
The content display device according to any one of claims 1 to 3, wherein the display unit deforms and displays the three-dimensional image according to the change at the recognized position. A content display method performed by a computer having storage means for reading information from a marker printed by superimposing a marker image and a digital watermark and displaying the content and linking and storing the code of the digital watermark and the content ,
A shooting step of shooting the marker;
A code extracting step of identifying a digital watermark of the photographed marker and extracting a code;
A content acquisition step of acquiring the content corresponding to the extracted code;
A position recognition step for recognizing the position of the marker image of the photographed marker;
A display step of displaying the acquired content at the recognized position;
A content display method comprising: A program for causing a computer to read information from a marker printed with a marker image and a digital watermark superimposed thereon and to function as a content display device that displays content,
Computer
Storage means for storing the electronic watermark code and the content in association with each other;
Photographing means for photographing the marker;
Code extraction means for identifying a digital watermark of the photographed marker and extracting a code;
Content acquisition means for acquiring the content corresponding to the extracted code;
Position recognition means for recognizing the position of the marker image of the marker taken;
Display means for displaying the acquired content at the recognized position;
Program to function as. A content display system in which information is read from a marker printed with a marker image and a digital watermark superimposed and displayed, and a database server is connected to the content server via a network,
The database server is
A storage means for storing the electronic watermark code and the content in association with each other;
The content display device includes:
Photographing means for photographing the marker;
Code extraction means for identifying the digital watermark of the photographed marker and extracting the code;
Content acquisition means for acquiring the content corresponding to the extracted code from the database server;
A position recognition means for recognizing the position of the marker image of the photographed marker;
Display means for displaying the acquired content at the recognized position;
A content display system comprising:

Images