WO2016063570A1

WO2016063570A1 - Image capturing and extraction device

Info

Publication number: WO2016063570A1
Application number: PCT/JP2015/067543
Authority: WO
Inventors: 将彦真野; 元喜紙谷; 隆行小尾; リッチウエストウッド
Original assignee: 株式会社日立ソリューションズ
Priority date: 2014-10-20
Filing date: 2015-06-18
Publication date: 2016-04-28
Also published as: JP2016082469A

Abstract

Provided is a device capable of clarifying in which image a digital watermark is embedded, and guiding an image capturing direction thereof. A user captures an image of printed matter 12 including a picture (object) 16 in which target data is embedded with a digital watermark by an image capturing unit 2. The image captured by the image capturing unit 2 is displayed on a display unit 8 by a captured image display means 4. A feature image means 6 superimposes and displays a feature figure of the picture in which the target data is embedded on the display unit 8. Consequently, the user can easily know in which picture the target data is embedded, and further capture an image in a right direction, with a right size, and at a right position by aligning the captured picture with the feature figure superimposed and displayed on the captured image. A target data extraction means 10 extracts the target data embedded as the digital watermark from the picture thus captured.

Description

Imaging extraction device

The present invention relates to an apparatus for extracting target data by imaging a target object in which target data is embedded by digital watermarking or the like.

An apparatus is used that reads a target object in which target data is embedded, such as a digital watermark or a QR code (trademark), as an image and extracts the target data. For example, a smartphone with a camera is used as such a device. The code reading application extracts the target data by grasping the QR code (trademark) or the like at an appropriate position and size while checking the image being picked up on the display.

At this time, even if the image such as the Q code is too large or too small, it cannot be read, and there is a problem that correct reading cannot be performed unless it is within a predetermined position on the screen (normally near the center). However, it is not always easy to image the QR code (trademark) at an appropriate position and size.

In order to solve such a problem, for example, in Patent Document 1, as shown in FIGS. 12A and 12B, the marker 102 and the marker 104 are displayed so as to overlap the image being captured, thereby making it easy to perform alignment. Has been disclosed.

JP-A-2005-250554

Even if the QR code (trademark) is imaged from a direction different from the predetermined direction (for example, even when the image is rotated 90 degrees), the QR code (trademark) itself has a mark indicating the direction. Therefore, the code can be read accurately by the image rotation process.

Also, the QR code (trademark) that is to be read can be clearly distinguished from other pictures and characters from its unique design, so it is clear which location should be read.

However, in an image in which target data is embedded with a digital watermark, it is difficult to visually determine in which image the target data is embedded. For this reason, there is a problem that it is not determined whether there is target data unless such new images are taken one after another.

Also, in the case of digital watermarking, unlike the QR code (trademark), a mark indicating the direction is not attached to the code itself, so there is a problem that the image must be taken from the correct direction.

An object of the present invention is to solve the above-described problems, to clarify which image has an electronic watermark embedded therein, and to provide an apparatus capable of guiding the imaging direction.

Some features that can be independently employed in the imaging extraction apparatus or imaging apparatus according to the present invention are shown below.

(1) (2) An imaging extraction device according to the present invention is an imaging extraction device for extracting target data by imaging a target in which target data is embedded by digital watermarking, and imaging the target An image capturing unit for capturing the image, a captured image display unit for displaying a captured image captured by the image capturing unit on the display unit, and a feature image of the image of the target object or the vicinity of the target object superimposed on the captured image. Characteristic image display means, and target data extraction means for extracting target data from the captured image of the object, wherein the feature image is not rotationally symmetric.

Therefore, it is possible to easily determine in which image the target data is embedded, and it becomes easy to correctly capture the image including its direction at the time of imaging.

(3) (4) An imaging device according to the present invention is an imaging device for imaging an object in which target data is embedded by digital watermarking, and an imaging unit for imaging the object, and an imaging unit Captured image display means for displaying the captured image captured by the display unit on the display unit, and feature image display means for displaying the feature image of the image of the object or the vicinity of the object superimposed on the captured image, The feature image is not rotationally symmetric.

(5) The device according to the present invention is characterized in that the feature image further includes at least an element that is not rotationally symmetric of the object.

Therefore, it is possible to make the direction during imaging more correct.

(6) The device according to the present invention is characterized in that the feature image is one of the following. 1) Outline of the object, 2) When the object is a picture of an animal, the outline of the eyes, nose or mouth or a combination thereof, 3) Characters, figures or figures displayed superimposed on the picture of the object Symbol or a combination of these 4) A picture that is the same as the object and has transparency.

Therefore, it is possible to appropriately generate a feature image for aligning captured images.

(7) The apparatus according to the present invention is characterized in that the captured image display means enlarges, reduces, and rotates the captured image so that an object in the captured image matches the feature image.

Therefore, the captured image can be matched with the feature image without the user moving the imaging unit.

(8) The apparatus according to the present invention is characterized in that the feature image display means generates a feature image based on a captured image of the target object when the target data is embedded in the captured target object. Yes.

Therefore, the present invention can be carried out without recording a feature image in advance.

(9) (10) An imaging apparatus according to the present invention is an imaging apparatus for imaging an object, and displays an imaging unit for imaging the object and a captured image captured by the imaging unit An image pickup apparatus comprising: a picked-up image display means to be displayed on the display unit; and a feature image display means for displaying the feature image of the object superimposed on the picked-up image, wherein the feature image is not rotationally symmetric.

“Captured image display means” corresponds to step S1 in the embodiment.

“Feature image means” corresponds to step S3 in the embodiment.

In the embodiment, “object data extraction means” corresponds to step S4.

“The target data is embedded” means that the target data is embedded with a digital watermark or the like in a state that humans can detect or cannot detect.

“Program” is a concept that includes not only a program that can be directly executed by the CPU but also a source-format program, a compressed program, an encrypted program, and the like.

It is a functional block diagram of the imaging extraction apparatus by one Embodiment of this invention. It is a hardware configuration of an imaging extraction device. 5 is a flowchart of an imaging extraction program 46. It is an example of the picture book 50 on which the pattern 52 is printed. It is a figure which shows the characteristic image 64 displayed on the touch panel. It is a figure which shows the overlap of the captured image 66 and the characteristic image 64. FIG. It is an example of the displayed content data. It is an example of the table which matched and recorded content ID and content data. It is an example of the characteristic image 64 by another example. This is an example of another feature image 64. It is a figure which shows other embodiment. It is a figure which shows the guide for the conventional QR code (trademark).

1. Overall Configuration FIG. 1 shows a functional block diagram of an imaging extraction apparatus according to an embodiment of the present invention. The user images the printed matter 12 including the design (object) 16 in which the target data is embedded by digital watermarking by the imaging unit 2. The image captured by the imaging unit 2 is displayed on the display unit 8 by the captured image display unit 4.

At this time, the feature image means 6 displays the feature image of the pattern in which the target data is embedded on the display unit 8 (see the captured image 14). As a result, the user can easily know in which pattern the target data is embedded, and by matching the captured image to the feature image displayed superimposed on the captured image, the correct direction can be obtained. The image can be taken at the correct size, but at the position.

The target data extraction means 10 extracts target data embedded as a digital watermark from the picture thus captured.

2. Hardware Configuration FIG. 2 shows a hardware configuration of an imaging extraction apparatus according to an embodiment of the present invention. In this embodiment, a camera-equipped smartphone is used as an imaging extraction device. The CPU 30 is connected to a memory 32, a touch panel 34 (display having a touch input function), a nonvolatile memory 36, a communication circuit 38, and a camera 40. Note that hardware related to the telephone function is omitted.

The communication circuit 38 is a circuit for connecting to the Internet. An operating system 44 and an imaging extraction program 46 are recorded on the hard disk 36. The imaging extraction program 46 performs its function in cooperation with the operating system 44. These programs are downloaded via the Internet and installed on the hard disk 36.

3. Imaging Extraction Processing FIG. 3 shows a flowchart of the imaging extraction program 46. When the imaging extraction program 46 is activated, the CPU 30 displays a feature image recorded in advance on the hard disk 36 on the touch panel 34 (step S1). FIG. 5 shows an example of the feature image 64 displayed on the touch panel 34 of the smartphone 60. In this example, an image representing the feature of the smartphone image is displayed. In this embodiment, the feature image 64 is an image that is not rotationally symmetric. Thereby, when aligning a captured image, it can align correctly also including a direction.

The user images the picture 52 (target object) shown on the picture book page shown in FIG. 4 with the camera 40 of the smartphone 60. CPU30 acquires the imaged image (step S2). Further, the CPU 30 displays the captured image 66 and the feature image 64 in an overlapping manner (step S3). FIG. 6 shows an example in which the feature image 64 is superimposed on the captured image 66 and displayed. The display position of the feature image 64 on the touch panel 34 is fixed. The captured image 66 changes depending on the user's operation such as the direction and distance of the camera 40. The user changes the orientation and position of the camera so that the pattern 52 captured in the captured image 66 overlaps the feature image 64.

At this time, since the feature image 64 is displayed on the touch panel 34, it is possible to easily know which pattern should be imaged. Further, since the feature image 64 is not rotationally symmetric, it is possible to know the correct imaging direction. In addition, in this embodiment, the feature image 64 is represented by a line segment, and other parts are transparent (having transparency). Therefore, when the captured image 66 is superimposed on the feature image 64, the captured image 66 can be easily confirmed and operated.

The CPU 30 performs a process of extracting target data from the captured image 66 (step S4). For example, digital watermarking can be realized by a technique such as embedding target data in a high-frequency region (spatial frequency) that is difficult for human eyes to perceive.

Digital watermark generation methods include embedding watermark information that cannot be identified by the human eye in the noise part of the image, and embedding watermark information that cannot be identified by the human eye in parts where similar image components continue. For example, watermark information composed of noise components that cannot be identified by human eyes can be embedded in a part of the noise components. Alternatively, it is also possible to use a method of embedding watermark information that is difficult to be identified by the human eye by adding a small amount of change to the image in a certain pattern in a region having a high spatial frequency. The luminance information of the image can be used as an object to which a small amount of change is applied to the image.

If the pattern 52 is captured in a predetermined size and a predetermined direction in the captured image 66, the CPU 30 can extract the target data embedded in the pattern 52 as a digital watermark. on the other hand. If the captured image 66 does not include the pattern 52, or is included in the captured image 66, it is too small or tilted, and the CPU 30 cannot extract the target data.

If the target data cannot be extracted, the CPU 30 repeatedly executes step S1 and subsequent steps.

If the target data can be extracted, the CPU 30 accesses a predetermined server device (not shown) (step S6). At the time of this access, the extracted target data is transmitted to the server device. In this embodiment, the content ID is described as the target data.

In response to this, the server device reads the content data recorded in advance corresponding to the content ID and sends it back to the smartphone 60. FIG. 8 shows a table recorded in the server device. Content data is described corresponding to the content ID. If the content ID “00001” is transmitted from the smartphone 60, the content data “Ferris wheel is spinning” is returned.

The CPU 30 receives this content data and displays it on the touch panel 34 (step S7). Thus, the example of the content data displayed on the smart phone 60 is shown in FIG. When a mother or the like is reading a picture book for a child, the mother can see an explanation of the picture on the smartphone 60.

4). Other
(1) In the above embodiment, the content ID is described as the target data. However, the content data may be directly embedded as the target data. In this way, it is not necessary to communicate with the server device.

(2) In the above embodiment, characters are displayed on the smartphone 60 as shown in FIG. However, sound may be output from the smartphone 60. For example, an elephant's cry may be output as a voice corresponding to an elephant picture. In this case as well, as in (1) above, the content ID may be transmitted to the server device, and the corresponding audio data may be received by the smartphone 60, or the audio data itself may be embedded as the target data. Also good.

Also, the text data may be converted into voice on the smartphone 60 side and output.

(3) In the above embodiment, a picture book has been described as an example, but it can be applied to a general printed matter such as a normal book. Moreover, the present invention can be applied not only to printed matter but also to TV broadcast screens, moving images distributed on the Internet, still images, and the like.

(4) In the above embodiment, the content data is acquired based on the extracted target data and displayed. That is, the target data is used for acquiring content data. However, the extracted target data may be used to achieve the purpose of the other application by passing it to the other application.

(5) In the above embodiment, an image that is not rotationally symmetric is used as the feature image. This is to correctly capture the object (an image in which target data is embedded) including its direction. More specifically, it may not be sufficient that the feature image is not rotationally symmetric.

For example, in the above embodiment, it is assumed that a feature image 64 as shown in FIG. 9 is used. This feature image 64 is not rotationally symmetric. However, in the relationship with the pattern 52, there is a possibility that an image is captured in a state rotated by 180 degrees. This is because, in the pattern 52, the feature elements 68 of the feature image 64 exist symmetrically on the left and right. Considering such a case, not only the feature image 64 itself is not rotationally symmetric, but also the feature image 64 needs to include elements that are not rotationally symmetric in the pattern 52.

(6) In the above embodiment, a feature image 64 as shown in FIG. 5 is used. However, as long as it shows the characteristics of the pattern 52 (image that is the object), the one shown in FIG. 10 may be used. In addition, as in the above embodiment, the outer shape of the pattern 52 can be used as the feature image 64, but a feature image 64 that does not include the outer shape (for example, a line drawing representing a person's eyes, nose, or mouth) may be used. .

(7) In the above embodiment, the target data is embedded in the pattern 52 itself. However, the target data may be embedded in the image in the background without embedding the target data in the pattern 52 itself. For example, as shown in FIG. 11, there are few local features in the image in which the target data is embedded, and this is suitable when the target data is embedded in a part of the image.

The target data is embedded in the cloud image of the background part of the character 80 called CLOUD. Also, other target data is embedded in the cloud image of the background portion of the character 82 called CLOUD. In this case, the feature image 64 has the characters CLOUD. As a result, it is possible to appropriately image the portion in which the target data is embedded. Also in this case, the feature image 64 can be a character, a figure, a symbol, a picture, a photograph, or the like.

(8) In the above embodiment, the feature image 64 is configured by a diagram. However, instead of a diagram, the target image itself may be a transparent image and used as the feature image 64.

(7) In the above embodiment, the feature image 64 is fixed to one. However, a plurality of feature images may be prepared and selected and used by a swap operation or the like.

(8) In the above embodiment, the user moves the smartphone 60 so that the pattern 52 matches the feature image 64. However, if the CPU 30 recognizes the pattern 52 within the imaging range, the captured image 52 may be electronically rotated, enlarged, or reduced so that the pattern 52 matches the feature image 64.

(9) In the above embodiment, the feature image 64 is recorded in advance and used. However, if the target data is embedded in the captured image, the CPU 30 may extract the feature of the image and generate the feature image 64.

In this case, if the entire pattern 52 is watermarked, it is easy to recognize that the watermarked part is the pattern 52, and the feature image 64 can be easily generated. For example, it is possible to obtain a feature image by making the outline of the watermarked portion 52 a diagram. Further, if the pattern 52 can be specified, the characteristic image 64 may be obtained by making the pattern 52 transparent (increasing the transmittance of the image, that is, recognizing the superimposed image).

If the feature image 64 is automatically generated as described above, the feature image 64 can be freely changed. If it is a book or the like, the first page is provided with a page for generating the feature image 64 (a picture 52 that is easy to identify, such as only the picture 52 is displayed on a white background). The feature image 64 may be generated by capturing this.

(10) Although the smartphone 60 is used in the above embodiment, a digital camera or the like may be used as the imaging device or the imaging extraction device.

(11) The above embodiment has been described as the imaging extraction device. However, it can also be constructed as an imaging device that performs imaging and outputs imaging data to an extraction device or the like.

(12) In the above embodiment, the content data is acquired from the server device each time. However, if the correspondence table is downloaded to the smartphone 60 in advance, the content data can be acquired only by the smartphone 60 (communication) Can be used even in poor environments).

(13) In the above-described embodiment, the present invention is used as alignment for extracting embedded target data. However, the idea of the present invention can be used to determine the imaging position regardless of the presence or absence of target data.

Claims

An imaging and extracting device for imaging a target in which target data is embedded by digital watermarking and extracting the target data,
An imaging unit for imaging the object;
Captured image display means for displaying a captured image captured by the imaging unit on a display unit;
Feature image display means for displaying a feature image of an image of the object or the vicinity of the object superimposed on the captured image;
Objective data extraction means for extracting objective data from the captured image of the object;
With
An imaging extraction apparatus characterized in that the feature image is not rotationally symmetric.
An imaging extraction program for realizing, by a computer, an imaging extraction device for imaging a target in which objective data is embedded by digital watermarking and extracting the objective data, and the computer is imaged by an imaging unit Captured image display means for displaying a captured image on a display unit;
Feature image display means for displaying a feature image of an image of the object or the vicinity of the object superimposed on the captured image;
In an imaging extraction program for functioning as objective data extraction means for extracting objective data from a captured image of an object,
An imaging extraction program characterized in that the feature image is not rotationally symmetric.
An imaging device for imaging an object in which target data is embedded using a digital watermark,
An imaging unit for imaging the object;
Captured image display means for displaying a captured image captured by the imaging unit on a display unit;
Feature image display means for displaying a feature image of an image of the object or the vicinity of the object superimposed on the captured image;
With
An image pickup apparatus, wherein the feature image is not rotationally symmetric.
An imaging program for realizing an imaging device for imaging an object in which target data is embedded with a digital watermark by a computer, and causing the computer to display a captured image captured by the imaging unit on a display unit Image display means;
In an imaging program for causing a feature image of an image of the object or the vicinity of the object to function as a feature image display unit that displays the image superimposed on the captured image,
An imaging program, wherein the feature image is not rotationally symmetric.
The apparatus or program according to claims 1 to 4,
The feature image further includes at least an element that is not rotationally symmetric of the object.
The apparatus or program according to claims 1 to 5,
The characteristic image is one of the following, or an apparatus or program.
1) Outline of the object 2) If the object is a picture of an animal, the outline of the eyes, nose or mouth or a combination of these 3) Characters, figures or symbols displayed on the object picture Combination of these 4) The same picture as the object, but a transparent picture.
The apparatus or program according to any one of claims 1 to 6,
The captured image display means is configured so that an object in the captured image matches the feature image.
An apparatus or program for enlarging, reducing, or rotating a captured image.
The apparatus or program according to any one of claims 1 to 7,
The feature image display means generates a feature image based on a captured image of a target object if the target data is embedded in the captured target object.
An imaging device for imaging an object,
An imaging unit for imaging the object;
Captured image display means for displaying a captured image captured by the imaging unit on a display unit;
A feature image display means for displaying the feature image of the object superimposed on the captured image;
With
An imaging extraction apparatus characterized in that the feature image is not rotationally symmetric.
An imaging program for realizing an imaging apparatus for imaging an object by a computer, and a captured image display means for displaying a captured image captured by the imaging unit on the display unit on the computer,
In an imaging program for causing a feature image of the object to function as a feature image display unit that displays the feature image on the captured image,
An imaging program, wherein the feature image is not rotationally symmetric.