JP2014225753A - Marker detection device, marker detection method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform inverse projection transformation to a projection-transformed image and video on the basis of markers embedded into the image and video without impairing the design of contents.SOLUTION: A marker detection device includes an image input unit that receives the input of one or a plurality of consecutive images, and a marker detection unit that detects four points including a predetermined contrast pattern in a color component in which an image is difficult to be perceived, and when a compound ratio calculated from the detected four points is a predetermined value, determines that markers having a contrast pattern common to four places are located on a straight line having a concentric circular fringe pattern and passing through the center of the concentric circle.

Description

  The present invention relates to a marker detection apparatus and marker detection that detect a marker embedded in a still image or a moving image to detect a digital watermark or the like even when projective transformation is performed on the still image or the moving image. The present invention relates to a method and a program.

  In the distribution of content such as images and videos, other information is embedded in the content so that it is not perceived by the human eye for purposes such as content identification and management, content copyright protection and management, and related information provision. A method using a digital watermark technique is known. In particular, in the digital watermark technology for image / video content, a method based on minute change of the pixel value is common. As a specific method, for example, when embedding a digital watermark, based on embedding target component position information, spread spectrum is generated by independently changing the real and imaginary components of the complex matrix, and a watermark pattern is generated independently of the input image. When an embedded image is generated by adding an actual image pattern and a digital watermark is detected, a detection target sequence is generated based on detection target component position information, and offset information is extracted and detected. There is a digital watermarking method that performs spectral despreading after correcting a target sequence and detects a digital watermark embedded in a cut-out pixel block (for example, Patent Document 1).

  In addition, in digital watermarking, it is necessary that a digital signal representing digital content is resistant to various modifications received after embedding the digital watermark, that is, the digital watermark can be detected after being subjected to various modifications. . An important modification that should be particularly resistant to digital watermarking is geometric transformation. Resistance to geometric transformation is an indispensable requirement when a content such as a target image or video is re-photographed with a camera and a digital watermark embedded in the content is detected. This is because geometric transformation occurs in the content according to the difference in the distance and angle at which the content is shot.

  As a method of providing resistance to geometric transformation, the edge of the image in which the digital watermark is embedded is detected, and the positions of the four corners of the image are specified to perform reverse projection transformation, and the state before the transformation is determined. There is a method of restoring (Patent Document 2). With this method, a digital watermark can be detected from an image accompanied by deformation by projective transformation, but in order to stably detect the edge of the image, it is necessary to surround the image with a black frame or a white frame. Since this is perceived by human eyes, there is a problem that the design of the content itself is greatly impaired.

  On the other hand, when a digital watermark is embedded as a spatial repetitive pattern and is detected, a method of correcting using the autocorrelation of the pattern (Patent Document 3), a marker is embedded in a frequency space of an image invisible to humans, There is a method (Patent Document 4 and Patent Document 5) for correction using this. These are not perceived by the human eye, and can detect digital watermarks from images with geometric transformation, but can support affine transformations represented by rotation, scale transformation, etc., and support projection transformation There is a problem that cannot be done.

JP 2003-219148 A Japanese Patent No. 4020093 Japanese translation of PCT publication No. 2003-510931 Japanese Patent No. 3949679 JP 2010-232886 A

  As described above, in the inventions and techniques described in Patent Documents 1 to 5, the original image or video is obtained by reverse projection transformation on the image or video that has undergone projection transformation without impairing the design of the content. There is a problem that can not be converted to.

  The present invention has been made to solve the above problems, and its purpose is to detect markers for obtaining information for performing reverse projection transformation from markers embedded in images and videos without impairing the design of the content. An apparatus, a marker detection method, and a program are provided.

  According to an aspect of the present invention, an image input unit that inputs one or a plurality of continuous images and four points where a predetermined light and shade pattern exists in a color component that is difficult to perceive are detected and detected 4 When the cross ratio calculated from one point is a predetermined value, it is determined that four common markers having the above-mentioned grayscale pattern are located on a straight line having a concentric striped pattern and passing through the center of the concentric circle. A marker detection unit, wherein the marker detection unit is a first box filter defined in accordance with the shading pattern of the marker and using a plurality of first box filters, A raster scan is sequentially performed on each pixel of the color component of the image, a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and the raster scan is detected among the detected pixels. For each marker detected by the marker position candidate detection unit, a marker position candidate detection unit that determines whether or not the marker is positioned based on a cross-ratio calculated based on a positional relationship between four pixels continuous in the direction of A second box filter that is defined in accordance with the shading pattern of the marker and is different from the first box filter, and has a plurality of second box filters as a starting point. Scanning is performed in a direction perpendicular to the direction of the raster scan, pixels corresponding to the maximum value or minimum value in the vicinity of each pixel are detected, and among the detected pixels, from the positional relationship of four pixels close to the center point Based on the calculated cross ratio, a marker position candidate redetermining unit for redetermining the detection of the marker, the first box filter, and the second box Using a sfilter, circle detection is performed based on a pixel corresponding to the maximum value or minimum value in the vicinity of each pixel obtained by scanning the color component of the image, and based on the obtained circular region, A marker detection device comprising: a marker position candidate limiting unit that narrows down the markers detected by the marker position candidate detection unit.

  According to another aspect of the present invention, an image input unit that inputs one or a plurality of continuous images and four points where a predetermined light and shade pattern exists in a color component that is difficult to perceive are detected and detected. When the cross ratio calculated from the four points is a predetermined value, the marker having the concentric striped pattern and the four common shade patterns located on the straight line passing through the center of the concentric circle is located. A marker detection unit for determining, using the first box filter of a plurality of sizes, which is a first box filter defined according to the shading pattern of the marker, A raster scan is sequentially performed on each pixel of the color component of the image, a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and the raster scan is detected among the detected pixels. A marker position candidate detection unit that determines whether or not the marker is positioned based on a multi-ratio calculated based on a positional relationship between four pixels that are continuous in the direction of the scan, and the marker detected by the marker position candidate detection unit A center point of the marker using a second box filter having a plurality of sizes, which is a second box filter different from the first box filter, which is determined according to the shading pattern of the marker. The pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected in a direction perpendicular to the raster scan direction starting from the position, and the positions of four pixels near the center point among the detected pixels A marker position candidate re-determination unit that re-determines the detection of the marker based on the cross ratio calculated from the relationship. That.

  According to another aspect of the present invention, an image input unit that inputs one or a plurality of continuous images and four points where a predetermined light and shade pattern exists in a color component that is difficult to perceive are detected and detected. When the cross ratio calculated from the four points is a predetermined value, the marker having the concentric striped pattern and the four common shade patterns located on the straight line passing through the center of the concentric circle is located. A marker detection unit for determining, using the first box filter of a plurality of sizes, which is a first box filter defined according to the shading pattern of the marker, A raster scan is sequentially performed on each pixel of the color component of the image, a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and the raster scan is detected among the detected pixels. A marker position candidate detection unit that determines whether or not the marker is positioned based on a multi-ratio calculated based on a positional relationship between four pixels that are continuous in the direction of the channel, the first box filter, and the marker The color component of the image is scanned using a second box filter that is determined according to the shading pattern and is different from the first box filter and has a plurality of sizes. Marker position candidate limitation that performs circle detection based on the pixel corresponding to the maximum value or minimum value in the vicinity of each obtained pixel, and narrows down the marker detected by the marker position candidate detection unit based on the obtained circular area It is a marker detection apparatus characterized by having a part.

In addition, according to one aspect of the present invention, in the above-described marker detection device, a projective transformation correction unit that performs a projective transformation on the image based on the positions of the plurality of markers detected in the image, and the projective transformation correction And an information detecting unit for detecting information related to the image from the partial area specified by the detected marker in the image subjected to projective transformation by the unit.
One embodiment of the present invention is characterized in that the marker detection apparatus further includes an information output unit that outputs a position of the marker detected in the image.
Further, according to one aspect of the present invention, in the above-described marker detection device, the marker detection device further includes an integrated image calculation unit that calculates an integrated image in the color component of the image, and the marker detection unit is based on the integrated image, The position of the marker in the image is detected.

  Further, according to one aspect of the present invention, an image input step of inputting one or a plurality of continuous images and four points where a predetermined light and shade pattern exists in a color component that is difficult to perceive are detected and detected. When the cross ratio calculated from the four points is a predetermined value, the marker having the concentric striped pattern and the four common shade patterns located on the straight line passing through the center of the concentric circle is located. A marker detecting step for determining, wherein the marker detecting step uses a first box filter having a plurality of sizes, which is a first box filter determined according to the shading pattern of the marker. Then, raster scanning is sequentially performed on each pixel of the color component of the image to detect a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel, and the detected image is detected. Among the marker position candidate detection step for determining whether or not the marker is positioned based on the cross ratio calculated based on the positional relationship between the four pixels continuous in the raster scan direction, and the marker position candidate detection step. For each of the markers, a second box filter which is determined according to the shading pattern of the marker and is different from the first box filter and has a plurality of sizes is used. Scanning is performed in a direction perpendicular to the direction of the raster scan starting from the center point, and a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and among the detected pixels, four pixels close to the center point are detected. Based on the cross ratio calculated from the pixel positional relationship, the marker position candidate redetermination step for redetermining the detection of the marker. And a circle based on a pixel corresponding to a maximum value or a minimum value in the vicinity of each pixel obtained by scanning the color component of the image using the first box filter and the second box filter. And a marker position candidate limiting step for narrowing down the marker detected in the marker position candidate detection step based on the obtained circular region.

  Further, according to one aspect of the present invention, an image input step of inputting one or a plurality of continuous images and four points where a predetermined light and shade pattern exists in a color component that is difficult to perceive are detected and detected. When the cross ratio calculated from the four points is a predetermined value, the marker having the concentric striped pattern and the four common shade patterns located on the straight line passing through the center of the concentric circle is located. A marker detecting step for determining, wherein the marker detecting step uses a first box filter having a plurality of sizes, which is a first box filter determined according to the shading pattern of the marker. Then, raster scanning is sequentially performed on each pixel of the color component of the image to detect a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel, and the detected image is detected. Among the marker position candidate detection step for determining whether or not the marker is positioned based on the cross ratio calculated based on the positional relationship between the four pixels continuous in the raster scan direction, and the marker position candidate detection step. For each of the markers, a second box filter which is determined according to the shading pattern of the marker and is different from the first box filter and has a plurality of sizes is used. Scanning is performed in a direction perpendicular to the direction of the raster scan starting from the center point, and a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and among the detected pixels, four pixels close to the center point are detected. Based on the cross ratio calculated from the pixel positional relationship, the marker position candidate redetermination step for redetermining the detection of the marker. A marker detecting method characterized by comprising and.

  Further, according to one aspect of the present invention, an image input step of inputting one or a plurality of continuous images and four points where a predetermined light and shade pattern exists in a color component that is difficult to perceive are detected and detected. When the cross ratio calculated from the four points is a predetermined value, the marker having the concentric striped pattern and the four common shade patterns located on the straight line passing through the center of the concentric circle is located. A marker detecting step for determining, wherein the marker detecting step uses a first box filter having a plurality of sizes, which is a first box filter determined according to the shading pattern of the marker. Then, raster scanning is sequentially performed on each pixel of the color component of the image to detect a pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel, and the detected image is detected. A marker position candidate detection step for determining whether or not the marker is positioned by a cross ratio calculated based on a positional relationship between four pixels that are continuous in the raster scan direction, and the first box filter, The color component of the image is determined by using a second box filter that is determined according to the shading pattern of the marker and is different from the first box filter and has a plurality of sizes. A marker that performs circle detection based on a pixel corresponding to the maximum value or minimum value in the vicinity of each pixel obtained by scanning, and narrows down the marker detected in the marker position candidate detection step based on the obtained circular region It is a marker detection method characterized by having a position candidate limitation step.

Further, according to one aspect of the present invention, in the marker detection method described above, a projective transformation correction step for performing a projective transformation on the image based on the positions of the plurality of markers detected in the image, and the projective transformation correction An information detecting step of detecting information related to the image from the partial area specified by the detected marker in the image subjected to projective transformation in the step is further characterized.
One embodiment of the present invention is characterized in that the above-described marker detection method further includes an information output step of outputting a position of the marker detected in the image.
Further, one aspect of the present invention further includes an integrated image calculation step of calculating an integrated image in the color component of the image in the marker detection method described above, and in the marker detection step, based on the integrated image, The position of the marker in the image is detected.

  One embodiment of the present invention is a program for causing a computer to execute each step in the marker detection method described above.

  According to the present invention, in a color component that is difficult to perceive in an image, a cross ratio that is not affected by projective transformation is calculated from four points detected by the shading pattern of the marker, whereby the position of the marker in the image is calculated. Can be detected. By detecting a plurality of markers, it is possible to acquire information for performing reverse projection transformation on the projection transformation performed on the image.

It is a figure which shows an example of the marker used in this embodiment. It is a figure which shows an example of the area | region which embeds a marker in an image in this embodiment. It is a figure which shows an example of the Box filter used for the scanning of a horizontal direction in this embodiment. It is a figure which shows an example of the Box filter used for the scanning of a vertical direction in this embodiment. It is a figure which shows an example of the scanning of the vertical direction performed in this embodiment. It is the schematic which shows an example of the circle detection performed in this embodiment. It is a block diagram which shows the structural example of the marker embedding apparatus 1 in this embodiment. It is a figure which shows an example of the characteristic shading pattern which the marker in this embodiment has. It is a flowchart which shows the marker embedding process which the marker embedding apparatus 1 in this embodiment performs. It is a block diagram which shows the structural example of the marker detection apparatus 2 in this embodiment. It is a figure which shows the outline | summary of the Box filter which is a Box filter which the marker position candidate detection part 221 in this embodiment uses, and has a different magnitude | size. It is a flowchart which shows the marker detection process which the marker detection apparatus 2 in this embodiment performs. It is a flowchart which shows the marker detection process of step S22 in the marker detection process shown in FIG.

  Hereinafter, a marker detection device, a marker detection method, and a program according to an embodiment of the present invention will be described with reference to the drawings.

  The marker embedding device and the marker detection device according to the present embodiment detect the conversion performed on the original image or video when converting the image or video on which the projective transformation has been performed into the original image or video. A concentric shading pattern as shown in FIG. 1 is used. FIG. 1 is a diagram illustrating an example of a marker used in the present embodiment. As shown in the figure, the marker is formed of a concentric striped pattern, and −1, +2 from the center point of the concentric circle toward the outer circumference with reference to the color difference (or luminance, etc.) of the circle at the center of the marker. , -2, +2, and -1 are equally spaced shading patterns having color differences according to the ratio. The marker embedding device superimposes markers at four corners of a predetermined area on an image or video. The marker embedding device converts the image or video from the RGB color space to the YCbCr (or YPbPr) color space when embedding the marker in the image or video, or the Cb component that is most difficult to be perceived by humans in terms of visual characteristics, or its Next, a marker is embedded in the Cr component that is difficult to perceive. In the RGB color space, a marker may be embedded in a B component that is difficult for humans to perceive among RGB components. In addition, when an image or video is expressed in another color space, a marker is placed on the color component that is most difficult to be perceived by humans or the color component that is next difficult to be perceived. It may be embedded. Hereinafter, a case where a marker is embedded in the Cb component will be described.

  FIG. 2 is a diagram illustrating an example of a region in which a marker is embedded in an image in the present embodiment. In the figure, an example is shown in which markers are embedded in the four corners of a predetermined rectangular area having the same size as the image. Thus, even if the image in which the markers are embedded in the four corners is subjected to projective transformation, each marker is detected, and by performing reverse projective transformation that transforms the area that can be grasped by the position of each marker into a predetermined area, Can be restored. In the example shown in FIG. 2, markers are embedded in the four corners of a predetermined area having the same size as the image, and areas other than the area where the marker is embedded in the image are areas where the digital watermark is embedded.

  In the detection of the digital watermark, the marker detection device detects all the embedded markers from the Cb component of the photographed image obtained by photographing the image in which the marker is embedded with a camera. The marker detection device performs restoration to convert the captured image into the shape and size of the image by performing reverse projection conversion on the captured image based on the position of the marker in the captured image. By performing processing such as detection of a digital watermark from an image obtained by restoration, it is possible to obtain resistance of the digital watermark to geometric transformation such as projective transformation.

  Hereinafter, the marker used in this embodiment will be described. The marker must be able to detect the marker stably even if the shape of the marker is changed by geometric transformation such as projective transformation. Therefore, a cross ratio which is a projective transformation invariant is used. Specifically, as shown in FIG. 1, four points (point A, point B, point C, and point D) are defined on a straight line L passing through the center of the marker (the center point of a concentric circle). Four points (point A, point B, point C, and point D) are detected using a Box filter (box filter). Here, the position of each point on the marker is determined so that the four points (point A, point B, point C, and point D) satisfy the following expression (1).

In Formula (1), n is an arbitrary real number. When the expression (1) is satisfied, the cross ratio calculated from the positions of the four points (point A, point B, point C, and point D) is expressed by the following expression (2). Regardless of the value. For example, when n = 3, the cross ratio is ((3 × 5) / (4 ² ) = 15/16).

  Here, a Box filter for detecting a marker having a striped pattern as shown in FIG. 1 will be described. FIG. 3 is a diagram illustrating an example of a Box filter used for horizontal scanning in the present embodiment. In the figure, the horizontal direction is the X direction and the vertical direction is the Y direction. The Box filter shown in the figure is a 9 pixel × 9 pixel Box filter, and “0”, “−1”, and “+2” are weighting factors for each pixel (region) divided by hatching. It is. If the coordinates (x, y) of the pixel in the upper left corner are (0, 0), ((x, y); 1 ≦ x ≦ 9, 1 ≦ y ≦ 3) and ((x, y); 1 ≦ For pixels with x ≦ 9, 7 ≦ y ≦ 9), the Cb component is weighted by “0”. In the pixel of ((x, y); 1 ≦ x ≦ 3, 4 ≦ y ≦ 6) and ((x, y); 7 ≦ x ≦ 9, 4 ≦ y ≦ 6), the Cb component Is weighted by “−1”. In the pixel of ((x, y); 4 ≦ x ≦ 6, 4 ≦ y ≦ 6), the Cb component is weighted by “+2”.

When a marker is searched by raster scanning for each pixel of the image using this Box filter, for example, a stripe of “+2” including a point A on the marker is ((x, y); 4 ≦ x ≦ 6, 4 ≦ y ≦ 6), and the fringe of “−1” on the outer peripheral side in contact with the stripe including the point A on the marker is ((x, y); 1 ≦ x ≦ 3, 4 ≦ y ≦ 6) Is located at ((x, y); 7 ≦ x ≦ 9, 4 ≦ y ≦ 6). The value calculated using the Box filter is larger than the surrounding pixels. When detecting a marker, a point (pixel) where the calculated value is larger than the surroundings is detected, and a cross ratio is calculated from four points adjacent to the raster scan direction among the detected points (pixels). calculate.
When the calculated cross ratio matches the marker cross ratio or the difference from the marker cross ratio is within a predetermined range, the four points corresponding to the calculated cross ratio are four points (point A on the marker). , Point B, point C, and point D), the marker is detected.

  The extreme value of the value calculated sequentially based on the Cb component (the largest value or the smallest value in the vicinity) that is obtained sequentially by raster scanning the captured image using the Box filter as shown in FIG. ) Is detected. Points (pixels) corresponding to the detected extreme values and four extreme values continuous in the scanning direction are defined as a point A ′, a point B ′, a point C ′, and a point D ′. Then, the cross ratio is calculated by the following equation (3) from four points (point A ′, point B ′, point C ′, and point D ′) corresponding to the detected extreme value.

  When the cross ratio calculated using the equation (3) is equal to the cross ratio in the marker embedded in the image in advance, the four points (point A ′, point B ′, point C ′, and point D ′) are This is a point on a straight line passing through the center of the marker and corresponding to a characteristic shading pattern possessed by the marker. In this way, candidate points of the four feature points (point A, point B, point C, and point D) on the marker are detected from the photographed image using the Box filter, and the cross ratio of the detected candidate points is If it is equal to the cross ratio, the candidate point is determined to be four feature points (point A, point B, point C, and point D) on the marker. This makes it possible to detect a marker embedded in the original image in the captured image and specify its position.

  Further, starting from the position of the marker detected by the raster scan using the Box filter, scanning is performed in the direction (vertical direction) perpendicular to the direction of the raster scan, that is, the vertical direction, using the Box filter shown in FIG. . FIG. 4 is a diagram illustrating an example of a Box filter used for vertical scanning in the present embodiment. In the same figure, as in FIG. 3, the horizontal direction is the X direction and the vertical direction is the Y direction. The Box filter shown in FIG. 4 is a 9 × 9 Box filter, and “0”, “−1”, and “+2” are weighting factors for each pixel (region) divided by hatching. It is. As shown in the figure, if the coordinates (x, y) of the pixel at the upper left corner are (0, 0), ((x, y); 1 ≦ x ≦ 3, 1 ≦ y ≦ 9) and (( x, y); 7 ≦ x ≦ 9, 1 ≦ y ≦ 9), the Cb component is weighted by “0”. Further, in the pixels of ((x, y); 4 ≦ x ≦ 6, 1 ≦ y ≦ 3) and ((x, y); 4 ≦ x ≦ 6, 7 ≦ y ≦ 9), the Cb component And multiply by “−1”. In the pixel of ((x, y); 4 ≦ x ≦ 6, 4 ≦ y ≦ 6), the Cb component is weighted by “+2”.

  Using this Box filter, a scan that is performed upward from the center of the detected marker and a scan that is performed downward from the vertical direction is performed. FIG. 5 is a diagram illustrating an example of vertical scanning performed in the present embodiment. As shown in the figure, it is a value obtained by scanning the captured image in the vertical direction of upward and downward with the center point of the marker detected using the Box filter shown in FIG. The extreme value (the largest value or the smallest value in the vicinity) of the value calculated based on the above is detected. Then, points (pixels) corresponding to the detected extreme values are set as a point A ″, a point B ″, a point C ″, and a point D ″. The cross ratio is calculated by the following equation (4) from four points (point A ″, point B ″, point C ″, and point D ″) corresponding to the detected extreme value.

  When the cross ratio calculated using the equation (4) is equal to the cross ratio in the marker embedded in the image in advance, four points (point A ″, point B ″, point C ″, and point D) '') Can be determined to be a point on a straight line passing through the center of the marker and corresponding to the characteristic shading pattern of the marker, and the position of the marker detected by the raster scan ( Re-determination (verification) can be performed on the center point. In this re-determination, when the cross ratio calculated from the four points (point A ″, point B ″, point C ″, and point D ″) does not match the value calculated by equation (2) The position of the marker detected by the raster scan is excluded as being erroneously detected. In addition, the center point calculated from the four points (point A ″, point B ″, point C ″, and point D ″) is the position that is the base point of the vertical scan (in the horizontal scan) Even if it does not match the detected center point), it is excluded as a false detection.

  Further, circle detection is performed based on pixel positions (coordinates) corresponding to extreme values detected by horizontal and vertical scans using a Box filter. At this time, if the marker detected using the Box filter protrudes from the circle obtained by the circle detection, the marker is excluded as being erroneously detected. The marker may not maintain a circular shape when the image is projectively transformed, but it is possible to detect a circle that includes the marker by performing circle detection using the Hough transform. It is. Utilizing this, by performing a screening process on a marker detected by a scan using a Box filter, it is possible to eliminate the erroneously detected marker and improve the marker detection accuracy. An improvement in the accuracy of detecting a marker leads to an improvement in the accuracy of specifying the digital watermark embedding region and the reverse projection transformation, and thus the digital watermark reading performance can be improved.

  FIG. 6 is a schematic diagram illustrating an example of circle detection performed in the present embodiment. As shown in the figure, circle detection is performed based on the position of the extreme value detected using the Box filter, and it is determined whether or not the detected marker is located in the area inside the detected circle. Thus, a screening process is performed.

Hereinafter, specific configuration examples of the marker embedding device and the marker detection device in the present embodiment will be described.
FIG. 7 is a block diagram illustrating a configuration example of the marker embedding device 1 according to the present embodiment. The marker embedding device 1 embeds markers having the above-described features at four corners of a predetermined rectangular area on image content or video content for input image content (still image) or video content (moving image). . Furthermore, the marker embedding device 1 embeds and outputs a predetermined digital watermark in a rectangular area in which markers are arranged at four corners. The marker embedding device 1 includes an image input unit 11, a marker superimposing unit 12, a digital watermark superimposing unit 13, and an embedded image output unit 14 as shown in FIG.

  The image input unit 11 inputs image content or video content prepared in advance. When video content is input, each frame image continuous in time series of video is input one by one. The input image content or video content is input as digital data.

  The marker superimposing unit 12 superimposes markers with appropriate sizes on the four corners of a predetermined rectangular area with respect to each frame image of the image or video input to the image input unit 11. The marker to be superimposed is a marker having the shading pattern shown in FIG. Further, the size of the marker may be a predetermined size or may be determined according to the size of the digital watermark superimposed on the rectangular area.

  The marker embedded in the image by the marker superimposing unit 12 is a concentric pattern, and there are two configurations in which four characteristic grayscale patterns common to any straight line passing through the center of the concentric pattern exist. have. Regarding the latter configuration, for example, in the marker shown in FIG. 1, there is a change in shading as shown in FIG. 8 around point A, point B, point C, and point D.

  FIG. 8 is a diagram illustrating an example of a characteristic shading pattern possessed by the marker in the present embodiment. In the figure, the horizontal axis corresponds to a straight line passing through the center of the marker, and the vertical axis represents the light and shade when the center of the marker is used as a reference on the straight line. The marker shown in FIG. 1 has four characteristic shade patterns (point A, point B, point C, and point D) shown in FIG. For example, the point A is (−1, +2, −2), the point B is (−2, +2, −1), and the lightness is positive (+) with respect to the point (or region) of interest. Is given, and negative (-) shades are given to points (or regions) adjacent to the point (or region) of interest. The marker shown in FIG. 1 is an example, and may be a different shape from the marker shown in FIG. 1 as long as it is a marker composed of the light and shade pattern having the two configurations described above.

  Specifically, the marker superimposing unit 12 converts the input image from the RGB color space signal to the YCbCr color space signal as a superposition method for superimposing the marker on the image. The marker superimposing unit 12 superimposes markers at four corners of a predetermined rectangular area on the image, and adds the gray value of the marker to the Cb component at the position (pixel) where the marker is superimposed. When the marker shown in FIG. 1 is used, the numerical value shown in FIG. 1 is a value added to the Cb component. Here, addition is performed when the value is a positive value (plus), and subtraction is performed when the value is a negative value (minus). In addition, in order to increase the intensity of light and shade by marker superimposition, a fixed value a set in advance as a parameter may be multiplied as a coefficient to the light and dark value of the marker and then added to the Cb component of the image. That is, a value obtained by multiplying the gray value of the marker by a fixed value a may be added to the Cb component of the image.

  In addition, as a geometric transformation for image content and video content, when rotation of ± 45 degrees or more is applied, in order to ensure that the correspondence of the markers at the four corners can be taken correctly when detecting the markers, Invert the sign of one of the markers and add it. Hereinafter, as in the marker superposition example shown in FIG. 2, a case will be described in which superpositions obtained by reversing the sign of the lower right marker are applied.

  The digital watermark superimposing unit 13 superimposes the digital watermark information in the rectangular area indicated by the four markers superimposed on the image by the marker superimposing unit 12. The superimposition method follows a digital watermark method to be superimposed. Various schemes such as the scheme disclosed in Patent Document 1 have been proposed for the digital watermark scheme, and any scheme may be employed. In addition, the digital watermark superimposed by the digital watermark superimposing unit 13 may not be resistant to geometric transformation itself.

  The embedded image output unit 14 outputs the image processed by the marker superimposing unit 12 and the digital watermark superimposing unit 13 as an “embedded image” in which the marker and the digital watermark are superimposed. When video content is input to the image input unit 11, the embedded image is output as a single video content by connecting it to a series of video sequences as a frame. The output destination of the embedded image output unit 14 is, for example, an HDD, an electronic recording medium such as a DVD-ROM, or a display device. When the output destination is an electronic recording medium, the embedded image is recorded on the electronic recording medium, and when the output destination is a display device, the embedded image is displayed on the display device.

FIG. 9 is a flowchart showing marker embedding processing performed by the marker embedding device 1 according to the present embodiment. When the marker embedding device 1 starts the marker embedding process, an image as a target for embedding the marker is input to the image input unit 11 (step S11).
The marker superimposing unit 12 superimposes four markers on the four corners of a predetermined rectangular area with respect to the Cb component of the image input to the image input unit 11, and displays an image in which the four markers are superimposed on the digital watermark superimposing unit 13. Output (step S12).

The digital watermark superimposing unit 13 superimposes the predetermined digital watermark information in a predetermined rectangular area on the image input from the marker superimposing unit 12, that is, a rectangular area in which markers are arranged at the four corners. The superimposed image is output to the embedded image output unit 14 (step S13).
The embedded image output unit 14 outputs the image input from the digital watermark superimposing unit 13 to an external device (step S14).

  The embedded image output unit 14 determines whether or not the processing has been completed for all the images (step S15), and when the processing has been completed for all the images (step S15: YES). The marker embedding process is terminated. On the other hand, if the processing has not been completed for all the images (step S15: NO), the processing is returned to step S11, and the processing from step S11 to step S15 is repeated.

  As described above, in the marker embedding device 1, the image content or video content input to the image input unit 11 undergoes a series of processing in the marker superimposing unit 12, the digital watermark superimposing unit 13, and the embedded image output unit 14. As a result, markers are superimposed on the four corners of a predetermined rectangular area in the image content or video content, and a digital watermark is superimposed on the rectangular area and output.

  FIG. 10 is a block diagram illustrating a configuration example of the marker detection device 2 in the present embodiment. The marker detection device 2 determines whether four predetermined markers are superimposed on the input image content or video content. When four markers are detected, the marker detection device 2 detects a digital watermark superimposed on the image content or video content based on the markers, and outputs information indicated by the detected digital watermark. The marker detection device 2 includes an image input unit 21, a marker detection unit 22, a projective transformation correction unit 23, a digital watermark detection unit 24, and a detection result output unit 25, as shown in FIG. The marker detection unit 22 includes a marker position candidate detection unit 221, a marker position candidate redetermination unit 222, and a marker position candidate limitation unit 223.

  The image input unit 21 processes image data or video content in which a digital watermark to be detected is captured by a camera or the like, and image content or video content in which the digital watermark to be detected is embedded (for example, , Projective transformation) is input. When the embedded digital watermark requires a plurality of images that are continuous in time series at the time of detection, the required number of images are input to the image input unit 21 in time series in a time series. The

  The marker detection unit 22 identifies the position where the marker is superimposed on the image input to the image input unit 21. A case where the marker embedded in the image is the marker shown in FIG. 1 will be described. In the marker detection unit 22, the input image is converted from a signal in the RGB color space to a signal in the YCbCr color space, and a marker is detected for the Cb component of the image.

  The marker position candidate detection unit 221 performs raster scanning from the upper left to the lower right by repeatedly scanning the Cb component image in the horizontal direction for each of a plurality of box filters having a plurality of sizes prepared in advance. , Using a Box filter. Here, a case will be described in which three different sized Box filters are prepared in advance as shown in FIG.

  FIG. 11 is a diagram illustrating an outline of a Box filter used by the marker position candidate detection unit 221 in the present embodiment and having different sizes. As the Box filter, for example, a Box filter of 3 pixels × 3 pixels, 5 pixels × 5 pixels, 7 pixels × 7 pixels, and 15 pixels × 15 pixels is prepared in advance. Note that the size of each Box filter and the number of Box filters may be determined in advance according to the target content and the size of the marker superimposed on the content. The marker position candidate detection unit 221 calculates a value for each pixel in a raster scan using each Box filter.

  Here, on the three-dimensional scale space composed of the scan direction, the direction perpendicular to the scan direction, and the size direction of the Box filter, an area adjacent to each direction is defined as 26 neighborhoods. The marker position candidate detection unit 221 compares the calculated value of the current scan position (pixel) with the calculated value in the vicinity of 26, and sets the calculated value of the scan position as a maximum value when the calculated value of the scan position becomes the maximum value. To detect. The marker position candidate detection unit 221 uses the point corresponding to the four local maximum values in the scanning direction as points A ′, B ′, C ′, and D ′ to calculate the cross ratio using Equation (3). calculate. The marker position candidate detection unit 221 compares the cross ratio calculated using Expression (3) with the cross ratio calculated using Expression (2) defined for the marker. It determines with existing in the position defined by two points (point A ', point B', point C ', and point D'), and memorize | stores the center point of a marker. The marker position candidate detection unit 221 performs processing for detecting the center point of the marker using the Box filter on the entire image. Note that the local maximum value may be detected by comparing the calculated values in the vicinity of 8 on the two-dimensional scale space formed by either the scanning direction or the direction perpendicular to the scanning direction and the size direction of the Box filter.

  In addition, the marker position candidate detection unit 221 performs the following for the marker corresponding to the lower right of the rectangular area. Note that the marker corresponding to the lower right corner of the rectangular area has a tone pattern opposite to that of the other three markers, so the value calculated by the Box filter described above is near the feature point of the marker. Smaller than this pixel. The marker position candidate detection unit 221 compares the calculated value of the current scan position (pixel) with the calculated value in the vicinity of 26, and sets the calculated value of the scan position as a minimum value when the calculated value of the scan position is minimized. To detect. The marker position candidate detection unit 221 uses Expression (3) as points A ′, B ′, C ′, and D ′ as points corresponding to four minimum values (minimum values) continuous in the scan direction. To calculate the cross ratio. The marker position candidate detection unit 221 compares the cross ratio calculated using Expression (3) with the cross ratio calculated using Expression (2) defined for the marker. It determines with existing in the position defined by two points (point A ', point B', point C ', and point D'), and memorize | stores the center point of a marker.

  The marker position candidate detection unit 221 is determined for the marker and the cross ratio calculated based on the detected four points (point A ′, point B ′, point C ′, and point D ′). If the difference from the cross ratio calculated by Expression (2) does not match and is within a predetermined range, it may be determined that a marker exists at a position defined by the detected four points. Through the above processing, the marker position candidate detection unit 221 detects a marker position candidate that is a candidate for the position where the marker is superimposed in the image input to the marker detection device 2. At this time, the marker position candidate detection unit 221 stores the detected marker position candidates as candidates even if there are four or more detected marker position candidates, and outputs them to the marker position candidate re-determination unit 222.

  The marker position candidate re-determination unit 222 determines whether the marker position candidate detected by the marker position candidate detection unit 221 is valid. Specifically, the marker position candidate re-determination unit 222 performs a vertical scan starting from the point indicated by the marker position candidate, so that the point (feature point) corresponding to the characteristic shading pattern of the marker is detected. Perform detection. At this time, the marker position candidate re-determination unit 222 performs vertical scanning using, for example, a Box filter as illustrated in FIG. Further, the marker position candidate re-determination unit 222 compares the calculated value of the current scan position (pixel) with the calculated value in the vicinity of 8 using a plurality of sized box filters in the same manner as the marker position candidate detection unit 221. When the calculated value of the scan position becomes the maximum value, the calculated value of the scan position is detected as the maximum value.

  That is, as shown in FIG. 5, the marker position candidate re-determination unit 222 uses the point indicated by the marker position candidate as the starting point of the scanning in the vertical direction, and selects points corresponding to two local maximum values close to the starting point. Detect for each of the upward and downward directions. In FIG. 5, two points that are close to the starting point in the upward direction are points A ″ and B ″, and two points that are close to the starting point in the downward direction are point C ″ and point D ″. It is. The marker position candidate re-determination unit 222 uses the equation (4) to calculate the compound from four points (point A ″, point B ″, point C ″, and point D ″) detected by the vertical scanning. Calculate the ratio. For the marker position candidate corresponding to the marker obtained by inverting the marker shown in FIG. 1, the marker position candidate redetermining unit 222 detects the point corresponding to the minimum value and performs the same processing.

  The marker position candidate re-determination unit 222 compares the calculated multi-ratio with the multi-ratio calculated by Expression (2) in which the calculated multi-ratio is determined for the marker. The marker position candidate re-determination unit 222 determines that the marker position candidate is valid when the two cross ratios are equal. If the two cross ratios are not equal, the marker position candidate re-determination unit 222 excludes the marker position candidate as erroneously detected. At this time, the marker position candidate re-determination unit 222 may determine that the marker position candidate is valid when the difference between the two cross ratios is within a predetermined range.

  The marker position candidate redetermining unit 222 calculates the center point of the marker based on the positions of the four points detected by the vertical scan, and determines whether the calculated center point matches the marker position candidate. You may make it determine. At this time, the marker position candidate re-determination unit 222 determines that the marker position candidate is valid when the two points match or the distance between the two points is within a predetermined range. Thus, the marker position candidate re-determination unit 222 performs a re-determination using the vertical cross ratio and the position of the center point on the marker position candidate, so that the marker position candidate detection unit 221 is erroneously detected. Marker position candidates can be excluded.

  The marker position candidate limiting unit 223 converts each of the Box filter for detecting the change in the horizontal direction as shown in FIG. 3 and the Box filter for detecting the change in the vertical direction as shown in FIG. Is used to detect the point between the maximum and minimum values. That is, the marker position candidate limiting unit 223 detects a feature point corresponding to a characteristic shading pattern of the marker from the Cb component of the image. At this time, the marker position candidate limiting unit 223 scans the Cb component of the image in multiple scales using a plurality of size Box filters, similarly to the marker position candidate detection unit 221 and the marker position candidate re-determination unit 222.

The marker position candidate limiting unit 223 obtains a union of detected points and performs circle detection as shown in FIG. At this time, like the marker position candidate detection unit 221 and the marker position candidate re-determination unit 222, the marker position candidate limiting unit 223 performs extreme values by scanning in the horizontal direction and the vertical direction using a plurality of size Box filters. The maximum or minimum value in the vicinity is calculated. There are various known methods for circle detection. For example, Hough transform described in Reference 1 may be used.
[Reference 1]: Supervised by the Digital Image Processing Editorial Committee, “Digital Image Processing”, Association for Promotion of Image Information Education (CG-ARTS Association), pages 211-214, July 2004

  The marker position candidate limiting unit 223 limits the marker detection area based on the circle obtained by the circle detection. Specifically, the marker position candidate limiting unit 223 leaves the marker position candidates located in the area inside the obtained circle and excludes the marker position candidates located in the area outside the obtained circle. As a result, the marker position candidate limiting unit 223 performs screening on the marker position candidates detected by the marker position candidate detection unit 221 by scanning in the horizontal direction. Alternatively, the marker position candidate limiting unit 223 performs screening on marker position candidates remaining as a result of redetermination by the marker position candidate redetermining unit 222. As described above, the marker position candidate limiting unit 223 performs screening on the marker position candidates based on the circle detection, so that the detection accuracy of the marker position candidates can be increased.

  The marker position candidate limiting unit 223 determines whether the three marker positions other than the marker position where the grayscale pattern is inverted correspond to the upper left, upper right, or lower left of the rectangular area. Correlation is performed based on the positional relationship between the center of gravity and the lower right marker (inverted marker). When there are five or more detected marker positions, the marker position candidate limiting unit 223 selects three marker positions having a larger maximum value and selects one marker position having a smaller minimum value. By selecting, four markers are specified.

  Through the above processing, the marker detection unit 22 can specify all the positions of the markers indicating the four corners of the rectangular area in which the digital watermark is embedded. In addition, even if the embedded marker has a shape other than that shown in FIG. 1, by using a filter that can extract the “common characteristic grayscale pattern” that is the marker configuration in this embodiment. Similarly, the position of the marker can be detected. When the marker detection unit 22 can detect only three or less marker positions, the marker detection unit 22 determines that the digital watermark is not embedded or the marker detection has failed, and does not perform the subsequent processing.

  The projection conversion correction unit 23 performs reverse projection conversion on the input image based on the positions of the four markers detected by the marker detection unit 22. At this time, the projective transformation correcting unit 23 performs the reverse projective transformation so that the positions of the four markers are the positions of the four corners of the rectangular area in which the marker embedding device 1 has embedded the four markers. At this time, the projective transformation correction unit 23 performs reverse projective transformation by applying the same technique as that described in Patent Document 2, for example.

  The digital watermark detection unit 24 is an electronic device in which the digital watermark superimposing unit 13 of the marker embedding device 1 superimposes the image on the image from a rectangular area where the markers on the image obtained by the reverse projection conversion by the projective conversion correction unit 23 are arranged at the four corners. Detect watermarks. At this time, when the embedded digital watermark requires a plurality of images in time series, the marker detection unit 22 and the projective transformation correction unit 23 repeatedly perform processing for the required number of images, A digital watermark is detected after obtaining the required number of images corrected by reverse projection transformation. The digital watermark detection unit 24 outputs information indicated by the detected digital watermark to the detection result output unit 25.

  The detection result output unit 25 outputs information output from the digital watermark detection unit 24, that is, digital watermark information indicated by the digital watermark, to the outside as digital data.

FIG. 12 is a flowchart showing marker detection processing performed by the marker detection device 2 according to this embodiment. When the marker detection process is started in the marker detection device 2, an image (or each frame image of a video) as a target for detecting the marker and digital watermark information is input to the image input unit 21 (step S21).
The marker detection unit 22 performs marker detection processing for detecting the center point of the marker using a Box filter in the Cb component image of the image input to the image input unit 21 (step S22).

The marker detection unit 22 determines whether or not the number of detected markers is three or less (step S23). If the number of detected markers is three or less (step S23: YES), the marker processing is terminated.
On the other hand, when the detected number of markers is not three or less (step S23: NO), the projective transformation correction unit 23 detects the input image based on the positions of the four markers detected by the marker detection unit 22. Reverse projection transformation is performed to match the partial area indicated by the marked marker with a rectangular area having a predetermined size (step S24).

The digital watermark detection unit 24 detects the digital watermark in the rectangular area indicated by the marker on the image obtained by the reverse projection conversion by the projection conversion correction unit 23 (step S25).
The digital watermark detection unit 24 determines whether the digital watermark information indicated by the detected digital watermark is correct (step S26). If the digital watermark information is correct (step S26: YES), the digital watermark information is output ( Step S27), the marker detection process is terminated.
On the other hand, if the digital watermark information is not correct, the digital watermark detection unit 24 returns the process to step S21 and repeats the process from step S21 to step S26.

  FIG. 13 is a flowchart showing the marker detection process of step S22 in the marker detection process shown in FIG. When the marker detection process (step S22) is started in the marker detection unit 22, the marker position candidate detection unit 221 detects Boxes having a plurality of sizes in order to detect points corresponding to the characteristic shading pattern of the marker. A raster scan using a filter is performed on the Cb component of the image. The marker position candidate detection unit 221 detects, from the detected points, a set in which the cross ratio obtained from the positional relationship between the four points continuous in the scan direction satisfies Expression (2), and is estimated from the four points Is set as a marker position candidate (step S221).

  For each marker position candidate detected by the marker position candidate detection unit 221, the marker position candidate re-determination unit 222 is directed upward from the point indicated by the marker position candidate as a vertical scan using a plurality of size Box filters. Do each downwards. The marker position candidate redetermining unit 222 calculates the cross ratio and the center point of the marker based on the four nearest feature points detected by the vertical scanning. The marker position candidate re-determination unit 222 performs re-determination on the marker position candidates corresponding to the cross ratio and the center point based on the calculated cross ratio and center point, and narrows down the marker position candidates (step S222).

  The marker position candidate limiting unit 223 uses the horizontal Box filter (for example, FIG. 3) and the vertical Box filter (for example, FIG. 4) to detect points corresponding to the characteristic grayscale pattern of the marker, and Cb of the image. The component is scanned in multiscale to detect points corresponding to extreme values. The marker position candidate limiting unit 223 performs circle detection based on the detected points. The marker position candidate limiting unit 223 narrows down the marker position candidates by extracting the marker position candidates located inside the detected circle (step S223).

  Among the marker position candidates detected by the marker position candidate detection unit 221 in the marker detection unit 22, a marker whose center point is the marker position candidate narrowed down by the marker position candidate re-determination unit 222 and the marker position candidate limitation unit 223. Are treated as markers detected from the image. Accordingly, even if the position where the marker is not embedded may be erroneously detected as the position where the marker is embedded, depending on the shading included in the image or video to be detected, the marker position candidate re-determination unit Screening by 222 and the marker position candidate limiting unit 223 can exclude erroneously detected positions and improve the accuracy of marker detection.

  As described above, in the marker detection device 2, the image content or the video content input to the image input unit 21 is subjected to reverse projection conversion based on the four markers detected by the marker detection unit 22, and the reverse projection conversion is performed. A digital watermark is detected from the processed image. In this way, when the projective transformation is performed on the image content or the video content on which the digital watermark that is not resistant to the projection transformation is superimposed by performing the reverse projection transformation based on the marker having the tolerance to the projection transformation. In this case, a digital watermark can be detected.

By using the marker embedding device 1 and the marker detection device 2 described above, a marker capable of detecting a cross ratio is embedded in a Cb component that is most difficult to be perceived by humans in visual characteristics, and the marker is detected by a Box filter. It becomes possible.
By defining four common characteristic grayscale patterns on a straight line passing through the center of the marker, the characteristic grayscale patterns can be detected by filtering. Also, the cross ratio determined by the four points on the straight line passing through the center of the marker is the same as the cross ratio calculated from the four points in the marker after the projective transformation, even if the marker is deformed by the projective transformation. To do. By detecting the marker using this cross ratio, it is possible to detect the marker from the image subjected to the projective transformation.

In the filter processing, since the raster scan using the Box filter corresponding to the characteristic gray pattern of the marker is performed, the calculation is performed when the characteristic gray pattern area overlaps with the Box filter area. The value obtained is larger than the neighboring value. Thereby, specification of the position of a marker can be made easy. By performing reverse projection conversion based on the identified marker position, it is possible to restore the image content and video content embedded with information such as digital watermarks to the state before geometric conversion.
As a result, reverse projection conversion is applied to images that have undergone projective transformation that occurs when image content or video content embedded with information such as digital watermarks is captured by a camera, etc. Information can be acquired.

In the present embodiment, as shown in FIG. 2, as an example, a marker is superimposed on the four corners of an image to be embedded with a digital watermark, and almost all areas of the image are set as areas on which the digital watermark is superimposed. Indicated. However, the present invention is not limited to this, and a specific partial region in the target image may be used as a region where a digital watermark is superimposed. In this case, markers may be superimposed on the four corners of the partial area. For example, assuming that two persons appear in the target image, it is possible to embed two different types of digital watermarks in partial areas corresponding to the faces of the respective persons. In this way, information corresponding to each person can be detected from a single image content. Furthermore, the user may be guided to other content (for example, a Web information site) based on the detected information.
When embedding a plurality of pieces of information such as a digital watermark, the marker detection unit 22 detects a plurality of five or more markers. In this case, a combination of markers in which regions defined by the four markers do not overlap is sequentially selected. Thus, a plurality of areas may be detected.

  In the present embodiment, the configuration in which the marker detection device 2 detects a digital watermark after reverse projection conversion has been described. However, information other than the digital watermark may be detected. For example, in a similar application using image recognition (for example, an application that provides information for accessing a Web information site related to an object photographed by a camera) in addition to the purpose of specifying an embedded area of a digital watermark, It can be used as preprocessing for image recognition. In this case, the marker detection device 2 may include an information output unit that outputs information indicating the position in the image of the partial region specified based on the detected marker or the position of the marker. However, the object including the area where the information is embedded is a planar object, and is limited to the case where the marker can be embedded in time by printing or the like. Of course, when such use is assumed, the digital watermark superimposing unit 13 in the marker embedding device 1 and the projective transformation correcting unit 23 and the digital watermark detecting unit 24 in the marker detecting device 2 are unnecessary, and the detection result output unit 25 is Alternatively, the position information of the marker on the embedded image may be output. The recognition module can improve recognition accuracy by cutting out an object and correcting geometric transformation based on the output marker position information. Further, the recognition accuracy may be improved by calculating the relative spatial positional relationship between the object and the camera based on the marker position information and using it together with the recognition result.

  In addition, the case where the area specified by the four markers is a rectangular area has been described as an example. However, the area may be other than the rectangular area, and information such as a digital watermark may be superimposed on the area specified by the four markers. It may be. In this case, the shape and size of the area specified by the four markers are determined in advance, and the marker superimposing unit 12 and the projective transformation correcting unit 23 perform processing based on the determined shape and size. Further, although the case where the region is specified by four markers has been described as an example, information such as a digital watermark may be superimposed on a polygonal region that can be specified by four or more markers.

  In this embodiment, the case where the cross ratios of the four markers that specify the partial areas corresponding to the rectangular areas are the same has been described. However, the cross ratios of the four markers may be different. In this case, even when a rotation of ± 45 degrees or more is applied, the original position of each marker can be grasped, so that it is not necessary to invert the density of one marker as shown in FIG.

  Also, in the raster scan using the Box filter described in the above embodiment, by calculating an integral image of the target image before performing the raster scan, the amount of calculation by filtering is greatly reduced. (Reference document 2: Yasushi Yagi, Hideo Saito, “Computer Vision Advanced Guide 2”, Adcom Media Corporation, p. 46, June 2010). Specifically, an integrated image calculation unit that calculates a total value of Cb components for each region multiplied by each weighting coefficient from the integrated image is provided between the image input unit 21 and the marker detection unit 22. The marker detection unit 22 calculates the sum of values obtained by multiplying the total value of the Cb components of each region obtained from the integrated image by a weighting coefficient. As a result, the amount of computation that increases in accordance with the size of the Box filter can be suppressed, and the amount of computation required for marker detection processing can be reduced.

  In the above embodiment, the marker position candidate redetermining unit 222 and the marker position candidate limiting unit 223 narrow down the marker position candidates detected by the marker position candidate detecting unit 221. However, the marker position candidate redetermining is described. The marker position candidates may be narrowed down by either the unit 222 or the marker position candidate limiting unit 223. The order of narrowing down by the marker position candidate redetermining unit 222 and the marker position candidate limiting unit 223 may not be the order shown in the flowchart of FIG. For example, after performing narrowing by circle detection in the marker position candidate limiting unit 223, narrowing by scanning in the vertical direction in the marker position candidate re-determination unit 222 may be performed.

  Further, in the process in which the marker position candidate limiting unit 223 detects the feature points using the Box filter, the result of the horizontal box filter processing by the marker position candidate detection unit 221 and the vertical direction by the marker position candidate re-determination unit 222 are detected. You may make it divert the result of a Box filter process. Thereby, the marker detection accuracy can be maintained while reducing the amount of calculation in the marker position candidate limiting unit 223.

  You may make it implement | achieve the marker embedding apparatus 1 (FIG. 7) and the marker detection apparatus 2 (FIG. 10) in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  The present invention can also be applied to a use in which reverse projection transformation for returning an image obtained by performing projection transformation to an original image is indispensable.

DESCRIPTION OF SYMBOLS 1 ... Marker embedding apparatus 2 ... Marker detection apparatus 11 ... Image input part 12 ... Marker superimposition part 13 ... Digital watermark superimposition part 14 ... Embedded image output part 21 ... Image input part 22 ... Marker detection part 23 ... Projection transformation correction part 24 ... Digital watermark detector (information detector)
25 ... Detection result output unit 221 ... Marker position candidate detection unit 222 ... Marker position candidate re-determination unit 223 ... Marker position candidate limitation unit

Claims (13)

An image input unit for inputting one or more continuous images;
When four points where a predetermined light and shade pattern exists in the color component that is difficult to perceive in the image are detected and the cross ratio calculated from the detected four points is a predetermined value, a concentric striped pattern And a marker detection unit that determines that markers having four common shade patterns are located on a straight line passing through the center of the concentric circles,
With
The marker detection unit
A raster scan is sequentially performed on each pixel of the color component of the image using a first box filter having a plurality of sizes, which is a first box filter defined according to the shading pattern of the marker. The pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and the cross ratio calculated based on the positional relationship of the four pixels consecutive in the raster scan direction among the detected pixels is used. A marker position candidate detection unit for determining whether or not the marker is located;
For each of the markers detected by the marker position candidate detection unit, a second box filter that is determined in accordance with the shading pattern of the marker and is different from the first box filter and has a plurality of sizes. Using a box filter, scan in a direction perpendicular to the raster scan direction starting from the center point of the marker, detect pixels corresponding to the maximum value or minimum value in the vicinity of each pixel, and detect the detected pixel A marker position candidate redetermining unit for redetermining the detection of the marker based on the cross ratio calculated from the positional relationship of the four pixels close to the center point.
Using the first box filter and the second box filter, circle detection is performed based on a pixel corresponding to a maximum value or a minimum value in the vicinity of each pixel obtained by scanning the color component of the image. And a marker position candidate limiting unit that narrows down the markers detected by the marker position candidate detection unit based on the obtained circular region. An image input unit for inputting one or more continuous images;
When four points where a predetermined light and shade pattern exists in the color component that is difficult to perceive in the image are detected and the cross ratio calculated from the detected four points is a predetermined value, a concentric striped pattern A marker detection unit that determines that four markers having the common shading pattern are located on a straight line passing through the center of the concentric circles,
The marker detection unit
A raster scan is sequentially performed on each pixel of the color component of the image using a first box filter having a plurality of sizes, which is a first box filter defined according to the shading pattern of the marker. The pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and the cross ratio calculated based on the positional relationship of the four pixels consecutive in the raster scan direction among the detected pixels is used. A marker position candidate detection unit for determining whether or not the marker is located;
For each of the markers detected by the marker position candidate detection unit, a second box filter that is determined in accordance with the shading pattern of the marker and is different from the first box filter and has a plurality of sizes. Using a box filter, scan in a direction perpendicular to the raster scan direction starting from the center point of the marker, detect pixels corresponding to the maximum value or minimum value in the vicinity of each pixel, and detect the detected pixel A marker position candidate re-determination unit that re-determines the detection of the marker based on a cross ratio calculated from the positional relationship of four pixels close to the center point. An image input unit for inputting one or more continuous images;
When four points where a predetermined light and shade pattern exists in the color component that is difficult to perceive in the image are detected and the cross ratio calculated from the detected four points is a predetermined value, a concentric striped pattern A marker detection unit that determines that four markers having the common shading pattern are located on a straight line passing through the center of the concentric circles,
The marker detection unit
A raster scan is sequentially performed on each pixel of the color component of the image using a first box filter having a plurality of sizes, which is a first box filter defined according to the shading pattern of the marker. The pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and the cross ratio calculated based on the positional relationship of the four pixels consecutive in the raster scan direction among the detected pixels is used. A marker position candidate detection unit for determining whether or not the marker is located;
Using the first box filter and a second box filter having a plurality of sizes, which is a second box filter that is determined according to the shading pattern of the marker and is different from the first box filter The circle position is detected based on the pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel obtained by scanning the color component of the image, and the marker position candidate detection unit based on the obtained circle area A marker position candidate limiting unit that narrows down the markers detected by the marker detection device. In the marker detection apparatus according to any one of claims 1 to 3,
A projective transformation correction unit that performs projective transformation on the image based on the positions of the plurality of markers detected in the image;
A marker detection apparatus, further comprising: an information detection unit that detects information related to the image from a partial region specified by the detected marker in the image subjected to the projection conversion by the projection conversion correction unit. In the marker detection apparatus according to any one of claims 1 to 3,
The marker detection apparatus further comprising: an information output unit that outputs the position of the marker detected in the image. In the marker detection apparatus according to any one of claims 1 to 5,
Further comprising an integral image calculation unit for calculating an integral image in the color component of the image;
The marker detection unit
A marker detection device that detects a position of the marker in the image based on the integrated image. An image input step for inputting one or more continuous images;
When four points where a predetermined light and shade pattern exists in the color component that is difficult to perceive in the image are detected and the cross ratio calculated from the detected four points is a predetermined value, a concentric striped pattern And a marker detection step for determining that markers having four common shade patterns are located on a straight line passing through the center of the concentric circles,
Have
The marker detection step includes
A raster scan is sequentially performed on each pixel of the color component of the image using a first box filter having a plurality of sizes, which is a first box filter defined according to the shading pattern of the marker. The pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and the cross ratio calculated based on the positional relationship of the four pixels consecutive in the raster scan direction among the detected pixels is used. A marker position candidate detection step for determining whether or not the marker is located; and
For each of the markers detected in the marker position candidate detection step, a second box filter that is determined in accordance with the shading pattern of the marker and is different from the first box filter and has a plurality of sizes. Using a box filter, scan in a direction perpendicular to the raster scan direction starting from the center point of the marker, detect pixels corresponding to the maximum value or minimum value in the vicinity of each pixel, and detect the detected pixel A marker position candidate redetermination step for redetermining the detection of the marker based on the cross ratio calculated from the positional relationship of the four pixels close to the center point,
Using the first box filter and the second box filter, circle detection is performed based on a pixel corresponding to a maximum value or a minimum value in the vicinity of each pixel obtained by scanning the color component of the image. And a marker position candidate limiting step for narrowing down the marker detected in the marker position candidate detection step based on the obtained circular region. An image input step for inputting one or more continuous images;
When four points where a predetermined light and shade pattern exists in the color component that is difficult to perceive in the image are detected and the cross ratio calculated from the detected four points is a predetermined value, a concentric striped pattern And a marker detection step for determining that markers having four common shade patterns are located on a straight line passing through the center of the concentric circles,
Have
The marker detection step includes
A raster scan is sequentially performed on each pixel of the color component of the image using a first box filter having a plurality of sizes, which is a first box filter defined according to the shading pattern of the marker. The pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and the cross ratio calculated based on the positional relationship of the four pixels consecutive in the raster scan direction among the detected pixels is used. A marker position candidate detection step for determining whether or not the marker is located; and
For each of the markers detected in the marker position candidate detection step, a second box filter that is determined in accordance with the shading pattern of the marker and is different from the first box filter and has a plurality of sizes. Using a box filter, scan in a direction perpendicular to the raster scan direction starting from the center point of the marker, detect pixels corresponding to the maximum value or minimum value in the vicinity of each pixel, and detect the detected pixel A marker position candidate re-determination step for re-determining the detection of the marker based on a cross ratio calculated from the positional relationship of four pixels close to the center point. An image input step for inputting one or more continuous images;
When four points where a predetermined light and shade pattern exists in the color component that is difficult to perceive in the image are detected and the cross ratio calculated from the detected four points is a predetermined value, a concentric striped pattern And a marker detection step for determining that markers having four common shade patterns are located on a straight line passing through the center of the concentric circles,
Have
The marker detection step includes
A raster scan is sequentially performed on each pixel of the color component of the image using a first box filter having a plurality of sizes, which is a first box filter defined according to the shading pattern of the marker. The pixel corresponding to the maximum value or the minimum value in the vicinity of each pixel is detected, and the cross ratio calculated based on the positional relationship of the four pixels consecutive in the raster scan direction among the detected pixels is used. A marker position candidate detection step for determining whether or not the marker is located; and
Using the first box filter and a second box filter having a plurality of sizes, which is a second box filter that is determined according to the shading pattern of the marker and is different from the first box filter Detecting a circle based on a pixel corresponding to a maximum value or a minimum value in the vicinity of each pixel obtained by scanning the color component of the image, and detecting the marker position candidate based on the obtained circle region And a marker position candidate limiting step for narrowing down the marker detected in step 1. A marker detection method comprising: In the marker detection method according to any one of claims 7 to 9,
A projective transformation correction step for performing a projective transformation on the image based on the positions of the plurality of markers detected in the image;
A marker detection method, further comprising: an information detection step of detecting information related to the image from a partial region specified by the detected marker in the image subjected to the projective transformation in the projective transformation correction step. In the marker detection method according to any one of claims 7 to 9,
A marker detection method, further comprising: an information output step of outputting the position of the marker detected in the image. In the marker detection method according to any one of claims 7 to 11,
An integrated image calculating step of calculating an integrated image in the color component of the image;
In the marker detection step,
A marker detection method, comprising: detecting a position of the marker in the image based on the integrated image.   The program for making a computer perform each step in the marker detection method as described in any one of Claims 7-12.

Images