JP2018005263A - Image processing method, program, photographic paper, and image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform information processing of image data by using a symbol that does not become an eyesore.SOLUTION: An image processing method includes: a step for acquiring a photographic image I by photographing a photographing object O with photographic paper P as a background in a state where the photographing object O arranged on the photographic paper P with a plurality of fine patterns printed in a prescribed area; a step for using the plurality of fine patterns to correct distortion of the photographic image I; and a step for extracting an image of the photographing object O by using the plurality of fine patterns to segment the contour of the photographing object O from a photographic image I' whose distortion corrected.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing method for performing a predetermined process on a photographed image obtained by photographing a photographing object, a program for executing the image processing method, a photographing sheet used when photographing a photographed image, and The present invention relates to an image processing apparatus for the captured image.

2. Description of the Related Art Conventionally, it is known that an image including an object and / or drawing or the like is taken with a shooting sheet on which an object and / or drawing or the like is placed as a background, and various types of information processing are performed on the image. Yes. For example, Patent Document 1 discloses image data obtained by photographing a photographing object arranged on a photographing sheet having a positioning symbol formed by a symbol code composed of a multi-valued graphic at at least two predetermined positions. It is disclosed that information processing is executed.
In this information processing, a positioning symbol is detected from the obtained image data, and processing based on the positioning symbol is executed.

JP 2012-68746 A

In the information processing based on the positioning symbols described above, the region defined by the positioning symbols can be extracted from the image data, but the process of extracting only each photographing target included in the image data cannot be performed.
The positioning symbol is printed on a photographing sheet or the like with a size that can be visually recognized by the user. This is because the user needs to adjust the position of the camera while checking the positioning symbols appearing on the camera so that the camera can correctly recognize the positioning symbols when shooting the shooting target with a camera or the like. . For this reason, when the user uses the notebook, a large positioning symbol enters the field of view and becomes annoying.

  An object of the present invention is to perform image information processing using symbols that do not obstruct the use.

An image processing method according to an aspect of the present invention includes the following steps.
A step of obtaining a photographed image by photographing a photographing object with the photographing paper as a background in a state where the photographing object is arranged on the photographing paper on which a plurality of fine patterns are printed in a predetermined area.
A step of correcting distortion of a photographed image using a plurality of fine patterns.
A step of extracting an image to be imaged by cutting out the outline of the image to be imaged from the image that has been corrected for distortion using a plurality of fine patterns.

  In this image processing method, a plurality of fine patterns are printed on a shooting sheet on which a shooting target is placed when shooting a shot image. Since the plurality of fine patterns are fine and difficult to visually recognize, the plurality of fine patterns do not become annoying to the user when an object to be photographed is arranged.

The plurality of fine patterns may include a first pattern with a first color and a second pattern with a second color. The second color is a color different from the first color. In this case, the high frequency pattern may be formed on the photographing paper by alternately arranging the first pattern and the second pattern at a predetermined cycle.
Accordingly, it is possible to accurately execute the distortion correction of the captured image and the process of extracting the captured object from the captured image using the high-frequency pattern in which the first color and the second color appear alternately in a predetermined cycle.

  In the step of correcting distortion, the distortion of the photographed image may be corrected by matching the high-frequency pattern in the photographed image with the high-frequency pattern printed on the photographed paper. Thereby, distortion of the captured image can be corrected by simpler calculation.

  In the step of correcting the distortion, the distortion of the captured image may be corrected using a plurality of fine patterns existing in the vicinity of the imaging target. Thereby, the calculation amount of the information processing performed with respect to a picked-up image can be reduced.

  A program according to another aspect of the present invention is a program for causing a computer to execute the above-described image processing method.

  A photographing sheet according to still another aspect of the present invention is used for photographing a photographing target. The photographic paper is used for correcting the distortion of the photographic image obtained by photographing the photographic object placed on the photographic paper, and extracting the photographic object image by cutting out the outline of the photographic object from the photographic image after the distortion correction. A plurality of fine patterns are arranged.

  When a plurality of fine patterns on a photographing sheet used for photographing a photographing target are made fine so that the plurality of fine patterns are difficult to visually recognize and the photographing target is arranged, the plurality of fine patterns Can be made unobtrusive to the user.

The plurality of fine patterns may include a first pattern with a first color and a second pattern with a second color different from the first color. At this time, a high frequency pattern is formed by alternately arranging the first pattern and the second pattern at a predetermined cycle.
Accordingly, it is possible to accurately execute the distortion correction of the captured image and the process of extracting the captured object from the captured image using the high-frequency pattern in which the first color and the second color appear alternately in a predetermined cycle.

The high-frequency pattern may be a rectangular checkered pattern formed by alternately arranging a first pattern that is a square of the first color and a second pattern that is a square of the second color.
Thereby, a pattern in which the first color and the second color change in a predetermined cycle in a predetermined direction of the photographing paper can be formed. As a result, it is possible to accurately execute distortion correction of the captured image and extraction of the image from the corrected captured image.

The high-frequency pattern may be a rhombus checkered pattern formed by alternately arranging a first pattern that is a rhombus of the first color and a second pattern that is a rhombus of the second color.
Thereby, a pattern in which the first color and the second color change in a predetermined cycle in a predetermined direction of the photographing paper can be formed. As a result, it is possible to accurately execute distortion correction of the captured image and extraction of the image from the corrected captured image.

An image processing apparatus according to still another aspect of the present invention includes a storage unit, a distortion correction unit, and an image extraction unit. The storage unit stores captured images. The photographed image is an image obtained by photographing a subject to be photographed arranged on a photographing sheet on which a plurality of fine patterns are printed in a predetermined area.
The distortion correction unit corrects the distortion of the captured image using a plurality of fine patterns. The image extraction unit extracts an image of the photographing target by cutting out the contour of the photographing target from the photographed image subjected to distortion correction using a plurality of fine patterns.

  In this image processing apparatus, a plurality of fine patterns are printed on a shooting sheet on which a shooting target is arranged when shooting a shot image. Since the plurality of fine patterns are fine and difficult to visually recognize, the plurality of fine patterns do not become annoying to the user when an object to be photographed is arranged.

The plurality of fine patterns may include a first pattern with a first color and a second pattern with a second color. The second color is a color different from the first color. In this case, the high frequency pattern may be formed on the photographing paper by alternately arranging the first pattern and the second pattern at a predetermined cycle.
As a result, the image processing apparatus uses the high-frequency pattern in which the first color and the second color appear alternately in a predetermined cycle to accurately perform the distortion correction of the captured image and the process of extracting the captured object from the captured image. Can be executed well.

  The distortion correction unit may correct the distortion of the photographed image by matching the high-frequency pattern in the photographed image with the high-frequency pattern printed on the photographed paper. Thereby, distortion of the captured image can be corrected by simpler calculation.

  The distortion correction unit may correct the distortion of the captured image using a plurality of fine patterns existing in the vicinity of the imaging target. Thereby, the calculation amount of the information processing performed with respect to a picked-up image can be reduced.

  A captured image can be processed by using a pattern that does not obstruct the use.

The figure which shows the structure of an image processing apparatus. The figure which shows the structure of photography paper. 6 is a flowchart illustrating a method for processing a captured image captured using a photographing sheet. The figure which shows an example of arrangement | positioning to the imaging | photography paper of imaging | photography object. The figure which shows an example of a picked-up image. The figure which shows an example of the selected feature point of a 1st pattern. The figure which shows an example of the calculation method of the coordinate value of four selected corners of the area | region in which the high frequency pattern was provided. The figure which shows distortion correction typically. The figure which shows an example of the picked-up image after distortion correction. The figure which shows typically an example of the method of extracting the image of the imaging | photography object from a picked-up image.

1. First Embodiment (1) Image Processing Device Hereinafter, an image processing device 100 according to a first embodiment will be described.
First, the configuration of the image processing apparatus 100 according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating a configuration of an image processing apparatus. The image processing apparatus 100 includes, for example, a CPU, a ROM, a RAM, a mass storage device (for example, SSD, a hard disk), an input / output interface (for example, a USB interface, a keyboard, a touch panel), and a communication interface (for example, Ethernet (registered). (Registered trademark) controller, wireless LAN controller, Bluetooth (registered trademark) controller, etc.), etc., and a computer system such as a smartphone or a tablet terminal. The image processing apparatus 100 may include an image sensor such as a CCD sensor.
Some or all of the functions of each component of the image processing apparatus 100 described below may be realized as a program (application) that can be executed by a general-purpose computer, a smartphone, or a tablet terminal that configures the image processing apparatus 100. Good. The program may be stored in a mass storage device of a computer system that constitutes the image processing apparatus 100.

The image processing apparatus 100 includes a storage unit 1. The storage unit 1 is a part or all of a storage area of a storage device of a computer system that constitutes the image processing apparatus 100. The storage unit 1 stores various data.
The image processing apparatus 100 includes a captured image acquisition unit 3. The photographed image acquisition unit 3 acquires a photographed image I photographed in a state where the photographing object O is arranged on the photographing paper P on which a high-frequency pattern FP (described later) is printed, and stores the photographed image I in the storage unit 1.

  The above-mentioned “arranging the photographing object O” means placing an object that is the photographing object O on the photographing paper P and / or drawing and / or printing a picture that is the photographing object O on the photographing paper P. Say to do. In addition, drawing and / or printing a picture on the photographing paper P and further placing an object on the photographing paper P are also included in the “arranging the photographing object O”. In this case, the drawn picture and the placed object become the photographing target O. Further, the number of photographing objects O arranged on the photographing paper P is arbitrary.

  The captured image acquisition unit 3 inputs a captured image I from a camera or an image scanner connected to an input / output interface such as a USB interface provided in the image processing apparatus 100 and stores the captured image I in the storage unit 1. The captured image acquisition unit 3 may input the captured image I by communication from an external computer system or the like via the communication interface of the image processing apparatus 100 and store the captured image I in the storage unit 1.

  Furthermore, when the image processing apparatus 100 includes an image sensor such as a CCD sensor, the captured image acquisition unit 3 generates a captured image I using data obtained by the image sensor, and stores it in the storage unit 1. You may remember.

  The image processing apparatus 100 includes a distortion correction unit 5. The distortion correction unit 5 corrects distortion of the captured image I. Specifically, the distortion correction unit 5 corrects the distortion of the photographed image I using the high-frequency pattern FP that is included in the photographed image I by photographing the photographing object O with the photographing paper P as the background. .

  The distortion correction unit 5 converts, for example, a matrix (homography matrix (described later)) that converts a coordinate system representing the high-frequency pattern FP captured in the photographed image I into a coordinate system representing the high-frequency pattern printed on the photographing paper P. Use to correct the distortion of the captured image I. The distortion correction method using the above matrix will be described in detail later.

  The image processing apparatus 100 includes an image extraction unit 7. The image extraction unit 7 extracts an image (extracted image E) of the shooting target O by cutting out a contour representing the shooting target O from the shot image I ′ whose distortion is corrected by the distortion correction unit 5, and stores it in the storage unit 1. Remember. In the present embodiment, the image extraction unit 7 applies the low-pass filter to the captured image I ′ whose distortion has been corrected, thereby extracting the extracted image E corresponding to the imaging target O from the captured image I ′. . The method of extracting the extracted image E in the image extracting unit 7 will be described in detail later.

  By having the above configuration, the image processing apparatus 100 can capture a captured image I acquired by an external device such as a camera, an image scanner, or another computer via communication, or a captured image I (image captured by itself). When the processing apparatus 100 has an image sensor), the distortion correction process and the extraction process of the extracted image E can be executed.

(2) Photographing Sheet Next, the photographing sheet P on which the photographing object O is arranged when photographing the photographed image I will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration of the photographing paper.
The photographing paper P has a high-frequency pattern FP printed on a predetermined area of the photographing paper P. In the present embodiment, the high-frequency pattern FP is provided with a first pattern SP1 that is a fine pattern with a first color (black in the example of FIG. 2) and a second color (white in the example of FIG. 2). The second pattern SP2 which is a fine pattern is formed.

  Specifically, the high-frequency pattern FP is a first color square in the width direction (X direction) and the length direction (Y direction) of the photographing paper P, for example, as shown in FIG. It is formed by alternately arranging (printing) one pattern SP1 and a second pattern SP2 that is a square of the second color (rectangular checkered pattern). In this case, the high frequency pattern FP is a pattern in which the first color and the second color alternately change in a predetermined cycle in the X direction and the Y direction.

Alternatively, for example, as shown in FIG. 2B, the first pattern SP1 that is a rhombus of the first color and the second pattern SP2 that is a rhombus of the second color in the width direction and the length direction of the photographing paper P. The high frequency pattern FP can also be formed by alternately arranging and (rhombus checkered pattern).
In this case, the high-frequency pattern FP is a pattern in which the first color and the second color are alternately changed at a predetermined period in a direction having an angle of 45 ° with respect to the X direction (or Y direction).

  By using the rectangular checkerboard pattern or the rhombus checkerboard pattern as the high frequency pattern FP, a pattern in which the first color and the second color change in a predetermined cycle in the predetermined direction of the photographing paper P can be formed. As a result, the distortion correction of the captured image I, which will be described later, and the extraction of the extracted image E from the corrected captured image I ′ can be performed with high accuracy.

As a period of change between the first color and the second color in the high-frequency pattern FP, for example, 0.2 Hz per pixel of the captured image I (that is, from the first color to the second color in five pixels, Or, conversely, the color can be selected from a range of 1 to 2 Hz (that is, the color changes from the first color to the second color in one pixel or vice versa).
The period of change between the first color and the second color of the high-frequency pattern FP can be set to an arbitrary value by adjusting the sizes of the first pattern SP1 and the second pattern SP2, for example.

As the print color of the fine pattern, for example, the first color of the first pattern SP1 is a light color such as gray or light blue, and the second pattern SP2 of the second pattern SP2 is a color that does not affect the visual perception of the user of the photographing paper P. The two colors can be white (or the second pattern SP2 is not printed).
When drawing a picture on the photographic paper P, the first color and the second color are light and / or light so that the first pattern SP1 and the second pattern SP2 can be filled with the color of the picture. Is preferred.

  In the image processing apparatus 100, the brightness difference between the first color and the second color is equal to or greater than a predetermined value so that the first pattern SP1 and the second pattern SP2 included in the captured image I can be clearly identified. It is preferable. Specifically, the brightness difference between the first color and the second color may be 10 or more, preferably 50 or more, and more preferably 100 or more.

  Since the photographing paper P on which the photographing object O is placed when photographing the photographed image I has the above-described configuration, the photographed image I can include the high-frequency pattern FP. As a result, the image processing apparatus 100 can execute various types of information processing (distortion correction processing and image extraction processing) on the captured image I using the high-frequency pattern FP included in the captured image I.

  Since the period of change between the first color and the second color in the high-frequency pattern FP is 0.2 Hz to 2 Hz per pixel of the captured image I, the first pattern SP1 and the second pattern SP2 can be reduced. As a result, the first pattern SP1 and the second pattern SP2 (high-frequency pattern FP) printed on the photographing paper P are not annoying to the user when the photographing object O is arranged on the photographing paper P.

(3) Operation of Image Processing Device Next, a processing method of the captured image I photographed using the photographing paper P, which is executed in the image processing device 100, will be described with reference to FIG. FIG. 3 is a flowchart showing a method for processing a photographed image photographed using photographing paper.
First, a captured image I to be processed is acquired. Specifically, first, an object to be photographed O is placed on the photographing paper P and / or a picture to be photographed O is drawn on the photographing paper P (step S1).
For example, as shown in FIG. 4, a shooting object O1, which is a cylindrical object, is placed on an area of the shooting paper P on which a high-frequency pattern FP is provided, and a shooting object O2, which is a triangular picture, is placed in the area shape. Suppose that it was drawn. FIG. 4 is a diagram illustrating an example of an arrangement on a photographing sheet to be photographed.

After the shooting object O is placed on the shooting paper P, shooting is performed using the shooting paper P as a background using a shooting device such as a camera or a shooting application provided in the image processing apparatus 100 (when the image processing apparatus 100 includes an image sensor). The subject O is photographed and a photographed image I is acquired (step S2).
After acquiring the captured image I, the imaging device is connected to the input / output interface of the image processing apparatus 100, and the captured image acquisition unit 3 of the image processing apparatus 100 stores the captured image I captured by the imaging apparatus in the image processing apparatus 100. Captured and stored in the storage unit 1. Alternatively, a shooting application that is the shot image acquisition unit 3 stores the acquired shot image I in the storage unit 1.

  Assume that, for example, a photographed image I as shown in FIG. 5 is obtained as a result of photographing the photographing objects O1 and O2 arranged on the photographing paper P shown in FIG. 4 by a photographing device or a photographing application. FIG. 5 is a diagram illustrating an example of a captured image. In the example shown in FIG. 5, since the field of view of the photographing device such as a camera is not parallel to the surface of the photographing paper P on which the high-frequency pattern FP is provided, the rectangular photographing paper P is a distorted quadrangle. A captured image I is obtained.

After acquiring the captured image I, the distortion correction unit 5 reads the captured image I stored in the storage unit 1 and executes distortion correction on the read captured image I (step S3). In the present embodiment, the distortion correction unit 5 corrects the distortion of the photographed image I by matching the high-frequency pattern FP in the photographed image I with the high-frequency pattern FP printed on the photographed paper P.
Specifically, the distortion correcting unit 5 performs a matrix (homography matrix H and matrix) for performing an operation for matching the high frequency pattern FP in the captured image I with the high frequency pattern FP printed on the photographing paper P as follows. And the distortion of the captured image I is corrected by applying the homography matrix H to the coordinate value of each pixel in the captured image I.

First, in order to calculate the homography matrix H, feature points (before conversion) of the high-frequency pattern FP in the captured image I are extracted.
The feature points (before conversion) of the high frequency pattern FP in the captured image I are extracted as follows. When the rectangular checkered pattern as shown in FIG. 2A is provided on the photographing paper P as the high-frequency pattern FP, for example, the first pattern that exists at the uppermost left end of the high-frequency pattern FP in the photographed image I. Extract four corners of SP1.
For example, as shown in FIG. 6, the coordinates (px1, py1) (the coordinates of the upper left corner) of the first pattern SP1 at the upper left corner of the region where the high frequency pattern FP in the captured image I is provided, (px2) , Py2) (upper right corner coordinates), (px3, py3) (lower left corner coordinates), and (px4, py4) (lower right corner coordinates) are extracted as feature points. FIG. 6 is a diagram illustrating an example of selected feature points of the first pattern.

  Next, the coordinates of the four corners of the first pattern SP1 are extended, and the coordinate values of the four corners of the region where the high frequency pattern FP in the captured image I is provided are calculated. When the feature points at the four corners of the first pattern SP1 are determined as described above, for example, as shown in FIG. 7, the coordinate values at the four corners of the region where the high frequency pattern FP is provided in the captured image I are calculated. it can. FIG. 7 is a diagram illustrating an example of a method of calculating coordinate values of four selected corners of a region where a high frequency pattern is provided.

  Specifically, the coordinate value (x1, y1) of the upper left end of the region where the high frequency pattern FP is provided is, for example, the upper left end of the first pattern SP1 existing at the upper left end of the region where the high frequency pattern FP is provided. As coincident with the coordinates, x1 = px1, and y1 = py1 can be determined.

  The coordinate value (x2, x2) at the upper right end is obtained by extending a vector represented by w1 * (px2-px1, py2-py1) (w1: scalar quantity) from the coordinates (x1, y1) at the upper left end, for example. The coordinates of the end point of the vector at that time, that is, x2 = px1 + w1 * (px2-px1), y2 = py1 + w1 * (py2-py1) can be calculated.

  For example, the coordinate value (x3, x3) of the lower left corner is obtained by extending a vector represented by w2 * (px3-px1, py3-py1) (w2: scalar quantity) from the coordinates (x1, y1) of the upper left corner. The coordinates of the end point of the vector at that time, that is, x3 = px1 + w2 * (px3−px1), y3 = py1 + w2 * (py3−py1) can be calculated.

  The coordinate value (x4, x4) of the lower right end (which is the right end or the lower end of the region where the high frequency pattern FP is provided, but does not necessarily correspond to the lower right end) is, for example, from the upper left end coordinates (x1, y1). , W3 * (px4-px1, py4-py1) (w3: scalar quantity), the coordinates of the end point of the vector when extended, that is, x4 = px1 + w3 * (px4-px1), y4 = py1 + w3 * It can be calculated as (py4-py1).

  The above-described scalar amounts w1, w2, and w3 are the total number of first patterns SP1 and second patterns SP2 arranged in the width direction (X direction) of the photographing paper P, and the first pattern in the length direction (Y direction). This is the total number of arrangements of SP1 and second pattern SP2, and the total number of arrangements of first pattern SP1 and second pattern SP2 in a 45-degree oblique direction (the direction in which Y = X holds). Therefore, the scalar quantities w1, w2, and w3 are constant positive integers determined in advance by the arrangement of the first pattern SP1 and the second pattern SP2.

Thereafter, using the coordinate values of the four corners of the region where the high-frequency pattern FP is provided in the photographed image I and the coordinate values of the four corners corresponding to the region of the photographing paper P where the high-frequency pattern FP is provided, Thus, the homography matrix H is calculated.
For example, the coordinate values (x1, y1), (x2, y2), (x3, y3), and (x4, y4) in the photographed image I are the regions of the photographing paper P where the high-frequency pattern FP is provided. The corresponding coordinate values (X1 ′, Y1 ′), (X2 ′, Y2 ′), (X3 ′, Y3 ′), (X4 ′, Y4 ′) of the corresponding four corners are assumed.

  Note that the coordinate values (X1 ′, Y1 ′), (X2 ′, Y2 ′), (X3 ′, Y3 ′), and (X4 ′, Y4 ′) of the four corners are the first pattern SP1 on the photographing paper P. And based on the arrangement relationship of the second pattern SP2, it is uniquely determined in advance.

In this case, the relationship between the coordinate values of the four corners of the photographed image I and the coordinates of the four corners of the corresponding photographing paper P is expressed by the following formula 1 using the homography matrix H. Can express. In the following formula, w is an arbitrary scalar quantity (constant).

In the above case, the homography matrix H is a matrix of 3 rows and 3 columns, and can be expressed as the following Expression 2.

  The above formula 1 represents the origin and four coordinate values (X1 ′, Y1 ′), (X2 ′, Y2 ′), (X3 ′, Y3 ′), (X4 ′, Y4 ′) on the photographing paper P. A vector calculated by connecting the coordinate values (x1, y1), (x2, y2), (x3, y3), (x4, y4) of the four corners in the captured image I to the origin is a homography matrix H. This indicates that the vector is parallel to the vector connecting the coordinate values after coordinate conversion.

Therefore, the coordinate value (X1 ′, Y1 ′), (X2 ′, Y2 ′), (X3 ′, Y3) on the photographing paper P is utilized by utilizing that the outer product of two vectors parallel to each other is 0 (vector). The following numbers are present between '), (X4', Y4 ') and coordinate values (x1, y1), (x2, y2), (x3, y3), (x4, y4) in the captured image I. 3 holds. In the following Equation 3, “x” means an outer product.

If the above equation 3 is further arranged, it can be rewritten as the following equation 4. That is, for each element h11 to h33 of the homography matrix H, h33 = 1 is established, and simultaneous equations composed of equations for 8 h11 to h32 are established. Specific numerical values of the elements h11 to h33 of the homography matrix H can be calculated from the simultaneous equations.

After the homography matrix H is specifically calculated, the distortion correction unit 5 applies the homography matrix H (a matrix obtained by dividing each element of the captured image I by the scalar quantity w) to each pixel included in the captured image I (that is, , H (x / w, y / w, 1 / w) ^T (superscript “T” means transposition of a matrix with respect to the coordinate value (x, y, 1) of each pixel of the captured image I. ) Is calculated), as shown in FIG. 8, the coordinate values (x1, y1), (x2, y2), (x3, y3), (x4) of the four corners in the captured image I indicated by the circles. , Y4) are coordinate values (X1 ′, Y1 ′), (X2 ′, Y2 ′), (X3 ′, Y3 ′), (X4) of the corresponding four corners on the photographing paper P indicated by triangles. ', Y4'). FIG. 8 is a diagram schematically showing distortion correction.

  As a result of the above coordinate movement, the distortion correction unit 5 matches the region where the high-frequency pattern FP is provided in the photographic image I in which the distortion is generated with the region where the high-frequency pattern FP is provided in the photographic paper P without distortion. At the same time, distortion of the captured image I can be corrected.

  By correcting the distortion of the captured image I using the high-frequency pattern FP, the distortion of the captured image I can be corrected with a relatively simple calculation such as using the homography matrix H.

After executing the distortion correction of the photographed image I as described above, the image extraction unit 7 uses the high frequency pattern FP (fine pattern) to cut out the outlines of the photographing objects O1 and O2 from the photographed image I ′ whose distortion has been corrected. Thus, the images of the photographing targets O1 and O2 are extracted as the extracted image E (step S4). The image extraction unit 7 stores the extracted extracted image E in the storage unit 1.
As described above, the image extraction unit 7 applies the low-pass filter to the photographed image I ′ to delete (transparent) the data in the region where the high-frequency pattern FP appears in the photographed image I ′. The high-frequency pattern FP holds the data of the area hidden by the photographing objects O1 and O2.

Specifically, for example, when a rectangular checkered pattern as shown in FIG. 2A is used as the high-frequency pattern FP, the image extraction unit 7 moves in the X direction or the Y direction of the captured image I ′. Each pixel is scanned to obtain the color value of each pixel. For example, in the photographed image I ′ shown in FIG. 9, when a pixel existing between A and A ′ shown by the alternate long and short dash line in FIG. 9 is scanned in the X direction, the color as shown in FIG. Suppose a change is obtained.
FIG. 9 is a diagram illustrating an example of a captured image after distortion correction. FIG. 10 is a diagram schematically illustrating an example of a method for extracting an image to be captured from a captured image.

In FIG. 10A, in a certain region in the X direction, the color of the pixel changes between the first color and the second color at a predetermined cycle. The color change occurs when the first pattern SP1 and the second pattern SP2 are alternately arranged in the X direction to form a high frequency pattern FP that is a rectangular checkered pattern.
On the other hand, in FIG. 10A, in other regions in the X direction, the color is constant or changes slowly (there may be a small high-frequency color change such as noise). ing. The change in color means that the imaging target O1 exists in the other area (generally, the color of the imaging target O does not change as frequently as the high-frequency pattern FP).

The image extracting unit 7 has a color that changes in the arrangement period of the first pattern SP1 and the second pattern SP2 in the high-frequency pattern FP, and the change in the color changes between the first color and the second color. Pixel data existing in the above-changed region is deleted from the captured image I ′ (or the color is made transparent). On the other hand, pixel data existing in other regions is held in the captured image I ′.
As a result, for example, as shown in FIG. 10B, a captured image I ′ (that is, an extracted image E) in which only pixel data corresponding to the imaging target O1 is retained is obtained.

As shown in FIG. 9, when a plurality of shooting targets O1 and O2 are included in the shot image I ′, the image extraction unit 7 selects the plurality of shooting targets O1 and O2 (for example, color Recognizing the difference (by a difference or the like), the respective images of the plurality of photographing objects O1 and O2 are extracted as individual extracted images.
Alternatively, when the user of the image processing apparatus 100 designates areas including a plurality of shooting targets O1 and O2, and the image extraction unit 7 executes the above-described image extraction for each of the designated areas. The images of the plurality of shooting targets O1 and O2 may be extracted as individual extracted images.

  As described above, by taking an image with the background of the photographing paper P on which the high-frequency pattern FP in which the first color and the second color are alternately changed at a predetermined period as the photographed image I, the photographed image is obtained. Using the high-frequency pattern FP included in I, as described above, distortion correction of the captured image I and extraction processing of a specific image (image of each capturing object O) can be performed.

  By making the first pattern SP1 and the second pattern SP2 constituting the high-frequency pattern FP fine (fine pattern), the first pattern SP1 and the second pattern SP2 can be hardly seen. As a result, when the user places the photographing object O on the photographing paper P, the first pattern SP1 and the second pattern SP2 are not annoying for the user.

2. Other Embodiments Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the scope of the invention. In particular, a plurality of embodiments and modifications described in this specification can be arbitrarily combined as necessary.
(A) Another embodiment of a plurality of fine patterns used for information processing of a captured image (part 1)
In the first embodiment, the distortion correction and the extraction process of the image of the imaging target O are performed on the basis of the first pattern SP1 (or the second pattern SP2) at the four corners of the area where the high frequency pattern FP is provided. The first pattern SP1 and the second pattern SP2 included in the first pattern SP2 are executed. Thereby, the distortion correction of the captured image I and the extraction process of the imaging target O can be performed by a single calculation over the entire region where the high frequency pattern FP is provided.

As shown in the first embodiment, in the case of extracting a necessary image of the imaging target O from the captured image I (or the captured image I ′ after distortion correction), the entire region where the high frequency pattern FP is provided is not necessarily provided. There is no need to perform distortion correction on the image.
That is, the distortion correction unit 5 corrects distortion of the captured image I using a plurality of fine patterns (first pattern SP1 and second pattern SP2) existing in the vicinity of the imaging target O to be extracted from the captured image I. Also good.

For example, the distortion of the captured image I can be corrected using the first pattern SP1 and the second pattern SP2 that exist in the vicinity of the imaging target O as follows.
First, the image extraction unit 7 automatically recognizes the photographing object O in the photographed image I, has an area slightly larger than the area occupied by the recognized photographing object O in the photographed image I, and is photographed. An area including all the objects O is extracted from the captured image I. As a result, it is possible to extract an image including the imaging target O and having the first pattern SP1 and the second pattern SP2 at least at the ends.
Note that the image extraction unit 7 may extract a region that satisfies the above-described conditions from the captured image I without automatically recognizing the capturing object O.

Next, the distortion correction unit 5 selects, for example, the first pattern SP1 and / or the second pattern SP2 existing at the four corners of the extracted region, and performs the same as described in the first embodiment. Distortion correction is performed on the extracted area.
As described above, by extracting an image including the imaging target O and at least the first pattern SP1 and the second pattern SP2 at the end from the captured image I, and performing distortion correction on the extracted image, The amount of calculation for distortion correction in the distortion correction unit 5 can be reduced.

In addition, after correcting the distortion of the image extracted as described above, the image extraction unit 7 applies a low-pass filter to the extracted image after the distortion correction, and the high-frequency pattern FP ( The first pattern SP1 and the second pattern SP2) are deleted, and an image of the shooting target O is extracted from the image.
The calculation amount when the low-pass filter is applied to the extracted image is smaller than the calculation amount when the low-pass filter is applied to the entire region in which the high-frequency pattern FP is provided.

  Therefore, by performing distortion correction and extraction processing of the imaging target O on the image extracted from the captured image I as described above, the amount of information processing for the image can be reduced.

(B) Another embodiment of a plurality of fine patterns used for information processing of captured images (part 2)
In the first embodiment, the captured image I includes the entire area where the high frequency pattern FP is provided. That is, when photographing the photographing object O with the photographing paper P (the high-frequency pattern FP) as a background, the photographing device and the photographing paper P are arranged so that the entire region where the high-frequency pattern FP is provided is included in the field of view of the photographing device. The positional relationship was adjusted.

As shown in the first embodiment, if the high-frequency pattern FP is included in the captured image I, information processing (distortion correction and extraction processing of the imaging target O) can be performed on the captured image I. An image including the target O and a part of the region where the high frequency pattern FP is provided may be acquired as the captured image I.
In this case, for example, the first pattern SP1 and the second pattern SP2 existing at the four end portions (or the vicinity thereof) of the captured image I are used in the same manner as described in the first embodiment. The image I is subjected to distortion correction and the extraction of the image of the photographing object O.

  As described above, the distortion correction and the extraction process of the imaging target O can be performed even if an image including the imaging target O and a part of the region where the high frequency pattern FP is provided is used as the captured image I. The condition that the captured image I needs to include such as including the entire region where the high frequency pattern FP is provided can be relaxed. As a result, a burdensome work for the user, such as positioning the photographing device when acquiring the photographed image I, is not required.

  The present invention can be widely applied to an image processing apparatus and an image processing method for performing a predetermined process on a captured image obtained by photographing a photographing object.

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 1 Memory | storage part 3 Captured image acquisition part 5 Distortion correction | amendment part 7 Image extraction part O, O1, O2 Shooting object P Shooting paper FP High frequency pattern SP1 1st pattern SP2 2nd pattern I, I 'Captured image E Extracted image

Claims (13)

Obtaining a photographed image by photographing the photographing object with the photographing paper as a background in a state where the photographing object is arranged on a photographing paper having a plurality of fine patterns printed in a predetermined area;
Using the plurality of fine patterns to correct distortion of the captured image;
Extracting the image of the photographing target by cutting out the contour of the photographing target from the photographed image subjected to distortion correction using the plurality of fine patterns;
An image processing method comprising: The plurality of fine patterns include a first pattern with a first color and a second pattern with a second color different from the first color,
The image processing method according to claim 1, wherein a high-frequency pattern is formed by alternately arranging the first pattern and the second pattern at a predetermined period.   3. The image according to claim 2, wherein in the step of correcting the distortion, the distortion of the photographed image is corrected by matching the high-frequency pattern in the photographed image with the high-frequency pattern printed on the photographing paper. Processing method.   The image processing method according to claim 1, wherein in the step of correcting the distortion, distortion of the captured image is corrected using the plurality of fine patterns existing in the vicinity of the imaging target.   The program which makes a computer perform the image processing method in any one of Claims 1-4. A photographic paper used to shoot a subject,
A plurality of images used for correcting distortion of a captured image obtained by capturing the imaged object placed on the imaging paper and extracting the image of the imaged object by cutting out the outline of the imaged object from the captured image after distortion correction. Photographic paper with a fine pattern. The plurality of fine patterns include a first pattern with a first color and a second pattern with a second color different from the first color,
The photographing paper according to claim 6, wherein a high-frequency pattern is formed by alternately arranging the first pattern and the second pattern at a predetermined period.   The high-frequency pattern is a rectangular checkered pattern formed by alternately arranging the first pattern that is the square of the first color and the second pattern that is the square of the second color. Photographic paper described in 1.   The high-frequency pattern is a rhombus checkered pattern formed by alternately arranging the first pattern that is a rhombus of the first color and the second pattern that is a rhombus of the second color. Photographic paper described in 1. A storage unit for storing a photographed image obtained by photographing a photographing object arranged on a photographing sheet on which a plurality of fine patterns are printed in a predetermined area;
A distortion correction unit that corrects distortion of the captured image using the plurality of fine patterns;
An image extraction unit that extracts the image of the imaging target by cutting out the outline of the imaging target from the captured image subjected to distortion correction using the plurality of fine patterns;
An image processing apparatus comprising: The plurality of fine patterns include a first pattern with a first color and a second pattern with a second color different from the first color,
The image processing apparatus according to claim 10, wherein a high-frequency pattern is formed on the photographing paper by alternately arranging the first pattern and the second pattern at a predetermined period.   The image processing apparatus according to claim 11, wherein the distortion correction unit corrects distortion of the photographed image by matching the high-frequency pattern in the photographed image with the high-frequency pattern printed on the photographing paper. .   The image processing apparatus according to claim 10, wherein the distortion correction unit corrects distortion of the captured image using the plurality of fine patterns existing in the vicinity of the imaging target.

Images