JP2005275447A - Image processing device, image processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To acquire an image looking like being picked up as the front view by suitably correcting the image to be picked up. <P>SOLUTION: A digital camera 1 comprises a CPU and an image processing device. The image processing device creates a reduced binarized image from a picked up image of a white plate 2 as an imaging object. The image processing device performs labeling process for the reduced binarized image and changes the pixel value of the image into a pixel value of a background pixel with using an image with maximum area as a subject image and an image in other region as an image including an image reflected by a fluorescent lamp. The image processing device acquires a subject image from the reduced binarized image and the border and performs projection conversion by seeking a projection parameter. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  With the development of digital cameras and the low price of storage-type memory, digital cameras are not only used for taking pictures of landscapes and people, but are also used for paper documents, business cards, and blackboards written at meetings. An application is being considered in which what is displayed is photographed, and these images are digitally stored in a personal computer or the like for management.

  When photographing a document or a blackboard in this way, it is desirable to photograph the photographing object in front and vertically. However, it is difficult to shoot a document placed on the desk vertically and vertically. Also, when shooting a blackboard, it may be difficult to photograph the blackboard from the front depending on the position of the photographer. Even if the image can be taken from the front, it may be preferable to avoid taking the image from the front for reasons such as reflection of light. Thus, when an object such as a document is photographed from an oblique direction, characters and the like are distorted obliquely or trapezoidally. Of course, these characters can be interpreted. However, even if it can be read, it is very tiring to read. Moreover, since these images have distortion, it is difficult to reuse these images.

  As a solution to such a problem, for example, there is one in which image distortion is corrected using an image having two parallaxes (see, for example, Patent Document 1).

This image processing unit creates a corrected image that is synthesized by performing image conversion so that parallax is eliminated with respect to two or more images and combining them. Then, the image composition unit stores the created corrected image in the image memory.
JP 2001-134751 (5th page, FIG. 1)

  However, in such a conventional apparatus, when image editing is performed again, the image deteriorates if the corrected image stored in the image memory is edited. If the original image before correction is to be corrected again, it is necessary to store the original image and the corrected image in the image memory, and it is necessary to repeat similar image processing on the original image.

  Therefore, it is conceivable that the photographing apparatus has an image processing function for correcting the image of the photographing object by detecting the contour based on the edge image of the photographing object and cutting out the image of the subject.

  However, even when using such a photographing device, the photographing object cannot be captured to the full angle of view at the time of photographing. For example, when photographing an image projected on the screen by a projector or the like, In addition to the projection unit, a reflected image of the fluorescent lamp or the fluorescent lamp itself is reflected at the same time. Since such an image includes a bright part other than the object to be cut out at the same time, an edge other than a quadrangular subject appears in the above-described edge image, and this may be erroneously detected as a contour.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus, an image processing method, and a program that can correctly acquire an image to be captured.

In order to achieve the above object, an image processing apparatus according to the first aspect of the present invention provides:
In an image processing apparatus that acquires an image of a target shooting target from a shot image obtained by shooting and performs image processing,
An image acquisition unit that obtains an area in which pixels having pixel values corresponding to pixel values of the image to be photographed are continuous, and obtains an image of an area having the maximum area among the obtained areas as the image to be photographed; ,
An image processing unit that performs image processing on the image to be captured acquired by the image acquisition unit.

  The image acquisition unit scans each pixel of the captured image, acquires a pixel having a pixel value corresponding to a pixel value of the captured image as a target pixel, and acquires the acquired target pixel and a pixel adjacent to the target pixel It is preferable that the region is determined based on a correspondence relationship with the pixel value.

  The image acquisition unit determines that an image of an area smaller than the area of the image to be imaged among areas formed by the continuous pixels is an image outside the object to be imaged taken together with the object to be imaged, and It is preferable that the pixel value of an outside image is changed to a pixel value of a background pixel set in advance to acquire the image to be captured.

Comparing a pixel value of each pixel of the photographed image with a predetermined threshold, and a binarized image creating unit that creates a binarized image of the photographed image based on a comparison result,
It is preferable that the image acquisition unit acquires a pixel having a pixel value corresponding to the shooting target image from the binarized image created by the binarized image creation unit.

  When the image acquisition unit scans each pixel of the captured image to acquire the target pixel, the image acquisition unit refers to the pixel value of the pixel adjacent to the acquired target pixel, and the already scanned adjacent pixel is preset. If the pixel value of the background pixel has a unique value, the target pixel is newly assigned a unique value, and if the adjacent pixel already has a unique value, the neighbor pixel is adjacent to the target pixel. By assigning the same value as the pixel, the correspondence relationship between the target pixel and the pixel value of the pixel adjacent to the target pixel is obtained, and when all the pixels of the photographed image are scanned, each pixel is assigned a unique value. It is preferable that the area is discriminated with reference to a value.

The image processing unit
A shape acquisition unit that acquires the contour of the image to be captured acquired by the image acquisition unit and acquires the shape of the image;
A projection parameter acquisition unit that obtains a projection parameter indicating a relationship between the imaging target and the imaging target image by associating the shape of the imaging target image acquired by the shape acquisition unit with the shape of the imaging target;
It is preferable to include an image conversion unit that performs image conversion of the image to be captured using the projection parameter obtained by the projection parameter acquisition unit.

In order to achieve the above object, an image processing method according to a second aspect of the present invention includes:
An image processing method of a photographing apparatus for photographing a photographing object,
Obtaining a region in which pixels having pixel values corresponding to pixel values of a target image to be photographed are continuous from a photographed image obtained by photographing;
When there are a plurality of obtained regions, obtaining an image of the region having the largest area as the image to be captured;
And performing image processing on the acquired image to be photographed.

In order to achieve the above object, a program according to the third aspect of the present invention provides:
On the computer,
A procedure for obtaining a region in which pixels having pixel values corresponding to pixel values of a target image to be photographed are continuous from a photographed image obtained by photographing,
In the case where there are a plurality of obtained areas, a procedure for acquiring an image of an area with the largest area as the image to be captured,
A procedure for performing image processing of the acquired image to be imaged;
Is to execute.

  According to the present invention, it is possible to easily correct an image of a photographing object.

Hereinafter, an imaging device according to an embodiment of the present invention will be described with reference to the drawings.
In the embodiment, the photographing apparatus is described as a digital camera.

A configuration of a digital camera 1 according to the present embodiment is shown in FIG.
The digital camera 1 according to the present embodiment shoots characters, drawings, photographs, and the like such as the white board 2 as a target shooting target, detects and corrects image distortion from an image obtained by shooting, as if An image as if taken from the front is generated. The digital camera 1 includes a photographic lens unit 11, a liquid crystal monitor 12, and a shutter button 13.

  The photographic lens unit 11 includes a lens that collects light and the like, and collects light from characters such as the white plate 2, drawings, and photographs.

The liquid crystal monitor 12 is for projecting an image taken in through the photographing lens unit 11.
The shutter button 13 is pressed when shooting a shooting target.

  As shown in FIG. 2, the digital camera 1 includes an optical lens device 21, an image sensor 22, a memory 23, a display device 24, an image processing device 25, an operation unit 26, a computer interface unit 27, An external storage IO device 28 and a program code storage device 29 are provided.

  The optical lens device 21 includes a photographic lens unit 11 and a driving unit thereof, and forms an image on the image sensor 22 by condensing light from characters, drawings, photographs, and the like on the white plate 2. .

  The image sensor 22 is for capturing a formed image as digitized image data, and is constituted by a CCD or the like. If the image sensor 22 is controlled by the CPU 30 and the shutter button 13 is not pressed, digital image data having a low preview resolution is generated, and the image data is periodically stored in the memory 23 at intervals of about 30 sheets per second. To send. Further, when the shutter button 13 is pressed, the image sensor 22 generates image data with high resolution and sends the generated image data to the memory 23.

  The memory 23 temporarily stores low-resolution preview images from the image sensor 22, high-resolution image data, original image data processed by the image processing device 25, and processed image data. The memory 23 sends the temporarily stored image data to the display device 24 or the image processing device 25.

  The display device 24 includes the liquid crystal monitor 12 and displays the image on the liquid crystal monitor 12. The display device 24 displays a low-resolution preview image or a high-resolution image temporarily stored in the memory 23 on the liquid crystal monitor 12.

  The image processing device 25 is for performing image processing such as image data compression, image distortion correction, and image effect processing on the image data temporarily stored in the memory 23.

  As shown in FIG. 3A, when the digital camera 1 shoots characters on the white plate 2 from the left direction and the right direction, the liquid crystal monitor 12 displays the characters shown in FIGS. 3B and 3C. As shown, the white plate 2 and images such as letters, diagrams, and photographs are distorted and displayed. The image processing device 25 performs image processing on the images as shown in FIGS. 3B and 3C to generate an image taken from the front as shown in FIG. 3D. To do.

  In order to correct image distortion, the image processing device 25 cuts a square from the distorted image, and performs projective transformation of the captured image into the cut quadrangle.

More specifically, the image processing device 25 is controlled by the CPU 30 and mainly performs the following processing.
(1) Extraction of affine parameters from photographed image (2) Image conversion using extracted affine parameters (3) Adjustment processing of image conversion (4) Extraction of image effect correction parameters relating to luminance or color difference and image effect processing The processing contents of will be described later.

  The operation unit 26 includes switches and keys for controlling the document projection function. When the user presses these switches and keys, the operation unit 26 responds and transmits the operation information at this time to the CPU 30.

  As shown in FIG. 4, the operation unit 26 includes an upper reduction key 111, a lower reduction key 112, a right reduction key 113, a left reduction key 114, a right rotation key 115, and a left rotation key 116. .

  The upper reduction key 111, the lower reduction key 112, the right reduction key 113, and the left reduction key 114 are projective conversion keys for performing projective conversion. The upper reduction key 111 is a key that is pressed when the upper part of the image is compared with the lower part around the X axis and the upper part is large and the upper part is rotated downward toward the paper surface.

  The lower reduction key 112 is a key that is pressed when the upper part and the lower part of the image are compared around the X axis, and the lower part is rotated downward toward the paper surface when the lower part is large.

  The right reduction key 113 and the left reduction key 114 are keys that are pressed when adjusting left and right distortion around the Y-axis side, and the right reduction key 113 is a key that is pressed when the right is large, and left The reduction key 114 is a key that is pressed when the left side is large.

  The right rotation key 115 and the left rotation key 116 are rotation correction keys for adjusting the rotation of the image. The right rotation key 115 is a key that is pressed when the image is rotated to the right, and the left rotation key 116 is a key that is pressed when the image is rotated to the left.

  In addition, the operation unit 26 includes a photographing key, a reproduction key, a cursor key, a control key, and the like (not shown). The shooting key is a key for selecting a shooting mode when shooting a shooting target. The reproduction key is a key for selecting a reproduction mode for reproducing an image to be photographed obtained by photographing. The control key is a key having functions such as a YES key for confirming the operation, a NO key for canceling the operation, and an editing key for performing editing.

  The computer interface unit 27 operates as a USB store registration class driver when the digital camera 1 is connected to a computer (not shown). Thus, when the computer is connected to the digital camera 1, the computer handles the memory card 31 as an external storage device of the computer.

  The external storage IO device 28 inputs and outputs image data and the like with the memory card 31. The memory card 31 stores image data and the like supplied from the external storage IO device 28.

  The program code storage device 29 is for storing a program executed by the CPU 30, and is configured by a ROM or the like.

  The CPU 30 controls the entire system according to a program stored in the program code storage device 29. The memory 23 is also used as a work memory for the CPU 30.

  When operation information is transmitted from the operation unit 26 by pressing a switch or key of the operation unit 26, the CPU 30 performs image sensor 22, memory 23, display device 24, image processing device based on the operation information. 25 etc. are controlled.

  Specifically, when the operation information indicating that the photographing key is pressed is transmitted from the operation unit 26, the CPU 30 sets each unit to the photographing mode. The CPU 30 sets the image sensor 22 to the preview mode if the shutter button 13 is not pressed in the state set to the shooting mode, and reads the high-resolution shooting target image if the shutter button 13 is pressed. Set to. Further, when the operation information indicating that the reproduction key is pressed is transmitted, the CPU 30 sets each unit to the reproduction mode.

  Further, when the operation information indicating that the projection conversion key and the rotation correction key are pressed is transmitted from the operation unit 26, the CPU 30 transmits the operation information to the image processing device 25, and causes the image processing device 25 to operate. Control.

  When the image data is temporarily stored in the memory 23, the CPU 30 records the preview image and the high-resolution image data in different storage areas.

  Further, the CPU 30 records preview image and high-resolution image data on the memory card 31 via the external storage IO device 28, and reads the recorded image data from the memory card 31. For this reason, the CPU 30 creates an image file for storing image data in the memory card 31.

  Then, for example, the CPU 30 records the corrected image data compressed in the JPEG format into image files. Further, the CPU 30 creates a header information storage area of the image file for each image file and records the image data, such as the projection parameters and image effect processing correction parameters obtained by the image processing device 25 when the image data is recorded. Is recorded in this header information storage area. This header information will be described later.

Next, the operation of the digital camera 1 according to this embodiment will be described.
When the user turns on (turns on) the digital camera 1, the CPU 30 acquires program data stored in the program code storage device 29. When the user presses the shooting button, the operation unit 26 transmits this operation information to the CPU 30. The CPU 30 receives this operation information, and the CPU 30, the image processing device 25, etc. execute the photographing process according to the flowchart shown in FIG.

The CPU 30 sets the image sensor 22 to the preview mode (step S11).
The CPU 30 determines whether or not the shutter button 13 has been pressed based on the operation information transmitted from the operation unit 26 (step S12).

  If it is determined that the shutter button 13 has been pressed (Yes in step S12), the CPU 30 controls the image sensor 22 by switching the image sensor 22 from the preview mode to the high resolution mode (step S13).

  The CPU 30 records the high-resolution photographic target image data generated by the image sensor 22 in a storage area different from the preview image on the memory 23 (step S14).

The CPU 30 determines whether the reading of the image data has been completed (step S15).
If it is determined that the reading has not been completed (No in step S15), the CPU 30 continues to control the image sensor 22 to read the image data.

  If it is determined that all the image data has been read and the image transfer has been completed (Yes in step S15), the CPU 30 generates a low-resolution preview image from the captured image (high-resolution image), and uses the preview image for the preview image in the memory 23. The preview image data is written in the storage area (step S16).

The CPU 30 controls the image processing device 25 so as to create compressed data, and the image processing device 25 creates compressed data (step S17).
The CPU 30 records this compressed data on the memory card 31 via the external storage IO device 28, and stores the compressed data created by the image processing device 25 (step S18).

  Next, the image processing device 25 extracts projection parameters for creating a front image from the captured image under the control of the CPU 30 (step S19).

The CPU 30 and the image processing device 25 determine whether or not the projection parameters have been extracted (step S20).
If it is determined that the extraction has been completed (Yes in step S20), the image processing device 25 creates a projected transformation image based on the extracted projection parameters (step S21).

  When the projection conversion key and the rotation correction key of the operation unit 26 are pressed, the operation unit 26 transmits this operation information to the CPU 30. The CPU 30 transmits operation information from the operation unit 26 to the image processing device 25, and the image processing device 25 performs image conversion adjustment processing in accordance with the transmitted operation information (step S22).

  The image processing device 25 extracts image effect correction parameters (step S23) and performs image effect processing (step S24).

  The image processing device 25 performs compression processing on the image data that has been subjected to the image effect processing, and creates compressed data (step S25).

  The image processing device 25 records the created compressed data in the image file of the memory card 31 (step S26). Further, the CPU 30 stores the projection parameters used for the projective conversion and the image effect correction parameters used for the image effect processing in the image file of the memory card 31.

  On the other hand, when it is determined that the projection parameters could not be extracted (No in step S20), the CPU 30 performs a warning process (step S27).

  The CPU 30 and the like end the photographing process in this way. Note that the CPU 30 or the like repeatedly executes this photographing process unless the user operates the key.

Next, image processing performed by the image processing device 25 will be described.
First, the basic concept (implementation method) of affine transformation used by the image processing apparatus 25 for image processing will be described.

Affine transformation is widely applied to image spatial transformation. In this embodiment, projective transformation is performed using two-dimensional affine transformation without using three-dimensional camera parameters. This is because the coordinates (u, v) before conversion are related by the transformation such as movement, enlargement / reduction, and rotation, and the coordinates (x, y) after conversion are related by the following equations (1) and (2). Become. Projective transformation can also be performed by this affine transformation.
The final coordinates (x, y) are calculated by the following formula 2.
Expression 2 is an expression for projective transformation, and the coordinates x and y are reduced toward 0 according to the value of z ′. That is, the parameter included in z ′ affects the projection. This parameter is a13, a23, a33. Since other parameters can be normalized by the parameter a33, a33 may be 1.

  FIG. 6 shows the coordinates of each vertex of a rectangular captured image. The relationship between the quadrangle photographed with the digital camera 1 and the actual photographing object (white plate 2) will be described with reference to FIG.

In FIG. 7, the UVW coordinate system is a three-dimensional coordinate system of an image obtained by photographing with the digital camera 1. The A (Au, Av, Aw) vector and the B (Bu, Bv, Bw) vector represent the object to be imaged in the three-dimensional coordinate system UV-W.
The S (Su, Sv, Sw) vector indicates the distance between the origin of the three-dimensional coordinate system UV-W and the object to be imaged.

The projection screen shown in FIG. 7 is for projecting an image to be photographed.
If the coordinate system on the projection screen is x, y, the image projected on the projection screen may be considered as the image photographed by the digital camera 1. It is assumed that the projection screen is positioned vertically at a distance f from the W axis. An arbitrary point P (u, v, w) to be photographed is connected to the origin by a straight line, and there is a point where the straight line intersects the projection screen, and the XY coordinate of the intersection is p (x, y). And At this time, the coordinate p is expressed by the following equation 3 by projective transformation.
From Equation 3, the relationship shown in the following Equation 4 is obtained from the relationship between the points P0, P1, P2, P3 and the projection points p0, p1, p2, p3 on the projection screen as shown in FIG.
At this time, the projection coefficients α and β are expressed by the following equation (5).

Next, projective transformation will be described.
An arbitrary point P on the object to be imaged is expressed by the following equation 6 using S, A, and B vectors.
When the relational expression of Expression 4 is substituted into Expression 6, the coordinates x and y are expressed by the following Expression 7.

When this relationship is applied to the affine transformation formula, the coordinates (x ′, y ′, z ′) are expressed by the following equation (8).

  By substituting m and n into Equation 8, the corresponding point (x, y) of the captured image is obtained. Since the corresponding point (x, y) is not necessarily an integer value, the pixel value may be obtained using an image interpolation method or the like.

For m and n, an image size (0 ≦ u <umax, 0 ≦ v <vmax) for outputting the corrected image p (u, v) is given in advance, and a method of adjusting the image according to the image size is considered. It is done. According to this method, m and n are expressed by the following equation (9).
However, the aspect ratio of the corrected image to be created does not match the aspect ratio of the shooting target. Here, the relationship between the corrected image p (u, v) and the values of m and n is expressed by the following Expression 10 from Expression 3, Expression 4, and Expression 5.

If the focal length f of the lens, which is a camera parameter, is known, the aspect ratio k can be obtained according to Equation 10. Therefore, if the image size of the corrected image p (u, v) is (0 ≦ u <umax, 0 ≦ v <vmax), m and n are obtained according to the following equation 11 to obtain the same vertical and horizontal as the object to be imaged. The ratio k can be obtained.
If the camera has a fixed focus, the value of the focal length f of the lens can be obtained in advance. When a zoom lens or the like is present, the value of the focal length f of the lens changes depending on the zoom magnification of the lens. Therefore, a table indicating the relationship between the zoom magnification and the focal length f of the lens is created and stored in advance. Then, the focal length f can be read based on the zoom magnification, and projective transformation can be performed according to Equations 10 and 11.

In order to perform such affine transformation, the image processing device 25 first extracts projection parameters from the captured image to be captured (step S19 in FIG. 5).
Projection parameter extraction processing executed by the image processing device 25 will be described with reference to the flowchart shown in FIG.

  The image processing device 25 extracts coordinate points (rectangular outlines) at the four corners of the captured image from the captured image (step S31). The image processing device 25 extracts a rectangular outline according to the flowchart shown in FIG.

  That is, the image processing device 25 creates a reduced image from the input image in order to reduce the number of operations for image processing. Then, binarization processing is performed on the created reduced image based on a predetermined threshold value to create a reduced binary image (step S41). Note that the preview image generated in step S16 may be used as the reduced image.

Here, the threshold for binarization is obtained as follows.
The image processing device 25 assumes that there is a quadrilateral extraction target at the center of the image, and creates a histogram at the center of the image. Based on the above assumption, the peak value of the created histogram is the luminance value to be cut out, so the image processing apparatus 25 sets a value obtained by subtracting a certain value from this peak value as the threshold for binarization.

  Since this reduced binarized image may include not only the image to be photographed but also the reflected image of the fluorescent lamp that is not the object of photographing, etc., the image processing device 25 first performs this reduction binarization. A labeling process is performed on the image to extract the maximum connected area (step S42).

  The labeling process is a process of assigning a unique value (label) to a connected component (connected pixel group) in an image. The connected component to which the unique value is attached includes a photographing target image that is not a background such as a reflected image of a fluorescent lamp.

  The image processing device 25 further performs a labeling process, and then determines that the area having the same label and the largest area is the shooting target image, and images other than the label area having the largest area are captured at the same time as the shooting target image. It discriminate | determines from the imaging | photography outside image object, and clears pixels other than this label to 0 in a reduced binary image. Details of the labeling process will be described later.

  The image processing device 25 creates an edge image to be imaged from the reduced binarized image after the maximum connected area is extracted by the labeling process (step S43).

The image processing device 25 performs radon conversion in order to perform straight line detection from the edge image to be imaged (step S44). Details of the Radon conversion will be described later.
Using the data obtained by the Radon transform, a straight line parameter included in the edge image is detected (step S45). Details of the straight line detection will be described later.
The image processing apparatus 25 creates a quadrangle that is a candidate for forming the contour of the imaging target from the detected straight line parameter (step S46).

The description returns to FIG. 8 again. The image processing device 25 generates the candidate rectangles in this way, and assigns priorities to the generated rectangles (step S32 in FIG. 8).
The image processing device 25 selects a rectangle according to the priority order, and determines whether or not the selected rectangle has been extracted (step S33).

  If it is determined that the quadrangle could not be extracted (No in step S33), the CPU 30 ends the projection parameter extraction process.

  On the other hand, if it is determined that the quadrangle has been extracted (Yes in step S33), the CPU 30 acquires the extracted quadrangle from the image processing device 25, sends it to the display device 24, and displays the quadrangle preview image on the liquid crystal monitor 12. It is displayed (step S34).

  The CPU 30 determines whether the YES key or the NO key has been pressed based on the operation information transmitted from the operation unit 26 (step S35).

  If the CPU 30 determines that the NO key has been pressed (No in step S35), the image processing device 25 designates the next candidate rectangle (step S36).

  On the other hand, if it is determined that the YES key has been pressed (Yes in step S35), the CPU 30 calculates affine parameters from the extracted quadrangular vertices (step S37).

Next, the projection parameter extraction process will be described more specifically.
An example of the reduced binarized image generated by the image processing apparatus 25 in step S41 is shown in FIG.

  The image processing device 25 extracts a maximum connected region (processing in step S42) using a labeling process for the generated reduced binary image. Details of the labeling process will be described. As described above, in this reduced binarized image, as shown in FIG. 10B, not only the image to be photographed but also the reflected image of the fluorescent lamp is included as an image outside the object to be photographed. There is. For this reason, the image processing device 25 performs a labeling process on the reduced binarized image, and determines an image to be captured.

  In advance, an area for storing a label corresponding to each pixel of the reduced image is secured in the memory 23. The data in this area is called a label image. In the present embodiment, it is assumed that the label numbers added by the labeling process are 1 to 254.

  The image processing device 25 initializes the label image from the binary image. The place where the binary image is 0 is set as the background in the label image, and 0 is written here. On the other hand, when the binary image is 1, the initial label (255) is written in the label image.

  It is assumed that the image processing device 25 sequentially performs the process of scanning the pixels of the label image from the upper left of the label image to the right end in the x direction in the y direction. The image processing device 25 uses each pixel that is not a background pixel as a target pixel, and examines pixel values of eight pixels adjacent to the target pixel in the vertical and horizontal directions. The background pixel is a pixel having a preset background color pixel value (= 0).

  The image processing device 25 performs this labeling process based on the values and labels of eight pixels adjacent to the target pixel in the vertical and horizontal directions, and displays a region in which pixels having pixel values corresponding to the photographic target image are continuous. Ask. The image processing device 25 performs labeling processing according to the following conditions.

(1) Condition 1
As shown in FIG. 11A, the target pixel is not a background color, all four pixels adjacent to the upper and left sides are background pixels, and among the four pixels adjacent to the lower and right sides, the background pixel If the pixel is not included, the image processing device 25 attaches a new label to the target pixel.

(2) Condition 2
As shown in FIG. 11B, the target pixel is not a background color, and there are pixels that are not background pixels among the four pixels adjacent to the upper and left sides, and any one of the pixels has one type of label. In this case, the image processing device 25 attaches the same label as the adjacent pixel to the target pixel.

(3) Condition 3
As shown in FIG. 11C, the target pixel is not a background color, and there are pixels that are not background pixels among the four pixels adjacent to the top and left, and two or more types of labels are attached to a plurality of pixels. The image processing device 25 gives the target pixel the smallest label number of two or more types of label numbers, and stores that the pixels to which these labels are attached are the same connected component. When the image processing device 25 has scanned to the end, the image processing device 25 scans again and re-labels the same connected components stored.

(4) Condition 4
In order to reduce the number of labels at the time of labeling, as shown in FIG. 11D, when all the pixels adjacent to the target pixel are background pixels, the image processing device 25 displays a label on the target pixel as an isolated point. The initial value (= 255) is held without attaching.

  The image processing device 25 labels all connected components according to the conditions 1 to 4 described above. When the labeling is completed, the image processing device 25 detects the connected component label having the largest area among the connected components. The image processing device 25 holds the pixel with this label, assuming that the connected component having the largest area is that of the image to be photographed, and sets the pixel value of the other pixels as those based on the reflected image of the fluorescent lamp. Change to the pixel value of the background pixel.

  Based on such contents, the image processing device 25 executes a labeling process according to the flowchart shown in FIG.

  First, the image processing device 25 initializes all the pixels of the label image based on the binary image data. That is, 0 is written as the background in the label image corresponding to the pixel whose binary image is 0. On the other hand, if the binary image is 1, the initial label is written in the label image (step S51).

  Subsequently, the image processing device 25 sets the upper left pixel (pixel) of the label image as a target pixel (step S52).

  The image processing device 25 determines whether or not the target pixel is the background color (= 0) (step S53). If it is determined that the color is the background color (step S53: Yes), the process jumps to step S61 and proceeds to the process of step S61 described later. On the other hand, when it is determined that the target pixel is not the background color (step S53: No), the image processing device 25 is the pixel adjacent to the upper side or the left side of the eight pixels adjacent to the periphery of the target pixel. It is determined whether or not there is a pixel value other than (step S54).

  If it is determined that the pixels other than the background pixels are adjacent to the upper side or the left side (step S54: Yes), the image processing apparatus 25 determines whether there are a plurality of adjacent pixels with different types of labels. A determination is made (step S55). If it is determined that there are no adjacent pixels with a plurality of types of labels (step S55: No), the process jumps to step S58, and the image processing apparatus 25 performs the process described later.

  If it is determined that the pixels of the plurality of types of labels are adjacent to the upper side or the left side (step S55: Yes), the image processing device 25 selects the label with the smallest number from the plurality of types of labels and labels the target pixel. (Step S56). Further, the image processing device 25 stores in the memory 23 via the CPU 30 that these detected labels indicate the same connected component (step S57), and proceeds to the processing of step S59 described later. .

  On the other hand, if it is determined in step S54 that a pixel having a pixel value other than the background pixel is not adjacent to the upper side or the left side (step S54: No), the image processing apparatus 25 sets the pixel having a pixel value other than the background pixel to the lower side. Alternatively, it is determined whether or not it is adjacent to the right side (step S58). If it is determined that they are not adjacent to each other (step S58: No), the process jumps to step S61 as it is and performs the processing described later. If it is determined that they are adjacent (step S58: Yes), the image processing apparatus 25 performs new labeling on the target pixel (step S59), and then proceeds to step S61 to perform the process described later.

  If it is determined in step S55 that there are no adjacent pixels with a plurality of types of labels on the upper or left side of the target pixel, that is, only adjacent pixels with one type of label exist (step S55: No), the image processing apparatus 25 labels the target pixel with the same type of label as the adjacent pixel (step S60).

  The image processing device 25 determines whether or not one scan has been completed, that is, whether or not processing has been performed from the upper left to the lower right of the image (step S61). If it is determined that the scanning has not ended (step S61: No), the image processing apparatus 25 selects a pixel on the right side of the current pixel with the new pixel as a target pixel. If the target pixel is already positioned on the rightmost side, the image processing device 25 selects the leftmost pixel on the next lower line (step S62). The image processing apparatus 25 returns to step S53 and repeats a series of processes.

  The image processing device 25 scans again from the upper left to the lower right of the image, and this time, the same labeling is performed on the connected component pixels stored in the memory 23 in step S57 (step S63). Then, the image processing apparatus 25 detects the labeled connected component having the largest area (step S64).

  The label having the maximum area detected here is defined as Label_m.

  The image processing device 25 returns the scan target to the reduced binary image, searches for a pixel of 1 in the binary image, and checks whether the label of the pixel of the label image corresponding to the pixel is Label_m. If it is not Label_m, the pixel of the reduced binary image is rewritten to 0, assuming that it is not a shooting target (step S65).

This labeling process will be described more specifically.
As illustrated in FIG. 13, when the pixel p1 is the target pixel by scanning from the upper left of the captured image to the right end in the x direction, the pixel p1 satisfies the condition 1. In this case, the image processing apparatus 25 attaches Label_1 to the pixel p1 as a new label.

  When the pixel p2 is the target pixel, the pixel p2 satisfies the condition 2. The pixel adjacent to the left of the pixel p2 is the pixel p1, and the label of the pixel p1 is Label_1. In this case, the image processing apparatus 25 attaches Label_1 to the pixel p2 as the same label as the pixel p1.

  When the pixel p3 in the next line is the target pixel, the pixel p3 satisfies the condition 1. In this case, the image processing device 25 attaches Label_2 as a new label to the pixel p3.

  When the pixel p4 is the target pixel, the label of the pixel p1 adjacent to the pixel p4 is Label_1, and the label of the pixel adjacent to the left of the pixel p4 is Label_2. For this reason, the pixel p4 corresponds to the condition 3. In this case, the image processing apparatus 25 attaches Label_1 to the pixel p4 as the smallest label number among two or more types of label numbers. Then, the image processing device 25 stores the pixel p1 to which Label_1 is attached and the pixel adjacent to the left of the pixel p4 as the same connected component.

  When the pixel p5 is the target pixel, the pixels adjacent to the pixel p5 are all background pixels, and therefore the pixel p5 satisfies the condition 4. For this reason, the image processing device 25 does not label the pixel p5 with the pixel p5 as an isolated point.

  The image processing device 25 executes such a process for each pixel that is not a background pixel, and scans again when all the processes are completed. When scanning is performed again, the image processing device 25 stores the pixel p1 and the pixel adjacent to the left of the pixel p4 as the same connected component, and therefore re-labels the pixel adjacent to the left of the pixel p4.

  In this way, Label_1 is attached to a connected component that is not a background pixel in this region. Then, Label_1 to Label_4 as shown in FIG. 14 are attached to each connected component of the reduced binarized image shown in FIG.

  Among these, the connected component having the largest area is the connected component to which Label_4 is attached. Accordingly, the image processing device 25 holds the pixel with this label, assuming that the connected component having the maximum area is that of the image to be captured, and sets the other pixels as the reflection image of the fluorescent lamp, etc. And In this way, a reduced binarized image as shown in FIG. 15A obtained by extracting the maximum connected region portion using the labeling process is acquired.

  The image processing device 25 generates an edge image as shown in FIG. 15B from the reduced binarized image from which the maximum connected area has been extracted, using an edge detection filter called a Roberts filter. (Step S42). The Roberts filter is a filter that detects an edge of an image by weighting two 4-neighboring pixels, obtaining two filters Δ1 and Δ2, and averaging them.

FIG. 16A shows the coefficients of the filter Δ1, and FIG. 16B shows the coefficients of the filter Δ2. When the coefficients of the two filters Δ1 and Δ2 are applied to the pixel value f (x, y) of a certain coordinate (x, y), the converted pixel value g (x, y) Represented by

  The edge image shown in FIG. 15B includes a straight line parameter. The image processing device 25 performs a Radon transform from this edge image to detect a straight line parameter (processing in step S43).

The Radon transform is an integral transform that associates n-dimensional data with (n-1) -dimensional projection data. Specifically, as shown in FIGS. 17A and 17B, an r-θ coordinate system rotated by an angle θ from the xy coordinate system with image data f (x, y) is considered. Image projection data p (r, θ) in the θ direction is defined by the following equation (13).

This conversion by Equation 13 is called Radon conversion.
Image data f (x, y) as shown in FIG. 17A is converted into image projection data p (r, θ) as shown in FIG. 17B by Radon transformation.

  According to such a Radon transform, a straight line L in the xy coordinate system as shown in FIG. 18A is represented by ρ = x cos α + y sin α in the polar coordinate system. Since this straight line L is entirely projected on the point p (ρ, α), it is possible to detect the straight line by detecting the peak of p (r, θ). Using this principle, the image processing device 25 generates data p (r, θ) from the edge image by Radon transform.

  Next, the image processing device 25 extracts the peak point from the generated data p (r, θ). For this reason, the image processing apparatus 25 executes this peak point extraction processing according to the flowchart shown in FIG.

The image processing device 25 searches for the maximum value pmax at P (r, θ) (step S71).
The image processing device 25 calculates a threshold value pth = pmax * k (k is a constant value between 0 and 1) (step S72).

The image processing device 25 generates a binary image B (r, θ) from P (r, θ) using pth as a threshold (step S73).
The image processing device 25 performs image labeling of B (r, θ). The number of labels obtained at this time is set to N1 (step S74).

  The image processing device 25 examines r and θ where P (r, θ) has the maximum value in each label area. The image processing device 25 acquires these values as ri, θi (i = 1 to Nl), respectively (step S75). This is a straight line parameter.

  Next, when the image processing apparatus 25 generates a quadrangle candidate using the detected straight line parameter (the process of step S45 in FIG. 9), as shown in FIG. 20A, the four straight lines form a quadrangle. In this case, for example, the opposing straight lines a1 and a3 have two straight lines a2 and a4 other than the opposing straight lines a1 and a3 and intersections p1, p4, p2, and p3, respectively. In this case, the CPU 30 determines that the quadrangle has been extracted (Yes in step S33 in FIG. 8).

  On the other hand, as shown in FIGS. 20B and 20C, if there are no four intersections, the CPU 30 determines that the quadrangle could not be extracted (No in step S33).

Next, the image processing device 25 selects the most suitable rectangle as the one representing the side to be imaged from the extracted rectangle candidates.
Several methods are conceivable. In the present embodiment, the outermost quadrangle is selected from the captured quadrangles. As shown in FIG. 6, the outermost quadrangle is a rectangle that is formed by surrounding a candidate quadrangle with parallel lines of the X and Y axes, and has the largest area.

If the coordinates of the four vertices of the rectangle Ri are (x0, y0), (x1, y1), (x2, y2), and (x3, y3), the outline area Si of the quadrangle is given by expressed.

The image processing device 25 selects this rectangle according to the flowchart shown in FIG.
The image processing device 25 selects one of the candidate number Nr of rectangles (step S81).
The image processing device 25 obtains the area Si of the selected square according to Equation 14 (step S82).

The image processing device 25 decrements the number of candidates Nr (step S83).
The image processing apparatus 25 determines whether the number of candidates Nr has become 0 (step S84).
When it is determined that the number of candidates Nr is not 0 (No in step S84), the image processing device 25 executes the processes in steps S81 to S83 again.

  When it is determined that the number of candidates Nr has become 0 by repeating such processing (Yes in step S84), the image processing device 25 arranges the candidate quadrilateral data in descending order of the obtained area Si. Change (step S85).

  Then, the image processing apparatus 25 sets the first quadrangle as the highest priority quadrilateral outline. Thus, even if there are a plurality of quadrangle candidates, the quadrilateral with the maximum outline is always preferentially selected. In this way, the maximum outline rectangle is preferentially selected. Normally, zoom adjustment and adjustment of the shooting position are performed so that the subject is maximally within the shooting angle of view, and the maximum outline rectangle is selected. This is because the contour of the subject is considered.

  Therefore, in general, it is expected that the contour of the photographing target is automatically extracted. In addition, even if a quadrangle is extracted by mistake, the user generally confirms sequentially from the quadrangle of the maximum outline. For this reason, if a straight line indicating the outline of the imaging target has been extracted, the true imaging target rectangle is selected by sequentially pressing the NO key.

  In this way, using the coordinates (x0, y0), (x1, y1), (x2, y2), (x3, y3) of the four points of the selected quadrangle to be imaged, According to 8, affine parameters that are each element in the matrix shown in Equation 8 can be obtained.

(1) Extraction of Affine Parameter from Image to be Captured Based on such a concept, the image processing device 25 acquires an affine parameter from a rectangular vertex. This process will be described based on the flowchart shown in FIG.

  The image processing device 25 calculates projection coefficients α and β according to Equation 5 from the vertex coordinates (x0, y0), (x1, y1), (x2, y2), and (x3, y3) of the four points of the rectangle. (Step S91).

The image processing device 25 calculates the aspect ratio k of the subject to be photographed according to Equation 10 (step S92).
The image processing device 25 designates the center point (uc, vc) of the image (step S93).
The image processing device 25 compares the maximum image size vmax / umax with the aspect ratio k expressed by Equation 10 (step S94).

  When vmax / umax ≦ k (No in step S94), assuming that the aspect ratio k is not changed, the image processing apparatus 25 determines that the maximum image size umax on the U-axis side (horizontal) is larger than the image size to be captured. Judged to be large. Then, the image processing device 25 obtains the values of m and n in accordance with the condition (1) of Formula 11 so that the maximum image size on the V-axis side matches the image size of the subject (Step S95).

  When vmax / umax> k (Yes in step S94), assuming that the aspect ratio k is not changed, the image processing apparatus 25 determines that the maximum image size vmax on the V-axis side (vertical) is larger than the image size to be captured. Judged to be large. Then, the image processing device 25 obtains the values of m and n according to the condition (2) of Equation 11 so that the maximum image size on the U-axis side matches the image size of the imaging target (step S96).

The image processing device 25 calculates the affine transformation from the calculated vertex coordinates (x0, y0), (x1, y1), (x2, y2), (x3, y3) of the four points of the quadrangle according to Equation 8 A matrix Af is obtained (step S97).
The image processing apparatus 25 acquires the affine parameter A using each element of the affine transformation matrix Af as the affine parameter A (step S98).

  Note that, as in the case where the quadrangle cannot be recognized (No in step S33 in FIG. 8), the image capturing condition or the like is inappropriate, and the projection parameter may not be obtained. In such a case, a warning message as shown in FIG. 23A is displayed on the liquid crystal monitor 12 to appropriately warn the photographer that the area could not be detected, and to the photographer, FIG. It is preferable to change the shooting conditions to the camera shooting setting mode as shown in FIG. Furthermore, it may be preferable to warn again to change the shooting conditions.

  When the proper projection parameter cannot be obtained as described above, the CPU 30 performs a warning process according to the flowchart shown in FIG. 24 (step S27 in FIG. 5).

  The CPU 30 controls the display device 24 to display a warning text as shown in FIG. 23A on the liquid crystal monitor 12 (step S101).

  The CPU 30 determines which of the YES key and the NO key has been pressed based on the operation information transmitted from the operation unit 26 (step S102).

  If it is determined that the NO key has been pressed (No in step S102), the CPU 30 ends the warning process.

  On the other hand, if it is determined that the YES key has been pressed (Yes in step S102), the CPU 30 switches to the shooting mode (step S103) and ends this warning process.

(2) Image Conversion Using Extracted Affine Parameters Next, an image processing method for creating a corrected image using the obtained affine parameters will be described.
First, when projective transformation or other affine transformation is performed using affine parameters, as shown in FIG. 25, the point p (x, y) of the original image is transformed by projective transformation using the transformation matrix Ap (after). Assume that it corresponds to the point P (u, v) of the image. In this case, it is better to obtain the point p (x, y) of the original image corresponding to the point P (u, v) of the converted image than to obtain the point P of the converted image corresponding to the point p of the original image. preferable.

Note that when obtaining the coordinates of the point P of the converted image, an interpolation method based on the bilinear method is used. In the bilinear interpolation method, the coordinate point of the other image (transformed image) corresponding to the coordinate point of one image (original image) is searched, and the (pixel) values of four points around the coordinate point of the one image are found. Is a method for obtaining a (pixel) value of a point P (u, v) of the converted image. According to this method, the pixel value P of the point P of the converted image is calculated according to the following Expression 15.

In order to obtain the point p (x, y) of the original image corresponding to the point P (u, v) of the converted image, the image processing device 25 executes the process of the flowchart shown in FIG.
The image processing device 25 initializes the pixel position u of the converted image to 0 (step S111).

The image processing device 25 initializes the pixel position v of the converted image to 0 (step S112).
The image processing device 25 substitutes the pixel position (u, v) of the converted image using the affine parameter A obtained by Equations 5 and 8, and according to Equation 2, the pixel position (x, y) of the original image Is obtained (step S113).

The image processing device 25 obtains the pixel value P (u, v) from the obtained pixel position (x, y) according to the formula 15 by the bilinear method (step S114).
The image processing device 25 increments the coordinate v of the corrected image by one (step S115).

  The image processing device 25 compares the coordinate v of the corrected image with the maximum value vmax of the coordinate v, and determines whether or not the coordinate v of the corrected image is greater than or equal to the maximum value vmax (step S116).

  When it is determined that the coordinate v is less than the maximum value vmax (No in step S116), the image processing device 25 executes steps S113 to S115 again.

  When it is determined that the coordinate v has reached the maximum value vmax by repeating the processes in steps S113 to S115 (Yes in step S116), the image processing device 25 increments the coordinate u of the corrected image by one ( Step S117).

The image processing device 25 compares the coordinate u with the maximum value umax of the coordinate u, and determines whether or not the coordinate u is equal to or greater than the maximum value umax (step S118).
When it is determined that the coordinate u is less than the maximum value umax (No in step S118), the image processing device 25 executes the processes of steps S112 to S117 again.

  When it is determined that the coordinate u has reached the maximum value umax by repeating the processing of steps 112 to S117 (Yes in step 118), the image processing device 25 ends the image conversion processing.

(3) Image Conversion Adjustment Next, adjustment (step S22 in FIG. 5) performed on an image that has been once converted will be described.
If the extracted quadrangular vertex coordinates include some errors and the like, the result of projection with the obtained affine parameters may be undesirable as shown in FIG. For this reason, the digital camera 1 according to the present embodiment is configured so that the user can adjust the projective transformation in order to adjust the image once converted and obtain an image as shown in FIG. Has been.

  When the user operates the projection conversion key and the rotation correction key of the operation unit 26, the operation unit 26 transmits this operation information to the CPU 30 in response to the user's operation. The CPU 30 determines this operation information and controls the image processing device 25 according to the determination result.

  As shown in FIG. 28, when the interpolation pixel Q (u ′, v ′) of the corrected image is obtained, the inverse transformation Ai is performed on the interpolation pixel Q (u ′, v ′) to obtain the interpolation pixel Q. A corrected image P (u, v) corresponding to (u ′, v ′) is obtained, and further the inverse transformation is performed on the corrected image P (u, v) to obtain p (x, y) of the original image. Then, pixel interpolation is performed on the image p (x, y). Further, when performing two-stage image conversion such as projective conversion and enlargement conversion, a conversion matrix obtained by combining the two conversions is obtained, and the original image is converted at a time. In this case, an image can be obtained at a higher speed than when conversion is performed twice, and image deterioration can be reduced.

The rotation inverse transformation matrix Ar for obtaining the image before conversion from the image after conversion obtained by rotating the image before conversion by the angle θ about the X axis and the Y axis is expressed by the following equation (16). expressed.
An enlargement matrix Asc in the case of obtaining an image before conversion from an image after conversion obtained by enlarging the image before conversion by Sc magnification around the X axis and the Y axis is expressed by the following Expression 17. The

  Note that once an image is enlarged, a rounding error may be processed by adjusting or calculating an affine parameter. For this reason, before enlarging the image, it is necessary to restore the original affine parameters at the same magnification.

The movement matrix As for obtaining the image before conversion from the image after conversion obtained by moving the image before conversion in the X and Y directions by Tx and Ty, respectively, is expressed by the following equation (18). Is done.

The projection effect matrix Ap in the case of acquiring the image before conversion from the image after conversion obtained by inclining the image before correction in the X and Y directions by α and β, respectively, is given by expressed.
When performing two-stage inverse transformation, the inverse transformation matrix A is expressed by the following equation (20).

  In this embodiment, the projection effect parameters α and β are set in increments of 0.1, and the angle correction parameter θ is set in increments of 1 degree. However, the adjustment range of each parameter can be determined while checking the actual correction effect.

The image conversion adjustment process executed based on this concept will be described based on the flowchart shown in FIG.
The CPU 30 sets the center coordinates Xc and Yc of the image (step S121).
The CPU 30 determines whether or not the projection correction key has been pressed based on the operation information transmitted from the operation unit 26 (step 122).
If it is determined that the projection conversion key has been pressed (Yes in step S122), the CPU 30 has pressed one of the upper reduction key 111, the lower reduction key 112, the right reduction key 113, and the left reduction key 114 as the projection conversion key. The key type is determined (steps S123 to S126).

  When determining that the pressed projective transformation key is the up-reduction key 111, the CPU 30 assigns α = 0.1 and β = 0 to the projective effect matrix Ap shown in Equation 19 to obtain an inverse transformation matrix Ai = Ap. Is acquired (step S123).

  If it is determined that the pressed projective transformation key is the lower reduction key 112, the CPU 30 assigns α = −0.1 and β = 0 to the projective effect matrix Ap shown in Equation 19 to obtain an inverse transformation matrix Ai = Ap is acquired (step S124).

  If it is determined that the pressed projective transformation key is the right reduction key 113, the CPU 30 substitutes α = 0 and β = 0.1 for the projection effect matrix Ap shown in Equation 19 to obtain an inverse transformation matrix Ai = Ap. Is acquired (step S125).

  When determining that the pressed projective transformation key is the left reduction key 114, the CPU 30 assigns α = 0 and β = −0.1 to the projective effect matrix Ap shown in Equation 19, respectively, and the inverse transformation matrix Ai = Ap is acquired (step S126).

When it is determined that the projection conversion key has not been pressed (No in Step 122), the CPU 30 determines whether or not the rotation correction key has been pressed (Step S127).
If it is determined that the rotation correction key has been pressed (Yes in step S127), the CPU 30 determines the type of the rotation correction key that has been pressed (steps S128, S129).

  When determining that the pressed rotation correction key is the right rotation key 115, the CPU 30 substitutes θ = −1 into the rotation inverse transformation matrix Ar shown in Expression 16 to obtain the inverse transformation matrix A = Ar. (Step 128).

  If it is determined that the pressed rotation correction key is the left rotation key 116, the CPU 30 substitutes θ = 1 for the rotation inverse transformation matrix Ar shown in Equation 16 to obtain the inverse transformation matrix Ai = Ar ( Step 129).

  Further, when it is determined that the projective transformation key or the rotation correction key has been pressed and the inverse transformation matrix Ai is set (steps S123 to S126, S128, S129), the CPU 30 obtains the inverse transformation matrix A according to Equation 20 ( Step S130).

The CPU 30 supplies the obtained conversion matrix A to the image processing device 25, and controls the image processing device 25 to perform image conversion based on the inverse conversion matrix A. The image processing device 25 performs image conversion by affine transformation based on the supplied inverse transformation matrix A (step S131), and ends this image transformation adjustment processing.
On the other hand, if it is determined that the rotation correction key has not been pressed (No in step S127), the CPU 30 ends the adjustment processing of the image conversion as it is.

  By performing image conversion by the above-described method using the affine parameters obtained in this manner, the image can be further manually adjusted with respect to the corrected image.

  For example, when adjusting an image as shown in FIG. 27, since the image is distorted leftward, first, when the right rotation key 115 is pressed, the image processing device 25 rotates the image to the right. If the right rotation key 115 is pressed as it is and the character string is correctly displayed and the right rotation key 115 stops being pressed, the image processing device 25 stops the right rotation of the image.

  Next, since the left side of the image is larger than the right side, when the left reduction key 114 is pressed, the image processing device 25 adjusts the left and right sides of the image. If the left reduction key 114 is pressed and the left and right balances are aligned and the pressing of the left reduction key 114 stops, the image processing device 25 stops the left reduction process.

  In this method, an image can be obtained by performing only one image conversion on the original image, so that an image with better image quality than rotating and projecting the image once subjected to projection correction can be obtained. Can do.

(3) Extraction of image effect correction parameters relating to luminance or color difference and image effect processing Next, processing for extracting image effect correction parameters from the image thus obtained, and image effects performed using these parameters Processing will be described. The image effect process is a process for obtaining a clearer image.

  The image thus obtained is an image obtained by cutting out the white board 2 or the document as a result. When performing image effects such as histogram correction, more effective parameters are expected to be obtained by obtaining parameters from the cut out image than by obtaining correction parameters from the original image and performing correction.

  In the present embodiment, a histogram is created from luminance (Y) from the corrected image data, and image effect processing is performed according to the histogram.

  The image effect correction parameters are variables necessary for image effect processing such as the maximum value, minimum value, and peak value of the luminance histogram.

  In order to extract the image effect correction parameter, it is necessary to generate a luminance histogram. First, processing necessary for extracting image effect correction parameters will be described.

  The luminance histogram indicates the distribution of luminance values (Y) existing in the image, and is generated by counting the number of pixels for each luminance value. FIG. 30 shows an example of a luminance histogram. In FIG. 30, the horizontal axis indicates the luminance value (Y), and the vertical axis indicates the number of pixels. In order to correct the image effect, it is necessary to obtain a maximum value (Ymax), a minimum value (Ymin), and a peak value (Ypeak) as image effect correction parameters.

  The maximum value counts the number of pixels for each luminance value, and is a value indicating the maximum luminance among luminance values having a predetermined number or more set in advance, and the minimum value is a predetermined number or more This is a value indicating the minimum luminance among the luminance values having the counted value. The peak value is a luminance value that maximizes the count value. The peak value is considered to indicate the luminance value of the background color to be photographed.

  Note that in order to obtain an image with excellent visibility by correcting the image effect, the correction effect differs depending on the background color of the shooting target. Therefore, it is necessary to change the correction method of the image effect depending on the background color of the shooting target. For this reason, it is necessary to determine the background color of the shooting target. The background color to be photographed is determined from the peak values of the luminance histogram and the color difference histogram.

  Here, the background color to be photographed is classified into three. The first is a case where the background color is white, such as a whiteboard or notebook. The second is a case where the background color is black, such as a blackboard. The third is a case where the background color is other than white or black, such as a magazine or a pamphlet.

Specifically, the background color of the shooting target is determined according to the following discriminant.
(2-a) White Determination Condition The white determination condition is expressed by the following Expression 21, and when the condition shown in Expression 21 is satisfied, the background color of the shooting target is determined to be white (W).
(2-b) Black Determination Condition The black determination condition is expressed by the following Expression 22, and when the condition shown in Expression 22 is satisfied, the background color of the shooting target is determined to be black (b).

  If the conditions shown in Equations 21 and 22 are not satisfied, the background color to be imaged is determined to be color (C). For example, the color threshold is set to 50 and 128, the white determination threshold is set to 128, and the black determination threshold is set to 50, for example.

Based on such an idea, the image processing device 25 executes image effect correction parameter extraction processing according to the flowchart shown in FIG.
The image processing device 25 counts the number of pixels having each luminance (Y) value, and generates a luminance histogram as shown in FIG. 30 (step S141).

  The image processing device 25 acquires the maximum value (Ymax), the minimum value (Ymin), and the peak value (Ypeak) of the luminance from the generated luminance histogram (step S142).

The image processing device 25 discriminates the background color of the subject to be photographed from the peak value (Ypeak) of the luminance histogram according to the judgment conditional expressions shown in Equations 21 and 22 (step S143).
The image processing device 25 stores the image effect correction parameter and the background color data to be photographed in the memory 23 (step S144).

Next, the image processing device 25 performs image effect processing (step S24 in FIG. 5) using the image effect correction parameters extracted in this way.
As described above, in order to effectively perform the image effect processing, it is necessary to change the processing contents depending on the background color.

  When the background color is white, such as a whiteboard or notebook, luminance conversion as shown in FIG. When the background color is black, such as a blackboard, luminance conversion as shown in FIG. 32B is performed. When the background color is other than white or black, such as a magazine or pamphlet, the conversion shown in FIG. 32C is performed. In FIGS. 32A, 32B, and 32C, the horizontal axis indicates the input value of the pixel value, and the vertical axis indicates the output value of the pixel value.

  When the background color is white, as shown in FIG. 32A, the inclination angle of the luminance conversion line is changed with the peak value as a boundary. The predetermined luminance value is set to 230, for example, and the input luminance peak value is raised to the luminance value 230. Then, the maximum value is raised to the maximum brightness. Therefore, the luminance conversion line is represented by two line segments as shown in FIG.

  When the background color is black, luminance conversion is performed so that the peak value becomes a certain luminance value (20), as shown in FIG. Also in this case, as shown in FIG. 32B, the luminance conversion line is represented by two line segments.

  When the background color is a color other than white or black, as shown in FIG. 32C, the minimum value or less and the maximum value or more are cut and expressed as one line segment as in the normal enlargement process. The luminance conversion line is set as follows.

  It should be noted that a conversion table of background luminance (Y) and output (Y ′) may be set in advance and stored in the memory card 31 as described above. According to the created conversion table, each output value is obtained from the input pixel value, and image effect processing is performed. In the image thus converted, bright pixels are brighter and dark pixels are darker, so that the luminance distribution is widened and the image is excellent in visibility.

Based on such a concept, the image processing apparatus 25 executes image effect processing according to the flowchart shown in FIG.
The image processing device 25 reads out the stored image effect correction parameters from the memory 23 (step S151).

The image processing device 25 determines whether or not the background is white (step S152).
When it is determined that the background is white (Yes in step S152), the image processing device 25 performs luminance conversion as illustrated in FIG. 32A so that the background is whiter and the visibility is improved. The brightness histogram is adjusted (step S153).

  If it is determined that the background is not white (No in step S152), the image processing apparatus 25 determines whether the background is black (step S154).

  When it is determined that the background is black (Yes in step S154), when the background is black, the image processing device 25 performs luminance conversion as illustrated in FIG. 32B and adjusts the luminance histogram (step). S155).

  If it is determined that the background is not black (No in step S154), the image processing device 25 performs luminance conversion as shown in FIG. 32C and performs histogram adjustment according to the background color of the shooting target ( Step S156).

In the digital camera 1 of the present embodiment, the mode can be set not only to the shooting mode but also to the playback mode.
When the playback key of the operation unit 26 is pressed, the operation unit 26 transmits this operation information to the CPU 30.

When the CPU 30 determines that the mode is the playback mode based on the operation information, the CPU 30 executes the playback process according to the flowchart shown in FIG.
The CPU 30 selects one image selected by the user from the image files recorded in the memory card 31 (step S161).

The CPU 30 reads the selected image file from the memory card 31 and writes it in the memory 23 (step S162).
The CPU 30 creates a reduced image from the read image (step S163).

  The CPU 30 writes in the preview image storage area on the memory 23 (step S164). Thus, the reduced image is displayed from the display device 24 as in the photographing mode. The user can confirm the reproduced image by viewing this display.

If the user visually recognizes this display and presses the NO key to view another image, the operation unit 26 transmits this operation information to the CPU 30.
The CPU 30 determines whether or not the next image has been designated according to the operation information (step S165).

  When it is determined that the next image has been designated (Yes in step S165), the CPU 30 selects the next image (step S166) and executes steps S162 to S164 again.

When it is determined that the next image has not been designated (No in step S165), the CPU 30 determines whether or not the reproduction mode has ended (step S167).
If the play key is not pressed (No in step S167), CPU 30 determines again whether the next image has been designated (step S165).

  On the other hand, when the playback key is pressed, the CPU 30 receives this operation information from the operation unit 26 and determines that the playback mode has ended (Yes in step S167), and ends the playback process.

Next, writing parameters to the file header will be described.
When both the original image and the corrected image are recorded as JPEG format images, the file name of the original image is recorded in the header (option data) area of the image file. When it is desired to correct the corrected image again, if the corrected image file is corrected, an image with little deterioration can be generated by generating the corrected image file from the original image. This image editing may be performed in the digital camera 1. However, image editing can also be performed on a computer. If image editing is performed on a computer, it is expected that more advanced image correction can be performed.

  Further, by recording the parameters when image processing is performed in the header, the computer can easily create a corrected image again using the parameters for the original image. For this reason, it becomes easy for the user to make a change immediately from the previous corrected image.

As shown in FIG. 35, the header information includes the type of data (data name), the number of bytes of each data, and the contents thereof.
The CPU 30 sequentially stores the header information in the header information storage area, the image file name of the original image data, the image size of the corrected image, the affine parameters, and an input / output data set for creating a histogram table.

  The input / output data set is a set of data indicating the relationship between the input data and the output data, and is arranged in order from the smallest of the input data for each change point where the slope of the straight line indicating the input / output relationship changes. This is a set of output data.

If the input data at the change point is x1, x2, ..., xm in order from the smallest, and the corresponding output data is y1, y2, ..., ym, the data set is (x1, y1), ( x2, y2),..., (xm, ym). In this case, if the input data x is xi <x <xi + 1, the output data y is expressed by the following equation (23).

  For example, when the relationship between the input data and the output data is as shown in FIG. 32A, the CPU 30 determines (0, 0), (minimum value, 0), (peak value, constant luminance value ( = 230)), (maximum value, 255), and (255, 255) are stored in the memory card 31.

  Similarly, when the relationship between input and output data is as shown in FIG. 32B, the CPU 30 determines (0, 0), (minimum value, 0), (peak value, constant luminance value (= 20). )), (Maximum value, 255), and (255, 255) are stored in the memory card 31.

  When the relationship between the input and output data is as shown in FIG. 32C, the CPU 30 determines (0, 0), (minimum value, 0), (maximum value, 255), (255, 255). Four data sets are stored in the memory card 31.

  By recording such a brightness value data set, various histograms can be described. The CPU 30 records such data on the memory card 31 simultaneously with the recording of the corrected image data (step S26 in FIG. 5).

Next, correction image editing processing will be described.
In some cases, after image data is stored once, it is desired to correct the image data again. In this case, as described above, it is preferable to collectively correct the original image data in that image deterioration is small.

When the user operates the control key to designate an image editing mode, the CPU 30 executes an image editing process according to the flowchart shown in FIG.
The CPU 30 selects a corrected image according to the operation information transmitted from the operation unit 26 (step S171).

The CPU 30 reads header information from the header information storage area of the memory card 31 (step S172).
The CPU 30 reads the original image (step S173).
The CPU 30 sends the read corrected image, header information, and original image to the image processing device 25, and controls the image processing device 25 to correct again (step S174).

  The image processing device 25 acquires the corrected image, the header information, and the original image from the CPU 30, creates a projective conversion image (step S175), and performs image effect processing (step S176). The image processing device 25 sends the data of the created projective transformation image to the display device 24, and the display device 24 displays this image on the liquid crystal monitor 12.

  When the user operates the projection conversion key and the rotation correction key, the operation unit 26 transmits this operation information to the CPU 30. The CPU 30 acquires this operation information and controls the image processing apparatus 25 to perform the projection manual correction process.

The image processing apparatus 25 performs a projection manual correction process (image conversion adjustment process) according to the flowchart shown in FIG. 29 (step S177).
The image processing device 25 records the corrected image subjected to the projection manual correction process together with the header information on the memory card 31 (step S178), and ends the editing process.

  As described above, according to the present embodiment, the image processing device 25 obtains a region in which pixels having the same pixel value as that of the image to be captured are obtained by performing a labeling process, and the area of the obtained regions is the area. The image of the area where the maximum is set as the image to be captured. In addition, the image processing device 52 changes the pixel value of the image in the region smaller than the region of the shooting target image to the pixel value of the background pixel, and generates an edge image of the shooting target image.

Therefore, even if a reflected image such as a fluorescent lamp or the fluorescent lamp itself is reflected at the same time by photographing a projected image projected on a white plate by a projector, an edge other than a quadrangular subject appears on the edge image of the document. Never appear. For this reason, the outline of the subject can be accurately detected, and the image to be captured can be acquired correctly.
This process is also effective when photographing a white document placed on a table with low brightness and correcting projection distortion.

  In addition, by labeling so that isolated pixels that are the background 8 pixels are not labeled, an increase in the number of labels due to noise at the isolated points can be prevented, and the memory used for calculation is reduced. Image processing.

  Further, when the image processing device 25 performs the projection conversion and the image effect processing, the corrected image data is recorded in the image file of the memory card 31, and the CPU 30 stores the original image data name, the obtained projection parameter, and The image processing correction parameters are stored in the image file of the memory card 31. Therefore, when image editing is performed again, image editing can be easily performed and the image can be corrected again.

  The image processing device 25 acquires an outline from the image of the shooting target (white plate 2), acquires the shape of the shooting target, obtains a projection parameter from the vertex position of the shooting target quadrangle, and obtains the image of the shooting target. Changed to projective transformation.

  Therefore, if the object to be imaged is a rectangle, the distortion of the image can be automatically corrected easily. As a result, an image with high legibility that is equal to the original image to be photographed can be obtained.

  Further, assuming that the subject object exists in the center of the image and obtaining the binarization threshold value from the histogram only in the center of the image, it is possible to increase the efficiency of the arithmetic processing and perform high-speed image processing.

  Further, even when the shooting target is a quadrangle whose aspect ratio is unknown and the shooting target is shot from any viewpoint / posture, the distortion of the image is corrected by using the focal length of the digital camera 1. can do.

  In addition, since a reduced image having a low resolution used for preview or the like is used to obtain the projection parameters, the number of operations can be reduced, the operation processing can be performed efficiently, and the image processing can be performed at high speed.

  In addition, when a plurality of rectangles are extracted from the captured image, the largest outline rectangle is preferentially selected, and the rectangle is selected in order from the larger one by the NO key, the cursor key, or the like. Therefore, even if there are a plurality of contour candidates for the photographing target, the contour of the photographing target can be quickly acquired, and as a result, a highly readable image can be obtained by a simple method.

  In addition, since the image processing correction parameters are obtained from the distortion-corrected image, better image effect correction can be performed.

In carrying out the present invention, various forms are conceivable and the present invention is not limited to the above embodiment.
For example, in the present embodiment, the labeling process is performed on the reduced binarized image. However, for example, if a range is set for the pixel value of the photographic image so as to correspond to the pixel value of the photographic target image and the labeling process is performed, the image to be subjected to the labeling process is not necessarily a binarized image. There is no need.

  Further, it is not necessary to obtain a region in which pixels having pixel values corresponding to the image to be captured are continuous by the labeling process as in the above embodiment. For example, the pixel values of all the pixels are obtained, and among the obtained pixels, pixels having pixel values corresponding to the image to be photographed are grouped to determine a region where pixels having pixel values corresponding to the image to be photographed are continuous. You may make it ask.

  In the above embodiment, the parameters used to create the corrected image are stored in the corrected image file. However, this parameter can also be stored in the original image file. If the parameters are saved in the original image file, the corrected image can be easily created from the original image without saving the corrected image.

  As shown in FIG. 37, the header information in this case includes only the image size, projection parameters, and image effect correction parameters.

Further, when performing the photographing process, the digital camera 1 immediately performs the projection parameter extraction process without storing the converted image. The contents of the photographing process (2) are shown in the flowchart of FIG.
The image processing device 25 creates a projection corrected image (steps S181 to 189), performs image conversion adjustment, extraction of image effect correction parameters, and image effect processing (steps S190 to S191).

The CPU 30 creates header information shown in FIG. 37 (step S192).
The image processing device 25 compresses the original image data, and the CPU 30 records the created header information together with the compressed original image data in the memory card 31 (steps S193 and 194).

Also, the image editing process is different from the process shown in FIG. That is, the image editing process is performed not on the corrected image but on the original image. The contents of this image re-editing process are shown in the flowchart of FIG.
The CPU 30 selects the original image and reads header information from the original image file (steps S201 and S202).

The image processing device 25 immediately creates a projection corrected image and performs image effect processing (steps S203 and S204).
The CPU 30 and the image processing device 25 perform a projection manual correction process. After the correction is completed, the CPU 30 creates header information based on the changed projection correction parameter, and records the created header information in the original image file again. (Steps S205 to S207).

  In the above embodiment, the image correction is performed by the digital camera 1. However, image correction can also be performed by a computer. In this case, a computer is connected to the computer interface unit 27, and the computer executes image editing processing according to the flowchart shown in FIG. If the computer performs such image editing processing, the computer includes a mouse and the like, and it becomes easier to input operation information than the digital camera 1, so that the operability is improved. Further, since the display device of the computer is generally larger than the liquid crystal monitor 12 of the digital camera 1, the image can be visually confirmed in detail and image correction can be performed with high accuracy.

  In the above embodiment, a warning is given when a quadrangle cannot be acquired. However, instead of issuing a warning, the photographed image can be displayed and the user can designate four rectangular points using a control key or the like. Then, the affine parameters can be obtained using the designated four points.

  In the above embodiment, the binarization threshold is set based on the peak value of the histogram at the center of the image. However, the threshold setting method is not limited to this. For example, binarization may be performed using a predetermined threshold. By performing processing based on a certain standard as described above, it is possible to stably acquire an image of a document regardless of the characteristics of a subject to be photographed.

  In the above embodiment, the program is described as being stored in advance in a memory or the like. However, a program for causing a computer to operate as the whole or a part of the apparatus or to execute the above-described processing is a flexible disk, a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), The information may be stored in a computer-readable recording medium such as an MO (Magneto Optical disk) and distributed, installed in another computer, operated as the above-described means, or the above-described steps may be executed.

  Furthermore, the program may be stored in a disk device or the like included in a server device on the Internet, and may be downloaded onto a computer by being superimposed on a carrier wave, for example.

It is explanatory drawing which shows a state when image | photographing a white board with the digital camera which concerns on embodiment of this invention. It is a block diagram which shows the structure of the digital camera shown in FIG. It is explanatory drawing for demonstrating the function of the image processing apparatus shown in FIG. It is explanatory drawing for demonstrating each key with which the operation part shown in FIG. 2 is provided. It is a flowchart which shows the content of the imaging | photography process which a digital camera performs. It is explanatory drawing which shows the square of the clipping object of the image processing apparatus shown in FIG. It is explanatory drawing for demonstrating the basic concept of extraction of a projection parameter, and an affine transformation. It is a flowchart which shows the content of the projection parameter extraction process which the image processing apparatus shown in FIG. 2 performs. It is a flowchart which shows the content of the square outline extraction process which the image processing apparatus shown in FIG. 2 performs. (A) is explanatory drawing which shows an example of the reduction | restoration binarization image which the image processing apparatus shown in FIG. 2 acquired, (b) is an image of the image contained in the reduction | restoration binarization image of (a). It is explanatory drawing. It is explanatory drawing of the labeling process which the image processing apparatus shown in FIG. 2 performs, (a)-(d) shows the conditions 1-4 in the case of a labeling process, respectively. It is a flowchart which shows the content of the labeling process which the image processing apparatus shown in FIG. 2 performs. It is explanatory drawing which shows the specific content of the labeling process which the image processing apparatus shown in FIG. 2 performs. It is explanatory drawing which shows the result of having performed the labeling process which the image processing apparatus shown in FIG. 2 performs. (A) is a reduced binarized image obtained after labeling, and (b) is an explanatory diagram showing an edge image generated based on the reduced binarized image shown in (b). It is explanatory drawing which shows the coefficient of a Roberts filter. It is an image figure for demonstrating the principle of radon conversion. It is explanatory drawing for demonstrating the operation | movement which acquires the data of a polar coordinate system by radon-transforming the straight line of a X, Y coordinate system. It is a flowchart which shows the content of the peak point detection process from the data of the polar coordinate system which the image processing apparatus shown in FIG. 2 performs. It is explanatory drawing which shows the view which detects a square from the straight line which detected and extracted the peak point. It is a flowchart which shows the content of the selection process of the detected square which the image processing apparatus shown in FIG. 2 performs. It is a flowchart which shows the content of the acquisition process of the affine parameter which acquires an affine parameter from the square vertex performed by the image processing apparatus shown in FIG. It is explanatory drawing which shows the content of the warning when a square cannot be extracted. It is a flowchart which shows the content of the warning process which CPU shown in FIG. 2 performs. It is explanatory drawing for demonstrating the inverse transformation for obtaining the original image from the image after projective transformation. It is a flowchart which shows the content of the image conversion process by the affine transformation which the image processing apparatus shown in FIG. 2 performs. It is explanatory drawing which shows the example which could not correct | amend the distortion of an image by projective transformation. It is explanatory drawing for demonstrating the correspondence with an original image, a projection conversion image, and the enlarged projection conversion image. 3 is a flowchart showing the contents of image conversion adjustment processing executed by the image processing apparatus shown in FIG. 2. It is explanatory drawing which shows an example of a brightness | luminance histogram. 3 is a flowchart showing the contents of image effect correction parameter extraction processing executed by the image processing apparatus shown in FIG. 2. It is explanatory drawing which shows an image effect process, (a) shows an image effect process in case a background color is white, (b) shows an image effect process in case a background is black, (c), The image effect processing when the background is other than white or black is shown. It is a flowchart which shows the content of the image effect process which the image processing apparatus shown in FIG. 2 performs. It is a flowchart which shows the content of the reproduction | regeneration processing which CPU and image processing apparatus which are shown in FIG. 2 perform. It is explanatory drawing which shows the content of header information. 3 is a flowchart showing the content of a correction image editing process executed by the CPU and image processing apparatus shown in FIG. It is explanatory drawing which shows the content of the header information in the case of preserve | saving only an original image. 3 is a flowchart showing the contents of a photographing process (2) executed by the CPU and image processing apparatus shown in FIG. 2 when only the original image is stored. FIG. 3 is a flowchart showing details of a correction image re-editing process executed by the CPU and the image processing apparatus shown in FIG. 2 when only the original image is stored.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Digital camera, 2 ... White board, 11 ... Shooting lens part, 12 ... Liquid crystal monitor, 13 ... Shutter button, 21 ... Optical lens apparatus, 22 ... Image sensor, 23 ... Memory, 24 ... Display device, 25 ... Image processing device, 30 ... CPU, 31 ... Memory card

Claims (8)

In an image processing apparatus that acquires an image of a target shooting target from a shot image obtained by shooting and performs image processing,
An image acquisition unit that obtains an area in which pixels having pixel values corresponding to pixel values of the image to be photographed are continuous, and obtains an image of an area having the maximum area among the obtained areas as the image to be photographed; ,
An image processing unit that performs image processing of the shooting target image acquired by the image acquisition unit,
An image processing apparatus. The image acquisition unit scans each pixel of the captured image, acquires a pixel having a pixel value corresponding to a pixel value of the captured image as a target pixel, and acquires the acquired target pixel and a pixel adjacent to the target pixel Determining the region based on the correspondence with the pixel value of
The image processing apparatus according to claim 1. The image acquisition unit determines that an image of an area smaller than the area of the image to be imaged among areas formed by the continuous pixels is an image outside the object to be imaged taken together with the object to be imaged, and Changing the pixel value of the outside image to the pixel value of the background pixel set in advance, and acquiring the shooting target image;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. Comparing a pixel value of each pixel of the photographed image with a predetermined threshold, and a binarized image creating unit that creates a binarized image of the photographed image based on a comparison result,
The image acquisition unit acquires a pixel having a pixel value corresponding to the shooting target image from the binarized image created by the binarized image creation unit.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. When the image acquisition unit scans each pixel of the captured image to acquire the target pixel, the image acquisition unit refers to the pixel value of the pixel adjacent to the acquired target pixel, and the already scanned adjacent pixel is preset. If the pixel value of the background pixel has a unique value, the target pixel is newly assigned a unique value, and if the adjacent pixel already has a unique value, the neighbor pixel is adjacent to the target pixel. By assigning the same value as the pixel, the correspondence relationship between the target pixel and the pixel value of the pixel adjacent to the target pixel is obtained, and when all the pixels of the photographed image are scanned, each pixel is assigned a unique value. Determine the region by referring to the value;
The image processing apparatus according to claim 2. The image processing unit
A shape acquisition unit that acquires the contour of the image to be captured acquired by the image acquisition unit and acquires the shape of the image;
A projection parameter acquisition unit for associating the shape of the imaging target image acquired by the shape acquisition unit with the shape of the imaging target to obtain a projection parameter indicating a relationship between the imaging target and the imaging target image;
An image conversion unit that performs image conversion of the shooting target image using the projection parameter obtained by the projection parameter acquisition unit,
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. An image processing method of a photographing apparatus for photographing a photographing object,
Obtaining a region in which pixels having pixel values corresponding to pixel values of a target image to be photographed are continuous from a photographed image obtained by photographing;
When there are a plurality of obtained regions, obtaining an image of the region having the largest area as the image to be captured;
Performing image processing of the acquired image to be photographed,
An image processing method for a photographing apparatus. On the computer,
A procedure for obtaining a region in which pixels having pixel values corresponding to pixel values of a target image to be photographed are continuous from a photographed image obtained by photographing,
In the case where there are a plurality of obtained areas, a procedure for acquiring an image of an area with the largest area as the image to be captured,
A procedure for performing image processing of the acquired image to be imaged;
A program for running