JPH11238123A - Image correction processing method, image correction processor, digital camera and information recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correct distortions of images caused by the fact that an image pickup plate and an object plane are not parallel. SOLUTION: In a digital camera, image elements for estimating the relative angle of an image pickup plate and an object plane are instructed on a display screen where a photographed image is displayed (205) and relation between the image elements is instructed (215). The image elements are segmented, attribute information is extracted (215) and the relative angle is calculated from the relation of the image elements and the attribute information (220). The distortion is corrected by executing a geometrical operation to the photographed image based on the relative angle (230) and the corrected photographed image is stored/outputted (250).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for correcting a geometric distortion caused by a non-parallel relationship between an image pickup plate included in an image of a subject and a plane of the subject. The present invention relates to a digital camera that takes into account various correction processes.

[0002]

2. Description of the Related Art Flat subjects such as digital cameras,
For example, drawings affixed to walls, letters and figures written on a blackboard,
When a signboard or timetable is photographed, the image pickup board (CC
D imaging plate) and the plane of the subject are generally not parallel, so that even if the distortion of the lens is zero, geometric distortion uniquely determined by the relative angle between the plane of the subject and the imaging plate occurs in the captured image. This distortion may not be a serious problem when the shot images are used individually one by one. However, in a case where a large subject is divided into a plurality of images, and the divided photographed images are combined to be combined into one large image, high-precision image combining is required unless such image distortion is corrected. Is impossible.

[0003] At the time of photographing, if the posture of the camera or the subject can be adjusted so that the image pickup plate and the subject plane are exactly parallel, the occurrence of such image distortion can be prevented. However, it is extremely difficult and often impossible to make such adjustments each time shooting is performed.

Further, as a conventional technique related to distortion correction by image processing, there is an image processing method and apparatus described in Japanese Patent Application Laid-Open No. 8-263626. In this conventional technique, the upper surface of the document table is divided and photographed in a state where no document is placed on the document table, and the coordinates of feature points arranged on the upper surface of the document table are detected on each divided image, and no document is determined based on the coordinates. A coordinate conversion formula between the divided image and the reference rectangle is determined. Next, the original placed on the platen was divided and photographed so as to include some feature points on the platen, and each of the divided images corresponded to the corresponding feature point in the documentless divided image. The image obtained by correcting the geometric distortion of the divided image with the original by converting the pixels of the divided image with the original into pixels within the reference rectangle in accordance with the coordinate conversion formula after moving in parallel so as to overlap the feature point. obtain.

[0005]

The above-mentioned prior art is effective when dividing and photographing a document of a predetermined size placed on a special platen to be assumed. However, it is possible to photograph an arbitrary subject in an arbitrary environment. If you do, you are powerless. Further, in the conventional technique, even in the case of photographing a document on a special platen to be assumed, in addition to the photographing of the document, the photographing of the platen without the document must be performed. It is.

Accordingly, an object of the present invention is to provide a more generally applicable image correction processing method and apparatus for correcting distortion of a photographed image due to the non-parallelism between the imaging plate and the object plane at the time of photographing the object. Is to do. Another object of the present invention is to provide a digital camera which facilitates such distortion correction. Another object of the present invention is to provide a digital camera capable of directly obtaining a photographed image in which such distortion has been corrected.

[0007]

A flat subject often has characteristic patterns such as "right-angle corners", "sets of parallel straight lines", and "rectangular figures". A relatively simple relationship is recognized between the manner of distortion when these are photographed and the relative angle between the subject plane and the imaging plate. Therefore, if such a characteristic pattern included in the captured image can be identified, the relative angle between the subject plane and the imaging plate can be obtained by calculation, and distortion correction can be performed based on the calculated angle. The calculation method of the relative angle is well known, and is detailed in, for example, "Image and Space" by Koichiro Deguchi (Shokodo).
However, it is generally difficult to automatically identify these characteristic patterns in an image obtained by capturing an arbitrary subject under arbitrary capturing conditions. Thus, it is realistic to be able to identify those characteristic patterns, and it would be desirable in terms of accuracy. The present invention aims to achieve the above-mentioned object based on such considerations.

According to a first aspect of the present invention, there is provided an image correction processing method comprising the steps of: inputting an image of a subject; Indicating a specific image element to be used for estimating the relative angle with the object plane, cutting out the specified image element from the input image, indicating a relationship between the specified image elements,
Calculating the relative angle between the imaging plate and the object plane at the time of capturing the input image based on the attribute information of the clipped image element and the specified relationship between the image elements; and inputting based on the calculated relative angle. The image processing apparatus includes a step of performing a geometric deformation operation on the image to correct a geometric distortion caused by the imaging plate and the object plane not being parallel.

According to a second aspect of the present invention, there is provided an image correction processing apparatus for inputting an image of a subject, displaying the input image on a display screen, and displaying the input image on the display screen when the image is captured. Means for designating a specific image element used for estimating the relative angle between the imaging plate and the object plane, means for cutting out the designated image element from the input image, and designating the relationship between the designated image elements Means for calculating the relative angle between the imaging plate and the object plane at the time of capturing the input image based on the attribute information of the clipped image element and the specified image element relationship, and
The apparatus includes means for performing a geometric deformation operation on the input image based on the calculated relative angle to correct geometric distortion caused by the imaging plate and the object plane not being parallel.

The digital camera according to the third aspect of the present invention is a digital camera for use in estimating a relative angle between an image pickup plate and a plane of a subject on a display screen on which a photographed image of the subject or an image before photographing is displayed. And a means for storing or externally outputting the information of the designated image element in association with the data of the captured image.

A digital camera according to a fourth aspect of the present invention
Means for indicating a specific image element to be used for estimating a relative angle between the imaging plate and the object plane on a display screen on which a captured image of the subject or a pre-imaging image is displayed, and the designated image element It is configured to include means for instructing the relationship between them, and means for storing or externally outputting the designated image element and information on the relationship between the image elements in association with the data of the captured image.

A digital camera according to a fifth aspect of the present invention
Means for indicating a specific image element to be used for estimating the relative angle between the imaging plate and the object plane on a display screen on which a captured image of the object or a pre-image image is displayed, and corresponding to the specified image element Means for displaying the identification symbol on the display screen, means for indicating the relationship between the specified image elements using the displayed identification symbol, and the specified image element and image element Means for storing or externally outputting the information of the interrelationship in association with the data of the photographed image.

[0013] The digital camera of the invention according to claim 6 is
Means for indicating a specific image element to be used for estimating a relative angle between the imaging plate and the object plane on a display screen on which a captured image of the subject or a pre-imaging image is displayed, and the designated image element From the captured image, means for indicating the relationship between the specified image elements, and associating the attribute information of the extracted image element and the information on the specified relationship between the image elements with the data of the captured image. And a means for storing or outputting to the outside.

The digital camera according to the invention described in claim 7 is:
A means for indicating a specific image element used for estimating the relative angle between the imaging plate and the object plane on a display screen on which a captured image or a pre-imaging image of the object is displayed, and the specified image element Means for cutting out from the photographed image, means for instructing the relationship between the designated image elements, and means for capturing the photographed image based on the attribute information of the clipped image element and the information on the designated relationship between the image elements. It is configured to include means for calculating a relative angle between the imaging plate and the object plane, and means for storing or externally outputting information on the calculated relative angle in association with data of the captured image.

The digital camera according to the invention of claim 8 is
Means for indicating a specific image element to be used for estimating a relative angle between the imaging plate and the object plane on a display screen on which a captured image of the subject or a pre-imaging image is displayed, and the designated image element Means for cutting out a captured image, means for indicating the relationship between designated image elements, and means for capturing a captured image based on attribute information of the clipped image elements and information on the designated relationship between image elements. Means for calculating a relative angle between the imaging plate and the object plane, and correcting a geometric distortion caused by the imaging plate and the object plane not being parallel to the captured image based on the calculated relative angle. And means for storing or externally outputting data of the captured image on which the geometric deformation operation has been performed.

According to a ninth aspect of the present invention, there is provided an information recording medium, wherein a program for causing a computer to execute a procedure of each processing step of the image correction processing method according to the first aspect of the invention is recorded. I do.

[0017]

Embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 shows an image correction processing apparatus 1 according to the present invention.
2 shows the relationship between the conventional digital camera 2 and the digital camera 3 according to the present invention. The processing contents of the image correction processing apparatus 1 of the present invention vary depending on whether the processing is for a normal image captured by the conventional digital camera 2 or the like or for an image captured by the digital camera 3 according to the present invention. Further, even when the image captured by the digital camera 3 according to the present invention is targeted, the content of the information provided in association with the captured image data (the configuration of the digital camera 3 at the time of the image capturing, that is, a mode to be described later). The processing content changes depending on the difference.

The image correction processing apparatus 1 of the present invention comprises a CPU 11, a memory 12, an auxiliary storage device 13, a drive 16 for reading and writing on an information recording medium 17 such as a floppy disk, and a display device 14 as shown in FIG. ,
A bus 20 such as a user input device 15 such as a pointing device such as a mouse or a keyboard, a drive 19 for reading and writing a memory card of a digital camera, and an external interface unit 18 for connecting a cable to an external device such as a digital camera or a printer. It is realized by a program on the hardware of a general computer connected by. However, it is needless to say that it can be realized by dedicated hardware. The memory 12 stores C
It is used as a work area for the PU 11 and as a storage area for fixed information 22 such as a processing program 21 for executing the procedure of each step of the image correction processing according to the present invention and other control programs. The processing program 21 is loaded into the memory 12 from the information recording medium 17 via the drive 16 or is temporarily stored in the auxiliary storage device 13 and loaded into the memory 12 when necessary. Data of a captured image to be processed is input via the drive 19 or 16 or via the external interface unit 18.

When a normal image photographed by the conventional digital camera 2 or the like is targeted, the processing program 2
FIG. 3 shows the processing flow performed by the first embodiment. In other words, means for executing the procedure of each processing step shown in this processing flow is realized on the hardware of the computer by the processing program 21.

First, in step 51, image data is read into the memory 22, and in step 52, the input image is displayed on the display device 14 on a screen. While observing the input image on the display screen, the operator can use the “right angle corner”, “set of parallel straight lines”, “rectangle” to be used for estimating the relative angle between the imaging plate and the object plane when capturing the input image. Recognize a characteristic pattern such as a figure or a circle, and indicate a specific image element for identifying it and the relationship between them.

That is, the operator operates the user input device 15
By operating a mouse (pointing device) included in the above, a required image element (specifically, a point, a straight line, an arc, etc.) is indicated on the display screen (step 53). The method of instructing is to place a mouse cursor on an image element and click it, or to move the mouse cursor while holding down the mouse button to create a rectangular area surrounding the image element, as is generally used in drawing software. A method of setting or setting an area surrounding an image element with a free handwriting line may be adopted. When an image element is designated in this way, a symbol for identifying each image element is automatically given and displayed on the display screen in association with the image element (step 54). The instruction of the image element and the display of the identification symbol can be repeated as many times as necessary.
Although it is not explicitly described, it is also possible to modify it as needed.

For example, on the subject plane in the input image, parallel straight lines 71, 73, 74 as shown in FIG.
When a straight line 72 orthogonal to the straight line 71 is included and these are designated as image elements, as shown in FIG. 4B, identification symbols A, B, C, and D are given to the respective image elements. Displayed in association. In FIG. 4B, a straight line 7
Numerals 1, 72, and 73 indicate a case where an instruction is given by a broken handwritten line area, and a straight line 74 indicates a case where a continuous black pixel group pointed by an arrow is indicated as an object. An example of indicating a point is not shown in FIG.

When the operator instructs the end of the image element instruction, the specified image element is cut out (step 55). An example of this cutout processing is shown in FIG. 5 and described. First, an outline enhancement process is performed on the entire input image (step 80). This may be performed by a general method of performing a binarization process after performing a second differentiation process. Next, it is checked whether all of the designated image elements are designated as a point, designated as a region, or designated as an object (step 81). If the designated region is a region, a black pixel group in the region is contour-emphasized. Extract from the image (step 82)
If it is an object, a group of black pixels connected to the target black pixel is extracted from the contour emphasized image (step 83). Then, the attribute of the extracted black pixel group, that is, the attribute of the image element is determined (step 84). For example, if it is a point, its coordinate value is obtained; if it is a straight line, its inclination, length, and coordinates of both ends are obtained; if it is a circular arc, its center coordinate and radius are obtained.

When the attributes for all the specified image elements are determined, the operator uses, for example, a keyboard included in the user input device 15 to determine the correlation between the specified image elements and the identification assigned to each image element. Instruction is given using symbols (step 56). For example, A // C indicates that the straight lines 71 and 73 in FIG. 4 are parallel, A_ | _B indicates that the straight lines 71 and 72 are orthogonal, and A indicates that the lengths of the straight lines 71 and 73 are equal. = C, and A = C = D = ho
r indicates that the straight lines 71, 73, 74 are horizontal (that is, the subject plane including them is horizontal).

When such an instruction is completed, the relative angle between the imaging plate and the object plane at the time of capturing the input image is determined based on the attribute information of the image elements extracted in the clipping process and the specified relationship between the image elements. It is calculated (step 57). Next, based on the calculated relative angle, the input image is subjected to a geometric operation for correcting image distortion due to the image plane being not parallel to the object plane (step 58).

For example, straight lines 71, 73, 74 shown in FIG.
It is assumed that the relationship of A // C // D and A = C = D = hor is instructed in step 56. That is, the straight lines 71, 7
3, 74 are parallel and horizontal. In this case, the vanishing points of the parallel straight lines (A, C, D, ie, the straight lines 71, 7)
Assuming that the coordinates of the intersection of (3, 74) are (Xd, Yd), the horizontal angle α and the vertical angle β of the subject plane with respect to the imaging plate, which include these three lines, are α = arctan (Xd / F) β = Arctan (Yd / F). Here, F is the focal length of the camera lens. Here, it is assumed that the image was taken with the bottom side of the imaging plate being horizontal. Such calculations are detailed in the book. And the relative angles α, β
X = x (cosα + sinα · tan (α + γ)); γ = arctan (x /
F) Y = y (cosβ + sinβ · tan (β + δ)): δ = arctan (y /
The image (x, y) can be converted into an image (X, Y) in which geometric distortion due to non-parallelism between the imaging plate and the object plane has been corrected by the inverse projection transformation of F).

The image corrected in this way is stored, for example, in the auxiliary storage device 13 or recorded on the information recording medium via the drive 16 (step 5).
9).

Although not shown in FIG. 3, if it is desired to process a plurality of images shot under the same shooting conditions,
By executing the entire processing described above for the second image,
For the subsequent images, the processing of step 58 may be executed immediately using the calculated relative angle. Further, the order of Step 55 and Step 56 can be reversed.

FIG. 6 shows an example of a block configuration of the digital camera 3 of the present invention. In FIG. 6, 100 is an imaging system,
101 is an image signal processing system, 102 is a frame memory used for the image signal processing system, 103 is a display device such as a liquid crystal display panel used as a monitor of a captured image or an electronic finder, and 104 is an image signal processing system 101
A memory card 105 for reading and writing data; 105, an external interface unit for inputting and outputting signals between the image signal processing system 101 and the outside; 106, a CPU for controlling the image signal processing system 101 and the imaging system 100;
A RAM 108 used as a work area for the reference numeral 06, a ROM 108 for storing fixed information such as a program,
Reference numeral 11 denotes an operation unit for inputting various instructions and the like for a shooting operation. The imaging system 100 includes an optical lens, an aperture and a mechanism for adjusting focus, an imaging plate such as a CCD imaging device,
An image signal of the optical image formed on the imaging plate is output. The image signal processing system 101 converts an image signal input from the imaging system 100, assembles a frame image, automatically controls exposure, focus, and white balance or detects a signal therefor, generates a video signal for the display device 103, It performs compression / decompression processing of image data.

In the digital camera 3 of the present invention, a function related to the correction of the distortion of the photographed image due to the non-parallelism between the imaging plate and the object plane is added. Is realized by a program. Additional function program 10 stored in ROM 108
9 is the program. However, some or all of the additional means may be realized as hardware.

The digital camera 3 of the present invention may be configured as a single unit as a whole,
It may be divided into a camera main unit including 0 and one or more accessory units. In the case of the split configuration, the means added according to the present invention can be provided on the camera body unit side or on the accessory unit side depending on the configuration of each unit.

The digital camera 3 of the present invention can select a plurality of modes in relation to the additional functions according to the present invention, and the configuration related to the additional functions is changed depending on the selected mode. Each mode will be described with reference to FIGS.

FIG. 7 shows a processing flow in the mode (1). In this mode, first, a target object is photographed (step 200). The photographed image is displayed on the display device 10.
3 is displayed on the screen. The operator searches the display screen for a characteristic pattern to be used for estimating the relative angle between the imaging plate and the object plane, and sequentially instructs image elements (such as straight lines) using a pointing device included in the operation unit 111. (Step 205). The method of this instruction is shown in FIG.
The method may be the same as that described in connection with step 53 of FIG.
When an image element is designated, its identification symbol is given and displayed in association with the display screen (step 2).
10). When the instruction of the required image element is completed and the end is instructed by the operator, the information of the specified image element and the identification symbol is stored in the memory card together with the captured image data, for example, as a part of header information of the captured image data. The data is written to the memory 104 or is output to, for example, the image correction processing apparatus 1 of the present invention via the interface unit 105 (step 250).

As described above, in this mode (1), means for designating an image element on the display screen, means for giving and displaying an identification symbol, and information on the image element and the identification symbol are associated with the photographed image data. Means for outputting or storing data are involved, and these means are realized by the additional function program 109 on the block configuration shown in FIG.

In the image correction processing apparatus 1 according to the present invention, when a photographed image in this mode (1) is to be processed, information on image elements and identification symbols is read together with the photographed image data in step 51 in FIG. At 52, it is displayed on the display screen. Therefore, steps 53 and 54 can be skipped, and the process can immediately proceed to step 55.

FIG. 8 shows a processing flow in the mode (2). In this mode, the screen of the display device 103 is observed, and the target object is fixed in a state of being put in the field of view of the imaging system 100. The image elements are sequentially shown for the image on the display screen at this time (the image before photographing, that is, the electronic finder image) as in the mode (1) (step 305). An identification symbol is assigned to each image element and displayed in association with each other (step 310). When the instruction of the image element is completed, the operator performs a shutter operation to actually photograph the subject (step 350). The information of the designated image element and the identification symbol is written to the memory card 104 together with the captured image data, for example, as a part of header information of the captured image data, or output to the outside via the interface unit 105. . If two or more shots are required under the same shooting conditions, the shooting operation and data storage / output (steps 350 and 370) are repeated.

In this mode (2), for example, when it is desired to photograph a plurality of images under the same photographing conditions or to photograph a large subject in a divided manner, only a part of the subject is used to estimate the relative angle between the imaging plate and the subject plane. This is effective when there is no characteristic pattern that can be used.

Means for designating image elements required for this mode (2), means for giving and displaying identification symbols, means for outputting or storing information on image elements and identification symbols in association with photographed image data include: This is realized by the additional function program 109 on the block configuration in FIG.

In the image correction processing apparatus 1 of the present invention, the processing for a captured image in mode (2) is the same as the processing for a captured image in mode (1).

FIG. 9 shows a processing flow in the mode (3). The difference between the mode (3) and the mode (1) is that the relationship between the designated image elements is designated in step 215, and in step 250, the information on the relationship between the image elements in addition to the image elements and the identification symbols is taken. Writing to the memory card 104 together with the data, or the external interface unit 10
5 to the outside. The method of indicating the relationship between image elements in step 215 may be the same as the method described with reference to step 56 in FIG.

In this mode (3), means corresponding to step 215 is further required, but this is realized by the additional function program 109. In the image correction processing apparatus 1 of the present invention, when the image data in the mode (3) is targeted, the information of the image element, the identification symbol, and the relation between the image elements is read together with the image data in Step 51. Steps 53 and 54 and step 56 can be skipped.

FIG. 10 shows a processing flow in the mode (4). The difference between this mode (4) and mode (2) is
Instructing the relationship between the designated image elements in step 315, and writing the information on the relationship between the image elements in addition to the image elements and identification symbols to the memory card 104 together with the photographed image data in step 370, or an external interface unit. That is, output to the outside through 105. In the image correction processing apparatus 1 of the present invention, the processing for a captured image in mode (4) is the same as the processing for a captured image in mode (3). In addition, this mode (4)
Is effective for the same purpose as in mode (2).

FIG. 11 shows a processing flow in the mode (5). The difference between this mode (5) and mode (3) is
In step 213, the specified image element is cut out, and in step 250, the attribute information of the image element obtained by the cut out processing is stored / output together with the captured image data. The contents of the cutout processing in step 213 and the attribute information to be extracted are:
This is the same as step 55 in FIG.

In this mode (5), means corresponding to step 213 is further required, but this is realized by the additional function program 109. When the image correction processing apparatus 1 of the present invention targets photographed image data in mode (5), the attribute information of the image element is read together with the photographed image data in step 51, so steps 53 to 56 are skipped. The process can immediately proceed to step 57.

FIG. 12 shows a processing flow in the mode (6). The difference between this mode (6) and mode (4) is
In step 313, the designated image element is cut out, and in step 370, the attribute information of the image element obtained by the cut out processing is stored / output together with the photographed image data. The contents of the extraction processing in step 313 and the attribute information to be extracted are
This is the same as step 55 in FIG. In the image correction processing apparatus 1 according to the present invention, the processing for a captured image in mode (6) is the same as the processing for a captured image in mode (5). The mode (6) is effective for the same purpose as the mode (2).

FIG. 13 shows a processing flow in the mode (7). The difference between this mode (7) and mode (5) is
Calculating the relative angle between the imaging plate and the object plane in step 220 based on the attribute information of the image element extracted in step 213 and the relationship between the image elements specified in step 215; and information on the calculated relative angle. Is stored / output together with the photographed image data in step 250, for example, as a part of header information.

In this mode (7), means corresponding to step 220 is further required, but this is realized by the additional function program 109. In the image correction processing apparatus 1 of the present invention, when the image data in the mode (7) is targeted, the relative angle is read together with the image data in the step 51, so that the steps 53 to 5 are performed.
7 is skipped, and the process can proceed to step 58 immediately.

FIG. 14 shows a processing flow in the mode (8). The difference between this mode (8) and mode (6) is
In step 320, the relative angle between the imaging plate and the object plane is calculated, and information on the calculated relative angle is stored / output together with the captured image data in step 370. In the image correction processing apparatus 1 of the present invention, the processing for a captured image in mode (8) is the same as the processing for a captured image in mode (7). The mode (8) is effective for the same purpose as the mode (2).

FIG. 15 shows a processing flow in the mode (9). The difference between this mode (9) and mode (7) is
In step 230, the captured image is corrected based on the relative angle, and in step 250, the corrected captured image data is stored / output. A means corresponding to step 230 is further required, and this is realized by the additional function program 109.

FIG. 16 shows a processing flow in the mode (10). The difference between mode (10) and mode (8) is that the captured image is corrected in step 360
0 means storing / outputting the corrected photographed image data. This mode (10) is effective for the same purpose as mode (2).

[0052]

According to the image correction processing method or apparatus according to the first or second aspect of the present invention, the operator actually observes an image obtained by photographing an arbitrary subject, and determines the relative angle between the subject plane and the imaging plate. By judging and indicating a specific image element that can be used for estimation, and indicating the relationship between the image elements, it is possible to accurately correct distortion due to the non-parallelism between the object plane and the imaging plate. For example, if the divided images are subjected to distortion correction by the same method or apparatus, it is possible to create a high-accuracy combined image.

According to the digital camera of the third aspect of the present invention, at the time of photographing, the operator can observe a photographed image or an image to be photographed, and use it for estimating the relative angle between the image pickup plate and the object plane. Since the image element can be determined and instructed, and the information can be stored or output in association with the photographed image data, the information of the image element can be used when the photographed image is processed by an external processing device. The work and processing for correcting distortion caused by the plate and the object plane not being parallel are simplified.

According to the digital camera of the fourth or fifth aspect of the present invention, at the time of photographing, the operator observes the photographed image or the image to be photographed, and can use the image element for the relative angle between the imaging plate and the object plane. Can be determined and instructed, and the relationship between image elements can be instructed, and the information can be stored or output in association with the captured image data. Therefore, when the captured image is processed by an external processing device, The information can be used, and the work and processing for correcting distortion caused by the imaging plate and the object plane not being parallel are simplified. Further, in the digital camera according to the fifth aspect of the present invention, by using the identification symbol given to the image element, the operation of instructing the relation between the image elements becomes easy, and the contents of the instruction can be easily understood and the mistake of the instruction can be prevented. It is hard to get up.

According to the digital camera of the present invention, the operator observes the photographed image or the image to be photographed at the time of photographing, and estimates the relative angle between the imaging plate and the object plane. When the photographed image is processed by an external processing device, it is possible to instruct the elements and the relationship between them and store or output the information of the relationship between the image elements and the attribute information of the image elements in association with the photographed image. In addition, such information can be used, and work and processing for correcting distortion caused by the imaging plate and the object plane not being parallel are facilitated.

According to the digital camera of the present invention, an image element for estimating the relative angle between the image pickup plate and the object plane by observing the photographed image or the image to be photographed at the time of photographing. And by indicating the relationship between them, the relative angle can be calculated and stored or output in association with the captured image data,
When a captured image is processed by an external processing device, the relative angle can be used, and distortion due to the imaging plate and the object plane not being parallel can be easily corrected.

According to the digital camera of the present invention, the image plate for estimating the relative angle between the image pickup plate and the object plane and the relationship between them are designated at the time of photographing. High-accuracy captured image data in which distortion due to non-parallelism has been corrected can be stored or output. Therefore, if this digital camera is used for divisional photographing of a subject, a high-accuracy combined image can be easily created.

According to the information recording medium of the ninth aspect of the present invention, the image distortion can be corrected by the image correction processing method of the first aspect of the present invention using a computer.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the relationship between an image correction processing device of the present invention, a digital camera of the present invention, a conventional digital camera, and the like.

FIG. 2 is a block diagram illustrating an example of a configuration of an image correction processing device according to the present invention.

FIG. 3 is a flowchart showing a processing flow when processing an image captured by a conventional digital camera or the like in the image correction processing apparatus of the present invention.

FIG. 4 is a diagram for explaining an image element used for estimating a relative angle between an imaging plate and a subject plane and a method of indicating the image element.

FIG. 5 is a flowchart illustrating an example of an image element cutout process.

FIG. 6 is a block diagram illustrating an example of a block configuration of a digital camera according to the present invention.

FIG. 7 is a flowchart showing a processing flow in a mode (1).

FIG. 8 is a flowchart showing a processing flow in a mode (2).

FIG. 9 is a flowchart showing a processing flow in a mode (3).

FIG. 10 is a flowchart showing a processing flow in a mode (4).

FIG. 11 is a flowchart showing a processing flow in a mode (5).

FIG. 12 is a flowchart showing a processing flow in a mode (6).

FIG. 13 is a flowchart showing a processing flow in a mode (7).

FIG. 14 is a flowchart illustrating a processing flow in a mode (8).

FIG. 15 is a flowchart showing a processing flow in a mode (9).

FIG. 16 is a flowchart showing a processing flow in a mode (10).

[Explanation of symbols]

 Reference Signs List 11 CPU 12 Memory 13 Auxiliary storage device 14 Display device 15 User input device 16 Drive such as floppy disk 17 Information recording medium such as floppy disk 18 External interface unit 19 Drive of memory card 21 Image correction processing program 100 Imaging system 101 Image signal processing System 102 Frame memory 103 Display device 104 Memory card 105 External interface unit 106 CPU 107 RAM 108 ROM 109 Additional function program 111 Operation unit

Continuing from the front page (72) Inventor Kaikatsu Seki, 3-chome, Chuo-ku, Tokyo Ricoh System Development Co., Ltd. (72) Inventor Takuyuki Sakamoto, 3-chome, 12-chome, Chuo-ku, Tokyo Ricoh System Development Co., Ltd.

Claims (9)

[Claims] A step of inputting an image obtained by photographing a subject, and estimating a relative angle between an image pickup plate and a plane of the subject at the time of photographing in the input image on a display screen on which the input image is displayed. A step of designating a specific image element to be used, a step of cutting out the designated image element from the input image, a step of designating a relationship between the designated image elements, Calculating the relative angle between the imaging plate and the object plane when capturing the input image based on the relationship between the image elements,
Performing, on the input image, a geometric deformation operation for correcting a geometric distortion caused by the imaging plate and the object plane not being parallel based on the calculated relative angle. Image correction processing method. 2. A means for inputting an image obtained by photographing a subject,
Means for displaying an input image on a display screen, and means for indicating a specific image element used for estimating a relative angle between the imaging plate and the object plane at the time of shooting in the input image on the display screen Means for cutting out the designated image element from the input image; means for designating the relationship between the designated image elements; and, based on the attribute information of the cropped image element and the designated relationship between the image elements. Means for calculating a relative angle between the imaging plate and the object plane at the time of capturing the input image; and a geometry caused by the imaging plate and the object plane not being parallel to the input image based on the calculated relative angle. Means for performing a geometrical deformation operation for correcting a spatial distortion. 3. A digital camera having a display screen for displaying a photographed image or a pre-shooting image of a subject, wherein a relative position of an imaging plate and a subject plane is displayed on the display screen on which the photographed image or the pre-shooting image is displayed. A means for designating a specific image element to be used for estimating an angle, and a means for storing or externally outputting information on the designated image element in association with data of a photographed image. Digital camera. 4. A digital camera provided with a display screen for displaying a photographed image or a pre-shooting image of a subject, wherein a relative position between an imaging plate and a subject plane is displayed on the display screen on which the photographed image or the pre-shooting image is displayed. Means for indicating a specific image element to be used for estimating the angle, means for indicating the relationship between the specified image elements, and information on the specified image element and the relationship between the image elements. A digital camera comprising means for storing or externally outputting the data in association with data of the photographed image. 5. A digital camera provided with a display screen for displaying a photographed image or a pre-shooting image of a subject, wherein a relative position between an imaging plate and a subject plane is displayed on the display screen on which the photographed image or the pre-shooting image is displayed. Means for indicating a specific image element to be used for the estimation of the angle, means for displaying the identification symbol on the display screen in association with the specified image element, and using the displayed identification symbol, Means for instructing the relationship between the designated image elements, and means for storing or externally outputting information on the designated image elements and the relationship between the image elements in association with the data of the captured image. Digital camera. 6. A digital camera provided with a display screen for displaying a photographed image or a pre-shooting image of a subject, wherein a relative position of an imaging plate and a subject plane is displayed on the display screen on which the photographed image or the pre-shooting image is displayed. Means for indicating a specific image element to be used for estimating the angle, means for cutting out the specified image element from the captured image, means for indicating the relationship between the specified image elements, and Means for storing or externally outputting the acquired attribute information of the image elements and the information on the designated relationship between the image elements in association with the data of the photographed image. 7. A digital camera provided with a display screen for displaying a photographed image or a pre-shooting image of a subject, wherein a relative position between an imaging plate and a subject plane is displayed on the display screen on which the photographed image or the pre-shooting image is displayed. Means for indicating a specific image element to be used for estimating the angle, means for cutting out the specified image element from the captured image, means for indicating the relationship between the specified image elements, and Means for calculating the relative angle between the imaging plate and the object plane at the time of capturing the captured image based on the attribute information of the captured image element and the information on the designated relationship between the image elements, and captures the information on the calculated relative angle. A digital camera comprising means for storing or externally outputting in association with image data. 8. A digital camera having a display screen for displaying a photographed image or a pre-shooting image of a subject, wherein a relative position between an imaging plate and a subject plane is displayed on the display screen on which the photographed image or the pre-shooting image is displayed. Means for indicating a specific image element to be used for estimating the angle, means for cutting out the specified image element from the captured image, means for indicating the relationship between the specified image elements, and Means for calculating the relative angle between the imaging plate and the object plane at the time of photographing the photographed image based on the attribute information of the acquired image element and the information on the designated relationship between the image elements, and photographing based on the calculated relative angle. Means for performing a geometric deformation operation on the image to correct geometric distortion caused by the imaging plate and the object plane not being parallel to each other; Means for storing or externally outputting shadow image data. 9. A machine-readable information recording medium on which a program for causing a computer to execute a procedure of each processing step of the image correction processing method according to claim 1 is recorded.