JP2000172846A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image processor which facilitates the handling of a camera in image pickup operation and is advantageous in size reduction by finding the angles that segments connecting peak positions detected as to respective intensity distribution images contain and correcting the tilts of the images according to the found angles. SOLUTION: A power spectrum image Q is scanned vertically and horizontally to individually detect the peak positions of luminance level and the segments dX and dY connecting the pixels at the detected peak positions are found. Then the angles that segments DX and DY corresponding to the found segments dX and dY contain are found. The both are not always equal, so the mean of the angle between the segments dX and DX and the angle between the segment dY and DY is found and regarded as a target angle to secure constant precision. Then the tilt of an image B is corrected in the direction where the angle decreases and thus equalized to the tilt of an image A.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for generating a stereoscopic image, and more particularly to an image processing apparatus having a feature in correcting inclination and position of two images.

[0002]

2. Description of the Related Art With the rapid spread of computer systems in recent years, particularly personal computers, the demand for various peripheral devices is also increasing. In particular, digital still cameras that can directly capture captured photographic images as digital data have attracted attention as peripheral devices for capturing images into personal computers.

Generally, an image picked up by a digital still camera is printed out after some image processing. An image processing apparatus proposed in Japanese Utility Model Laid-Open No. 9-416 detects an overlapping position of adjacent images when a plurality of images which are divided and taken so that their edges overlap each other are provided. Then, by splicing the two at that position and printing them out as a single image, for example, a large subject that does not fit in the imaging range of one image is divided into a plurality of partial regions, and the image is captured over a plurality of images. The purpose of the present invention is to obtain a large print output with good resolution.

By the way, when two images of the same subject imaged at a fixed interval are juxtaposed and each image is directly viewed at the same time in association with one of the right eye and the left eye, a person is in the image. It is generally known that a subject feels depth. Hereinafter, an image paired to provide such an effect will be referred to as a stereoscopic image. This stereoscopic image is often used for learning teaching materials, toys and the like. As an apparatus for creating a stereoscopic image, two objective lenses respectively associated with the right eye and the left eye are arranged at a fixed interval, and a light beam transmitted through each objective lens is reflected by a reflecting mirror or a prism. Induced using
There are cameras configured so that they form an image in the right half and the left half of the screen, respectively.

[0005] Further, based on the same principle as that of the stereoscopic image, two moving images picked up by two video cameras installed at a fixed interval are alternately transmitted at a predetermined timing to a television. A moving image reproducing apparatus which displays on the screen and synchronizes with the timing of the display switching, and alternately intercepts the field of view of the right eye or the left eye of the viewer with the goggles worn by the viewer to give the viewer a sense of depth. Has already been commercialized.

[0006]

In such a stereoscopic image, if there is a shift between the imaging ranges of the images respectively associated with the right eye and the left eye, the effect obtained is diminished. At that time, it is necessary to pay close attention to make the two imaging ranges the same. In particular,
In the case of separately capturing images associated with the right eye and the left eye using two imaging systems as in the above-described moving image reproducing apparatus, the center position and the inclination of the mutual imaging range from the start to the end of the imaging. It was not easy to handle, because it was necessary to keep the state of matching. This will be described below with reference to the drawings.

FIG. 8 is an explanatory diagram for explaining a deviation of the imaging range of the two imaging systems. In the figure, C _R and C
_L is an imaging system associated with the right eye and the left eye, respectively, and its imaging range is represented by rectangular regions R and L. The area X where the rectangular areas R and L overlap is hatched. The region X is a region where an expected effect can be obtained for a stereoscopic image. However, the greater the deviation between the rectangular regions R and L, the less the effect.

In a camera for capturing a stereoscopic image as described above, there are provided two objective lenses arranged at a fixed interval in association with the right eye and the left eye, respectively, and a mechanism for guiding light beams transmitted therethrough. However, there is a problem that the above-mentioned camera becomes an obstacle to miniaturization of the camera.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and receives input of two systems of images or moving images respectively associated with the right eye and the left eye, and shifts the imaging range between the systems. It is an object of the present invention to provide an image processing apparatus which facilitates handling of a camera at the time of imaging by performing image processing so as to correct the image, and which is advantageous for miniaturization of the apparatus.

[0010]

An image processing apparatus according to a first aspect of the present invention provides two images of the same subject,
An image processing apparatus that performs image processing for matching the inclinations of both subjects, and performs orthogonal transform on each image to obtain respective transform coefficients. Means for detecting the peak position of the luminance level on each of the plurality of scanning lines in the direction, means for obtaining the angle formed by the line segments connecting the pixels at the plurality of peak positions detected for each intensity distribution image, and the obtained angle And means for correcting the inclination of one of the images so as to reduce the inclination of the other image.

FIG. 6 is an explanatory diagram for explaining the concept of the present invention. In the figure, A represents an example of an image, and P represents a power spectrum (intensity distribution) image of frequency components obtained by performing a two-dimensional discrete Fourier transform (hereinafter, referred to as DFT) on the image A. . The power spectrum image P is an image gray-scale-displayed based on the intensity of the frequency component using a polar coordinate system having the origin at the center of the image. Here, in the power spectrum image P, only the characteristic partial region is schematically represented by a line segment U having a substantially rhombic shape, and the gradation expression of the luminance level of each pixel is omitted. Specifically, the line segment U is a line segment connecting pixels having substantially the same luminance level, and in the actual power spectrum image P, the luminance level increases as the inner side approaches the origin, On the outside, the luminance level decreases as the distance from the origin increases.

[0012] On the other hand, B shows an example of another image taken of the same subject as the image A, and the imaging range of the image A and the imaging range of the image A have a slight difference in inclination. Q represents a power spectrum image of a frequency component obtained by performing the DFT on the image B, and its shading expression is based on the power spectrum image P.

When the power spectrum of the DFT is displayed in the polar coordinate system in this manner, a feature related to the direction of the image may appear as a change in the luminance level near an angle corresponding to the direction of the power spectrum image. Generally known. For example, an image captured by adjusting the horizontal line of an outdoor landscape so that its horizontal line is parallel to the horizontal direction of the imaging range has a general feature that the horizontal component and the vertical component of the directivity are increased. In this case, in the power spectrum image of the DFT of the image, the peak of the luminance level appears along the horizontal and vertical line segments passing through the origin.

In other words, when an image of an outdoor scene is photographed, when the peak of the luminance level appears along a line segment in a direction different from the horizontal direction and the vertical direction in the power spectrum image of the DFT, adjustment is performed at the time of imaging. It can be determined that the horizontal direction of the obtained imaging range is inclined from the horizontal line of the outdoor scenery. Regarding the slope, a line segment (corresponding to the horizontal component of the captured image) connecting the pixels at the peak position when the power spectrum image is scanned in the vertical direction or the pixel at the peak position when the power spectrum image is scanned in the horizontal direction is connected. A line segment (corresponding to a vertical component of the captured image) can be obtained by an angle between the horizontal direction and the vertical direction of the power spectrum image.

In the power spectrum image Q, the line segments D _X and D _Y represented by broken lines connect the pixels at the peak positions when the power spectrum image P of the image A is scanned in the vertical and horizontal directions, respectively. A line segment is expressed, and an angle formed by this and an equivalent line segment for the image B is obtained.
Used for tilt correction.

Specifically, a power spectrum image Q is
Each scanned individually detects the peak position of the brightness level in the vertical direction (arrow Q _Y) and horizontal (arrow Q _X),
Line segment d _X and line segment d connecting the pixel at the detected peak position
_Ask for _Y. Next, determine the angle determined and the line segment D _X and the line segment D _Y line segments d _X and line segment d _Y and respectively corresponding the forms.
Since both not always agree, an average between the angle and the line segment d _Y and the line segment D _Y is the angle which the line segment d _X and the line segment D _X forms, this constant by the angle of the object Ensure accuracy. Then, the inclination of the image B is made to match the inclination of the image A by correcting the inclination of the image B in the direction in which the angle becomes smaller.

In the present invention, the deviation of the inclination of the imaging range between the systems is corrected by image processing, so that the camera can be easily handled at the time of imaging. In addition, since images or moving images of each system individually received are to be processed, it is not necessary to provide a mechanism for simultaneously forming images associated with the right eye and the left eye on one screen.

An image processing apparatus according to a second aspect of the present invention is an image processing apparatus for performing image processing on two images obtained by capturing the same subject so that the two subjects have the same inclination.
Means for performing orthogonal transform on each image to obtain respective transform coefficients, and means for detecting the peak position of the luminance level on a plurality of scanning lines in a predetermined direction for the obtained intensity distribution image of each transform coefficient, Means for determining an angle formed by line segments connecting pixels at a plurality of peak positions detected for each intensity distribution image, and means for correcting the inclination of one of the intensity distribution images based on the determined angle so as to reduce the angle. And a means for performing an orthogonal inverse transform on a transform coefficient related to the corrected intensity distribution image to obtain an image.

According to the present invention, based on the determined angle,
By correcting the inclination of the power spectrum image Q in a direction in which the angle becomes smaller, and performing an inverse two-dimensional discrete Fourier transform (hereinafter referred to as IDFT) on the corrected image, the image B whose inclination matches the inclination of the image A Can be obtained.

In the present invention, the deviation of the inclination of the imaging range between the systems is corrected by image processing, so that the camera can be easily handled at the time of imaging. In addition, since images or moving images of each system individually received are to be processed, it is not necessary to provide a mechanism for simultaneously forming images associated with the right eye and the left eye on one screen.

An image processing apparatus according to a third aspect of the present invention is an image processing apparatus for performing image processing for matching the positions of two subjects on two images of the same subject,
Means for cutting out an area of a predetermined size at a predetermined position of one image, means for applying a high-pass filter to the area cut out of the image to obtain a first inspection image, and means for obtaining a first inspection image; Means for cutting out a region of a predetermined size at a plurality of positions different from the above, applying a high-pass filter to each of the cut out regions of the image, and obtaining respective second inspection images; Means for calculating the sum of the differences in the luminance levels between the pixels whose positions correspond to each of the sets with the second inspection image; means for comparing the calculated sums to identify the set with the smaller sum; Means for correcting the position of one of the images based on the difference between the cutout positions of the respective regions according to the set so as to reduce the difference.

According to a fourth aspect of the present invention, in the image processing apparatus, the high-pass filter performs an orthogonal transform on the cut-out region of the image to obtain a transform coefficient, and removes a low-frequency component of the obtained transform coefficient. And a means for performing an orthogonal inverse transform on the transform coefficient from which the low-frequency component has been removed.

FIG. 7 is an explanatory diagram for explaining another concept of the present invention. In the figure, E represents an example of an image, and R represents a filtered image obtained by applying a high-pass filter to a partial region of the image E. Here, in the filter image R, only the feature is schematically represented by a straight line in the vertical direction, and the gradation expression of the luminance level of each pixel is omitted. Specifically, in the partial area, the luminance level on the left side and the luminance level on the right side change largely with the substantially center as a boundary. That is, the straight line in the vertical direction in the filter image R corresponds to the edge between the hatched area of the image E and another area.

On the other hand, F shows an example of another image taken of the same subject as the image E, and the imaging range of the image E and the imaging range of the image E are slightly shifted in position. S represents a filter image obtained by applying a high-pass filter to the image F, and its shading expression conforms to that of the filter image R. In the filter image R and the filter image S obtained in this way, a difference in luminance level is calculated for each pixel, a position where the sum of the differences is minimized is obtained, and used for correction.

More specifically, first, a rectangular area of a predetermined size is cut out near the center of the image E, and a high-pass filter is applied to the rectangular area to obtain a filtered image R. Next, the rectangular area is cut out near the center of the image F, and a high-pass filter is applied to the cut-out area to obtain a filtered image S. Further, a difference in luminance level between pixels corresponding to the positions of the filter image R and the filter image S is calculated, and further a sum thereof is calculated.

Furthermore, the rectangular area is cut out at a position slightly shifted in any direction from the position related to the previous cut out of the image F, and a high-pass filter is applied to this to obtain a new filter image S. , The sum of differences in luminance levels between pixels corresponding to the positions of the filter images R and S. Then, the position at which the obtained sum is minimized is specified, and the position deviation is corrected for the entire image based on the relative positional relationship between the position and the position of the filtered image R, so that the image E and the position are corrected. A matched image can be obtained.

In the third and fourth aspects of the present invention, in order to correct the displacement of the position of the imaging range between the systems by image processing,
Handling of the camera during imaging is facilitated. In addition, since images or moving images of each system individually received are to be processed, it is not necessary to provide a mechanism for simultaneously forming images associated with the right eye and the left eye on one screen.

[0028]

FIG. 1 is a block diagram showing a configuration of a signal processing system of a digital still camera which is an embodiment of an image processing apparatus according to the present invention. In the figure, reference numeral 1 denotes an objective lens. The light beam incident from the objective lens 1 forms an image on the light receiving surface of the CCD 2 which is an image sensor, and is converted into an analog electric signal corresponding to the brightness. This CCD2
Is periodically scanned at an appropriate timing, whereby a signal for one frame is serially transferred to the analog / digital (A / D) converter 3. The A / D converter 3 converts the serially input analog signal into a digital signal and converts the analog signal into a digital signal.
Send to The signal processing circuit 4 performs appropriate processing on the digital signal input serially to generate bitmap image data, and sends it to the selector 5.

A VRAM 6 and a CPU 7 are connected to the selector 5 in addition to the signal processing circuit 4 described above, and a selection instruction signal given from a system controller (not shown) is provided.
The signal source and destination are physically selected and connected according to the SS. Specifically, when the display setting for displaying the camera image on the monitor has been made, the selector 5 transfers the image data sent from the signal processing circuit 4 to the VRAM 6. When the display setting for displaying the image input from the external device on the monitor is made, the selector 5
The image data sent from the PU 7 is transferred to the VRAM 6. Furthermore, when the shutter is pressed, the selector 5 converts the image data sent from the signal processing circuit 4 into C data.
Transfer to PU7.

The image data sent to and stored in the VRAM 6 by the transfer operation of the image data by the selector 5 is displayed on the liquid crystal monitor 8 as a finder. The bitmap image data stored in the VRAM 6 is converted into an analog NTSC signal by a digital / analog (D / A) converter 9 and output from a television (TV) terminal 10. Therefore, this TV terminal
When the TV monitor 11 is connected to the TV monitor 10, the same image as the image displayed on the liquid crystal monitor 8 is displayed.
Is also displayed.

The CPU 7 compresses one frame of bitmap image data sent from the selector 5 when the shutter is pressed by an appropriate compression method, for example, the JPEG method, and compresses one frame of the compressed image data. Convert to data. The converted data is stored in the image area 12a of the flash memory 12. The data stored in the image area 12a is read out by the CPU 7, expanded to an original image, sent from the selector 5 to the VRAM 6, and stored therein.
It is displayed on the liquid crystal monitor 8 or the TV monitor 11. The flash memory 12 is provided with a program area 12b for storing a program to be described later.
It is read and executed by the PU 7.

On the other hand, the image area 12a of the flash memory 12
Can be transferred from a PC terminal 13 conforming to the RS-232C interface standard, which is a general personal computer serial terminal, to a personal computer (PC) 14, for example. Conversely, data compressed by the compression method (JPEG method) can be sent from the PC 14 through the PC terminal 13 and written in the image area 12a of the flash memory 12.

FIG. 2 is a flowchart showing a processing procedure for correcting the inclination of the image processing apparatus according to the present invention. Inputs of two systems of images are individually received (S1). DF for each image
Each power spectrum is obtained by performing T, and a power spectrum image expressing these in a polar coordinate system is obtained (S2). The luminance level is scanned in a predetermined direction of each power spectrum image, specifically, in the vertical and horizontal directions to detect a peak position, and each line segment connecting the peak position is set (S
3). An angle formed by both line segments is obtained (S4). One of the images is corrected so that the obtained angle becomes smaller (S5), and the process ends.

FIG. 3 is a flowchart showing another processing procedure relating to the inclination correction of the image processing apparatus according to the present invention.
The input of two images is individually received (S11). DFT is performed on each image to obtain respective power spectra, and a power spectrum image expressing these in a polar coordinate system is obtained (S1
2) The luminance level is scanned in a predetermined direction of each power spectrum image, specifically, in the vertical and horizontal directions to detect a peak position, and each line segment connecting the peak position is set (S13). The angle formed by both line segments is obtained (S14). One of the power spectrum images is corrected so that the obtained angle becomes smaller (S15). By subjecting the corrected power spectrum image to IDFT, an image in which the deviation in inclination has been corrected is obtained (S16), and the process ends.

FIG. 4 is a flowchart showing a processing procedure for correcting the position of the image processing apparatus according to the present invention. Receiving input of two systems of images and the number of repetitions individually (S21)
. Cut out a rectangular area near the center of one image (S22)
. A first inspection image is obtained by applying a high-pass filter to the cut-out rectangular area of the image according to a processing procedure described later.
(S23). A cutout position is set in the other image, and the rectangular area is cut out at the cutout position (S24). The high-pass filter is applied to the cut-out rectangular area of the image to obtain a second inspection image (S25).

The sum of the difference in luminance level between the pixels corresponding to the positions of the first inspection image and the second inspection image is calculated (S26), and the number of repetitions is decremented (S27). It is determined whether or not the number of repetitions has become 0, and if not, the process proceeds to S24.
And repeat the subsequent steps. When it is determined in S28 that the number of repetitions is 0, the group having the smallest sum is specified (S29). Then, one of the images is corrected so that the distance between the cut-out areas of the specified group is reduced (S30), and the process is terminated.

FIG. 5 is a flowchart showing an example of a processing procedure relating to the high-pass filter. A power spectrum is obtained by performing DFT on the cut-out rectangular area (S31). Then, the low-frequency component of the power spectrum is removed and IDFT is performed.
To obtain an inspection image (S32), and return to the calling source.

The high-pass filter is not limited to the method using the DFT as described above, but may be inspected using various edge detection operators, for example, Laplacian operators, which are frequently used in the field of image processing. A configuration for obtaining an image may be adopted.

[0039]

According to the image processing devices of the first and second inventions and the image processing devices of the third and fourth inventions as described above, deviations in the inclination and position of the imaging range between the systems are corrected by image processing. Therefore, an excellent effect is obtained in simplifying the handling of the camera at the time of imaging.

Furthermore, since images or moving images of each system received individually are to be processed, there is no need to provide a mechanism for simultaneously forming images associated with the right eye and the left eye on a single screen. And an excellent effect of miniaturization of the apparatus can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a signal processing system of an image processing apparatus according to the present invention.

FIG. 2 is a flowchart illustrating a processing procedure for correcting inclination of the image processing apparatus according to the present invention.

FIG. 3 is a flowchart showing another processing procedure relating to the correction of the inclination of the image processing apparatus according to the present invention.

FIG. 4 is a flowchart illustrating a processing procedure relating to position correction of the image processing apparatus according to the present invention.

FIG. 5 is a flowchart illustrating an example of a processing procedure relating to a high-pass filter.

FIG. 6 is an explanatory diagram for explaining the concept of the present invention.

FIG. 7 is an explanatory diagram for explaining another concept of the present invention.

FIG. 8 is an explanatory diagram for explaining a shift between imaging ranges of two imaging systems.

[Explanation of symbols]

 7 CPU 12 Flash memory 12a Image area 12b Program area

Continued on front page F-term (reference) 2F065 AA03 AA07 AA32 DD02 EE00 FF05 JJ03 JJ26 QQ03 QQ16 QQ29 QQ33 2H059 AA35 5B050 BA09 DA07 EA03 EA13 EA18 EA26 FA02 FA06 5B057 CA08 CA12 CA16 CB08 DC02 DB02

Claims (4)

[Claims] 1. An image processing apparatus for performing image processing on two images obtained by imaging the same subject so that the inclinations of the two subjects match each other. Means for obtaining, for the obtained intensity distribution image of each transform coefficient, means for respectively detecting the peak position of the luminance level on a plurality of scanning lines in a predetermined direction, and pixels for the plurality of peak positions detected for each intensity distribution image An image processing apparatus, comprising: means for calculating an angle formed by line segments connecting., And means for correcting the inclination of one of the images based on the obtained angle so as to reduce the angle. 2. An image processing apparatus for performing image processing for matching the inclinations of two subjects obtained by capturing two images of the same subject, wherein each of the images is subjected to an orthogonal transformation, and each transform coefficient is calculated. Means for obtaining, for the obtained intensity distribution image of each transform coefficient, means for respectively detecting the peak position of the luminance level on a plurality of scanning lines in a predetermined direction, and pixels for the plurality of peak positions detected for each intensity distribution image Means for determining the angle formed by the segments connecting the two, and means for correcting the inclination of one of the intensity distribution images based on the obtained angle so that the angle becomes smaller. Means for performing conversion to obtain an image. 3. An image processing apparatus for performing image processing for matching the positions of two subjects of two images obtained by capturing the same subject, wherein a region of a predetermined size is cut out at a predetermined position of one of the images. Means for applying a high-pass filter to the cut-out region of the image to obtain a first inspection image; and cutting out a region of a predetermined size at the predetermined position and a plurality of positions different from the other image, Means for applying a high-pass filter to each of the regions cut out of the image to obtain respective second inspection images, and a position corresponding to each set of the obtained first inspection image and each second inspection image. Means for calculating the sum of the differences in the luminance levels between the pixels to be calculated; means for comparing the calculated sums to determine a set with a smaller sum; and a difference in the cutout position of each region relating to the specified set. This difference Means for correcting the position of one image so as to reduce the size of the image. 4. The high-pass filter performs a quadrature transform on an area cut out of an image to obtain a transform coefficient, a means for removing a low-frequency component of the obtained transform coefficient, 4. The image processing apparatus according to claim 3, further comprising: means for performing an orthogonal inverse transform on the removed transform coefficient.