JP2004128588A - Imaging unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging unit for easily and accurately photographing images of an object having a consecutive positional relation to each other. <P>SOLUTION: The imaging apparatus 1 satisfying the requirement above displays a photographing auxiliary image generated from an image photographed before in performing panorama photographing or stereoscopic photographing. Thus, a photographer utilizes the photographing auxiliary image to be capable of easily and accurately photographing an image in matching with the image photographed earlier. That is, since a plurality of images subjected to the panorama photographing by the imaging apparatus 1 result from accurately consecutively photographing an object, compositing the images by a PC or the like can obtain a panorama image with high consistency. Further, since a plurality of the images stereoscopically photographed by the imaging unit 1 result from photographing all faces of the object at a proper angle, at a proper position, and at a prescribed angular interval, compositing the images by the PC or the like can obtain a stereoscopic image with high consistency. <P>COPYRIGHT: (C)2004,JPO

Description

【００４５】
また、第１および第２の実施の形態において、撮影補助画像を射影変換する場合について述べたが、水平面上のＸ−Ｙ軸と鉛直方向に伸びるＺ軸とからなる座標系を用い、これらの座標軸を回転軸とすることが可能である。また、撮影者が撮影し易いように、原点を含め、座標系を変化させることとしてもよい。
また、第１および第２の実施の形態において、撮影補助画像を生成するために、撮像部２０が備える撮像素子の全てによる出力信号を用いるのではなく、例えば、表示部７０の表示する画像に対応する画素のみを用いることが可能である。例えば、３００万画素の品質で撮影可能な撮像素子において、表示部７０が３０万画素の表示能力であれば、全撮像素子のうち、表示部７０の３０万画素に対応する画素のみを用いて撮影補助画像を生成することが可能である。
【００４６】
このように、撮像素子の出力信号を間引いて使用することにより、次に撮影するアングルを決定する間における消費電力を低減させることができる。
【００４７】
【発明の効果】
本発明によれば、パノラマ撮影あるいは立体撮影等の特殊撮影を行う際、先に撮影された画像から生成された撮影補助画像が表示される。
したがって、撮影者は、撮影補助画像を利用して、先に撮影された画像と整合する画像を容易かつ正確に撮影することができる。
【００４８】
即ち、このようにしてパノラマ撮影された複数の画像は、被写体が正確に連続して撮影されたものであるため、ＰＣ（Ｐｅｒｓｏｎａｌ　Ｃｏｍｐｕｔｅｒ）等によって合成することにより、整合性の高いパノラマ画像とすることができる。
また、このようにして立体撮影された複数の画像は、被写体のすべての面を所定角度おきに、適切な位置から適切なアングルで撮影したものであるため、ＰＣ等で合成することにより、整合性の高い立体映像とすることができる。
【図面の簡単な説明】
【図１】撮像装置１の機能構成を示すブロック図である。
【図２】撮像装置１が行う処理を示すフローチャートである。
【図３】ガイド付画像の表示形態の例を示す図である。
【図４】撮像装置２の機能構成を示すブロック図である。
【図５】撮像装置２が行う処理を示すフローチャートである。
【図６】ガイド付画像の表示形態の例を示す図である。
【符号の説明】
１，２　撮像装置，１０　シャッタ，２０　撮像部，３０，３１　処理部，４０
画像情報記憶部，５０　入力部，６０，６１　画像合成部，７０　表示部[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an imaging device capable of capturing a still image of a subject.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, so-called panoramic photography has been performed using an imaging device such as a digital camera. In the case of performing panoramic photography, as one method, subjects such as landscapes are continuously photographed while the photographing direction is slightly shifted, and the photographed images are combined later (for example, see Patent Document 1). ).
[0003]
At this time, first, the subject is photographed at a predetermined angle of view. Then, the photographer stores a characteristic portion in the angle of view, and determines an adjacent angle of view with reference to the memorized portion. Then, the photographer shoots at the determined angle of view, and thereafter, repeats the same operation as appropriate to obtain a plurality of images, and then synthesizes the images.
Also, with regard to flowers, hair styles, etc., there is a demand for capturing almost the entire surface of a three-dimensional subject, so that a plurality of angles of view that the photographer can photograph the entire surface such as the front, side, rear, and top surfaces are considered. Is appropriately selected and each of them is photographed (hereinafter, such a photographing method is referred to as “stereoscopic photographing”).
[0004]
[Patent Document 1]
JP-A-11-098397
[Problems to be solved by the invention]
However, when shooting a subject by dividing it into a plurality of angles of view, such as panoramic shooting and stereoscopic shooting, it is not easy to shoot the subject without gaps and without unnecessary duplication. It was very difficult for a beginner.
[0006]
An object of the present invention is to easily and accurately capture images of a subject having a continuous positional relationship with each other.
[0007]
[Means for Solving the Problems]
In order to solve the above problems, the invention described in claim 1 is
An image capturing apparatus including a display device that displays an image of a subject at the time of capturing the subject, the subject having a continuous positional relationship with the subject (for example, a continuous The image processing apparatus is characterized in that a shooting assistance image for shooting an image of a scene to be shot, a different surface of one object, or the like) is generated and can be displayed on the display device.
[0008]
The invention according to claim 2 is
The imaging device according to claim 1,
The imaging assistance image is characterized in that it includes an image that transparently displays a captured image of a previously captured subject.
Here, “transparently” displaying an image refers to a display mode in which one image is displayed thinly on another image so that any content of the superimposed image can be visually recognized.
[0009]
The invention according to claim 3 is
The imaging device according to claim 1,
The photographing auxiliary image includes an image indicating an edge in a photographed image of a subject photographed before.
The invention according to claim 4 is
The imaging device according to claim 1,
It is characterized in that, based on a previously captured image of a predetermined landscape, the auxiliary shooting image for capturing an image of a landscape adjacent to the predetermined landscape is generated.
[0010]
That is, as a main use of the present invention, a case of performing panoramic shooting is assumed.
The invention according to claim 5 is
The imaging device according to claim 1,
The photographing auxiliary image is an image obtained by rotating a photographed image of a subject previously photographed three-dimensionally.
[0011]
That is, as a main application of the present invention, a case where stereoscopic imaging is performed is assumed.
Here, the “three-dimensional rotation” means that an image displayed three-dimensionally on a two-dimensional plane is rotated.
The invention according to claim 6 is
The imaging device according to claim 5,
It is provided with a distance detecting means for detecting a distance to at least a part of the photographed subject, and generates an image in which the subject is three-dimensionally rotated based on the detected distance.
[0012]
The invention according to claim 7 is
The imaging device according to claim 6,
The distance detecting means is capable of detecting a distance from a subject portion corresponding to a location arbitrarily selected in the subject image (for example, a location pointed out by a photographer using a predetermined pointing method). .
[0013]
The invention according to claim 8 is
The imaging device according to claim 6,
The distance detecting means can detect a distance from a subject portion corresponding to a predetermined location (for example, a central location of each area of a screen divided into a predetermined pattern) in an image of the subject. I have.
[0014]
The invention according to claim 9 is
An imaging device according to any one of claims 1 to 8,
An imaging unit including a plurality of imaging elements is provided, and the imaging assistance image is generated by an image generated based on output signals of some of the plurality of imaging elements.
[0015]
For example, it is possible to use only an output signal of an image sensor corresponding to a pixel displayed on a finder liquid crystal display device or the like among all the provided image sensors.
According to the present invention, when performing special shooting such as panoramic shooting or stereoscopic shooting, a shooting auxiliary image generated from a previously shot image is displayed.
[0016]
Therefore, the photographer can easily and accurately photograph an image matching the previously photographed image by using the photographing auxiliary image.
That is, since a plurality of images taken in this manner are panorama-photographed, since the subject is photographed accurately and continuously, the images are synthesized by a PC (Personal Computer) or the like to obtain a highly consistent panoramic image. be able to.
[0017]
In addition, since a plurality of images stereoscopically photographed in this manner are images obtained by photographing all surfaces of the subject at a predetermined angle from an appropriate position and at an appropriate angle, the images are synthesized by a PC or the like, so that matching is performed. It is possible to make a stereoscopic image with high quality.
[0018]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of an imaging device according to the present invention will be described in detail with reference to the drawings.
(First Embodiment)
First, the configuration will be described.
FIG. 1 is a block diagram illustrating a functional configuration of an imaging device 1 according to the first embodiment of the present invention. 1, the imaging apparatus 1 includes a shutter 10, an imaging unit 20, a processing unit 30, an image information storage unit 40, an input unit 50, an image synthesis unit 60, and a display unit 70. You.
[0019]
The shutter 10 outputs a signal (hereinafter, referred to as a “shutter signal”) for instructing shooting to the processing unit 30 in response to being pressed.
The imaging unit 20 includes an imaging element such as a charge coupled diode (CCD), converts an image of a subject received through a lens into an electric signal, and generates an image signal indicating a predetermined image. Then, the imaging unit 20 outputs the generated image signal to the processing unit 30 and the image combining unit 60.
[0020]
The processing unit 30 receives a shutter signal when the shutter 10 is pressed down, and always receives an image signal from the imaging unit 20. Then, based on the image based on the image signal at the time when the shutter signal is input (that is, the photographed image), the processing unit 30 performs an image for assisting photographing in panoramic photographing or stereoscopic photographing (hereinafter referred to as “photographing assist” Image ").
[0021]
Specifically, the processing unit 30 first performs edge detection processing (edge detection and edge strength detection) on the captured image, and binarizes the detected edge with a predetermined threshold. Then, the processing unit 30 performs the projective transformation by rotating the binarized edge image (shooting auxiliary image) by a predetermined amount as necessary. The amount of rotation corresponds to the angle of view of the subject to be shot next (for example, turning around 15 degrees), and can be set by the photographer.
[0022]
Then, the processing unit 30 outputs the generated shooting assistance image to the image information storage unit 40.
Here, in order to rotate the shooting auxiliary image and perform projective conversion, the distance to the subject must be known. Therefore, means for detecting the distance to the subject is required.
[0023]
As the means, the following method is possible.
First, the photographer points out a point considered to be important in performing the projective transformation, such as a point having a large change in distance, in the photographing auxiliary image, and acquires the distance at that point. As means for pointing out a point, the subject is displayed such that the point to be pointed out is located at the center of the display unit 70 (the position where focus is achieved), and a predetermined button is pressed (for example, the shutter is half-pressed). The imaging device 1 may be provided with a pointing device capable of detecting a distance or arbitrarily indicating an image displayed on the display unit 70, and detecting a distance from the subject at the indicated point. It is possible.
[0024]
In pointing out the points, it is desirable to point out the points as uniformly as possible in the entire image.
Next, the distance at each point is determined using an autofocus function conventionally used in digital cameras and the like.
At this time, as described above, in addition to the mode in which the photographer points out the point, the entire screen is divided into a plurality of regions in advance, and the distance in each region is detected, or the distance is detected by each image sensor. It is also possible.
[0025]
Then, using the distance of each point obtained in this way, the coordinates in a portion between the points are interpolated, and the projection transformation is performed.
The image information storage unit 40 stores the photographing auxiliary image input from the processing unit 30. The input unit 50 includes an input button and the like, and receives an instruction input from a photographer. That is, the input unit 50 receives an input from the photographer regarding whether to perform panoramic imaging or stereoscopic imaging (imaging method) and an imaging direction, and outputs the input content to the image synthesizing unit 60.
[0026]
Here, in the panoramic shooting, the shooting direction refers to whether the subject is shifted in the vertical or horizontal direction, and in the stereoscopic shooting, the subject is moved in any direction around the subject. It means whether to shoot.
The image synthesizing unit 60 receives a photographing method (either panoramic photographing or stereoscopic photographing) and a photographing direction from the input unit 50, and always receives an image signal from the photographing unit 20.
[0027]
Then, the image synthesizing unit 60 synthesizes the photographing auxiliary image stored in the image information storage unit 40 with the image signal based on the photographing method and the photographing direction input from the input unit 50, and forms the photographing auxiliary The image is output to the display unit 70 as an image on which the images are superimposed (hereinafter, referred to as an “image with guide”).
In addition, in the panoramic shooting, the ratio of overlapping the shooting auxiliary image with the shot image can be set in advance by the photographer.
[0028]
The display unit 70 is configured by a liquid crystal display device in a finder or a larger liquid crystal monitor, and displays a guided image input from the image synthesizing unit 60.
Next, the operation will be described.
FIG. 2 is a flowchart illustrating a process performed by the imaging device 1.
In FIG. 2, when the shutter 10 is pressed (step S1), the imaging device 1 detects an edge from the captured image (step S2), and binarizes the detected edge to obtain a predetermined imaging auxiliary image ( Here, an image composed of edges is generated (step S3).
[0029]
Next, when stereoscopic imaging is selected, the imaging device 1 performs projective transformation of the photographing auxiliary image (step S4), and stores the converted photographing auxiliary image in the image information storage unit 40 (step S5). . When panoramic shooting is selected, a part of the image obtained by translating the shooting auxiliary image is used as the shooting auxiliary image.
Then, the imaging device 1 combines the image captured by the imaging unit 20 and the shooting assistance image to generate a guided image (step S6).
[0030]
The guide-equipped image thus generated is displayed on the display unit 70 in a form as shown in FIG. In FIG. 3, the ridge line of the building and the mountain displayed on the upper left side of the screen is an edge image indicating a photographed subject portion generated as a photographing auxiliary image, and the other portions are currently displayed by the imaging unit 20. It is an image that has been taken.
[0031]
Then, the photographer matches the edge displayed in the image with a guide with a part of the subject, so that a continuous image (an image of an adjacent angle of view of panoramic imaging or an image of an angle of view before and after stereoscopic imaging) is obtained. To shoot.
In the first embodiment, the process of binarizing an edge is performed to make the edge easier to see. A predetermined threshold value is set in advance, and if the detected edge is smaller than the threshold value. The luminance can be set to “0”, and conversely, if it is large, the luminance can be set to “255”. The photographer can set an arbitrary threshold value.
[0032]
Further, in the first embodiment, the edge image generated as the photographing auxiliary image can change the strength of the edge based on the distance of each part of the edge from the subject.
For example, at each position of the detected edge, an edge closer to the subject is a strong edge, while an edge farther from the subject is a weak edge. Specifically, the brightness of the edge of the relatively close subject portion is set to “255”, and the brightness of the edge of the relatively far subject portion is set to “128”.
[0033]
Further, in the first embodiment, the color of the displayed edge can be determined corresponding to the color of the portion detected as the edge in the captured image. For example, when an edge is detected for a blue portion of the captured image, the edge color is set to red, and when an edge is detected for a black portion of the captured image, the edge color is set to white. In addition, it is possible to display edges with colors having complementary colors to each other. By doing so, the photographing auxiliary image becomes easy to see.
(Second embodiment)
Next, a second embodiment of the present invention will be described.
[0034]
First, the configuration will be described.
FIG. 4 is a block diagram illustrating a functional configuration of the imaging device 2 according to the second embodiment of the present invention. 4, the functional configuration of the imaging device 2 is the same as that of the imaging device 1 shown in FIG. 1 except that the image information storage unit 40 is not provided and the functions of the processing unit 31 and the image synthesizing unit 61 are different. , Only the processing unit 31 will be described, and other parts will be denoted by the same reference numerals and description thereof will be omitted.
[0035]
4, a shutter signal is input to the processing unit 31 when the shutter 10 is pressed, and an image signal is constantly input from the imaging unit 20. Then, the processing unit 30 stores an image (that is, a captured image) based on the image signal when the shutter signal is input, as a photographing auxiliary image for assisting photographing.
At this time, when stereoscopic imaging is selected, the processing unit 31 stores the input captured image after projecting the image by rotating it by a predetermined amount as necessary. The amount of rotation corresponds to the angle of view of the subject to be shot next (for example, turning around by 15 degrees).
[0036]
Then, the processing unit 31 outputs the stored shooting assistance image to the image combining unit 60.
The image synthesizing unit 61 receives from the input unit 50 a shooting method (either panoramic shooting or stereoscopic shooting) and a shooting direction, and an image signal from the imaging unit 20 at all times.
[0037]
Then, the image synthesizing unit 61 synthesizes the photographing auxiliary image stored in the image information storage unit 40 by performing an α blending process with the image signal based on the photographing method and the photographing direction input from the input unit 50. The image is output to the display unit 70 as a guide-added image in which the shooting auxiliary image is superimposed on the image of the subject.
At this time, it is possible to vary the transmittance at the time of performing the α blending process on each part of the photographing auxiliary image according to the distance to the subject.
[0038]
Note that the image synthesizing unit 61 may perform synthesis by a method other than the α blending process as long as the image can be synthesized transparently.
Next, the operation will be described.
FIG. 5 is a flowchart illustrating a process performed by the imaging device 2.
In FIG. 5, when the shutter 10 is pressed (step P1), the imaging device 2 stores the captured image as a photographing auxiliary image in the processing unit 31 (step P2). At this time, when stereoscopic imaging is selected, the imaging device 2 performs projective transformation on the input captured image. When panoramic shooting is selected, a part of an image obtained by translating the shot image is used as a shooting auxiliary image.
[0039]
Next, the imaging device 2 generates an image with a guide by synthesizing the image photographed by the imaging unit 20 and the photographing auxiliary image.
At this time, the imaging device 2 combines the photographed image and the photographing auxiliary image by the α blending process. Therefore, the photographing auxiliary image is combined with the photographed image in a semi-transparent state.
[0040]
The image with a guide generated in this way is displayed on the display unit 70 in a form as shown in FIG. In FIG. 6, a translucent image indicating a photographed subject portion generated as a photographing auxiliary image is displayed in a superimposed manner on a shaded portion on the left side of the screen. The image indicated by the dotted line included in the shaded portion is a photographing auxiliary image, and the other images are images currently photographed by the imaging unit 20.
[0041]
Then, the photographer superimposes the translucent image displayed on the guide-attached image with a part of the subject to form a continuous image (an image having an adjacent angle of view in panoramic imaging or an angle of view before and after stereoscopic imaging). Image).
As described above, when performing the panoramic imaging or the stereoscopic imaging, the imaging devices 1 and 2 to which the present invention is applied display the imaging auxiliary image generated from the previously captured image.
[0042]
Therefore, the photographer can easily and accurately photograph an image matching the previously photographed image by using the photographing auxiliary image.
That is, since a plurality of images taken in this manner are panorama-photographed, since the subject is photographed accurately and continuously, the images can be combined with a PC or the like to obtain a highly consistent panoramic image.
[0043]
In addition, since a plurality of images stereoscopically photographed in this manner are images obtained by photographing all surfaces of the subject at a predetermined angle from an appropriate position and at an appropriate angle, the images are synthesized by a PC or the like, so that matching is performed. It is possible to make a stereoscopic image with high quality.
In the first and second embodiments, the edge detection process is performed on the captured image. However, the edge detection is performed by superimposing the following matrix on the luminance component of the captured image. It is possible.
[0044]
(Equation 1)

[0045]
Further, in the first and second embodiments, the case where the shooting auxiliary image is projectively transformed has been described. However, a coordinate system composed of an XY axis on a horizontal plane and a Z axis extending in a vertical direction is used. The coordinate axis can be a rotation axis. Further, the coordinate system including the origin may be changed so that the photographer can easily take a picture.
Further, in the first and second embodiments, in order to generate a photographing auxiliary image, instead of using output signals from all of the image pickup devices included in the image pickup unit 20, for example, an image displayed on the display unit 70 is used. It is possible to use only the corresponding pixels. For example, in an image sensor capable of shooting with a quality of 3 million pixels, if the display unit 70 has a display capability of 300,000 pixels, only pixels corresponding to 300,000 pixels of the display unit 70 among all the image sensors are used. It is possible to generate a shooting assistance image.
[0046]
As described above, by decimating and using the output signal of the image sensor, it is possible to reduce power consumption during determination of an angle to be photographed next.
[0047]
【The invention's effect】
According to the present invention, when performing special shooting such as panoramic shooting or stereoscopic shooting, a shooting auxiliary image generated from a previously shot image is displayed.
Therefore, the photographer can easily and accurately photograph an image matching the previously photographed image by using the photographing auxiliary image.
[0048]
That is, since a plurality of images taken in this manner are panorama-photographed, since the subject is photographed accurately and continuously, the images are synthesized by a PC (Personal Computer) or the like to obtain a highly consistent panoramic image. be able to.
In addition, since a plurality of images stereoscopically photographed in this manner are images obtained by photographing all surfaces of the subject at a predetermined angle from an appropriate position and at an appropriate angle, the images are synthesized by a PC or the like, so that matching is performed. It is possible to make a stereoscopic image with high quality.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a functional configuration of an imaging device 1.
FIG. 2 is a flowchart illustrating a process performed by the imaging apparatus.
FIG. 3 is a diagram illustrating an example of a display mode of an image with a guide.
FIG. 4 is a block diagram showing a functional configuration of the imaging device 2.
FIG. 5 is a flowchart showing a process performed by the imaging device 2.
FIG. 6 is a diagram illustrating an example of a display mode of an image with a guide.
[Explanation of symbols]
1, 2 imaging device, 10 shutter, 20 imaging unit, 30, 31 processing unit, 40
Image information storage unit, 50 input unit, 60, 61 image synthesis unit, 70 display unit

Claims (9)

An imaging apparatus including a display device that displays an image of a subject when shooting the subject,
A photographing auxiliary image for photographing an image of a subject having a continuous positional relationship with the subject is generated based on a photographed image of the subject previously photographed, and can be displayed on the display device. Imaging device. The imaging apparatus according to claim 1, wherein the photographing auxiliary image includes an image that transparently displays a photographed image of a subject previously photographed. The imaging apparatus according to claim 1, wherein the photographing auxiliary image includes an image indicating an edge in a photographed image of a previously photographed subject. The photographing auxiliary image for photographing an image of a scenery adjacent to the scenery is generated based on a photographed image of a predetermined scenery previously photographed, according to claim 1. Imaging device. The imaging apparatus according to any one of claims 1 to 3, wherein the imaging assistance image is an image obtained by rotating a captured image of a subject previously captured three-dimensionally. 6. The image processing apparatus according to claim 5, further comprising a distance detecting unit configured to detect a distance to at least a part of the photographed subject, and generating an image obtained by rotating the subject three-dimensionally based on the detected distance. Imaging device. 7. The imaging apparatus according to claim 6, wherein the distance detection unit is capable of detecting a distance from a subject portion corresponding to a position arbitrarily selected in the subject image. 7. The imaging apparatus according to claim 6, wherein the distance detection unit is capable of detecting a distance from a subject portion corresponding to a predetermined location in an image of the subject. 9. The image capturing device according to claim 1, further comprising: an image pickup unit including a plurality of image pickup devices, wherein the image pickup auxiliary image is generated by an image generated based on a part of output signals of the plurality of image pickup devices. An imaging device according to any one of the above.

Images