JP2012169851A - Omnidirectional imaging device and imaging condition adjustment method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately adjust an imaging condition.SOLUTION: An omnidirectional imaging device includes: a convex mirror 1 having a rotationally symmetric shape; a reference subject part 2a (2b) being a reference subject that is arranged around the top of the convex mirror 1; a lens 3 for forming the image of light from the convex mirror 1 and the reference subject part 2a (2b); an imaging element 4 for converting the light whose image is formed by the lens 3 into image signals; and an adjustment part 53 (63) for adjusting an imaging condition in accordance with an area where the reference subject part 2a (2b) is being imaged between the image signals from the imaging element 4.

Description

  The present invention relates to an omnidirectional photographing apparatus capable of simultaneously photographing omnidirectional images and a photographing condition adjusting method thereof.

  As a method for photographing an omnidirectional image, a photographing apparatus using a fisheye lens and a photographing apparatus using a rotationally symmetric convex mirror are known. The latter includes a convex mirror having a reflecting surface and a camera facing the convex mirror, and is configured to collect light from a 360 ° field of view centered on the optical axis of the camera onto a camera lens (patent) Reference 1).

  By the way, an auto white balance adjustment method for automatically adjusting the white balance of a photographed image in accordance with the light source of the photographing environment in a normal camera has been proposed. For example, a filter that scatters milky white light rays is arranged in front of the camera, and the ratio of the R (red) component to the B (blue) component in the light rays that are input to the filter is obtained. A method of making balance data (first white balance adjustment method) is known. Also, before shooting, a neutral gray standard reflector is shot and recorded as white balance data, and the TTL method (second white balance adjustment) is used to adjust the white balance of the shot image based on the white balance data. Method) is known.

  On the other hand, as a method of automatically adjusting the appropriate exposure in a normal camera, the average luminance level of the photographed image is obtained as the photometric output of the subject, and the appropriate exposure value (aperture value and exposure time) is obtained from the average value. Is generally known. However, this method has a problem that the subject to be watched becomes dark when a light source such as the sun is present in the imaging region. In view of this, there has been disclosed a method of dividing a region and estimating a subject region to be watched and performing appropriate exposure adjustment (see Patent Document 2).

JP 11-174603 A Japanese Patent Laid-Open No. 3-99585

  The conventional white balance adjustment method has the following problems. First, in the first white balance adjustment method, the direction of the light source illuminating the subject is not always constant, and there is a problem in the accuracy of the white balance data because it depends on the color of surrounding objects, etc. Perfect white balance reproduction was impossible. In the second white balance adjustment method, for example, when shooting in a state where the color temperature of the light source changes with time, such as when shooting under natural light in the morning and evening, Since the obtained white balance data is not real-time data when a still image is actually taken, there is a problem that the white balance is more out of order for images taken later in time. In particular, when taking an omnidirectional image, it is difficult to automatically adjust the white balance because the shooting range is wide and the object is often not clearly determined.

  On the other hand, the conventional exposure adjustment method has the following problems. In other words, when shooting outdoors in the daytime using an omnidirectional imaging device, the sun is often reflected directly and the proportion of the sky in the entire image increases, so the average value of all pixels is relatively high. Cheap. If exposure adjustment using the average value described above is performed in such an environment, the signal level in areas other than sunlight and sky will be unnecessarily lowered, and an appropriate exposure will be obtained such as a dark image overall. Is difficult. In addition, since there are few situations in which a subject to be watched in order to photograph all directions is present only in a certain region, it is difficult to obtain an appropriate exposure even by the method described in Patent Document 2.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an omnidirectional photographing apparatus capable of appropriately adjusting photographing conditions and a photographing condition adjusting method thereof.

  In order to solve the above problems, an omnidirectional imaging apparatus according to an embodiment of the present invention is an apparatus that captures an omnidirectional image, and includes a rotationally symmetric convex mirror (1) and a top of the convex mirror (1). A reference subject portion (2a, 2b) which is a reference subject disposed in the periphery, a lens (3) for imaging light from the convex mirror (1) and the reference subject portion (2a, 2b), and the lens ( According to the imaging device (4) for converting the light imaged in 3) into an image signal, and the region of the image signal from the imaging device (4) where the reference subject portion (2a, 2b) is photographed And an adjustment unit (53, 63) for adjusting the photographing condition.

  In the omnidirectional photographing apparatus, the reference subject portions (2a, 2b) are reference color subject portions (2a) that are subjects serving as reference colors, and the image signal from the image sensor (4) is converted into an RGB image signal. An RGB processing unit (51) for conversion, and a white balance detection unit (52) for detecting white balance from an area where the reference color subject part (2a) is photographed in the RGB image signal from the RGB processing unit (51). ) And a white balance adjustment unit (53) that adjusts the white balance of the RGB image signal from the RGB processing unit (51) according to the white balance data from the white balance detection unit (52). .

  In the omnidirectional photographing apparatus, the white balance detection unit (52) may use an average value of white balance in the photographing region of the reference color subject part (2a) as white balance data.

  In the omnidirectional photographing apparatus, the white balance detection unit (52) calculates the white balance of each pixel in the photographing region of the reference color subject portion (2a) from the center of the photographing region of the reference color subject portion (2a). The white balance data may be calculated by weighting according to the distance.

  In the omnidirectional photographing apparatus, the reference subject portion (2a, 2b) is a reference exposure subject portion (2b) that is a subject serving as a reference exposure, and performs image processing for processing an image signal from the image sensor (4). Part (61), an exposure detection part (62) for detecting an exposure value from an area where the reference exposure subject part (2b) is photographed in the luminance image signal from the image processing part (61), and the exposure An exposure adjustment unit (63) that adjusts the exposure so as to obtain an appropriate exposure according to the exposure value from the detection unit (62), and an exposure amount variation that varies the exposure amount according to an adjustment signal from the exposure adjustment unit (63). Part (64).

  In the omnidirectional photographing apparatus, the exposure detection unit (62) may use an average value of luminance in the photographing region of the reference exposure subject portion (2b) as an exposure value.

  In the omnidirectional photographing apparatus, the exposure detection unit (62) may calculate an exposure value by performing weighting according to a distance from the center of the photographing region of the reference exposure subject part (2b).

  In order to solve the above-described problems, an imaging condition adjustment method according to an embodiment of the present invention is an imaging condition adjustment method for an omnidirectional imaging apparatus that captures an omnidirectional image, and includes a rotationally symmetric convex mirror (1). A reference subject portion (2a, 2b), which is a reference subject, is disposed around the top, and light from the convex mirror (1) and the reference subject portion (2a, 2b) is imaged by a lens (3), and the lens ( The light imaged in 3) is converted into an image signal by the image sensor (4), and the image signal from the image sensor (4) is selected according to the area where the reference color subject portion (2a, 2b) is photographed. And adjusting the shooting conditions.

  According to the present invention, the subject is arranged around the top of the convex mirror, and the photographing condition is adjusted by detecting the reference value from the region where the subject is photographed. Therefore, the photographing condition can be adjusted appropriately. It is possible to provide an omnidirectional photographing apparatus capable of performing the above and a photographing condition adjusting method thereof.

It is a block diagram of the omnidirectional imaging device in 1st Embodiment. It is a figure which shows an example of the image signal obtained with the image pick-up element in 1st Embodiment. It is a block diagram of the image processing part in 1st Embodiment. It is a flowchart which shows operation | movement of the image process part in 1st Embodiment. It is a block diagram of the omnidirectional imaging | photography apparatus in 2nd Embodiment. It is a figure which shows an example of the image signal obtained with the image pick-up element in 2nd Embodiment. It is a block diagram of the signal processing part in 2nd Embodiment. It is a flowchart which shows the flow of the exposure adjustment process in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a configuration diagram of an omnidirectional photographing apparatus according to a first embodiment of the present invention. This omnidirectional imaging apparatus is an apparatus that captures omnidirectional images. As shown in FIG. 1, the omnidirectional imaging apparatus is a convex mirror 1, a reference color subject portion 2a, a lens 3, an image sensor 4, and an image processing unit 50. And. The convex mirror 1 is a reflecting mirror having a rotationally symmetric shape. The reference color subject portion 2a is a subject disposed around the top of the convex mirror 1, and is a subject that is an achromatic reference color such as normally gray. The lens 3 is disposed at a position facing the top of the convex mirror 1, and forms an image of light from the convex mirror 1 and the reference color subject portion 2a. The image sensor 4 converts an image formed by the lens 3 into an image signal and outputs the image signal. The image processing unit 50 processes the image signal from the image sensor 4. A light beam from a 360 ° surrounding subject is reflected by the convex mirror 1 and forms an image on the image sensor 4 via the lens 3.

  FIG. 2 is a diagram illustrating an example of an image signal obtained by the image sensor 4. As shown in this figure, the omnidirectional subject image 6 has an annular shape, and an image 7a of the reference color subject portion 2a is taken at the center thereof. Conventionally, since there is no reference color subject portion 2a, the central portion of the image signal obtained by the image sensor 4 is an invalid region because an effective image cannot be obtained because the lens 3 is reflected. Therefore, in this embodiment, appropriate white balance adjustment is performed using this invalid area.

  FIG. 3 is a configuration diagram of the image processing unit 50. As shown in this figure, the image processing unit 50 includes an RGB processing unit 51, a white balance detection unit 52, a white balance adjustment unit 53, a gamma processing unit 54, and a color difference processing unit 55. The RGB processing unit 51 converts the image signal from the image sensor 4 into an RGB image signal. The white balance adjustment unit 53 detects the white balance from the region in which the reference color subject portion 2 a is captured in the RGB image signal from the RGB processing unit 51. The white balance adjustment unit 53 adjusts the white balance of the RGB image signal from the RGB processing unit 51 according to the white balance data from the white balance detection unit 52. The gamma processing unit 54 performs gamma processing on the image signal from the white balance adjustment unit 53. The color difference processing unit 55 converts the image signal from the gamma processing unit 54 into a Y color difference signal and outputs it.

  FIG. 4 is a flowchart showing the operation of the image processing unit 50. Hereinafter, the configuration of the image processing unit 50 will be described in more detail with reference to FIG. First, when the image signal obtained by the image sensor 4 is input, the RGB processing unit 51 converts the image signal into an RGB image signal according to the pixel arrangement and the color filter characteristics of the image sensor 4 (step S1). . For example, when the image sensor 4 is a single-plate Bayer array, Bayer decoding is performed, and when the image sensor 4 is a complementary color system, RGB conversion is performed according to the characteristics of the complementary color filter. Next, the white balance detection unit 52 detects the white balance from the portion where the reference color subject portion 2 a at the center of the image signal from the RGB processing unit 51 is imaged and inputs the detected white balance to the white balance adjustment unit 53. (Step S2). A specific example of the white balance detection method will be described later. Next, the white balance adjustment unit 53 adjusts the white balance of the image signal from the RGB processing unit 51 according to the white balance data detected by the white balance detection unit 52 (step S3). The white balance adjustment method is not particularly limited. Next, the gamma processing unit 54 performs gamma processing on the image signal from the white balance adjustment unit 53 (step S4). Finally, the color difference processing unit 55 converts the image signal from the gamma processing unit 54 into a Y color difference signal and outputs it (step S5).

  Here, a specific example of the white balance detection method will be described. For example, the white balance detection unit 52 may use the average value of the white balance of the shooting area of the reference color subject unit 2a as white balance data. At this time, since the shooting area of the reference color subject portion 2a is achromatic, that is, an adjustment value is required so that R, G, and B have the same value, white balance is set according to the ratio of GR and GB. Calculate the data. Alternatively, the white balance detection unit 52 calculates the white balance data by weighting the white balance of each pixel in the shooting region of the reference color subject portion 2a according to the distance from the center of the shooting region of the reference color subject portion 2a. May be. For example, if the weight of the central portion is increased and the peripheral portion is reduced, the influence of subjects other than the reference color subject portion 2a mixed in the peripheral portion can be reduced.

  As described above, according to the present embodiment, the reference color subject portion 2a is arranged around the vertex of the convex mirror 1, and the white balance is detected from the area where the reference color subject portion 2a is photographed in the image signal. Therefore, the white balance can be adjusted appropriately. That is, since an achromatic image serving as a reference can be obtained in real time under any circumstances, white balance can be easily adjusted.

(Second Embodiment)
FIG. 5 is a configuration diagram of an omnidirectional photographing apparatus according to the second embodiment of the present invention. As shown in FIG. 5, this omnidirectional photographing apparatus includes a convex mirror 1, a reference exposed subject portion 2 b, a lens 3, an electronic shutter 64, an image sensor 4, and a signal processing unit 60. The convex mirror 1, the lens 3, and the image sensor 4 are the same as in the first embodiment. The reference exposure subject portion 2b is a subject that is disposed around the top of the convex mirror 1, and is a subject that becomes a standard exposure such as normally gray. The electronic shutter 64 is an exposure amount variation unit for changing the exposure amount. The signal processing unit 60 processes the image signal from the image sensor 4. A light beam from a 360 ° surrounding subject is reflected by the convex mirror 1 and forms an image on the image sensor 4 via the lens 3.

  FIG. 6 is a diagram illustrating an example of an image signal obtained by the image sensor 4. As shown in this figure, the image 6 of the omnidirectional subject has an annular shape, and an image 7b of the reference exposure subject portion 2b is taken at the center thereof. Conventionally, since there is no reference exposure subject portion 2b, the central portion of the image obtained by the image sensor 4 is an invalid region because an effective image cannot be obtained because the lens 3 is reflected. Therefore, in the present embodiment, this invalid area is used to perform appropriate exposure adjustment.

  FIG. 7 is a configuration diagram of the signal processing unit 60. As shown in this figure, the signal processing unit 60 includes an image processing unit 61, an exposure detection unit 62, and an exposure adjustment unit 63. The image processing unit 61 processes an image signal from the image sensor 4. Specifically, a luminance color difference signal is output by performing various image processing such as signal processing, gamma processing, color difference processing, and the like according to the image sensor 4 such as a Bayer array and a complementary color filter. The exposure detection unit 62 detects an exposure value from a region where the reference exposure subject portion 2b is photographed in the luminance image signal from the image processing unit 61. The exposure adjustment unit 63 adjusts the exposure so as to achieve an appropriate exposure according to the exposure value from the exposure detection unit 62. Specifically, an adjustment signal is output to the electronic shutter 64. The electronic shutter 64 varies the exposure amount (exposure time) according to the adjustment signal from the exposure adjustment unit 63. Here, the exposure adjustment is performed only by the electronic shutter, but in practice, other exposure fluctuation portions such as an aperture and a gain control may be combined.

  FIG. 8 is a flowchart showing the flow of exposure adjustment processing in the present embodiment. First, when the luminance image signal is input from the image processing unit 61, the exposure detection unit 62 detects an exposure value from an area where the reference exposure subject portion 2b is photographed in the luminance image signal (step S11). For example, the average value of the luminance of the shooting area of the reference exposure subject portion 2b is set as the exposure value. A method is also conceivable in which the exposure value is calculated by weighting according to the distance from the center of the photographing area of the reference exposure subject portion 2b instead of the average value. Next, the exposure adjustment unit 63 determines whether or not the exposure value from the exposure detection unit 62 is proper exposure (step S12). For example, the exposure target value that is the target value of the luminance of the reference exposure subject portion 2b is set in advance as Yt, and the exposure value from the exposure detection unit 62 is assumed to be Ya. In this case, if Ya = Yt, the process is terminated with proper exposure. On the other hand, if Ya ≠ Yt, it is determined that the exposure is not appropriate, and the process proceeds to step S13. Here, Ya = Yt is used as the determination criterion, but the determination criterion is not limited to this. For example, a method may be considered in which the target value is given a certain range, the minimum target value is Ytmin, the maximum target value is Ytmax, and Ytmin ≦ Ya ≦ Ytmax is determined as appropriate exposure. If it is determined that the exposure is not appropriate, exposure adjustment is performed so as to approach the exposure target value (step S13), an adjustment signal is output to the electronic shutter 64, and the process ends. For example, if the current exposure time of the electronic shutter 64 is Sc, the exposure target value is Yt, and the exposure value from the exposure detection unit 62 is Ya, Sn that is Sn = Sc + F (Yt, Ya) is output to the electronic shutter 64. . Here, F (Yt, Ya) is a function represented by the exposure target value and the exposure value. For example, when the relationship between the exposure time and the exposure value is linear, it is expressed as F (Yt, Ya) = K × (Yt−Ya) (K is a proportional constant). Actually, in many cases, it is not linear, and it is necessary to determine F (Yt, Ya) according to the characteristics of the image sensor 4.

  As described above, according to the present embodiment, the reference exposure subject portion 2b is arranged around the vertex of the convex mirror 1, and the exposure value is detected from the region where the reference exposure subject portion 2b is photographed in the image signal. Therefore, the exposure value can be adjusted appropriately. That is, it is possible to perform appropriate exposure adjustment even when shooting outdoors during the daytime.

  In addition, although the case where white balance or exposure was adjusted was demonstrated here, this invention is not limited to this. That is, as long as a subject is arranged around the top of the convex mirror 1 and some information is detected from the region where the subject is photographed to adjust the photographing condition, it is included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Convex mirror 2a Reference color subject part 2b Reference exposure subject part 3 Lens 4 Image pick-up element 6 Omni-directional subject imaging area 7a Reference color subject imaging area 7b Reference exposure subject imaging area 50 Image processing part 51 RGB processing part 52 White balance detection part 53 White Balance adjustment unit 54 Gamma processing unit 55 Color difference processing unit 60 Signal processing unit 61 Image processing unit 62 Exposure detection unit 63 Exposure adjustment unit 64 Electronic shutter

Claims (8)

An omnidirectional imaging device that captures omnidirectional images,
A rotationally symmetric convex mirror;
A reference subject portion that is a reference subject disposed around the top of the convex mirror;
A lens that forms an image of light from the convex mirror and the reference subject portion;
An image sensor that converts light imaged by the lens into an image signal;
An adjustment unit that adjusts imaging conditions according to an area in which the reference subject portion is imaged in the image signal from the image sensor;
An omnidirectional photographing apparatus comprising: The reference subject portion is a reference color subject portion that is a subject that becomes a reference color;
An RGB processing unit for converting an image signal from the image sensor into an RGB image signal;
A white balance detection unit that detects white balance from an area in which the reference color subject portion is imaged in the RGB image signal from the RGB processing unit;
A white balance adjustment unit that adjusts the white balance of the RGB image signal from the RGB processing unit in accordance with white balance data from the white balance detection unit;
The omnidirectional imaging apparatus according to claim 1, further comprising:   3. The omnidirectional imaging apparatus according to claim 2, wherein the white balance detection unit uses, as white balance data, an average value of white balance in a shooting region of the reference color subject portion.   The white balance detection unit calculates white balance data by weighting the white balance of each pixel in the shooting region of the reference color subject portion according to the distance from the center of the shooting region of the reference color subject portion. The omnidirectional imaging apparatus according to claim 2, characterized in that: The reference subject portion is a reference exposure subject portion that is a subject serving as a reference exposure.
An image processing unit for processing an image signal from the image sensor;
An exposure detection unit that detects an exposure value from an area in which the reference exposure subject portion is captured in the luminance image signal from the image processing unit;
An exposure adjustment unit that adjusts the exposure so as to obtain an appropriate exposure according to an exposure value from the exposure detection unit;
An exposure amount variation unit that varies the exposure amount according to an adjustment signal from the exposure adjustment unit;
The omnidirectional imaging apparatus according to claim 1, further comprising:   6. The omnidirectional photographing apparatus according to claim 5, wherein the exposure detection unit uses an average value of luminance of a photographing region of the reference exposed subject portion as an exposure value.   The omnidirectional imaging apparatus according to claim 5, wherein the exposure detection unit calculates an exposure value by performing weighting according to a distance from a center of the imaging region of the reference exposure subject part. An imaging condition adjustment method for an omnidirectional imaging device that captures an omnidirectional image,
Place the reference subject part that is the reference subject around the top of the rotationally symmetrical convex mirror,
The light from the convex mirror and the reference subject portion is imaged by a lens,
The light imaged by the lens is converted into an image signal by an image sensor,
An imaging condition adjustment method for an omnidirectional imaging apparatus, characterized in that imaging conditions are adjusted according to a region in which the reference color subject portion is imaged in an image signal from the imaging element.

Images