JP2010258669A - Omnidirectional imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a panoramic image of an excellent image even though characteristic differences with lapse of time among respective cameras occur in an omnidirectional imaging apparatus in which a plurality of cameras are arranged radially to image a panoramic image of the full 360 degrees. <P>SOLUTION: The omnidirectional imaging apparatus includes: a first imaging system A composed of a plurality of imaging units arranged such that edge portions of photographic field angles of adjacent imaging units A1 to A4 overlap each other to image an object image of the full 360 degrees; a second imaging system B composed of a plurality of imaging units B1 to B4 provided adjacently to the respective imaging units A1 to A4 of the first imaging system A; and a driving means for moving the second imaging system B relatively with respect to the first imaging system A to change pairs of the imaging units of the first imaging system A and the imaging units of the second imaging system B which are adjacent. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an omnidirectional imaging apparatus that arranges a plurality of imaging units and captures a subject image of 360 degrees around.

  Patent Documents 1 and 2 listed below describe omnidirectional imaging devices that can arrange a plurality of cameras (imaging units) in a ring shape and capture a 360-degree panoramic image. In such an omnidirectional imaging apparatus, it is preferable to prepare a plurality of cameras having a small characteristic difference, that is, a small individual difference because of the panoramic composition of the captured images of the cameras using a plurality of cameras.

  However, if the omnidirectional imaging device is used for a long time, the characteristics of each camera will vary over time, and when panoramic synthesis is performed, the result is that one image part has high brightness and another image part has severe shading. End up.

  If the difference in characteristics of each camera can be corrected without disassembling the omnidirectional imaging apparatus after a certain period of use, an image with uniform characteristics can be obtained with all cameras. For example, in the compound eye camera described in Patent Document 3 below, individual difference information is extracted from the compound eye camera and the compound eye camera is controlled.

  However, since this compound-eye camera captures the same subject, the individual difference between the two cameras can be obtained. However, in the omnidirectional imaging apparatus, for example, the captured image of the camera in the direction of backlight against sunlight is The subject and exposure amount are completely different from the captured image of the camera in the direction of light, and the difference between the subject images is too large to determine the characteristic difference between the cameras.

Japanese Patent Laid-Open No. 11-98342 JP 2004-61808 A JP-A-11-146425

  An object of the present invention is to enable an omnidirectional imaging device that arranges a plurality of cameras and shoots a subject image of 360 degrees around to obtain an omnidirectional captured image having uniform characteristics over time.

  An omnidirectional imaging device according to the present invention includes a first imaging system configured by a plurality of imaging units that are arranged so that end portions of shooting field angles of adjacent imaging units overlap and that capture a subject image of 360 degrees around the imaging unit. A second imaging system comprising a plurality of imaging units provided adjacent to each imaging unit of the first imaging system; and the adjacent by moving the second imaging system relative to the first imaging system Driving means for changing a pair of the imaging unit of the first imaging system and the imaging unit of the second imaging system.

  According to the present invention, even if a characteristic difference occurs over time between a plurality of imaging units, a relative characteristic difference between each imaging unit can be easily detected, so image correction that absorbs the characteristic difference is performed, It is possible to obtain an omnidirectional captured image with always good image quality.

It is the side view (a) and top view (b) of the omnidirectional imaging device which concern on one Embodiment of this invention. It is a block block diagram of the camera shown in FIG. It is a function block diagram of the control part shown in FIG. It is a flowchart which shows the process sequence at the time of the individual difference detection of the omnidirectional imaging device shown in FIG. It is explanatory drawing of the omnidirectional imaging device which concerns on other embodiment of this invention.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1A is a side view of an omnidirectional imaging apparatus according to an embodiment of the present invention, and FIG. 1B is a top view thereof. The omnidirectional imaging apparatus 1 of this embodiment includes an imaging system A and an imaging system B. The imaging system A includes four cameras (imaging units) A1 to A4, and the imaging system B also includes four cameras (imaging). Part) B1-B4 are provided.

  The imaging system A includes a cylindrical housing 2, and cameras A1, A2, A3, A4 are fixedly installed on the peripheral wall portion every 90 degrees. The imaging system B also includes a cylindrical housing 3, and cameras B1, B2, B3, and B4 are fixedly installed every 90 degrees around the casing 3. And the housing | casing 2 is attached with respect to the housing | casing 3 so that rotation is possible as shown by the double-headed arrow X centering on the axis | shaft 4. FIG.

  The cameras A1 to A4 arranged in a ring shape and a radial shape around the axis 4 of the imaging system A are installed so that the respective shooting angles of view overlap at the end portions, and subject images of 360 degrees around the axis 4 Can be photographed by four cameras A1 to A4. Similarly, the cameras B1 to B4 of the imaging system B are also installed so that the respective shooting angles of view overlap at the end portions, and subject images of 360 degrees around the axis 4 are taken by the four cameras B1 to B4. It can be done.

  In the example shown in FIG. 1A, the imaging system B is installed on the ground side, and the camera A4 on the sky side (upper side) and the lower camera B4 form a pair of adjacent compound cameras. The incident optical axes of the camera A4 and the camera B4 are parallel to each other, and the shooting angle of view is shifted by the incident optical axis position difference (camera position difference) between the cameras A4 and B4. The same applies to the other compound eye cameras (A1, B1) (A2, B2) (A3, B3).

  Here, since the image pickup system A is provided so as to be rotatable with respect to the image pickup system B, when the image pickup system A is rotated by 90 degrees, the pair of upper and lower cameras of the pair becomes (A1, B2) (A2, B3 ) (A3, B4) (A4, B1). If the camera is further rotated 90 degrees, the paired upper and lower camera pairs are (A1, B3) (A2, B4) (A3, B1) (A4, B2). The upper and lower pairs are (A1, B4) (A2, B1) (A3, B2) (A4, B3).

  That is, all combinations of the cameras A1 to A4 of the imaging system A and the cameras B1 to B4 of the imaging system B can be realized. As described above, the paired camera pair is parallel to the incident optical axis because the shooting angle of view is shifted by the camera position difference (position difference between the cameras A4 and B4 in FIG. 1A). Most of them will shoot the same subject. Therefore, it is possible to obtain the characteristic difference between the two cameras from the difference between the captured images of the two cameras.

  The present invention is not limited to the embodiment in which an image of 360 degrees is captured by the four cameras shown in FIG. 1B, and may be an embodiment in which eight cameras are installed every 45 degrees, for example. In addition, although 12 cameras may be installed every 30 degrees, for convenience of explanation, the number is 4 in this embodiment. In addition, it is also possible to have a configuration in which two cameras using fisheye lenses are arranged back to back to capture a 360-degree image.

  FIG. 2 is a configuration diagram of the camera A1 shown in FIG. The other cameras A2 to A4 and B1 to B4 have the same configuration. The camera A1 includes a solid-state image sensor 11. A diaphragm 12 is disposed in front of the solid-state image sensor 11, and a photographing lens 13 is disposed in front thereof. A picked-up image signal output from the solid-state image pickup device 11 includes a CDSVGA section (CDS: correlated double sampling processing, VGA: variable gain amplifier) 14, an analog / digital (A / D) conversion section 15, and a white balance (WB) calculation. The signal is input to the control unit 9 through the unit 16. In the present embodiment, the control unit 9 is provided in common for the eight cameras A1 to A4 and B1 to B4, and the omnidirectional imaging device 1 is centrally controlled by the single control unit 9.

  In addition, the camera A1 has a timing generator (TG) 17 that generates various timing pulses based on an instruction signal from the control unit 9, and an aperture drive that adjusts the opening amount of the diaphragm 12 based on the instruction signal from the control unit 9. A circuit 18, a lens drive circuit 19 that adjusts the focal position of the photographic lens 13 based on an instruction signal from the control unit 9, and an image sensor that drives the image sensor 11 by an image sensor drive pulse output from the timing generator 17. And a drive circuit 20. Further, the CDSVGA unit 14, the A / D conversion unit 15, and the WB calculation unit 16 operate according to each timing pulse output from the timing generator 17.

  FIG. 3 is a functional configuration diagram of the control unit 9. The control unit 9 controls the rotation of the imaging system A at a predetermined angle relative to the imaging system B in response to an instruction from the central processing unit (CPU) 9a for managing and controlling the entire omnidirectional imaging apparatus 1 and the CPU 9a. A rotation control unit 9b for displaying, image data captured and processed by each camera, a display unit 9c for displaying a menu screen, an operation mode, and the like. A drive signal generation unit 9d that outputs to the generator 17, an operation unit 9e that inputs an instruction from an operator, and a memory 9f that is used for storing adjustment parameters and captured image data of each camera and image processing. Prepare.

  The controller 9 further includes a white balance controller 9g that comprehensively controls the white balance between the images of the image signals output from the eight cameras A1 to A4 and B1 to B4, and the exposure value of each camera. Exposure control unit 9h that determines and outputs to the aperture driving circuit 18, a focus control unit 9i that comprehensively determines the focus position of each camera and outputs it to the lens driving circuit 19, and a time measuring unit 9j that measures time Then, well-known image processing such as γ correction, offset correction, and RGB / YC conversion processing is performed on the captured image signal output from each camera, and a 360-degree panoramic image is synthesized to generate a JPEG image or And a digital signal processing unit 9k for compressing the MPEG image or the like.

  In the omnidirectional imaging apparatus 1 of the embodiment, as shown in FIG. 2, the CDSVGA 14, A / D 15,..., The lens drive circuit 19 and the like are provided in each camera. In the case where reduction is intended, these can be provided in the control unit 9. However, since the image sensor driving circuit 20 is easily affected by noise, the image sensor driving circuit 20 is preferably arranged in the vicinity of the image sensor 11 in each camera.

  When the omnidirectional imaging device 1 is manufactured, it is usual to select and mount eight imaging elements 11 having the same characteristics as the imaging elements 11 used for the eight cameras. Characteristics include sensitivity, noise, luminance shading, and color shading.

  However, the characteristics do not completely match, and even if the image sensor 11 whose characteristics are completely matched is used, the omnidirectional imaging device 1 can be used as a fixed point camera, a monitoring camera, or the like for a long time. If used, the characteristics will vary over time between the eight cameras. For example, a camera that often receives direct sunlight is more likely to deteriorate the characteristics of the image sensor 11 due to the influence of ultraviolet rays than a camera that does not receive direct sunlight.

  However, even if variations occur between the characteristics of the eight image sensors 11 and individual differences occur, by performing appropriate correction processing based on the individual differences on the captured image signals output from the image sensors 11, the characteristics can be obtained. An image captured by the deteriorated image sensor does not stand out as a deteriorated image in a 360 degree panoramic image.

  For example, in the case of sensitivity, correction is performed by multiplying the gain of the VGA unit 14 by a certain coefficient and adjusting the coefficient to correct the sensitivity difference. In the case of noise, correction is performed by adjusting a parameter that determines the strength of noise reduction in image signal processing. In the case of luminance shading or color shading, correction is performed by multiplying each output value at a specified position in the image effective area by a fixed value. Usually, the area is subdivided into m × n (m and n are integers), and the multiplication value at each point is stored in the memory as two-dimensional map data. The above coefficients and parameters used for image correction, fixed values to be multiplied, and the like are hereinafter referred to as adjustment parameters.

  When performing image correction based on such individual differences, it is necessary to detect individual differences of the image sensor 11 with high accuracy. In the omnidirectional imaging apparatus 1 of the present embodiment, the imaging system A and the imaging system B are configured as a pair, and both systems A and B can be relatively rotated so that the combination of the paired cameras, that is, the imaging elements can be changed. Thus, even individual differences over time between the image pickup devices 11 can be detected.

  FIG. 4 is a flowchart illustrating a driving method when image characteristic correction (individual difference) is detected and image correction is performed. When the system is started to operate the omnidirectional imaging apparatus 1, first, correction of individual differences of the initial camera pair, in this case (A1, B1) (A2, B2) (A3, B3) (A4, B4), is performed. (Step S1), and proceeds to the next imaging operation (step S2).

  In step S1, for example, the cameras A1 and B1 are paired to capture the same subject with the same brightness and size, and therefore, by comparing the captured images, the image sensor mounted on the camera A1 and the camera B1 are compared. Individual differences between mounted image sensors can be obtained, and correction for absorbing individual differences is possible. The same applies to other camera pairs.

  There are individual differences among the cameras A1, A2, A3 and A4, and individual differences between B1, B2, B3 and B4. However, since an image sensor with a small characteristic difference is selected and used in the initial stage of operation, the characteristics are not so great. There is no difference. Even if there is a characteristic difference, by performing the following steps S2 to S10, correction for absorbing the characteristic difference becomes possible.

  In the next step S3, whether the elapsed time from operating the omnidirectional imaging apparatus 1 has elapsed for a predetermined time based on the time measurement result of the time measuring unit 9j, or a change over time is detected in the characteristic difference between the imaging systems. If the predetermined time has not elapsed or if there is no change over time, the process returns to step S2 to continue the imaging operation.

  As a result of the determination in step S3, when a predetermined time has elapsed from the start of the operation or when a change in the characteristic difference equal to or greater than the threshold value has occurred over time, the process proceeds to the next step S4, and the imaging system A is moved 90 Rotate it and change the camera pair. Now, since the camera pair (upper and lower pairs shown in FIG. 1A) in the initial state in step S1 is (A1, B1) (A2, B2) (A3, B3) (A4, B4), In step S4, the camera pair is changed to (A1, B2) (A2, B3) (A3, B4) (A4, B1).

  Then, when the imaging operation is performed in the next step S5, each camera pair images the same subject. That is, the paired camera A1 and camera B2 capture images of the same subject with the same brightness and size. The same applies to the other camera pairs. Thereby, the individual difference between the image sensor of the camera A1 and the image sensor of the camera B2 can be detected by comparing both captured images, and similarly, the individual difference between the image sensors of the camera A2 and the camera B3 is detected. You can do it. The same applies to the other pairs.

  In the next step S6, the imaging system A is further rotated 90 degrees with respect to the imaging system B. As a result, the camera pair becomes (A1, B3) (A2, B4) (A3, B1) (A4, B2). By performing the imaging operation in step S7, each imaging of the camera A1 and the camera B3 is performed. It becomes possible to detect individual differences between elements, individual differences between the image sensors of the cameras A2 and B4, and so on.

  When the imaging system A is further rotated 90 degrees with respect to the imaging system B in the next step S8, the camera pair becomes (A1, B4) (A2, B1) (A3, B2) (A4, B3), and step S9. When the imaging operation is performed, it is possible to detect individual differences between the image sensors of the cameras A1 and B4, individual differences between the image sensors of the cameras A2 and B1, and so on.

  In the next step S10, the captured images between the pairs obtained in steps S1, S5, S7, and S9 are compared, correction for absorbing individual differences among all image sensors is performed, and the process returns to step S2. This step S10 is processed by the CPU 9a using the subordinate digital signal processing unit 9k, and the adjustment parameter obtained for this correction is stored in the memory 9f or is updated sequentially every time step S10 is performed. Used for etc.

  For example, in the case of only the imaging system A, it is not easy to obtain an individual difference between the imaging element 11 of the camera A1 and the imaging element 11 of the camera A3 at a position 180 degrees relative to this. The same applies to the conventional omnidirectional imaging devices described in Patent Documents 1 and 2. However, in the case of this embodiment, if the same subject is imaged by pairing camera A1 and camera B1, and then the same subject is imaged by pairing camera A3 and camera B1, the individual difference between camera A1 and camera A3 Can be detected.

  As described above, in this embodiment, since all the combinations of the camera constituting the imaging system A and the camera constituting the imaging system B can be realized, it is possible to perform correction to absorb individual differences of the imaging elements used in each camera. It becomes.

  The correction that absorbs the individual differences between the image sensors may be, for example, correction that matches the characteristics of the other cameras A2 to A4 and B1 to B4 with the characteristics of the image sensor mounted on the camera A1. The characteristic average value of each image sensor of the eight cameras may be used as a reference, and correction may be performed so as to match this reference.

  The omnidirectional imaging apparatus 1 of the embodiment shown in FIG. 1 has a stereo camera configuration in which two cameras constituting a pair are arranged up and down, so that the distance to the subject can be accurately detected based on each camera parallax. It is. However, because the pair is in the vertical direction, unlike a normal “stereo camera”, the operator cannot see the stereoscopic image of the subject even when viewing the captured image of the pair.

  FIG. 5 shows an embodiment in which a stereoscopic image can be viewed. In the omnidirectional imaging apparatus of the embodiment shown in FIG. 5A, the pair cameras are (A1, B1) (A2, B2) (A3, B3) (A4, B4), and the two cameras constituting the pair are used. It arrange | positions on either side so that an incident optical axis may become parallel.

  For example, the pair cameras (A1, B1) face the east, the pair cameras (A2, B2) face the south, the pair cameras (A3, B3) face the west, and the pair cameras (A4, B4) face the north. The panorama image of 360 degrees around can be synthesized by only the four cameras A1 to A4 constituting the imaging system A, and the circumference 360 by only the four cameras B1 to B4 constituting the imaging system B. Each camera angle of view is set so that a panoramic image can be synthesized.

  Four cameras B1 to B4 arranged in a ring shape and in a radial shape constituting the imaging system B are fixed to an installation base (ground side), and four cameras arranged in a ring shape and a radial shape constituting the imaging system A A <b> 1 to A <b> 4 are connected by a vertical movement / rotation mechanism 25. The imaging system A is moved up as indicated by an arrow L in FIG. 5B, then rotated 90 degrees as indicated by an arrow M, and moved down as indicated by an arrow N in FIG. 5C. When moved, the paired camera is changed. Thus, in the same manner as in the embodiment shown in FIG. 1, the relative individual differences between the camera-equipped imaging elements can be easily detected during the imaging operation without disassembling the omnidirectional imaging device, and the individual differences can be detected. Absorbing image correction can be performed, and quality degradation of the captured image of the omnidirectional imaging apparatus can be avoided.

  As described above, the omnidirectional imaging apparatus according to the embodiment includes a plurality of imaging units that are arranged so that the end portions of the shooting angle of view of adjacent imaging units overlap with each other and capture a subject image of 360 degrees around. A first imaging system; a second imaging system comprising a plurality of imaging units provided adjacent to each imaging unit of the first imaging system; and the second imaging system relative to the first imaging system And driving means for changing a pair of the imaging unit of the first imaging system and the imaging unit of the second imaging system adjacent to each other.

  Further, the driving unit of the omnidirectional imaging apparatus according to the embodiment moves the position of a certain imaging unit of the first imaging system in order relatively, and sequentially with all the imaging units of the second imaging system. It is characterized by pairing.

  In addition, the omnidirectional imaging apparatus according to the embodiment includes a control unit that changes a combination of the pairs to obtain a relative characteristic difference between the imaging units and performs image correction that absorbs the characteristic difference.

  In addition, the omnidirectional imaging apparatus according to the embodiment includes a storage unit that stores adjustment parameters obtained by the control unit in order to perform the image correction.

  In addition, the omnidirectional imaging apparatus according to the embodiment is characterized in that the process of obtaining the characteristic difference is performed every predetermined time.

  In addition, the omnidirectional imaging apparatus according to the embodiment performs the image correction when it is detected that a characteristic difference equal to or greater than a threshold value has occurred between the imaging units constituting the pair.

  In the omnidirectional imaging apparatus of the embodiment, the incident optical axis of the imaging unit of the first imaging system and the incident optical axis of the imaging unit of the second imaging system that constitute the pair are provided in parallel. It is characterized by.

  The omnidirectional imaging apparatus according to the embodiment is characterized in that a stereoscopic image is captured by the imaging unit of the first imaging system and the imaging unit of the second imaging system that constitute the pair.

  In addition, the omnidirectional imaging device of the embodiment obtains a distance from the captured image by the imaging unit of the first imaging system and the captured image by the imaging unit of the second imaging system constituting the pair to the subject. It is characterized by.

  According to the embodiment described above, even if a characteristic difference occurs with time between a plurality of cameras (imaging units), it is possible to easily detect a relative characteristic difference between the cameras, and thus absorb the characteristic difference. Image correction is performed, and an omnidirectional captured image with always good image quality can be obtained.

  The omnidirectional imaging device of the present invention can always detect a characteristic difference between image pickup elements mounted on the camera used and easily correct the image and correct the image. It is useful when applied to fixed-point cameras, surveillance cameras, and the like.

1, 23 Omnidirectional imaging device 2 Cylindrical casing 3 on movable side Cylindrical casing 4 on fixed side Rotating shaft 9 Controller 9a CPU
9b Rotation controller 11 Image sensor 13 Shooting lens 25 Vertical movement / rotation mechanism

Claims (9)

  A first imaging system configured by a plurality of imaging units that are arranged so that end portions of shooting field angles of adjacent imaging units overlap with each other and image a subject image of 360 degrees around, and each imaging unit of the first imaging system A second imaging system composed of a plurality of imaging units provided adjacent to each other, and the imaging of the adjacent first imaging system by moving the second imaging system relative to the first imaging system And an omnidirectional imaging device comprising: a driving unit that changes a pair of the imaging unit of the second imaging system.   2. The omnidirectional imaging apparatus according to claim 1, wherein the driving unit relatively sequentially moves a position of a certain imaging unit of the first imaging system to perform all of the imaging of the second imaging system. An omnidirectional imaging device that allows the pair to be assembled in order with the unit.   3. The omnidirectional imaging apparatus according to claim 1, further comprising: a control unit that performs image correction to obtain a relative characteristic difference of the imaging unit by changing the combination of the pairs and absorb the characteristic difference. An omnidirectional imaging apparatus.   4. The omnidirectional imaging apparatus according to claim 3, further comprising a storage unit that stores adjustment parameters obtained by the control unit for performing the image correction.   The omnidirectional imaging apparatus according to claim 3 or 4, wherein the processing for obtaining the characteristic difference is performed every predetermined time.   5. The omnidirectional imaging apparatus according to claim 3 or 4, wherein the image correction is performed when it is detected that a characteristic difference equal to or greater than a threshold value has occurred between the imaging units constituting the pair. .   7. The omnidirectional imaging apparatus according to claim 1, wherein an incident optical axis of the imaging unit of the first imaging system and the imaging unit of the second imaging system constituting the pair are configured. An omnidirectional imaging device provided with an incident optical axis in parallel.   The omnidirectional imaging apparatus according to claim 7, wherein the stereoscopic imaging is performed by the imaging unit of the first imaging system and the imaging unit of the second imaging system configuring the pair.   The omnidirectional imaging apparatus according to claim 7, wherein a distance from the captured image by the imaging unit of the first imaging system and the captured image by the imaging unit of the second imaging system constituting the pair to a subject An omnidirectional imaging device for obtaining

Images