JP2012227846A - Multocular imaging apparatus, multocular imaging method, and personal digital assistant - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reliably obtain an image with excellent image quality by determining one imaging part as a main imaging part when the one imaging part has image quality which is more excellent than that of the other imaging part.SOLUTION: A multocular imaging apparatus 20 has a function of selecting a main imaging part to be used in a monocular camera operation mode (in 2D imaging) between multiple imaging parts, e.g. a first imaging part 2 and a second imaging part 3. The apparatus includes: the first imaging part 2 and the second imaging part 3; an image quality information storage part 10a for storing respective kinds of image data being image quality information held by the respective imaging parts 2, 3; a determination information calculation part 10b for calculating an SN ratio being image quality determination information with respect to the respective imaging parts 2, 3 on the basis of the respective kinds of image data being the image quality information; a determination information storage part 10c for storing the SN ratio being the image quality determination information calculated by the determination information calculation part 10b; and an imaging part changeover control part 10d for determining the main imaging part to be used in monocular imaging (in 2D imaging) on the basis of the SN ratio being the image quality determination information.

Description

  The present invention relates to a multi-view imaging device and a multi-view imaging method capable of generating a moving image and a still image / image including stereoscopic vision, and a digital camera or a camera-equipped mobile phone using the multi-view imaging device as an imaging unit. The present invention relates to a portable information terminal device such as a device.

  This type of conventional multi-lens imaging device is used in a digital camera or the like that can synthesize an image signal acquired from two or more optical systems and capture a 3D image. It is used for a portable information terminal device with a photographing function, such as a mobile phone device with a camera used in the unit. In this case, the “digital camera” includes a digital still camera that captures still images and a digital video camera that captures still images and moving images.

  As a multi-lens image pickup device having a plurality of image pickup units, various multi-eye image pickup devices capable of, for example, 3D (three-dimensional) shooting and panoramic shooting have been proposed. In such a conventional multi-lens image pickup device, the same plurality of image pickup units are arranged side by side, and a plurality of images acquired at the same time in each image pickup unit are subjected to 3D composition processing to enable stereoscopic viewing. Images and panoramic images are generated.

  Patent Document 1 discloses a conventional integrated compound-eye image processing apparatus including two imaging optical systems for 3D (three-dimensional) imaging. As described above, in the conventional compound eye image processing apparatus capable of 3D (3D) imaging, when performing normal 2D (2D) imaging, only a predetermined main imaging unit among the two imaging units is used. It is used by operating.

  In a conventional portable information terminal device using a conventional multi-view imaging device capable of 3D (three-dimensional) imaging as an imaging unit, only a predetermined main imaging unit is operated during normal 2D (two-dimensional) imaging. The case will be described with reference to FIG.

  FIG. 10 is a block diagram illustrating a configuration example of a main part of a conventional portable information terminal device using a conventional multi-view imaging device as an imaging unit.

  In FIG. 10, a conventional portable information terminal device 100 is provided with an optical lens, an imaging device, a CDS / AGC / ADC, and the like, and a main imaging unit 101 that performs 2D (2D) or 3D (3D) imaging, and an optical A lens, an imaging device, a CDS / AGC / ADC, and the like are provided, and a sub imaging unit 102 that performs 3D (three-dimensional) imaging, and in the case of 3D, left and right image signals from the main imaging unit 101 and the sub imaging unit 102 Is input and 3D synthesis is performed for each frame, or in the case of 2D, the memory unit 103 temporarily stores image data for each frame, and signal processing for performing predetermined signal processing on the image signal from the memory unit 103 Unit 104, display device 105 that displays on the display screen based on the image signal from signal processing unit 104, and automatic focus driving of the optical lens with respect to sub-imaging unit 102 At the same time, the lens driving / image sensor driving unit 106 for driving the image sensor to read out the image pickup signal, and the lens driving for reading out the image pickup signal by driving the image pickup device while driving the optical lens to the main image pickup unit 101 automatically. / The image sensor driving unit 107, and the system control unit 108 for controlling the driving of the lens driving / image sensor driving units 106 and 107 using the image signal from the signal processing unit 104. Reference numeral 109 denotes a lateral distance device, and reference numeral 110 denotes a convergence angle control mechanism.

JP-A-10-32842

  In the conventional configuration, two imaging units 101 and 102 are prepared for 3D (three-dimensional) imaging. When this is used for 2D (two-dimensional) imaging, one of the two imaging units 101 and 102 is defined as, for example, the main imaging unit 101 and the main imaging unit 101 is used. If the main imaging unit 101 is determined in advance, the main imaging unit 101 is determined in advance even if the sub imaging unit 102 has better image quality than the main imaging unit 101 due to manufacturing variations or the like. There was a problem that 2D photography can only be performed.

  The present invention solves the above-described conventional problems. When the image quality of the other imaging unit is better than that of the one imaging unit, the other imaging unit is determined as the main imaging unit. To provide a multi-view imaging device and a multi-view imaging method capable of obtaining a good image more reliably, and a portable information terminal device such as a digital camera or a camera-equipped mobile phone using the multi-view imaging device as an imaging unit. With the goal.

  The multi-lens image pickup device of the present invention is a multi-eye image pickup device provided with at least two image pickup units for 3D shooting, and in the case of 2D shooting, any one of the at least two image pickup units is defined as a main image pickup unit. Image quality determination information is obtained in at least one of the inside and the outside based on the image data from the at least two image pickup units, and the image pickup unit with the best image quality is selected as the main image pickup unit based on the image quality determination information It has a control part, and the said objective is achieved by it.

  Preferably, the control unit in the multi-lens imaging device of the present invention is based on an image quality information storage unit that stores each image data from the at least two imaging units, and each image data from the image quality information storage unit. A determination information calculation unit that calculates image quality determination information, a determination information storage unit that stores image quality determination information calculated by the determination information calculation unit, and a determination information from the determination information storage unit. An imaging unit switching control unit that determines the imaging unit as the main imaging unit.

  Further preferably, the determination information calculation unit in the multi-lens imaging device of the present invention calculates an SN ratio as the image quality determination information from the output value of each image data from the image quality information storage unit.

  Further preferably, the determination information calculation unit in the multi-view imaging apparatus of the present invention calculates a shading characteristic as the image quality determination information from the output value of each image data from the image quality information storage unit.

  Further preferably, the determination information calculation unit in the multi-lens imaging device of the present invention calculates a color shading characteristic as the image quality determination information from the output value of each image data from the image quality information storage unit.

  Further preferably, the image quality information storage unit in the multi-lens imaging device of the present invention is configured such that each of the at least two imaging units captures each image data from the at least two imaging units in which a subject having a uniform luminance is captured. Remember.

  Further preferably, the image quality information storage unit in the multi-lens imaging device of the present invention is configured such that each of the at least two imaging units captures each image data from the at least two imaging units in which a subject having a uniform luminance is captured. The ratio of the color signals of at least two colors is stored.

  Further preferably, the imaging unit switching control unit in the multi-lens imaging device of the present invention selects which one of the two imaging units is used during 2D imaging based on a selection command issued by the user from the outside. It further has a function to control.

  Furthermore, preferably, each image data from the two imaging units in the multi-lens imaging device of the present invention is any one of a total effective pixel region, a single partial effective pixel region, and a plurality of partial effective pixel regions. Each image data from the region.

  A portable information terminal device according to the present invention uses the multi-eye imaging device according to the present invention as an imaging unit, and thereby achieves the above object.

  Preferably, in the portable information terminal device of the present invention, an image signal from at least one of the at least two imaging units is displayed on a display screen based on image data after performing predetermined signal processing for display. A display device is provided.

  Furthermore, it is preferable that the portable information terminal device of the present invention further includes a memory unit capable of recording image data after performing predetermined signal processing for recording an image signal from at least one of the at least two imaging units. .

  Further preferably, in the portable information terminal device of the present invention, a communication unit capable of performing communication processing on image data after performing predetermined signal processing for communication of an image signal from at least one of the at least two imaging units. Have

  The multi-eye imaging method of the present invention is a multi-eye imaging method that uses one of the at least two imaging units as a main imaging unit when performing 2D imaging using at least two imaging units for 3D imaging. A control unit that obtains image quality determination information based on each image data from the at least two imaging units, and selects an imaging unit having the best image quality as the main imaging unit based on the image quality determination information. This achieves the above object.

  Preferably, in the multi-view imaging method of the present invention, the control step includes: an image quality information storage step for storing each image data from the at least two imaging units in an image quality information storage unit; A determination information calculation step for calculating image quality determination information based on each image data from the information storage unit; a determination information storage step for storing the image quality determination information calculated in the determination information calculation step in the determination information storage unit; The unit switching control unit includes an imaging unit switching control step of determining an imaging unit to be used at the time of 2D imaging as the main imaging unit based on the determination information from the determination information storage unit.

  Further preferably, the determination information calculation step in the multi-view imaging method of the present invention calculates an SN ratio as the image quality determination information from an output value of each image data from the image quality information storage unit.

  Further preferably, the determination information calculation step in the multi-view imaging method of the present invention calculates a shading characteristic as the image quality determination information from the output value of each image data from the image quality information storage unit.

  Further preferably, the determination information calculation step in the multi-view imaging method of the present invention calculates a color shading characteristic as the image quality determination information from the output value of each image data from the image quality information storage unit.

  Further preferably, the image quality information storing step in the multi-lens imaging method of the present invention is characterized in that each of the image data from the at least two imaging units in which a subject having a uniform luminance is captured on each of the at least two imaging units. Remember.

  Further preferably, the image quality information storing step in the multi-lens imaging method of the present invention includes a step in which each of the at least two imaging units is supplied with the brightness from the at least two imaging units on which an achromatic subject is photographed. A ratio of color signals of at least two colors of each image data is stored.

  With the above configuration, the operation of the present invention will be described below.

  In the present invention, at least two imaging units are provided for 3D imaging, and in the case of 2D imaging, at least two imaging units are used in a multi-eye imaging apparatus that uses at least one of the two imaging units as a main imaging unit. A control unit that obtains image quality determination information based on each image signal from the unit and selects an imaging unit with the best image quality based on the image quality determination information as a main imaging unit;

  As a result, when the image quality of the other imaging unit is better than that of the one imaging unit, it is possible to more reliably obtain an image with better image quality by determining the other imaging unit as the main imaging unit. Become.

  As described above, according to the present invention, when the image quality of the other image pickup unit is better than that of the one image pickup unit, the other image pickup unit is determined as the main image pickup unit, so that an image with higher image quality can be more reliably obtained. Can get to.

It is a block diagram which shows the principal part structural example of the portable information terminal device which used the multiview imaging device in Embodiment 1 of this invention for the imaging part. 3 is a flowchart for explaining an imaging unit switching control operation in the portable information terminal device of FIG. 1. It is a block diagram which shows the principal part structural example of the portable information terminal device which used the multiview imaging device in Embodiment 2 of this invention for the imaging part. 4 is a flowchart for explaining an imaging unit switching control operation in the portable information terminal device of FIG. 3. It is a top view of the imaging region which shows an example of an image data position. It is a block diagram which shows the principal part structural example of the portable information terminal device which used the multiview imaging device in Embodiment 3 of this invention for the imaging part. It is a flowchart for demonstrating the imaging part switching control operation | movement in the portable information terminal device of FIG. It is a top view of the imaging region which shows another example of an image data position. It is a block diagram which shows the principal part structural example of the portable information terminal device which used the multiview imaging device in Embodiment 4 of this invention for the imaging part. It is a block diagram which shows the principal part structural example of the conventional portable information terminal device which used the conventional multi-view imaging device for the imaging part.

  Embodiments 1 to 4 of a portable information terminal device such as a digital camera or a camera-equipped mobile phone device using the multi-eye imaging device of the present invention as an imaging unit will be described below in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram illustrating a configuration example of a main part of a portable information terminal device using the multiview imaging device according to the first embodiment of the present invention as an imaging unit.

  In FIG. 1, the portable information terminal device 1 using the multi-view imaging device 20 of the present invention as an imaging unit is provided with a first imaging unit 2 and a second imaging unit 3 as at least two imaging units for 3D imaging, In the case of 2D shooting, the better image quality of at least two imaging units is determined and used as the main imaging unit.

  The portable information terminal device 1 is provided with an optical lens, an infrared cut filter, a microlens, an image sensor, a CDS / AGC / ADC, and the like, and performs at least 3D imaging of 2D (2D) and 3D (3D). A first imaging unit 2, an optical lens, an infrared cut filter, a microlens, an imaging device, a CDS / AGC / ADC, and the like are provided, and at least 3D imaging of 2D (2D) and 3D (3D) is performed. The left and right image signals from the two image pickup units 3 and the first image pickup unit 2 and the second image pickup unit 3 are input and 3D combined for each frame, or the first image pickup unit 2 and the second image pickup unit 3 A memory unit 4 to which a 2D image signal from any one of them is input and temporarily stored, a signal processing unit 5 that performs predetermined signal processing on each image signal from the memory unit 4, and a signal The display device 6 that displays on the display screen based on the image signal from the processing unit 5 and the second imaging unit 3 are automatically focused on the optical lens by the contrast AF method, and the imaging element is driven. Lens driving / imaging device driving unit 7 for reading an imaging signal and lens driving / imaging for reading the imaging signal by driving the imaging device while automatically driving the optical lens to the first imaging unit 2 using a contrast AF method or the like. An element driving unit 8, a system control unit 9 that controls driving of the lens driving / imaging element driving units 7 and 8 using an image signal from the signal processing unit 5, and the first imaging unit 2 and the second imaging unit 3. Communication is performed after predetermined signal processing is performed on the image signal from the signal processing unit 5 and the control unit 10 that selects an imaging unit with a better image quality by obtaining an SN ratio as image quality determination information from each image signal. A communication unit 11 such as a transmission / reception device capable of processing, a memory unit 12 (storage unit) such as a recording medium capable of recording data after image signals from the signal processing unit 5 have been subjected to predetermined signal processing for recording, and a system An operation unit 13 is provided for instructing the control unit 7 to input each mode such as 2D shooting and 3D shooting.

  The first imaging unit 2, the second imaging unit 3, the memory unit 4, the signal processing unit 5, the lens driving / imaging element driving units 7 and 8, the system control unit 9, the control unit 10, and the operation unit 13 perform multi-eye imaging. A device 20 is configured. The display device 6 and the memory unit 12 are added to the mobile information terminal device 1 such as a digital camera. Further, the communication unit 11 is added to form another portable information terminal device 1 such as a mobile phone device with a camera. Here, a mobile phone device with a camera is shown as the portable information terminal device 1. In short, the portable information terminal device 1 may have at least one of the display device 6, the communication unit 11, and the memory unit 12.

  The control unit 10 includes an image quality information storage unit 10a such as a pixel data storage unit that stores each captured image data (a plurality of pixel data) captured by the first imaging unit 2 and the second imaging unit 3, and an image quality information storage unit. A determination information calculation unit 10b for calculating an SN ratio as image quality determination information from each captured image data stored in 10a, and a determination information storage unit for storing the SN ratio as image quality determination information calculated by the determination information calculation unit 10b Imaging that determines which imaging unit of the first imaging unit 2 and the second imaging unit 3 is the main imaging unit based on 10c and the SN ratio as the image quality determination information stored in the determination information storage unit 10c One having a larger S / N ratio, obtaining an S / N ratio as image quality determination information from each captured image data (a plurality of pixel data) of the first imaging unit 2 and the second imaging unit 3. Tsuma The image quality is good imaging unit for 2D imaging is controlled so as to select as a main image pickup unit.

  The operation unit 13 may be provided with input buttons for operating each mode on the surface of the casing, or the display screen of the display device 6 may be configured with a touch panel and each mode input area may be provided. Each mode can be input using the input button and input area of each mode.

  The image quality information storage device 10a calculates RGB of each captured image data (a plurality of pixel data) in all effective pixels in the first imaging unit 2 and the second imaging unit 3 for calculating the SN ratio as the image quality determination information. Data is stored in advance in the manufacturing process.

  The determination information calculation unit 10b calculates the SN ratio by taking the logarithm of the average value of the image data divided by the standard deviation from each captured image data (a plurality of pixel data) stored in the image quality information storage device 10a. The SN ratios of the imaging units 2 and 3 are stored in the determination information storage unit 10c as image quality determination information calculated by the determination information calculation unit 10b.

  The imaging unit switching control unit 10d compares the SN ratios that are image quality determination information of the imaging units 2 and 3 calculated by the determination information calculation unit 10b, and selects an imaging unit that has a higher SN ratio that is a determination information value. Control is performed so that the main image pickup unit (the first image pickup unit 2 or the second image pickup unit 3) with good image quality is determined and the determined main image pickup unit is used in the monocular mode (2D shooting).

  The operation of the above configuration will be described below.

  FIG. 2 is a flowchart for explaining an imaging unit switching control operation in the portable information terminal device of FIG.

  As shown in FIG. 2, first, in step S1, an object having uniform brightness (or luminance) and color, for example, a completely white object in a manufacturing process or a shipping inspection process, is captured by a first imaging unit 2 and a second imaging unit. Take a picture at 3 at the same time. Thereafter, the image data for all effective pixels (one frame) is recorded in the image quality information recording unit 10a from the respective captured image data (a plurality of pixel data) captured by the first imaging unit 2 and the second imaging unit 3.

Next, the determination information calculation unit 10b calculates the SN ratio from each captured image data recorded in the image quality information recording unit 10a in step S2. Subsequently, in step S3, the SN ratio calculated in step S2 is used as the image quality determination information, and the SN ratio that is the image quality determination information is recorded in the determination information storage device 10c. When σ _S is an average value of image data and σ _N is a standard deviation, the SN ratio can be calculated by the following equation. The larger the S / N ratio, the better the image quality without noise.

さらに、ステップＳ４において、ステップＳ３で記憶した画質判定情報であるＳＮ比に基づいて、第１撮像部２および第２撮像部３のうちのどの撮像部を主撮像部にするかを決定する。つまり、切り替え制御部１０ｄが、第１撮像部２のＳＮ比と第２撮像部３のＳＮ比とを比較し、単眼撮影モード時（２Ｄ撮影時）に判定情報であるＳＮ比が一番大きい値を持つ第１撮像部２および第２撮像部３のいずれかの撮像部を主撮像部として使用することにより撮像部切替処理を行う。

Further, in step S4, based on the SN ratio that is the image quality determination information stored in step S3, it is determined which of the first imaging unit 2 and the second imaging unit 3 is to be the main imaging unit. That is, the switching control unit 10d compares the SN ratio of the first imaging unit 2 with the SN ratio of the second imaging unit 3, and the SN ratio that is the determination information is the largest in the monocular imaging mode (2D imaging). The imaging unit switching process is performed by using any one of the first imaging unit 2 and the second imaging unit 3 having a value as the main imaging unit.

  As described above, according to the first embodiment, at least two imaging units are provided for 3D imaging, and in the case of 2D imaging, at least one of the two imaging units 2 and 3 is determined as a main imaging unit and used. In the eye imaging device 20, a control unit 10 that obtains image quality determination information based on each image signal from at least two imaging units 2 and 3 and selects an imaging unit with the best image quality as the main imaging unit based on the image quality determination information. have.

  That is, the multi-lens imaging device 20 has a function of selecting a main imaging unit to be used in a monocular camera operation mode (during 2D shooting) from a plurality of imaging units, for example, the first imaging unit 2 and the second imaging unit 3. The first imaging unit 2 and the second imaging unit 3, the image quality information storage unit 10a that stores the image data that is the image quality information that each of the imaging units 2 and 3 has, and the respective imaging units from the image data that is the image quality information 2 and 3, a determination information calculation unit 10b that calculates an SN ratio that is image quality determination information, a determination information storage unit 10c that stores an SN ratio that is image quality determination information calculated from the determination information calculation unit 10b, and an image quality determination An imaging unit switching control unit 10d that determines a main imaging unit to be used during monocular imaging (during 2D imaging) based on the SN ratio that is information.

  Thereby, each captured image data is obtained by each imaging unit 2, 3, and an SN ratio which is image quality determination information for each imaging unit 2, 3 is calculated by the determination information calculation unit 10 b from each image data. Next, the SN ratio, which is image quality determination information, is stored in the determination information storage unit 10c. Thereafter, based on the S / N ratio that is the stored image quality determination information, the imaging unit switching control unit 10d performs appropriate imaging unit switching in the monocular camera operation mode (2D imaging).

  As described above, even when there is a difference in image quality between the imaging units 2 and 3 due to the manufacturing process, each image data as image quality information is stored for each imaging unit 2 and 3, and from this image quality information, Since the S / N ratio that is the determination information is calculated and the imaging unit is switched based on the S / N ratio that is the determination information, the one that can compare the image quality of each of the imaging units 2 and 3 is used as the main imaging unit and monocular imaging At the time (2D shooting), it is possible to obtain high-quality image data with less noise. Therefore, it is possible to prevent deterioration in image quality such as noise due to variations in the imaging units 2 and 3.

  In the first embodiment, RGB data in all effective pixel regions of the first imaging unit 2 and the second imaging unit 3 are stored in advance in the image quality information storage unit 10a in the manufacturing process, and the determination information calculation unit 10b The S / N ratio is calculated as the image quality determination information from each image data of all effective pixels stored in the information storage unit 10a. However, the first imaging unit 2 and the second imaging are not limited to each image data in all effective pixels. Of all the effective pixel regions of the unit 3, the same central region may be used in the first imaging unit 2 and the second imaging unit 3. In this case, RGB data of each image data in the same central region of the first imaging unit 2 and the second imaging unit 3 is stored in advance in the image quality information storage unit 10a in the manufacturing process, and the determination information calculation unit 10b stores the image quality information storage. The S / N ratio may be calculated as image quality determination information from each image data in the same central area stored in the unit 10a. This same central area is at the same position in the first imaging unit 2 and the second imaging unit 3, but the SN ratio is calculated as image quality determination information from each image data that is an extreme story and has few adjacent 4 pixels or 9 adjacent pixels. You may make it do. In short, in this case, each image data from the two imaging units 2 and 3 is each image data from the entire effective pixel region, a single partial effective pixel region, and a plurality of partial effective pixel regions.

  In the first embodiment, the S / N ratio is obtained as the image quality determination information from the image data of the first imaging unit 2 and the second imaging unit 3, and the imaging unit with better image quality is selected as the main imaging unit. Although the control unit 10 is provided to perform control so that the selected image pickup unit having a better image quality at the time of shooting is used, the present invention is not limited to this. The image capturing unit 3 captures images simultaneously, and from each image data from the first image capturing unit 2 and the second image capturing unit 3, a resolution inspection is performed to determine how fine the image appears, and the image capturing unit with better image quality is selected as the main image capturing unit. In addition, a control unit may be provided that performs control so that shooting is performed using the imaging unit with the better selected image quality during 2D shooting.

(Embodiment 2)
In the first embodiment, the case has been described in which the image pickup unit with the better image quality is selected as the main image pickup unit using the SN ratio as the image quality determination information. However, in the second embodiment, as the image quality determination information, the screen center and the screen are selected. A case will be described in which a shading characteristic indicating a difference in brightness (or luminance) from the peripheral portion is used to select an imaging unit having a smaller difference, that is, a better image quality as the main imaging unit.

  FIG. 3 is a block diagram illustrating a configuration example of a main part of a portable information terminal device using the multiview imaging device according to the second embodiment of the present invention as an imaging unit. In FIG. 3, the same member numbers are used for the description of the structural members that have the same effects as the structural members of FIG.

  In FIG. 3, the portable information terminal device 1 </ b> A using the multi-lens imaging device 20 </ b> A of the present invention as an imaging unit is different from the case of the first embodiment in that the first embodiment replaces the control unit 10 of the first embodiment. From the image signals at the screen center and the screen periphery of the image capturing unit 2 and the second image capturing unit 3, a shading characteristic indicating a difference in brightness (or luminance) between the screen center and the screen periphery is obtained as image quality determination information. In other words, a control unit 10A for selecting an image pickup unit with better image quality is provided.

  The first imaging unit 2, the second imaging unit 3, the memory unit 4, the signal processing unit 5, the lens driving / imaging element driving units 7 and 8, the system control unit 9, the control unit 10 </ b> A, and the operation unit 13 perform multi-eye imaging. The apparatus 20A is configured. The display device 6 and the memory unit 12 are added to the mobile information terminal device 1A such as a digital camera. Further, the communication unit 11 is added to form another portable information terminal device 1A such as a camera-equipped mobile phone device. Here, a mobile phone device with a camera is shown as the portable information terminal device 1A. In short, the portable information terminal device 1 </ b> A only needs to include at least one of the display device 6, the communication unit 11, and the memory unit 12.

  The control unit 10A includes a pixel data storage unit that stores each captured image data composed of a plurality of pixels at the center of the screen and the periphery of the screen from each captured image data captured by the first imaging unit 2 and the second imaging unit 3, respectively. Difference in brightness (or brightness) between the center of the screen and the peripheral portion of the screen as image quality determination information from the image quality information storage portion 10a1 and the captured image data of the screen center and the peripheral portion of the screen stored in the image quality information storage portion 10a. A determination information calculation unit 10b1 for calculating a shading characteristic indicating a brightness, a determination information storage unit 10c1 for storing a shading characteristic of brightness (or luminance) as image quality determination information calculated by the determination information calculation unit 10b1, and determination information Which imaging unit of the first imaging unit 2 and the second imaging unit 3 is based on the shading characteristics as the image quality determination information stored in the storage unit 10c1. An imaging unit switching control unit 10d1 that determines whether to use the main imaging unit, and from each of the captured image data (a plurality of pixel data) of the screen center and the screen periphery of the first imaging unit 2 and the second imaging unit 3. As the image quality determination information, the shading characteristic of brightness (or luminance) is obtained, and the one with the smaller shading characteristic of brightness (or luminance), that is, the imaging unit with better image quality is selected as the main imaging unit for 2D shooting. Is controlling.

  The image quality information storage device 10a1 calculates the shading characteristics of brightness (or luminance) as image quality determination information, and the center of the screen among all the effective pixels in the first imaging unit 2 and the second imaging unit 3. The average value of the color signal of each photographed image data composed of a plurality of pixels in the periphery of the screen is stored in advance in the manufacturing process.

  The determination information calculation unit 10b1 takes a ratio (screen center / screen periphery) from the average values of the screen center and the screen periphery stored in the image quality information storage device 10a1, and shows a shading characteristic indicating a difference in brightness (or luminance). And the shading characteristics of the imaging units 2 and 3 are stored in the determination information storage unit 10c1 as image quality determination information as the image quality determination information calculated by the determination information calculation unit 10b1.

  The imaging unit switching control unit 10d1 compares the shading characteristics of the imaging units 2 and 3 calculated by the determination information calculation unit 10b1, and the shading characteristics that are determination information values are smaller, that is, the ratio of the average values is smaller. The small imaging unit is determined as the main imaging unit (the first imaging unit 2 or the second imaging unit 3) with good image quality, and control is performed so that the determined main imaging unit is used in the monocular mode (2D shooting). Is done.

  In this case, the shading characteristics of the center of the screen and the periphery of the screen are obtained by taking the difference between the average value of the image data at the center of the screen and the average value of the image data at the periphery of the screen. May be determined as the main imaging unit (the first imaging unit 2 or the second imaging unit 3) with good image quality. In addition, each shading characteristic at the screen center and the screen periphery is the difference between the brightness (or brightness value) of the image data at the center of the screen and the brightness (or brightness value) of the image data at the screen periphery corresponding to this. For each color signal, the values having the worst difference are compared, and the imaging unit with the smallest value is determined as the main imaging unit (the first imaging unit 2 or the second imaging unit 3) with good image quality. Also good.

  The operation of the above configuration will be described below.

  FIG. 4 is a flowchart for explaining the imaging unit switching control operation in the portable information terminal device 1A of FIG.

  As shown in FIG. 4, first, in step S11, the first imaging unit 2 and the second imaging are performed on a subject with uniform brightness (or luminance) and color, for example, a completely white subject in the manufacturing process or product shipment inspection process. Part 3 is shot simultaneously. After that, among the captured image data (a plurality of pixel data) captured by the first imaging unit 2 and the second imaging unit 3, the screen center area A and the peripheral corner area B of the entire captured screen as shown in FIG. The output value of the image data of each pixel for each fixed point is obtained, and the output value of the image data for each fixed point is recorded in the image quality information recording unit 10a1.

  Next, from each captured image data recorded in the image quality information recording unit 10a1 in step S12, the determination information calculation unit 10b1 has a shading characteristic indicating a difference in brightness (or luminance) between the screen center region A and the peripheral corner region B. calculate. Subsequently, in step S13, the brightness (or luminance) shading characteristic calculated in step S12 is used as the image quality determination information, and the brightness (or luminance) shading characteristic which is the image quality determination information is recorded in the determination information storage device 10c1. To do.

R _c , Gr _c , B _c , Gb _c are average values of image data from each central pixel in the screen center area A, and R _p , Gr _p , B are average values of image data from peripheral pixels in the peripheral corner area B. _p , Gb _p , Ave_c is the average value of the image data from each central pixel in the screen center area A, and Ave_p is the average value of the image data from the peripheral pixels in the peripheral corner area B. ) Is calculated.
Ave_c = Ave (Gr _c , R _c , B _c , Gb _c )
Ave_p = Ave (Gr _p , R _p , B _p , Gb _p )
S1 = Ave_c / Ave_p
Each shading characteristic is calculated by the following equation when the difference between the average value of the image data at the center of the screen and the average value of the image data at the periphery of the screen is the shading characteristic S1.
S1 = Ave_c−Ave_p
Also, the difference between the brightness (or brightness value) of the image data at the center of the screen and the brightness (or brightness value) of the corresponding image data at the periphery of the screen is taken for each color signal, and the difference is the worst value. Is the shading characteristic S1, the following formula is used.
_{S1 = Max (Gr c -Gr p} , R c -R p, B c -B p, Gb c -Gb p)
However, Ave () represents an average value, and Max () represents a maximum value (worst value).
After that, in step S14, based on the shading characteristic S1 that is the determination information calculated as described above, the imaging unit switching control unit 10d1 compares the shading characteristic S1 that is the determination information of each of the imaging units 2 and 3, In the monocular imaging mode (at the time of 2D imaging), imaging unit switching processing is performed so as to select an imaging unit with good image quality having the smallest value of the shading characteristic S1 as determination information as the main imaging unit.

  As described above, according to the second embodiment, the determination information calculation unit 10b1 calculates the shading characteristic of brightness (luminance) as the image quality determination information from the output value of each pixel data from the image quality information storage unit 10a1, and this calculation is performed. By comparing the respective shading characteristics and determining the imaging unit having the smallest value as the main imaging unit with good image quality, it is possible to more reliably obtain an image with better image quality. Therefore, it is possible to prevent deterioration in image quality such as shading due to variations in the imaging units 2 and 3.

(Embodiment 3)
In the second embodiment, as the image quality determination information, a shading characteristic indicating a difference in brightness (luminance) between the screen center and the screen periphery is used, and the imaging unit with the smaller difference, that is, the better image quality is used as the main imaging unit. In the third embodiment, as the image quality determination information, each color is used by using a color shading characteristic indicating a difference (color variation) in output values for each color between the screen center and the screen peripheral part. A case will be described in which an imaging unit with a smaller output value difference (color variation), that is, a better image quality is selected as the main imaging unit.

  FIG. 6 is a block diagram illustrating a configuration example of a main part of a portable information terminal device using the multiview imaging device according to the third embodiment of the present invention as an imaging unit. In FIG. 6, the same member numbers are used for the description of the structural members that have the same operational effects as the structural members of FIG.

  In FIG. 6, a portable information terminal device 1B using the multi-view imaging device 20B of the present invention as an imaging unit is different from the case of the second embodiment in that the first alternative is the control unit 10A of the second embodiment. A difference in output values (color variation) for each color between the screen center and the screen periphery is shown as image quality determination information from the image signals at the screen center and the screen periphery of the image capturing unit 2 and the second image capturing unit 3. This is provided with a control unit 10B that obtains color shading characteristics and selects an image pickup unit with better image quality.

  These first imaging unit 2, second imaging unit 3, memory unit 4, signal processing unit 5, lens driving / imaging element driving unit 7, 8, system control unit 9, control unit 10 </ b> B and operation unit 13 perform multi-view imaging. A device 20B is configured. The display device 6 and the memory unit 12 are added to the mobile information terminal device 1B such as a digital camera. Further, the communication unit 11 is added to form another portable information terminal device 1B such as a camera-equipped mobile phone device. Here, a mobile phone device with a camera is shown as the portable information terminal device 1B. In short, the portable information terminal device 1 </ b> B only needs to include at least one of the display device 6, the communication unit 11, and the memory unit 12.

  The control unit 10B obtains the captured image data (plural pixel data) at the center of the screen and the peripheral portion of the screen from the captured image data (plural pixel data) captured by the first imaging unit 2 and the second imaging unit 3, respectively. From the image quality information storage unit 10a2 such as the pixel data storage unit to be stored, and the captured image data of the screen center and the screen periphery stored in the image quality information storage unit 10a, as the image quality determination information, each color of the screen center and the screen periphery A determination information calculation unit 10b2 for calculating a color shading characteristic indicating a difference (color variation) between output values for each, and a color shading characteristic of each imaging unit as image quality determination information calculated by the determination information calculation unit 10b2. And the first imaging unit 2 and the second imaging unit 2 based on the color shading characteristics as the image quality determination information stored in the determination information storage unit 10c2. An imaging unit switching control unit 10d2 that determines which imaging unit of the image unit 3 is the main imaging unit, and each of the screen center and the screen peripheral unit of the first imaging unit 2 and the second imaging unit 3 A color shading characteristic is obtained as brightness difference for each color as image quality determination information from captured image data (a plurality of pixel data), and an image pickup unit having a smaller color shading characteristic, that is, a higher image quality is mainly used for 2D shooting. Control is performed so as to select the imaging unit.

  The image quality information storage device 10a2 uses each effective pixel in the first imaging unit 2 and the second imaging unit 3 to calculate the shading characteristics of the output value difference (color variation) for each color as the image quality determination information. The ratio (R / G, B / G) of the RGB data of each captured image data (a plurality of pixel data) in the center of the screen and the periphery of the screen in (one frame) is stored in advance in the manufacturing process.

  The determination information calculation unit 10b2 calculates the average brightness of the image data at the center of the screen and the periphery of the screen from the captured image data (plural pixel data) at the screen center and the screen periphery stored in the image quality information storage device 10a2. As the image quality determination information calculated by the determination information calculation unit 10b2, the difference between the average brightness of the image data of each unit is taken for each color, the difference between the best color and the bad color value is calculated as a color shading characteristic. A color shading characteristic indicating a difference in output values (color variation) for each color of the imaging units 2 and 3 is stored in the determination information storage unit 10c2 as image quality determination information.

  The imaging unit switching control unit 10d2 compares the color shading characteristics of the imaging units 2 and 3 calculated by the determination information calculation unit 10b2, and selects an imaging unit having a smaller color shading characteristic, which is a determination information value, as a main image with good image quality. Control is performed so that the determined main imaging unit is used in the monocular mode (at the time of 2D imaging).

  In this case, considering the possibility of color shading on one side of the screen, each color shading characteristic has the output value (or level) of the image data at the periphery of the screen for each color and the other screen periphery on the diagonal line. The color value with the worst difference is used as the color shading characteristic of each image pickup unit, and the image pickup unit with the smaller color shading characteristic is the main one with good image quality. The imaging unit (the first imaging unit 2 or the second imaging unit 3) may be determined.

  The operation of the above configuration will be described below.

  FIG. 7 is a flowchart for explaining the imaging unit switching control operation in the portable information terminal device 1B of FIG. In FIG. 7, components having the same operational effects as the components of FIG.

  As shown in FIG. 7, first, in step S21, the first imaging unit 2 and the second imaging unit 3 simultaneously photograph an achromatic object with uniform brightness (or luminance) in the manufacturing process or product shipment inspection process. To do. After that, among the respective photographed image data (a plurality of pixel data) photographed by the first imaging unit 2 and the second imaging unit 3, it is point-symmetric with the optical center pixel in the screen center area A of the entire photographing screen as shown in FIG. For the peripheral pixels in the peripheral corner region C at the position, the output value of the image data of each pixel for each fixed point is obtained, and the output value of this image data is recorded in the image quality information recording unit 10a2.

  Next, in step S22, the determination information calculation unit 10b2 outputs the output value (level) for each color at the screen center and the screen peripheral part from the captured image data of the screen center and the screen peripheral part recorded in the image quality information recording unit 10a2. ) To calculate image quality determination information. Subsequently, in step S23, the color shading characteristic calculated in step S22 is used as image quality determination information, and the color shading characteristic that is the image quality determination information is recorded in the determination information storage device 10c2.

The output value of the image data at the center (center pixel) of the screen is R _c , Gr _c , B _c , Gb _c , and the output value of the image data at the screen peripheral part (peripheral pixel) is R _p1 , Gr _p1 , B _p1 , Gb _p1. , R _p2 , Gr _p2 , B _p2 , Gb _p2 are the output values of the peripheral pixels at the point symmetry position, and the ratio of R and B R _c / Gr _c , B _c / Gb _c , R _p1 / Gr _p1 , for each pixel, B _p1 / _{Gb p1,} the _{R p2 / Gr p2, B p2} / Gb p2 is calculated.
The color shading characteristic S2 for each pixel is calculated from the following equation. However, Ave () represents an average value, Max () represents a maximum value (worst value), and Min () represents a minimum value (best value).

AveR _p1 = Ave (R _p1 / Gr _p1 ) −Ave (R _c / Gr _c )
AveR _p2 = Ave (R _p2 / Gr _p2 ) −Ave (R _c / Gr _c )
AveB _p1 = Ave (B _p1 / Gb _p1 ) −Ave (B _c / Gb _c )
_{AveB p2 = Ave (B p2 /} Gb p2) -Ave (B c / Gb c)
_{Dis_P1 = Max (AveR p1, AveB} p1) - Min (AveR p1, AveB p1)
_{Dis_P2 = Max (AveR p2, AveB} p2) - Min (AveR p2, AveB p2)
S2 = Max (Dis_P1, Dis_P2)
In addition, when taking the difference between the image data of two points on the diagonal line at the periphery of the screen and using the color value having the worst difference as the color shading characteristic S2, the following formula is used.

Dis_R = Max (AveR _p1 , AveR _p2 ) −Min (AveR _p1 , AveR _p2 )
Dis_B = Max (AveB _p1 , AveB _p2 ) −Min (AveB _p1 , AveB _p2 )
S2 = Max (Dis_R, Dis_B)
Thereafter, in step S24, based on the color shading characteristic S2 that is the determination information calculated as described above, the image capturing unit switching control unit 10d2 compares the color shading characteristic S2 that is the determination information of each of the image capturing units 2 and 3, In the monocular imaging mode (at the time of 2D imaging), the imaging unit switching process is performed by using an imaging unit with good image quality having the smallest value of the color shading characteristic S2 as determination information as the main imaging unit.

  As described above, according to the third embodiment, the determination information calculation unit 10b2 calculates the color shading characteristics for each color as the image quality determination information from the output value of each pixel data from the image quality information storage unit 10a2, and each of the calculated colors By comparing the shading characteristics and determining the imaging unit having the smallest value as the main imaging unit with good image quality, an image with better image quality can be obtained more reliably. Therefore, it is possible to prevent deterioration in image quality such as color shading due to variations in the imaging units 2 and 3.

  In the first embodiment, a case where an image capturing unit with better image quality is selected as the main image capturing unit using the SN ratio as the image quality determination information will be described. In the second embodiment, the image center determination information is the center of the screen. A case will be described in which a shading characteristic indicating a difference in brightness (luminance) from the peripheral portion of the screen is used to select an imaging unit with a small difference, that is, a better image quality as the main imaging unit. The image quality judgment information uses color shading characteristics that indicate the difference in output value (level) for each color between the center of the screen and the periphery of the screen, and the difference in the output value (level) for each color is small. However, the present invention is not limited to this, and the image quality determination information includes the SN ratio, resolution, shading characteristics, and color shading characteristics. Without even it can also be applied to the present invention for the case of selecting an imaging unit towards better image quality as a main image pickup unit using the two items.

(Embodiment 4)
In the first to third embodiments, the main imaging of the imaging unit having the better image quality using the image quality determination information among the first imaging unit 2 and the second imaging unit 3 at the time of manufacturing or at the time of shipping inspection after manufacturing. In the fourth embodiment, in addition to this, which of the first imaging unit 2 and the second imaging unit 3 is used for monocular imaging mode (at the time of 2D imaging) is described. A case will be described in which a user selection mode that allows the user to forcibly select is provided.

  FIG. 9 is a block diagram illustrating a configuration example of a main part of a portable information terminal device using the multiview imaging device according to the fourth embodiment of the present invention as an imaging unit. In FIG. 9, the same member numbers are given to the structural members that exhibit the same operational effects as the structural members of FIG. 1.

  In FIG. 9, the portable information terminal device 1 </ b> C using the multi-lens imaging device 20 </ b> C of the present invention as an imaging unit is different from the case of the first embodiment in that the imaging unit switching control unit of the control unit 10 of the first embodiment. 10d, that is, each SN ratio that is the image quality determination information of each of the imaging units 2 and 3 calculated by the determination information calculation unit 10b is compared, and an imaging unit having a higher SN ratio that is the determination information value is compared with the image quality. This is determined as a good main imaging unit (the first imaging unit 2 or the second imaging unit 3), and control is performed so that the determined main imaging unit is used in the monocular mode (2D shooting). In addition, it has a user selection mode in which the user can forcibly select which one of the first imaging unit 2 and the second imaging unit 3 is used in the monocular imaging mode (2D imaging). . In this user selection mode, one of the first imaging unit 2 and the second imaging unit 3 is controlled by controlling the imaging unit switching control unit 10d3 of the control unit 10C via the system control unit 9 by an operation input from the operation unit 13C. Is a mode in which the user can forcibly select.

  The first imaging unit 2, the second imaging unit 3, the memory unit 4, the signal processing unit 5, the lens driving / imaging element driving units 7 and 8, the system control unit 9, the control unit 10C, and the operation unit 13C perform multi-view imaging. A device 20C is configured. The display device 6 and the memory unit 12 are added to the mobile information terminal device 1C such as a digital camera. Further, the communication unit 11 is added to form another portable information terminal device 1C such as a camera-equipped mobile phone device. Here, a mobile phone device with a camera is shown as the portable information terminal device 1C. In short, the portable information terminal device 1 </ b> C only needs to include at least one of the display device 6, the communication unit 11, and the memory unit 12.

  Therefore, according to the fourth embodiment, even if there is a difference in image quality between the imaging units 2 and 3 due to the manufacturing process, each image data that is image quality information is stored for each imaging unit 2 and 3. Since the S / N ratio that is the determination information is calculated from the image quality information and the imaging unit is switched based on the S / N ratio that is the determination information, the main imaging unit is the one that can compare the image quality of each of the imaging units 2 and 3. In monocular photography (2D photography), high-quality image data with less noise can be obtained. This is the same as the effect of the first embodiment, but in addition to this, as the main image capturing unit is used as the main image capturing unit, the one having the better image quality of the image capturing units 2 and 3 during monocular shooting (2D shooting). Even when the image quality deteriorates due to a change or the like, the user can forcibly select another imaging unit by his / her own intention, and the deterioration of the image quality can be eliminated by replacing with another imaging unit. . In addition, by switching the determination information to be used depending on the shooting situation (for example, the SN ratio because noise is conspicuous in the dark mode, and color shading is used as the determination information to increase the color in the landscape mode). A high-quality image data can be obtained.

  In the fourth embodiment, the user selection mode is applied to the first embodiment in which the S / N ratio is obtained as the image quality determination information. However, the present invention is not limited to this, and shading is obtained using the user selection mode as the image quality determination information. The present invention may also be applied to the second embodiment. Similarly, this user selection mode may be applied to the third embodiment in which color shading is obtained as image quality determination information.

  In the first to fourth embodiments, the case where the image quality information and the determination information are stored in each storage unit in the digital camera is described. However, the present invention is not limited to this, and the image quality information is stored in a place other than the digital camera ( It may be configured to store in advance in a storage device outside the digital camera, and to perform imaging unit switching processing with good image quality using this. In short, at least two image pickup units 2 and 3 are provided for 3D shooting, and in the case of 2D shooting, at least one of the two image pickup units 2 and 3 is used as a main image pickup unit and A control unit that obtains image quality determination information based on the image data from at least two image capturing units 2 and 3 and selects the image capturing unit having the best image quality as the main image capturing unit based on the image quality determination information. 10, 10A to 10C. In addition, as described above, the chart for resolution inspection is simultaneously photographed by the first imaging unit 2 and the second imaging unit 3, and from each image data from the first imaging unit 2 and the second imaging unit 3 to where finely. A determination information calculation unit that performs resolution inspection to see if it is visible, and a determination information storage unit that stores resolution inspection results are provided outside the multi-lens imaging device, and the imaging unit switching control unit inside the multi-lens imaging device has better image quality. Is selected as the main imaging unit, and control is performed so as to shoot using the imaging unit with the better selected image quality during 2D shooting.

  Although not specifically described in the first to fourth embodiments, the image quality information is obtained by photographing a subject with uniform brightness and color, but the subject and photographing conditions are changed depending on the nature of the determination information. May be.

  Although not particularly described in the first to fourth embodiments, the image quality information is stored for each of the RGB colors, the determination information is calculated, and the imaging unit switching process is performed on all the RGB colors as the determination targets. Of these, only one or two colors, for example, only R, only G, or only R and B may be used.

  Although the image quality information (each image data) to be stored is the entire screen, which will be finally described and further described in the first embodiment, the present invention is not limited to this, and pixel data at any limited position is not limited thereto. You may remember. In short, each image data from the two imaging units 2 and 3 is each image data from any one of the entire effective pixel region, the single partial effective pixel region, and the plurality of partial effective pixel regions.

  In the second embodiment, the image quality information to be stored is set to a plurality of pixel data (each image data) in a pixel area in a limited place as shown in FIG. Pixel data (each image data) at an arbitrary position (area) in (full screen) may be stored in the image quality information storage unit.

  In the third embodiment, the image quality information to be stored is set at the pixel positions (two places) on the diagonal line as shown in FIG. The position (region) and pixel data (each image data) of a plurality of fixed points in the region may be stored in the image quality information storage unit.

  In the second and third embodiments, the image quality information to be stored calculates the determination information by averaging the output values of a plurality of pixels. However, an arbitrary number of pixels in one or a plurality of partial effective pixel regions are used. The determination information may be calculated from the data (each image data) and stored in the determination information storage unit.

  In the second embodiment, the ratio of the average values of the image data (pixel data) in the center and the peripheral area is calculated as the shading characteristic of the brightness (or luminance). However, the present invention is not limited to this. The shading characteristic of brightness (or luminance) may be calculated with.

  In the third embodiment, the color shading characteristic for each color is calculated from the variation in the average value of the image data (pixel data) in the center and the peripheral area of the screen. The characteristic may be calculated.

  In the first to third embodiments, the image data (pixel data) may be an image signal from an image sensor, an image signal obtained by signal processing of the image signal, a luminance signal, an RGB signal, or a color signal other than RGB. It doesn't matter.

  In the first to fourth embodiments, the control units 10, 10 </ b> A to 10 </ b> C are provided outside the system control unit 9. However, the present invention is not limited thereto, and the control units 10, 10 </ b> A to 10 </ b> C are respectively connected to the system control unit 9. You may provide inside.

  As mentioned above, although this invention has been illustrated using preferable Embodiment 1-4 of this invention, this invention should not be limited and limited to this Embodiment 1-4. It is understood that the scope of the present invention should be construed only by the claims. It is understood that those skilled in the art can implement an equivalent range from the description of specific preferred embodiments 1 to 4 of the present invention based on the description of the present invention and the common general technical knowledge. Patents, patent applications, and documents cited herein should be incorporated by reference in their entirety, as if the contents themselves were specifically described herein. Understood.

  The present invention relates to a multi-view imaging device and a multi-view imaging method capable of generating a moving image and a still image / image including stereoscopic vision, and a digital camera or a camera-equipped mobile phone using the multi-view imaging device as an imaging unit. In the field of portable information terminal devices such as devices, when the image quality of the other image pickup unit is better than that of one image pickup unit, the other image pickup unit is determined as the main image pickup unit, so that an image with better image quality can be obtained. It can be obtained more reliably.

DESCRIPTION OF SYMBOLS 1, 1A, 1B, 1C Portable information terminal device 2 1st imaging part 3 2nd imaging part 4 Memory part 5 Signal processing part 6 Display apparatus 7, 8 Lens drive / imaging element drive part 9 System control part 10, 10A, 10B 10C Control unit 10a, 10a1, 10a2 Image quality information storage unit 10b, 10b1, 10b2 Determination information calculation unit 10c, 10c1, 10c2 Determination information storage unit 10d, 10d1, 10d2, 10d3 Imaging unit switching control unit 11 Communication unit 12 Memory unit 13 Operation unit 20, 20A, 20B, 20C Multi-eye imaging device

Claims (20)

  At least two imaging units are provided for 3D shooting, and in the case of 2D shooting, at least one of the inside and outside of the multi-lens imaging device that uses any one of the at least two imaging units as a main imaging unit, A multi-view imaging apparatus having a control unit that obtains image quality determination information based on each image data from the at least two imaging units and selects an imaging unit with the best image quality based on the image quality determination information as the main imaging unit.   The control unit includes an image quality information storage unit that stores image data from the at least two imaging units, a determination information calculation unit that calculates image quality determination information based on the image data from the image quality information storage unit, A determination information storage unit that stores the image quality determination information calculated by the determination information calculation unit, and an imaging unit that determines, as the main imaging unit, an imaging unit to be used during 2D shooting based on the determination information from the determination information storage unit The multi-view imaging apparatus according to claim 1, further comprising a switching control unit.   The multi-view imaging apparatus according to claim 2, wherein the determination information calculation unit calculates an SN ratio as the image quality determination information from an output value of each image data from the image quality information storage unit.   The multi-view imaging apparatus according to claim 2, wherein the determination information calculation unit calculates a shading characteristic as the image quality determination information from an output value of each image data from the image quality information storage unit.   The multi-view imaging apparatus according to claim 2, wherein the determination information calculation unit calculates a color shading characteristic as the image quality determination information from an output value of each image data from the image quality information storage unit.   3. The multi-view imaging according to claim 2, wherein the image quality information storage unit stores each image data from the at least two imaging units in which a subject having uniform brightness is captured in each of the at least two imaging units. apparatus.   The image quality information storage unit stores, in each of the at least two imaging units, a ratio of color signals of at least two colors of each image data from the at least two imaging units in which a subject with uniform brightness is captured. The multi-eye imaging device according to claim 2.   The imaging unit switching control unit further has a function of performing selection control of which of the two imaging units is used for imaging based on a selection command externally provided by a user during 2D imaging. Multi-eye imaging device.   2. Each image data from the two image pickup units is each image data from any one of a whole effective pixel region, a single partial effective pixel region, and a plurality of partial effective pixel regions. The multi-lens imaging device according to any one of 2, 6, and 7.   The portable information terminal device which used the multiview imaging device in any one of Claims 1-9 for the imaging part.   The portable information according to claim 10, further comprising: a display device that displays an image signal from at least one of the at least two imaging units on a display screen based on image data after performing predetermined signal processing for display. Terminal device.   The portable information terminal device according to claim 10, further comprising: a memory unit that can record image data after performing predetermined signal processing on an image signal from at least one of the at least two imaging units for recording.   The portable information terminal device according to claim 10, further comprising: a communication unit that enables communication processing of image data after performing predetermined signal processing for image signals from at least one of the at least two imaging units for communication. When performing 2D shooting using at least two imaging units for 3D shooting, in the multi-eye imaging method using one of the at least two imaging units as a main imaging unit,
The control unit obtains image quality determination information based on the image data from the at least two imaging units, and has a control step of selecting an imaging unit having the best image quality as the main imaging unit based on the image quality determination information. Eye imaging method.   The control step includes an image quality information storage step for storing each image data from the at least two imaging units in an image quality information storage unit, and a determination information calculation unit based on each image data from the image quality information storage unit. A determination information calculation step for calculating determination information, a determination information storage step for storing the image quality determination information calculated in the determination information calculation step in a determination information storage unit, and an imaging unit switching control unit from the determination information storage unit The multi-eye imaging method according to claim 14, further comprising: an imaging unit switching control step for determining an imaging unit to be used at the time of 2D imaging as the main imaging unit based on the determination information.   The multi-view imaging method according to claim 15, wherein the determination information calculation step calculates an SN ratio as the image quality determination information from an output value of each image data from the image quality information storage unit.   The multi-view imaging method according to claim 15, wherein the determination information calculation step calculates a shading characteristic as the image quality determination information from an output value of each image data from the image quality information storage unit.   The multi-view imaging method according to claim 15, wherein the determination information calculation step calculates a color shading characteristic as the image quality determination information from an output value of each image data from the image quality information storage unit.   The multi-view imaging according to claim 15, wherein the image quality information storing step stores each image data from the at least two imaging units in which a subject with uniform brightness is captured in each of the at least two imaging units. Method.   In the image quality information storing step, color signals of at least two colors or more of each image data from the at least two imaging units in which an achromatic subject having a uniform luminance is captured on each of the at least two imaging units. The multi-eye imaging method according to claim 15, wherein the ratio is stored.

Images