JP2008172523A - Multifocal camera device, and control method and program used for it - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve image quality of a synthetic image, and to smooth connection between consecutive images when a plurality of images of the same photography subject photographed by a plurality of cameras with different focal distances are selected and synthesized to track the photography subject, and an output image is generated. <P>SOLUTION: At least one out of zoom control, exposure control, focus control, color temperature control, and depth-of-field control of a photography module of a camera 1 is controlled so as to reduce at least one change in a zooming scale factor, exposure, focus, color temperature, and a depth of field accompanying track of the photography subject of the image obtained by selecting and synthesizing. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a multifocal camera device having a plurality of cameras with different focal lengths for photographing the same subject, and a control method and program used therefor.

  2. Description of the Related Art Conventionally, a camera device capable of shooting in a long shooting distance range as well as a wide range of angle of view is known for in-store monitoring, crop or disaster observation, accident verification, or sports event shooting. .

  For example, Patent Literature 1 includes, for example, a plurality of cameras with different focal lengths for photographing the same subject such as a person, each camera having one or more photographing modules, and a plurality of images photographed by each photographing module. There has been proposed a multifocal camera device that generates an image obtained by selecting and synthesizing these images so as to reproduce an imaging target with a certain size as an output image.

  The multiple shooting modules provided in this multifocal camera device include a shooting module equipped with a wide-angle lens with a short focal length and a wide angle of view, and a subject with a long focal length and a narrow angle of view. This includes an imaging module equipped with a telephoto lens that magnifies the image, and each imaging module has a certain size for the subject image obtained by imaging the imaging area of the allocated imaging distance in the entire space within the imaging field of view. Is output as an image. As a result, a plurality of images obtained by a plurality of imaging modules that capture a spatial region including the position of the imaging target moving within the imaging field of view are selected and combined, and the imaging target is always regarded as a substantially constant size. Images can be acquired.

In conventional ordinary cameras, the same subject, i.e., the subject, is large in the vicinity and small in the distance, so we used a zoom lens to shoot a large subject far away. Accordingly, it is necessary to change the focal length of the lens. However, if such a multifocal camera is used, since any camera always captures the subject to be photographed at an appropriate size, a similar image can be obtained by selecting and combining later. There is an effect that it is not necessary to perform an operation for changing the focal length at the time of shooting.
JP 2006-14291 A

  Since the multifocal camera device of the above-mentioned patent document 1 shoots the entire space within the shooting field of view by sharing a plurality of shooting modules, it is optimized for each set shooting range for each shooting module. By performing such controls as zoom control, exposure control, focus control, color temperature control, and depth of field control, the images obtained by each shooting module are optimal for observation of shooting targets in each shooting range Can be. For example, if shade and sun are mixed in the field of view, and multiple shooting modules have shade or sun as the main shooting range, set a higher exposure value for the shooting module that uses the shade as the main shooting range. However, exposure control suitable for observing a subject to be photographed in each photographing range, such as setting a low exposure value, can be performed for a photographing module whose main photographing range is the sun.

  By the way, by moving the shooting target, an output image including the shooting target in a substantially central portion and an image shot by one or more shooting modules for shooting a spatial region including a new position of the shooting target are selected and combined. If the image is selected as a new output image, the zooming magnification, exposure, focus, color temperature, or depth of field of the image selected as the new output image may differ greatly from the previous output image, When the characteristic values of a plurality of images used in the image are greatly different from each other, the image quality of the output image obtained by the synthesis is deteriorated, and the connection between the continuous output images is poor, and the image is used for observing the photographing target. There is a problem that the performance of the system deteriorates.

  In view of the above circumstances, the present invention improves the image quality of an output image obtained by combining a plurality of images shot by a plurality of shooting modules to generate an output image, and continuously An object of the present invention is to provide a multifocal camera device that smoothes the connection between output images to be output, a control method used therefor, and a program.

  The multifocal camera device of the present invention includes a plurality of cameras with different focal lengths for photographing the same subject, and each of the cameras has one or more photographing modules, and each of the plurality of photographing images taken by the photographing modules. In a multifocal camera device that records an image obtained by selecting and combining images so as to track an imaging target, the imaging module independently performs zoom control, exposure control, focus control, color temperature control, and coverage. At least one control of depth of field control is possible, and tracking of an imaging target is performed by at least one of zooming magnification, exposure, focus, color temperature, and depth of field of an image obtained by selection and synthesis. Control means for controlling the photographing module is provided so as to reduce the accompanying change.

  In the above-described apparatus, a tracking unit that tracks the position of the shooting target from a plurality of images shot by each shooting module, and an image that is used for selecting and combining the shooting modules from the tracking information of the shooting target by the tracking unit. It may have a selection means for selecting a photographing module to be photographed.

  The control method of the present invention includes a plurality of cameras each having one or more imaging modules having different focal lengths for imaging the same imaging target, and tracks the imaging target with a plurality of images captured by each imaging module. Controlling at least one of zoom control, exposure control, focus control, color temperature control, and depth-of-field control of the imaging module in a multi-focus camera device that records images obtained by selecting and combining A control method for photographing so as to reduce a change caused by tracking of an object to be photographed, at least one of zooming magnification, exposure, focus, color temperature, and depth of field of an image obtained by selection and synthesis. The module is controlled.

  In the above method, the position of the shooting target is tracked from a plurality of images shot by each shooting module, and the image used for selecting and combining among the shooting modules from the tracking information of the shooting target obtained by tracking. It is also possible to select an imaging module for imaging.

  The program of the present invention comprises a computer having a plurality of cameras each having one or more photographing modules having different focal lengths for photographing the same photographing object, and a plurality of images photographed by each photographing module are selected as photographing objects. At least one of zoom control, exposure control, focus control, color temperature control, and depth of field control of an imaging module in a multifocal camera device that records an image obtained by selecting and combining for tracking This is a program for realizing control, and at least one change in zooming magnification, exposure, focus, color temperature, and depth of field of an image obtained by selecting and synthesizing is used for tracking an imaging target. This is to realize control of the imaging module so as to reduce the accompanying reduction.

  In the above program, the position of the shooting target is tracked from a plurality of images shot by each shooting module on the computer, and the shooting target tracking information obtained by tracking is used for selecting and combining each shooting module. It may be for realizing selecting a photographing module for photographing an image to be captured.

  Here, the object to be photographed means an object to be photographed, such as a person's face, a pet such as a dog or cat, or a car.

  Here, the “tracking information” may include a stepwise change in the position of the photographing target and the position of the photographing target in a plurality of time-sequential images. It may be information only.

  According to the multifocal camera device of the present invention and the control method and program used therefor, the camera includes a plurality of cameras with different focal lengths for photographing the same subject, each camera having one or more photographing modules, In a multifocal camera device that records images obtained by selecting and synthesizing a plurality of images shot by each of the shooting modules to track a shooting target, zoom control, exposure control, focus control of the shooting module, At least one of color temperature control and depth of field control is controllable, and at least one of zooming magnification, exposure, focus, color temperature, and depth of field of an image obtained by selecting and combining, Since it controls the shooting module so as to reduce the change accompanying the tracking of the shooting target, it was obtained by selection and synthesis. It can be smoothly connected between the images to be continued.

  In the multifocal camera device of the present invention and the control method and program used therefor, the position of the shooting target is tracked from a plurality of images shot by each shooting module, and from the tracking information of the shooting target obtained by tracking, Select the shooting module that captures the image to be used for selection and composition, and select the selected shooting module, zooming magnification, exposure, focus, color temperature, and exposure If the control is to reduce at least one change in the depth of field due to tracking of the shooting target, select the shooting module that captures the image that is supposed to be used for selection and synthesis. Zooming magnification, exposure, focus, color temperature, and depth of field of the image obtained by selecting and compositing only the selected shooting module It is possible to control so as to reduce a change in at least one, can improve the efficiency of control.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  The multifocal camera device 1 according to an embodiment of the present invention captures a plurality of images obtained by photographing a subject to be photographed, for example, the face H of the person at three different focal lengths when a person is moving within the photographing field of view. The image obtained by selecting and synthesizing the person's face H is recorded, and the face at the next shooting is recorded from the tracking information of the face H obtained by tracking the person's face H. Predict the position of H, select the shooting module that captures the image that is supposed to be used for selection and synthesis, align the settings of the selected shooting module, improve the image quality of the image obtained by selection and synthesis It is something to be made.

  FIG. 1 is a schematic block diagram showing a configuration of a multifocal camera device according to an embodiment of the present invention, FIG. 2 is a schematic diagram showing an imaging region by the multifocal camera device of FIG. 1, and FIG. It is a figure which shows an example of the image acquired by image | photographing with the multifocal camera apparatus of FIG.

The multifocal camera device 1 includes an imaging unit 10 that includes one or more imaging modules M that respectively constitute _three cameras C ₁ , C ₂ , and C ₃ having different focal lengths for imaging the same face H; A temporary storage unit 20 that stores images P _{1 to} P ₁₄ photographed by the photographing modules M _{1 to} M ₁₄ of the photographing unit 10 and a plurality of images photographed by the photographing modules M _{1 to} M ₁₄ of the photographing unit 10. A tracking unit 30 that tracks the position of the face H from P _{1 to} P _14, and a plurality of images P _{1 to} P captured by the imaging modules M _{1 to} M ₁₄ using the position T of the face H in the tracking unit 30. ₁₄ is selected and combined so as to track the face H, a recording unit 50 that records the image U obtained by selecting and combining, and the position T and the change direction V of the face H by the tracking unit 30. From each shooting module M Selected images to be used in the synthesis selects the camera module R ₁ to R _N to shoot, change in the exposure of the image to be using the selected imaging module R ₁ to R _N, a synthetic reduction in the synthesis section 40 As described above, a control unit 60 that controls exposure is provided. The recording unit 50 is connected to a monitor 70 that is a display device via an external interface (not shown).

The photographing unit 10 shoots the photographing areas A ₁ , A ₂ , and A ₃ whose photographing distances in the entire space in the photographing field are d ₁ , d ₂ , and d ₃ , and three cameras C ₁ , is intended to shoot with C _{2, C 3,} the focal length of the cameras _{C 1,} C 2, _{C 3} is a _{f 1,} f _{2, f} 3, respectively _{(f 1 <f 2 <f} 3) . Each camera C ₁ , C ₂ , C ₃ has one or more photographing modules M, and the longer the focal length f, the narrower the field of view of the photographing modules constituting the camera. The entire field of view is divided and photographed by a plurality of photographing modules. Specifically, the camera C ₁ has one photographing module M ₁ , the camera C ₂ has four photographing modules M _{2 to} M ₅ , and the camera C ₃ has nine photographing modules M _{6 to} M _14. The imaging modules M _{1 to} M ₁₄ output images P _{1 to} P ₁₄ having the same size.

In addition, each camera C ₁ , C ₂ , C ₃ is not limited to the set focus distance d ₁ , d ₂ , d ₃ set for each camera, but when the face H exists in any place in the field of view. Also have in-focus ranges (range of depth of field) δ ₁ , δ ₂ , δ ₃ so that the face H can be clearly imaged, and adjacent in-focus ranges δ ₁ , δ ₂ , and in-focus. The focal ranges δ ₂ and δ ₃ are set to be continuous with respect to the shooting direction.

Further, all the imaging modules M _{1 to} M ₁₄ are configured to perform imaging at a timing synchronized by a clock generated by the control unit 60, for example.

  Hereinafter, the configuration of each imaging module M will be described. FIG. 4 is a diagram illustrating an example of the configuration of the imaging module M.

  The optical system of the photographing module M includes an imaging lens 12 having a required focal length, a diaphragm 13 and a shutter 14 for adjusting exposure of an image obtained by photographing, and the diaphragm diameter and opening / closing of the shutter. Is controlled by aperture value data and shutter speed data output from the control unit 60 described later. Note that the configuration of the imaging lens 12 of each photographing module M differs depending on the focal length.

  Behind the optical system, it has a photographing element 15 such as a CCD and an A / D converter 16, and is obtained by subjecting subject light that has passed through the optical system to photoelectric conversion by the photographing element 15 such as a CCD. The analog image signal is converted into a digital image signal by the A / D converter 16, and the converted digital image signal is output to a memory.

Temporary storage unit 20 is provided with the same number of memory _me 1 _{~me 14} respectively corresponding to the plurality of imaging modules _M 1 ~M ₁₄ of the imaging unit, simultaneously the image data from all of the camera module _M 1 ~M ₁₄ In addition, it is possible to capture and store continuously in parallel. Incidentally, the temporary storage unit 20 has been described for the case with the same number of memory me ₁ ~me ₁₄ respectively corresponding to the plurality of imaging modules M ₁ ~M ₁₄ of the imaging unit 10 is not limited to this, two or more One memory may be provided for each of the imaging modules.

Each of the memories me _{1 to} me ₁₄ of the temporary storage unit 20 temporarily and sequentially stores digital image data of photographed images continuously output from the photographing modules M _{1 to} M ₁₄ to which the memories are connected. For example, a semiconductor memory such as SRAM or DRAM, or a combination of these can be used.

FIG. 5 is a diagram illustrating a configuration of the tracking unit 30. The tracking unit 30 is connected to all of the plurality of memories me _{1 to} me ₁₄ of the temporary storage unit 20, and images P _{1 to} P ₁₄ captured by each of the plurality of imaging modules M _{1 to} M ₁₄ from the plurality of memories. The position of the face H is acquired from the acquired image and includes a position detection unit 31 that detects the position of the face H and a movement direction detection unit 32 that detects the movement direction of the face H.

Position detector 31, a second detecting a first position detecting unit 31a for detecting the position of the face H in a plane perpendicular to the photographing direction from the image P ₁ to P ₁₄ obtained, the position of the face H for photographing direction The position detecting unit 31b is configured.

First position detection unit 31a temporarily reads a plurality of memory _me 1 _{~me 14} images _P 1 to P ₁₄ stored in the storage unit 20, detects an area of the face H from the images (face region) The approximate center position of the area is acquired as the current position T (x, y) of the face H.

Specifically, as described in Japanese Patent Application Laid-Open No. 2006-139369, the feature amount of each pixel in a plurality of sample images whose detection targets such as faces are known, that is, the density at each pixel on the image changes. By learning in advance the direction to be performed and the magnitude of the change, a discriminator for identifying whether or not it is a detection target is acquired corresponding to the feature amount, and the target image PP ₁ (image P obtained) for each camera is acquired. ₁ ), PP ₂ (an image made up of images P _{2 to} P ₅ ), PP ₃ (an image made up of images P _{6 to} P ₁₄ ), and scanning the discriminator to detect a detection target. it can.

At this time, in an image shot by a normal camera, the size of the face H of the person included in the image changes depending on shooting conditions such as a shooting distance from the camera to the face H. It is necessary to repeatedly perform detection while changing the size of the discriminator to be scanned from large to small. Here, in the multifocal camera 1 of the present invention, the focal length f _j (j = 1 to 3) and setting of each camera are set. The in-focus distance d _j (j = 1 to 3) and the size p _j (j = 1 to 3) of the subject image per pixel in the obtained image are known, and the actual size of the face H Using H _j , the size h _j (h _j = f _j H _j / d _j ) of the image of the face H and the size of the face H in the image when each camera can clearly photograph the face H ( h _j / p _j pixel = f _j H _j / p _j d _j pixel) The detection efficiency can be improved by detecting the face H using a discriminator having a size corresponding to the size.

That is, the multifocal camera 1 captures a subject to be photographed, for example, the face H with a plurality of cameras C ₁ , C ₂ , and C ₃ having different focal lengths f. Among the target images PP ₁ , PP ₂ , and PP ₃ photographed by the above, the image of the face H most clearly captured by the camera having the focus setting distance closest to the photographing distance. By performing detection using a discriminator having a size corresponding to the height, the detection efficiency and further the detection accuracy can be improved. Thereby, the face area can be accurately detected from the target image obtained by at least one camera, and the face area can be easily detected also from the target image obtained by another camera based on the detection result. . Then, the approximate center position of the detected face area is acquired as the current position of the face H.

In the face detection method described above, a case has been described in which the discriminator scans the entire area of the target images PP ₁ , PP ₂ , and PP ₃ captured by the imaging module group of each camera to detect the face area. By detecting the difference between the current image and the past reference image from the current image, the area of the moving person is detected, and the face area is detected by scanning only a part of the area including the extracted area. You can also. In addition to the method of detecting the face H by scanning the discriminator on the target image, a plurality of object regions are extracted and extracted from the target image as described in JP-A-2004-62604. By identifying the object type for each of the plurality of object areas and determining whether or not the certainty level of the identified type is greater than a predetermined threshold, it is possible to detect any other type of detection target. A photographing target such as the face H may be detected by a method.

The second position detection unit 31b performs spatial frequency analysis on the area of the face H detected from the target images PP ₁ , PP ₂ , and PP ₃ by the first position detection unit 31a, and the face H with respect to the shooting direction. The position of is detected.

Since the sharper image in focus changes the brightness of the contour and pattern clearly, the focus level can be measured by examining the frequency component of the spatial frequency of the image representing the degree of this change. Thereby, spatial frequency analysis is performed on each face area detected from each of the target images PP ₁ , PP ₂ , PP ₃ , the focus level y is measured, and a target including a face area with the highest focus level y The camera that captured the image is determined, and the focus setting distance d in the determined camera is detected as the position of the face H with respect to the shooting direction.

  The movement direction detection unit 33 detects a movement direction of the face H with respect to a plane direction orthogonal to the shooting direction, and a second movement direction detection detects a movement direction of the face H with respect to the shooting direction. It is comprised from the part 33b.

The first movement direction detection unit 33a detects the current position T _NEW (x, y) of the face H detected by the first position detection unit 31a and the first position detection unit 31a in the same manner during the previous shooting. The moving direction V (x, y) from the previous position T _OLD (x, y) to the current position T _NEW (x, y) is calculated using the previous position T _OLD (x, y) of the face H. calculate.

The second moving direction detection unit is configured to detect the movement of the face H based on the increase / decrease of the focus levels y ₁ , y ₂ , y _{3 in} the cameras C ₁ , C ₂ , C ₃ obtained by the second position detection 31b. The moving direction with respect to the photographing direction is detected.

FIG. 6 shows the position of the subject with respect to the shooting direction of the camera, that is, the subject using the cameras C ₁ , C ₂ , and C ₃ whose focus setting distances are d ₁ , d ₂ , and d ₃ with respect to the shooting distance d, respectively. It is a graph which shows the relationship of the focus level at the time of imaging | photography. As shown in the figure, the focus level of each camera is the maximum when the subject's shooting distance d matches the focus setting distance of the camera, and gradually decreases as it deviates from the focus setting distance. By examining the focus level and the change direction of the focus level, the moving direction V (z) of the face H relative to the photographing direction can be detected.

As shown on the graph of FIG. 6, the focus level determination section by the obtained face region relative focus level each y _{1 NEW} of each _{camera, y 2NEW,} a y _3NEW, similarly focus level during the previous photographing each camera focus level each _y 1OLD for face region obtained by the determination _unit, y _2OLD, if it is _{y 3OLD,} change of focus level of each camera, _{respectively, y 1NEW -y 1OLD> 0,} y 2NEW since -y _{2OLD> 0,} a y _{3NEW -y 3OLD} <0, it is possible to detect the distance with respect to the photographing direction of a person is moving from the place closest to _{d 1} in the direction of _{d 2.}

The combining unit 40 selects and combines the images P _{1 to} P ₁₄ captured by each of the plurality of imaging modules M _{1 to} M ₁₄ so as to track the person's face H, and generates an output image U. Is. First, a camera that can capture the face H calculated by the second position detection unit at the highest focus level is selected, and the face T is detected using the position T of the face H detected by the first position detection unit 31. It is determined whether or not H is included in a predetermined range at a substantially central portion of any of the images captured by each of the plurality of imaging modules constituting the camera. When it is determined that the face H is included in a predetermined range at a substantially central portion of any image, an image including the face H within a predetermined range at a substantially central portion is selected as the output image U. . On the other hand, if it is determined that the image is not included in the predetermined range in the substantially central portion of all the images, the face of the plurality of images captured by each of the plurality of imaging modules constituting the selected camera is displayed. One image having a face H in the substantially central portion is synthesized by synthesizing all images including at least a part of a certain range of regions having an H in the substantially central portion (region corresponding to the size of the output image). An output image U is generated.

  FIG. 7 is a diagram illustrating a configuration of the control unit 60. The control unit 60 includes a shooting module selection unit 61 that selects a shooting module for shooting an image that is assumed to be used for selection / combination in the combining unit 40 at the next shooting, and an aperture value and a shutter of the selected shooting module. The control value determination unit 62 determines the speed.

First, the imaging module selection unit 61 first detects the movement direction V of the person detected by the movement direction detection unit 33a at the current position T _NEW (x, y, z) of the person detected by the position detection unit 31 of the tracking unit 30. By adding (x, y, z), the predicted position T _EXP (x, y, z) of the person at the time of the next photographing is obtained. If the person's face H exists at the calculated predicted position T _EXP , a camera capable of photographing the face H at the highest focus level is obtained, and among the plurality of imaging modules constituting the camera, the person's face H is obtained. All of the imaging modules R ₁ to R including at least a part of a certain area (area larger than the size of the output image) within the imaging range having the predicted face position T _EXP at the substantially central portion. to select the R _N.

The control value determining unit 62 selects the images P _{1 to} P ₁₄ photographed by each of the plurality of imaging modules M _{1 to} M ₁₄ at the next photographing so as to track the face H of the target person in the composition unit. Shooting an image presumed to be used for selection / composition so as to reduce the change in exposure in the image obtained by selection / combination when generating an output image U by compositing and it controls the aperture value and the shutter speed of the photographing module R ₁ to R _N, which is selected by the photographing module selection unit 61 as a module in advance.

Specifically, the exposure of each of the selected one or more photographing modules R _{1 to} R _N is B _{1 to} B _N , respectively, and is obtained by selecting and synthesizing from the image obtained by the current photographing by the synthesizing unit. when exposed in was image for output is B _U, as shown in the following formula (1), in their respective exposure values B ₁ .about.B _N and B _U, weight W determined by the method described below after multiplying ₁ to _W-N or _{W U,} each of these values by summing the calculated exposure value _{B k,} the exposure value _{B k} obtained, one or more imaging modules the selected _R 1 to R _N the exposure set value, and controls the aperture value and the shutter speed of the photographing module R ₁ to R _N.

B _k = W ₁ B ₁ + W ₂ B ₂ +... + W _N B _N + W _U B _U (1)
Here, the sum of the weights W _{1 to} W _N and W _U is 1, and the weights W _{1 to} W _N and W _U can be determined by various methods. For example, a weight for an exposure value of a photographing module that captures an image predicted to include a predicted position T _EXP of a person's face is set to 1, and the exposure value of the photographing module is set to be completely imitated by another photographing module. A method for setting a greater weight as the distance from the predicted position T _EXP of the person's face to the center position of the image obtained by each imaging module is closer, and an exposure value in an image generated as an output image in the current shooting how always sets the weight of the constant value B _U, etc., can be determined by any setting method. As a result, the photographing module selection unit 61 selects the images photographed by each of the plurality of imaging modules at the next photographing so as to reduce the change in exposure in the image obtained by selecting and synthesizing by the synthesizing unit. The exposure setting values of the photographing modules R _{1 to} R _N can be determined.

  The recording unit 50 is connected to the synthesizing unit 40, and records and stores the digital image data of the synthesized image generated by the synthesizing unit 40 so as to be readable and writable. As this recording unit, for example, a volatile memory such as SRAM or DRAM, or a semiconductor storage element (memory) such as a nonvolatile memory such as EPROM, EEPROM, or flash memory, a hard disk, an optical disk, a magneto-optical disk, a magnetic disk, a magnetic disk, etc. A recording medium such as a recording tape or a combination of these is used.

  The recording unit 50 is connected to a monitor 70 for displaying an image stored in the recording unit via an external interface (not shown).

Next, the operation of the multifocal camera device 1 will be described. First, a plurality of images P _{1 to} P ₁₄ obtained by photographing the same face H with the photographing modules M _{1 to} M _{14 at} three different focus setting distances d _{1 to} d ₃ by the photographing unit ₁₀ are temporarily stored. 20 stored. Next, the position detection unit 31 of the tracking unit 30 acquires images P _{1 to} P ₁₄ captured by each of the plurality of imaging modules M _{1 to} M ₁₄ stored in the temporary storage unit 20, and from the acquired images. detecting a current position T _NEW face H, and outputs the current position T _NEW of the detected face H in the moving direction detection unit 32 and the combining unit 40. In the synthesis unit 40, the face H is tracked with the images P _{1 to} P ₁₄ captured by each of the plurality of imaging modules M _{1 to} M ₁₄ based on the current position of the face H input by the position detection unit 31. Thus, the output image U is generated by selecting and combining. Then, the generated image U is recorded in the recording unit 50 and displayed on the monitor 70 via the external interface.

The moving direction detection unit 32 uses the current position T _NEW of the face H input by the position detection unit 31 and the previous position T _OLDE of the face H input by the position detection unit 31 at the same time as the previous shooting. The moving direction V of H is detected. Next, using the current position T _{NEW of} the face H and the moving direction V of the face H detected by the moving direction detecting unit 32 by the shooting module selecting unit 61 of the control unit 60, the composition unit performs the following shooting. selecting a camera module R ₁ to R _N taking an image suspected of being used in the selection and combining. Thereafter, the control value determination unit 62 selects the aperture value and shutter speed of the imaging module selected by the imaging module selection unit 61, and the synthesis unit 40 selects and captures images captured by each of the plurality of imaging modules at the next imaging. It determines so that the change of the exposure in the image U obtained by combining may be reduced. The determined aperture value and the shutter speed is set to the photographing module R ₁ to R _N, the next photographing is performed. And the process mentioned above is performed repeatedly.

According to the above-described embodiment, a plurality of cameras C ₁ , C ₂ , and C ₃ having different focal lengths for photographing the same subject S are provided, each of which has one or more photographing modules M, An imaging module in a multifocal camera device that records an image U obtained by selecting and synthesizing a plurality of images P _{1 to} P ₁₄ captured by the imaging modules M _{1 to} M ₁₄ so as to track an imaging target. In the exposure control method of M _{1 to} M ₁₄ , the imaging module is controlled so as to reduce the change accompanying the tracking of the imaging target of the exposure of the image U obtained by selection and synthesis. The connection between successive images obtained by selection and synthesis can be smoothed, and the image quality of the image obtained by synthesis can be improved.

Further, the position of the photographing target is tracked from the plurality of images P _{1 to} P ₁₄ photographed by the photographing modules M _{1 to} M ₁₄ , and each photographing module M ₁ to M ₁ is obtained from the tracking information of the photographing target obtained by tracking. selection of M _14, captures an image used in the synthesis selects the camera module R ₁ to R _N, a photographing module R ₁ to R _N, which is selected, selection, the exposure of an image obtained by compositing The control is performed so as to reduce the change, and the efficiency of the control can be improved.

  The present invention is not limited to the above embodiment. For example, in the above-described embodiment, the case where the photographing module can independently control the exposure and the photographing module is controlled so as to reduce the change in the exposure of the image obtained by selection and synthesis has been described. The photographing module can independently control at least one of zoom control, exposure control, focus control, color temperature control, and depth of field control, and a zoom magnification of an image obtained by selecting and combining The imaging module may be controlled to reduce at least one change in exposure, focus, color temperature, and depth of field.

  For example, when the subject is moving somewhat in the photographing direction of the point camera device 1, the camera that picks up and synthesizes the image obtained by selecting and combining the subject to be tracked by moving the subject is taken by the telephoto lens. When switching from a module to a wide-angle lens shooting module, the telephoto lens shooting module has the same aperture value, but the depth of field is shallower than that of the wide-angle lens shooting module. When switching from the wide-angle lens photographing module to the telephoto lens photographing module, the aperture value in the telephoto lens photographing module can be set higher than usual.

  Further, in the above embodiment, before selecting / combining the images captured by each of the plurality of imaging modules by the combining unit, the images used for selection and combining are stored in the imaging module at the time of acquiring each image. Between consecutive images obtained by selecting and combining by performing image correction based on the setting value of at least one of zoom control, exposure control, focus control, color temperature control, and depth of field control It is also possible to improve the image quality of an image obtained by smoothing the connection between the images.

  For example, when a plurality of images used for composition in the composition unit are obtained by photographing modules controlled with different setting values, image correction is performed so that the joints of the plurality of images become smooth. By doing so, it is possible to improve the image quality of the image obtained by the synthesis.

  In the above embodiment, the case where the tracking unit 30, the combining unit 40, the recording unit 50, and the control unit 60 are integrally configured in the multifocal camera device 1 has been described. It may be provided as a separate unit outside the focus camera device 1.

Further, the case where the three cameras C ₁ , C ₂ , C ₃ having different focal lengths are provided with a predetermined different number of imaging modules M has been described, but each camera C ₁ , C ₂ , C _{3 has been described.} Need only have one or more photographing modules M, and may have the same number of photographing modules.

  Further, when the photographing target is a person's face, the photographing module selection unit detects in advance when the photographing module for photographing an image presumed to be used for selection / combination in the composition unit 40 is selected. It is also possible to select a photographing module that captures an image that is estimated to be used for selection / combination by predicting the moving direction of the face reflecting the orientation of the person's face.

1 is a schematic block diagram showing the configuration of a multifocal camera device according to an embodiment of the present invention. The schematic diagram which shows the imaging | photography area | region by the multifocal camera apparatus of FIG. The figure which shows an example of the image acquired by image | photographing with the multifocal camera apparatus of FIG. The figure which shows an example of a structure of the imaging | photography module of FIG. The figure which shows the structure of the tracking part of FIG. The graph for demonstrating the movement direction detection method by the 2nd movement direction detection part of the tracking part of FIG. The figure which shows the structure of the control part of FIG.

Explanation of symbols

1 Multifocal camera device f Focal length C Camera M Shooting module P Image H Face (shooting target)
T Shooting target position 13 Aperture (exposure control)
14 Shutter (exposure control)
30 Tracking unit (tracking means)
40 synthesis unit 50 recording unit 60 control unit (control means)
61 Selection part (selection means)

Claims (6)

A plurality of cameras having different focal lengths for photographing the same photographing object are provided, each of the cameras has one or more photographing modules, and the photographing object is tracked with a plurality of images photographed by the photographing modules. In a multifocal camera device that records an image obtained by selecting and combining
The photographing module can independently control at least one of zoom control, exposure control, focus control, color temperature control, and depth of field control,
Control the photographing module so as to reduce a change caused by tracking the photographing target of at least one of zooming magnification, exposure, focus, color temperature, and depth of field of the image obtained by the selection and synthesis. A multifocal camera device comprising control means for performing the control. Tracking means for tracking the position of the subject to be photographed from a plurality of images photographed by each of the photographing modules;
2. The apparatus according to claim 1, further comprising: a selecting unit that selects an imaging module that captures an image to be used for the selection and synthesis from the imaging modules, based on the tracking information of the imaging target by the tracking unit. The multifocal camera device described. A plurality of cameras each having one or more photographing modules having different focal lengths for photographing the same photographing object are selected, and a plurality of images photographed by each photographing module are selected and combined so as to track the photographing object. A control method for controlling at least one of zoom control, exposure control, focus control, color temperature control, and depth-of-field control of the photographing module in a multifocal camera device that records an image obtained in this manner, ,
Control the photographing module so as to reduce a change caused by tracking the photographing target of at least one of zooming magnification, exposure, focus, color temperature, and depth of field of the image obtained by the selection and synthesis. A control method characterized by: Tracking the position of the shooting target from a plurality of images shot by each shooting module,
4. The control method according to claim 3, further comprising: selecting a photographing module for photographing an image used for the selection and synthesis from the photographing modules from the tracking information of the photographing target obtained by tracking. On the computer,
A plurality of cameras each having one or more photographing modules having different focal lengths for photographing the same photographing object are selected, and a plurality of images photographed by each photographing module are selected and combined so as to track the photographing object. To control at least one of zoom control, exposure control, focus control, color temperature control, and depth of field control of the photographing module in a multifocal camera device that records an image obtained in this manner The program of
Control the photographing module so as to reduce a change caused by tracking the photographing target of at least one of zooming magnification, exposure, focus, color temperature, and depth of field of the image obtained by the selection and synthesis. A program to make it happen. On the computer,
Tracking the position of the shooting target from a plurality of images shot by each shooting module,
The program according to claim 5, for realizing the selection of a photographing module for photographing an image used for the selection and synthesis from the photographing modules from the tracking information of the photographing target obtained by tracking.

Images