JP2009188953A - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and method for easily performing self-photographing within predetermined composition. <P>SOLUTION: The imaging apparatus includes: an imaging portion 13 for imaging a subject and converting an image into image data; a correlation operating portion 59 for detecting a degree of matching between first image data 53b obtained by imaging a background using the imaging portion and second image data 53a obtained by imaging a photographer under the background; a decision portion 65 for determining whether the degree of matching is settled within a predetermined range; an imaging operation control portion 15 for causing the imaging portion to start imaging operation when the decision portion determines that the degree of matching is settled within a predetermined range; and a recording portion for recording third image data obtained by starting the imaging operation by the imaging portion by operating the imaging operation control portion. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method that can easily perform self-taking with a predetermined composition.

  An imaging apparatus such as a digital camera is required to perform so-called self-photographing in which a photographer photographs himself without causing a photographing mistake.

The digital camera described in Patent Document 1 detects the outline of a face included in a subject based on an imaging signal from an imaging element, and determines whether or not the detected outline of the face falls within a preset imaging range. The result is output.
As a result, mistakes such as part of the face being cut off are less likely to occur in selfies.
JP 2005-217768 A

However, the technique disclosed in Patent Document 1 does not consider the background image.
For example, when a composition including a background image is determined in advance and a self-photographing is to be performed with the composition, the technique disclosed in Patent Document 1 allows the photographer himself to capture without mistake, but the composition intended for the background image It is not guaranteed that

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging apparatus and an imaging method capable of easily taking a selfie with a predetermined composition.

  In order to solve the above problems, the present invention provides an imaging unit that captures an image of a subject and converts it into image data, first image data obtained by imaging a background with the imaging unit, and imaging based on the background A correlation calculation unit that detects the degree of coincidence with the second image data obtained by imaging the person, a determination unit that determines whether the degree of coincidence is within a predetermined range, and the degree of coincidence in the determination unit Obtained when the imaging operation control unit that starts the imaging operation by the imaging unit and the imaging operation control unit operates to start the imaging operation by the imaging unit is determined. And a recording unit that records three image data.

  Further, the present invention is the imaging apparatus according to the above-described invention, further comprising an output unit for notifying a photographer of a shooting direction so that the degree of coincidence falls within a predetermined range.

  In the imaging apparatus according to the above-described invention, the output unit notifies the start of shooting when the degree of coincidence falls within a predetermined range.

  According to the present invention, in the imaging apparatus according to the above-described invention, the imaging apparatus automatically starts photographing after the degree of coincidence falls within a predetermined range.

  The present invention also includes a step of capturing a background to acquire first image data, a step of capturing a photographer under the background to acquire second image data, the first image data, and the first image data. A step of performing a correlation operation for detecting the degree of coincidence of the two image data, a step of determining whether or not the degree of coincidence is within a predetermined range, and when the degree of coincidence is within the predetermined range An imaging method comprising: a step of performing an imaging operation; and a step of recording third image data obtained by performing the imaging operation in a recording unit.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device and imaging method which can perform a self-portrait simply with a predetermined composition can be provided.

[First Embodiment]
FIG. 1 is a perspective view of an imaging apparatus according to the first embodiment of the present invention. FIG. 2 is a block diagram showing a schematic configuration of the imaging apparatus according to the first embodiment of the present invention.
The configuration and operation of the imaging apparatus will be described with reference to FIGS.

  An imaging light beam from a subject (not shown) enters the imaging element 12 via an imaging lens system 11 including an imaging lens 11a and a diaphragm 11b. The image sensor 12 is configured using a solid-state image sensor such as a CCD or a CMOS. The imaging light beam incident on the imaging device 12 is photoelectrically converted into an electrical signal here, and then reaches the motion vector detection circuit 23 via the imaging circuit 13.

  The imaging circuit 13 converts the analog signal output from the imaging device 12 into digital image data. Then, when shooting in the self-shooting mode, the image data is output to the motion vector detection circuit 23 at a predetermined timing.

  The motion vector detection circuit 23 determines from the input image data whether the self-portrait image has a predetermined composition. If the motion vector detection circuit 23 determines that the self-portrait image has a predetermined composition, it supplies the captured image data to the image processing circuit 24. Details of the motion vector detection circuit 23 will be described later.

The image processing circuit 24 performs digital signal processing such as γ correction, white balance correction, and automatic exposure adjustment of the image signal. Further, for example, image compression / decompression is performed based on the JPEG standard. The compressed / decompressed image data is transferred to and stored in the video memory 26, and can be displayed on the image display LCD 35 via the video output circuit 33, and can be output to a video device from the video output terminal 36. is there.
Instead of the image display LCD 35, a display device using an organic EL (Electroluminescence) display, LED, or the like may be used.

  The image data subjected to the image processing can be recorded from the I / F 27 to the removable memory 31 that is detachable from the camera via the card slot 30. Here, the removable memory 31 may be an optical storage medium such as a flash memory or a DVD (Digital Versatile Disk) that is a non-volatile memory. Further, the image data subjected to the image processing can be input / output to / from an external device via the external input / output terminal 37 via the external I / F 28.

  Furthermore, the imaging apparatus 1 is provided with a system controller 15 that controls the operation of the entire imaging apparatus. The system controller 15 is connected to a lens driving unit 17, a strobe light emitting unit 18, an operation unit 19, a speaker 40, and a direction display unit 41.

  The lens driving unit 17 moves the imaging lens 11a in the optical axis direction. The strobe light emitting unit 18 emits strobe light. The operation unit 19 includes a release switch 42, power on, various mode settings, switches and buttons for performing battery display selection, and the like. The speaker 40 instructs the photographer with the direction of the camera by voice so that the self-portrait image has a predetermined composition. The direction display unit 41 displays the direction of the camera for the photographer so that the self-portrait image has a predetermined composition.

The system controller also includes a motion vector detection circuit 23, an image processing circuit 24, a buffer memory 25, a video memory 26, an interface (I / F) 27, and an external interface (via the common bus 21. I / F) 28 is electrically connected.
The power supply unit 39 is provided to convert the voltage of the battery as a power supply into a voltage required for each circuit of the imaging device 1 and supply the voltage.

FIG. 3 is a block diagram showing a schematic configuration of the motion vector detection circuit 23.
The motion vector detection circuit 23 includes an imaging data acquisition unit 51, a Y / C separation unit 52, a frame memory 53 having an evaluation frame 53a and a reference frame 53b, an evaluation region setting unit 56, a reference region setting unit 57, An inter-frame correlation calculation unit 59, a reliability determination unit 60, a motion vector detection unit 61, a motion vector determination unit 64, and a determination unit 65 are configured.

  The imaging data acquisition unit 51 is a means for acquiring imaging data from the imaging circuit 13 in order to acquire the evaluation frame 53a and the reference frame 53b. The evaluation frame 53a is data imaged at a predetermined time point, and the reference frame 53b is data imaged a predetermined time before the predetermined time point.

Next, the operation of the motion vector detection circuit 23 will be described.
A luminance component is extracted from the data output from the imaging data acquisition unit 51 by the Y / C separation unit 52. The extracted luminance component data is stored in the frame memory 53 as an evaluation frame 53a and a reference frame 53b in accordance with a shooting operation in a self-shooting mode described later.
The evaluation area setting unit 56 is an evaluation area setting unit for setting a predetermined area of the evaluation frame 53a obtained from the imaging data acquisition unit 51 as an evaluation area. Similarly, the reference area setting unit 57 is a reference area setting unit for setting a reference area corresponding to the evaluation area in the reference frame 53b obtained from the imaging data acquisition unit.

FIG. 4 is a diagram for explaining a method of setting an evaluation area.
FIG. 4A shows an example in which a reference area is set for the reference frame 53b. The reference frame 53b is divided into 3 × 3 squares, and each square is set as a reference area.
FIG. 4B shows an example in which an evaluation area is set for the evaluation frame 53a. A small area at the center of each set reference area is set as an evaluation area.
The inter-frame correlation calculation unit 59 as a correlation calculation unit between the evaluation frame 53a and the reference frame 53b includes an evaluation region set by the evaluation region setting unit 56 and a reference region set by the reference region setting unit 57. The correlation coefficient between the evaluation region and the reference region is calculated for each positional relationship while shifting and changing the relative positional relationship between the evaluation region and the reference region.

FIG. 5 is a diagram illustrating an example of shifting the positional relationship between the evaluation area and the reference area.
For example, the evaluation area is shifted rightward by one pixel from the upper left position of the reference area. When the shift is made to the right end position, the pixel is shifted to the right end position in the right direction by one pixel from the left end position of the reference area with the evaluation area lowered by one pixel. This operation is performed until the evaluation area reaches the lower right position of the reference area.

Then, the interframe correlation calculation unit 59 calculates a correlation coefficient at each shift position.
Here, the correlation coefficient is a value obtained by integrating the absolute value of the difference between corresponding pixel data in the evaluation region and the reference region in the region. Therefore, when the correlation coefficient is relatively small, it indicates that the degree of correlation between the evaluation region and the reference region is relatively high.

  In accordance with the correlation coefficient calculated by the inter-frame correlation calculation unit 59, a motion vector detection unit 61, which is a motion vector detection means, obtains a motion vector for each evaluation area (block) from the positional relationship having the highest degree of correlation.

  For example, as shown in FIG. 5, when the correlation coefficient when the evaluation area is placed at the upper left end position of the reference area is the lowest value, the motion vector of this evaluation area is a vector A. Here, the vector A is a vector having the start point as the center point of the reference area and the end point as the center point of the evaluation area at the upper left position of the reference area.

  At this time, the reliability of the detection result of the motion vector detection unit 61 is determined by the reliability determination unit 60 which is a reliability determination unit. For example, when the lowest correlation coefficient obtained by the above-described calculation is a value higher than a predetermined level, the reliability determination unit 60 determines that the detection result of the motion vector detection unit 61 is “no reliability”. .

The motion vector determination unit 64 determines a motion vector of the entire evaluation frame 53a from a plurality of motion vectors in units of evaluation regions. In this case, the determination unit 65 determines whether the motion vector of the entire evaluation frame 53a is reliable.
When the determination unit 65 determines that the motion vector of the entire evaluation frame 53a is reliable, the composition is determined in advance according to the motion vector determined by the motion vector determination unit 64. The direction of moving the imaging device is instructed to the photographer. This instruction
This is done by outputting to the speaker 40 or the direction display unit 41 via the system controller 15.

  Then, when predetermined conditions are satisfied thereafter, the image data from the imaging data acquisition unit 51 is sent to the image processing circuit 24.

  Note that the basic configuration of the above-described motion vector detection circuit 23 is an application of the electronic camera shake correction difference matching circuit described in JP-A-2005-260481, for example.

FIG. 6 is a schematic flowchart showing a self-portrait operation of the imaging apparatus according to the present embodiment.
In step S01, the photographer operates the operation unit 19 to set the photographing mode to “selfie mode”. In step S02, the imaging apparatus 1 stands by until the release switch 42 is pressed.

  In step S03, the photographer performs reference photographing. That is, the photographer operates the release switch 42 with a background composition. In step S04, the image data from the imaging circuit 13 is stored in the reference frame 53b.

  In step S05, the photographer holds the imaging device 1 in his hand and points the lens in his / her direction with the subject photographed in the standard photographing as a background. The imaging apparatus 1 shifts to a standby state for actual shooting. That is, the operations from step S06 to step S11 are executed.

  In step S06, image data including the background and the photographer as subjects is extracted from the imaging circuit 13 and stored in the evaluation frame 53a. In step S07, a motion vector is obtained by the processing described with reference to FIGS. 4 and 5, and a framing shift amount is obtained by a correlation matching method.

FIG. 7 is a diagram for explaining processing for obtaining a framing shift amount by a correlation matching method.
FIG. 7 (1) shows a background image obtained by reference photographing, and FIG. 7 (2) shows an image stored in the evaluation frame 53a. FIG. 7 (3) shows the result of the correlation matching process.

  A reference region is set for the background image obtained by the standard imaging, and a motion vector is obtained for each evaluation region by performing a correlation calculation between the reference region and the evaluation region set corresponding to the reference region. The right side of FIG. 7 (3) shows the motion vector for each evaluation area obtained in this way.

  Here, in the evaluation region including the photographer's face, the correlation coefficient with the background image has a high value. For this reason, as described above, the reliability determination unit 60 determines that the motion vector of this evaluation region is “no reliability”. Therefore, the motion vector of the evaluation area determined as “unreliable” is excluded, and the motion vector of the entire frame is obtained from the remaining plurality of motion vectors. For example, an average value of motion vectors for each evaluation area may be obtained, or a median value may be obtained.

In step S08, it is checked using the threshold value matrix whether or not the entire motion vector is equal to or less than the threshold value.
FIG. 8 is a diagram illustrating a threshold matrix.
In the threshold matrix, a two-dimensional coordinate system having the origin at the center position of the reference area is defined and divided into nine areas (area [1] to area [9]). Here, if the positive threshold value in the X-axis direction is Tx and the positive threshold value in the Y-axis direction is Ty, the region [5] is a region surrounded by −Ty ≦ Y ≦ Ty and −Tx ≦ X ≦ Tx. . This area [5] is referred to as an area below the threshold. Then, on this threshold value matrix, when the start point of the entire motion vector is set as the coordinate origin, it is examined to which region the end point position of the vector belongs.

  If NO in step S08, that is, if the end point position of the motion vector is other than the region [5], in step S09, the imaging apparatus gives an instruction to the photographer based on the instruction matrix.

FIG. 9 is a diagram showing an instruction matrix.
In this instruction matrix, contents instructed to the photographer are described corresponding to the area to which the end point position of the vector belongs. For example, when the end point position of the vector belongs to the area [1], an instruction “shift to the lower right” is issued. This instruction may be given by voice through the speaker 40, or the direction in which the image pickup apparatus is shifted may be lit on the direction display unit 41.

  Then, after a predetermined time has elapsed, the process returns to step S06, and a motion vector is obtained based on the acquired new subject image data.

If YES in step S08, that is, if the end point position of the motion vector is in the area [5], in step S10, the imaging apparatus blinks and displays the direction display unit 41 that the composition is within the predetermined position. To notify the photographer of the start of shooting. For example, “Shooting starts after x seconds” is notified by voice using the speaker 40.
Thereby, the photographer can stop and hold the movement of the image pickup apparatus, and can secure a preparation time for adjusting the facial expression for photographing.

  In step S <b> 11, the imaging apparatus performs the main photographing operation after x seconds have elapsed. That is, the imaging apparatus performs the main photographing operation when the end point position of the motion vector is still in the region [5] after x seconds have elapsed. Then, the imaging data acquisition unit 51 outputs the image data acquired in the actual imaging to the image processing circuit 24.

FIG. 10 is a diagram for explaining a function for assisting a framing operation for a photographer in self-portrait.
The photographer uses sound, light, etc. to detect a difference value indicating the degree of coincidence with the reference image photographed on the first image and a self-portrait image photographed on the second image, and to correct the difference value. To inform. If the difference value falls within the threshold value, this is immediately notified by light, and the start of shooting is notified by voice.

[Effect of the embodiment]
In the embodiment described above, various effects can be obtained.

(1) The imaging device has a self-portrait mode, takes a background image for the first time, and takes a self-portrait for the second time. When taking a self-portrait, the photographer is informed of the direction in which the imaging device is moved by sound or light so as to obtain a predetermined composition.
In addition, when the composition is determined, this is immediately notified to the photographer by light. As a result, the photographer can quickly stop the movement of the imaging device at that position. In addition, it is configured to notify the start of shooting so that preparation time for shooting can be secured. As described above, the imaging apparatus has a function for smoothly performing self-portrait.

  (2) In addition, the imaging apparatus automatically shifts to the imaging operation when the determined composition is obtained in the self-portrait operation. For this reason, the photographer does not need to operate the release switch, is easy to handle, and there is no fear of image quality deterioration such as blur due to the release operation.

  (3) Further, the imaging apparatus obtains a direction in which the imaging apparatus is moved by image processing with respect to a predetermined composition. In this image processing, a motion vector is used, but this basic processing method applies a difference matching circuit for electronic camera shake correction. Therefore, since this circuit can be shared, it is possible to configure an inexpensive image pickup apparatus equipped with both electronic camera shake correction and self-shooting functions.

  Each function described in the above embodiment may be configured using hardware, or may be realized by reading a program describing each function into a computer using software. Each function may be configured by appropriately selecting either software or hardware.

  Furthermore, each function can be realized by causing a computer to read a program stored in a recording medium (not shown). Here, as long as the recording medium in the present embodiment can record a program and can be read by a computer, the recording format may be any form.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

1 is a perspective view of an imaging apparatus according to a first embodiment of the present invention. 1 is a block diagram showing a schematic configuration of an imaging apparatus according to a first embodiment of the present invention. The block diagram which shows schematic structure of a motion vector detection circuit. The figure for demonstrating the method of setting an evaluation area | region. The figure which shows the example which shifts the positional relationship of an evaluation area | region and a reference area. FIG. 3 is a schematic flowchart showing a self-taking operation of the imaging apparatus. The figure explaining the process which calculates | requires the deviation | shift amount of a framing with a correlation matching method. The figure which shows a threshold value matrix. The figure which shows an instruction | indication matrix. The figure explaining the function which assists the framing operation | movement to the photographer in self-portrait.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... Imaging lens system, 11a ... Imaging lens, 12 ... Imaging element, 13 ... Imaging circuit, 15 ... System controller, 17 ... Lens drive part, 19 ... Operation part, 23 ... Motion vector detection circuit, 24 Image processing circuit, 25 Buffer memory, 26 Video memory, 27 Interface, 28 External interface, 30 Card slot, 31 Detachable memory, 33 Video output circuit, 35 Image display LCD, 40 Speaker DESCRIPTION OF SYMBOLS 41 ... Direction display part, 42 ... Release switch, 51 ... Imaging data acquisition part, 53 ... Frame memory, 53a ... Evaluation frame, 53b ... Reference frame, 56 ... Evaluation area setting part, 57 ... Reference area setting part, 59 ... Frame Inter-correlation calculation unit, 60 ... reliability determination unit, 61 ... motion vector detection unit, 64 ... motion vector determination unit, 5 ... the determination unit.

Claims (5)

An imaging unit for imaging a subject and converting it into image data;
A correlation calculation unit for detecting a degree of coincidence between the first image data obtained by imaging the background with the imaging unit and the second image data obtained by imaging the photographer under the background;
A determination unit for determining whether or not the degree of coincidence is within a predetermined range;
An imaging operation control unit that starts an imaging operation by the imaging unit when the determination unit determines that the degree of coincidence is within a predetermined range;
A recording unit that records the third image data obtained by operating the imaging operation control unit and starting the imaging operation by the imaging unit;
An imaging apparatus comprising:   2. The imaging apparatus according to claim 1, further comprising an output unit that notifies a photographer of a shooting direction so that the degree of coincidence falls within a predetermined range.   The imaging apparatus according to claim 2, wherein the output unit notifies the start of shooting when the degree of coincidence falls within a predetermined range.   The imaging apparatus according to claim 3, wherein the imaging apparatus automatically starts imaging after the degree of coincidence falls within a predetermined range. Imaging a background to obtain first image data;
Capturing a photographer under the background and obtaining second image data;
Performing a correlation operation for detecting the degree of coincidence between the first image data and the second image data;
Determining whether the degree of coincidence is within a predetermined range;
Performing an imaging operation when it is determined that the degree of coincidence is within a predetermined range;
Recording the third image data obtained by performing the imaging operation in a recording unit;
An imaging method comprising:

Images