JP2011130378A - Image recording apparatus and image recording method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve a probability of photographing an effective still image during still picture photographing to be performed under a motion picture photographing mode. <P>SOLUTION: An image recording apparatus is provided with: a face detecting means for detecting whether or not a face is included in an image; a face specific state detecting means for detecting a specific state in a face detection region detected by the face detecting means; a still image recording means having an automatic specific state recording function of automatically recording a still image when the specific state is detected by the face specific state detecting means during a photographing mode operation; a composition stability determining means for determining whether or not the composition of the image is stabilized; and a control means for changing a condition, required for implementing the automatic specific state recording function, based on a result of the determination by the composition stability determining means between the still picture photographing mode and the motion picture photographing mode. Thus, trouble is prevented that a still image may be recorded by the automatic recording function during panning or zooming of motion picture photographing under the motion picture mode. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image recording apparatus and an image recording method, and more particularly to a technique suitable for use in capturing still images during moving image shooting.

  In recent years, image processing technology for performing digital arithmetic processing based on an image signal has been rapidly progressing. For example, there is a technology for detecting a human face from an image. For example, a luminance difference between two pixels in the face image is learned as a feature amount. Then, an estimated value indicating whether or not the predetermined area in the input image is a face is calculated based on the feature amount, and finally whether or not the image in the area is a face is calculated from the one or more estimated values. A face detection technique that makes it possible to discriminate between the two has been put into practical use.

  Such face detection technology has been increasingly installed in digital imaging devices such as digital still cameras that capture images with a solid-state imaging device. Recently, technology for discriminating detected facial expressions has also been realized. ing.

There are image pickup apparatuses that apply such a technique and have a function of shooting a still image at the moment when a person who is a subject smiles. Furthermore, in recent years, a moving image recording apparatus has been developed that can detect a smiling face even during moving image shooting and record a still image of the smiling face during moving image shooting.
Further, as a method for automatically capturing an image, a method for performing automatic recording by voice analysis has been proposed (for example, Patent Document 1).

JP 2008-219450 A

  When recording a still image using the automatic recording function in the still image mode as described above, it is possible for the user to communicate the intention of shooting to the subject after taking an appropriate composition. Therefore, the automatic shooting function also operates at a relatively intended timing. For this reason, in the still image mode, there is a high possibility that the best shot can be taken.

  On the other hand, when still images are automatically recorded during moving image shooting in the conventional technology, still images may be recorded by the automatic recording function during panning or zooming of movie shooting in the movie mode, which is not necessarily in a good state. However, there is a problem that still images cannot be recorded.

Furthermore, when recording a moving image and a still image at the same time, the recording band on the recording medium is preferentially assigned to the moving image recording, so that the recording time of the still image tends to be extended. As a result, the interval at which a still image can be taken is extended, and there is a problem that the probability of missing the best shot increases. Note that the simultaneous recording of moving images and still images here refers to an operation of interrupting moving image recording as necessary during moving image recording, switching to still image recording, and returning to moving image recording again.
In view of the above-described problems, an object of the present invention is to improve the probability of capturing an effective still image in still image shooting performed in the moving image shooting mode.

  An image recording apparatus of the present invention is an image recording apparatus having a still image shooting mode for recording still images and a moving image shooting mode for recording moving images, and having a function of recording still images even in the moving image shooting mode. A face detection unit that detects whether or not a face is included in the image, a face specific state detection unit that detects a specific state in the face detection area detected by the face detection unit, the still image shooting mode, and the Still image recording means having a specific state automatic recording function for automatically recording a still image when a specific state is detected by the face specific state detection means during operation in the moving image shooting mode, and stable image composition A composition stability determining means for determining whether or not the image is recorded and a control means for changing a condition for executing the specific state automatic recording function between the still image shooting mode and the moving image shooting mode. The control unit controls the operation of the still image recording unit to execute the specific state automatic recording function only in the moving image shooting mode, on the condition that the determination result by the composition stability determination unit is stable. Alternatively, in the moving image shooting mode, the threshold value for determining that the composition stability determining means determines that the image composition is stable is set stricter than in the still image shooting mode.

  According to the present invention, a still image with a stable composition can be taken even in still image recording using an automatic recording function during moving image shooting.

It is a block diagram which shows 1st Embodiment and shows the structural example of a video camera. It is a flowchart explaining the 1st example of a smile imaging | photography process procedure. It is a figure explaining the mode of a smile photographing process. It is a flowchart explaining the 2nd example of a smile imaging | photography process procedure. It is a flowchart explaining a composition stabilization process procedure. It is a characteristic view explaining the example of determination of whether a video camera is panning.

(First embodiment)
Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
As an embodiment of the present invention, a video camera 100 as an image recording apparatus having a function of simultaneously recording still images even in the moving image shooting mode will be described with reference to FIG. As described above, simultaneous recording refers to an operation of interrupting shooting of a moving image, switching to shooting of a still image as necessary, and returning to shooting of a moving image while shooting a moving image.
In FIG. 1, a control unit 101 controls the operation of the entire video camera. It also accepts user camera operations such as still image recording timing instructions by pressing the shutter button. Various operation instructions are given to each block to be described later depending on the operation content.

  The lens 102 can change the focal length and the state of the aperture. The image sensor 103 is an image sensor such as a charge coupled device (CCD), for example, and has a function of converting light imaged by the lens 102 into an electric signal.

  The signal processing unit 104 performs predetermined signal processing such as white balance and gamma correction on the input moving image signal, and temporarily records it as image data in a memory 114 described later. H. The H.264 encoding / decoding unit 105 is an H.264 encoder. The moving image data temporarily recorded in the memory 114 is converted into the H.264 format by the H.264 method. H.264 or H.264 It has a function of decoding moving image data from H.264 stream data.

  The JPEG encoding / decoding unit 106 has a function of encoding still image data temporarily recorded in the memory 114 into JPEG or decoding still image data from the JPEG data and writing it back to the memory 114.

  The microphone 108 has a function of converting sound into an audio signal. The speaker 109 has a function of converting an audio signal into sound, contrary to the microphone 108. The audio signal (Audio) encoding / decoding unit 110 has a function of encoding an audio signal input from the microphone 108 into a predetermined code format and temporarily recording it in the memory 114. Further, it has a function of decoding an audio signal from predetermined encoded data held in the memory 114. As a code format, AC3 (Audio Code Number 3) is used in this embodiment.

  The disk control unit 111 controls the operation of the hard disk drive 112 that is a recording medium. In the present embodiment, an ATA (AT Attachment) controller is used as the disk control unit 111. The hard disk drive 112 corresponds to ATA, and is used for data communication between the controller that handles communication, recording, and reading of ATA commands and data with the disk control unit 111 and the magnetic recording disk and control unit 101. It is mainly composed of buffer memory.

  Stored in the memory 114. H.264 data and audio data are multiplexed and recorded as a stream data file on the disc. On the contrary, the stream data recorded in the hard disk drive 112 is read, and H.264 is read from the read data. H.264 data and audio data are separated and written in a predetermined area of the memory 114. Similarly, JPEG data and management data files can be recorded on the disc.

  The flash memory 113 is used for storing a program operated by the control unit 101, storing various setting data, and the like. Since the data recorded in the flash memory 113 is retained even when the power of the video camera 100 of the present embodiment is turned off, it is also used for retaining management data and the like that need to be retained even after the power is turned off. .

  As described above, the memory 114 stores the H.264 memory. H.264 data buffer, audio data buffer, work memory, etc. The display unit 115 also has a function as a display device for using the decoded video data as a display device and a viewfinder at the time of shooting. Further, it also has a function as an operation screen when the user operates this apparatus.

  The data bus 116 is used for access from the control unit 101 to each functional block, data transfer between the functional blocks via the memory 114, and the like. The acceleration sensor 117 detects acceleration applied in three orthogonal directions. With this acceleration sensor 117, the moving direction of the video camera 100 can also be calculated.

The face detection unit 118 has a function of determining whether or not a person's face area exists in the captured image temporarily recorded in the memory 114 by the signal processing unit 104. As the determination result by the face detection unit 118, parameters such as the number of faces and the position (coordinates), size, rotation angle, and reliability of each face are output.
The user operation unit 119 performs various settings of the video camera 100 in accordance with a user operation, and performs settings for enabling or disabling a specific state automatic recording function described later.

  The video camera 100 of this embodiment has a smile detection function. That is, the face detection unit 118 determines whether the face state is “smile” by the software processing of the control unit 101 for the region determined to be a human face. Furthermore, the smile detection function is used to automatically record a still image at the moment of smile by using the above-mentioned smile detection function at the still image recording timing.

Next, the operation mode of the video camera 100 of this embodiment will be described.
The video camera 100 of the present embodiment has a moving image mode and a still image mode in recording mode and playback mode, respectively, and has a total of four operation modes. In the recording mode, the effective area of the image sensor 103, the number of read pixels, the angle of view, signal processing, and the like are switched according to the moving image mode and the still image mode. Even in the moving image mode, a still image can be recorded by the simultaneous recording function. In the playback mode, a list of moving images and still images is displayed according to the moving image mode and the still image mode, respectively, and the user can play back by selecting an arbitrary image.

  When the above-described smile shooting function is used in the video camera 100 of the present embodiment, the smile shooting function is enabled in the main unit setting. In the case of invalidity, the still image is recorded as the user presses the shutter as usual. When the smile shooting function is enabled, automatic recording of still images is performed according to the following smile shooting processing.

Next, the smile photographing process will be described.
In the video camera 100 of the present embodiment, the smile shooting process is changed between simultaneous recording of a moving image still image in the moving image shooting mode and still image shooting recording in the still image shooting mode. A flowchart of each smile photographing process is shown in FIGS. Moreover, the execution timing of each procedure in the smile photographing process is shown in FIG.

FIG. 2A is a flowchart illustrating a processing procedure when smile shooting is enabled in the moving image shooting mode.
In S201, when shooting of a moving image is started with moving image mode smile shooting enabled, in S202, a face detection determination is performed to detect whether or not a face is included in the moving image being shot. If the result of this determination is that no face has been detected, the system waits until face detection is performed. If a face is detected, the process proceeds to S203.

  In S203, it is determined whether the composition is stable. If the result of this determination is that the composition is unstable, the process returns to S202. If the composition is stable as a result of the determination in S203, the process proceeds to S204.

  In S204, it is determined whether a smile is detected. If no smile is detected as a result of this determination, the process returns to S202. If a smile is detected as a result of the determination in S204, the process proceeds to S205, where the moving image shooting is interrupted and still image shooting is performed. Next, proceeding to S206, the still image recorded in S205 is recorded as a JPEG file on the hard disk drive 112. Thereafter, the process returns to moving image shooting, returns to S202, and repeats the processing described above. As described above, in the moving image mode smile shooting, there are three conditions for executing the specific state automatic recording function: “face detection”, “composition stabilization”, and “smile detection”.

  In the case of the video camera 100, in the video shooting mode, face detection is always performed from the face detection image 301 so that autofocus (AF) and automatic exposure (AE) can be preferentially applied to the area where the face is detected. Is going. When a face is detected from the face detection image 301, a face detection area 302 is generated. Next, it is determined whether or not the composition of the face detection image 301 is stable, and a composition stable image 303 is obtained.

Here, the composition stability determination process for obtaining the composition stable image 303 will be described with reference to FIG.
When the composition stability determination process is started in S501, in S502, the control unit 101 confirms whether the input zoom operation of the user and other various camera operations have been performed for the past one second or more.

  As a result of the confirmation, if the camera operation has not been performed for the past one second or more, the process proceeds to S503 to confirm whether panning is in progress. As a result of this confirmation, it is stable when panning is not in progress. That is, when it is determined that there is no user operation for one second and panning is not being performed, the composition stability determination process determines <stable> (S504).

  On the other hand, if it is determined in S502 that the camera operation has been performed within the past one second, or if it is determined that panning is being performed in S503, the process proceeds to S505 and <unstable> is determined. In S503, whether or not panning is in progress is determined by integrating the output value of the acceleration sensor 117 (integration of output values every 1/30 seconds) and obtaining the speed.

Next, determination of whether or not the video camera 100 is panning will be described with reference to FIG.
The characteristic diagram shown in FIG. 6A shows the output value history of the acceleration sensor 117 (here, the acceleration direction is represented by one axis for convenience). A positive acceleration is detected between time t0 and t2 (reference numeral 601), and a negative acceleration is detected between time t3 and t5 (reference numeral 602), which is the generation of acceleration due to the start and end of panning, respectively. .

  The acceleration output value is integrated by the control unit 101 and converted into a speed, and the characteristic diagram of FIG. The speed increases from the time of acceleration occurrence at time t0, maintains the speed for a while (reference numeral 603), then decelerates from time t3, and becomes speed 0 (still) at t5. In the meantime, there is a period in which the speed threshold value Vt is exceeded from time t1 to t4, and during this period, it can be determined that panning is in progress. Such composition stability determination is a special process performed only when the smile shooting function is enabled in the moving image shooting mode.

  When it is determined that the composition is stable, the composition stable image 303 is secured, and as described above, the control unit 101 detects whether the face existing in the face detection area 302 is a smile in S204. When a smile detection image 304 is detected in the face detection area 302 by this detection, in S205, the moving image shooting is interrupted and a still image is shot (symbol 305) as smile shooting.

  The captured still image is converted into JPEG data by the JPEG encoding / decoding unit 106 in S206 described above, and recorded as a JPEG file on the hard disk drive 112 in parallel with the recording of the moving image data. When the file recording is completed, the process returns to the process of S202 to return to the moving image shooting, and the smile shooting processing is resumed.

  As described above, in the smile shooting process in the moving image shooting mode, a camera operation by the user, panning, or the like can always occur. Therefore, as in the present embodiment, by adding the composition stability determination process in S203, it is possible to record better moving images and still images simultaneously.

FIG. 2B is a flowchart for explaining a processing example when the smile photographing function is enabled in the still image photographing mode.
In the present embodiment, when the smile shot is enabled in S207, face detection is always performed from continuously generated images (S208), as in the moving image mode. Then, when a face is detected in S208, the process proceeds to S209, and smile detection processing is immediately performed on that area (face detection area 302 in FIG. 3).

  If a smile is detected as a result of the smile detection process in S209, the process proceeds to S210, and a still image is shot as smile shooting. Next, in S211, the captured still image is converted into JPEG data by the JPEG encoding / decoding unit 106, and still image recording is performed in which it is recorded as a JPEG file on the hard disk drive 112. When the file recording is completed, the process returns to S208, and the smile photographing process is resumed. As described above, in the still image mode smile shooting, there are two conditions for executing the specific state automatic recording function: “face detection” and “smile detection”.

  As described above, the smile shooting process in the still image shooting mode is only for still image recording, and since the smile shooting is enabled as in movie shooting, panning and the like are unlikely to occur, so the composition is stable. The determination process is omitted and shooting is performed. In addition, since recording on the hard disk drive 112 (HDD) is only a still image, the recording interval can be shortened, and effective smile shooting can be expected by repeatedly performing smile shooting.

  In this embodiment, the user operation input or the output value of the acceleration sensor is used as the composition stability determination. However, the use of a motion vector of the face area detected by the signal processing unit 104 is also conceivable. For example, the motion vectors of the face area are integrated every 1/30 seconds, and if the integrated value for one second is equal to or less than a predetermined value, it is determined that the face position is stable, and the image is also used as the composition stable image 303. Good.

In the present embodiment, as shown in FIG. 3A, processing is performed in the order of acquisition of the composition stable image 303 → acquisition of the smile detection image 304. However, as shown in FIG. 3B, these determinations may be performed simultaneously, and a still image may be recorded when both determinations are Yes.
As described above, by switching the smile shooting process between the moving image shooting mode and the still image shooting mode, effective smile shooting recording can be performed in either mode.

(Second Embodiment)
In the present embodiment, an example is described in which the same configuration as the video camera of the first embodiment is used, but the smile shooting process in the still image mode is changed. About others, it is the same as that of 1st Embodiment, and abbreviate | omits description.
FIG. 4 shows a flowchart of the smile photographing process of the present embodiment. Since the smile photographing process in the moving image mode is the same as that in the first embodiment (FIG. 2A), the description of the processing procedure is omitted.

  In smile photography in the still image mode of the present embodiment, smile photography starts when the user fully presses the shutter button. In this way, compared to the first embodiment, the user can explicitly specify the start of smile shooting, and thus recording with a more intentional composition is possible.

FIG. 4 shows processing when smile shooting is enabled in the still image shooting mode. Unlike the first embodiment, face detection is not always performed in this embodiment.
In S401, smile shooting is enabled, and in S402, the user fully presses the shutter to advance to S403 and start face detection. That is, the “shutter full press determination” performed in S402 is a process of detecting the presence or absence of a “still image recording instruction” by the user.

  As a result of the determination in S403, if a face is detected, the process proceeds to S404, where the control unit 101 detects a smile and detects the face detection area 302 described above. Next, in S404, it is determined whether or not the face detection area 302 detected by the face detection process in S403 is a smile.

  As a result of the determination in S404, if it is determined that the face in the face detection area 302 is a smile, the process proceeds to S405, and a still image is shot as smile shooting. As described above, the face detection and smile detection processes are repeated until still image shooting is performed. After smiling is detected in S404 and still image recording is performed in S406, the process returns to S402 and waits until the user presses the shutter again. Therefore, in this embodiment, since face detection and smile detection are not performed until the user fully presses the shutter, there is an effect of suppressing power consumption.

  In the embodiment described above, the example of smile detection is shown as an example of detecting the face specific state during the shooting mode operation. However, other specific facial expressions are detected, or states other than facial expressions are detected. It doesn't matter. For example, in addition to this example, a serious face or an eye-opening may be detected, or it may be detected that the face is facing the front. In the above-described embodiment, the composition stability determination is performed only when the face identification state is detected in the moving image shooting mode. However, the present invention is not limited to this. In both movie shooting mode and still image shooting mode, composition stability is determined when a face-specific state is detected, and the threshold for determining that the composition is stable in movie shooting mode is compared to that in still image shooting mode. However, it may be set strictly.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program.

101 control unit, 102 lens, 103 image sensor, 104 signal processing unit, 105 H.P. H.264 encoding / decoding unit, 106 JPEG encoding / decoding unit, 108 microphone, 109 speaker, 110 audio signal encoding / decoding unit, 111 disk control unit, 112 hard disk drive, 113 flash memory, 114 memory, 115 display unit, 116 data bus, 117 acceleration sensor, 118 face detection unit, 119 user operation unit, 302 face detection area

Claims (8)

An image recording apparatus having a still image shooting mode for recording still images and a moving image shooting mode for recording moving images, and having a function of recording still images even in the moving image shooting mode,
Face detection means for detecting whether a face is included in the image;
A face specific state detecting means for detecting a specific state in the face detection area detected by the face detecting means;
Still image recording means having a specific state automatic recording function for automatically recording a still image when a specific state is detected by the face specific state detection means during operation in the still image shooting mode and the moving image shooting mode. When,
Composition stability determination means for determining whether the composition of the image is stable;
Control means for changing a condition for executing the specific state automatic recording function between the still image shooting mode and the moving image shooting mode;
The control means controls the operation of the still image recording means so as to execute the specific state automatic recording function on the condition that the determination result by the composition stability determination means is stable only in the moving image shooting mode. An image recording apparatus. An image recording apparatus having a still image shooting mode for recording still images and a moving image shooting mode for recording moving images, and having a function of recording still images even in the moving image shooting mode,
Face detection means for detecting whether a face is included in the image;
A face specific state detecting means for detecting a specific state in the face detection area detected by the face detecting means;
Still image recording means having a specific state automatic recording function for automatically recording a still image when a specific state is detected by the face specific state detection means during operation in the still image shooting mode and the moving image shooting mode. When,
Composition stability determination means for determining whether the composition of the image is stable;
Control means for changing a condition for executing the specific state automatic recording function between the still image shooting mode and the moving image shooting mode;
The image recording apparatus characterized in that the control means sets a threshold value for determining that the composition of the image is stable in the moving image shooting mode as compared with the still image shooting mode. . Recording timing instruction means for instructing the recording timing of the still image;
The control means controls the operation of the still image recording means to execute the specific state automatic recording function only in the still image shooting mode on the condition that the recording instruction is given by the recording timing instruction means. The image recording apparatus according to claim 1, wherein:   The image according to any one of claims 1 to 3, wherein the face specific state detection means detects a specific facial expression or a face orientation as the specific state in the face detection region. Recording device. An image recording method having a still image shooting mode for recording a still image and a moving image shooting mode for recording a moving image, and recording a still image even in the moving image shooting mode,
A face detection step for detecting whether a face is included in the image;
A face specific state detection step of detecting a specific state in the face detection region detected in the face detection step;
A still image recording step having a specific state automatic recording function that automatically records a still image when a specific state is detected in the face specific state detection step during the operation of the still image shooting mode and the moving image shooting mode. When,
A composition stability determination step for determining whether the composition of the image is stable;
A control step of changing a condition for executing the specific state automatic recording function between the still image shooting mode and the moving image shooting mode;
In the control step, only in the moving image shooting mode, the operation of the still image recording step is controlled to execute the specific state automatic recording function on the condition that the determination result in the composition stability determination step is stable. An image recording method. An image recording method having a still image shooting mode for recording a still image and a moving image shooting mode for recording a moving image, and recording a still image even in the moving image shooting mode,
A face detection step for detecting whether a face is included in the image;
A face specific state detection step of detecting a specific state in the face detection region detected in the face detection step;
A still image recording step having a specific state automatic recording function that automatically records a still image when a specific state is detected in the face specific state detection step during the operation of the still image shooting mode and the moving image shooting mode. When,
A composition stability determination step for determining whether the composition of the image is stable;
A control step of changing a condition for executing the specific state automatic recording function between the still image shooting mode and the moving image shooting mode;
In the control step, in the moving image shooting mode, an image recording method is characterized in that a threshold value for determining that the composition of the image is stable in the composition stability determination step is set stricter than in the still image shooting mode. .   A program that causes a computer to execute each step of the image recording method according to claim 5.   A computer-readable storage medium storing the program according to claim 7.

Images