JP5854861B2 - Imaging device, control method thereof, and control program - Google Patents

Description

  The present invention relates to an imaging apparatus that automatically performs imaging in accordance with the movement of a subject, a control method thereof, and a control program.

  In an imaging apparatus such as a digital camera, it is known that a subject (user) who is a subject shoots a desired image, so that a predetermined motion is detected in the subject and still image shooting is performed. Yes.

  For example, there is one in which images of both eyes of a subject are continuously detected based on a moving image captured by an image sensor, and a wink that is a predetermined movement is detected according to the detection result. Here, when a wink is detected, the shutter is released to capture a still image (see Patent Document 1).

  In addition to still image shooting triggered by detection of a predetermined motion, for example, a predetermined motion such as a wink is detected by image processing, and various systems are remotely operated according to the detection result. There are also things.

JP 2008-72183 A

  By the way, in Japanese Patent Application Laid-Open No. 2004-133620, when a subject that is a subject is photographed remotely, a still image is photographed at a timing desired by the subject. The person's photography is executed.

  For this reason, a wink that is a predetermined movement cannot be performed. For example, if both eyes are meditated to wink, it is not determined to be a wink (for example, it is determined that the eyes are meditated). Therefore, if the person who is difficult or unable to wink (a person to be photographed), an instruction cannot be given to the imaging apparatus by the action of an eye called wink. Here, it is conceivable that the imaging apparatus is configured not only to meditate one eye as in the case of wink but also to medicate both eyes. Another problem arises that would be to give instructions.

  Therefore, an object of the present invention is to provide an imaging apparatus, a control method therefor, and a control program that can give a shooting instruction by opening and closing an eye even for a person who is not good at winking.

In order to achieve the above object, an imaging apparatus according to the present invention is an imaging apparatus that captures an image of a subject and records image data, and the subject is one based on a plurality of image data sequentially read in a shooting standby state. Determination means for determining that the subject has meditated , and that the subject has closed both eyes, and imaging means for photographing the subject when the subject is in the first state or the second state; The first state is a state in which the subject is in a state where one eye is meditated for a predetermined first determination time or longer, and the second state is a state in which the subject is in both eyes. The meditated state is a state of continuing for a second determination time longer than the first determination time .

A control method according to the present invention is a control method for an imaging apparatus that captures an image of a subject and records the image data, and the subject relaxes one eye based on a plurality of image data sequentially read in a shooting standby state . And a determination step of determining that the subject has meditated on both eyes, and a photographing step of photographing the subject when the subject is in the first state or the second state, The first state is a state in which the subject is meditated on one eye for a predetermined first determination time or longer, and the second state is a state in which the subject is meditated on both eyes. It is the state which continues more than 2nd determination time longer than 1st determination time, It is characterized by the above-mentioned.

A control program according to the present invention is a control program used in an imaging apparatus that images a subject and records image data, and is based on a plurality of image data sequentially read out in a shooting standby state by a computer included in the imaging apparatus. A step of determining that the subject has meditated one eye and the subject has meditated both eyes; and when the subject is in the first state or the second state, The first state is a state in which the subject is in a state where one eye is meditated for a predetermined first determination time or longer, and the second state is The state where the subject is in a state where both eyes are meditated is a state in which the subject continues for a second determination time longer than the first determination time .

  According to the present invention, even a person who is not good at a predetermined movement, such as wink, can take a picture at a desired timing if the predetermined determination condition is satisfied even in a state where both eyes are meditated.

It is a block diagram which shows the structure of the digital camera which is an example of the imaging device by this invention. 3 is a flowchart for explaining a photographing operation in the digital camera shown in FIG. 1. It is a figure for demonstrating determination of the eye area | region performed by the system control circuit shown in FIG.

  Hereinafter, an imaging device according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a digital camera which is an example of an imaging apparatus according to the present invention.

  In FIG. 1, a digital camera 100 has a photographic lens 10 and a mechanical shutter 12 having a diaphragm function. An optical image of a subject or the like is formed on the image sensor 14 via the taking lens 10 and the shutter 12. The image sensor 14 outputs an electrical signal (analog signal) corresponding to the optical image. This analog signal is converted into a digital signal (image data) by the A / D converter 16.

  A clock signal and a control signal are supplied from the timing generation circuit 18 to the image sensor 14 and the A / D converter 16, and the image sensor 14 and the A / D converter 16 operate in accordance with the clock signal and the control signal. The timing generation circuit 18 is controlled by a memory control circuit 22 that operates under the control of the system control circuit 50.

  If the reset timing of the image sensor 14 is controlled by the timing generation circuit 18, the charge accumulation time of the image sensor 14 can be controlled. That is, a so-called electronic shutter is realized by reset timing control by the timing generation circuit 18.

  The image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the image data supplied from the A / D converter 16 or the image data supplied from the memory control circuit 22. The image processing circuit 20 performs an electronic zoom function by cutting out and scaling the image from the image data.

  Further, the image processing circuit 20 performs predetermined arithmetic processing according to the image data obtained as a result of imaging. Then, the system control circuit 50 controls the exposure control unit 40 and the distance measurement control unit 42 based on the calculation result. Thereby, the exposure control unit 40 and the distance measurement control unit 42 perform AF (automatic focus) processing, AE (automatic exposure) processing, and EF processing by a so-called TTL (through the lens) method.

  In addition, the image processing circuit 20 performs a TTL AWB (Auto White Balance) process based on a calculation result obtained by performing a predetermined calculation process according to image data obtained as a result of imaging.

  The memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the memory 30, and the compression / decompression circuit 32. The image data output from the A / D converter 16 is written into the memory 30 via the image processing circuit 20 and the memory control circuit 22 or directly via the memory control circuit 22.

  The image display unit 28 is composed of, for example, a TFT LCD, and the image data written in the memory 30 is displayed on the image display unit 28 as an image by the memory control circuit 22. If images are sequentially displayed on the image display unit 28, it can be used as an electronic viewfinder function.

  Further, the display of the image display unit 28 is turned on or off under the control of the system control circuit 50. If the display is turned off, power consumption in the imaging apparatus 100 can be significantly reduced.

  As described above, the image data obtained as a result of shooting is stored in the memory 30 (this image data includes still image data and moving image data). The memory 30 has a sufficient storage capacity for storing a predetermined number of still image data and a predetermined time of moving image data.

  As a result, a large amount of image data can be written in the memory 30 at high speed even during so-called continuous shooting or panoramic shooting in which a plurality of still images are continuously shot. The memory 30 can also be used as a work area for the system control circuit 50.

  The nonvolatile memory 31 is, for example, a flash ROM, and a program code executed by the system control circuit 50 is stored in the nonvolatile memory 31. The system control circuit 50 sequentially reads out the program code from the nonvolatile memory 50 and executes it.

  The non-volatile memory 31 is set with an area for storing system information and an area for storing user setting information, and the system control circuit 50 reads various information and settings at the next startup.

  The compression / decompression circuit 32 compresses and decompresses image data by adaptive discrete cosine transform (ADCT) or the like. For example, the compression / decompression circuit 32 reads image data stored in the memory 30, performs compression processing or decompression processing, and writes the processed image data in the memory 30.

  The exposure control unit 40 controls the shutter 12 having a diaphragm function. The exposure control unit 40 also has a flash light control function that works in conjunction with the flash 48.

  The distance measurement control unit 42 controls the focusing of the taking lens 10. The zoom control unit 44 controls zooming of the taking lens 10. The flash 48 has an AF auxiliary light projecting function and a flash light control function.

  As described above, the exposure control unit 40 and the distance measurement control unit 42 are controlled by the TTL method, and the system control circuit 50 controls the exposure control unit 40 and the distance measurement control unit 42 based on the calculation result by the image processing circuit 20. To do.

  As shown in the figure, a mode dial switch 60, shutter switches 62 and 64, a display switch 66, an operation unit 70, and a zoom switch 72 are connected to the system control circuit 50.

  The mode dial switch 60 is used to switch and set function modes such as power-off, automatic shooting mode, shooting mode, panoramic shooting mode, moving image shooting mode, and HDR (dynamic range extended shooting) moving image shooting mode.

  The shutter switch 62 is turned on during the operation of a shutter button (not shown), and sends the first shutter switch signal SW1 to the system control circuit 50. As a result, the system control circuit 50 instructs the start of operations such as AF (autofocus) processing, AE (automatic exposure) processing, and AWB (auto white balance) processing.

  The shutter switch 64 is turned on when the operation of the shutter button is completed, and sends a second shutter switch signal SW2 to the system control circuit 50. Thus, for example, in the case of flash photography, the system control circuit 50 performs the EF (flash pre-emission) process, and then exposes the image sensor 14 for the exposure time determined by the AE process.

  In the case of flash photography, the flash 48 emits light during the exposure time, and when the exposure time ends, the exposure control unit 40 controls the shutter 12 under the control of the system control circuit 50 to block light. Exposure to the image sensor 14 is terminated.

  As described above, the analog signal read from the image sensor 14 is written as image data in the memory 30 via the A / D converter 16 and the memory control circuit 22 (reading process). The development processing using the calculation results in the image processing circuit 20 and the memory control circuit 22 and the compression processing in which the image data is read from the memory 30 and compressed by the compression / decompression circuit 32 are performed under the control of the system control circuit 50. Is called. Then, the compressed image data is written in a recording medium 200 described later (recording process).

  By the operation of the display switching switch 66, the display switching of the image display unit 28 is performed under the control of the system control circuit 50. When photographing is performed using the optical viewfinder 104 by this display switching, the power supply to the image display unit 28 is interrupted as described above to save power.

  The operation unit 70 includes, for example, various buttons, a touch panel, and a rotary dial. Although not shown, various buttons include a menu button, a set button, a macro button, a multi-screen playback page break button, a flash setting button, and a single shooting / continuous shooting / self-timer switching button.

  The zoom switch 72 is operated by the user, and the operation of the zoom switch 72 causes the system control circuit 50 to instruct a magnification change of the captured image. The zoom switch 72 has a tele switch that changes the imaging angle of view to the telephoto side and a wide switch that changes the wide angle side.

  By operating the zoom switch 72, the zoom control unit 44 changes the imaging field angle of the imaging lens 10 under the control of the system control circuit 50 and executes optical zoom. Further, by operating the zoom switch 72, an electronic zooming change of the imaging angle of view such as image clipping and pixel interpolation processing by the image processing circuit 20 is performed.

  The power source 86 includes, for example, a primary battery such as an alkaline battery, or a secondary battery such as a NiCd battery, a NiMH battery, or a Li ion battery, and an AC adapter.

  An interface (I / F) 90 is an interface between the recording medium 200 such as a memory card or a hard disk and the digital camera 100, and the I / F 90 is connected to the recording medium 200 by a connector 92.

  The recording medium 200 includes a recording unit 202 such as a semiconductor memory or a magnetic disk and an interface (I / F) 204, and the recording medium 200 is connected to the digital camera 100 by a connector 206.

  The communication unit 110 includes, for example, USB and IEEE 1394 functions, a LAN function, a wireless communication function, and the like. The communication unit 110 is connected to an external device (not shown) by the connector 112. Note that an antenna is connected to the connector 112 when the wireless communication function is used.

  FIG. 2 is a flowchart for explaining a photographing operation in the digital camera 100 shown in FIG. Here, a description will be given by taking as an example a process of taking a still image by detecting a wink.

  Here, the digital camera 100 is in a shooting standby state, and in this shooting standby state, a plurality of image signals (analog signals) are output from the image sensor 14 at a higher speed than a reading process at the time of still image shooting such as 60 fps. Are read sequentially. The image signal is A / D converted by the A / D converter 16 and subjected to image conversion or scaling processing in the image processing circuit 20 as described above.

  Thereafter, the memory control circuit 22 writes the image data into the memory 30. Further, the image data written in the memory 30 is displayed on the image display unit 28 for the purpose described above. In this way, the image data displayed on the image display unit 28 is called through image data. In the shooting standby state, the process from the reading process of the image sensor 14 to the display of an image is repeatedly performed.

  In the above-described shooting standby state, the system control circuit 50 performs face detection and main face determination (step S201). Here, the main face determination refers to a process of determining and specifying a first priority person when a plurality of faces are detected, for example. In the face detection, the system control circuit 50 detects a face area in the person area of the image indicated by the image data by using a well-known face detection process for the image data written in the memory 30.

  Known face detection methods include, for example, a method using a neural network, and a part having a characteristic shape such as an eye, nose, and mouth is searched from an image, and the similarity is equal to or higher than a predetermined threshold using template matching. If so, there is a method to consider it as a face. Further, as a face detection method, a method using statistical analysis by detecting image feature amounts such as a distance between both eyes and a face outline is also known. In order to improve face detection accuracy, it is desirable to perform detection processing by combining these methods. Japanese Patent Laid-Open No. 2002-251380 describes a method for detecting a face using wavelet transform and image feature amounts.

  In the main face determination, the size of the detected face, the distance from the screen center of the face, and the determination result of each item such as whether or not the pre-registered face and the detected face are the same person are used. Accordingly, each item is weighted. Then, the first priority face is determined as the main face according to the weighting.

  For example, for each face on the image, the larger the face size and the closer the face position is to the center of the screen, the larger the weighting coefficient (evaluation value) is, and the face with the highest evaluation value is the main face. Judge as. Further, the priority order of the person may be stored in advance in the nonvolatile memory 31 or the like as the priority order data, and the system control circuit 50 may determine the main face according to the priority order data.

  When specifying a person, template matching is performed on a part having a characteristic shape such as eyes, nose and mouth of a person image registered in advance in the nonvolatile memory 31 or the like.

  Subsequently, the system control circuit 50 determines whether the left and right eyes of the main face are in an eye-meditation state (step S202). In the eye meditation determination, the system control circuit 50 first determines a region for determining eye meditation. Here, the system control circuit 50 obtains the size of the region (eye region) to be determined according to the size of the face, using the center of the eye obtained at the time of face detection as a reference.

  FIG. 3 is a diagram for explaining the eye area determination performed by the system control circuit 50 shown in FIG.

  In FIG. 3, a frame 301 indicated by a dotted line is a frame for specifying a face (also referred to as a face area) detected by face detection. In the frame 301, the center of the double circle 302 indicates the center of the left eye in the face area. Then, the system control circuit 50 determines an eye region 303 (left eye region) indicated by a dotted line according to the size of the frame 301 with the center of the left eye as a reference. Similarly, the system control circuit 50 determines the right eye region according to the size of the frame 301 with reference to the center of the right eye. The system control circuit 50 sets each of the left eye region and the right eye region as a determination region for determining whether or not the eyes are meditated.

  Further, in the determination of eye meditation, the pupil region is extracted and specified for each of the left eye region and the right eye region (pupil region determination). For the extraction of the pupil region, a method using a neural network or template matching or the like is used as in face detection. Alternatively, an area having a color or luminance estimated as a pupil may be extracted as a pupil area.

  After determining the pupil region (that is, the left pupil region and the right pupil region), the system control circuit 50 obtains the ratio of the area of the pupil region to the area of the determination target region (that is, the left eye region and the right eye region) (hereinafter referred to as the pupil). Called the area ratio). Note that the ratio of the area of the left pupil region to the area of the left eye region is a first ratio, and the ratio of the area of the right pupil region to the area of the right eye region is a second ratio.

  Subsequently, the system control circuit 50 compares the preset eye meditation threshold level with the (first threshold level) pupil area ratio to determine whether the pupil area ratio is less than the eye meditation threshold level. (Pupil area ratio determination). This pupil region ratio determination is performed for each of the left eye region and the right eye region.

  If the pupil area ratio is less than the eye meditation threshold level (less than the first threshold level), the system control circuit 50 determines that the eye (left eye or right eye) is in a meditation state. On the other hand, if the pupil area ratio is equal to or higher than the eye meditation threshold level (first threshold level or higher), the system control circuit 50 determines that the eye (left eye or right eye) is in an open state.

  Next, the system control circuit 50 determines whether or not both eyes are in a relaxed state (step S203). Here, if both the left eye and the right eye are in a meditated state, the system control circuit 50 is in a state in which both eyes are meditated.

  If both eyes are open (YES in step S203), system control circuit 50 returns to the process in step S201.

  On the other hand, if both eyes are not open (NO in step S203), the system control circuit 50 determines whether or not both eyes are in a relaxed state (step S204).

  If the left eye or the right eye is in an open state, that is, if either the left eye or the right eye is in a relaxed state (NO in step S204), the system control circuit 50 determines the predetermined determination time Ta (first Is set (step S205).

  Subsequently, the system control circuit 50 determines whether or not the eye determined to have been meditated has been meditated for the determination time Ta (step S206). That is, the system control circuit 50 determines whether the state where one eye is meditated is a first state that continues for a predetermined first determination time or more.

  At the time of the determination in step S206, the system control circuit 50 performs eye meditation determination for each frame. If the frame is in the frame meditation state corresponding to the determination time Ta, the system control circuit 50 remains in the eye meditation state for the determination time Ta or longer. Suppose there was.

  If the eye-meditation state is less than determination time Ta (NO in step S206), system control circuit 50 returns to the process of step S201. On the other hand, if the eye-meditation state is equal to or longer than determination time Ta (YES in step S206), system control circuit 50 executes still image shooting (step S207).

  In still image shooting in step S207, the system control circuit 50 controls the timing generation circuit 18 via the memory control circuit 22 to read out a still image from the image sensor 14. The read still image (analog signal) is A / D converted by the A / D converter 16 and then subjected to image conversion by the image processing circuit 20.

  Thereafter, the image data is subjected to image compression in the compression / decompression circuit 32, and the compressed data is temporarily stored in the memory 30 via the memory control circuit 22. Then, under the control of the system control circuit 50, the memory control circuit 22 reads the compressed data stored in the memory 30 and stores it in the recording medium 200.

  If both eyes are in a meditated state (YES in step S204), the system control circuit 50 sets the above-mentioned eye meditating threshold level to the threshold level for both eyes (second threshold level) (step S208). This binocular threshold level is lower than the eye meditation threshold level. In other words, the system control circuit 50 tightens the criteria for determining eye meditation. As a result, the system control circuit 50 makes it difficult to determine that the eye is in a closed state when it is not in a complete closed state.

  Subsequently, the system control circuit 50 compares the pupil area ratio with the binocular threshold level for both the left and right eyes, and determines whether the pupil area ratio is less than the binocular threshold level (less than the second threshold level). (Step S209).

  If the pupil area ratio of both eyes is greater than or equal to the threshold level for both eyes (NO in step S209), that is, if it is not determined that the eyes are in a closed state, the system control circuit 50 returns to the process of step S201.

  If the pupil area ratio is less than the threshold level for both eyes (YES in step S209), that is, if it is confirmed that the eyes are in a closed state, the system control circuit 50 sets a determination time Tb (second determination time) ( Step S210). This determination time Tb is longer than the above determination time Ta.

  Subsequently, the system control circuit 50 proceeds to the process of step S206, and determines whether or not the eyes determined to have been meditated (that is, both eyes) have been meditated for the determination time Tb or more. That is, the system control circuit 50 determines whether or not the state where both eyes are closed is the second state that continues for the second determination time or longer.

  As described above, in the embodiment of the present invention, the determination criterion for determining that one eye is in an eye-mealing state is set to be looser than the determination criterion for determining that both eyes are in an eye-mealing state. Thereby, it is possible to easily detect the wink detection.

  Except for special circumstances such as entering the eyes, it is rare to meditate only one eye in normal life, so even if you loosen the criteria for judging that one eye is meditation, misjudgments increase There is almost nothing.

  Further, as described above, since there are few erroneous determinations that one eye is in a closed state, the determination time Ta is made shorter than the determination time Tb. As a result, the time until the start of photographing can be shortened.

  On the other hand, when it is determined that both eyes are in a closed state, the determination time Tb is set longer than the determination time Ta in order to clearly distinguish it from normal blinking. Desirably, the determination time Tb is set sufficiently longer than the time required for the normal blinking operation.

  As described above, in the embodiment of the present invention, even when both eyes are meditated, it can be used as a trigger as a cue to start shooting using a predetermined determination criterion. As a result, even a person who cannot wink or is difficult to wink Shooting can be started by an eye signal. As a result, even a person who cannot wink or has difficulty can take a still image at a desired timing.

  In the above-described embodiment, eye meditation detection is performed on a person who is determined to be the main face, but a specific person registered in advance is detected and eye meditation detection is performed on the person. You may do it.

  Further, although the eye meditation determination is performed based on the pupil area ratio, other indices may be used as long as the indices indicate the degree of eye meditation.

  Further, if both eyes are in a state of being meditated, the system control circuit 50 sets the eye meditating threshold level to the threshold level for both eyes in step S208, but this process is not necessarily required and may be omitted.

As is clear from the above description, in the example shown in FIG. 1, the system control circuit 50 functions as a determination unit, and the system control circuit 50, the memory control circuit 22, the image processing circuit 20, the timing generation circuit 18, and the A / D. The converter 16, the image sensor 14 and the like function as an imaging unit . The system control circuit 50 functions as an eye region determining unit , a pupil region extracting unit, and a ratio calculating unit .

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a control method, and this control method may be executed by the imaging apparatus. Further, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the imaging apparatus. The control program is recorded on a computer-readable recording medium, for example.

  The present invention can also be realized by executing the following processing. That is, software (program) for realizing the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

DESCRIPTION OF SYMBOLS 11 Shooting lens 14 Image pick-up element 18 Timing generation circuit 20 Image processing circuit 22 Memory control circuit 28 Image display part 30 Memory 50 System control circuit 62, 64 Shutter switch 200 Recording medium

Claims (5)

An imaging device that images a subject and records image data,
Based on the plurality of image data are sequentially read in a photographing standby state, determination means for determining that said that the subject Shut one eye, and wherein the subject is Shut both eyes,
Photographing means for photographing the subject when the subject is in the first state or the second state;
The first state is a state in which the subject is meditated on one eye for a predetermined first determination time or longer, and the second state is a state in which the subject is meditated on both eyes. An imaging apparatus characterized by being in a state of continuing for a second determination time longer than the first determination time . The determination means includes
Eye region determining means for determining a left eye region and a right eye region that respectively define a left eye and a right eye of the subject from the plurality of image data ;
Pupil region extraction means for extracting a left pupil region and a right pupil region indicating left and right pupils of the subject from the left eye region and the right eye region;
Together determine the ratio of the area of the left pupil region to an area of the eye region as a first percentage of, organic and ratio calculating means for calculating a ratio of the area of the right pupil region to the area of the right-eye region as a second percentage of And
It is determined whether each of the first ratio and the second ratio is less than a predetermined first threshold level, and the first ratio is less than the first threshold level. , determines that the state with shut the left, according to claim 1, wherein the second percentage of, wherein the determining that the state with shut the right eye is less than the first threshold level Imaging device. Said determining means, said first fraction and each of the second fraction of it is determined whether it is smaller than the second threshold level lower than said first threshold level, said first rate and When each of the second fraction of less than the second threshold level, the imaging apparatus according to claim 2, characterized in that determining that the state with shut both eyes. A method of controlling an imaging apparatus that images a subject and records image data,
Based on the plurality of image data are sequentially read in a photographing standby state, a determination step that said subject is Shut one eye, and wherein the subject is Shut both eyes,
A photographing step of photographing the subject when the subject is in the first state or the second state;
The first state is a state in which the subject is meditated on one eye for a predetermined first determination time or longer, and the second state is a state in which the subject is meditated on both eyes. A control method characterized by being in a state of continuing for a second determination time longer than the first determination time . A control program used in an imaging device that images a subject and records image data,
In the computer provided in the imaging device,
Based on the plurality of image data are sequentially read in a photographing standby state, a determination step that said subject is Shut one eye, and wherein the subject is Shut both eyes,
A photographing step of photographing the subject when the subject is in the first state or the second state;
The first state is a state in which the subject is meditated on one eye for a predetermined first determination time or longer, and the second state is a state in which the subject is meditated on both eyes. A control program characterized by being in a state of continuing for a second determination time longer than the first determination time .

Images