JP2006340281A - Image pickup apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image pickup apparatus capable of enlarging a specific region in a picked-up image through simple operation. <P>SOLUTION: Photographed images are temporarily stored in a temporary storage section 13 and displayed on a liquid crystal monitor 3 as an entire picture. Furthermore, a face image extraction section 16 extracts the face of a human figure of an object from the temporarily stored image data and when a user presses a select button 5, the extracted face image is enlarged and displayed on the liquid crystal monitor 3. The user checks the expression of the enlarged object and then, image data in the temporary storage section 13 are stored and recorded in an image recording section 15. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image capturing apparatus, and more particularly to an image capturing apparatus capable of confirming a captured image before storing and recording the captured image.

  Conventionally, in an image capturing apparatus such as a digital still camera (hereinafter referred to as a digital camera), it is possible to display a captured image on a liquid crystal monitor built in the camera. Before the image is recorded in the memory, it is possible to confirm what kind of image has been taken, for example, the state of focus.

However, with the recent miniaturization of digital cameras, there are many cases where the screen size of the liquid crystal monitor must be reduced. In such a small LCD monitor, it is difficult to check whether or not the subject is in focus accurately because the display image is small, and it is a so-called out-of-focus image that is not in focus only after being displayed on a large monitor. Is often found.
On the other hand, as the performance of digital camera image sensors increases, the number of pixels in the captured image increases, that is, the image size increases, so it is more convenient to check the focus of the subject on a small LCD monitor. It has become even more difficult.

In view of this problem, a method has been proposed in which a specific in-focus portion of a captured image is enlarged and displayed on an image display device (for example, see Patent Document 1). According to this method, since the set specific in-focus portion is enlarged and displayed by a simple operation, it can be easily determined whether or not the captured subject is in focus. In addition, a method in which the user selects and confirms the portions to be enlarged one by one can be considered.
JP 2004-64259 A

By the way, when a person is photographed as a subject, the facial expression of the subject is often an important point for determining success or failure of photographing. For example, re-photographing is often performed because the subject in the captured image has closed eyes or is not smiling.
In photographing using a digital camera, immediately after photographing, the photographed image can be displayed on the spot to check the expression of the subject, so it is possible to immediately determine whether or not to re-shoot. Then, according to the method described in Patent Document 1, with respect to a person reflected in a focused portion, the portion of the image is enlarged, and the facial expression (and focus state) of the subject is confirmed by a simple operation. be able to.

  However, depending on the composition of the photograph, the person is not necessarily brought to the in-focus portion of the captured image. In such a case, the method described in Patent Document 1 cannot be used to enlarge and confirm the expression of the person. Can not. That is, according to the method described in Patent Document 1, only a specific focus portion that is set can be displayed in an enlarged manner. For example, a person or a group photo taken in an area other than the focus portion can be displayed. For a plurality of persons included in the image, it was impossible to enlarge the face part (of all persons). Therefore, in such a case, there is a problem that the user has to enlarge the desired portion by manual operation, and the operation is very complicated.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an image imaging apparatus capable of enlarging a specific area in a captured image by a simple operation.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems. The invention according to claim 1 is an image pickup means for picking up a subject and generating image data of the subject, and temporarily storing the image data. Temporary storage means, image extraction means for extracting at least one image area having a specific image pattern from the image data stored in the temporary storage means, and converting the extracted image area into an enlarged image An image capturing apparatus comprising: an image enlarging unit that performs display, a display unit that displays an image, and an image recording unit that records the image data for storage.
According to the present invention, the image area having the specific image pattern is extracted from the temporarily stored image data and is enlarged and displayed, so that the user can enlarge and display the specific portion in the photographed image by a simple operation. This makes it possible to easily check an image before saving and recording on a recording medium. In addition, since the image area to be enlarged is extracted by pattern recognition based on a specific image pattern, a desired portion can be accurately enlarged and displayed regardless of where the subject is in the image.

According to a second aspect of the present invention, in the image pickup apparatus according to the first aspect, the image enlarged by the image enlargement means and the image before the enlargement are switched to the display means and displayed. The display control means is further provided.
According to the present invention, the display can be switched between the whole image and the enlarged image, and the convenience at the time of image check is improved.

According to a third aspect of the present invention, in the image pickup apparatus according to the first or second aspect, the display unit enlarges and displays the plurality of image areas extracted by the image extraction unit. The image display device further comprises image selection means for selecting an image area to be performed.
According to the present invention, the display can be switched between a plurality of extracted enlarged images, and convenience at the time of image check is improved.

  According to a fourth aspect of the present invention, in the image imaging device according to any one of the first to third aspects, the specific image pattern is a human face image pattern stored in advance. It is characterized by that.

According to a fifth aspect of the present invention, in the image imaging device according to any one of the first to third aspects, the specific image pattern is a human eye image pattern stored in advance. It is characterized by that.
According to the fourth and fifth aspects of the present invention, human face or eye image patterns are stored in advance, respectively, and the face or eye portion of the subject is detected from images captured using these image patterns. Extracting. As a result, the image can be enlarged and displayed on the face or eye part of the photographed subject.

The invention according to claim 6 is the image pickup device according to claim 5, wherein the eye part image pattern is an image pattern of an eye opening, and the eye opening extracted by the image extraction means is opened. In addition, the information processing device further includes notification means for performing notification based on the degree.
According to the present invention, an eye pattern of a subject in a captured image is detected by storing an image pattern related to the eye opening in advance, and notification is made that the subject closes the eye if necessary. Therefore, it is possible to surely check the image.

  According to the present invention, it is possible to realize an image capturing apparatus capable of enlarging a specific area in a captured image by a simple operation, whereby the user can easily and where the subject is in the image. Regardless of this, an image check can be performed before saving and recording on a recording medium.

Embodiments of the present invention will be described below with reference to the drawings.
<< First Embodiment >>
FIG. 1 is an external view of an image pickup apparatus (digital camera 1) according to a first embodiment of the present invention, and is a view of the camera as viewed from the rear (surface on which a liquid crystal monitor 3 is provided) obliquely from above.

  The liquid crystal monitor 3 is a liquid crystal display that electronically displays an image captured by the image pickup unit 11 and is used for determining the composition of a subject by displaying an image in real time during shooting and after shooting. Used to check the image by displaying the captured image (display in normal size or enlarged display). In addition, various information such as an operation menu, date, time, and remaining battery level are displayed as necessary.

  The release button 4 is a button for performing release (releasing the shutter), and is provided in the vicinity of the right side (viewed from the back) end of the camera so that the user can press it with the right index finger. Further, the release button 4 can perform a first-stage input for performing a pressing operation with a light force and a second-stage input for performing a pressing operation with a stronger force from the first-stage input state. It is a two-stage button.

The selection button 5 is a button for selecting a portion to be enlarged and displayed when there are a plurality of portions to be enlarged in the image.
The switching button 6 is a button for switching between the entire display (normal size display) and the enlarged display of the liquid crystal monitor 3.
The memory slot 7 is a portion into which a removable external recording medium (image recording unit 15) for recording captured image data is inserted.

FIG. 2 is a block diagram showing a configuration of an electric processing system in the digital camera 1.
An image of a subject is formed on an image pickup device (CCD; charge coupled device) by a lens and converted into electric charge by the image pickup device, but the image pickup unit 11 generates image data from an output signal of the image pickup device. It is. The lens and the image sensor are not shown.

The image processing unit 12 is a processing unit that performs predetermined image processing (for example, electronic camera shake correction) on the output of the imaging unit 11 that is a raw image as it is captured.
The temporary storage unit 13 is a RAM (random access memory) for temporarily storing image processed image data output from the image processing unit 12.

  The image recording unit 15 is a recording medium that records captured image data for storage. The image data is compressed and recorded in the image recording unit 15 in a predetermined format. The compression / decompression unit 14 is a processing unit that performs compression (for recording) and expansion (for reading) for that purpose.

The face image extraction unit 16 is a processing unit that detects a face portion of a person who is a subject from image data stored in the temporary storage unit 13, extracts the face portion image, and extracts a face image. . If a plurality of persons are shown, face images are extracted for the plurality of persons.
When a plurality of human face images are input from the face image extraction unit 16, the image selection unit 17 receives the output signal from the selection button 5 and transmits the face image data selected by the user to the image enlargement unit 18. Output.

The image enlargement unit 18 is a processing unit that enlarges the input face image data to a predetermined magnification.
In accordance with the output signal from the switching button 6, the display control unit 19 displays one of the image data (entire image) in the temporary storage unit 13 and the face image data (enlarged image) input from the image enlargement unit 18 as a liquid crystal. Output to monitor 3.

Next, the operation of the digital camera 1 will be described with reference to the flowchart shown in FIG. 3 and the photographing example (display image of the liquid crystal monitor 3) shown in FIG.
First, when the user lightly presses the release button 4 to input the first stage, the shutter is operated after the predetermined focus / exposure mechanism is operated (step S1), and the imaging unit 11 and the image processing unit are operated. 12 is stored in the temporary storage unit 13 (step S2).

  The temporarily stored image data is immediately sent to the display control unit 19 as it is, and the entire image is displayed on the liquid crystal monitor 3 (step S3, FIG. 4A). At this time, the output from the switching button 6 is not performed. Here, it is assumed that three persons are shown in the photographed image as shown in FIG. 4A, and for the convenience of the following description, each person's face is modeled and a round face (left person) is shown. ), A rhombus face (middle person), and a pentagon face (right person).

Next, the temporarily stored image data is sent to the face image extracting unit 16 to detect the face of the person included in the image and extract the face image (step S4).
Here, a known detection (pattern recognition) technique using an image pattern of a human face is applied to face detection. For example, a method for detecting a face by extracting a contour portion in an image and calculating a bend angle of the contour line, a method for recognizing the skin color area as a face by detecting the skin color of the face, and the like are known. Yes. There is also a method of preparing (storing) a large number of image data in which a human face, eyes, nose, and mouth are drawn only by contour lines, and performing pattern matching with the extracted contour lines. When a plurality of faces are shown in the image, a plurality of face portions are detected by these methods.

  When the detection process is completed, the detected portion is extracted as a face image, and selection numbers are sequentially assigned to the extracted face image from “1” for image selection described later. Further, the number of detected face parts is displayed in the lower right part of the screen of the liquid crystal monitor 3 and is notified to the user as a detection result (see FIG. 4A). In the example of FIG. 4A, it is assumed that the face image selection numbers “1”, “2”, and “3” are assigned in order from the left person. The number of detections is displayed as 3.

Since the above notification is given to the liquid crystal monitor 3, the user can know the completion of the face image extraction. Next, in order to select the face image to be enlarged and displayed, the user sets the release button 4 to the one-stage input state. The selection button 5 is pressed down ("Yes" in step S8).
Then, a signal from the selection button 5 is input to the image selection unit 17, and the image selection unit 17 thereby causes the left person (round face) in the image of FIG. 4A to which the selection number “1” is assigned. ) Is selected. Then, the face image is sent to the image enlargement unit 18 and enlarged to a predetermined magnification, and then output to the display control unit 19. When receiving the input from the image enlargement unit 18, the display control unit 19 switches the display on the liquid crystal monitor 3 from the entire image to the inputted enlarged face image. Thus, the face portion of the first person shown in the photographed image is enlarged and displayed on the liquid crystal monitor 3 (step S9).

If a plurality of face images are extracted, the selection number is incremented (step S10), and then steps S7 to S9 are repeated. The face images enlarged each time the selection button 5 is pressed are successively displayed on the LCD monitor 3. Is displayed.
In the example shown in FIG. 4, as shown in FIGS. 4B to 4D, the round face person, the rhombus face person, and the pentagon face person are arranged in this order according to the assigned selection number. The face image is enlarged and displayed.
If the switch button 6 is pressed while the release button 4 is in the one-stage input state (“as is” in step S7 → “No” in step S8 → “Yes” in step S11), the display on the liquid crystal monitor 3 is displayed as the entire image. Switching to the enlarged image (step S12).

  By appropriately operating the selection button 5 and the switching button 6 as described above, the user checks the face of the person in the photographed image displayed in an enlarged manner. Then, there is no problem in each person's facial expression (whether eyes are closed, etc.) or focus, and the release button 4 held in the one-stage input state when recording the image (entire image). Press strongly to enter the second stage input state ("Deep press" in step S7). Then, the captured image stored in the temporary storage unit 13 is recorded in the image recording unit 15 after being compressed by the compression / expansion unit 14 (step S13). Thereafter, the camera 1 enters a next photographing standby state.

  On the other hand, when there is a problem with the facial expression of the person and the user wants to re-photograph, the release button 4 in the one-stage input state is released (finger is released) (“open” in step S7). Then, the captured image data in the temporary storage unit 13 is cleared without being saved and recorded (step S14), and the camera 1 moves to the next shooting standby state.

  As described above, according to the first embodiment, the face portion of the subject person is extracted from the image data temporarily stored in the temporary storage unit 13 before the captured image is stored and recorded in the image recording unit 15. This is enlarged and displayed on the liquid crystal monitor 3. Accordingly, the user can enlarge and display the face portion of the subject by a simple operation (pressing the selection button 5), and can determine whether to save or re-shoot the image after checking the detailed facial expression of the face. It has become.

In addition, since the area to be enlarged is determined by extracting the face of the subject by pattern recognition (rather than being a fixed portion in the image), the user can select this regardless of where the face is in the image. You can check the facial expression by enlarging it.
Furthermore, since a plurality of extracted faces are selected and displayed in a simple operation, or the display is switched to the entire image, the convenience of image checking after shooting can be improved.

As described above, the first embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to that described above, and various designs can be made without departing from the scope of the present invention. It is possible to make changes.
For example, in the above-described embodiment, a plurality of enlarged face image are displayed one after another by the selection button 5 and the whole image and the enlarged image are switched by the switching button 6, but the selection button 5 has the function of the switching button 6. You can also. That is, by pressing down the selection button 5, in the case of FIG. 4, (b) → (c) → (d) → (a) → (b). An image may be displayed. Thereby, the number of buttons can be reduced.

  Further, regarding the operation of selecting the detected face portion, instead of the method of sequentially displaying the enlarged face image by pressing the selection button 5 as described above, the detected face portion is detected on the liquid crystal monitor 3 displaying the entire image. A square “frame” indicating the detected portion may be displayed on the face portion, and the frame may be freely selected using a cross key or the like. This is convenient because the selection operation can be performed intuitively and visually even when a large number of people appear in the group photo.

  In the above-described embodiment, the photographed image is recorded in the image recording unit 15 by pressing the release button 4 deeply. However, the image recording operation can take other methods. For example, a dedicated button for recording may be provided, or the release button 4 is a one-stage button and the temporarily stored photographed image is checked (the entire photographed image or an enlarged image is displayed on the liquid crystal monitor 3. When the release button 4 is pressed once again when the release button 4 is once released), the image recording unit 15 may be recorded. In the latter case, for example, the operation of moving to re-photographing without saving the image may be performed by, for example, pressing the release button 4 for a long time (holding it down for several seconds). If these methods are adopted, there is no need to keep the release button 4 in the one-stage input state during the image check as in the above-described embodiment, and there is an advantage that a comfortable operation can be realized.

<< Second Embodiment >>
Next explained is the second embodiment of the invention.
FIG. 5 is an external view of an image pickup apparatus (digital camera 2) according to the second embodiment of the present invention, and is a view as seen from diagonally above the back surface (surface on which the liquid crystal monitor 3 is provided) of the camera. In the figure, the configuration of the camera 2 is the same as that of the camera 1 of the first embodiment described above except that the LED 8 is added and the selection button 5 is deleted.
The LED 8 is a light emitting diode that notifies the user by lighting (or flashing) that an image captured in a predetermined case is a failed shot.

FIG. 6 is a block diagram showing a configuration of an electric processing system in the digital camera 2. Also in the figure, the face image extraction unit 16 is replaced with the eye image extraction unit 21, the image selection unit 17 is deleted, and the eye opening degree detection unit 22 is added. Since it is the same structure as the camera 1, description is abbreviate | omitted.
The eye part image extraction unit 21 detects the eye part of the person who is the subject from the image data stored in the temporary storage part 13, extracts the eye part image, and extracts the eye part image. It is.
The eye opening degree detection unit 22 calculates the eye opening degree (the degree of eye opening) in the eye image, and determines whether or not the value is equal to or less than a predetermined value.

Next, the operation of the digital camera 2 will be described with reference to a flowchart shown in FIG. 7 and a photographing example (display image on the liquid crystal monitor 3) shown in FIG.
First, from when the subject is photographed until the entire image is displayed on the liquid crystal monitor 3, the same process as in the first embodiment is performed (steps S1 to S3). However, it is assumed here that the subject is one person as shown in FIG.

Next, the image data stored in the temporary storage unit 13 is sent to the eye image extraction unit 21 to detect the eyes of the person included in the image and extract the eye image (step S5a). .
Here, as in the face detection in the first embodiment, a known detection (pattern recognition) technique using an image pattern of the human eye is applied to the detection of the human eye. For example, a method of performing pattern matching of a large number of eye shape data prepared in advance for an outline extracted from an image, or a black area (pupil) in a skin color area (face) in an image having a certain size If it is included, a method of recognizing the periphery of the black area as an eye is applied.
When the detection process is completed, the detected part is extracted as an eye part image, and the extracted eye part image is sent to the eye opening degree detection part 22.

Then, the eye opening degree detection unit 22 calculates the eye opening degree in the input eye part image (step S5b). Specifically, as an opening calculation method, image data with a wide open eye is prepared in the pattern matching and collation is performed, or a black portion ( A known method using an image pattern of the opening degree of the human eye, such as calculating the area ratio of the pupil), is used. Note that the opening degree is defined to be a larger value as the eyes are opened wide.
Next, it is determined whether or not the calculated opening degree of the eyes is smaller than a preset value (step S6a).

If the result of determination is smaller than the set value, it is determined that there is a possibility of a failed shot in the image because the person who is the subject has closed eyes, and the LED 8 is turned on (step S6b). Further, the eye image extracted in step S5a is sent to the image enlarging unit 18, and the same processing as that in step S9 in the first embodiment is performed to display the enlarged eye image on the liquid crystal monitor 3. (Step S6c).
The display screen of the liquid crystal monitor 3 at this time is, for example, as shown in FIG. According to the figure, the eye part of the subject person whose one eye is closed is enlarged and displayed on the liquid crystal monitor 3.

In this way, the digital camera 2 notifies the user of the possibility of a failed shot (the subject person closed his eyes) by turning on the LED 8 and switching the liquid crystal monitor 3 to the enlarged display. As a result, the user makes a final decision as to whether or not the shot is actually a failed shot after checking the enlarged image of the eye with his / her own eyes.
Then, when the user performs an operation of deeply pressing or releasing the release button 4 according to the determination result (step S7), the image is stored and recorded in the image recording unit 15 (step S7) as in the first embodiment. S13), or clearing the image data in the temporary storage unit 13 (step S14).

  Note that when the switch button 6 is pressed while the release button 4 is in the one-stage input state, the operation of switching the display on the liquid crystal monitor 3 between the entire image and the enlarged image is the same as in the first embodiment ( In step S7, “as is” → “Yes” in step S11 → step S12).

As described above, according to the second embodiment, the eye portion of the subject can be enlarged and displayed on the liquid crystal monitor 3 as in the first embodiment.
Further, the opening degree of the subject's eye is calculated, and if the opening degree is not sufficient, the fact is notified. Therefore, the user can surely perform an image check before saving and recording.

As described above, the second embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various designs can be made without departing from the scope of the present invention. Changes can be made.
For example, regarding the operation method for recording the photographed image in the image recording unit 15, it is needless to say that the alternative method described in the first embodiment can be similarly applied.

Further, the lighting method of the LED 8 in step S6b may be blinked instead of continuous lighting. Furthermore, it may replace with LED8 or you may make it alert | report by an electronic alarm sound or an electronic sound with LED8. Further, the way of notification (how to turn on and sound) may be varied according to the calculated eye opening.
Further, similarly to the first embodiment, a selection button 5 may be provided to select a plurality of extracted eye images.

1 is an external view of an image pickup apparatus according to a first embodiment. FIG. 2 is a block diagram illustrating a configuration of an electric processing system in the image pickup apparatus of FIG. 1. 3 is a flowchart illustrating an operation of the image capturing apparatus in FIG. 1. 2 is an example of an image displayed on a liquid crystal monitor in the image pickup apparatus of FIG. 1. It is an external view of the image pick-up device by a 2nd embodiment. FIG. 6 is a block diagram illustrating a configuration of an electric processing system in the image pickup apparatus of FIG. 5. 6 is a flowchart illustrating an operation of the image capturing apparatus in FIG. 5. 6 is a shooting example displayed on a liquid crystal monitor in the image pickup apparatus of FIG. 5.

Explanation of symbols

1, 2 ... Digital camera 3 ... LCD monitor 4 ... Release button 5 ... Selection button 6 ... Switch button 7 ... Memory slot 8 ... LED 11 ... Imaging unit 12 ... Image processing unit 13 ... Temporary storage unit 14 ... Compression / decompression unit 15 ... Image recording unit 16 ... face image extraction unit 17 ... image selection unit 18 ... image enlargement unit 19 ... display control unit 21 ... eye part image extraction unit 22 ... eye opening degree detection unit

Claims (6)

Imaging means for imaging a subject and generating image data of the subject;
Temporary storage means for temporarily storing the image data;
Image extracting means for extracting at least one image region having a specific image pattern from the image data stored in the temporary storage means;
Image enlarging means for converting the extracted image region into an enlarged image;
Display means for displaying an image;
Image recording means for recording the image data for storage;
An image pickup apparatus comprising: The image capturing apparatus according to claim 1, further comprising: a display control unit configured to switch the display unit to display the image enlarged by the image enlargement unit and the image before the enlargement. The image selecting means for selecting an image area to be enlarged and displayed on the display means from among a plurality of image areas extracted by the image extracting means. Image pickup device. The image capturing apparatus according to any one of claims 1 to 3, wherein the specific image pattern is a human face image pattern stored in advance. The image capturing apparatus according to any one of claims 1 to 3, wherein the specific image pattern is a human eye part image pattern stored in advance. The eye image pattern is an image pattern of eye opening,
The image capturing apparatus according to claim 5, further comprising notification means for performing notification based on an eye opening degree extracted by the image extraction means.

Images