JP5623247B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus suitable for use in generating an image of a subject including shadows, a control method for the imaging apparatus, a program, and a storage medium.

  In general, in photography, it is widely known that the impression of a photographed photograph changes depending on how light is applied to a subject (hereinafter referred to as “lighting”). Therefore, various techniques have been developed in the case where light hits the subject. Patent Document 1 includes illumination means having a plurality of light-emitting surfaces on which light-emitting elements are arranged, and control means for individually controlling the light amount of the light-emitting elements and the direction of the light-emitting surface. There has been disclosed a person photographing apparatus that changes the amount of light and the light distribution more finely. This makes it possible to obtain a face photograph having a desired shadow.

  In addition, the face image processing apparatus described in Patent Document 2 adjusts the face direction of a plurality of face three-dimensional shape models stored in advance according to the input two-dimensional face image, and adjusts the three-dimensional shape model of the adjusted three-dimensional shape model. A light source direction estimated from a two-dimensional face image is applied to each to create a two-dimensional shadow image. Then, by synthesizing the 3D shape model corresponding to the shadow image most similar to the 2D face image and the 2D face image, a 3D face image corresponding to the face of the person is created.

  In addition, the face image matching device described in Patent Document 3 combines the input two-dimensional face image used for personal registration and a standard frame model representing a three-dimensional shape of a standard face stored. A three-dimensional face model corresponding to the person's face is generated. Then, a plurality of reference face images are generated from the generated three-dimensional face model according to a plurality of rendering conditions. This makes it possible to easily perform identity registration and perform face image matching with a low error rate.

International Publication No. 2007/108321 JP 2009-2111151 A JP 2003-6645 A

  However, the method disclosed in Patent Document 1 requires a lighting unit having at least a plurality of light emitting surfaces, and thus costs a lot. In addition, the methods disclosed in Patent Document 2 and Patent Document 3 do not require a plurality of illumination means, but require data processing of a three-dimensional shape, resulting in an enormous amount of calculation. That is, an expensive processor with a high processing load and high computing capacity and a worm memory with a large capacity are required, and the power consumption increases. As described above, the conventional technique has a problem that it is difficult to obtain a face photograph having a desired shadow by using hardware resources provided in a digital camera that is required to reduce cost and save power consumption.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to make it possible to obtain a subject photograph having a desired shadow by eliminating a plurality of illumination means and suppressing a processing load.

An image pickup apparatus according to the present invention includes an image pickup unit that picks up a subject and generates an image signal, a detection unit that detects main elements of the subject picked up by the image pickup unit, and a plurality of shadow patterns for light rays from different directions. And the position and shape of the shadow pattern designated by the user among the plurality of shadow patterns stored in the storage means based on the position of the main element of the subject detected by the detection means possess a changing unit for changing, the shaded pattern modified by the change means to synthesize the image according to the image signals so as to overlap with the object, and a display control means for displaying the synthesized image on the display means, The change means is a shadow pattern stored in the storage means based on the position of the main element of the subject detected by the detection means. At least one of the shadow patterns other than the shadow pattern specified by the user is changed, and the storage means is other than the shadow pattern specified by the user whose position and shape are changed by the changing means. The shading pattern is memorized .

  According to the present invention, it is possible to obtain a subject photograph having a desired shadow with a smaller processing load while eliminating the need for a plurality of illumination means.

It is a figure which shows the example of an external appearance structure of the digital camera which concerns on embodiment. It is a block diagram which shows the example of an internal structure of the digital camera which concerns on embodiment. It is a flowchart which shows an example of the process sequence which gives a lighting effect. It is a figure which shows an example of the shadow pattern memorize | stored in the non-volatile memory. It is a figure which shows an example of a mode that the position of the main element is specified. It is a figure which shows an example of the shadow pattern changed according to the position of the main element. It is a figure which shows an example of the image which synthesize | combined the shadow pattern. It is a figure which shows an example of the screen which instruct | indicates and selects a desired light beam direction.

Preferred embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
FIG. 1 is a diagram illustrating an external configuration example of a digital camera 100 as an example of an imaging apparatus in the present embodiment.
In FIG. 1, a display unit 101 is a display unit that displays images and various types of information. The shutter button 102 is a button for giving a shooting instruction. The mode dial 103 is an operation member for switching the operation mode to any one of a still image shooting mode, a moving image shooting mode, a reproduction mode, and the like. The connector 104 is for connecting the connection cable 111 and the digital camera 100.

  The operation unit 105 is an operation unit including operation members such as various switches, buttons, and a touch panel for receiving various operations from the user. The controller wheel 106 is a rotatable operation member included in the operation unit 105, and is used when a selection item is designated together with a direction button. The power switch 107 is a switch for switching between power-on and power-off.

  The recording medium 108 is a recording medium such as a memory card or a hard disk, and includes a semiconductor memory or a magnetic disk. The recording medium slot 109 is a slot for storing the recording medium 108. When the recording medium 108 is stored in the recording medium slot 109, communication between the recording medium 108 and the digital camera 100 becomes possible. A lid 110 is a lid of the recording medium slot 109.

FIG. 2 is a block diagram illustrating an internal configuration example of the digital camera 100 according to the present embodiment.
In FIG. 2, a photographing lens 201 is a lens including a focus lens, and a shutter 202 is a shutter having an aperture function. The imaging unit 203 includes a CCD, a CMOS, or the like that converts an optical image into an electrical signal. The A / D converter 204 is used to convert an analog signal output from the imaging unit 203 into a digital signal. The barrier 205 covers the photographing lens 201 of the digital camera 100 and prevents the imaging system including the photographing lens 201, the shutter 202, and the imaging unit 203 from becoming dirty or damaged.

  The image processing unit 206 performs resize processing and color conversion processing such as predetermined pixel interpolation and reduction on the data input from the A / D converter 204 or the memory control unit 207. The image processing unit 206 performs predetermined calculation processing using the captured image data, and the system control unit 212 performs exposure control and distance measurement control based on the obtained calculation result. Thereby, AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-emission) processing of the TTL (through-the-lens) method are performed. Further, the image processing unit 206 performs predetermined arithmetic processing using the captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained arithmetic result.

Data output from the A / D converter 204 is directly written into the memory 208 via the image processing unit 206 and the memory control unit 207 or via the memory control unit 207. The memory 208 stores image data converted into digital data by the A / D converter 204 and image data to be displayed on the display unit 210. The memory 208 has a storage capacity sufficient to store a predetermined number of still image data, moving image data and audio data for a predetermined time.
The memory 208 also serves as an image display memory (video memory).

  The D / A converter 209 converts the image display data stored in the memory 208 into an analog signal and supplies the analog signal to the display unit 210. In this manner, the display image data written in the memory 208 is displayed on the display unit 210 via the D / A converter 209. The display unit 210 performs display according to the analog signal output from the D / A converter 209 on a display such as an LCD.

  The nonvolatile memory 211 is an electrically erasable / recordable memory, and for example, an EEPROM or the like is used. The nonvolatile memory 211 stores constants, programs, and the like for operation of the system control unit 212. Here, the program is a program for executing various flowcharts described later in the present embodiment.

  The system control unit 212 controls the entire digital camera 100. By executing the program stored in the non-volatile memory 211 described above, each process of the present embodiment to be described later is realized. The system control unit 212 also performs display control by controlling the memory 208, the D / A converter 209, the display unit 210, and the like. For example, a RAM is used as the system memory 213, and constants and variables for operation of the system control unit 212, programs read from the nonvolatile memory 211, and the like are expanded in the system memory 213.

  The mode changeover switch (mode dial) 103, the first shutter switch 214, the second shutter switch 215, and the operation unit 105 are operation means for inputting various operation instructions to the system control unit 212.

  The first shutter switch 214 is turned on when the shutter button 102 provided in the digital camera 100 is being operated, so-called half-press (shooting preparation instruction), and generates a first shutter switch signal SW1. Then, in response to the first shutter switch signal SW1, operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-emission) processing are started.

  The second shutter switch 215 is turned on when the operation of the shutter button 102 is completed, so-called full press (shooting instruction), and generates a second shutter switch signal SW2. In response to the second shutter switch signal SW2, the system control unit 212 starts a series of shooting processing operations from reading an image signal from the imaging unit 203 to writing image data on the recording medium 108.

  Each operation member of the operation unit 105 shown in FIG. 1 is appropriately assigned a function for each scene by selecting and operating various function icons displayed on the display unit 210, and functions as various function buttons. Examples of the function buttons include an end button, a return button, an image advance button, a jump button, a narrowing button, and an attribute change button. For example, when a menu button is pressed, various setting menu screens are displayed on the display unit 210. The user can make various settings intuitively using the menu screen displayed on the display unit 210, the four-way button, and the SET button.

  When the controller wheel 106 shown in FIG. 1 included in the operation unit 105 is rotated, an electrical pulse signal is generated according to the operation amount. Based on this pulse signal, the system control unit 212 is a digital camera. 100 parts are controlled. From this pulse signal, it is possible to determine the angle at which the controller wheel 106 is rotated, how many rotations, and the like. The controller wheel 106 may be any operation member that can detect a rotation operation other than the example shown in FIG. For example, it may be a dial operation member that generates a pulse signal by rotating the controller wheel 106 itself in response to a user's rotation operation. Further, an operation member formed of a touch sensor may be used to detect a rotation operation of a user's finger on the controller wheel 106 such as a touch wheel without rotating the controller wheel 106 itself.

  The power control unit 216 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is attached, the type of battery, and the remaining battery level. The power supply control unit 216 controls the DC-DC converter based on the detection result and an instruction from the system control unit 212, and supplies a necessary voltage to each unit including the recording medium 108 for a necessary period.

  The power supply unit 217 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like. The interface 218 is an interface with the recording medium 108 such as a memory card.

  The digital camera 100 according to the present embodiment has a person detection function for detecting a person in an image. The person detection function will be described below. The system control unit 212 sends image data that is a target for detecting a person to the image processing unit 206. Then, under the control of the system control unit 212, the image processing unit 206 causes a horizontal bandpass filter to act on the image data, and further causes a vertical bandpass filter to act on the processed image data. Edge components are detected from the image data by the band pass filters in the horizontal and vertical directions.

  Thereafter, the system control unit 212 performs pattern matching on the detected edge component, and extracts a candidate group of main elements of the person such as eyes, nose, mouth, and ears. Next, for example, for eyes, those that satisfy a preset condition (for example, distance between two eyes, inclination, etc.) from the extracted candidate group are determined as eye pairs, and there are eye pairs. Narrow down as the only candidate group. Then, the narrowed-down eye candidate group is associated with other parts (nose, mouth, ears) that form a person, and the person is detected by passing through a preset non-person condition filter. Generate face information.

  The system control unit 212 outputs the face information according to the person detection result, and ends the process. At this time, information on the detected outline of the person and the positions of the main elements of the person such as eyes, nose, mouth, and ears is stored in the memory 208.

  FIG. 3 is a flowchart illustrating an example of a processing procedure in a shooting mode (hereinafter referred to as a lighting control mode) in which the digital camera 100 according to the present embodiment applies a lighting effect to a captured image according to a user's selection. Note that the processing shown in FIG. 3 is realized by developing a program recorded in the nonvolatile memory 211 in the system memory 213 and executing it by the system control unit 212.

  First, when the mode dial 103 or the operation unit 105 is operated on the menu screen to switch to the lighting control mode which is one of the still image shooting modes, the processing of FIG. 3 is started.

  First, in step S <b> 301, the image capturing instruction by the full press described above is performed, and the process waits until the second shutter switch signal SW <b> 2 is input to the system control unit 212. When a shooting instruction is given, the process proceeds to the next S302.

  Next, in S302, shooting processing is performed. This photographing process is a general main photographing process until an image signal photographed by the photographing unit 203 is recorded on the recording medium 108 as a still image file, and the detailed procedure is as described above with reference to FIG. .

  Next, in S303, the system control unit 212 performs person detection processing from the image captured in S302. Details of this processing are as described above.

  Next, in S304, the system control unit 212 selects a shadow selected by the user from a plurality of shadow patterns stored in the nonvolatile memory 211 based on the positions of the main elements of the person detected by the person detection process in S303. Change the position and shape of the pattern. If it is before accepting an operation for selecting a shadow pattern from the user, the position and shape of the default shadow pattern are changed.

  FIG. 4 is a diagram illustrating an example of a shadow pattern stored in the nonvolatile memory 211. FIG. 4 shows shadow patterns for light rays from eight directions of upper, upper right, right, lower right, lower, lower left, left, and upper left. FIG. 5 is a diagram showing an example of detecting a person in the image and specifying the position of the main element. In FIG. 5, the detected outline, eyes, nose, mouth, and ear positions of the person are displayed by bold lines.

  For example, it is assumed that the main elements (eyes, nose, mouth, ears, contour) of the person detected by the person detection process in S303 are positions as shown in FIG. Then, it is assumed that the shadow pattern selected by the user is the shadow pattern for the light rays from the upper right direction shown in FIG. In this case, in S304, the shadow pattern for the light rays from the upper right direction is changed to a shadow pattern as shown in FIG. 6 based on the positions of the main elements of the person shown in FIG.

  Next, in S305, the system control unit 212 combines the shadow pattern changed in S304 with an overlay effect on the person in the image taken in S302. Then, the combined image is displayed on the display unit 210. This is performed as so-called REC review display in which an image photographed immediately after photographing is reviewed and displayed in the photographing mode. FIG. 7 shows an example of an image obtained by combining shadow patterns.

  Next, in S306, it is determined whether or not an operation for switching the specified shadow pattern to another shadow pattern (an operation for selecting a shadow pattern) has been received from the user. As a result of this determination, if it is determined that a shadow pattern change operation has been accepted, the process proceeds to S304, where the shadow pattern is changed and the process is repeated. On the other hand, if the shadow pattern changing operation has not been received, the process proceeds to S307.

  An example of the selection operation method will be described. It is assumed that a predetermined order is predetermined for the plurality of shadow patterns stored in the nonvolatile memory 211. If it is determined in step S306 that the right button of the four-direction button included in the operation unit 105 has been pressed, the selection is switched to the shadow pattern that is next in order from the currently selected shadow pattern. Switch selection to previous shadow pattern.

  Next, in S307, the system control unit 212 determines whether or not to end the REC review display. The REC review display is determined to end when a display end instruction operation or a next shooting preparation instruction is received from the user or when a predetermined time has elapsed. If the result of this determination is that it does not end, processing returns to S306 and the processing is repeated, and if it ends, processing proceeds to S308.

  Next, in S308, an image obtained by combining the shadow pattern changed in S304 with an overlay effect on the person captured in S302 (the image displayed on the display unit 210 immediately before) is combined into an image file. To the recording medium 108.

  In step S309, the system control unit 212 determines whether to end the lighting control mode. If the result of this determination is that the lighting control mode is not terminated, processing returns to S301 and the processing is repeated. On the other hand, when the lighting control mode is ended, the processing of FIG. 3 is ended as it is. Exiting the lighting control mode means automatically switching to another shooting mode or playback mode, turning off the power, or automatically saving power if no operation is performed for a while. For example, auto power off is activated when the power is turned off.

  According to the processing described above, a face photograph having a shadow desired by the user can be obtained by selecting from a plurality of shadow patterns and changing and combining the selected shadow patterns according to the person detected from the image. Can do. Accordingly, there is no need to provide other illumination means, and the processing load can be reduced as compared with a method of processing data of a three-dimensional shape. That is, even with an inexpensive processor with a lower computing capacity than before and an inexpensive work memory with a smaller capacity, it is possible to obtain a face photograph with a shade desired by the user. In addition, power consumption can be saved as the amount of computation is reduced. Furthermore, if the hardware resources are the same as the conventional one, processing can be performed at a higher speed as the amount of computation is suppressed, leading to an improved response.

  In S304 of FIG. 3 described above, the example in which the position and shape of the shadow pattern selected by the user is changed has been described. On the other hand, other than the shadow pattern selected by the user, the positions and shapes of the preceding and following shadow patterns may be changed and held in the system memory 213. By doing in this way, the response with respect to a user's shadow pattern change operation can be improved.

  In addition, although the REC review display has been described in the present embodiment, the processing of S303 to S306 in FIG. 3 may be performed on the reproduction target image read from the recording medium 108 in the reproduction mode. In this case, when there is an operation for switching an image to be played back (image forwarding operation) or an operation for ending the playback mode, the image displayed on the display unit 210 immediately before is displayed as an image file as in S308. Recording is performed on the recording medium 108.

  Further, the processing of S303 to S306 in FIG. 3 may be performed on a through image displayed in a so-called through display in which an image captured by the imaging unit 203 in the imaging mode is displayed in real time.

  Moreover, although the example which detects the main elements of a person's face by S303 was demonstrated, main elements (a head, a neck, a shoulder, an arm, a hand, a torso, a leg, etc.) other than a person's face are detected, and a shadow pattern is detected. It may be changed. In this case, the shadow pattern before change stored in the non-volatile memory 211 is not the face portion as shown in FIG. 4 but a shadow pattern for the entire human body. Furthermore, any subject can be used as long as it is a detectable subject. For example, an animal such as a dog or a cat may be detected, and a shadow pattern adapted to these animals may be stored.

(Second Embodiment)
In the present embodiment, an example will be described in which, in a digital camera equipped with a touch panel, the shadow pattern selection operation that can be accepted in S306 of FIG. 3 described above is an operation on the touch panel. Since the other aspects of the digital camera, the internal configuration, and the procedure for applying the lighting effect are the same as those in the first embodiment, description thereof will be omitted.

  The touch panel is an input device that outputs coordinate information according to a position touched with respect to an input unit configured in a planar manner, for example. The digital camera 100 according to this embodiment is configured integrally with a display, and associates input coordinates on the touch panel with display coordinates on the display (display unit 101). Thereby, it is possible to configure a GUI as if the user can directly operate the screen displayed on the display.

FIG. 8 is a diagram showing an example of a screen displayed on the display unit 101 in order to allow the user to instruct and select a desired light direction in the digital camera 100 according to the present embodiment.
In FIG. 8, a small circle is displayed as a guide in the eight directions above, upper right, right, lower right, lower, lower left, left, upper left of the person in the image, and when one point is touched by the user, The shadow pattern for the light ray from the direction corresponding to the touched point is read out. As a result, the user can instruct and select a desired light beam direction in association with the display coordinates of the display. Therefore, it is possible to realize a special operation as if the light beam direction can be directly changed with respect to the displayed image.

(Other embodiments)
Note that the control of the system control unit 212 may be performed by a single piece of hardware, or the entire apparatus may be controlled by a plurality of pieces of hardware sharing the processing. Although the present invention has been described in detail based on the preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms without departing from the gist of the present invention are also included in the present invention. included. Furthermore, each embodiment mentioned above shows only one embodiment of this invention, and it is also possible to combine each embodiment suitably.

  In the above-described embodiment, the case where the present invention is applied to a digital camera has been described as an example. However, this is not limited to this example. That is, the present invention can also be applied to a personal computer, a PDA, a mobile phone terminal, a portable image viewer, a display for selecting and confirming a print image provided in a printer, a digital photo frame, and the like. As described above, any image processing apparatus that can synthesize and display a selected one of the stored shadow patterns can be applied.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed. In this case, the program and a computer-readable storage medium storing the program constitute the present invention.

203 Imaging unit 208 Memory 210 Display unit 212 System control unit

Claims (7)

Imaging means for imaging a subject and generating an image signal;
Detecting means for detecting main elements of the subject imaged by the imaging means;
Storage means for storing a plurality of shadow patterns for light rays from different directions;
Changing means for changing the position and shape of the shadow pattern designated by the user among the plurality of shadow patterns stored in the storage means based on the position of the main element of the subject detected by the detection means;
The shaded pattern modified by the change means to synthesize the image according to the image signals so as to overlap with the object, to have a display control means for displaying the synthesized image on the display means,
The changing means is based on the position of the main element of the subject detected by the detecting means, and among the shadow patterns stored in the storage means, the position and shape of a shadow pattern other than the shadow pattern designated by the user Change at least one
The image pickup apparatus , wherein the storage unit stores a shadow pattern other than a shadow pattern designated by a user whose position and shape are changed by the changing unit. The display means is a touch panel,
The imaging apparatus according to claim 1, wherein the changing unit changes a position and a shape of a shadow pattern corresponding to the touched position in response to the touch on the touch panel. 3. The imaging apparatus according to claim 1, wherein the plurality of shadow patterns are patterns for light rays from directions of upper, lower, right, left, upper right, lower right, upper left, and lower left. The imaging apparatus according to any one of claim 1 to 3, further comprising a recording means for recording the image displayed on the display means as an image file. An imaging step of imaging an object and generating an image signal;
A detection step of detecting main elements of the subject imaged in the imaging step;
Of the plurality of shadow patterns for light rays from different directions stored in the storage means, the position and shape of the shadow pattern designated by the user are based on the positions of the main elements of the subject detected in the detection step. Change process to change,
Wherein the modified shading patterns in changing step synthesizes the image according to the image signals so as to overlap with the object, to have a display control step of displaying the composite image on the display means,
In the changing step, based on the position of the main element of the subject detected in the detecting step, among the shadow patterns stored in the storage unit, the position of the shadow pattern other than the shadow pattern designated by the user and Change at least one shape,
The storage unit stores a shadow pattern other than a shadow pattern designated by a user whose position and shape have been changed in the changing step . An imaging step of imaging an object and generating an image signal;
A detection step of detecting main elements of the subject imaged in the imaging step;
Of the plurality of shadow patterns for light rays from different directions stored in the storage means, the position and shape of the shadow pattern designated by the user are based on the positions of the main elements of the subject detected in the detection step. Change process to change,
Combining the image related to the image signal so that the shadow pattern changed in the changing step overlaps the subject, and causing the computer to execute a display control step of displaying the combined image on a display unit ,
In the changing step, based on the position of the main element of the subject detected in the detecting step, among the shadow patterns stored in the storage unit, the position of the shadow pattern other than the shadow pattern designated by the user and Change at least one shape,
The storage means stores a shadow pattern other than a shadow pattern designated by a user whose position and shape have been changed in the changing step . A computer-readable storage medium storing the program according to claim 6 .

Images