JP2011055251A - Imaging apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To surely take a shutter chance in simple configuration. <P>SOLUTION: When it is determined in a step S22 that a photographing mode is a photographer mode, it is determined in a step S23 whether or not a smiling face (smile) of a photographer is detected. When it is determined that the smile of the photographer is detected, an image of the photographer is captured in a step S24 and an image of a subject is captured in a step S27. When it is determined in a step S25 that the photographing mode is a subject mode, it is determined in a step S26 whether or not a smiling face (smile) of the subject is detected. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and method, and more particularly, to an imaging apparatus and method that can reliably capture a photo opportunity with a simple configuration.

  For example, in a digital camera or the like, various schemes have been proposed for recording a subject image by determining the recognition of the face of a person who is the subject, the recognition of facial expressions such as a smile, and the action.

  In such a digital camera, it is necessary to recognize the subject and to recognize the facial expression and motion of the subject, and the recognized facial expression and motion of the subject are triggers for shooting.

  On the other hand, when a photographer shoots a subject by pressing the shutter button of the camera at his / her own will, it is necessary to always pay attention to the shutter chance to record the decisive moment of the subject, and sometimes it is Sometimes missed. For this reason, a mechanism (using a ring memory or the like) that starts recording a few frames back from the shutter release timing has been proposed.

  There has also been proposed a camera that performs release by detecting the wink state of a photographer in a non-contact manner (see, for example, Patent Document 1).

JP 7-191405 A

  However, taking only the facial expression and motion of the subject as a trigger for shooting functions effectively, for example, when shooting a person, although it works effectively for shooting a person on the premise that there is sufficient shooting preparation time, etc. May not be suitable for

  Further, by using the technique of Patent Document 1, for example, the operability at the time of shooting at a higher shutter speed can be improved. However, if the photographer does not stare at the subject, the shutter chance is missed. There is a possibility.

  Furthermore, in a mechanism that starts recording a few frames past the shutter release timing, it is necessary to use a ring memory or the like, or a high-speed CMOS image sensor.

  For this reason, a large-capacity memory for recording a large number of image data is necessary, and a very troublesome operation such as checking and selecting an image after shooting is necessary.

  The present invention has been made in view of such a situation, and makes it possible to reliably capture a photo opportunity with a simple configuration.

  One aspect of the present invention is a first camera block including an optical system, a sensor, and a signal processing unit that collects light, and an optical system and sensor that collects light at a position different from that of the first camera block. And a second camera block comprising a signal processing unit, and a characteristic region in an image corresponding to an image signal output from the first camera block or the second camera block, or a second camera block According to the state of the characteristic region in the image corresponding to the image signal output from the image signal, and the state detection means for outputting a detection signal, and when imaging in the first mode, is output from the first camera block. Recording of an image corresponding to the image signal output from the first camera block is controlled based on a detection signal output according to the state of a characteristic region in the image corresponding to the image signal And a detection signal output in accordance with the state of the characteristic area in the image corresponding to the image signal output from the second camera block when imaging in the second mode. And a second recording control means for controlling recording of an image corresponding to the image signal output from the second camera block together with an image corresponding to the image signal output from the first camera block. It is an imaging device provided.

  The optical system of the first camera block collects light in the direction in which the subject is located, and the optical system of the second camera block collects light in the direction in which the photographer is located. can do.

  The detection means may detect a human face area as the characteristic area and output the detection signal in accordance with the degree of smile of the face.

  The image processing apparatus further includes display means for displaying the recorded image, and the display means displays an image corresponding to an image signal output from the second camera block whose recording is controlled by the second recording means. PinP display can be performed on an image corresponding to an image signal output from the first camera block.

  When imaging in the third mode, the detection signal output in accordance with the state of the characteristic region in the image corresponding to the image signal output from the first camera block, and the second camera block Based on the detection signal output according to the state of the characteristic area in the image corresponding to the output image signal, the second image together with the image corresponding to the image signal output from the first camera block. Third recording control means for controlling recording of an image corresponding to an image signal output from the camera block can be further provided.

  The second camera block may be configured as a mobile phone and used as a camera block used during a videophone call.

  One aspect of the present invention is that an image output from a first camera block including an optical system, a sensor, and a signal processing unit that collects light when the first recording control unit captures an image in the first mode. Controlling recording of an image corresponding to the image signal output from the first camera block based on a detection signal output in accordance with a state of a characteristic region in the image corresponding to the signal; When the recording control means captures an image in the second mode, it is output from a second camera block including an optical system, a sensor, and a signal processing unit that collects light at a position different from that of the first camera block. From the second camera block together with the image corresponding to the image signal output from the first camera block, based on the detection signal output according to the state of the characteristic region in the image corresponding to the image signal An imaging method comprising the step of controlling the recording of the image corresponding to the image signal strength.

  In one aspect of the present invention, when an image is captured in the first mode, an image corresponding to an image signal output from the first camera block including an optical system, a sensor, and a signal processing unit that collects light is included. When the recording of an image corresponding to the image signal output from the first camera block is controlled based on the detection signal output according to the state of the characteristic area of The state of the characteristic region in the image corresponding to the image signal output from the second camera block including the optical system, the sensor, and the signal processing unit that collects light at a position different from that of the first camera block And an image corresponding to the image signal output from the second camera block together with the image corresponding to the image signal output from the first camera block based on the detection signal output accordingly. Recording is controlled.

  According to the present invention, it is possible to reliably capture a photo opportunity with a simple configuration.

It is a block diagram which shows the structural example of the conventional digital camera. It is a figure which shows the example of the external appearance of the digital camera which concerns on one embodiment of this invention. It is a figure which shows the example of the external appearance of the digital camera which concerns on one embodiment of this invention. It is a block diagram which shows the internal structural example of the digital camera which concerns on one embodiment of this invention. It is a flowchart explaining the example of an image acquisition process. It is a figure explaining acquisition of an image in photographer mode. It is a figure explaining acquisition of an image in photographer mode. It is a figure which shows the example in case of PinP display. It is a block diagram which shows another example of an internal structure of the digital camera which concerns on one embodiment of this invention. It is a flowchart explaining another example of an image acquisition process. It is a flowchart explaining another example of an image acquisition process. It is a figure which shows the example of the external appearance of the mobile telephone which concerns on one embodiment of this invention. It is a figure which shows the example of the external appearance of the mobile telephone which concerns on one embodiment of this invention. It is a figure which shows the example of the external appearance of the mobile telephone which concerns on one embodiment of this invention. It is a block diagram which shows the structural example of the sub camera block of the mobile telephone of this invention. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

  Embodiments of the present invention will be described below with reference to the drawings.

  First, the configuration of a conventional digital camera will be described.

  FIG. 1 is a block diagram illustrating a configuration example of a conventional digital camera.

  In the figure, the digital camera 10 is provided with a main camera block 20. The main camera block 20 includes a lens / mechanical shutter 21, a lens driving unit 22, a sensor 23, and a signal processing unit 24.

  The lens / mechanical shutter 21 is configured as an optical system, for example, and is driven by a lens driving unit 22.

  The light condensed through the lens / mechanical shutter 21 is supplied to the sensor 23. The sensor 23 is constituted by a photoelectric conversion sensor such as a CMOS image sensor or a CCD image sensor, and photoelectrically converts light collected through the lens / mechanical shutter 21. The sensor 23 converts the analog electrical signal obtained by photoelectric conversion into a digital signal and outputs the digital signal to the signal processing unit 24.

  The signal processing unit 24 processes a signal supplied from the sensor 23. For example, the signal processing unit 24 converts a digital signal supplied from the sensor 23 into a color difference signal and outputs the color difference signal to the image processing unit 51.

  For example, the image processing unit 51 performs JPEG encoding / decoding processing, image scaling / resizing processing, and the like on the image signal supplied from the signal processing unit 24.

  For example, in the viewfinder mode in which the image of the subject is displayed on the liquid crystal panel 53 for confirmation before the image is acquired, the image processing unit 51 resizes the image signal supplied from the signal processing unit 24 and performs the liquid crystal panel controller 52. To be output. When acquiring an image of a subject, the image processing unit 51 performs JPEG encoding processing of the image based on the control of the control unit 30 and outputs the encoded image data to the memory manager 61. Yes.

  The liquid crystal panel controller 52 converts the image data output from the image processing unit 51 into a size and format that can be displayed on the liquid crystal panel 53 and displays the image on the liquid crystal panel 53.

  The memory manager 61 writes image data to the recording medium 62 or reads image data from the recording medium 62 as necessary. The memory manager 61 is also adapted to add thumbnails to image data, add Exif information, and the like.

  The main camera block 20 and the image processing unit 51 are controlled by the control unit 30.

  The control unit 30 is actually composed of, for example, a microcomputer. Here, only the block related to acquisition (imaging) of the image of the digital camera 10 in the control unit 30 will be described.

  Acquisition of the image of the digital camera 10 is performed by the capture controller 33 based on a control signal output from the capture manager 31.

  When the release button 41 is pressed, the capture manager 31 receives a release instruction signal and outputs a control signal for image acquisition to the capture controller 33.

  The capture controller 33 controls to acquire an image for each of the lens driving unit 22, the sensor 23, the signal processing unit 24, the image processing unit 51, and the memory manager 61 at a timing when a control signal is received from the capture manager 31. It is made to do. The acquired image data is finally written to the recording medium 62.

  Further, the control unit 30 is provided with a state detection unit 32. For example, the state detection unit 32 detects a subject in the image corresponding to the digital signal output from the signal processing unit 24. Here, the subject is, for example, a person's face.

  Then, the state detection unit 32 detects the state of the detected subject. Here, the detected state of the subject means, for example, a state in which a human face is smiling. That is, the state detection unit 32 is a functional block for realizing a function called so-called smile detection.

  Note that various inventions and documents are known for techniques such as face detection and smile detection, and the state detection unit 32 is realized by these techniques.

  For example, the state detection unit 32 detects that a person's face has become a smile and outputs a detection signal indicating that the person has become a smile to the capture manager 31. In addition, it is also possible to output the degree of smile (smile state) as a degree.

  The capture manager 31 outputs a control signal to the capture controller 33 at the timing when the detection signal is output from the state detection unit 32.

  In this way, the digital camera 10 can acquire an image at the moment when the person who is the subject smiles.

  However, as in the digital camera 10, using only the facial expression of the subject as a trigger for shooting functions effectively, for example, when shooting a person on the premise that there is sufficient shooting preparation time, but the subject to act on It may not be suitable for shooting.

  Therefore, in the present invention, not only the facial expression of the subject but also the facial expression of the photographer can be used as a shooting trigger.

  2 and 3 are diagrams showing an example of the appearance of a digital camera according to an embodiment of the present invention.

  As shown in FIG. 2, the digital camera 100 is provided with a subject photographing lens 102, a release button (shutter) 103, and a flash 104. The portion shown in FIG. 2 is a surface directed to the subject at the time of shooting.

  The portion shown in FIG. 3 corresponds to the back surface of the surface shown in FIG. 2 in the digital camera 100, and is a surface that is directed to the photographer side during shooting. As shown in FIG. 3, the digital camera 100 is provided with a photographer photographing lens 105, a liquid crystal panel 106, a zoom lever 107, a recorded image reproduction button 108, and an operation switch 109.

  The photographer photographing lens 105 is disposed at an upper portion of the liquid crystal panel 106 and is disposed at a position where light corresponding to an image of a user (photographer) who observes the liquid crystal panel 106 can be condensed. In other words, the photographer's photographing lens 105 can photograph an image of the upper body of the user who photographs the subject with the digital camera 100. The angle of view of the photographer photographing lens 105 is preferably, for example, at least 50 degrees horizontally.

  FIG. 4 is a block diagram illustrating a configuration example of the digital camera 100 according to the embodiment of the present invention.

  In the figure, the digital camera 100 is provided with a main camera block 120. The main camera block 120 includes a lens / mechanical shutter 121, a lens driving unit 122, a sensor 123, and a signal processing unit 124.

  The lens / mechanical shutter 121 is configured as an optical system, for example, and is driven by a lens driving unit 122. A part of the lens / mechanical shutter 121 is shown as a subject photographing lens 102 in FIG.

  The light condensed through the lens / mechanical shutter 121 is supplied to the sensor 123. The sensor 123 is constituted by a photoelectric conversion sensor such as a CMOS image sensor or a CCD image sensor, and photoelectrically converts light collected through the lens / mechanical shutter 121. The sensor 123 converts the analog electrical signal obtained by photoelectric conversion into a digital signal and outputs the digital signal to the signal processing unit 124.

  The signal processing unit 124 processes a signal supplied from the sensor 123. For example, the signal processing unit 124 converts a digital signal supplied from the sensor 123 into a color difference signal and outputs the color difference signal to the image processing unit 151.

  As shown in the figure, the digital camera 100 is provided with a sub camera block 180. The sub camera block 180 includes a lens 181, a sensor 182, and a signal processing unit 183.

  The lens 181 is configured as an optical system, for example. A part of the lens 181 is shown as the photographer photographing lens 105 in FIG.

  The light collected through the lens 181 is supplied to the sensor 182. The sensor 182 is composed of, for example, a photoelectric conversion sensor such as a CMOS image sensor or a CCD image sensor, and photoelectrically converts light collected through the lens 181. Then, the sensor 182 converts the analog electrical signal obtained by photoelectric conversion into a digital signal and outputs the digital signal to the signal processing unit 183.

  The signal processing unit 183 is configured to process the signal supplied from the sensor 182. For example, the signal processing unit 183 converts a digital signal supplied from the sensor 182 into a color difference signal and outputs the color difference signal to the image processing unit 151.

  For example, the image processing unit 151 performs JPEG encoding / decoding processing, image scaling / resizing processing, and the like on the image signal supplied from the signal processing unit 124 or the signal processing unit 183.

  For example, in the viewfinder mode in which the image of the subject is displayed on the liquid crystal panel 153 for confirmation before the image is acquired, the image processing unit 151 resizes the image signal supplied from the signal processing unit 124 and performs the liquid crystal panel controller 152. To be output. When acquiring the subject image or the photographer image, the image processing unit 151 performs JPEG encoding processing of the image under the control of the control unit 130 and outputs the encoded image data to the memory manager 161. It is made to do.

  The liquid crystal panel controller 152 converts the image data output from the image processing unit 151 into a size and format that can be displayed on the liquid crystal panel 153, and causes the liquid crystal panel 153 to display an image. The liquid crystal panel 153 corresponds to the liquid crystal panel 106 in FIG.

  The memory manager 161 writes image data to the recording medium 162 or reads image data from the recording medium 162 as necessary. The memory manager 161 also adds thumbnails to image data, adds Exif information, and the like.

  The main camera block 120, the sub camera block 180, and the image processing unit 151 are controlled by the control unit 130.

  The control unit 130 is actually composed of, for example, a microcomputer. Here, only the block related to acquisition (imaging) of the image of the digital camera 100 in the control unit 130 will be described.

  Acquisition of an image of the digital camera 100 is performed by the capture controller 133 based on a control signal output from the capture manager 131.

  When the release button 141 is pressed, the capture manager 131 receives a release instruction signal and outputs a control signal for image acquisition to the capture controller 133.

  The capture controller 133 performs control so as to acquire images for each of the lens driving unit 122, the sensor 123, the signal processing unit 124, the image processing unit 151, and the memory manager 161 at the timing when the control signal is received from the capture manager 131. It is made to do. The acquired image data is finally written to the recording medium 162.

  The control unit 130 is provided with a state detection unit 132. For example, the state detection unit 132 detects a subject in an image corresponding to the digital signal output from the signal processing unit 124 and an image corresponding to the digital signal output from the signal processing unit 183. Here, the subject is, for example, a person's face.

  Then, the state detection unit 132 detects the state of the detected subject. Here, the detected state of the subject means, for example, a state in which a human face is smiling. That is, the state detection unit 132 is a functional block for realizing a function called so-called smile detection.

  Note that various inventions and documents are known for techniques such as face detection and smile detection, and the state detection unit 132 is realized by these techniques.

  For example, the state detection unit 132 detects that a person's face has become a smile, and outputs a detection signal indicating that the person has become a smile to the capture manager 131. In the present invention, it is detected that a person's face in the image corresponding to the signal output from the main camera block 120 is smiling. In the present invention, it is detected that the photographer's face in the image corresponding to the signal output from the sub camera block 180 has become a smile. In addition, it is also possible to output the degree of smile (smile state) as a degree.

  The capture manager 131 outputs a control signal to the capture controller 133 at the timing when the detection signal is output from the state detection unit 132.

  In this way, the digital camera 100 can acquire an image at the moment when the person who is the subject smiles or when the photographer smiles.

  In the digital camera 100, for example, the shooting mode can be set by operating the operation switch 109. For example, when the shooting mode is set to the subject mode, an image at the moment when the person who is the subject smiles is acquired. For example, when the shooting mode is set to the photographer mode, an image at the moment when the photographer smiles is acquired.

  For example, when the shooting mode is set to the subject mode, only the image data of the person who is the subject acquired through the main camera block 120 is written to the recording medium 162. Also, for example, when the shooting mode is set to the photographer mode, the image data of the person who is the subject acquired via the main camera block 120 and the image of the photographer acquired via the sub camera block 180 are displayed. Data is written to the recording medium 162. When the photographing mode is set to the photographer mode, the image data of the person who is the subject and the image data of the photographer are recorded in association with each other.

  The shooting mode set in the digital camera 100 is stored in the capture manager 131 and the state detection unit 132.

  When the shooting mode is set to the subject mode, the state detection unit 132 detects that the person's face in the image corresponding to the signal output from the main camera block 120 has become a smile. When the photographer mode is set, the state detection unit 132 detects that the photographer's face in the image corresponding to the signal output from the sub camera block 180 becomes a smile. For example, when the state detection unit 132 outputs a signal according to the degree of smile, the detection signal is generated by determining the threshold value of the signal.

  Next, an example of image acquisition processing by the digital camera 100 will be described with reference to the flowchart of FIG.

  In step S21, the capture manager 131 determines whether the release button 141 has been pressed. If it is determined in step S21 that the release button 141 has not been pressed, the process proceeds to step S22.

  In step S22, the capture manager 131 determines whether or not the shooting mode is the photographer mode. If it is determined in step S22 that the shooting mode is the photographer mode, the process proceeds to step S23.

  In step S <b> 23, the capture manager 131 determines whether a detection signal is output from the state detection unit 132. That is, it is determined whether or not a photographer's smile (smile) has been detected.

  If it is determined in step S23 that the photographer's smile has been detected, the process proceeds to step S24, and the capture manager 131 outputs a control signal for image acquisition to the capture controller 133, so that the photographer's smile is detected. Get an image. At this time, the image processing unit 151 performs JPEG encoding processing on the image signal supplied from the signal processing unit 183 and outputs the encoded image data to the memory manager 161. The memory manager 161 writes the image data to the recording medium 162.

  After the process of step S24, the process proceeds to step S27, and the capture manager 131 causes the capture controller 133 to output a control signal for image acquisition, thereby acquiring an image of the subject. At this time, the image processing unit 151 performs JPEG encoding processing on the image signal supplied from the signal processing unit 124, and outputs the encoded image data to the memory manager 161. The memory manager 161 writes the image data to the recording medium 162. In this case, for example, the memory manager 161 adds the Exif information to the image data, and the image data of the subject acquired in the process of step S27 is added to the photographer acquired in the process of step S24. Recording is performed in association with image data.

  On the other hand, if it is determined in step S22 that the shooting mode is not the photographer mode, the process proceeds to step S25.

  In step S25, the capture manager 131 determines whether or not the shooting mode is the subject mode. If it is determined in step S25 that the shooting mode is the subject mode, the process proceeds to step S26.

  In step S <b> 26, the capture manager 131 determines whether a detection signal is output from the state detection unit 132. That is, it is determined whether a smile (smile) of the subject has been detected.

  If it is determined in step S26 that the smile of the subject has been detected, the process proceeds to step S27, and the capture manager 131 outputs a control signal for image acquisition to the capture controller 133, so that the subject image is displayed. Get it. At this time, the image processing unit 151 performs JPEG encoding processing on the image signal supplied from the signal processing unit 124, and outputs the encoded image data to the memory manager 161. The memory manager 161 writes the image data to the recording medium 162.

  On the other hand, if it is determined in step S21 that the release button 141 has been pressed, the process proceeds to step S27. In step S <b> 27, the capture manager 131 outputs a control signal for image acquisition to the capture controller 133 to acquire an image of the subject. At this time, the image processing unit 151 performs JPEG encoding processing on the image signal supplied from the signal processing unit 124, and outputs the encoded image data to the memory manager 161. The memory manager 161 writes the image data to the recording medium 162.

  If it is determined in step S23 that the photographer's smile has not been detected, or if it is determined in step S26 that the subject's smile has not been detected, the process ends.

  When the shooting mode is the photographer mode, for example, the detection of the photographer's smile is performed preferentially, and when the photographer's smile is not detected, the shooting is performed with the detection of the smile of the subject as a trigger. It may be. That is, if it is determined in step S23 that the photographer's smile has not been detected, the process may proceed to step S26.

  In this way, the image acquisition process is executed. As described above, according to the present invention, not only the detection of the smile of the subject can be used as a trigger for shooting but also the detection of the smile of the photographer can be used as a trigger for shooting.

  6 and 7 are diagrams for explaining image acquisition in the photographer mode. For example, as illustrated in FIG. 6, an example in which a user 211 who is a photographer acquires an image of a subject 212 will be described. The subject 212 shown in FIG. 6 is a penguin. That is, in this case, the user 211 who is a photographer is not trying to acquire an image with a person as a subject.

  As described above, in the photographer mode, the photographer's smile is detected and an image is acquired. That is, the user 211 acquires the image of the subject 212 while observing the image displayed on the liquid crystal panel 106 of the digital camera 100. At this time, an image of the upper body of the user 211 is continuously captured through the photographer photographing lens 105. When the smile of the user 211 is detected, the image of the subject 212 and the image of the user 211 are acquired and recorded.

  FIG. 7 a is a diagram illustrating an example of a photographer image acquired in the photographer mode. A smiling user 211 is shown in the image 221 in FIG. FIG. 7B is a diagram illustrating an example of an image of a subject acquired in the photographer mode. In the image 222 in the figure, the subject 212 when the smile of the user 211 is detected is shown. Note that the image 221 and the image 222 are recorded in association with each other. Therefore, for example, it is possible to reproduce the image of the subject while confirming the photographer's facial expression.

  Alternatively, for example, the image 221 may be combined with the upper left end of the image 222 and recorded as one image.

  As described above, according to the present invention, for example, even when shooting an object such as an animal, whose facial expression cannot be detected, it is possible to shoot without missing a photo opportunity. In particular, by acquiring an image at the moment when the photographer smiles, an image of the subject that the photographer wants can be acquired at a more natural timing.

  Further, by acquiring an image at the moment when the photographer smiles, the operability at the time of shooting at a higher shutter speed can be improved, for example. In this case, it is possible to prevent a photo opportunity from being missed even if the photographer does not stare at the subject. Furthermore, since there is no need for a mechanism for starting recording back several frames from the shutter release timing, the camera structure can be simplified.

  In addition, since the subject image acquired in the photographer mode and the photographer image at the moment when the image was triggered are recorded in association with each other, for example, the user can reproduce the acquired image. Can give more fun.

  For example, when an acquired image is played back, PinP (picture-in-picture) display may be performed on the image displayed on the liquid crystal panel 106.

  FIG. 8 is a diagram illustrating an example in which PinP display is performed. In FIG. 8, an image of the subject 212 (image 222 in FIG. 7 b) is displayed large on the liquid crystal panel 106. As shown in the figure, an image of the photographer (user 211) (image 221 in FIG. 7a) is displayed small in the upper left area 106a of the liquid crystal panel 106.

  By doing so, it is possible to give the user more enjoyment when reproducing the acquired image.

  Alternatively, the photographer's image may be used as a thumbnail image, and the image of the subject may be reproduced by causing the user to select a thumbnail image.

  Although the smile of the subject or the photographer is detected here, other facial expressions may be detected. For example, in the photographer mode, a surprised facial expression or the like may be detected, and an image of the moment when the photographer was surprised may be acquired. Alternatively, an action or the like may be detected instead of a facial expression.

  By the way, in the example described above with reference to FIG. 4, the state detection unit 132 detects that the face of the subject has become a smile (smile) in the subject mode, and in the photographer mode, the state of the photographer. He explained that he would detect that his face was smiling. However, if it is detected that the subject's face is smiling and the photographer's face is smiling, it is considered that a more enjoyable image can be acquired for the user.

  FIG. 9 is a block diagram illustrating a configuration example of the digital camera 100 when it is detected that the subject's face is smiling and the photographer's face is smiling. 4 is a diagram corresponding to FIG. 4, and the same functional blocks as those in FIG. 4 are denoted by the same reference numerals, and thus detailed description of these functional blocks is omitted.

  In the configuration of FIG. 9, unlike the example of FIG. 4, the sub camera block 180 is provided with a state detection unit 184. Similarly to the state detection unit 132, the state detection unit 184 detects a subject in the image corresponding to the digital signal output from the signal processing unit 183, for example. In this case, the subject is the photographer's image. The state detection unit 184 detects the state of the detected subject. Here, the detected state of the subject means, for example, a state in which the photographer's face is smiling. That is, the state detection unit 184 is a functional block for realizing a function called so-called smile detection. In addition, it is also possible to output the degree of smile (smile state) as a degree.

  For example, the state detection unit 184 detects that the photographer's face has become a smile, and outputs a detection signal indicating that the photographer has become a smile to the capture manager 131.

  In the case of the example of FIG. 9, the capture manager 131 outputs a control signal to the capture controller 133 at a timing when detection signals are simultaneously output from the state detection unit 132 and the state detection unit 184.

  In other words, in the example of FIG. 9, the digital camera 100 detects both the subject smile and the photographer smile to acquire an image. In this way, the shooting mode in which both the subject smile and the photographer smile are detected and an image is acquired is referred to as a double smile mode.

  Next, an example of image acquisition processing when the digital camera 100 is configured as shown in FIG. 9 will be described with reference to the flowcharts of FIGS. 10 and 11.

  In step S51, the capture manager 131 determines whether or not the release button 141 has been pressed. If it is determined in step S51 that the release button 141 has not been pressed, the process proceeds to step S52.

  In step S52, the capture manager 131 determines whether or not the shooting mode is the double smile mode. If it is determined in step S52 that the shooting mode is the double smile mode, the process proceeds to step S53.

  In step S53, the capture manager 131 determines whether or not a detection signal is output from the state detection unit 184, and waits until it is determined that a detection signal is output from the state detection unit 184. That is, in step S53, it is determined whether or not a photographer's smile (smile) has been detected.

  If it is determined in step S53 that the photographer's smile has been detected, the process proceeds to step S54.

  In step S <b> 54, the capture manager 131 determines whether a detection signal is output from the state detection unit 132. That is, in step S54, it is determined whether or not a smile (smile) of the subject has been detected.

  If it is determined in step S54 that the smile of the subject has been detected, the process proceeds to step S55, and the capture manager 131 outputs a control signal for acquiring an image to the capture controller 133, whereby the image of the photographer is captured. To get. At this time, the image processing unit 151 performs JPEG encoding processing on the image signal supplied from the signal processing unit 183 and outputs the encoded image data to the memory manager 161. The memory manager 161 writes the image data to the recording medium 162.

  After the process of step S55, the process proceeds to step S56, and the capture manager 131 outputs a control signal for image acquisition to the capture controller 133, thereby acquiring an image of the subject. At this time, the image processing unit 151 performs JPEG encoding processing on the image signal supplied from the signal processing unit 124, and outputs the encoded image data to the memory manager 161. The memory manager 161 writes the image data to the recording medium 162. In this case, for example, the memory manager 161 adds the Exif information to the image data, and the image data of the subject acquired in the process of step S56 is obtained from the photographer acquired in the process of step S55. Recording is performed in association with image data.

  On the other hand, if it is determined in step S51 that the release button 141 has been pressed, the process proceeds to step S56. In step S <b> 56, the capture manager 131 causes the capture controller 133 to acquire a subject image by outputting a control signal for image acquisition. At this time, the image processing unit 151 performs JPEG encoding processing on the image signal supplied from the signal processing unit 124, and outputs the encoded image data to the memory manager 161. The memory manager 161 writes the image data to the recording medium 162.

  If it is determined in step S54 that the subject's smile has not been detected, the process ends.

  On the other hand, if it is determined in step S52 that the mode is not the double smile mode, the process proceeds to step S62 in FIG. Since the processing from step S62 to step S67 in FIG. 11 is the same as the processing from step S22 to step S27 in FIG. 5, respectively, detailed description thereof is omitted.

  In this way, the image acquisition process is executed. As described above, since the double smile mode for detecting both the smile of the subject and the smile of the photographer is provided, it is possible to acquire a more enjoyable image for the user.

  In the case of the example of FIG. 9, in the double smile mode, the capture manager 131 outputs the control signal to the capture controller 133 at the timing when the detection signal is simultaneously output from the state detection unit 132 and the state detection unit 184. did. However, for example, the capture manager 131 may output a control signal to the capture controller 133 at a timing when a detection signal is output from any one of the state detection unit 132 or the state detection unit 184. Of course it is possible.

  In this case, in the shooting in the double smile mode, either the subject smile or the photographer smile is used as a shooting trigger to acquire the subject image and the photographer image.

  In the above, an example in which the present invention is applied to a digital camera has been described. However, for example, the present invention can also be applied to a mobile phone.

  12 to 14 are diagrams showing an example of the appearance of the mobile phone according to one embodiment of the present invention. The mobile phone 200 has a function of calling with other mobile phones, and also functions as a digital camera that acquires images.

  As shown in FIG. 12, a photographer-capturing lens 202 and a liquid crystal panel 203 are provided on the surface on which the numeric keypad of the mobile phone 200 is arranged. A release (shutter) button 204 is provided on the side surface of the mobile phone 200.

  For example, when a user makes a call with another mobile phone using the mobile phone 200, the vertical direction of the mobile phone 200 is held in the same manner as in the drawing with the surface shown in FIG. 12 facing the user side.

  When an image is acquired by the mobile phone 200, as shown in FIG. 13, the mobile phone 200 is held in a state rotated by 90 degrees. For example, when the state shown in FIG. 12 is the portrait orientation of the mobile phone 200, the mobile phone 200 is landscape orientation in the state shown in FIG.

  The portion shown in FIG. 14 corresponds to the back surface of the surface shown in FIG. 13 of the mobile phone 200, and is the surface that is directed to the subject side during shooting. As shown in the figure, the cellular phone 200 is provided with a subject photographing lens 206 and an LED light 205.

  The internal configuration of the mobile phone 200 is the same as that described above with reference to FIG. 4 or FIG. However, for example, the sub camera block 180 of FIG. 4 is configured as shown in FIG.

  FIG. 15 is a block diagram illustrating a configuration example of a sub camera block of the mobile phone 200. In the case of the example in the figure, unlike the case described above with reference to FIG. 4, an image rotation unit 185 is provided.

  As described above with reference to FIG. 13, when an image is acquired by the mobile phone 200, as shown in FIG. 13, the mobile phone 200 is held in a state rotated by 90 degrees. On the other hand, in the mobile phone 200, an image photographed through the sub camera block 180 is also used as an image during a videophone call, for example. Therefore, in the configuration in which the present invention is applied to a mobile phone, the aspect ratio is reversed between the pixel arrangement in the sensor 123 of the main camera block 120 and the pixel arrangement in the sensor 182 of the sub camera block 180. For example, in the sensor 182, when the liquid crystal panel 203 of FIG. 12 is vertically arranged, pixels are arranged so as to form a horizontal 4: vertical 3 imaging surface.

  Therefore, the image rotation unit 185 is an image corresponding to the image signal output from the signal processing unit 183 and rotates the photographer's image by 90 degrees.

  Other configurations are the same as those described above with reference to FIG. 4 or FIG. In this case, a part of the lens / mechanical shutter 121 is shown as a subject photographing lens 206 in FIG. A part of the lens 181 is shown as a photographer photographing lens 202 in FIGS.

  Thus, the present invention can be applied to a mobile phone. In mobile phones, for example, a self-portrait camera for a videophone (that is, a sub camera block) is often attached. In addition, mobile phones often have a face detection and smile detection function by software processing for images taken with a videophone selfie camera. Therefore, when the present invention is applied to a mobile phone, for example, it is unnecessary to change the design of an existing mobile phone, and as a result, the configuration of the present invention can be realized at low cost.

  Alternatively, the present invention can be applied to a camcorder such as a video camera.

  The series of processes described above can be executed by hardware, or can be executed by software. When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium into a computer incorporated in dedicated hardware. Further, by installing various programs, the program is installed from a network or a recording medium into a general-purpose personal computer 700 as shown in FIG. 16 that can execute various functions.

  In FIG. 16, a CPU (Central Processing Unit) 701 executes various processes according to a program stored in a ROM (Read Only Memory) 702 or a program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. To do. The RAM 703 also appropriately stores data necessary for the CPU 701 to execute various processes.

  The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output interface 705 is also connected to the bus 704.

  The input / output interface 705 is connected to an input unit 706 composed of a keyboard, a mouse, etc., a display composed of an LCD (Liquid Crystal display), etc., and an output unit 707 composed of a speaker. The input / output interface 705 is connected to a storage unit 708 composed of a hard disk and a communication unit 709 composed of a network interface card such as a modem and a LAN card. The communication unit 709 performs communication processing via a network including the Internet.

  A drive 710 is also connected to the input / output interface 705 as necessary, and a removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted. Then, the computer program read from these removable media is installed in the storage unit 708 as necessary.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network such as the Internet or a recording medium such as a removable medium 711.

  The recording medium shown in FIG. 16 is a magnetic disk (including a floppy disk (registered trademark)) on which a program is recorded, which is distributed to distribute the program to the user separately from the apparatus main body, Removable media consisting of optical disks (including CD-ROM (compact disk-read only memory), DVD (digital versatile disk)), magneto-optical disks (including MD (mini-disk) (registered trademark)), or semiconductor memory It includes not only those configured by 711 but also those configured by a ROM 702 storing a program, a hard disk included in the storage unit 708, and the like that are distributed to the user in a state of being incorporated in the apparatus main body in advance.

  Note that the series of processes described above in this specification includes processes that are performed in parallel or individually even if they are not necessarily processed in time series, as well as processes that are performed in time series in the order described. Is also included.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  100 Digital Camera, 102 Subject Shooting Lens, 103 Release Button, 105 Photographer Shooting Lens, 120 Main Camera Block, 121 Lens / Mechanical Shutter, 122 Lens Drive Unit, 123 Sensor, 124 Signal Processing Unit, 130 Control Unit, 131 Capture Manager, 132 state detection unit, 133 capture controller, 141 release button, 180 sub camera block, 181 lens 182 sensor, 183 signal processing unit, 184 state detection unit, 185 image rotation unit, 200 mobile phone, 202 lens for photographer photographing , 204 Release button, 206 Subject shooting lens

Claims (7)

A first camera block comprising an optical system for condensing light, a sensor, and a signal processing unit;
A second camera block including an optical system, a sensor, and a signal processing unit that collects light at a position different from that of the first camera block;
A characteristic region in an image corresponding to an image signal output from the first camera block or the second camera block, or a feature in an image corresponding to an image signal output from the second camera block State detection means for outputting a detection signal according to the state of the target area;
When imaging in the first mode, the first camera is based on a detection signal output in accordance with a state of a characteristic region in an image corresponding to an image signal output from the first camera block. First recording control means for controlling recording of an image corresponding to an image signal output from the block;
When imaging in the second mode, the first camera is based on a detection signal output in accordance with a state of a characteristic region in an image corresponding to an image signal output from the second camera block. An image pickup apparatus comprising: a second recording control unit that controls recording of an image corresponding to the image signal output from the second camera block together with an image corresponding to the image signal output from the block. The optical system of the first camera block collects light in a direction in which the subject is located;
The imaging apparatus according to claim 1, wherein the optical system of the second camera block collects light in a direction in which a photographer is located. The detection means includes
The imaging apparatus according to claim 2, wherein a human face area is detected as the characteristic area, and the detection signal is output in accordance with a degree of smile of the face. Further comprising display means for displaying the recorded image,
The display means corresponds to an image signal output from the second camera block whose recording is controlled by the second recording means, and corresponds to an image signal output from the first camera block. The imaging device according to claim 1, wherein PinP display is performed on an image. When imaging in the third mode, the detection signal output in accordance with the state of the characteristic region in the image corresponding to the image signal output from the first camera block, and the second camera block Based on the detection signal output according to the state of the characteristic area in the image corresponding to the output image signal, the second image together with the image corresponding to the image signal output from the first camera block. The imaging apparatus according to claim 1, further comprising a third recording control unit that controls recording of an image corresponding to an image signal output from the camera block. Configured as a mobile phone,
The imaging device according to claim 1, wherein the second camera block is a camera block used during a videophone call. When the first recording control means captures an image in the first mode, an image corresponding to an image signal output from a first camera block including an optical system, a sensor, and a signal processing unit that collects light is included. Based on the detection signal output according to the state of the characteristic area of the control of the recording of the image corresponding to the image signal output from the first camera block,
When the second recording control means captures an image in the second mode, the second recording block includes an optical system, a sensor, and a signal processing unit that collects light at a position different from that of the first camera block. Based on the detection signal output according to the state of the characteristic area in the image corresponding to the output image signal, the second image together with the image corresponding to the image signal output from the first camera block. An imaging method including a step of controlling recording of an image corresponding to an image signal output from a camera block.

Images