WO2009113383A1 - 撮像装置及び画像再生装置 - Google Patents

撮像装置及び画像再生装置 Download PDF

Info

Publication number
WO2009113383A1
WO2009113383A1 PCT/JP2009/053243 JP2009053243W WO2009113383A1 WO 2009113383 A1 WO2009113383 A1 WO 2009113383A1 JP 2009053243 W JP2009053243 W JP 2009053243W WO 2009113383 A1 WO2009113383 A1 WO 2009113383A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
face
composition adjustment
composition
imaging
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2009/053243
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
幸夫 森
安八 濱本
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sanyo Electric Co Ltd
Original Assignee
Sanyo Electric Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sanyo Electric Co Ltd filed Critical Sanyo Electric Co Ltd
Priority to US12/921,904 priority Critical patent/US20110007187A1/en
Publication of WO2009113383A1 publication Critical patent/WO2009113383A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present invention relates to an imaging apparatus such as a digital still camera and an image reproduction apparatus for reproducing an image.
  • composition setting at the time of shooting is particularly difficult for beginners of shooting, and images having a good composition (for example, highly artistic images) cannot be obtained under shooting conditions (including composition) determined by the user. There are also many. If an image having a good composition according to the state of the subject can be automatically acquired, it is beneficial for the user.
  • This method is a method for obtaining an image for face authentication. According to this method, for example, an image centered on the face can be obtained, which may be useful as a face authentication technique. However, it is difficult to say that the composition of such an image is excellent when photographing a person by a general user.
  • the zoom lens is driven and controlled to have a wider angle of view than the user's set angle of view, and then a wide angle image is captured by the CCD, and a plurality of images are cut out from the wide angle image. It has been proposed (see Patent Document 3 below). However, with this method, the cutout position is set regardless of the state of the subject, and thus an image having a composition corresponding to the state of the subject cannot be obtained.
  • an object of the present invention is to provide an imaging device that contributes to acquisition of an image having a good composition according to the state of a subject. It is another object of the present invention to provide an image reproduction apparatus that contributes to reproduction of an image having a good composition according to the state of a subject included in an input image.
  • the first image pickup apparatus includes an image pickup device that outputs a signal corresponding to an optical image projected on itself by photographing, an image moving unit that moves the optical image on the image pickup device, and the image pickup device. Detecting a face of a person as a subject from a determination image based on the output signal and detecting a position and orientation of the face on the determination image, and detecting the detected position and orientation of the face And a composition control unit configured to control the image moving unit based on the output signal and generate a composition adjustment image from the output signal of the imaging device after the control.
  • composition adjustment image having a good composition in which the composition is adjusted according to the position and orientation of the face.
  • the composition control means controls the image moving means so that a target point corresponding to the detected position of the face is arranged at a specific position on the composition adjustment image.
  • the specific position is set based on the orientation of the face.
  • the composition control means sets the specific position from the opposite side of the direction in which the face is facing, with the center of the composition adjustment image as a reference.
  • the specific position is formed by two lines that divide the composition adjustment image into three equal parts in the horizontal direction and two lines that divide the composition adjustment image into three equal parts in the vertical direction. Any one of the four intersection positions.
  • composition control means generates one or more composition adjustment images as the composition adjustment image, and determines the number of the composition adjustment images to be generated based on the detected face orientation.
  • the composition control means may be configured such that the detected face orientation is front-facing.
  • m specific positions different from each other are set to generate a total of m composition adjustment images corresponding to the m specific positions
  • one specific position when the detected orientation of the face is horizontal is set to generate one composition adjustment image
  • n different specific positions are set to generate a total of n composition adjustment images corresponding to the n specific positions.
  • the first imaging apparatus includes: a shooting instruction receiving unit that receives a shooting instruction from outside; and a recording control unit that performs recording control for recording image data based on an output signal of the imaging element on a recording medium.
  • the composition control unit further generates the composition adjustment image according to the photographing instruction, and generates a basic image different from the composition adjustment image from the output signal of the image sensor, and the recording control unit includes the composition adjustment image.
  • An image and image data of the basic image are associated with each other and recorded on the recording medium.
  • a second imaging device is configured to output an image sensor that outputs a signal corresponding to an optical image projected on itself by photographing, and to detect a human face as a subject from a determination image based on the output signal of the image sensor.
  • a face detection unit that detects and detects the position and orientation of the face on the image for determination, and an image different from the image for determination obtained from the image for determination or the output signal of the image sensor is handled as a basic image
  • a composition control unit that generates a composition adjustment image by cutting out a part of the basic image, and the composition control unit extracts the composition adjustment image based on the detected position and orientation of the face. It is characterized by controlling.
  • composition adjustment image having a good composition in which the composition is adjusted according to the position and orientation of the face.
  • the composition control means sets the cut-out position so that a target point corresponding to the detected position of the face is arranged at a specific position on the composition adjustment image.
  • the specific position is set based on the detected orientation of the face.
  • the second imaging apparatus includes: a shooting instruction receiving unit that receives a shooting instruction from outside; and a recording control unit that performs recording control for recording image data based on an output signal of the imaging element on a recording medium.
  • the composition control means generates the basic image and the composition adjustment image according to the photographing instruction, and the recording control means associates the image data of the composition adjustment image and the basic image with each other on the recording medium. Let me record.
  • An image reproduction device is obtained by detecting a human face from an input image and detecting a position and orientation of the face on the input image, and cutting out a part of the input image.
  • Composition control means for outputting image data of the composition adjustment image, wherein the composition control means controls the cut-out position of the composition adjustment image based on the detected position and orientation of the face.
  • an imaging device that contributes to acquisition of an image having a good composition according to the state of the subject.
  • an image reproduction apparatus that contributes to reproduction of an image having a good composition according to the state of the subject included in the input image.
  • FIG. 1 is an overall block diagram of an imaging apparatus according to an embodiment of the present invention. It is an internal block diagram of the imaging part of FIG. (A) And (b) is a figure which shows the mode of the movement of the optical image on an image pick-up element by the movement of the correction lens of FIG. It is a figure for defining up and down, right and left about an image.
  • FIG. 2 is a partial functional block diagram of the imaging apparatus in FIG. 1 involved in a first composition adjustment shooting operation.
  • (A), (b), and (c) are the figures which respectively show the face facing front, the face facing left, and the face facing right in the image. It is a flowchart showing the flow of the 1st composition adjustment photography operation.
  • FIG. (A), (b), (c), (d), and (e) are basic images generated by the first composition adjustment photographing operation, and the first, second, third, and fourth compositions, respectively. It is a figure which shows an adjustment image.
  • FIG. 6 is a partial functional block diagram of the imaging apparatus in FIG. 1 involved in a third composition adjustment shooting operation. 12 is a flowchart illustrating a flow of a third composition adjustment photographing operation.
  • FIG. 2 is a partial functional block diagram of the imaging apparatus of FIG. 1 involved in an automatic trimming reproduction operation. It is a flowchart showing the flow of automatic trimming reproduction
  • FIG. 1 is an overall block diagram of an imaging apparatus 1 according to an embodiment of the present invention.
  • the imaging device 1 is a digital video camera, for example.
  • the imaging device 1 can shoot moving images and still images, and can also shoot still images simultaneously during moving image shooting. Note that the moving image shooting function may be omitted, and the imaging apparatus 1 may be a digital still camera capable of shooting only a still image.
  • the imaging apparatus 1 includes an imaging unit 11, an AFE (Analog Front End) 12, a video signal processing unit 13, a microphone 14, an audio signal processing unit 15, a compression processing unit 16, and a DRAM (Dynamic Random Access Memory).
  • an internal memory 17 such as an SDRAM (Synchronous Dynamic Random Access Memory)
  • an external memory 18 such as an SD (Secure Digital) card or a magnetic disk
  • a decompression processing unit 19 a video output circuit 20, an audio output circuit 21, A TG (timing generator) 22, a CPU (Central Processing Unit) 23, a bus 24, a bus 25, an operation unit 26, a display unit 27, and a speaker 28 are provided.
  • the operation unit 26 includes a recording button 26a, a shutter button 26b, an operation key 26c, and the like. Each part in the imaging apparatus 1 exchanges signals (data) between the parts via the bus 24 or 25.
  • the TG 22 generates a timing control signal for controlling the timing of each operation in the entire imaging apparatus 1, and gives the generated timing control signal to each unit in the imaging apparatus 1.
  • the timing control signal includes a vertical synchronization signal Vsync and a horizontal synchronization signal Hsync.
  • the CPU 23 comprehensively controls the operation of each unit in the imaging apparatus 1.
  • the operation unit 26 receives an operation by a user. The operation content given to the operation unit 26 is transmitted to the CPU 23.
  • Each unit in the imaging apparatus 1 temporarily records various data (digital signals) in the internal memory 17 during signal processing as necessary.
  • FIG. 2 is an internal configuration diagram of the imaging unit 11 of FIG.
  • the imaging device 1 is configured to generate a color image by shooting.
  • the imaging unit 11 includes an optical system 35, a diaphragm 32, an imaging element 33, and a driver 34.
  • the optical system 35 includes a plurality of lenses including a zoom lens 30, a focus lens 31, and a correction lens 36.
  • the zoom lens 30 and the focus lens 31 are movable in the optical axis direction, and the correction lens 36 is movable in a direction having an inclination with respect to the optical axis.
  • the correction lens 36 is installed in the optical system 35 so as to be movable on a two-dimensional plane orthogonal to the optical axis.
  • the driver 34 drives and controls the positions of the zoom lens 30 and the focus lens 31 and the opening of the diaphragm 32 based on a control signal from the CPU 23, so that the focal length (angle of view), the focal position of the imaging unit 11, The amount of light incident on the image sensor 33 is controlled. Incident light from the subject enters the image sensor 33 through the lenses and the diaphragm 32 constituting the optical system 35. Each lens constituting the optical system 35 forms an optical image of the subject on the image sensor 33.
  • the TG 22 generates a drive pulse for driving the image sensor 33 in synchronization with the timing control signal, and applies the drive pulse to the image sensor 33.
  • the image sensor 33 is composed of, for example, a CCD (Charge Coupled Devices), a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or the like.
  • the image sensor 33 photoelectrically converts an optical image incident through the optical system 35 and the diaphragm 32 and outputs an electrical signal obtained by the photoelectric conversion to the AFE 12.
  • the image sensor 33 includes a plurality of light receiving pixels arranged two-dimensionally in a matrix, and in each photographing, each light receiving pixel stores a signal charge having a charge amount corresponding to the exposure time.
  • the electrical signal from each light receiving pixel having a magnitude proportional to the amount of the stored signal charge is sequentially output to the subsequent AFE 12 in accordance with the drive pulse from the TG 22.
  • the magnitude (intensity) of the electrical signal from the image sensor 33 increases in proportion to the exposure time.
  • the AFE 12 amplifies the analog signal output from the image sensor 33, converts the amplified analog signal into a digital signal, and outputs the digital signal to the video signal processing unit 13.
  • the degree of amplification of signal amplification in the AFE 12 is controlled by the CPU 23.
  • the video signal processing unit 13 performs various types of image processing on the image represented by the output signal of the AFE 12, and generates a video signal for the image after the image processing.
  • the video signal is composed of a luminance signal Y representing the luminance of the image and color difference signals U and V representing the color of the image.
  • the microphone 14 converts the ambient sound of the imaging device 1 into an analog audio signal
  • the audio signal processing unit 15 converts the analog audio signal into a digital audio signal.
  • the compression processing unit 16 compresses the video signal from the video signal processing unit 13 using a predetermined compression method.
  • the compressed video signal is recorded in the external memory 18 at the time of shooting and recording a moving image or a still image.
  • the compression processing unit 16 compresses the audio signal from the audio signal processing unit 15 using a predetermined compression method.
  • the video signal from the video signal processing unit 13 and the audio signal from the audio signal processing unit 15 are compressed while being associated with each other in time by the compression processing unit 16, and after compression, Recorded in the external memory 18.
  • the recording button 26a is a push button switch for instructing start / end of moving image shooting and recording
  • the shutter button 26b is a push button switch for instructing shooting and recording of a still image.
  • the operation mode of the imaging apparatus 1 includes a shooting mode capable of shooting moving images and still images, and a playback mode for reproducing and displaying moving images and still images stored in the external memory 18 on the display unit 27. Transition between the modes is performed according to the operation on the operation key 26c.
  • shooting mode shooting is sequentially performed at a predetermined frame period, and an image sequence arranged in time series is acquired from the image sensor 33. Each image forming this image sequence is called a “frame image”.
  • the recording button 26a When the user presses the recording button 26a in the shooting mode, under the control of the CPU 23, the video signal of each frame image and the corresponding audio signal obtained after the pressing are sequentially sent via the compression processing unit 16 to the external memory 18. To be recorded.
  • the recording button 26a again after starting the moving image shooting the recording of the video signal and the audio signal in the external memory 18 is finished, and the shooting of one moving image is completed.
  • the shutter button 26b In the shooting mode, when the user presses the shutter button 26b, a still image is shot and recorded.
  • the compressed video signal representing the moving image or the still image recorded in the external memory 18 is expanded by the expansion processing unit 19 and then the video output circuit. 20 is sent.
  • the video signal processing unit 13 In the shooting mode, the video signal processing unit 13 normally generates video signals sequentially regardless of the operation contents of the recording button 26 a and the shutter button 26 b, and the video signals are sent to the video output circuit 20. It is done.
  • the video output circuit 20 converts the given digital video signal into a video signal (for example, an analog video signal) in a format that can be displayed on the display unit 27 and outputs the video signal.
  • the display unit 27 is a display device including a liquid crystal display panel and an integrated circuit that drives the liquid crystal display panel, and displays an image corresponding to the video signal output from the video output circuit 20.
  • a compressed audio signal corresponding to the moving image recorded in the external memory 18 is also sent to the expansion processing unit 19.
  • the decompression processing unit 19 decompresses the received audio signal and sends it to the audio output circuit 21.
  • the audio output circuit 21 converts a given digital audio signal into an audio signal in a format that can be output by the speaker 28 (for example, an analog audio signal) and outputs the audio signal to the speaker 28.
  • the speaker 28 outputs the sound signal from the sound output circuit 21 to the outside as sound.
  • the video signal from the video output circuit 20 and the audio signal from the audio output circuit 21 are supplied to an external device (such as an external display device) via an external output terminal (not shown) provided in the imaging device 1. It is also possible.
  • the shutter button 26b can be pressed in two stages. When the photographer lightly presses the shutter button 26b, the shutter button 26b is half pressed, and when the shutter button 26b is further pressed from this state. The shutter button 26b is fully pressed.
  • the correction lens 36 is installed in the optical system 35 so as to be movable on a two-dimensional plane orthogonal to the optical axis. For this reason, as the correction lens 36 moves, the optical image projected onto the image sensor 33 moves on the image sensor 33 in a two-dimensional direction parallel to the imaging surface of the image sensor 33.
  • the imaging surface is a surface on which an optical image is projected, on which each light receiving pixel of the image sensor 33 is arranged.
  • the CPU 23 outputs a lens shift signal for changing the position of the correction lens 36 to the driver 34, and the driver 34 moves the correction lens 36 according to the lens shift signal. Since the optical axis changes due to the movement of the correction lens 36, the control for moving the correction lens 36 is called optical axis shift control.
  • FIGS. 3A and 3B show how the optical image is moved by moving the correction lens 36.
  • FIG. Light from the point 200 that is stationary in the real space enters the image sensor 33 through the correction lens 36, and an optical image of the point 200 is formed at a certain point on the image sensor 33.
  • the optical image is formed at the point 201 on the image sensor 33, but the position of the correction lens 36 is changed from the state of FIG. 3A to the state of FIG. 3B. Is changed, the optical image is formed at a point 202 on the image sensor 33 that is different from the point 201.
  • the top, bottom, left and right are defined (this definition is common to all images).
  • the image is a two-dimensional image having a rectangular outer shape. Assuming a two-dimensional orthogonal coordinate plane with the X axis and Y axis orthogonal to each other as coordinate axes, one vertex of the image is arranged at the origin O on the coordinate plane. Images are arranged in the positive direction of the X axis and the positive direction of the Y axis with the origin O as a base point.
  • the side toward the negative direction of the X axis is on the left side
  • the side toward the positive direction of the X axis is on the right side
  • the side toward the negative direction of the Y axis is on the upper side
  • the positive direction on the Y axis The side that faces is the lower side.
  • the horizontal direction matches the horizontal direction of the image
  • the vertical direction matches the vertical direction of the image.
  • the imaging device 1 can execute a characteristic operation.
  • This characteristic operation is called composition adjustment photographing operation.
  • the composition adjustment photographing operation the first to third composition adjustment photographing operations will be individually described below. If no contradiction arises, it is possible to apply the matters described in a certain composition adjustment photographing operation to other composition adjustment photographing operations.
  • description of image data may be omitted for simplification of description in a sentence explaining that some processing (recording, saving, reading, etc.) is performed on image data of a certain image. is there.
  • the expression “recording image data of a still image” is synonymous with the expression “recording of a still image”.
  • FIG. 5 is a partial functional block diagram of the imaging apparatus 1 involved in the first composition adjustment shooting operation.
  • the functions of the face detection unit 51 and the image acquisition unit 53 are mainly realized by the video signal processing unit 13 of FIG. 1
  • the function of the shooting control unit 52 is mainly realized by the CPU 23 of FIG. 1
  • the function of the recording control unit 54 is mainly the CPU 23.
  • the compression control unit 16 is also involved in realizing the functions of the parts referenced by reference numerals 51 to 54 as necessary.
  • the face detection unit 51 detects the face of a person from the input image based on the image data of the input image given to itself, and extracts a face area including the detected face.
  • Various methods are known as a method for detecting a face included in an image, and the face detection unit 51 can employ any method.
  • a face (face region) may be detected by extracting a skin color region from an input image, as in the method described in Japanese Patent Application Laid-Open No. 2000-105819, or Japanese Patent Application Laid-Open No. 2006-211139 or Japanese Patent Application Laid-Open No. 2006.
  • a face (face area) may be detected using the method described in Japanese Patent No. -72770.
  • the image of the region of interest set in the input image is compared with a reference face image having a predetermined image size to determine the similarity between both images, and the region of interest is determined based on the similarity. It is detected whether or not a face is included (whether or not the region of interest is a face region).
  • the region of interest is shifted in the horizontal direction or the vertical direction pixel by pixel.
  • the similarity between both images is determined again, and the same detection is performed.
  • the region of interest is updated and set, for example, while being shifted pixel by pixel from the upper left to the lower right of the input image.
  • the input image is reduced at a certain rate, and the same face detection process is performed on the reduced image. By repeating such processing, a face of any size can be detected from the input image.
  • the face detection unit 51 also detects the orientation of the face in the input image. That is, the face detection unit 51 can detect whether the face orientation detected in the input image is a front orientation, a left orientation, or a right orientation. The left direction and the right direction belong to the horizontal direction. As shown in FIG. 6 (a), when the face in the image appears as a face seen from the front, the face direction is detected to be front-facing, and in the image as shown in FIG. 6 (b). When a face appears as a face facing leftward, the face orientation is detected as being leftward, and as shown in FIG. 6C, the face in the image appears as a face facing rightward. The face is detected as facing right.
  • the front-facing face faces in the direction perpendicular to both the X-axis and the Y-axis
  • the left-facing face faces in the negative direction of the X-axis
  • the right-facing face faces in the positive direction of the X-axis.
  • the face detection unit 51 can employ any method. For example, as in the technique described in Japanese Patent Application Laid-Open No. 10-307923, face parts such as eyes, nose and mouth are found in order from the input image, and the position of the face in the image is detected. The direction of the face is detected based on the projection data of the part.
  • one front-facing face is divided into a left half (hereinafter referred to as a left face) and a right half (hereinafter referred to as a right face), and parameters relating to the left face and parameters relating to the right face are learned through learning processing. Is generated in advance.
  • the region of interest in the input image is divided into left and right, and the similarity between each divided region and the corresponding parameter of the two parameters is calculated. Then, when one or both of the similarities are equal to or greater than the threshold, it is determined that the region of interest is a face region. Furthermore, the orientation of the face is detected from the magnitude relationship of the similarity to each divided region.
  • the face detection unit 51 outputs face detection information representing the result of face detection by itself.
  • the face detection information for the input image specifies “face position, face orientation, and face size” on the input image.
  • the face detection unit 51 extracts a rectangular area including the face as a face area, and expresses the position and size of the face by the position and image size of the face area on the input image.
  • the face position indicates, for example, the center position of the face area for the face.
  • Face detection information for the input image is given to the imaging control unit 52 in FIG. If no face is detected by the face detection unit 51, face detection information is not generated and output, but instead, information indicating that fact is transmitted to the imaging control unit 52.
  • the imaging control unit 52 outputs a lens shift signal for obtaining a composition adjustment image to the driver 34 in FIG. 2 based on the face detection information.
  • the image acquisition unit 53 generates a basic image and a composition adjustment image from the output signal of the image sensor 33 (in other words, acquires image data of these images). The significance of the basic image and the composition adjustment image will become clear from the following description.
  • the recording control unit 54 records the image data of the basic image and the composition adjustment image in the external memory 18 in association with each other.
  • FIG. 7 is a flowchart showing the flow of the first composition adjustment photographing operation. The first composition adjustment photographing operation will be described along this flowchart.
  • step S1 the drive mode of the image sensor 33 is automatically set to the preview mode.
  • the preview mode a frame image is obtained from the image sensor 33 at a predetermined frame period, and the obtained frame image sequence is updated and displayed on the display unit 27.
  • step S ⁇ b> 2 the angle of view of the imaging unit 11 is adjusted by driving the zoom lens 30 according to the operation on the operation unit 26.
  • step S3 based on the output signal of the image sensor 33, AE (Automatic Exposure) control for optimizing the exposure amount of the image sensor 33 and AF (Automatic Focus) for optimizing the focus position of the imaging unit 11 are performed.
  • AE Automatic Exposure
  • AF Automatic Focus
  • step S4 the CPU 23 confirms whether or not the shutter button 26b is in a half-pressed state, and if it is in a half-pressed state, the process proceeds to step S5 and again the above exposure amount and focus. Perform position optimization. Thereafter, in step S6, the CPU 23 confirms whether the shutter button 26b is fully pressed, and if it is fully pressed, the process proceeds to step S10.
  • step S10 the imaging control unit 52 in FIG. 5 confirms whether or not a face having a predetermined size or more is detected from the determination image.
  • the determination image here is, for example, a frame image obtained immediately after or immediately before it is confirmed that the shutter button 26b is fully pressed.
  • the face detection unit 51 receives the determination image as an input image. Then, based on the face detection information for the determination image obtained by the face detection process, the imaging control unit 52 performs confirmation in step S10.
  • step S12 the image acquisition unit 53 acquires a basic image from the output signal of the AFE 12 after the shutter button 26b is fully pressed. More specifically, in step 12, the AFE 12 output signal itself (hereinafter referred to as “raw data”) for one frame image is temporarily written in the internal memory 17. The frame image represented by the signal written here is the basic image.
  • the basic image is an image of the photographing range itself set by the photographer.
  • step S13 the optical axis shift control by the imaging control unit 52 and the acquisition of the composition adjustment image by still image shooting after the optical axis shift control are executed as many times as necessary. Specifically, for example, the first to fourth composition adjustment images are acquired by repeating them four times.
  • steps S12 to S14 will be described in detail with reference to FIGS.
  • FIG 8 represents a plan view of the subject of the image pickup apparatus 1.
  • Reference numeral 301 denotes a photographing range when photographing a determination image
  • reference numeral 302 denotes a determination image. 8
  • two face regions 303 and 304 are extracted from the determination image 302 by the face detection unit 51. In this case, face detection information for each of the face regions 303 and 304 is generated.
  • a point 305 is an intermediate point between the center of the face area 303 and the center of the face area 304 in the determination image 302.
  • the imaging control unit 52 handles the intermediate point as a face target point.
  • the imaging control unit 52 detects the coordinate value of the face target point based on the face detection information of the face areas 303 and 304. This coordinate value specifies the position of the face target point on the coordinate plane of FIG.
  • the center of a main subject is preferably arranged at the intersection of lines that divide the image into three equal parts vertically and horizontally.
  • a composition with such an arrangement is also called a golden section composition.
  • FIG. 9 is formed by an image of interest, two lines that divide the image into three equal parts in the vertical direction, two lines that divide the image into three equal parts in the left-right direction, and these lines.
  • Four intersection points GA 1 to GA 4 are shown.
  • the intersection points GA 1 , GA 2 , GA 3, and GA 4 are intersection points located on the upper left side, the lower left side, the lower right side, and the upper right side, respectively, when viewed from the center of the focused image.
  • the imaging control unit 52 shifts the optical axis based on the coordinate value of the face target point in the determination image so that the face target point in the i-th composition adjustment image is positioned at the intersection point GA i on the i-th composition adjustment image. Control is performed (where i is 1, 2, 3 or 4).
  • Reference numeral 340 in FIG. 10A represents a basic image
  • reference numerals 341 to 344 in FIGS. 10B to 10E represent first to fourth composition adjustment images, respectively.
  • the shooting range 320 when shooting the basic image the shooting range 321 when shooting the first composition adjustment image, and the shooting when shooting the second composition adjustment image, respectively.
  • a range 322, a shooting range 323 at the time of shooting the third composition adjustment image, and a shooting range 324 at the time of shooting the fourth composition adjustment image are shown superimposed on the plan view 300 of the subject.
  • 10A to 10E shows two lines that divide the shooting range into three equal parts in the vertical direction and two lines that divide the photographing range into three equal parts in the left-right direction. Yes.
  • reference numerals 331 to 334 are assigned to the intersection points corresponding to the intersection points GA 1 to GA 4 , respectively.
  • the shooting range 320 when the basic image 340 is captured is the same as that when the determination image 302 is captured. If the difference in image quality is ignored, the basic image 340 and the determination image 302 are the same.
  • the photographing control unit 52 sets the face target point so that the photographing range of the imaging unit 11 becomes the photographing range 321 in FIG. 10B before the first composition adjustment image is photographed.
  • the optical axis shift control is performed so that is positioned, and then the raw data for one frame image is written in the internal memory 17.
  • the frame image represented by the signal written here is the first composition adjustment image.
  • the face target point in the first composition adjustment image is located at the intersection GA 1 on the first composition adjustment image.
  • the imaging control unit 52 sets the imaging range of the imaging unit 11 to the imaging range 322 in FIG. 10C prior to imaging the second composition adjustment image. That is, the optical axis shift control is performed so that the face target point is located at the intersection point 332, and then the raw data for one frame image is written in the internal memory 17.
  • the frame image represented by the signal written here is the second composition adjustment image.
  • the face target point in the second composition adjustment image is located at the intersection GA 2 on the second composition adjustment image.
  • the third and fourth composition adjustment images are acquired in the same manner.
  • the face target point in the third composition adjustment image is located at the intersection point GA3 on the third composition adjustment image
  • the face target point in the fourth composition adjustment image is the intersection point GA on the fourth composition adjustment image. 4 position.
  • step S15 the recording control unit 54 in FIG. 5 records these image data in the external memory 18 in association with each other, and then returns to step S1.
  • the image data is expressed by a YUV video signal. More specifically, the recording control unit 54 reads the raw data temporarily recorded in the internal memory 17 and the raw data of the first to fourth composition adjustment images, and those images obtained from the raw data.
  • Video signal (YUV signal) is JPEG compressed. Then, the compressed signals are associated with each other and recorded in the external memory 18.
  • JPEG compression means signal compression processing in accordance with the JPEG (Joint Photographic Experts Group) standard. Note that the Raw data itself can be recorded in the external memory 18 without performing JPEG compression.
  • step S10 when a face of a predetermined size or larger is not detected from the determination image, the process proceeds from step S10 to step S21, and the drive mode of the image sensor 33 is still image shooting suitable for still image shooting.
  • the mode is set, and then the processes of steps S22 and S23 are executed.
  • the process of step S22 is the same as the process of step S12, and thereby a basic image is acquired.
  • the image data of the basic image is recorded in the external memory 18 in step S23, and then the process returns to step S1.
  • an image having a golden section composition is automatically recorded just by giving a still image shooting instruction, and a highly artistic image can be provided to the user.
  • the size of each face is acquired from the face detection information of the determination image, and the largest face among the plurality of faces is the face of the main subject.
  • the center of the face area including the face of the main subject may be handled as the face target point.
  • the basic image is taken after the judgment image is taken. That is, although the determination image and the basic image are different, one frame image can be shared as the determination image and the basic image.
  • one frame image is acquired in the shooting image shooting mode, and the frame image is handled as a basic image and also used as a determination image. handle. Then, when a face larger than a predetermined size is detected from the determination image, the above-described processing of steps S13 to S15 is performed, and when it is not detected, the above-described processing of step S23 is performed. Like that.
  • the second composition adjustment shooting operation is a modification of a part of the first composition adjustment photographing operation, and operations and configurations not particularly described are the same as those shown in the first composition adjustment photographing operation.
  • a face larger than a predetermined size is detected from the determination image.
  • Step S10 when a face larger than a predetermined size is detected from the determination image in step S10 after the processing in steps S1 to S6 in FIG. 7, first, a basic image is shot. (Steps S11 and S12), and then the process proceeds to Step S13.
  • steps S13 and S14 the optical axis shift control by the imaging control unit 52 and the acquisition of the composition adjustment image by still image shooting after the optical axis shift control are executed as many times as necessary.
  • the imaging control unit 52 determines the number of executions and the composition adjustment image to be acquired according to the orientation of the face in the determination image.
  • the imaging control unit 52 specifies from the face detection information of the determination image whether the face direction in the determination image is the front direction, the left direction, or the right direction.
  • the face orientation specified here is referred to as the face orientation of interest.
  • the first to fourth composition adjustment images are acquired and recorded in the same manner as the first composition adjustment photographing operation.
  • the imaging control unit 52 determines that the face target point is the intersection GA 3 based on the coordinate value of the face target point (point 305 in the example of FIG. 8) in the determination image. arranged composition adjustment image, or performs optical axis shift control as the face target point is acquired composition adjustment image that is located at a cross point GA 4 (see FIG. 9).
  • composition adjustment image face target point is located at the intersection GA 1, or, the face target point intersection Optical axis shift control is performed so that a composition adjustment image arranged in GA 2 is acquired.
  • the determination image is the determination image 302 in FIG. 8, and two face regions 303 and 304 are extracted therefrom. . Further, it is assumed that the face size corresponding to the face area 303 is larger than that of the face area 304, and the face direction corresponding to the face area 303 (that is, the face direction of interest) is leftward.
  • the imaging control unit 52 performs optical axis shift control so that a composition adjustment image in which the face target point is arranged at the intersection point GA 3 or a composition adjustment image in which the face target point is arranged at the intersection point GA 4 is acquired.
  • a composition adjustment image in which the face target point is arranged at the intersection point GA 3 or a composition adjustment image in which the face target point is arranged at the intersection point GA 4 is acquired.
  • Photographing control unit 52 based on the positional relationship of the face area 303 and 304, of composition adjustment image face target point is located at a cross point GA 3 composition and face target point is placed composition adjustment image at the intersection GA 4 Determine which of the compositions is better.
  • the size of the face can also be considered.
  • FIGS. 11A and 11B show a shooting range 361 for acquiring the former composition adjustment image
  • FIG. 11B shows a shooting range 362 for acquiring the latter composition adjustment image, respectively. It is shown superimposed on the plan view 300.
  • the face area 304 exists above the face area 303, the face area 304 is too positioned above the shooting area when the shooting range 362 is used, and corresponds to the face area 304.
  • part of the face or the head may protrude from the shooting range. Therefore, it is determined that the direction of the composition of the composition adjustment image face target point is located at a cross point GA 3 are excellent, so as to obtain the composition adjustment image.
  • the imaging range of the imaging unit 11 is based on the coordinate values of the face target points of the determination image.
  • the optical axis shift control is performed so that the imaging range 361 becomes the same, and then the raw data for one frame image is written in the internal memory 17.
  • the frame image represented by the signal written here is one composition adjustment image to be acquired in step S14. Face target point in the composition adjustment image is located at the intersection GA 3 on the composition adjustment image.
  • FIG. 12 shows the obtained composition adjustment image.
  • step S15 the recording control unit 54 in FIG. 5 uses the basic image obtained in step S12 and the image data of the composition adjustment image obtained in step S14 (total two pieces of image data). The data are recorded in the external memory 18 in association with each other, and then the process returns to step S1.
  • a specific method of this recording is as shown in the first composition adjustment photographing operation.
  • reference numeral 400 represents a plan view of the subject of the imaging apparatus 1
  • reference numeral 420 represents a shooting range at the time of shooting the determination image and the basic image
  • reference numeral 440 is acquired in step S12.
  • the center of the face area is set as a face target point.
  • the face orientation of the extracted face area is leftward. Therefore, after acquiring the basic image, the imaging control unit 52 arranges the face adjustment point at the intersection point GA 3 or the face adjustment point at the intersection point GA 4 based on the fact that the face direction is the left direction.
  • Optical axis shift control is performed so that the obtained composition adjustment image is acquired.
  • the imaging control unit 52 composes the composition adjustment image in which the face target point is arranged at the intersection GA 3 and the composition adjustment image in which the face target point is arranged at the intersection GA 4 based on the position of the face region. Which of the compositions is better is judged. At this time, the size of the face can also be considered. In the case of the example shown in FIG. 13A, since there is one person as a subject, it can be said that the composition is better when the entire image of the person is within the shooting range. Therefore, the shooting control unit 52 estimates the direction in which the torso of the person is located based on the face detection result, and determines that a composition in which the torso is more within the shooting range is a better composition.
  • FIG. 13 (a) in the example shown, because the body is located on the lower side of the image as a base point a face, so as to obtain a composition adjustment image face target point is located at a cross point GA 4.
  • FIG. 13B shows a shooting range 421 when the composition adjustment image is acquired, and the obtained composition adjustment image 441. Then, a total of two pieces of image data of the basic image and the composition adjustment image are associated with each other and recorded in the external memory 18 to complete one photographing operation.
  • the face target point is at the intersection point GA 3 instead of the composition adjustment image in which the face target point is arranged at the intersection point GA 4.
  • the arranged composition adjustment image may be acquired.
  • the optical axis shift control and still image shooting after the optical axis shift control are repeated twice. It makes may acquire both the composition adjustment image composition adjustment image and face target point face target point is located at a cross point GA 3 is located at a cross point GA 4. In this case, the image data of the two composition adjustment images and the basic image are recorded in the external memory 18 in association with each other.
  • FIG. 14 is a partial functional block diagram of the imaging apparatus 1 involved in the third composition adjustment shooting operation.
  • the functions of the face detection unit 61 and the cutout unit 63 are mainly realized by the video signal processing unit 13 in FIG. 1, and the function of the cutout region setting unit 62 is mainly realized by the CPU 23 (and / or the video signal processing unit 13) in FIG.
  • the functions of the recording control unit 64 are mainly realized by the CPU 23 and the compression control unit 16.
  • other parts for example, the internal memory 17 shown in FIG. 1 are also involved in realizing the functions of the parts referenced by reference numerals 61 to 64 as necessary.
  • the face detection unit 61 has the same function as the face detection unit 51 (see FIG. 5) shown in the first composition adjustment shooting operation, and extracts face detection information for the input image (determination image) to the cutout region setting unit 62. introduce. Image data of the basic image having the composition designated by the photographer is given to the clipping unit 63.
  • the cutout region setting unit 62 sets a cutout region for cutting out the composition adjustment image from the basic image based on the face detection information, and cuts out cutout region information that specifies the position and size of the cutout region on the basic image. This is transmitted to the unit 63.
  • the cutout unit 63 cuts out a partial image of the basic image according to the cutout area information, and generates an image obtained by the cutout (hereinafter referred to as a cutout image) as a composition adjustment image.
  • the recording control unit 64 records the generated composition adjustment image and basic image image data in the external memory 18 in association with each other.
  • FIG. 15 is a flowchart showing the flow of the third composition adjustment photographing operation.
  • the third composition adjustment photographing operation will be described along this flowchart. It is assumed that the position of the correction lens 36 is always fixed during this operation (however, the movement of the correction lens 36 for realizing optical camera shake correction can be executed).
  • steps S1 to S6 are executed.
  • the processes in steps S1 to S6 are the same as those in the first composition adjustment photographing operation (see FIG. 7). However, if it is confirmed in step S6 that the shutter button 26b is fully pressed, the process proceeds to step S31, where the drive mode of the image sensor 33 is set to a still image shooting mode suitable for still image shooting. Is done.
  • the cutout unit 63 acquires a basic image from the output signal of the AFE 12 after confirming that the shutter button 26b is fully pressed. More specifically, in step 32, the raw data for one frame image is temporarily written in the internal memory 17. The frame image represented by the signal written here is the basic image.
  • the basic image is an image of the photographing range itself set by the photographer.
  • step S ⁇ b> 33 the cutout region setting unit 62 confirms whether or not a face having a predetermined size or more is detected from the determination image based on the face detection information of the determination image provided from the face detection unit 61.
  • the basic image is used as the determination image.
  • step S34 the basic image data is recorded in the external memory 18, and the process returns to step S1.
  • step S35 one or more cut-out images are cut out from the basic image.
  • the processing contents of step S35 will be described with reference to FIGS. 16 (a) to 16 (e).
  • the image denoted by reference numeral 500 is the basic image acquired in step S32.
  • the face detection unit 61 generates face detection information of the determination image by handling the basic image 500 as a determination image and performing face detection processing. It is assumed that two face regions 503 and 504 are extracted from the determination image by the face detection unit 61. In this case, face detection information for each of the face regions 503 and 504 is generated.
  • a point 505 is an intermediate point between the center of the face area 503 and the center of the face area 504 in the determination image.
  • the cutout region setting unit 62 handles the intermediate point as a face target point.
  • the cutout area setting unit 62 detects the coordinate value of the face target point based on the face detection information of the face areas 503 and 504. The coordinate value specifies the position of the face target point on the coordinate plane of FIG.
  • the cutout region setting unit 62 is based on all or any of the first to fourth cutout images 521 to 524 shown in FIGS. 16 (b) to (e).
  • the cutout position and size are set so as to be cut out from the image 500, and cutout area information representing the set cutout position and size is sent to the cutout unit 63.
  • the cut-out area information is generated so that the face target point in the i-th cut-out image is located at the intersection point GA i on the i-th cut-out image (see FIG. 9) (where i is 1, 2, 3 or 4).
  • the cut-out area information is generated so that the image size of the cut-out image is as large as possible.
  • the cutout unit 63 generates all or one of the first to fourth cutout images 521 to 524 from the basic image 500 according to the cutout area information.
  • the first to fourth cut-out images are handled as first to fourth composition adjustment images, respectively.
  • step S35 the recording control unit 64 in FIG. 14 associates the image data of the basic image obtained in step S32 with the image data of one or more composition adjustment images obtained in step S35.
  • the data is recorded in the external memory 18, and then the process returns to step S1.
  • a maximum of five pieces of image data are recorded in the external memory 18.
  • the raw data of the basic image temporarily recorded in the internal memory 17 is read, and the video signal (YUV signal) of the basic image and the composition adjustment image is generated from the raw data. Thereafter, the video signal is JPEG compressed and recorded in the external memory 18. It is possible not to perform JPEG compression.
  • the image size (that is, the number of pixels in the horizontal and vertical directions) of the composition adjustment image to be recorded is smaller than that of the basic image in principle.
  • the image size of the composition adjustment image is increased using an interpolation process so that the image size is different between the two, and the image data (video signal) of the composition adjustment image after the image size increase is recorded in the external memory 18. You may do it.
  • composition adjustment image to be generated and recorded is selected from the cutout images 521 to 524 is determined by the method shown in the second composition adjustment photographing operation. That is, according to the method described in the second composition adjustment shooting operation, the face orientation of interest is detected based on the face detection information of the determination image. If the face of interest is front-facing, all of the cut-out images 521 to 524 are generated and recorded.
  • one of the cut-out images 523 and 524 is generated and recorded. That is, according to the method described in the second composition adjustment shooting operation, which of the cut-out images 523 and 524 is superior is based on the number of faces, the position of the face, the orientation, and the size in the determination image. Is generated, and a cut-out image of which the composition is determined to be excellent is generated and recorded. However, both the cutout images 523 and 524 may be generated and recorded.
  • one of the cutout images 521 and 522 is generated and recorded. That is, according to the method described in the second composition adjustment shooting operation, which of the cut-out images 521 and 522 is superior is based on the number of faces, the position of the face, the orientation, and the size in the determination image. Is generated, and a cut-out image of which the composition is determined to be excellent is generated and recorded. However, both the cutout images 521 and 522 may be generated and recorded.
  • an image having a golden section composition is automatically recorded just by giving a still image shooting instruction, and a highly artistic image can be provided to the user. Also. If the composition adjustment image to be recorded is selected according to the face orientation, the required processing time and the required recording capacity are reduced.
  • FIG. 17 shows the structure of one image file.
  • the image file is formed of a main body area and a header area.
  • additional information for the corresponding image is stored.
  • the header area is also called an Exif tag or an Exif area. It is possible to make the file format of the image file comply with an arbitrary standard.
  • an image file refers to an image file recorded in the external memory 18. The generation and recording of the image file is executed by the recording control unit 54 in FIG. 5 or the recording control unit 64 in FIG.
  • the basic image and the first to fourth composition adjustment images are acquired by the photographing and recording operations shown in the first composition adjustment photographing operation, and are associated with each other and recorded in the external memory 18.
  • “5 images” means five images including a basic image and first to fourth composition adjustment images.
  • a first recording format that can be employed will be described with reference to FIG.
  • the first recording format When the first recording format is adopted, five image files FL 1 to FL 5 for individually storing five images are generated and recorded in the external memory 18.
  • Image data of the basic image is stored in the main body area of the image file FL 1
  • image data of the first to fourth composition adjustment images are stored in the main body areas of the image files FL 2 to FL 5 , respectively.
  • the related image information is information for designating the image files FL 2 to FL 5 , and the image file FL 1 and the image files FL 2 to FL 5 are associated with this information.
  • the user can usually browse only the basic image, and the first to fourth composition adjustment images are played back on the display unit 27 only when a special operation is given to the imaging apparatus 1. Can be browsed.
  • the image file FL 1 ⁇ FL 5 collectively managed as one related files may be applied to file operation for the image file FL 1 to all the image files FL 1 ⁇ FL 5.
  • the file operation is an operation for instructing deletion of an image file, change of a file name, or the like.
  • the operation in the above-described reproduction mode is also applied to an image reproduction apparatus (not shown) different from the imaging apparatus 1 that has received the recording data of the external memory 18.
  • the user can usually browse only the basic image, and the first to fourth composition adjustment images are played back on the display unit 27 only when a special operation is given to the imaging apparatus 1. Can be browsed.
  • the composition adjustment image if a predetermined operation is performed on the imaging apparatus 1, all of the first to fourth composition adjustment images can be erased from the image file FL 6 all at once or individually. It is.
  • a predetermined operation that is, a composition adjustment image designated in an image file other than the image file FL 6 ). Can also be saved).
  • FIG. 20 is a partial functional block diagram of the imaging apparatus 1 involved in the automatic trimming playback operation.
  • the face detection unit 71, the cutout region setting unit 72, and the cutout unit 73 have functions equivalent to the face detection unit 61, the cutout region setting unit 62, and the cutout unit 63 in FIG.
  • the setting unit 62 and the cutout unit 63 can be used as they are.
  • Image data of an input image is given to the face detection unit 71 and the cutout unit 73 from the external memory 18 or the outside of the imaging device 1.
  • image data of an input image is given from the external memory 18.
  • This input image is, for example, an image shot and recorded without performing the above-described composition adjustment shooting operation.
  • the face detection unit 71 transmits face detection information for the input image to the cutout region setting unit 72.
  • the cutout region setting unit 72 sets a cutout region for cutting out the composition adjustment image from the input image based on the face detection information, and cuts out cutout region information for specifying the position and size of the cutout region on the input image. Transmitted to the unit 73.
  • the cutout unit 73 cuts out a partial image of the input image according to the cutout area information, and generates the cutout image as a composition adjustment image.
  • the composition adjustment image as the cut-out image is reproduced and displayed on the display unit 27.
  • FIG. 21 is a flowchart showing the flow of the automatic trimming playback operation.
  • the automatic trimming playback operation will be described along this flowchart.
  • Various instructions (automatic trimming instructions and the like) to be described later with respect to the imaging apparatus 1 are given to the imaging apparatus 1 by an operation on the operation unit 26, for example, and the CPU 23 determines whether or not there is an instruction.
  • step S51 when the image pickup apparatus 1 is activated and the operation mode of the image pickup apparatus 1 is set to the reproduction mode, a still image recorded in the external memory 18 is reproduced and displayed on the display unit 27 in accordance with a user instruction in step S51.
  • the still image here is called a playback basic image.
  • step S53 When the user gives an automatic trimming instruction regarding the playback basic image, the process proceeds to step S53 via step S52. If the automatic trimming instruction is not given, the process of step 51 is repeated.
  • step S53 the reproduction basic image in step S51 is provided as an input image to the face detection unit 71 and the clipping unit 73, and the face detection unit 71 performs face detection processing on the reproduction basic image to obtain face detection information. create.
  • the cutout region setting unit 72 Based on the face detection information, in subsequent step S54, the cutout region setting unit 72 confirms whether a face of a predetermined size or more is detected from the reproduction basic image. If it is detected, the process proceeds to step S55. If not detected, the process returns to step S51.
  • step S55 the cutout area setting unit 72 and the cutout unit 73 cut out and display one optimal composition adjustment image from the reproduction basic image.
  • the method of generating one composition adjustment image from the reproduction basic image by the cutout area setting unit 72 and the cutout unit 73 is a method of generating one composition adjustment image from the basic image described in the third composition adjustment shooting operation. The same.
  • step S51 For example, consider a case where the reproduction basic image in step S51 is the same as the basic image 500 shown in FIG.
  • face detection information for each of the face areas 503 and 504 is generated, and the cut-out area setting unit 72 sets an intermediate point 505 between the center of the face area 503 and the center of the face area 504 in the reproduction basic image as a face target point.
  • the coordinate value of the face target point is detected based on the face detection information of the face areas 503 and 504.
  • the coordinate value specifies the position of the face target point on the coordinate plane of FIG. Of the face areas 503 and 504, the center point of the face area corresponding to the larger face can be treated as a face target point.
  • the cutout region setting unit 72 is one of the first to fourth cutout images 521 to 524 shown in FIGS. 16B to 16E based on the coordinate value of the face target point in the reproduction basic image.
  • the cutout position and size are set so that one is cut out from the reproduction basic image, and cutout area information representing the set cutout position and size is sent to the cutout unit 73.
  • the cut-out area information is generated so that the face target point in the i-th cut-out image is located at the intersection point GA i on the i-th cut-out image (see FIG. 9) (where i is 1, 2, 3 or 4).
  • the cut-out area information is generated so that the image size of the cut-out image is as large as possible.
  • the cutout unit 73 cuts out and generates a cutout image 521, 522, 523, or 524 from the reproduction basic image according to the cutout region information, and outputs the generated single cutout image to the display unit 27 as an optimal composition adjustment image. .
  • any one of the first to fourth composition adjustment images is selected to perform shooting or clipping, but the same method as the selection method is used.
  • the composition adjustment image selected by use is handled as the optimum composition adjustment image. That is, the optimal composition adjustment image is selected from the first to fourth composition adjustment images on the basis of the number of faces detected from the reproduction basic image, the position, orientation and size of the face. If the face orientation detected from the playback basic image is front-facing, the optimum composition adjustment image cannot be reduced to one, and a message to that effect is displayed and the process returns to step S51.
  • a plurality of composition adjustment images that cannot be narrowed down may be displayed side by side on the display screen of the display unit 27.
  • step S56 After displaying the optimum composition adjustment image in step S55, it is confirmed in step S56 whether or not an instruction to replace the recorded image has been issued.
  • the replacement instruction is given, the playback basic image is deleted from the external memory 18 in step S57 under the control of the CPU 23, and then the optimum composition adjustment image is recorded in the external memory 18 in step S59, and the process returns to step S51. . If there is no replacement instruction, the process proceeds to step S58, and it is confirmed whether a recording instruction for instructing separate recording of the optimum composition adjustment image is made.
  • the recording instruction is given, under the control of the CPU 23, the optimum composition adjustment image is recorded in the external memory 18 in step S59 while maintaining the recording of the reproduction basic image, and the process returns to step S51.
  • the process returns to step S51 without executing the recording of the optimum composition adjustment image.
  • the image size of the composition adjustment image may be increased so that the image size of the composition adjustment image is the same as that of the reproduction basic image.
  • the correction lens 36 is used as an optical member for moving the optical image projected on the image sensor 33 on the image sensor 33.
  • a variangle prism (not shown) is used instead of the correction lens 36.
  • the movement of the optical image may be realized using The movement of the optical image may be realized by moving the image sensor 33 along a plane orthogonal to the optical axis without using the correction lens 36 or the vari-angle prism.
  • the automatic trimming playback operation may be realized by an external image playback device (not shown) different from the imaging device 1.
  • the face detection unit 71, the cutout region setting unit 72, and the cutout unit 73 may be provided in an external image playback device, and image data of the playback basic image may be provided to the image playback device.
  • the composition adjustment image from the cutout unit 73 provided in the image reproduction device is displayed on a display unit equivalent to the display unit 27 provided in the image reproduction device or on an external display device (all not (Illustrated).
  • the imaging apparatus 1 in FIG. 1 can be realized by hardware or a combination of hardware and software.
  • the arithmetic processing necessary for performing the composition adjustment photographing operation and the automatic trimming reproduction operation can be realized by software or a combination of hardware and software.
  • a block diagram of a part realized by software represents a functional block diagram of the part. All or part of the arithmetic processing necessary for performing the composition adjustment photographing operation and the automatic trimming reproduction operation is described as a program, and the program is executed on a program execution device (for example, a computer) to thereby execute the arithmetic processing. You may make it implement
  • the image moving means for moving the optical image projected on the image sensor 33 on the image sensor 33 is realized by the correction lens 36 and the driver 34 in the above-described embodiment.
  • the part including the imaging control unit 52 and the image acquisition unit 53 in FIG. 5 functions as a composition control unit that generates a composition adjustment image.
  • the part including the cutout region setting unit 62 and the cutout unit 63 in FIG. 14 functions as a composition control unit that generates a composition adjustment image.
  • the parts referred to by reference numerals 71 to 73 in FIG. 20 function as an image reproducing device. It may be considered that the display unit 27 is further included in this image reproduction device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)
  • Television Signal Processing For Recording (AREA)
  • Image Analysis (AREA)
PCT/JP2009/053243 2008-03-10 2009-02-24 撮像装置及び画像再生装置 Ceased WO2009113383A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/921,904 US20110007187A1 (en) 2008-03-10 2009-02-24 Imaging Device And Image Playback Device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2008-059756 2008-03-10
JP2008059756A JP4869270B2 (ja) 2008-03-10 2008-03-10 撮像装置及び画像再生装置

Publications (1)

Publication Number Publication Date
WO2009113383A1 true WO2009113383A1 (ja) 2009-09-17

Family

ID=41065055

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2009/053243 Ceased WO2009113383A1 (ja) 2008-03-10 2009-02-24 撮像装置及び画像再生装置

Country Status (3)

Country Link
US (1) US20110007187A1 (https=)
JP (1) JP4869270B2 (https=)
WO (1) WO2009113383A1 (https=)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009218807A (ja) * 2008-03-10 2009-09-24 Sanyo Electric Co Ltd 撮像装置及び画像再生装置
CN104243791A (zh) * 2013-06-19 2014-12-24 联想(北京)有限公司 一种信息处理方法及电子设备
CN104601890A (zh) * 2015-01-20 2015-05-06 广东欧珀移动通信有限公司 利用移动终端拍摄人物的方法及装置
CN105592262A (zh) * 2014-11-11 2016-05-18 奥林巴斯株式会社 摄像装置

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5423284B2 (ja) * 2009-09-28 2014-02-19 リコーイメージング株式会社 撮像装置
JP5427577B2 (ja) * 2009-12-04 2014-02-26 パナソニック株式会社 表示制御装置及び表示画像形成方法
US9721324B2 (en) * 2011-09-10 2017-08-01 Microsoft Technology Licensing, Llc Thumbnail zoom
JP5805503B2 (ja) * 2011-11-25 2015-11-04 京セラ株式会社 携帯端末、表示方向制御プログラムおよび表示方向制御方法
JP2013153375A (ja) 2012-01-26 2013-08-08 Sony Corp 画像処理装置、画像処理方法および記録媒体
CN104054332A (zh) 2012-01-26 2014-09-17 索尼公司 图像处理装置和图像处理方法
JP5978639B2 (ja) * 2012-02-06 2016-08-24 ソニー株式会社 画像処理装置、画像処理方法、プログラム、及び記録媒体
JP5880263B2 (ja) * 2012-05-02 2016-03-08 ソニー株式会社 表示制御装置、表示制御方法、プログラムおよび記録媒体
WO2015072166A1 (ja) * 2013-11-18 2015-05-21 オリンパスイメージング株式会社 撮像装置、撮像アシスト方法及び撮像アシストプログラムを記録した記録媒体
JP5880612B2 (ja) * 2014-03-28 2016-03-09 ブラザー工業株式会社 情報処理装置及びプログラム
JP6518409B2 (ja) * 2014-06-30 2019-05-22 オリンパス株式会社 撮影装置及び撮影方法
JP6459517B2 (ja) * 2015-01-06 2019-01-30 株式会社リコー 撮像装置、映像送信装置、及び映像送受信システム
JP6584259B2 (ja) * 2015-09-25 2019-10-02 キヤノン株式会社 像ブレ補正装置、撮像装置および制御方法
JP6873186B2 (ja) * 2019-05-15 2021-05-19 日本テレビ放送網株式会社 情報処理装置、スイッチングシステム、プログラム及び方法
JP7384200B2 (ja) * 2019-05-27 2023-11-21 ソニーグループ株式会社 構図制御装置、構図制御方法、プログラム
JP7547062B2 (ja) * 2020-03-25 2024-09-09 キヤノン株式会社 撮像装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005175684A (ja) * 2003-12-09 2005-06-30 Nikon Corp デジタルカメラおよびデジタルカメラの画像取得方法
JP2007036436A (ja) * 2005-07-25 2007-02-08 Konica Minolta Photo Imaging Inc 撮像装置、及びプログラム
JP2007174548A (ja) * 2005-12-26 2007-07-05 Casio Comput Co Ltd 撮影装置及びプログラム

Family Cites Families (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7298412B2 (en) * 2001-09-18 2007-11-20 Ricoh Company, Limited Image pickup device, automatic focusing method, automatic exposure method, electronic flash control method and computer program
JP2004109247A (ja) * 2002-09-13 2004-04-08 Minolta Co Ltd デジタルカメラ、画像処理装置、およびプログラム
US20040207743A1 (en) * 2003-04-15 2004-10-21 Nikon Corporation Digital camera system
US7453506B2 (en) * 2003-08-25 2008-11-18 Fujifilm Corporation Digital camera having a specified portion preview section
JP2005117316A (ja) * 2003-10-07 2005-04-28 Matsushita Electric Ind Co Ltd 撮影装置、撮影方法、およびプログラム
JP2005215750A (ja) * 2004-01-27 2005-08-11 Canon Inc 顔検知装置および顔検知方法
CN100448267C (zh) * 2004-02-06 2008-12-31 株式会社尼康 数码相机
JP4135100B2 (ja) * 2004-03-22 2008-08-20 富士フイルム株式会社 撮影装置
JP4824411B2 (ja) * 2005-01-20 2011-11-30 パナソニック株式会社 顔抽出装置、半導体集積回路
JP4399668B2 (ja) * 2005-02-10 2010-01-20 富士フイルム株式会社 撮影装置
JP4513699B2 (ja) * 2005-09-08 2010-07-28 オムロン株式会社 動画像編集装置、動画像編集方法及びプログラム
JP2007174269A (ja) * 2005-12-22 2007-07-05 Sony Corp 画像処理装置および方法、並びにプログラム
JP4025362B2 (ja) * 2006-02-15 2007-12-19 松下電器産業株式会社 撮影装置および撮影方法
CN101867679B (zh) * 2006-03-27 2013-07-10 三洋电机株式会社 缩略图生成装置及摄像装置
JP4948014B2 (ja) * 2006-03-30 2012-06-06 三洋電機株式会社 電子カメラ
JP4182117B2 (ja) * 2006-05-10 2008-11-19 キヤノン株式会社 撮像装置及びその制御方法及びプログラム及び記憶媒体
JP4804398B2 (ja) * 2007-03-30 2011-11-02 三洋電機株式会社 撮像装置及び撮影方法
KR101469246B1 (ko) * 2007-08-07 2014-12-12 삼성전자주식회사 로봇에서 사진 촬영하는 장치 및 방법
US8218033B2 (en) * 2007-09-10 2012-07-10 Sanyo Electric Co., Ltd. Sound corrector, sound recording device, sound reproducing device, and sound correcting method
US8253819B2 (en) * 2008-02-06 2012-08-28 Panasonic Corporation Electronic camera and image processing method
JP4869270B2 (ja) * 2008-03-10 2012-02-08 三洋電機株式会社 撮像装置及び画像再生装置
JP5202211B2 (ja) * 2008-09-25 2013-06-05 三洋電機株式会社 画像処理装置及び電子機器
US20100074557A1 (en) * 2008-09-25 2010-03-25 Sanyo Electric Co., Ltd. Image Processing Device And Electronic Appliance
JP5178441B2 (ja) * 2008-10-14 2013-04-10 三洋電機株式会社 電子カメラ
JP4623200B2 (ja) * 2008-10-27 2011-02-02 ソニー株式会社 画像処理装置、画像処理方法およびプログラム
JP4623199B2 (ja) * 2008-10-27 2011-02-02 ソニー株式会社 画像処理装置、画像処理方法およびプログラム
JP2011050038A (ja) * 2009-07-27 2011-03-10 Sanyo Electric Co Ltd 画像再生装置及び撮像装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005175684A (ja) * 2003-12-09 2005-06-30 Nikon Corp デジタルカメラおよびデジタルカメラの画像取得方法
JP2007036436A (ja) * 2005-07-25 2007-02-08 Konica Minolta Photo Imaging Inc 撮像装置、及びプログラム
JP2007174548A (ja) * 2005-12-26 2007-07-05 Casio Comput Co Ltd 撮影装置及びプログラム

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009218807A (ja) * 2008-03-10 2009-09-24 Sanyo Electric Co Ltd 撮像装置及び画像再生装置
CN104243791A (zh) * 2013-06-19 2014-12-24 联想(北京)有限公司 一种信息处理方法及电子设备
CN105592262A (zh) * 2014-11-11 2016-05-18 奥林巴斯株式会社 摄像装置
CN104601890A (zh) * 2015-01-20 2015-05-06 广东欧珀移动通信有限公司 利用移动终端拍摄人物的方法及装置
CN104601890B (zh) * 2015-01-20 2017-11-03 广东欧珀移动通信有限公司 利用移动终端拍摄人物的方法及装置

Also Published As

Publication number Publication date
JP4869270B2 (ja) 2012-02-08
JP2009218807A (ja) 2009-09-24
US20110007187A1 (en) 2011-01-13

Similar Documents

Publication Publication Date Title
JP4869270B2 (ja) 撮像装置及び画像再生装置
JP5398408B2 (ja) カメラ、カメラの制御方法、表示制御装置、および表示制御方法
JP4645685B2 (ja) カメラ、カメラ制御プログラム及び撮影方法
CN103248815B (zh) 摄像装置、摄像方法
JP6304293B2 (ja) 画像処理装置、画像処理方法及びプログラム
KR101737086B1 (ko) 디지털 촬영 장치 및 이의 제어 방법
JP5888614B2 (ja) 撮像装置、映像コンテンツ生成方法、及び、プログラム
CN102164242A (zh) 摄像装置以及摄像控制方法
JP2012160950A (ja) 画像処理装置、撮像装置および表示装置
KR20130031176A (ko) 디스플레이 장치 및 방법
JP4697078B2 (ja) 撮像装置及びそのプログラム
JP5266701B2 (ja) 撮像装置、被写体分離方法、およびプログラム
JP6460783B2 (ja) 画像処理装置及びその制御方法
JP2011239021A (ja) 動画像生成装置、撮像装置および動画像生成プログラム
JP2006303961A (ja) 撮像装置
JP5035614B2 (ja) 撮像装置及びプログラム
JP2005086219A (ja) デジタルカメラ及び撮影制御方法
JP4735166B2 (ja) 画像表示装置及びプログラム
JP4696614B2 (ja) 画像表示制御装置及びプログラム
JP2010237911A (ja) 電子機器
JP2004312218A (ja) デジタルカメラおよび画像再生装置
JP5003803B2 (ja) 画像出力装置、および、プログラム
JP2007228453A (ja) 撮像装置、再生装置、プログラム、および記憶媒体
JP2009212867A (ja) 撮影画像処理装置、撮影制御プログラム及び撮影制御方法
JP5332668B2 (ja) 撮像装置および被写体検出プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09719408

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 12921904

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09719408

Country of ref document: EP

Kind code of ref document: A1