JP2010074400A - Device for determining composition, method of determining composition, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a more appropriate composition according to an attribute of an object by automatically searching the object as a person to optimize the composition of the object. <P>SOLUTION: A composition detection device includes: an object detection means for detecting an object existing in content of inputted image data; and an attribute detection means for detecting attributes, such as age and sex, for the detected objects. Also, a face direction is detected for each object, and then the composition adapted to a specific relationship is determined for attributes for each object. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a composition determination apparatus for determining an optimum composition from image content of image data obtained by imaging, for example, and a method thereof. Further, the present invention relates to a program executed by the composition determination apparatus.

One of the technical elements for taking photos that viewers feel good is the composition setting. The composition referred to here is also called framing, and means, for example, the arrangement of a subject in an image frame as a photograph.
Although there are several general and basic methods for achieving a good composition, it is important for ordinary camera users to take photographs with good composition unless they have sufficient knowledge and skill in photography. It's never easy. From this, for example, a technical configuration capable of easily and easily obtaining a photographic image having a good composition is required.

For example, in Patent Document 1, as an automatic tracking device, a difference between images at a predetermined time interval is detected, a center of gravity of the difference between images is calculated, and a moving amount and a moving direction of the center of gravity are used for an imaging screen of a subject image. A technical configuration is disclosed in which a moving amount and a moving direction are detected to control an imaging apparatus, and a subject image is set within a reference area of an imaging screen.
In Patent Document 2, as an automatic tracking device, when a person is automatically tracked, 20% from the upper side of the person relative to the entire area of the person image on the screen so that the face of the person is at the center of the screen. A technique has been disclosed in which tracking is performed while reliably capturing a face of a person by tracking the position where the area becomes the center of the screen.
If these technical configurations are viewed from the viewpoint of composition determination, it is possible to automatically search for a subject as a person and place the subject in a certain composition on the shooting screen.

JP 59-208983 A JP 2001-268425 A

For example, if a person is assumed to be a subject, the subject has several elements that can be regarded as attributes, such as gender and age (age, generation). If an attempt is made to obtain a better composition as described above, the appropriate composition may be different depending on the attribute of the subject.
However, with the technique according to the above-mentioned patent document, it is only possible to arrange the tracked subject with a certain fixed composition. Therefore, it is impossible to change the composition in accordance with the subject situation or the like.
Therefore, the present invention aims to propose a technique for easily obtaining a good composition for an image such as a photograph, and also adapts to the attributes of the subject. The purpose is to make a more appropriate composition decision.

In view of the above-described problems, the present invention is configured as follows as a composition determination apparatus.
That is, subject detection means for inputting image data and detecting a subject existing in the image content of the image data, attribute detection means for detecting a predetermined attribute for each subject detected by the subject detection means, And a composition determination device including composition determination means for determining composition based on a predetermined relationship with respect to attributes for each subject detected by the attribute detection means.

  In the above configuration, subjects existing in the image content of the image data are detected, and attributes of these detected subjects are also detected. In addition, it is possible to determine a composition that matches a specific relationship regarding the attribute of each subject.

  With the above configuration, a composition that is more appropriate according to the attribute of the subject can be obtained depending on the present invention.

Hereinafter, the best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described in the following order.

1. Configuration of imaging system 2. Basic composition determination processing example Composition determination according to the age (first example)
4). Composition determination according to the age (second example)
5). 5. Composition determination according to gender 6. Composition determination according to face direction Algorithm 8. Modified example

1. Configuration of imaging system

Prior to the description of the imaging system of the present embodiment, the words composition, image frame, angle of view, imaging viewing angle, and composition will be used in describing the present embodiment.
Here, the composition is also referred to as framing, and refers to, for example, an arrangement state including the size setting for the subject in the image frame.
The image frame is, for example, an area range corresponding to one screen in which an image can be seen to be fitted, and generally has an outer frame shape as a vertically long or horizontally long rectangle.
The angle of view is also called a zoom angle or the like, and represents the range within an image frame determined by the position of the zoom lens in the optical system of the imaging apparatus. Generally, it is determined by the focal length of the imaging optical system and the size of the image plane (image sensor, film), but here, the element that can change according to the focal length is called the angle of view. The
In addition to the above-mentioned angle of view, the viewing angle for the panning (horizontal) direction and the tilt (vertical) direction for the range that fits within the image frame of the image obtained by the imaging device placed at a fixed position. It is determined by the angle (elevation angle, depression angle).
For example, the composition indicates the arrangement state of the subject that falls within the image frame determined by the image viewing angle.

  In this embodiment, the configuration based on the present invention is applied to an imaging system including a digital still camera and a camera platform to which the digital still camera is attached.

FIG. 1 is a front view showing an example of the external configuration of an imaging system corresponding to the present embodiment.
As shown in this figure, the imaging system of the present embodiment includes a digital still camera 1 and a pan head 10.
The digital still camera 1 generates still image data based on imaging light obtained by imaging with the lens unit 3 provided on the front panel of the main body, and stores the still image data in a storage medium loaded therein. It is possible. That is, it has a function of storing and saving an image taken as a photograph in a storage medium as still image data. When such a photograph is taken manually, the user presses a shutter (release) button provided on the upper surface of the main body.

  The digital still camera 1 can be fixed to the camera platform 10 so as to be fixed. That is, the pan head 10 and the digital still camera 1 are provided with a mechanism part for enabling mutual attachment.

The pan / tilt head 10 includes a pan / tilt mechanism for moving the attached digital still camera 1 in both the pan direction (horizontal direction) and the tilt direction.
For example, FIGS. 2A and 2B show how the digital still camera 1 moves in the pan and tilt directions provided by the pan / tilt mechanism of the camera platform 10. FIGS. 2A and 2B show the digital still camera 1 attached to the pan head 10 as viewed from the plane direction and the side direction, respectively.
First, regarding the pan direction, for example, the rotation direction + α is set with the rotation axis Ct1 as the rotation center with reference to the position state where the horizontal direction of the digital still camera 1 and the straight line X1 shown in FIG. A panning movement in the right direction is given by the rotation along. Further, by performing rotation along the rotation direction −α, a panning movement in the left direction is given.
With respect to the tilt direction, for example, rotation in the rotation direction + β is performed with the rotation axis Ct2 as the rotation center with reference to the position state in which the vertical direction of the main body of the digital still camera 1 coincides with the straight line Y1 in the vertical direction. , Given downward panning movement. Further, the panning movement in the upward direction is given by the rotation in the rotation direction −β.
Although the maximum movable rotation angle in each of the ± α direction and the ± β direction shown in FIGS. 2A and 2B is not mentioned, it should be considered that the number of opportunities for capturing the subject should be increased as much as possible. If so, it is preferable to make the maximum movable rotation angle as large as possible.

FIG. 3 shows an internal configuration example of the digital still camera 1.
In this figure, first, the optical system unit 21 includes a predetermined number of imaging lens groups including a zoom lens, a focus lens, and the like, a diaphragm, and the like. Form an image on the surface.
The optical system unit 21 is also provided with a drive mechanism unit for driving the zoom lens, the focus lens, the diaphragm, and the like. The operation of these drive mechanisms is controlled by so-called camera control such as zoom (viewing angle) control, automatic focus adjustment control, automatic exposure control, and the like executed by the control unit 27, for example.

  The image sensor 22 performs so-called photoelectric conversion in which imaging light obtained by the optical system unit 21 is converted into an electric signal. For this purpose, the image sensor 22 receives the imaging light from the optical system unit 21 on the light receiving surface of the photoelectric conversion element, and sequentially accumulates signal charges accumulated according to the intensity of the received light at a predetermined timing. It is made to output. Thereby, an electrical signal (imaging signal) corresponding to the imaging light is output. The photoelectric conversion element (imaging element) employed as the image sensor 22 is not particularly limited, but in the present situation, for example, a CMOS sensor, a CCD (Charge Coupled Device), or the like can be used. When a CMOS sensor is used, a device (component) corresponding to the image sensor 22 may include an analog-digital converter corresponding to an A / D converter 23 described below.

The imaging signal output from the image sensor 22 is input to the A / D converter 23, thereby being converted into a digital signal and input to the signal processing unit 24.
In the signal processing unit 24, for example, the digital imaging signal output from the A / D converter 23 is captured in a unit corresponding to one still image (frame image), and the still image unit captured in this way is captured. By performing necessary signal processing on the signal, it is possible to generate captured image data (captured still image data) that is image signal data corresponding to one still image.

When the captured image data generated by the signal processing unit 24 as described above is recorded as image information in the memory card 40 that is a storage medium, for example, captured image data corresponding to one still image is stored in the signal processing unit 24. To the encode / decode unit 25.
The encoding / decoding unit 25 performs compression encoding on the captured image data in units of still images output from the signal processing unit 24 by a predetermined still image compression encoding method, and performs control of the control unit 27, for example. Accordingly, a header or the like is added and converted into a format of captured image data compressed into a predetermined format. Then, the captured image data generated in this way is transferred to the media controller 26. The media controller 26 writes and records the transferred captured image data in the memory card 40 under the control of the control unit 27. The memory card 40 in this case is a storage medium having a configuration including, for example, a card-type outer shape conforming to a predetermined standard and having a nonvolatile semiconductor storage element such as a flash memory inside. Note that the storage medium for storing the image data may be of a type or format other than the memory card.

  Further, the signal processing unit 24 according to the present embodiment can execute image processing as subject detection using captured image data. The subject detection process in this embodiment will be described later.

  Further, the digital still camera 1 can display a so-called through image that is an image currently being captured by causing the display unit 33 to perform image display using captured image data obtained by the signal processing unit 24. It is said. For example, the signal processing unit 24 captures the imaging signal output from the A / D converter 23 as described above and generates captured image data corresponding to one still image, but this operation is continued. Thus, captured image data corresponding to frame images in the moving image is sequentially generated. The captured image data sequentially generated in this way is transferred to the display driver 32 under the control of the control unit 27. Thereby, a through image is displayed.

  The display driver 32 generates a drive signal for driving the display unit 33 based on the captured image data input from the signal processing unit 24 as described above, and outputs the drive signal to the display unit 33. To be. As a result, the display unit 33 sequentially displays images based on the captured image data in units of still images. If this is seen by the user, the image taken at that time is displayed on the display unit 33 as a moving image. That is, a through image is displayed.

The digital still camera 1 can also reproduce captured image data recorded on the memory card 40 and display the image on the display unit 33.
For this purpose, the control unit 27 designates captured image data and instructs the media controller 26 to read data from the memory card 40. In response to this command, the media controller 26 accesses the address on the memory card 40 where the designated captured image data is recorded, executes data reading, and sends the read data to the encoding / decoding unit 25. Forward.

  The encode / decode unit 25 extracts, for example, actual data as compressed still image data from the captured image data transferred from the media controller 26 under the control of the control unit 27, and decodes the compressed still image data with respect to compression encoding. Processing is executed to obtain captured image data corresponding to one still image. Then, the captured image data is transferred to the display driver 32. Thereby, on the display unit 33, the image of the captured image data recorded in the memory card 40 is reproduced and displayed.

  On the display unit 33, a user interface image can be displayed along with the through image and the reproduced image of the captured image data. In this case, for example, the control unit 27 generates display image data as a necessary user interface image according to the operation state at that time, and outputs the display image data to the display driver 32. As a result, the user interface image is displayed on the display unit 33. The user interface image can be displayed on the display screen of the display unit 33 separately from the through image and the reproduced image of the captured image data, such as a specific menu screen. It is also possible to display the image data so as to be superimposed and synthesized on a part of the reproduced image.

The control unit 27 actually includes a CPU (Central Processing Unit), for example, and constitutes a microcomputer together with the ROM 28, the RAM 29, and the like. The ROM 28 stores, for example, various setting information related to the operation of the digital still camera 1 in addition to a program to be executed by the CPU as the control unit 27. The RAM 29 is a main storage device for the CPU.
In this case, the flash memory 30 is provided as a non-volatile storage area used for storing various setting information that needs to be changed (rewritten) according to, for example, a user operation or an operation history. It is what For example, if a non-volatile memory such as a flash memory is adopted as the ROM 28, a partial storage area in the ROM 28 may be used instead of the flash memory 30.

  The operation unit 31 collects various operation elements provided in the digital still camera 1 and operation information signal output parts that generate operation information signals according to operations performed on these operation elements and output them to the CPU. As shown. The control unit 27 executes predetermined processing according to the operation information signal input from the operation unit 31. Thereby, the operation of the digital still camera 1 according to the user operation is executed.

  The pan head communication unit 34 is a part that performs communication according to a predetermined communication method between the pan head 10 side and the digital still camera 1 side. For example, the digital still camera 1 communicates with the pan head 10. In the attached state, a physical layer configuration for enabling transmission / reception of wired or wireless communication signals to / from the communication unit on the camera platform 10 side, and communication processing corresponding to a predetermined layer higher than this And a configuration for realizing.

FIG. 4 is a block diagram illustrating a configuration example of the camera platform 10.
As described above, the pan head 10 includes a pan / tilt mechanism, and includes a pan mechanism unit 53, a pan motor 54, a tilt mechanism unit 56, and a tilt motor 57 as corresponding parts. .
The pan mechanism unit 53 is configured to have a mechanism for giving the pan (lateral) movement shown in FIG. 2A with respect to the digital still camera 1 attached to the pan head 10. Is obtained by rotating the pan motor 54 in the forward and reverse directions. Similarly, the tilt mechanism unit 56 is configured to have a mechanism for imparting a motion in the tilt (vertical) direction shown in FIG. 2B for the digital still camera 1 attached to the camera platform 10. This mechanism movement is obtained by rotating the tilt motor 57 in the forward and reverse directions.

The control unit 51 includes a microcomputer formed by combining a CPU, a ROM, a RAM, and the like, for example, and controls the movement of the pan mechanism unit 53 and the tilt mechanism unit 56. For example, when the control unit 51 controls the movement of the pan mechanism unit 53, a control signal corresponding to the amount and direction of movement necessary for the pan mechanism unit 53 is output to the pan drive unit 55. The pan drive unit 55 generates a motor drive signal corresponding to the input control signal and outputs the motor drive signal to the pan motor 54. By this motor drive signal, the pan motor 54 rotates, for example, at a required rotation direction and rotation angle, and as a result, the pan mechanism unit 53 is also driven to move according to the corresponding movement amount and movement direction.
Similarly, when controlling the movement of the tilt mechanism unit 56, the control unit 51 outputs a control signal corresponding to the movement amount and movement direction necessary for the tilt mechanism unit 56 to the tilt drive unit 58. The tilt drive unit 58 generates a motor drive signal corresponding to the input control signal and outputs it to the tilt motor 57. By this motor drive signal, the tilt motor 57 rotates, for example, in a required rotation direction and rotation angle, and as a result, the tilt mechanism unit 56 is also driven to move according to the corresponding movement amount and movement direction.

  The communication unit 52 is a part that performs communication according to a predetermined communication method with the pan-head compatible communication unit 34 in the digital still camera 1 attached to the pan head 10. Similarly, it has a physical layer configuration for enabling transmission / reception of wired or wireless communication signals with a counterpart communication unit, and a configuration for realizing communication processing corresponding to a predetermined layer higher than this. It consists of

Next, the block diagram of FIG. 5 shows a functional configuration example realized by hardware and software (program) for the digital still camera 1 and the camera platform 10 constituting the imaging system corresponding to the present embodiment.
In this figure, the digital still camera 1 includes an imaging / recording block 61, a composition determination block 62, a pan / tilt / zoom control block 63, and a communication control processing block 64.

The imaging record block 61 is a part that obtains an image obtained by imaging as image signal data (captured image data) and executes control processing for storing the captured image data in a storage medium. This part includes, for example, an optical system for imaging, an image sensor (image sensor), a signal processing circuit that generates captured image data from a signal output from the image sensor, and also records the captured image data in a storage medium. This is a part having a recording control / processing system for (storing).
In this case, the recording (imaging recording) of the captured image data in the imaging recording block 61 is executed by the instruction and control of the composition determination block.

The composition determination block 62 captures and inputs the captured image data output from the imaging record block 61, first performs subject detection based on the captured image data, and finally executes a process for composition determination. To do.
In the present embodiment, at the time of this composition determination, an attribute described later is also detected for each subject detected by subject detection. In the composition determination process, an optimum composition is determined using the detected attribute. Further, composition adjustment control is performed to obtain captured image data of the image content according to the determined composition.
Here, the subject detection processing (including initial face frame setting) executed by the composition determination block 62 can be configured to be executed by the signal processing unit 24 in correspondence with FIG. The subject detection process by the signal processing unit 24 can be realized as an image signal process by a DSP (Digital signal Processor). That is, it can be realized by a program or instruction given to the DSP.
Further, the face frame correction, composition determination, and composition adjustment control executed by the composition determination block 62 can be realized as processing executed by the CPU as the control unit 27 according to a program.

The pan / tilt / zoom control block 63 executes pan / tilt / zoom control in accordance with an instruction from the composition determination block 62 so as to obtain a composition and an imaging viewing angle corresponding to the determined optimum composition. That is, as composition adjustment control, the composition determination block 62 instructs the pan / tilt / zoom control block 63 on the composition and the imaging viewing angle to be obtained according to, for example, the determined optimum composition. The pan / tilt / zoom control block 63 obtains the amount of movement of the pan / tilt mechanism for the pan / tilt head 10 so that the digital still camera 1 faces in the imaging direction in which the instructed composition and imaging viewing angle are obtained. A pan / tilt control signal for instructing movement according to the amount of movement is generated.
Further, for example, a zoom position for obtaining the determined appropriate angle of view is obtained, and the zoom mechanism that the imaging recording block 61 is provided is controlled so as to be the zoom position.

  The communication control block 64 is a part for executing communication with the communication control block 71 provided on the camera platform 10 side according to a predetermined communication protocol. The pan / tilt control signal generated by the pan / tilt / zoom control block 63 is transmitted to the communication control block 71 of the camera platform 10 through communication of the communication control block 64.

The camera platform 10 includes a communication control block 71 and a pan / tilt control processing block 72 as shown in the figure, for example.
The communication control block 71 is a part for executing communication with the communication control block 64 on the digital still camera 1 side. When the pan / tilt control signal is received, the pan / tilt control is performed. The signal is output to the pan / tilt control processing block 72.

The pan / tilt control processing block 72 corresponds to an execution function of processing related to pan / tilt control among control processing executed by a microcomputer or the like on the camera platform 10 (not shown).
The pan / tilt control processing block 72 controls a pan driving mechanism unit and a tilt driving mechanism unit not shown here according to the input pan / tilt control signal. Thereby, panning and tilting are performed to obtain the required horizontal viewing angle and vertical viewing angle according to the optimum composition.

  In this case, the composition determination block 62 performs subject detection processing as described later. When no subject is detected as a result of the subject detection processing, the pan / tilt / zoom control block 63 responds to a command, for example. Thus, pan / tilt / zoom control for searching for a subject can be performed.

2. Basic composition determination processing example

A basic composition determination process when a plurality of subjects are detected by the subject detection process by the imaging system of the present embodiment will be described. Here, for the sake of convenience of explanation, the case where two subjects are detected is taken as an example.

Here, the subject detection processing executed by the composition determination block 62 refers to processing for discriminating and detecting a subject as a person from the image content of the captured image data taken in.
As a specific method of the subject detection process, a face detection technique can be used. Several face detection methods and methods are known, but in this embodiment, which method should be adopted is not particularly limited, and is appropriate in consideration of detection accuracy and design difficulty. It is only necessary to adopt the method.
In addition, in the present embodiment, for example, in response to subject detection by applying face detection as described above, based on the detected subject, that is, the detected image area portion of the face, the center of gravity ( Set the subject center of gravity. As will be understood from the following description, the subject center of gravity is effectively used for the composition determination process.
There are several ways to set the subject center of gravity. In the simplest example, a rectangular frame (face frame) is set corresponding to the detected face area of the subject, and the intersection of square diagonal lines as the face frame is used as the subject center of gravity. .

  FIG. 6 shows a state where two subjects SBJ1 and SBJ2 are detected in the image frame 300 by subject detection processing. In addition, in this figure, the subject gravity centers G1 and G2 are shown together with the two detected subjects SBJ1 and SBJ2. The subject gravity center G1 is set corresponding to the subject SBJ1, and the subject gravity center G2 is set corresponding to the subject SBJ2.

  The image frame 300 in this figure is formed by arranging square cells on a matrix. Each square schematically represents a pixel that forms frame image data corresponding to the image frame 300. That is, in this figure, it is schematically shown that the image frame is formed by the arrangement of pixel data. The number of pixels adopted as an actual image frame (frame image region) is, for example, horizontal pixel number × vertical pixel number = 640 × 480.

In addition, when a plurality of subjects are detected in the image frame 300 in this way, it is possible to set a total center of gravity (total subject center of gravity) corresponding to all of these subjects.
There are several ways to set the total subject center of gravity. In FIG. 6, as a case corresponding to the case where there are two detected subjects, the midpoint of the line segment connecting the subject gravity centers G1 and G2 of the two subjects is set as the total subject gravity center Gt.

  Note that the total subject gravity center can be set by a predetermined algorithm corresponding to a case where the number of detected subjects is three or more. For example, a polygon may be formed by a line connecting the detected subject centroids of the subject, and the total subject centroid may be set based on the position of the centroid obtained for the polygon.

  As an example to the last, as a composition determination process, when a subject is detected as shown in FIG. 6, the composition relating to the arrangement position of these subjects is set as follows. Here, in order to simplify the explanation, it is assumed that the sizes of the subjects SBJ1 and SBJ2 in the image frame 300 are appropriate as composition elements.

  In FIG. 6, two virtual lines Lx and Ly are set on the image frame 300. The virtual line Lx is a vertical line passing through the midpoint of the horizontal size (corresponding to the number of horizontal pixels) Cx of the image frame 300. The virtual line Ly is an upper line of two horizontal lines that divide the vertical size Cy (corresponding to the number of vertical pixels) Cy of the image frame 300 into three equal parts (Cy / 3).

Then, the intersection of the virtual lines Lx and Ly is set as the target position TP. This target position TP is related to the composition element of the subject arrangement, and is the position (coordinates) in the image frame 300 where the total subject gravity center Gt should be in order to obtain the subject position for obtaining the optimum composition. That is, as the composition determination process in this case, when two subjects are detected in the image frame 300, the state where the subject center of gravity Gt is located at the intersection of the virtual lines Lx and Ly is the optimum composition. Judgment.
Incidentally, the target position TP shown in this figure is set according to a so-called three-division method. However, this is merely an example, and as a composition determination process, the concept and composition determination method for determining the target position TP in the image frame 300 should not be limited.

FIG. 7 shows the image contents when the total subject gravity center Gt is positioned with respect to the target position TP shown in FIG. In order to obtain the image content shown in FIG. 7 from the image content state shown in FIG. 6, for example, the pan / tilt / zoom control block 62 that has received an instruction from the composition determination block 62 moves the subject with respect to the target position TP. Control for driving the pan / tilt mechanism of the camera platform 10 is executed so that the center of gravity Gt is located.
Note that performing pan / tilt / zoom control so that the determined composition is obtained is also referred to as “composition adjustment control”. The composition determination process and composition adjustment control are also referred to as “composition control”.

In FIG. 7, the face portions of the subjects SBJ 1 and SBJ 2 are positioned almost just above the virtual line Ly in the image frame 300. In this case, in response to the fact that the two subjects SBJ1 and SBJ2 are substantially aligned on the left and right, in the left-right direction of the image frame 300, the subjects (SBJ1 and SBJ2 are not centered). They are arranged so as to be substantially equidistant from the virtual line Lx).
Incidentally, such an arrangement of the subject is considered to be one of the typical compositions that are good under the three-division method, for example.

3. Composition determination according to the age (first example)

Two subjects SBJa and SBJc are detected in the image frame 300 shown in FIG. 8, and the positions thereof are substantially side by side. This is the same as in the case of FIG.
However, in FIG. 8, it is assumed that the subject SBJa is an adult and the subject SBJc is a child with respect to the age attribute of a subject that is a person. That is, when viewed as the attributes of the subject existing in the image frame, there is a relationship between an adult and a child.

  In general, when adults and children are mixed as subjects, it is often the case that children are emphasized. That is, in this case, the importance of the plurality of subjects is not equal, and the child has a higher importance as the subject. In such a case, according to the basic composition determination according to FIGS. 6 and 7, the child is placed closer to the center than when the composition control is performed so that the two subjects are approximately equidistant from the center in the left-right direction. It is possible to obtain a composition that is better to be arranged.

Therefore, in the present embodiment, when a plurality of subjects are detected, composition determination is performed so that the children are arranged closer to the center with respect to age attributes of these subjects when adults and children are mixed.
The composition determination process for this is as follows in the case of FIG.
In this case, as in the case of the basic composition determination process, the subject gravity centers G1 and G2 are set for each of the adult subject SBJa and the child subject SBJc. Then, the total subject gravity center Gt is set on the line segment connecting the subject gravity centers G1 and G2. At this time, as shown in the drawing, the ratio according to a predetermined weight is set to be higher than the midpoint of the line segment. The position of the total subject gravity center Gt is set by shifting to the side closer to the gravity center G2 of the child subject SBJc.

Note that the above ratio sets the amount by which the position of the total subject center of gravity Gt is shifted from the midpoint of the line segment, but as a result, as will be understood from FIG. It has the effect of bringing the child's subject center of gravity G2 closer (biased) to the target position TP. For this reason, the ratio is also referred to as a bias rate g.
This deviation rate g can be defined as taking a range of −1 ≦ g ≦ 1, for example. When the deviation rate g = 0, the total subject gravity center Gt is located at the midpoint of the line connecting the subject gravity centers G1 and G2. As the deviation rate g approaches -1, the position of the total subject gravity center Gt becomes closer to the subject gravity center G1 of the adult subject SBJa, and coincides with the subject gravity center G1 when g = -1. Further, as the deviation rate g approaches 1, the position of the total subject gravity center Gt becomes closer to the subject gravity center G2 of the child subject SBJc, and coincides with the subject gravity center G2 when the deviation rate g = 1.

  In this case, there are two detected subjects, and they are in a positional relationship that can be regarded as being almost side by side, and in this respect, it is the same as in the case of FIG. Therefore, the target position TP in this case is set to the same position as in FIG. 6 according to the same rules as in FIG.

  FIG. 9 shows the result of performing composition adjustment control in accordance with the composition determined as shown in FIG. As in the case of the basic example in FIGS. 6 and 7, the composition adjustment control in this case also performs pan / tilt control so that the total subject gravity center Gt is positioned with respect to the target position TP.

  For example, as can be seen by comparing FIG. 9 and FIG. 7, in FIG. 9, the child subject SBJc is arranged closer to the center in the left-right direction than the adult subject SBJa. That is, a good composition suitable for the case where the two subjects are an adult and a child is obtained.

4). Composition determination according to the age (second example)

In the image frame 300 of FIG. 10, two subjects are detected, and as a classification by age attribute, a state in which one adult subject SBJa and one child subject SBJc are detected is shown. The relationship from the viewpoint of the adult / child attributes is the same as in FIG. 8, but in FIG. 10, the adult subject SBJa and the child subject SBJc are considered to be aligned along the vertical direction. The positional relationship is possible. Here, the adult subject SBJa is on the bottom and the child subject SBJc is on the top.

  Even in a state where a plurality of subjects are arranged in the vertical direction in this way, when the adult subject and the child subject are mixed with respect to the age attribute, for the same reason as in FIGS. Even in the vertical direction, it is possible to obtain a composition in which the child's subject is better positioned from the center.

Therefore, for example, in the case of FIG. 10, the following composition determination is performed.
Also in this case, subject gravity centers G1 and G2 are set for the adult subject SBJa and the child subject SBJc, respectively, by the same algorithm as in the basic example. Then, in accordance with the case of FIG. 8, the total subject gravity center Gt is set according to a predetermined deviation rate g on a line segment connecting the subject gravity centers G1 and G2.
Note that the deviation rate g at this time does not have to be the same as in the case of the first example in FIG. 9 (when the subject is horizontally arranged), and is appropriate according to the fact that the subject is vertically aligned. May be a value.

  In addition, when the two subjects are vertically aligned as shown in FIG. 10, two vertical virtual lines Lx1 and Lx2 that divide the horizontal size of the image frame 300 into three equal parts are set as virtual lines. In addition, two horizontal virtual lines Ly1 and Ly2 that divide the vertical size into three equal parts are set.

Depending on the setting of the virtual line, there are four intersections of the virtual lines in the image frame. In the case of FIG. 10, the virtual line Lx2 located at the lower right of these intersections and A target position TP is set for the intersection of the virtual line Ly2.
In the case of FIG. 10, the child subject SBJc is on the upper side and the adult subject SBJa is on the lower side. In the case of this positional relationship, the target position TP is first set as an imaginary line along the horizontal direction with respect to the intersection on the lower imaginary line Ly2.
There are two intersections on the virtual line Ly2: an intersection with the left virtual line Lx1 and an intersection with the right virtual line Lx2. Here, the target position TP is set at the right intersection. Here, as to whether the target position TP is set at the intersection point on the right side or the left side on the virtual line Ly2, it is conceivable that one of them is uniquely determined as one. Moreover, it is good also as determining which intersection is made into the right side or the left side based on other specific conditions, such as attributes other than the age, for example.

FIG. 11 shows the result of performing composition adjustment control in accordance with the composition determination result described with reference to FIG.
As can be seen from this figure, among the adult subject SBJa and the child subject SBJc that are in the vertical relationship, the child subject SBJc is arranged in the center in the vertical direction of the image frame 300. . In other words, a good composition is obtained in which children are arranged from the center rather than adults.

  For confirmation, in the case where the adult subject SBJa is on and the child subject SBJc is on the bottom and the positional relationship is opposite to that in FIG. A target position TP is set for the intersection of Lx2 (or virtual line Lx1) and the upper virtual line Ly1 along the horizontal direction. As a result of the composition adjustment, a composition in which the child subject SBJc is arranged from the center is obtained.

As can be seen from FIG. 11, in the positional relationship in which the adult subject SBJa and the child subject SBJc are up and down, the bias rate g is set to be high and the total subject gravity center Gt is set to the child subject gravity center G1 side. If it is too close, the composition tends to obtain a composition in which the child's subject SBJc is below the center as a result of composition control. In addition, there may be a case where the adult subject SBJa exceeds the allowable range and protrudes too much from the image frame, resulting in a composition that is not good. Therefore, it is preferable to set a smaller value (close to 0) than the case of FIG.
In reality, when there are three or more subjects, it is naturally possible that an adult subject is positioned both in the horizontal direction and in the vertical direction of the child subject. In consideration of this, the deviation rate g may be obtained separately for the horizontal direction component and the vertical direction component.

Although illustration is omitted, in the case of FIG. 10, when the composition determination process is performed in the basic example, the following is performed.
That is, the total subject gravity center Gt is set with respect to the midpoint of the line segment connecting the subject gravity centers G1 and G2, and the target position TP is, for example, within the image frame on the vertical virtual line Lx2 (or virtual line Lx1). Will be set for the midpoint. As a result, along the vertical direction, the composition is such that the adult subject SBJa and the child subject SBJc are arranged at an approximately equal distance from the center.

5). Composition determination according to gender

FIG. 12 shows a state in which two subjects are detected in the image frame 300, and regarding the gender attributes, the left side is a male subject SBJm and the right side is a female subject SBJw. That is, when viewed as a gender attribute, it has a relationship that is a combination bubble of men and women.

  Here, as one of the composition elements, the size of the subject in the image frame 300 can be given. That is, it is said that one of the conditions for obtaining a good composition is that the image frame is not too small and not too large and is of an appropriate size.

  For example, it is assumed that the size of the subjects SBJm and SBJw at the time of detection shown in FIG. 12 is too small to satisfy the optimum composition. Therefore, in this case, an enlargement ratio for enlarging the subjects SBJm and SBJw to an appropriate size is set by the composition determination process. Note that the enlargement ratio here is for an image obtained by imaging, and therefore corresponds to the zoom magnification set in the zoom mechanism in the imaging region.

The enlargement ratio (zoom magnification) is set as follows, for example.
As described above, when subject detection is performed using the face detection method, a frame is set as the detection result for the face portion of the detected subject. Here, this frame is called a face frame. In FIG. 12, it is shown as a face frame FR. The face frame FR in this case is arranged in a square shape corresponding to the face image portion in the detected subject.

Next, FIG. 13 shows an image content obtained by performing zoom control as composition adjustment control in accordance with the enlargement ratio set under the image content shown in FIG.
The image content shown in FIG. 12 is in a positional relationship that allows the subject to be considered to be substantially side by side. In such a case, the distance from the face frame FR of the leftmost subject to the face frame FR of the rightmost subject is handled as the horizontal (lateral direction) size K of the subject overall.
In this case, the appropriate subject size means that the horizontal size K of the subject overall occupies a fixed ratio (occupancy) n (0 <n> 1) with respect to the horizontal size Cx of the image frame 300. It prescribes that there is. That is, the enlargement ratio may be set so that K = Cx · n holds. For example, the horizontal size of the subject overall at the time of subject detection obtained corresponding to FIG. 12 is K1, the horizontal size of the subject overall after composition control obtained corresponding to FIG. If Z is taken as Z), the enlargement ratio Z can be obtained by Z = Cx · n / K1.
FIG. 13 shows the result of performing zoom control from the state shown in FIG. 12 with the zoom magnification corresponding to the enlargement factor Z thus obtained.

However, the enlargement ratio set here corresponding to FIG. 13 is assumed to be obtained according to a normal enlargement ratio setting algorithm when a plurality of subjects are detected.
The enlargement ratio obtained according to this normal algorithm is appropriate if, for example, two subjects are detected, and the sex attribute of these subjects is male or female and are of the same sex. And an appropriate composition can be obtained.
However, when two subjects are detected, if these subjects are a couple of men and women and are a so-called couple, it is better to use a subject size with a larger magnification than the normal magnification as a photographic image. The atmosphere is improved, that is, a better composition is obtained.

  Therefore, when the number of detected subjects is 2 and the attributes regarding the sexes of these subjects are male and female, respectively, the occupancy rate n corresponding to the normal time is set when setting the enlargement rate in the composition determination process. A larger occupancy m is used.

FIG. 14 shows the image contents obtained by performing zoom control with the enlargement ratio set using the occupation ratio m.
Since the occupancy ratios m and n are in the relationship of m> n, the distance K (= Cx · m) shown in FIG. 14 is larger than the distance K = (Cx · n) in FIG. Yes. That is, compared with FIG. 13, the size of the male subject SBJm and the female subject SBJw in the image frame 300 is greatly enlarged in FIG. That is, the composition is more appropriate according to the subject being a couple.

  Although illustration explanation is omitted, when the detected gender attributes of the two subjects are male and female, respectively, and the positional relationship between these subjects is up and down in the image frame In this case, according to the above description, the enlargement ratio may be set larger than that in the normal time. However, the enlargement ratio (occupancy ratio m) should be set to an appropriate value in accordance with the subject's positional relationship being up and down, and is not necessarily set when the subject's positional relationship is horizontal. It is not necessary to be the same as the rate (occupancy rate m).

6). Composition determination according to face direction

The composition determination block 62 of the present embodiment can be configured to detect the face direction of each detected subject in the subject detection process. The face direction here refers to the direction detected as the subject's face is facing in the image frame.
In addition, when detecting the face direction, there is a method of outputting the detection results in a stepwise manner, for example, front, left, right, etc. Here, a method of detecting the face direction by a vector is adopted. Shall. As for the face direction, it is possible to detect a horizontal component (corresponding to left, front, right) and a vertical component (corresponding to upward, front, downward) Here, for the sake of simplicity, it is assumed that only the lateral component is detected.

For the face direction vector corresponding to the horizontal direction, the coordinate axes shown in FIG. 17 are defined. That is, an X axis and a Y axis that are orthogonal to each other are defined. The X axis corresponds to the left-right direction, and the Y axis corresponds to the front-rear direction. However, since the face direction is not detected for the head facing backward, only the front side from the intersection is considered to be effective for the Y axis.
In addition, the face direction is expressed as a unit vector VT indicating the direction in which the face faces from the intersection of the X axis and the Y axis in this coordinate.
As shown in the figure, this unit vector VT is 0 ° when it coincides with the Y axis, 90 ° when it coincides with the X axis corresponding to the left side, and 270 ° when it coincides with the X axis corresponding to the right side. The angle ranges from 0 ° to 90 and 0 ° to 270 ° depending on the face direction along the left-right direction. The angle taken by this unit vector VT is represented as θ.
In this case, the quantity value (face direction vector amount) S indicating the orientation of the face along the left-right direction is:
S = VTsinθ
Can be represented by
That is, the face direction vector amount S is in the range of −1 ≦ S ≦ 1, and is 0 when facing the front. The face direction vector amount S is closer to 1 as it goes to the left, and closer to −1 as it goes to the right. Go.

  As for composition, when the subject is facing the other direction than the front, a large space is left on the side facing the subject in the image frame, that is, the side where the subject's line of sight is facing. The composition gives a good atmosphere. That is, a composition can be obtained by arranging the subject in a direction opposite to the direction in which the subject is facing in the image frame. In addition, if the amount of the subject (the distance away from the center) is changed according to the degree of the orientation of the subject, a better composition can be obtained.

Therefore, in the present embodiment, the face direction is detected as the subject attribute, and the position of the subject in the image frame is shifted based on the detected face direction.
In addition, the detection result of the face direction is output as a vector amount, and the amount by which the position of the subject is shifted in the image frame is changed according to the vector amount.

  FIG. 15 shows a state where two subjects SBJ1 and SBJ2 are detected in the image frame 300. These subjects SBJ1 and SBJ2 are both directed to the right side in the image frame 300 (screen). As a result of detecting the face direction, the face direction vector amount S1 is detected for the subject SBJ1, and the face direction for the subject SBJ2. It is assumed that the vector quantity S2 is detected.

  These subjects SBJ1 and SBJ2 can be considered to have a substantially horizontal positional relationship in the image frame 300. Therefore, as the composition determination process, first, the subject gravity centers G1 and G2 are set for each of the subjects SBJ1 and SBJ2, and the total subject gravity center Gt is set for the midpoint of the line segment connecting the subject gravity centers G1 and G2.

Here, when the basic examples of FIGS. 7 and 8 are followed, the target position TP is set for the intersection of the virtual lines Lx and Ly.
However, in this case, the movement amount Δx corresponding to the value obtained by adding the face direction vector amounts S1 and S2 is obtained using a predetermined function or the like. In this case, the position shifted in the horizontal direction by the movement amount Δx from the intersection of the virtual lines Lx and Ly is set as the target position TP.
The total value S1 + S2 of the face direction vector amounts S1 and S2 indicates the total face direction vector amount (S_total) obtained by combining all the detected subjects as one subject. This face direction vector amount S_total of the subject overall can be regarded as indicating the relationship between the individual subjects regarding the face direction attribute.
In the present embodiment, in correspondence with the face direction vector amount S_total of the entire subject, an amount of shifting the subject in the image frame opposite to the direction in which the face is facing, that is, an amount of movement Δx is determined. Is.

  Specifically, in FIG. 15, both the subjects SBJ1 and SBJ2 face the right side in the image frame 300 (screen). In other words, the face direction tends to be rightward as a whole subject. Therefore, the target position TP set by the movement amount Δx obtained based on the face direction vector amounts S1 and S2 of the subjects SBJ1 and SBJ2 is the intersection of the virtual lines Lx and Ly, that is, horizontal, as shown in the figure. It is located on the left side of the center in the direction. In addition, the distance from the intersection of the virtual lines Lx and Ly to the target position TP corresponds to S1 + S2.

  As composition adjustment control, pan / tilt control is performed by moving the total subject gravity center Gt with respect to the target position TP moved by the movement amount Δx in this way.

FIG. 16 shows the image content obtained by performing composition adjustment control in accordance with the composition determination result described with reference to FIG.
In this way, in a state where the total subject gravity center Gt is located with respect to the target position TP, the subjects SBJ1 and SBJ2 are arranged so as to be biased to the left as a whole in the image frame 300. In other words, the composition is such that a large space is left on the right side of the image frame 300, that is, on the right side of the left side in the direction in which the face of the entire subject is facing. In addition, the size of the space vacated on the right side, that is, the amount of deviation to the left side of the subjects SBJ1 and SBJ2 in the image frame, depends on the degree of face orientation (vector amount) of the subject overall.
That is, in FIG. 16, a good composition is obtained that is suitable for the direction in which the subject faces the face in the screen and the degree to which the face is directed.

  The simplest example for obtaining the movement amount Δx is that the face direction vector amount per unit is made to correspond to a certain number of pixels px, and the face direction vector amount of the subject overall is S_total, and Δx = S_total px can be calculated.

As described above, in the present embodiment, not only the face direction in the horizontal direction but also the face direction in the vertical direction can be detected. That is, it is possible to obtain a face direction vector amount corresponding to the vertical direction that indicates how much the face is facing upward or downward.
Then, in accordance with the description of FIG. 15 and FIG. 16 above, based on the face direction vector amount corresponding to the vertical direction, Δy, which is the movement amount of the target position TP in the vertical (vertical) direction, is obtained to obtain the target position TP. It is also possible to configure so as to move.

7). algorithm

The flowchart of FIG. 15 shows an example of a processing procedure for composition control as the present embodiment. The processing shown in this figure can be regarded as appropriately executed by each functional part of the digital still camera 1 shown in FIG. 5 as necessary. Further, the processing executed by each of these functional parts can be viewed as a control and processing procedure realized by the control unit (CPU) 27 of FIG. 1 executing a program.

  In FIG. 15, first, in step S101, the composition determination block 62 (signal processing unit 24) starts capturing (input) of captured image data that is assumed to be obtained at that time in the imaging recording block 61. To do.

In the next step S102, the composition determination block 62 (signal processing unit 24) executes subject detection processing using the captured image data that has been captured.
As the subject detection processing, for example, the face detection technology is applied as described above, and as a result of the detection, as described above, each detected subject is applied to the area of the image portion of the face. Correspondingly, the face frame FR is set. For example, basic information about the subject such as the number of subjects, the subject size at the time of subject detection, and the position in the image frame can be obtained from the number, size, position, etc. of the face frame FR. In addition, according to the setting of the face frame FR, the subject gravity centers G1, G2,... And the total subject gravity center Gt for each subject are also acquired at this stage.
There are several known face detection methods and methods, but in this embodiment, which method should be adopted is not particularly limited, taking into account detection accuracy, design difficulty, etc. It is only necessary to adopt an appropriate method.

  In step S103, it is determined whether or not at least one subject has been detected by the subject detection process in step S102. If a negative determination result is obtained here, the process returns to step S102, and subject detection processing for subject search is executed. The subject search here means that the subject exists by controlling the movement of the camera platform 10 in the pan / tilt direction on the digital still camera 1 side and changing the imaging viewing angle by performing zoom control. This means that the captured image data is obtained.

If an affirmative determination result is obtained in step S103 that the subject is detected, the process proceeds to step S104.
In step S104, for each subject detected in step S102, a process for detecting an attribute required for composition determination is executed.

In the explanation so far, composition control based on each subject attribute of age, gender, and face direction was described, but in the algorithm described here, these attributes of age, gender, and face direction are all comprehensive. After use, the composition determination process and composition control are executed. That is, in this algorithm, an appropriate composition is obtained by reflecting all attributes of age, gender, and face direction.
For this reason, in step S104, for each subject detected corresponding to step S102, the age (adult, child), sex (male, female), and face direction are detected as attributes.

The attribute detection process may be configured to be executed by the signal processing unit 24 as a DSP in practice.
For detection of age, sex, and face direction, for example, techniques and algorithms that have been known so far may be applied.

At the stage where the processing in step S104 is completed, for each detected subject, information on the face frame FR (position, size, etc.), subject gravity center and total subject gravity center, subject age, gender, face detected as subject information Information indicating the direction is obtained.
Therefore, depending on step S105, the composition determination processing is executed using the subject information described above as the processing of the composition determination block 62 executed by the control unit 27.
By this composition determination processing, correction of the total subject gravity center Gt, setting of the target position TP, zoom magnification (subject size enlargement ratio), and the like are determined. Information on the composition determination result obtained in step S105 is passed to, for example, the pan / tilt / zoom control block 63.

  In step S106, the pan / tilt / zoom control block 63 executes pan / tilt / zoom control for obtaining an imaging viewing angle corresponding to the composition determination result. That is, composition adjustment control is executed.

  After the composition adjustment control in step S106 is started, the composition determination block 62 determines that the composition actually obtained as an image of the captured image data at that time in step S107 is the composition determined in step S107. It is determined whether or not the state is considered to be the same (for example, a state where the degree of approximation exceeds a certain level) (whether or not the composition is OK).

Here, for example, if the composition does not become OK even if the pan / tilt / zoom drive is performed with the movement amount necessary for the composition adjustment for some reason, a negative determination result is obtained in step S107. can get. In this case, the subject search process is resumed by returning to step S102.
On the other hand, if it is determined in step S107 that the composition is OK, the process proceeds to step S108.

In step S108, for example, in a state where the composition as determined for the image content of the captured image data is obtained, the process waits for the timing to be captured and recorded.
For example, the digital still camera 1 of the present embodiment can detect at least a smile as the facial expression of the detected subject. In addition, for example, it is assumed that a mode in which imaging / recording is to be performed is set at a timing when the detected subject is detected to be smiling by a user operation or the like in advance. In step S108, for example, in accordance with such a smile shooting mode, it is determined whether or not it is time to shoot and record. That is, it is determined whether or not the facial expression of the subject detected in the currently obtained captured image data is a smile.

In step S109, it is determined whether or not the imaging recording timing is OK.
For example, it is assumed that it is detected that the detected facial expression of the subject is smiling during the recording timing determination processing period executed as step S108. Then, a positive determination result is obtained in step S109, and the process proceeds to step S110. On the other hand, if a smile is not detected for the detected facial expression of the subject even after the recording timing determination processing period, a negative determination result is obtained in step S109. In this case, the process returns to step S102, and subject detection processing with subject search is executed.

  In step S110, for example, the control unit 27 instructs the imaging recording block 61 to perform imaging recording. In response to this, an operation of recording the captured image data obtained at that time as a still image file in the memory card 40 is executed.

The flowchart in FIG. 19 shows an example of the procedure as the composition determination process in step S105 in FIG.
First, in step S201, the composition determination block 62 sets 1 as an initial value for the variable n indicating the number assigned to the detected subject.
Then, in step S202, a subject gravity center Gn for the nth subject is calculated and obtained. As described above, as the simplest example, this can be obtained as an intersection of diagonal lines of the face frame set at the time of subject detection.

In step S203, it is determined whether or not the variable n is the maximum value.
If a negative determination result is obtained here, a subject for which the subject center of gravity has not yet been obtained remains. Therefore, the variable n is incremented in step S204 and the process returns to step S202, whereby the subject gravity center Gn for the remaining subject is obtained.
If a positive determination result is obtained in step S203 by obtaining the subject gravity centers G1 to Gn for all the detected subjects, the process proceeds to step S205.

In step S205, the age attribute is referred to as the attribute detection result in step S104 in FIG. 18 to determine whether an adult and a child are mixed as a breakdown of the age of the detected subject. .
If a negative determination result is obtained that the detected subject is only an adult or only a child, the process proceeds to step S206.
In step S206, the total subject gravity center Gt is obtained by calculation using a normal function without considering the deviation rate g. For example, when there are two subjects, the total subject gravity center Gt is obtained as the midpoint of the line segment connecting the subject gravity centers G1 and G2 of each subject.

On the other hand, if a positive determination result is obtained in step S205, the procedure of steps S207 and S208 is executed.
In step S207, the bias rate g is set.
As described above, this deviation rate g is used for the purpose of bringing the child subject closer to the center in the horizontal and vertical directions when the child subject and the adult subject are mixed. This is a parameter used to bring the position of the total subject gravity center Gt closer to the subject gravity center of the child's subject.
Further, as described above, the deviation rate g is set to a value corresponding to the horizontal direction component and a value corresponding to the vertical direction component depending on the positional relationship between the child subject and the adult subject. Can do.

  In step S209, a basic target position TP is set. That is, the coordinates of the target position TP within the image frame are determined. The basic target position TP here is, for example, defined by a basic target position setting algorithm before being moved by the movement amount Δx corresponding to the face direction vector, as described with reference to FIGS. It is the target position TP at which the position within the frame is determined. As described above with reference to FIGS. 6 to 11 and the like, the basic target position TP is determined based on the number of subjects, the positional relationship, and the like.

In step S210, a movement amount Δx for the target position TP is calculated from the face direction vector amounts S1 to Sn obtained for each subject. In step S211, a process of moving the coordinates of the target position TP according to the calculated movement amount Δx is executed.
For confirmation, if all of the detected subjects are facing the front, this movement amount Δx must be zero.

In step 212, it is determined whether or not the detected subject is a couple of men and women. That is, it is determined whether the number of detected subjects is 2 and the sex attribute is a combination of male and female.
If a negative determination result is obtained in step S212, the process proceeds to step S213, and a normal enlargement ratio Z to be applied to subjects other than the couple is set.
On the other hand, if a positive determination result is obtained in step S212, the process proceeds to step S214, and an enlargement ratio Z corresponding to a couple of subjects is set. As described above, the enlargement ratio set here has a larger value than the normal enlargement ratio set in step S213.
For confirmation, the enlargement factor Z corresponds to the zoom magnification used for zoom control in composition adjustment control.

In step S215, the composition determination block 62 instructs the pan / tilt / zoom control block 63 to control composition adjustment. At this time, the composition determination block 62 provides the pan / tilt / zoom control block 63 with the total subject gravity center Gt obtained in step S206 or S208 as the parameter used for composition adjustment control, and the target position obtained in step S211. The coordinates of TP and the enlargement ratio Z obtained in step S213 or step S214 are output.
In response to the instruction from the composition determination block 62, the pan / tilt / zoom control block 63 performs composition adjustment control (pan / tilt / zoom control) as step S106 in FIG. Will be executed.

According to the processing procedure of FIG. 19, when an adult and a child are mixed as subjects, composition control is performed so that the child's subject comes closer to the center, and the subject should be matched within the image frame according to the face direction of the subject. The composition control for moving in the direction and the composition control for enlarging the subject size larger than usual according to the fact that the subject is a couple are combined.
That is, for example, if two subjects of adults and children are detected and both of these subjects are facing right, the subject is only an adult (or only a child) while the subject is brought to the left side in the image frame. The composition can be such that the child's subject is closer to the center than in the case of).

However, as an easy-to-understand example, if the subject is a couple of men and women and both of them are facing significantly right, the target position will be far to the left in the image frame based on the face direction vector. TP is set. In other words, the subject is arranged at a position considerably close to the left side.
In this case, as described with reference to FIG. 14, if the zoom magnification Z that satisfies the condition K = Cx · m is set and the subject size is enlarged as it is according to the zoom magnification Z, the face of the subject is drawn from the image frame. It is also possible that the situation will cause the image to protrude. In this case, the composition is rather bad.

  Therefore, in the present embodiment, with regard to composition control according to attributes, priority is given to composition control in the case of a mixture of adults and children and composition control according to face direction over composition control related to the size of the subject (zoom magnification). And With regard to the enlargement ratio Z that is set in correspondence with the subject being a couple, the subject's face is appropriately image framed within a range in which the enlargement ratio value that satisfies the condition K = Cx · m is the maximum value. Adjustment is performed so that the maximum value corresponding to the state that is within the range is set.

  The flowchart of FIG. 20 shows an example of a procedure when the zoom magnification Z is adjusted as the zoom magnification Z setting process when the subject in step S214 of FIG. 19 is a couple.

  In step S301 in FIG. 20, first, an enlargement ratio Z that satisfies the condition K = Cx · m described in FIG. 14 is calculated. The enlargement factor Z calculated here is a value as the enlargement factor Z that should be set corresponding to the couple's subject, and is the maximum value in the range of the enlargement factor Z that can be set corresponding to the couple's subject. Become.

In step S302, the arrangement state of the face frame of the subject under the composition obtained corresponding to the currently set enlargement ratio (current target zoom ratio) Z is calculated and obtained.
That is, the face when composition control is performed using the total subject gravity center Gt obtained in step S206 or S208 in FIG. 19 and the coordinates of the target position TP obtained in step S211 and the current enlargement ratio Z as parameters. Find the position size of the frame.

  Here, in the present embodiment, if the subject's face frame does not protrude from the side, it is assumed that a composition in which the subject's face is captured in the image frame is secured. Therefore, in step S303, it is determined whether or not the face frame protrudes from the image frame based on the position size information of the subject's face frame obtained in step S302.

If a positive determination result is obtained here, the current enlargement ratio Z is too large. In this case, the process proceeds to step S304, and the enlargement ratio Z is multiplied by a reduction coefficient dec (0 <dec <1) based on a predetermined fixed value to obtain a new current enlargement ratio Z. Then, the process returns to step S302.
By executing such processing, finally, a positive determination result is obtained in step S303 when the maximum enlargement ratio Z is set in a range where the subject's face frame does not protrude from the image frame. As a result, this process is exited. That is, the enlargement factor Z at this time is the enlargement factor Z set in step S214.

Note that, depending on the target position TP set according to the face direction, the possibility that the subject's face protrudes from the image frame as a result of the composition adjustment control can also be said for the normal enlargement ratio Z set in step S213.
Therefore, as a countermeasure for this, the processing according to FIG. 20 may be applied in step S213 of FIG. At this time, in step S301, the enlargement ratio Z that satisfies the condition K = Cx · n described with reference to FIG. 13 is calculated. Further, the reduction coefficient dec in this case is not necessarily the same as that in step S214, and a value suitable for the normal enlargement ratio setting may be set.

Depending on the way of thinking, for example, contrary to the above, composition control of the subject size (magnification rate) is given priority over setting of the target position TP according to the face direction (composition control for bringing the subject to one side of the image frame). Can be taken.
In this case, first, an enlargement ratio Z that satisfies the condition of K = Cx · n or K = Cx · m is set. In addition, the movement amount Δx obtained from the face direction vector amounts S1 to Sn for each subject may be set as the maximum value, and an appropriate movement amount Δx that does not protrude the subject's face frame within this range may be obtained.

8). Modified example

Next, modified examples of the imaging system as the present embodiment will be described.
First, in the imaging system shown in FIG. 21, in the digital still camera 1, the captured image data obtained by the imaging recording block 61 is transmitted from the communication control processing block 64 to the communication control block 71 on the pan head 10 side. Has been.

In FIG. 21, a communication control processing block 71, a pan / tilt control processing block 72, and a composition determination block 73 are shown as the configuration of the camera platform 10.
The captured image data received by the communication control processing block 71 is output to the composition determination block 73. For example, the composition determination block 73 shown in FIG. 5 is applied to the composition determination block 73. That is, subject detection, attribute detection, and composition determination processing are executed based on the input captured image data. In this case, for example, the movement amount of the pan mechanism unit and the tilt mechanism unit for obtaining the imaging direction (imaging viewing angle) in which the determined optimum composition is obtained is obtained, and the pan / tilt commanding the movement amount is obtained. A control signal is output to the pan / tilt control processing block 72. Thus, panning and tilting are performed so that the optimum composition determined by the composition determination block 73 is obtained.
In this way, the imaging system shown in FIG. 21 transmits the captured image data from the digital still camera 1 to the camera platform 10, and the camera platform 10 side determines the composition based on the captured image data and responds accordingly. The pan / tilt control is executed.
Further, in the configuration shown in FIG. 21, in order to enable zoom (view angle) control as the imaging viewing angle control, the communication between the communication control processing blocks 63 and 71 is used to control the pan head. The angle of view according to the optimum composition determined by the composition determination block 73 is instructed to the image-recording block, and the image-recording block 61 executes the driving of the zoom lens so that the angle of view is instructed. What is necessary is just to comprise.

FIG. 22 shows a configuration example as another modification of the imaging system corresponding to the present embodiment. In this figure, the same parts as those in FIG.
In this system, an imaging recording block 75 is provided on the camera platform 10 side. The imaging recording block 75 includes an optical system for imaging and an imaging element (image sensor), for example, similarly to the imaging recording block 61 of FIG. 5, so as to obtain a signal (imaging signal) based on imaging light. And a signal processing unit for generating captured image data from the captured image signal and a recording control system for captured image data.
The captured image data generated by the imaging recording block 75 is output to the composition determination block 73.
Note that the direction in which the imaging recording block 75 captures the imaging light (imaging direction) is preferably set so as to match the imaging imaging direction of the digital still camera 1 placed on the camera platform 10 as much as possible. That is, the imaging recording block 75 is provided on the camera platform 10 so as to be as similar as possible to the image captured by the imaging recording block 61 on the digital still camera 1 side.

In this case, the composition determination block 73 and the pan / tilt control processing block 72 execute composition determination and drive control of the pan / tilt mechanism according to the composition determination result in the same manner as in FIG.
However, the composition determination block 73 in this case instructs the digital still camera 1 to execute imaging recording via the communication control processing block 71 in response to the timing at which the digital still camera 1 executes imaging recording. An instruction signal is transmitted. The digital still camera 1 executes imaging recording in response to the reception of this instruction signal, and executes imaging recording of captured image data that is assumed to be obtained by the imaging recording block 61 at that time.
In this way, in another modified example, regarding the composition determination and composition acquisition control, all the control and processing other than the imaging recording execution operation can be completed on the camera platform 10 side.

In the above description, pan control and tilt control are performed by controlling the movement of the pan / tilt mechanism of the camera platform 10, but instead of the camera platform 10, for example, the digital still camera 1 The optical system unit (21) is configured such that imaging light reflected by the reflecting mirror is incident, and an image obtained based on the imaging light is panned and tilted. It is also possible to adopt a configuration for moving the reflected light.
Further, panning / tilting is performed by controlling the pixel area for capturing an effective imaging signal as an image from the imaging element (image sensor 22) of the digital still camera 1 in the horizontal direction and the vertical direction. The same result can be obtained. In this case, it is not necessary to prepare a pan / tilt device unit other than the pan head 10 or the digital still camera 1 according to this, and the digital still camera 1 alone corresponds to composition acquisition control as the present embodiment. It is possible to complete the operation.
The angle of view control (zoom control) can also be realized by executing image processing for cutting out a partial image region from the captured image data instead of driving the zoom lens.
Even if the optical system unit of the digital still camera 1 has a mechanism that can change the optical axis of the lens in the horizontal and vertical directions and controls the movement of this mechanism, panning and tilting can be performed. Is possible.

In the description so far, the digital still camera 1 is an imaging system attached to the camera platform 10. However, the composition determination and the imaging recording configuration based on the composition determination result are as follows. Even without a pan head, the digital still camera 1 can be realized alone.
That is, even in a situation where the digital still camera 1 of the present embodiment is simply placed in a fixed state, composition determination is performed according to the image captured there, and automatic imaging recording is performed according to the determination result. Is executed. Even in such a way of using the digital still camera 1, it is sufficiently useful depending on the situation.

Subsequently, an example in which the basic configuration of composition determination of this embodiment is applied to other than the imaging system will be described.
First, in FIG. 23, a configuration for composition determination as an embodiment is applied to a single imaging apparatus such as a digital still camera. For example, when the image captured by the image capturing apparatus in the image capturing mode has an appropriate composition according to the determination result, the image capturing apparatus notifies the user of this fact by display.
For this purpose, the composition determination block 81, the notification control processing block 82, and the display unit 83 are shown here as the configuration that the imaging apparatus should have. The composition determination block 81 here has the same configuration as the composition determination block 62 shown in FIG.
For example, it is assumed that the user sets the imaging device in the imaging mode, holds the imaging device in his hand, and can perform imaging recording by performing a release operation (shutter button operation) at any time.
Under such a state, the composition determination block 81 first captures captured image data obtained by imaging at that time, executes a series of composition determination processes, and finally determines the optimum composition.
In addition, the composition determination block 81 in this case further obtains the coincidence and similarity between the composition of the captured image data actually obtained at that time and the determined optimum composition. For example, when the degree of similarity becomes equal to or higher than a certain level, it is determined that the image content of the captured image data actually obtained by photographing has an optimal composition. For example, in practice, if a degree of similarity equal to or greater than a predetermined level is obtained to such an extent that the composition of the captured image data is considered to match the optimum composition, the algorithm may be configured so that the composition is determined as the optimum composition. It is done. In addition, various algorithms can be considered as to how to determine the coincidence and similarity here, and it varies depending on the composition forming element to be adopted. Do not mention.

  Information on the determination result that the screen content of the captured image data has the optimum composition in this way is output to the notification control processing block 82. In response to the input of the above information, the notification control processing block 82 displays on the display unit 83 in a predetermined manner for notifying the user that the currently captured image has the optimum composition. Execute display control. The notification control processing block 82 is realized by a display control function by a microcomputer (CPU) provided in the imaging device, a display image processing function for realizing image display on the display unit 83, and the like. Note that the notification to the user of the optimal composition here may be configured to be performed by sounds such as electronic sounds or synthesized sounds.

  The display unit 83 corresponds to, for example, the display unit 33 of the digital still camera 1 of the present embodiment. For example, the display unit 83 is provided such that the display panel is exposed to a predetermined position in the imaging apparatus, and the shooting mode is set. It is common to display an image captured at that time, sometimes referred to as a so-called through image. Therefore, in the actual image pickup apparatus, the display unit 83 displays an image with a content for notifying that the optimum composition is superimposed on the through image. The user performs a release operation when a display notifying that the optimum composition is displayed. This makes it possible for a user who is not skilled in photography and skill to easily take a photo with an appropriate composition, that is, good image content.

Also, in the example shown in FIG. 24, the composition determination configuration of the embodiment is applied to a single imaging apparatus such as a digital still camera, as in FIG.
First, in the configuration shown in this figure, as in FIG. 23, the composition determination block 81 executes a process for determining the optimum composition based on the input captured image data, and the imaging obtained at the subsequent timing. It is determined from the image content of the image data what the optimal composition is, and then it is determined that the image content of the captured image data has the determined optimal composition. When it is determined that the optimum composition has been achieved, this is notified to the release control processing block 84.
The release control processing block 84 is a part that executes control for recording (capturing and recording) captured image data, and is realized by, for example, control executed by a microcomputer included in the imaging apparatus. Upon receiving the above notification, the release control processing block 84 executes image signal processing and recording control processing so that the captured image data obtained at that time is stored in, for example, a storage medium.
With such a configuration, when the digital still camera 1 is imaged by hand, the captured image can be automatically recorded at the timing when the image content having the optimum composition is obtained for the captured image. .

  23 and 24 can be applied to a digital still camera as long as it is in the category of a still camera, for example, a so-called silver salt camera that records a captured image on a silver salt film or the like. The present invention can be applied by providing an image sensor that takes in imaging light obtained by an optical system by spectroscopically and a digital image signal processing unit that inputs and processes a signal from the image sensor.

  FIG. 25 is also an example of a configuration in which the basic configuration for composition determination according to the embodiment is applied to an imaging apparatus such as a digital still camera. The imaging apparatus 100 shown in this figure includes a composition determination block 101, a metadata creation processing block 102, and a file creation processing block 103 as shown in the figure. Here, the composition determination block 101 corresponds to the composition determination block 202 of FIG.

  Here, captured image data obtained by imaging with an imaging recording block (not shown) is input to the composition determination block 101 and the file creation processing block 103 in the imaging apparatus 100. In this case, the captured image data input into the imaging apparatus 100 is captured image data that should be stored in a storage medium in accordance with a release operation, for example, and is not illustrated here. It is generated based on the imaging signal obtained by imaging in the imaging recording block.

First, in the composition determination block 101, a composition determination process is repeatedly executed regularly.
In addition, as a composition determination process in this case, an image portion (trimming) with a predetermined aspect ratio that is determined to obtain the determined optimum composition in the entire image area of the input captured image data based on the determination result. A process for identifying where the image portion is) is executed. Then, information indicating the specified trimmed image portion is output to the metadata creation processing block 102.
When such a process is executed, the composition determination block 101 keeps the determination result history information (determination result history information) and refers to the determination result history information. If the determined composition is exhausted, the trimming image portion is not specified based on the same composition determination result thereafter, in accordance with FIG. Alternatively, the composition determination algorithm is changed according to FIG. 4, and the composition determination process and the trimmed image portion are specified.

  The metadata creation processing block 102 creates metadata (edited metadata) composed of information necessary for obtaining an image having an optimal composition from the corresponding captured image data based on the input information, and creates a file. Output to block 103. This editing metadata is, for example, information indicating where the trimmed image portion on the screen as the corresponding captured image data is.

In the imaging apparatus 100 shown in this figure, captured image data is recorded on a storage medium so as to be managed as a still image file in a predetermined format. In response to this, the file creation processing block 103 converts (creates) the captured image data into a still image file format.
First, the file creation processing block 103 performs image compression encoding corresponding to the image file format on the input captured image data, and creates a file body portion composed of the captured image data. At the same time, the editing metadata input from the metadata creation processing block 102 is stored in a predetermined storage position, and data portions such as a header and an additional information block are created. Then, a still image file is created from the file body part, header, additional information block, and the like, and this is output. As a result, as shown in the figure, a still image file to be recorded on the storage medium is obtained having a structure including metadata (edited metadata) together with captured image data.

FIG. 26 shows a configuration example of an editing apparatus that edits a still image file created by the apparatus of FIG.
The editing apparatus 110 shown in the drawing takes in still image file data and first inputs it to the metadata separation processing block 111. The metadata separation processing block 111 separates captured image data and metadata corresponding to the file body portion from the data of the still image file. The metadata obtained by the separation is output to the metadata analysis processing block 112, and the captured image data is output to the trimming processing block 113.

The metadata analysis processing block 112 is a part that executes processing for analyzing the captured metadata. Then, when the editing metadata is analyzed as an analysis process, the trimmed image portion where the optimum composition is obtained is recognized. Then, trimming instruction information for instructing trimming of the recognized image portion is output to the trimming processing block 113.
The trimming processing block 113 executes image processing for extracting an image portion indicated by the trimming instruction information input from the metadata separation processing block 112 from the captured image data input from the metadata separation processing block 111 side. The image portion is output as edited captured image data which is one independent image data.

According to the system composed of the imaging device and the editing device shown in FIGS. 25 and 26, original still image data (captured image data) obtained by, for example, photographing can be stored as it is without being processed. Thus, editing can be performed to extract an image portion having an optimum composition from the original still image data by using metadata. Also, the determination of the extracted image portion corresponding to such an optimal composition is performed automatically, and the editing is very easy for the user.
The function as the editing apparatus shown in FIG. 26 may be adopted, for example, as an image data editing application installed in a personal computer or an image editing function in an application for managing image data. .

FIG. 27 is an example in which the composition determination configuration of the embodiment is applied to an imaging apparatus capable of capturing and recording moving images such as a video camera.
Moving image data is input to the imaging device 120 shown in this figure. This moving image data is generated based on an imaging signal obtained by imaging by an imaging unit assumed to be included in the same imaging device 120, for example. This moving image data is input to the composition determination block 122 and the file creation / recording processing block 124 in the imaging apparatus 120.
In this case, the composition determination block 122 corresponds to the composition determination block 200 shown in FIG. The composition determination block 122 regularly performs a series of subject detection, attribute detection, and composition determination processing on the input moving image data image. In addition, the quality determination is performed by comparing the difference (approximation) between the actual image content of the moving image data and the optimum composition obtained as a determination result.
As a result of the comparison, if a certain degree of approximation is obtained for the composition obtained in the actual captured image and the determined optimum composition, it is determined that the composition is good, and the above If the similarity is below a certain level, it is determined that the composition is not good.
When the composition determination block 122 determines that a good composition has been obtained for the moving image data as described above, the above-described good composition for the moving image data is compared with the metadata creation processing block 123. Information (good composition section instruction information) indicating where the image section (good composition section) determined to have been obtained is output. The good composition section instruction information) is, for example, information indicating the start position and the end position of the good composition section in the moving image data.

  In addition, the composition determination block 122 holds composition determination result history information, and does not create the above-described good composition section instruction information according to the exhausted composition determination result based on the determination result history information. To be. As a result, it is possible to avoid a result in which only images having a similar composition are designated as good composition sections.

In this case, the metadata creation processing block 123 generates various required metadata for the moving image data recorded as a file on the storage medium by the moving image recording processing block 124 described below. In addition, when good composition section instruction information is input from the composition determination block 122 as described above, metadata indicating that the image section indicated by the input good composition section instruction information is a good composition is generated. And output to the moving image recording processing block 124.
The moving image recording processing block 124 executes control for recording the input moving image data on a storage medium so as to be managed as a moving image file in a predetermined format. When metadata is output from the metadata creation processing block 123, control is performed so that the metadata is recorded so as to be included in the metadata accompanying the moving image file. To do.
As a result, as shown in the figure, the moving image file recorded on the storage medium includes the moving image data obtained by imaging accompanied by metadata indicating an image section in which a good composition is obtained. Will have.
It should be noted that the image section in which a good composition is obtained, which is indicated by the metadata as described above, may be an image section based on a moving image having a certain time width, or extracted from the moving image data. It may be based on a still image. Also, instead of the above metadata, moving image data or still image data of an image section in which a good composition is obtained is generated, and this is added to secondary image data (or moving image data) attached to the moving image file. It is also conceivable to record as a separate file from the image file.
As shown in FIG. 27, in the configuration including the composition determination block 122 for the imaging apparatus 120, only a moving image section determined to be a good composition section by the composition determination block 122 is used as a moving image file. It may be configured to record. Furthermore, a configuration in which image data corresponding to an image section determined to have a good composition by the composition determination block 122 is output to an external device via a data interface or the like can be considered.

In addition to the editing apparatus shown in FIG. 26, a printing apparatus 130 shown in FIG. 28 can be considered as an apparatus corresponding to the imaging apparatus 100 shown in FIG.
In this case, the printing apparatus 130 captures a still image file as an image to be printed. This still image file includes, for example, a file generated and recorded by the imaging apparatus 100, and has a structure having the substance of image data as a still image and metadata as shown in the figure. Therefore, this metadata includes composition editing metadata having the same meaning as that in the still image file shown in FIGS.

  The metadata separation processing block 131 inputs the file taken in this way. The metadata separation processing block 131 separates image data corresponding to the file main body portion and metadata associated therewith from the still image file data in the same manner as the metadata separation processing block 111 of FIG. The metadata obtained by the separation is output to the metadata analysis processing block 132, and the image data is output to the trimming processing block 133.

  The metadata analysis processing block 132 performs analysis processing similar to the metadata separation processing block 111 of FIG. 26 on the fetched metadata, and outputs trimming instruction information to the trimming processing block 133.

  The trimming processing block 133 is similar to the trimming processing block 113 in FIG. 26, and the image portion indicated by the trimming instruction information input from the metadata separation processing block 132 is selected from the image data input from the metadata separation processing block 131. Image processing for extraction is executed. Then, the print format image data generated from the extracted image portion is output to the print control processing block 134 as print image data.

The print control processing block 134 executes control for operating a printing mechanism (not shown) using the input print image data.
By such an operation, depending on the printing apparatus 130, an image portion that is assumed to have an optimum composition is automatically extracted from the entire image of the input image data and printed as a single image. It will be.

  Note that devices, systems, application software, and the like to which the composition determination configuration according to the present invention can be applied are considered in addition to the imaging systems and imaging devices described above.

Further, in the embodiments so far, the body part frame which is the face frame is a square, but for example, the body part frame is a shape other than a square and has a size in a direction appropriate for this shape. An enlargement may be performed.
In addition, there are various ways to obtain a composition by composition control (composition determination processing, composition adjustment control) after correcting the size and shape of the face frame. Should not be done.

As described above, at least a part of the configuration based on the present application can be realized by causing a CPU or a DSP to execute a program.
Such a program is written and stored in a ROM or the like at the time of manufacture, for example, and is stored in a removable storage medium and then installed (including update) from this storage medium. It is conceivable to store the data in a corresponding non-volatile storage area, the flash memory 30, or the like. It is also conceivable that the program can be installed through a data interface such as USB or IEEE 1394 under the control of another host device. Further, it may be possible to store the data in a storage device such as a server on the network, and to give the digital still camera 1 a network function so that the digital still camera 1 can be downloaded and acquired from the server.

It is a figure which shows the digital still camera and pan head which comprise the imaging system as embodiment. It is a figure which shows typically the example of a motion along the pan direction and the tilt direction of the digital still camera attached to the camera platform for the imaging system of the embodiment. It is a block diagram which shows the internal structural example of the digital still camera which comprises the imaging system of embodiment. It is a block diagram which shows the internal structural example of the pan head which comprises the imaging system of embodiment. It is a block diagram which shows the internal system structural example about the imaging system in embodiment. It is a figure for demonstrating one specific example along the basics as a composition determination process. It is a figure for demonstrating one specific example along the basics as a composition determination process. It is a figure for demonstrating the composition determination process (1st example) when the subject of an adult and a child is mixed as embodiment. It is a figure for demonstrating the composition determination process (1st example) when the subject of an adult and a child is mixed as embodiment. It is a figure for demonstrating the composition determination process (2nd example) when the subject of an adult and a child is mixed as embodiment. It is a figure for demonstrating the composition determination process (2nd example) when the subject of an adult and a child is mixed as embodiment. It is a figure for demonstrating the composition determination process regarding a to-be-photographed object size (zoom magnification). It is a figure for demonstrating the composition determination process example (normal case) regarding subject size (zoom magnification). It is a figure for demonstrating the composition determination process example (in the case of a couple) regarding subject size (zoom magnification). It is a figure for demonstrating the example of a composition determination process according to a face direction. It is a figure for demonstrating the example of a composition determination process according to a face direction. It is a figure for demonstrating face direction vector amount. It is a flowchart which shows the example of an algorithm for composition control as embodiment. It is a figure which shows the example of a procedure as a composition determination process in the composition control of embodiment. It is a flowchart which shows the example of a process sequence for the zoom magnification setting in the composition determination process of embodiment. It is a block diagram which shows the other internal system structural example about the imaging system of embodiment. It is a block diagram which shows the other internal system structural example about the imaging system of embodiment. It is a block diagram which shows the example of application of a composition determination block in embodiment other than an imaging system. It is a block diagram which shows the example of application of a composition determination block in embodiment other than an imaging system. It is a block diagram which shows the example of application of a composition determination block in embodiment other than an imaging system. FIG. 26 is a block diagram illustrating a configuration example of an editing device corresponding to the imaging device in FIG. 25. It is a block diagram which shows the example of application of a composition determination block in embodiment other than an imaging system. FIG. 26 is a block diagram illustrating a configuration example of an editing device corresponding to the imaging device in FIG. 25.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Digital still camera, 2 Shutter button, 3 Lens part, 10 Head, 21 Optical system, 22 Image sensor, 23 A / D converter, 24 Signal processing part, 25 Encoding / decoding part, 26 Media controller, 27 Control part, 28 ROM, 29 RAM, 30 Flash memory, 31 Operation section, 32 Display driver, 33 Display section, 34 Pan head compatible communication section, 40 Memory card, 51 Control section, 52 Communication section, 53 Bread mechanism section, 54 Bread motor , 55 Pan drive unit, 56 Tilt mechanism unit, 57 Tilt motor, 58 Tilt drive unit, 61 Image recording block, 62 Composition determination block, 63 Pan / tilt / zoom control block, 64 Communication control processing block

Claims (7)

Subject detection means for inputting image data and detecting a subject present in the image content of the image data;
Attribute detection means for detecting a predetermined attribute for each subject detected by the subject detection means;
Composition determining means for determining a composition based on a predetermined relationship with respect to attributes for each subject detected by the attribute detecting means;
A composition determination apparatus comprising: The attribute detection means detects a face direction for each subject as an attribute,
The composition determination means determines the position of the subject in the image frame based on the face direction of the entire subject obtained based on the face direction detected for each subject;
The composition determination apparatus according to claim 1. The attribute detection means detects the age of the subject as an attribute,
The composition determination means is configured to display an image frame when an adult subject and a child subject are mixed based on the age detected for each subject than when an adult subject and a child subject are not mixed. Determine the position of the subject so that the child's subject is placed closer to the center
The composition determination apparatus according to claim 1 or 2. The attribute detecting means detects the sex of the subject as an attribute,
In the case where the number of detected subjects is 2 and the sexes of the two detected subjects are male and female, respectively, based on the sex detection result by the attribute detecting unit. Set the size of the subject for the male / female group that is larger than the size of the subject in the image frame set in the normal case.
The composition determination apparatus according to claim 1. When the composition determining means sets the enlargement ratio of the image according to the size of the subject for the male / female group, the composition determination means uses the position of the subject determined along the direction indicated by the face orientation of the subject overall. Setting the enlargement ratio corresponding to the size of the subject so that the face portions of the two subjects fit within the image frame;
The composition determination apparatus according to claim 2. A subject detection procedure for inputting image data and detecting a subject present in the image content of the image data;
An attribute detection procedure for detecting a predetermined attribute for each subject detected by the subject detection procedure;
A composition determination procedure for determining a composition based on a predetermined relationship with respect to attributes for each subject detected by the attribute detection procedure;
A composition determination method for executing. A subject detection procedure for inputting image data and detecting a subject present in the image content of the image data;
An attribute detection procedure for detecting a predetermined attribute for each subject detected by the subject detection procedure;
A composition determination procedure for determining a composition based on a predetermined relationship with respect to attributes for each subject detected by the attribute detection procedure;
Is a program that causes a composition determination apparatus to execute.