JP5440588B2 - Composition determination apparatus, composition determination method, and program - Google Patents

Description

  The present invention relates to a composition determination apparatus and method for determining, for example, image data as a still image and determining the composition of the image content. The present invention also relates to a program executed by such a device.

For example, composition setting is one of the technical elements for taking a photo that can give a good impression. The composition here is also referred to as framing, and means, for example, the arrangement of the subject in the screen as a photograph.
Although there are several general and basic methods for achieving a good composition, it is never possible for a general camera user to take a photo with a good composition unless they have sufficient knowledge and skills regarding photography. It ’s not easy. From this, for example, a technical configuration capable of easily and easily obtaining a photographic image having a good composition is required.

For example, in Patent Document 1, as an automatic tracking device, a difference between images at a predetermined time interval is detected, a center of gravity of the difference between images is calculated, and a moving amount and a moving direction of the center of gravity are used for an imaging screen of a subject image. A technical configuration is disclosed in which a moving amount and a moving direction are detected to control an imaging apparatus, and a subject image is set within a reference area of an imaging screen.
In Patent Document 2, as an automatic tracking device, when a person is automatically tracked, 20% from the upper side of the person relative to the entire area of the person image on the screen so that the face of the person is at the center of the screen. A technical configuration is disclosed in which a person's face can be tracked while being reliably tracked by tracking the position of the area as the center of the screen.
If these technical configurations are viewed from the viewpoint of composition determination, it is possible to automatically search for a subject as a person and place the subject in a certain composition on the shooting screen.

JP 59-208983 A JP 2001-268425 A

For example, it is considered that the optimum composition may differ depending on a predetermined situation and state regarding the subject. However, with the technique according to the above-mentioned patent document, it is only possible to arrange the tracked subject with a certain fixed composition. Therefore, it is impossible to change the composition in accordance with the subject situation or the like.
Therefore, in the present invention, for example, the aim is to propose a technique for easily obtaining a good composition for an image such as a photograph. The purpose is to make it possible to determine a more sophisticated and flexible composition.

In view of the above-described problems, the present invention is configured as follows as a composition determination apparatus.
That is, subject detection means for detecting the presence of a specific subject in an image based on image data, and composition determination means for determining the composition according to the number of detected subjects that are the subjects detected by the subject detection means. And the composition determination means determines whether the rectangular image is portrait or landscape based on the distance between the subjects located at the left and right ends when the number of detected subjects is equal to or greater than a predetermined value. It was decided.

  In the above-described configuration, the optimum composition is determined according to the number of subjects detected in the image based on the image data. For example, the optimum composition varies depending on the number of subjects existing in one screen. However, according to the present invention, an optimum composition can be obtained by adapting to changes in the situation of the number of subjects. It will be.

  In this way, according to the present invention, it is possible to obtain an optimal composition corresponding to the number of subjects with respect to the content of the image included in the image data. In other words, a more advanced and flexible composition determination is automatically performed as compared to the case where the subject is simply arranged with a certain fixed composition. As a result, a user who uses the apparatus to which the present invention is applied can obtain an image with the optimum composition without troublesome work. For example, it is possible to provide higher convenience than before. Become.

It is a figure which shows the example of an external appearance structure of the imaging system (digital still camera, pan head) as embodiment of this invention. It is a figure which shows typically the example of a motion along the pan direction and the tilt direction of the digital still camera attached to the camera platform as an operation of the imaging system of the embodiment. It is a figure which shows the structural example of the digital still camera of embodiment. It is a figure which shows the structural example of the pan head of embodiment. It is a figure which shows the function with which the digital still camera of embodiment is provided with corresponding to composition control by the structure of a block unit. It is a figure explaining the center of gravity of an individual subject and the total subject center of gravity for a plurality of individual subjects. It is a figure explaining the origin coordinate set to the screen of captured image data. It is a figure which shows typically the example of composition control in the case of the one individual subject detected in 1st composition control. It is a figure which shows typically the composition control example in the case of two detected individual subjects in 1st composition control. It is a figure which shows typically the example of composition control in case the detected individual subject is 3 or more in 1st composition control. It is a flowchart which shows the example of a process sequence for 1st composition control. It is a figure which shows typically the example of composition control in the case of the one individual subject detected in 2nd composition control. FIG. 10 is a diagram schematically illustrating an example of composition control in the second composition control when two detected individual subjects are detected and the distance between the individual subjects is detected (supplemented) in a state of a certain value or less. . It is a flowchart which shows the process sequence example for 2nd composition control. It is a figure for demonstrating the object discrimination in embodiment. It is a flowchart which shows the example of a process sequence for implement | achieving the object discrimination in embodiment. It is a figure which shows the structural example as a modification of the imaging system of embodiment. It is a figure which shows the structural example as another modification of the imaging system of embodiment. It is a figure which shows the example of application of the composition determination based on this invention. It is a figure which shows the example of application of the composition determination based on this invention. It is a figure which shows the example of application of the composition determination based on this invention. It is a figure which shows the example of application of the composition determination based on this invention. It is a figure which shows the example of application of the composition determination based on this invention. It is a figure which shows the example of application of the composition determination based on this invention. It is a figure which shows the example of application of the composition determination based on this invention. It is a figure which shows the example of application of the composition determination based on this invention.

  Hereinafter, the best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described. As an example of the present embodiment, the configuration based on the present invention is applied to an imaging system including a digital still camera and a camera platform to which the digital still camera is attached.

FIG. 1 is a front view showing an example of the external configuration of an imaging system according to the present embodiment.
As shown in this figure, the imaging system of the present embodiment includes a digital still camera 1 and a pan head 10.
The digital still camera 1 generates still image data based on imaging light obtained by imaging with the lens unit 3 provided on the front panel of the main body, and stores the still image data in a storage medium loaded therein. It is possible. That is, it has a function of storing and saving an image taken as a photograph in a storage medium as still image data. When such a photograph is taken manually, the user presses a shutter (release) button provided on the upper surface of the main body.

  The digital still camera 1 can be fixed to the camera platform 10 so as to be fixed. That is, the pan head 10 and the digital still camera 1 are provided with a mechanism part for enabling mutual attachment.

The pan / tilt head 10 includes a pan / tilt mechanism for moving the attached digital still camera 1 in both the pan direction (horizontal direction) and the tilt direction.
For example, FIGS. 2A and 2B show how the digital still camera 1 moves in the pan and tilt directions provided by the pan / tilt mechanism of the camera platform 10. FIGS. 2A and 2B show the digital still camera 1 attached to the pan head 10 as viewed from the plane direction and the side direction, respectively.
First, regarding the pan direction, for example, the rotation direction + α is set with the rotation axis Ct1 as the rotation center with reference to the position state in which the horizontal direction of the main body of the digital still camera 1 and the straight line X1 shown in FIG. A panning movement in the right direction is given by the rotation along. Further, by performing rotation along the rotation direction −α, a panning movement in the left direction is given.
With respect to the tilt direction, for example, the rotation in the rotation direction + β is performed with the rotation axis Ct2 as the rotation center with reference to one state where the vertical direction of the main body of the digital still camera 1 coincides with the straight line Y1 in the vertical direction. , Given downward panning movement. Further, the panning movement in the upward direction is given by the rotation in the rotation direction −β.
Although the maximum movable rotation angle in each of the ± α direction and the ± β direction shown in FIGS. 2A and 2B is not mentioned, it should be considered that the number of opportunities for capturing the subject should be increased as much as possible. If so, it is preferable to make the maximum movable rotation angle as large as possible.

The block diagram of FIG. 3 shows an internal configuration example of the digital still camera 1 of the present embodiment.
In this figure, first, the optical system unit 21 includes a predetermined number of imaging lens groups including a zoom lens, a focus lens, and the like, a diaphragm, and the like, and receives light of the image sensor 22 using incident light as imaging light. Form an image on the surface.
The optical system unit 21 is also provided with a drive mechanism unit for driving the zoom lens, the focus lens, the diaphragm, and the like. The operation of these drive mechanisms is controlled by so-called camera control such as zoom (viewing angle) control, automatic focus adjustment control, automatic exposure control, and the like executed by the control unit 27, for example.

  The image sensor 22 performs so-called photoelectric conversion in which imaging light obtained by the optical system unit 21 is converted into an electric signal. For this purpose, the image sensor 22 receives the imaging light from the optical system unit 21 on the light receiving surface of the photoelectric conversion element, and sequentially accumulates signal charges accumulated according to the intensity of the received light at a predetermined timing. It is made to output. Thereby, an electrical signal (imaging signal) corresponding to the imaging light is output. The photoelectric conversion element (imaging element) employed as the image sensor 22 is not particularly limited, but in the present situation, for example, a CMOS sensor, a CCD (Charge Coupled Device), or the like can be used. When a CMOS sensor is used, a device (component) corresponding to the image sensor 22 may include an analog-digital converter corresponding to an A / D converter 23 described below.

The imaging signal output from the image sensor 22 is input to the A / D converter 23, thereby being converted into a digital signal and input to the signal processing unit 24.
In the signal processing unit 24, for example, the digital imaging signal output from the A / D converter 23 is captured in a unit corresponding to one still image (frame image), and the still image unit captured in this way is captured. By performing necessary signal processing on the signal, it is possible to generate captured image data (captured still image data) that is image signal data corresponding to one still image.

When the captured image data generated by the signal processing unit 24 as described above is recorded as image information in a memory card 40 that is a storage medium (storage medium device), for example, a captured image corresponding to one still image. Data is output from the signal processing unit 24 to the encoding / decoding unit 25.
The encoding / decoding unit 25 performs compression encoding on the captured image data in units of still images output from the signal processing unit 24 by a predetermined still image compression encoding method, and performs control of the control unit 27, for example. Accordingly, a header or the like is added and converted into a format of captured image data compressed into a predetermined format. Then, the captured image data generated in this way is transferred to the media controller 26. The media controller 26 writes and records the transferred captured image data in the memory card 40 under the control of the control unit 27. The memory card 40 in this case is a storage medium having a configuration including, for example, a card-type outer shape conforming to a predetermined standard and having a nonvolatile semiconductor storage element such as a flash memory inside. Note that the storage medium for storing the image data may be of a type or format other than the memory card.

  In addition, the signal processing unit 24 according to the present embodiment can perform image processing as subject detection using captured image data acquired as described above. The subject detection processing in this embodiment will be described later.

  Further, the digital still camera 1 can display a so-called through image that is an image currently being captured by causing the display unit 33 to perform image display using captured image data obtained by the signal processing unit 24. It is said. For example, the signal processing unit 24 captures the imaging signal output from the A / D converter 23 as described above and generates captured image data corresponding to one still image, but this operation is continued. Thus, captured image data corresponding to frame images in the moving image is sequentially generated. The captured image data sequentially generated in this way is transferred to the display driver 32 under the control of the control unit 27. Thereby, a through image is displayed.

  The display driver 32 generates a drive signal for driving the display unit 33 based on the captured image data input from the signal processing unit 24 as described above, and outputs the drive signal to the display unit 33. To be. As a result, the display unit 33 sequentially displays images based on the captured image data in units of still images. If this is seen by the user, the image taken at that time is displayed on the display unit 33 as a moving image. That is, a monitor image is displayed. The display screen unit 5 described above with reference to FIG. 1 corresponds to the screen portion of the display unit 33 here.

The digital still camera 1 can also reproduce captured image data recorded on the memory card 40 and display the image on the display unit 33.
For this purpose, the control unit 27 designates captured image data and instructs the media controller 26 to read data from the memory card 40. In response to this command, the media controller 26 accesses the address on the memory card 40 where the designated captured image data is recorded, executes data reading, and sends the read data to the encoding / decoding unit 25. Forward.

  The encode / decode unit 25 extracts, for example, actual data as compressed still image data from the captured image data transferred from the media controller 26 under the control of the control unit 27, and decodes the compressed still image data with respect to compression encoding. Processing is executed to obtain captured image data corresponding to one still image. Then, the captured image data is transferred to the display driver 32. Thereby, on the display unit 33, the image of the captured image data recorded in the memory card 40 is reproduced and displayed.

  In addition to the monitor image and the reproduced image of the captured image data, a user interface image can be displayed on the display unit 33. In this case, for example, the control unit 27 generates display image data as a necessary user interface image according to the operation state at that time, and outputs the display image data to the display driver 32. As a result, the user interface image is displayed on the display unit 33. The user interface image can be displayed on the display screen of the display unit 33 separately from the monitor image and the reproduced image of the captured image data, such as a specific menu screen. It is also possible to display the image data so as to be superimposed and synthesized on a part of the reproduced image.

The control unit 27 actually includes a CPU (Central Processing Unit), for example, and constitutes a microcomputer together with the ROM 28, the RAM 29, and the like. In the ROM 28, for example, various setting information related to the operation of the digital still camera 1 is stored in addition to a program to be executed by the CPU as the control unit 27. The RAM 29 is a main storage device for the CPU.
Further, the flash memory 30 in this case is provided as a nonvolatile storage area used for storing various setting information that needs to be changed (rewritten) in accordance with, for example, a user operation or an operation history. It is For example, if a non-volatile memory such as a flash memory is adopted as the ROM 28, a partial storage area in the ROM 28 may be used instead of the flash memory 30.

  The operation unit 31 collects various operation elements provided in the digital still camera 1 and operation information signal output parts that generate operation information signals according to operations performed on these operation elements and output them to the CPU. As shown. The control unit 27 executes predetermined processing according to the operation information signal input from the operation unit 31. Thereby, the operation of the digital still camera 1 according to the user operation is executed.

  The pan head communication unit 34 is a part that performs communication according to a predetermined communication method between the pan head 10 side and the digital still camera 1 side. For example, the digital still camera 1 communicates with the pan head 10. In the attached state, a physical layer configuration for enabling transmission / reception of wired or wireless communication signals to / from the communication unit on the camera platform 10 side, and communication processing corresponding to a predetermined layer higher than this And a configuration for realizing.

FIG. 4 is a block diagram illustrating a configuration example of the camera platform 10.
As described above, the pan head 10 includes a pan / tilt mechanism, and includes a pan mechanism unit 53, a pan motor 55, a tilt mechanism unit 56, and a tilt motor 57 as corresponding parts. .
The pan mechanism unit 53 is configured to have a mechanism for giving the pan (lateral) movement shown in FIG. 2A with respect to the digital still camera 1 attached to the pan head 10. Is obtained by rotating the pan motor 54 in the forward and reverse directions. Similarly, the tilt mechanism unit 53 is configured to have a mechanism for imparting a motion in the tilt (vertical) direction shown in FIG. This mechanism movement is obtained by rotating the tilt motor 57 in the forward and reverse directions.

The control unit 51 includes a microcomputer formed by combining a CPU, a ROM, a RAM, and the like, for example, and controls the movement of the pan mechanism unit 53 and the tilt mechanism unit 56. For example, when the control unit 51 controls the movement of the pan mechanism unit 53, a control signal corresponding to the movement amount and movement direction necessary for the pan mechanism unit 53 is output to the pan driving unit 55. The pan drive unit 55 generates a motor drive signal corresponding to the input control signal and outputs the motor drive signal to the pan motor 55. In response to this motor drive signal, the pan motor 55 rotates, for example, in a required rotation direction and rotation angle, and as a result, the pan mechanism unit 53 is also driven to move according to the corresponding movement amount and movement direction.
Similarly, when controlling the movement of the tilt mechanism unit 56, the control unit 51 outputs a control signal corresponding to the movement amount and movement direction necessary for the tilt mechanism unit 56 to the tilt drive unit 58. The tilt drive unit 58 generates a motor drive signal corresponding to the input control signal and outputs it to the tilt motor 57. By this motor drive signal, the tilt motor 57 rotates, for example, in a required rotation direction and rotation angle, and as a result, the tilt mechanism unit 56 is also driven to move according to the corresponding movement amount and movement direction.

  The communication unit 52 is a part that performs communication according to a predetermined communication method with the pan-head compatible communication unit 34 in the digital still camera 1 attached to the pan head 10. Similarly, it has a physical layer configuration for enabling transmission / reception of wired or wireless communication signals with a counterpart communication unit, and a configuration for realizing communication processing corresponding to a predetermined layer higher than this. It consists of

In the imaging system including the digital still camera 1 and the camera platform 10 having the above-described configuration, for example, a person is treated as a main subject (hereinafter simply referred to as a subject), and a search for detecting the subject is performed. If the presence of the subject is detected, the composition (optimal composition) that is optimal as an image in which the subject is captured is obtained (so that framing is performed) and the pan head 10 Drives the pan / tilt mechanism. Then, at the timing when the optimum composition is obtained, the captured image data at that time is recorded in the storage medium (memory card 40).
That is, in the imaging system of the present embodiment, when taking a picture with a digital still camera, an operation of determining (determining) an optimum composition for the searched subject and performing shooting and recording is automatically executed. Accordingly, it is possible to obtain a correspondingly high quality photographic image without the user himself / herself judging the composition and taking a picture. Also, in such a system, it is not necessary for someone to take a picture with a camera, so that everyone in the place where the picture is taken can be a subject. Further, even if the user who is the subject is not particularly conscious of entering the viewing angle range of the camera, a photograph containing the subject can be obtained. In other words, the opportunity to shoot the natural appearance of the person at the shooting location increases, and it is possible to obtain many photographs with an atmosphere that did not exist so far.
In addition, it can be considered that the optimal composition differs depending on the number of subjects, but the composition determination in the present embodiment differs based on the number of detected subjects. An optimum composition can be determined. As a result, compared with the case where composition determination is performed without considering the number of subjects, for example, it is possible to obtain a better quality image in the present embodiment as a whole.

Hereinafter, the composition control in the present embodiment will be described.
FIG. 5 shows a configuration example of functional parts corresponding to the composition control of the present embodiment, which is provided on the digital still camera 1 side.
In this figure, a subject detection processing block 61 executes subject detection processing including subject search control using captured image data obtained by the signal processing unit 24 based on an imaging signal obtained by the image sensor 22. It is considered as a part to be The subject detection processing here refers to processing for discriminating and detecting a subject as a person from the image content of the captured image data screen. Information (detection information) obtained as a detection result here is: The number of subjects as a person, position information on the screen for each individual subject (individual subject), the size (occupied area) in the image for each individual subject, and the like. Note that, depending on the configuration of the composition determination algorithm or the like, if only the number of subjects is obtained as detection information, the composition control as the present embodiment can be realized.
Further, as a specific technique for subject detection processing here, a face detection technique can be used. Also, some face detection methods and methods are known, but in this embodiment, which method should be adopted is not particularly limited, considering detection accuracy and design difficulty. It is only necessary to adopt an appropriate method.
The subject detection processing executed by the subject detection processing block 61 can be realized as image signal processing in the signal processing unit 24. When the signal processing unit 24 is configured by a DSP as described above, this subject detection processing is realized by a program and instructions given to the DSP as the signal processing unit 24.
At the time of subject search control, a control signal for driving the pan / tilt mechanism is output via the communication control processing block 63 in order to control the pan / tilt mechanism of the camera platform 10.

Detection information that is a result of subject detection processing in the subject detection processing block 61 is input to the composition control processing block 62.
The composition control processing block 62 uses the input detection information about the subject to determine the composition (optimum composition) that is regarded as optimal. Then, control (composition control) for obtaining the determined optimal composition is executed. As composition control in this case, change control of the angle of view (in this embodiment, for example, a viewing angle that can be changed according to control of the zoom lens) and control of the imaging direction along the pan (left and right) direction (Pan control) and control of the imaging direction along the tilt (up and down) direction (tilt control). In order to change the angle of view, at least one of zoom control for moving the zoom lens in the optical system unit 21 of the digital still camera 1 and image signal processing such as image cutout for captured image data is performed. Further, pan control and tilt control are performed by controlling and moving the pan / tilt mechanism of the camera platform 10. When controlling the pan / tilt mechanism, the composition control processing block 62 transmits a control signal for setting the pan / tilt mechanism to an appropriate position state via the communication control processing block 63 to the pan head 10 side. Let
The composition determination and composition control processing executed by the composition control processing block 62 can be configured to be executed by the control unit 27 (CPU) based on a program, for example. Alternatively, a configuration in which the signal processing unit 24 performs processing executed based on a program may be considered. The communication control processing block 63 is a part configured to execute communication processing with the communication unit 52 on the camera platform 10 side according to a predetermined protocol, and includes a functional part corresponding to the camera support communication unit 34. Become.

Next, referring to FIG. 6, an example of subject detection processing that is executed by the subject detection processing block 61 will be described.
Here, it is assumed that the subject detection processing block 61 has captured captured image data having the image content shown in FIG. The image content of the captured image data is obtained by photographing an image in which one subject as a person exists. FIG. 6 (a) (and FIG. 6 (b)) shows a state in which one screen is divided into a matrix. This is because the screen as captured image data has a predetermined number of horizontal and vertical. It schematically shows that the pixel consists of a set of pixels.
By performing subject detection (face detection) on the captured image data of the image content shown in FIG. 6A, the face of one individual subject SBJ shown in the figure is detected. That is, one individual subject is detected here by detecting one face by the face detection process. As a result of detecting the individual subject in this manner, information on the number, position, and size of the individual subject is obtained as described above.
First, regarding the number of individual subjects, for example, the number of faces detected by face detection may be obtained. In the case of FIG. 6A, since one face is detected, the result that the number of individual subjects is 1 is also obtained.
Further, regarding the position information for each individual subject, at least the centroid G (X, Y) of the individual subject SBJ in the image as the captured image data is obtained. In this case, the X, Y origin coordinates P (0,0) on the screen of the captured image data serving as a reference for the center of gravity G (X, Y) in this case correspond to the screen size as shown in FIG. 7, for example. It is assumed that it is the intersection of the intermediate point of the width (horizontal image size) Cx in the X-axis direction (horizontal direction) and the intermediate point of the width (vertical image size) Cy in the Y-axis direction (vertical direction).
In addition, as to the definition of the position of the individual subject in the image with respect to the center of gravity G and how to set the center of gravity G, for example, a known subject center of gravity detection method may be adopted. it can.
As for the size of each individual subject, for example, the number of pixels in a region that is identified and detected as a face portion by face detection processing may be obtained.

If it is assumed that the captured image data shown in FIG. 6B is captured and the subject detection processing block 61 executes the subject detection processing, first, the presence of two faces is specified by face detection. Therefore, the result that the number of individual subjects is 2 is obtained. Here, of the two individual subjects, the left subject is the individual subject SBJ0, and the right subject is the individual subject SBJ1, which has discriminability. The coordinates of the center of gravity obtained for each of the individual subjects SBJ0 and SBJ1 are indicated as G0 (X0, Y0) and G1 (X1, Y1), respectively.
When two or more individual subjects are detected in this way, the total subject center of gravity Gt (Xg, Yg), which is the center of gravity when these individual subjects are viewed as a group of subjects (total subjects). ).
There are several ways to set the total subject gravity center Gt, but here, as the simplest example, among the detected individual subjects, the left and right edges of the screen In this example, an intermediate point on a line segment connecting the centroids of the individual subjects located at both ends is set as the total subject centroid Gt. The total subject gravity center Gt is information that can be used in composition control, for example, as will be described later, and is information obtained by calculation if information on the center of gravity of an individual subject is acquired. Accordingly, the total subject gravity center Gt may be obtained by the subject detection processing block 61 and output as detection information, but the composition control processing block 62 indicates the position of the center of gravity of the individual subject acquired as detection information. Of the information, the information on the center of gravity of the individual subject located at both the left and right ends may be used.
In addition, for example, a weighting coefficient is given according to the sizes of a plurality of individual subjects, and the setting is made with consideration given so that the position of the total subject gravity center Gt is close to, for example, a large individual subject. A method can also be considered.
As for the size of the individual subject, for example, for each of the individual subjects SBJ0 and SBJ1, the number of pixels that the detected face occupies may be obtained.

Subsequently, a composition obtained by composition control as a first example in the present embodiment will be described with reference to FIGS.
FIG. 8A shows a case where image content obtained by capturing one individual subject SBJ0 is obtained as captured image data obtained as a result of subject detection before composition control. In the present embodiment, when the camera platform 10 to which the digital still camera 1 is attached is normally installed, the orientation of the digital still camera 1 is set so that a horizontally long image is captured. . Therefore, in the first example, it is assumed that a horizontally long image is obtained by imaging.

When one individual subject is detected as shown in FIG. 8A, for example, as shown as a transition from FIG. 8A to FIG. 8B, the individual subject SBJ is detected. Then, zoom control is performed to narrow the angle of view so that the occupation ratio of the captured image data in the screen becomes a predetermined value, and the size of the individual subject is changed. 8A and 8B show a case where the change is made in the direction of narrowing the angle of view in order to enlarge the size of the individual subject SBJ0, but at the stage where the individual subject is detected. When the occupancy ratio of the individual subject in the screen exceeds the predetermined value, zoom control is performed to increase the angle of view by reducing the occupancy ratio to the predetermined value. become.
Further, regarding the position in the screen when there is one individual subject, in the case of the present embodiment, the position is set so as to be substantially in the center in the left-right direction. For this purpose, for example, the position of the center of gravity G of the individual subject SJB0 in the left-right direction may be set substantially at the center.

Next, as shown in FIG. 9A, when two individual subjects are detected, as a composition control, first, a distance (distance between subjects) K between these two individual subjects SBJ0 and SBJ1 is set. Ask. This distance K can be expressed, for example, by the difference (X1−X0) between the X coordinate (X0) of the centroid G0 of the individual subject SBJ0 and the X coordinate (X1) of the centroid G1 of the individual subject SBJ1. Then, the angle of view is adjusted so that the inter-subject distance K obtained as described above is 1/3 of the horizontal image size Cx (K = Cx / 3) as shown in FIG. 9B. Do. Also in this case, the area where the two individual subjects SBJ0 and SBJ1 are shown is arranged so as to be substantially at the center of the screen in the left-right direction. For this purpose, for example, these individual subjects are arranged. The total subject gravity center Gt for SBJ0 and SBJ1 is arranged at the center position in the left-right direction.
The inter-subject distance K is set to 1/3 of the horizontal image size Cx based on a composition setting method called a three-division method. The three-division method is one of the most basic composition setting methods, and a good composition is obtained by positioning the subject on a virtual line that divides the rectangular screen into three equal parts along the vertical and horizontal directions. It is about to try. When the inter-subject distance K = Cx / 3 and the total subject gravity center Gt is positioned at the center in the left-right direction as described above, the center G0 of the individual subject SBJ0 is the left side in the vertical direction of the screen. The center of gravity G1 of the individual subject SBJ1 is substantially located on the right virtual line along the vertical direction of the screen. That is, a composition according to the three-division method is obtained.

In addition, as shown in FIG. 10A, composition control when three individual subjects are detected includes an individual subject SBJ0 located on the leftmost side in the screen and an individual subject SBJ2 located on the rightmost side. A distance K between the subjects is obtained. In other words, the distance K between the subjects is obtained by assigning numbers 0 to n−1 to the individual subjects from the left to the right of the screen when n individual subjects are detected. The X coordinate of the center of gravity G0 of the individual subject SBJ0 located on the leftmost side of the screen is (X0), and the X coordinate of the center of gravity Gn-1 of the individual subject SBJ (n-1) located on the rightmost side of the screen is (Xn-1). ) As a general formula represented by (Xn-1)-(X0).
In this case, the angle of view is controlled so that the distance K between the subjects is ½ of the horizontal image size Cx as shown in FIG. 9B. As for the subject position in the left and right directions, the general subject Gt is positioned substantially at the center in the horizontal direction of the screen so that the area portion where the three individual subjects are present is approximately at the center in the horizontal direction of the screen. In the present embodiment, the composition control shown in FIG. 10 is performed when three or more individual subjects are detected.
When there are three or more individual subjects in the screen, for example, the subject-to-subject distance K with respect to the horizontal image size Cx is more than the subject-to-subject distance K is set to 1/3 of the horizontal image size Cx according to the three-division method In general, a better composition can be obtained by increasing the ratio of. Therefore, in the present embodiment, as described above, when three or more individual subjects are detected, a composition in which the inter-subject distance K = Cx / 2 is formed.
In this way, in the present embodiment, composition control is performed by adjusting the different angle of view for each of the detected individual subjects 1, 2, and 3 or more.

  FIG. 11 is executed by the subject detection processing block 61, the composition control processing block 62, and the communication control processing block 63 shown in FIG. 5 in response to the composition control as the first example described with reference to FIGS. An example procedure to be performed is shown. Further, the processing shown in this figure can be considered to be realized by the CPU in the signal processing unit 24 and the control unit 27 as a DSP executing a program. Such a program is written and stored in a ROM or the like at the time of manufacture, for example, and is stored in a removable storage medium and then installed (including update) from this storage medium. It is conceivable to store the data in a corresponding non-volatile storage area, the flash memory 30, or the like. It is also conceivable that the program can be installed through a data interface such as USB or IEEE 1394 under the control of another host device. Further, it may be possible to store the data in a storage device such as a server on the network, and to give the digital still camera 1 a network function so that the digital still camera 1 can be downloaded and acquired from the server.

First, steps S101 to S106 are procedures for searching for and detecting a subject, and the subject detection processing block 61 is mainly executed.
In step S101, captured image data based on the imaging signal from the image sensor 22 is acquired and acquired. In step S102, subject detection processing is executed using the captured image data acquired in step S101. As the subject detection processing here, for example, first, detection of whether or not an individual subject exists in the screen content as captured image data is performed by a method such as face detection described above. Is present, at least the number of individual subjects, the position (center of gravity) for each individual subject, and the size are obtained as detection information.

In step S103, it is determined whether or not the presence of an individual subject has been detected as a result of the subject detection process in step S102. If a negative determination result is obtained because the presence of the individual subject is not detected (the number of detected individual subjects is 0), the process proceeds to step S104, and the zoom lens for widening the field angle is obtained. The movement control (zoom-out control) is executed. By widening the angle of view in this way, a wider range is imaged, so that it becomes easier to capture individual subjects. At the same time, in step S105, control (pan / tilt control) for moving the pan / tilt mechanism of the pan / tilt head 10 for subject search is executed. At this time, the subject detection processing block 61 passes a control signal for pan / tilt control to the communication control processing block 63 and performs control so as to be transmitted to the communication unit 52 of the camera platform 10.
As the pan / tilt control for searching for the subject, the pattern in which the pan / tilt mechanism of the pan / tilt head 10 is moved is determined in consideration of, for example, efficient search. That's fine.
In step S106, 0 is set for the mode flag f (f = 0), and the process returns to step S101.
In this manner, the procedure from step S101 to step S106 is repeated until at least one individual subject is detected in the image content of the captured image data. At this time, the system including the digital still camera 1 and the camera platform 10 is in a state in which the digital still camera 1 is moved in the pan direction and the tilt direction in order to search for a subject.

  If it is determined that the presence of the individual subject is detected in step S103 and a positive determination result is obtained, the process proceeds to step S107 and subsequent steps. The procedure after step S107 is mainly executed by the composition control processing block 62.

In step S107, it is determined what the current mode flag f is.
If it is determined that f == 0, this indicates that the first rough subject capturing mode should be executed as composition control, and the procedure starting from step S108 as shown in FIG. Run.
In step S108, the total subject gravity center Gt is located at the origin coordinate P (0,0) (see FIG. 7) on the screen of the captured image data (screen obtained when the image content of the captured image data is represented). It is determined whether or not it is. Here, if a negative determination result is obtained because the total subject gravity center Gt is not yet located at the origin coordinates, in step S109, the total subject gravity center Gt is located at the origin coordinates, and the pan head Control for moving the pan / tilt mechanism 10 is executed, and the process returns to step S101. In this way, in the capture mode, which is the first composition control procedure in the state where the presence of an individual subject is detected, the total subject center of gravity Gt is first positioned with respect to the origin coordinates, which is the initial reference position. In this way, by controlling the pan / tilt mechanism of the camera platform 10, the image area where the detected individual subject is captured is positioned at the center of the screen.

An example of an algorithm for actually performing the pan / tilt control as step S109 is shown here.
In the state in which the individual subject is detected with the mode flag f == 0, the subject detection processing block 61 performs the calculation represented by the following (Equation 1) to perform the necessary movement amount Span in the pan direction and the tilt direction. The required movement amount Stilt is obtained. In the following (Equation 1), n indicates the number of detected individual subjects, and p (Xi, Yi) is the i-th individual subject among the individual subjects given numbers 0 to n-1. The X and Y coordinates of the center of gravity are shown. For confirmation, as shown in FIG. 7, the origin coordinate (0, 0) in this case is the intersection of the middle point in the horizontal direction and the middle point in the vertical direction on the screen.

For example, in step S108, it is determined whether or not the absolute values of the necessary movement amounts Span and Stilt obtained as described above are within a predetermined value (strictly, it is 0, but may be a value larger than 0). By determining, it is possible to perform determination equivalent to whether or not the total subject gravity center Gt is at the origin coordinate P. In step S109, pan / tilt control is executed such that the absolute values of the necessary movement amounts Span and Stilt are within a predetermined range. Note that the speeds of the pan mechanism unit 53 and the tilt mechanism unit 56 at the time of pan / tilt control at this time may be constant. For example, the speed is increased as the required movement amounts Span and Stilt increase. It is possible to make it variable by going on. In this way, even when the necessary movement amount due to panning or tilting becomes large, it becomes possible to bring the total subject gravity center Gt close to the origin coordinates in a relatively short time.
If it is determined in step S108 that the total subject gravity center Gt is located at the origin coordinate and a positive determination result is obtained, step S110 sets 1 for the mode flag f (f = 1), and the process returns to step S101. . The state in which 1 is set for the mode flag f in step S109 is a state in which the capture mode, which is the first procedure in composition control, is completed, and the next first composition adjustment control (composition adjustment mode) is to be executed. Indicates that there is.

  When the mode flag f == 1 and the first composition adjustment mode is to be executed, the process proceeds from step S107 to step S111. As will be understood from the following description, the first composition adjustment mode is a zoom (view angle) adjustment for obtaining an optimum composition according to the number of detected individual subjects. Depending on the angle of view adjustment, the size of the individual subject in the screen or the distance between the plurality of individual subjects may change.

In step S111, it is determined how many individual subjects are currently detected. If the number is 1, the procedure starting from step S112 is executed.
In step S112, it is determined whether or not the size of the detected individual subject is OK. The state where the size of the individual subject is OK is a state where the occupation ratio of the image portion as the individual subject within the screen is within a predetermined range value as shown in FIG. 8B. If a negative determination result is obtained in step S112, the process proceeds to step S113, and the zoom lens drive control (zoom) is performed so that the occupation ratio falls within the predetermined range value (contains within the predetermined range value). Control), and the process returns to step S101. At this time, the position corresponding to the X coordinate (X = 0) set in step S109 is maintained with respect to the position of the center G of the individual subject (total subject center Gt) in the horizontal direction (left-right direction). Thus, zoom control is performed. As a result, it is possible to maintain a state in which the individual subject is positioned substantially at the center in the left-right direction. Further, since zoom-out control is performed in step S104 when subject search and detection operations are performed, it is considered that zoom-in control is often performed during zoom control in step S113. However, if a negative determination result is obtained in step S112 in response to the screen content state such that the occupation ratio exceeds the predetermined range value for some reason, zoom-out is executed in step S113. Control is performed so that the occupation ratio falls within a predetermined range value.
If a positive determination result is obtained in step S112, the process proceeds to step S114, 2 is set for the mode flag f, and the process returns to step S101. Note that the mode flag f == 2 should be executed after the first composition adjustment is completed and the next second composition adjustment is performed, as will be understood from the following description. Indicates that there is.

If it is determined in step S111 that the number of individual subjects is 2, the procedure starting from step S115 is executed.
In step S115, the distance K between the two individual subjects on the captured image data screen is 1/3 of the horizontal image size Cx as shown in FIG. 9B (K = = Cx / 3) is determined. If a negative determination result is obtained here, the process proceeds to step S116, and zoom control is executed so that the state of K == Cx / 3 is obtained. At this time as well, the zoom control is performed so as to maintain the X coordinate (X = 0) set in step S109 for the position of the total subject gravity center Gt in the horizontal direction. This also applies to step S119 described later. If a positive determination result is obtained in step S115 that K == Cx / 3, the process proceeds to step S117, where 2 is set for the mode flag f, and the process returns to step S101.

If it is determined in step S111 that the number of individual subjects is 3 or more, the procedure starting from step S118 is executed.
In step S118, the distance K between subjects on the screen of the captured image data (in this case, the distance from the center of gravity of the leftmost individual subject in the screen to the center of gravity of the rightmost individual subject in the screen). As shown in FIG. 10B, it is determined whether or not the horizontal image size Cx is half (K == Cx / 2). If a negative determination result is obtained here, the process proceeds to step S119, and zoom control is executed so that K == Cx / 2. If a positive determination result is obtained assuming that K == Cx / 3, the process proceeds to step S120, 2 is set for the mode flag f, and the process returns to step S101.

  In this way, in the state where 2 is set for the mode flag f, the procedure up to composition control corresponding to the case where one, two, or three or more individual subjects are described with reference to FIGS. 8 to 10 is completed. Will be in the state. Therefore, when it is determined in step S107 that the mode flag f is 2, the second composition adjustment mode is executed according to the procedure after step S121.

For example, in the description of the composition control in FIGS. 8 to 10, reference is made to how to set the position of the center of gravity of the individual subject in the vertical direction of the screen in order to simplify the description. In practice, however, it may be better to move (offset), for example, a certain required amount upward from the center of the screen. Therefore, in the actual composition control of the present embodiment, the offset amount in the vertical (vertical) direction of the total subject gravity center Gt can be set so that a better optimal composition can be obtained. The procedure for this is the second composition adjustment mode, and is executed as step S121 and step S122 described next.
In step S121, the position of the total subject center of gravity Gt (or the center of gravity G of the individual subject when there is one individual subject) is a predetermined offset amount from the horizontal straight line (X axis) passing through the origin coordinate P on the screen. It is determined whether or not there is an offset state (whether or not the gravity center offset is OK).
If a negative determination result is obtained in step S121, tilt control is executed so that the tilt mechanism of the camera platform 10 moves by offsetting the center of gravity by the set offset amount in step S122. Return to step S101. Then, at the stage where a positive determination result is obtained in step S121, a state is obtained in which an optimum composition corresponding to the number of individual subjects is obtained.
Note that there are several methods for setting the value of the offset amount as the center-of-gravity offset corresponding to these steps 121 and S122, and therefore there is no particular limitation here. One of the simplest settings is to give an offset value having a length corresponding to 1/6 of the vertical image size Cy from the center position in the vertical direction based on, for example, a three-division method. Of course, for example, it may be configured to set different offset values according to the number of individual subjects according to a predetermined rule.

  If a positive determination result is obtained in step S121, the processing procedure corresponding to the release operation starting from step S123 is executed. Here, the release operation refers to an operation for storing the captured image data obtained at that time in a storage medium (memory card 40) as still image data. In other words, when a manual shutter operation is performed, the captured image data obtained at that time is recorded on the storage medium as still image data in response to the shutter operation.

In step S123, it is determined whether or not the conditions under which the release operation can be currently performed are satisfied. Examples of conditions include being in focus (when autofocus control is set to be effective), pan / tilt mechanism of pan head 10 being stopped, and the like.
If a negative determination result is obtained in step S123, the process returns to step S101. Accordingly, it is possible to wait for the condition for executing the release operation to be satisfied. If a positive determination result is obtained in step S123, the release operation is executed in step S104. In this way, in this embodiment, captured image data having an optimal composition can be recorded.
When the release operation is finished, initial setting is performed for necessary parameters in step S125. By this process, the initial value 0 is set for the mode flag f. Further, the position of the zoom lens is also returned to the preset initial position.
And after performing the process of step S125, it returns to step S101. By returning from step S125 to step S101 in this manner, an object is searched for, and an optimum composition corresponding to the number of individual objects detected by this search is obtained and image recording (release operation) is performed. Will be automatically and repeatedly executed.

  Note that the release operation in the case of FIG. 11 is an operation for recording a still image from a captured image on a recording medium. However, the release operation in the present embodiment is a broader sense of the above still image. Including recording on a recording medium, for example, it means obtaining necessary still image data from a captured image. Therefore, for example, an operation of acquiring still image data from a captured image so as to be transmitted to another recording apparatus or the like via the data interface by the digital still camera 1 of the present embodiment is also a release operation. Is.

Further, in FIG. 11, the zoom control corresponding to steps S112 and S113, the zoom control corresponding to steps S115 and S116, or the zoom control corresponding to steps S118 and S119 is executed according to the determination result in step S111. Thus, it can be considered that the composition determination method is changed according to the number of individual subjects currently detected.
The change of the composition determination method here means, for example, changing an algorithm for composition determination and composition control, or changing parameters for composition determination and composition control. When it is determined that the number of individual subjects detected in step S111 is 1, zoom control is performed based on the occupation ratio of the image portion as an individual subject in the screen in steps S112 and S113. When it is determined that the number of individual subjects detected in step S111 is 2 or more, zoom control is performed based on the inter-subject distance K instead of the above occupancy rate. It can be said that the algorithm for the composition determination and composition control related to the adjustment of the size of the individual subject is changed according to the number of detected individual subjects. Further, in the case where the number of detected individual subjects is 2 or more, the distance K between subjects determined to be the optimal composition between the case where the number of individual subjects is 2 and the case where it is 3 or more is Cx / 3. Different values are set like Cx / 2. It can be said that the parameters for composition determination and composition control relating to the adjustment of the size of the individual subject are changed according to the number of detected individual subjects.

Subsequently, the second composition control in the present embodiment will be described. In the second composition control, the screen setting (composition) as captured image data is switched between portrait and landscape according to the number of detected individual subjects as described below.
In the second composition control, first, an object is detected in a state where a horizontally long composition is set as an initial state.
For example, it is assumed that one individual subject SBJ0 is detected in the captured image data screen as shown in FIG. When the number of individual subjects detected in this way is one, in the second composition control, the composition is set to be vertically long as shown as a transition from FIG. 12 (a) to FIG. 12 (b). To be done.
In addition, for the individual subject SBJ0, size adjustment (zoom) control is performed so that the occupation ratio in the screen is within a predetermined range value. Further, in this case, the position of the individual subject SBJ0 in the horizontal direction is substantially the center, and the position in the vertical direction is offset upward from the center according to a predetermined rule. Is done.

  Regarding the composition in the case of one subject, it can be considered that, in particular, in the case of a person, the overall composition becomes better when the composition is landscape rather than portrait. Therefore, in the second composition control, based on this concept, when there is one individual subject, the composition control is performed such that the composition and the position and the position of the individual subject are adjusted after having a vertically long composition. Is.

In the present embodiment, in order to change the composition from landscape to portrait as described above, for example, an image area corresponding to the portrait size is cut out from captured image data obtained as a landscape composition, and thus obtained. It is conceivable to use the vertically long image data portion.
Alternatively, for example, the pan head 10 is provided with a mechanism capable of moving the digital still camera 1 between a state corresponding to the horizontal placement and a state where the digital still camera 1 is placed vertically, and the mechanism is driven and controlled. Thus, a configuration in which the composition can be changed can be considered.

Also, assume that two individual subjects SBJ0 and SBJ1 are detected in the screen of the captured image data as shown in FIG. In the second composition control, when the number of two individual subjects is detected in this way, is the subject-to-subject distance K at or below the predetermined threshold at the angle of view corresponding to the detection time? Judgment about whether or not.
When the inter-subject distance K is equal to or less than the threshold value, the two individual subjects are in a substantially close state to each other. In such a state, it is possible to take the idea that it is preferable to use a vertically long composition rather than a horizontally long composition. Therefore, in this case, the composition is changed to be vertically long as shown as a transition from FIG. 13 (a) to FIG. 13 (b). For example, the method for changing the composition is as described above. In addition, zoom control and pan / tilt control are performed so that the sizes and positions of the individual subjects SBJ0 and SBJ1 are appropriate. Also in this case, the position in the horizontal direction of the image portion composed of the individual subjects SBJ0 and SBJ1 in the screen is set substantially at the center, and the position in the vertical direction is higher than the center according to a predetermined rule To be in.
On the other hand, when the inter-subject distance K between the two detected individual subjects SBJ0 and SBJ1 exceeds the threshold value, the two individual subjects are separated from each other accordingly. Therefore, it is preferable to set it horizontally long. Therefore, in this case, composition control similar to that described above with reference to FIG. 9 is performed.

  Also, it is assumed that three or more individual subjects SBJ0 to SBJn (n is a natural number of 3 or more) are detected in the screen of the captured image data. Here, when three or more individual subjects are detected, it is assumed that the composition is preferably in landscape orientation when viewed as an overall composition. Therefore, in this case, the same composition control as described with reference to FIG. 10 is performed as the second composition control.

FIG. 14 shows an example of a procedure that is executed by the subject detection processing block 61, the composition control processing block 62, and the communication control processing block 63 shown in FIG. 5 corresponding to the second composition control.
In this figure, the procedure from step S201 to S210 is the same as the procedure from step S101 to S110 in FIG. However, in step S204, zoom-out control is executed in the same manner as in step S104, and when the composition set so far is vertically long, control for resetting it to the horizontally long initial state is also executed. To be done.

In the state where the mode flag f == 1, it is determined in step S211 whether the number of detected individual subjects is 1 or 2 or 3 or more in the same manner as in step S111. Has been.
First, when it is determined in step S211 that the number of individual subjects is 1, a procedure starting from step S212 is executed.
In step S212, if the composition set so far is horizontally long, control for changing it to vertically long is executed. As this processing, for example, as described above, signal processing for cutting out an image area of a vertically long size from captured image data obtained as a horizontally long composition may be executed. Such processing should be realized by the function as the composition control processing block 62 in the signal processing unit 24. When the procedure of step S212 is executed, the process proceeds to step S213.
Steps S213 to S215 are the same as steps S112 to S114 in FIG. 11, respectively.
The composition control described with reference to FIG. 12 (excluding the upward offset for the individual subject) is performed by the procedure from step S212 to step S215.

If it is determined in step S211 that the number of individual subjects is 2, the procedure starting from step S216 is executed.
In step S216, it is determined whether or not the individual subject distance K between the two detected individual subjects is equal to or less than a threshold value. If an affirmative determination result is obtained, the process proceeds to step S217. If the composition set so far is horizontally long, control for changing it to vertically long is executed. Then, steps S213 to S215 are executed.
However, when the process from step S217 to step S213 is performed, when executing the procedure of steps S213 to S215, it is different from the case where the number of individual objects is one, which is set corresponding to the fact that there are two individual objects. A predetermined range value for the occupation ratio is set. If the occupation ratios of the two individual subjects are within the predetermined range value, 2 is set for the mode flag f in step S215, assuming that the appropriate size of the individual subject is obtained.
On the other hand, if a negative determination result is obtained in step S216, the procedure starting from step S218 is executed.
In step S218, if the composition set so far is vertically long, control is performed to change it to horizontally long. Subsequent steps S219 to S221 are the same as steps S115 to S117 in FIG.
The composition control of the second example when the number of individual subjects is 2 is performed by the procedure from step S216 to S221 and the procedure from step S217 to step S213 to S215. In other words, the composition control in which the screen when the inter-subject distance K is short is vertically long and the composition control in which the screen is horizontal when the inter-subject distance K is long are selectively used.

Note that regarding whether to set the composition to portrait or landscape, in step S216, only the individual subject distance K corresponding to the horizontal direction of the screen is used as a determination element. However, in practice, for example, in addition to the distance K between individual subjects corresponding to the horizontal (left and right) direction of the screen, the distance between individual subjects (Kv) corresponding to the vertical (vertical) direction of the screen may be added to the determination element. It is done. The distance between individual subjects (Kv) can be defined as the distance between the center of gravity of the individual subject located at the top of the screen and the center of gravity of the individual subject located at the bottom.
For example, on an actual screen, the distance between two individual subjects may be considerably separated in the vertical direction. In such a case, even if the distance between the two individual subjects in the left-right direction is as far as possible, it may be considered that the overall composition may be better when the composition is vertically long.
As step S216, an example of an algorithm in the case where the individual subject distance Kv corresponding to the screen vertical direction and the individual subject distance Kv corresponding to the screen vertical direction are used as determination elements will be described.
For example, K / Kv is obtained as a ratio between the distance K between individual subjects corresponding to the horizontal direction of the screen and the distance Kv between individual subjects corresponding to the vertical direction of the screen. And it is discriminate | determined about this K / Kv being more than a predetermined threshold value. If K / Kv is equal to or larger than the threshold, compared with the distance between the individual subjects in the vertical direction, the more the distance between two individual subjects in the horizontal direction becomes correspondingly wider gutters Ukoto. In this case, a landscape composition is set in step S218. On the other hand, when K / Kv is less than the threshold, the distance between the individual subjects in the vertical direction is correspondingly increased. Therefore, in this case, a vertically long composition is set in step S217.
Alternatively, as in the previous description, the distance between individual subjects K corresponding to the horizontal direction of the screen is compared with a predetermined threshold value. Set the composition. On the other hand, when the distance K between individual subjects exceeds the threshold value, the individual subject distance Kv corresponding to the vertical direction of the screen is compared with a predetermined threshold value. Note that the threshold value to be compared with the individual subject distance Kv does not have to be the same value as the threshold value corresponding to the individual subject distance K, and if a value appropriately set corresponding to the individual subject distance Kv is used. Good. Then, if it is within the threshold value is subject-to-subject distance Kv, is to set the horizontal composition in step S218, if the exceeding the threshold value, to set the portrait composition in step S217.

  If it is determined in step S211 that the number of individual subjects is 3, the procedure starting from step S222 is executed. In step S222, if the composition set so far is vertically long, control is performed to change it to horizontally long. Subsequent steps S223 to S225 are the same as steps S118 to S120 in FIG.

As a result of the procedure so far, in a state where 2 is set for the mode flag f, the procedure starting from step S226 is executed.
The procedures in steps S226 and S227 are the same as S121 and S122 in FIG. By executing this procedure, for example, as described above with reference to FIGS. 12 and 13, it is possible to obtain a composition in which the individual subject is positioned above the center in the screen.

  Steps S228 to S230 are procedures related to the release operation in the same manner as steps S123 to S125 of FIG. By executing this procedure, captured image data for which an optimum composition is obtained by composition control can be recorded in a storage medium.

  Note that the composition control procedures shown in FIG. 11 and FIG. 14 are determined and determined based on the overall flow in accordance with the number of individual subjects detected. Thus, it can be considered that the zoom control and the pan / tilt control are appropriately executed so that the captured image data of the determined composition is actually obtained (reflected).

In each composition control procedure shown in FIGS. 11 and 14, basically, when the number of detected individual subjects is one, two cases are divided into three conditional branches in the case of three or more. The composition is determined in correspondence. However, this is merely an example. For example, when the number of individual subjects is three or more, the composition may be determined by further dividing the number of individual subjects.
For example , regarding the composition determination algorithm for determining which composition is portrait or landscape, in the case of FIG. 14, when the number of detected individual subjects is 2, portrait and landscape are selected according to the distance K between the individual subjects. However, when the number of detected individual subjects is 3 or more, the landscape is uniformly set. However, for example, even when the number of detected individual subjects is 3 or more, the threshold set so as to be adapted to each detected number of individual subjects was compared with the distance K between the individual subjects at that time. Based on the result, it may be configured to determine whether the composition is landscape or portrait. In other words, when the number of detected individual subjects is two or more, the composition can be configured such that the portrait composition and landscape composition are determined based on the distance K between the individual subjects. Also at this time, the individual subject distance Kv corresponding to the vertical direction described above in step S216 can be added to the determination element.

  By the way, when using the imaging system of the present embodiment, there is a situation in which composition control is performed only on one or more specific persons in an environment where there are correspondingly many people. It is believed that there is. However, assuming that the subject detection processing is based on face detection technology, the algorithm configuration that simply recognizes all detected faces as the number of individual subjects, as described above, only a specific person is detected. The target composition control is not performed properly. In particular, since the composition control according to the present embodiment sets different compositions according to the number of individual subjects, it is considered that the possibility of producing a result that the user does not want is likely to increase accordingly.

  Therefore, when the present embodiment is adapted to the above situation, the subject detection process in step S102 of FIG. 11 or step S202 of FIG. 14 is configured such that the following subject discrimination process is possible. can do.

In this case, first, an upper limit number of individual subjects (target individual subjects) to be subject to composition control can be set by, for example, an operation on the digital still camera 1. For example, the subject detection processing block 61 holds information on the set target individual subject upper limit number. Here, as a specific example, it is assumed that 2 is set as the target individual subject upper limit number.
For example, assume that captured image data having the image content shown in FIG. 15A is obtained as a result of executing the subject search operation (step 105, S205). Corresponding to the subject detection processing in step S101 or S202, the presence of four individual subjects is detected by face detection. The individual subject detected at this stage is treated as a “subject candidate” here. In the figure, the four candidate subjects in the screen are labeled with subject candidates DSBJ0, DSBJ1, DSBJ2, and DSBJ3 from left to right.

  In this way, four subjects (candidate subjects) are detected as a result of simple face detection. However, as described above, in this case, 2 is set as the target individual subject upper limit number. Based on this, the subject detection processing block 61 selects two candidate subjects in descending order of size from the four candidate subjects DSBJ0, DSBJ1, DSBJ2, and DSBJ3, and selects these candidate subjects as target individual subjects. To do. In this case, of the candidate subjects DSBJ0, DSBJ1, DSBJ2, and DSBJ3, the two largest sizes are the candidate subjects DSBJ2 and DSBJ3. Therefore, the subject detection processing block 61 treats the candidate subjects DSBJ2 and DSBJ3 as the target individual subjects SBJ0 and SBJ1, respectively, as shown in FIG. 15B, and the candidate subjects DSBJ0 and DSBJ1 are subject to the individual target. Ignore it as a non-subject. Then, when the procedure for composition control after step S107 in FIG. 11 or after step S207 in FIG. 14 is executed, only the target individual subject is controlled. By such subject discrimination, for example, even in an environment or situation where many people are in the vicinity, there is a person who wants to be subject to composition control at the closest position to the imaging system. By doing so, it is possible to perform photographing by appropriate composition control for only these persons.

The flowchart in FIG. 16 shows an example of the procedure for subject discrimination described above, which is executed as part of the subject detection process in step S102 in FIG. 11 or step S202 in FIG.
In this process, for example, first, all subjects detected by the face detection process are treated as subject candidates. In step S301, the process waits until at least one subject candidate is detected. If a subject candidate is detected, the process proceeds to step S302.

In step S302, it is determined whether or not the currently set target individual subject upper limit number is equal to or greater than the number of subject candidates detected corresponding to step S301.
If a positive determination result is obtained in step S302, the number of subject candidates does not exceed the target individual subject upper limit number. Therefore, in this case, all detected subject candidates are set to be handled as target individual subjects in step S303.
On the other hand, if a negative determination result is obtained in step S302, the number of subject candidates is larger than the target individual subject upper limit number. In this case, in step S304, subject candidates corresponding to the upper limit number of target individual subjects are selected in descending order of the size of the detected subject candidates. Then, the subject candidates selected in step S305 are set to be handled as target individual subjects. As a result, subject discrimination is performed.
Through the procedure of FIG. 16, the result of the subject detection process in step S102 of FIG. 11 or step S202 of FIG. 14 is the number of target individual subjects set in step S303 or step S205. Information such as the size and position of each subject is output to the composition control processing block 62 as detection information. The composition control processing block 62 executes composition control after step S107 in FIG. 11 or after step S207 in FIG. 14 using this detection information.

FIG. 17 shows a configuration example as a modified example of the imaging system of the present embodiment.
In this figure, first, the captured image data generated by the signal processing unit 24 based on imaging is transmitted from the digital still camera 1 to the camera platform 10 via the communication control processing block 63. ing.
In this figure, a communication control processing block 71, a pan / tilt control processing block 72, a subject detection processing block 73, and a composition control processing block 74 are shown as the configuration of the camera platform 10.
The communication control processing block 71 is a functional part corresponding to the communication unit 52 of FIG. 4, and performs communication processing with the communication control processing block unit 63 (the pan head compatible communication unit 34) on the digital still camera 1 side with a predetermined protocol. It is the part comprised so that it may perform according to.
The captured image data received by the communication control processing block 71 is passed to the subject detection processing block 73. The subject detection block 73 includes a signal processing unit capable of at least subject detection processing equivalent to the subject detection processing block 61 shown in FIG. 5, for example. Subject detection processing is performed on the captured image data. And the detection information is output to the composition control processing block 74.
The composition control processing block 74 can execute composition control equivalent to the composition control processing block 62 of FIG. Output to the tilt control processing block 72.
The pan / tilt control processing block 72 corresponds to an execution function of processing related to pan / tilt control, for example, among the control processing executed by the control unit 51 in FIG. Signals for controlling the movement of the unit 53 and the tilt mechanism unit 56 are output to the pan driving unit 55 and the tilt driving unit 58. Thereby, panning and tilting are performed so that the composition determined by the composition control processing block 62 is obtained.
In this manner, the imaging system shown in FIG. 17 transmits the captured image data from the digital still camera 1 to the camera platform 10, and the camera platform 10 side performs subject detection processing and composition control based on the captured image data. And are configured to execute.

FIG. 18 shows a configuration example as another modified example of the imaging system of the present embodiment. In this figure, the same parts as those in FIG.
In this system, an imaging unit 75 is provided on the camera platform 10 side. The imaging unit 75 includes, for example, an optical system for imaging and an imaging device (imager), and obtains a signal (imaging signal) based on imaging light, and captures captured image data from the imaging signal. It consists of a signal processing unit for generating. For example, the optical system unit 21, the image sensor 22, the A / D converter 23, and the signal processing unit 24 shown in FIG. The captured image data generated by the imaging unit 75 is output to the subject detection processing block 73. Note that the direction in which the imaging unit 75 captures the imaging light (imaging direction) is set to coincide with the imaging direction of the optical system unit 21 (lens unit 3) of the digital still camera 1 placed on the camera platform 10 as much as possible, for example. Is set.

In this case, the subject detection processing block 73 and the composition control processing block 74 execute subject detection processing and composition control processing in the same manner as in FIG. However, the composition control processing block 73 in this case transmits a release instruction signal from the communication control processing block 71 to the digital still camera 1 in response to the timing for executing the release operation in addition to the pan / tilt control. Let In the digital still camera 1, a release operation is executed in response to receiving a release instruction signal.
In this way, in another modified example, regarding the subject detection processing and composition control, all control / processing other than the release operation itself can be completed on the pan head 10 side.

Furthermore, the following modifications can be considered for subject detection and composition control by the imaging system of the present embodiment.
For example, the composition control in the horizontal direction (left and right direction) has not been described so far. For example, according to the three-division method or the like, the subject is not placed in the center but in the left or right direction. A good composition can be obtained by biasing. Therefore, in actuality, as composition control according to the number of individual subjects, for example, a required amount of movement in the left-right direction with respect to the subject gravity center (individual subject gravity center, total subject gravity center) may be performed.

In addition, the pan control and tilt control executed in the composition control shown in FIGS. 11 and 14 are performed by controlling the movement of the pan / tilt mechanism of the camera platform 10, but instead of the camera platform 10. For example, the result of panning and tilting an image obtained based on the imaging light after the imaging light reflected by the reflecting mirror is incident on the lens unit 3 of the digital still camera 1. It is also possible to adopt a configuration in which the reflected light is moved so that
Further, by performing control of shifting a pixel region for capturing an effective imaging signal as an image from the image sensor 22 of the digital still camera 1 in the horizontal direction and the vertical direction, it is equivalent to performing panning / tilting. Result can be obtained. In this case, there is no need to prepare a pan / tilt device unit other than the pan head or the digital still camera 1 according to this, and the composition control as the present embodiment is completed by the digital still camera 1 alone. It becomes possible.
In addition, it is possible to perform panning and tilting even if a mechanism is provided that can change the optical axis of the lens in the optical system unit 21 in the horizontal and vertical directions and control the movement of this mechanism. It is.

  Further, the configuration for composition determination based on the present invention can be applied to other than the imaging system described as the embodiment so far. Therefore, hereinafter, application examples of composition determination according to the present invention will be described.

First, FIG. 19 shows a case where the composition determination according to the present invention is applied to a single imaging apparatus such as a digital still camera. For example, when an image captured by the imaging apparatus in an imaging mode becomes an appropriate composition. In addition, this is to be notified to the user by display.
For this purpose, the object detection / composition determination processing block 81, the notification control processing block 82, and the display unit 83 are shown here as the configuration that the imaging apparatus should have.
The subject detection / composition determination processing block 81 takes captured image data and uses, for example, subject detection processing equivalent to the subject detection processing block 61 of FIG. 5 and detection information as a result of this subject detection processing, for example, This is a part that is configured to perform a composition determination process equivalent to that in FIG.
For example, it is assumed that the user sets the imaging device in the imaging mode, holds the imaging device in his hand, and can record the captured image at any time by performing a release operation (shutter button operation).
Under such a state, the subject detection / composition determination processing block 81 captures captured image data obtained by imaging at that time and performs subject detection. Then, depending on the composition control process, first, what the optimal composition is is determined according to the number of detected individual subjects and the like. In this case, as the composition determination process, Thus, the consistency and similarity between the composition of the image content of the captured image data obtained and the optimum composition is obtained. Then, for example, when the degree of similarity becomes equal to or higher than a certain level, it is determined that the image content of the captured image data actually obtained by photographing has an optimal composition. For example, in practice, when a degree of similarity equal to or higher than a predetermined level is obtained, the algorithm is configured so that the composition is determined to be the optimum composition when the composition of the image content of the captured image data and the optimum composition are considered to match. It is possible. In addition, since various algorithms can be considered as to how to obtain the coincidence and similarity here, specific examples thereof are not particularly mentioned here.
Information on the determination result that the screen content of the captured image data has the optimum composition in this way is output to the notification control processing block 82. In response to the input of the above information, the notification control processing block 82 displays on the display unit 83 in a predetermined manner for notifying the user that the currently captured image has the optimum composition. Execute display control. The notification control processing block 82 is realized by a display control function by a microcomputer (CPU) provided in the imaging device, a display image processing function for realizing image display on the display unit 83, and the like. Note that the notification to the user of the optimal composition here may be made by sound such as electronic sound or synthesized speech.
The display unit 83 corresponds to, for example, the display unit 33 of the digital still camera 1 according to the present embodiment. For example, the display unit 83 is provided so that the display panel is exposed to a predetermined position in the imaging apparatus. In the mode, the so-called through image is generally displayed and the image captured at that time is displayed. Therefore, in the actual image pickup apparatus, the display unit 83 displays an image with a content for notifying that the optimum composition is superimposed on the through image. The user performs a release operation when a display notifying that the optimum composition is displayed. As a result, even a user who is not good at photography knowledge and technology can easily take a photo with a good composition.

FIG. 20 also applies the composition determination according to the present invention to a single imaging apparatus such as a digital still camera as in FIG.
First, in the configuration shown in this figure, as in FIG. 19, the subject detection / composition determination processing block 81 captures captured image data obtained by imaging at that time and performs subject detection processing, as well as subject detection information. Based on this, it is determined whether or not the image content of the captured image data has an optimal composition. When it is determined that the optimum composition has been achieved, this is notified to the release control processing block 84.
The release control processing block 84 is a part that executes control for recording captured image data, and is realized by, for example, control executed by a microcomputer included in the imaging apparatus. Upon receiving the above notification, the release control processing block 84 executes image signal processing and recording control processing so that the captured image data obtained at that time is stored in, for example, a storage medium.
With such a configuration, for example, when an image with an optimal composition is captured, an imaging apparatus can be obtained in which the captured image is automatically recorded.

  19 and 20 can be applied to, for example, a digital still camera having a configuration as shown in FIG. 1, for example, in the category of a still camera, or so-called recording a captured image on a silver salt film or the like. Also applied to what is called a silver halide camera by providing an image sensor that spectrally captures imaged light obtained by an optical system and a digital image signal processing unit that inputs and processes signals from this image sensor Is possible.

FIG. 21 shows an example in which the present invention is applied to an editing apparatus that performs image editing on already existing image data.
In this figure, an editing device 90 is shown. Here, the editing device 90 is configured to obtain image data (reproduced image data) obtained by reproducing, for example, data stored in a storage medium, as already existing image data. In addition to the data reproduced from the storage medium, for example, data downloaded via a network may be imported. That is, the route through which the captured image data to be captured by the editing apparatus 90 is acquired is not particularly limited.

Reproduced captured image data taken by the editing device 90 is input to the trimming processing block 91 and the subject detection / composition determination processing block 92, respectively.
First, the subject detection / composition determination processing block 92 first executes subject detection processing similar to FIGS. 19 and 20, for example, and outputs detection information. Then, as a composition determination process using this detection information, in this case, an image portion having a predetermined aspect ratio (optimum composition) at which the optimum composition is obtained on the entire screen as the input reproduced captured image data. Identify where the image part is). When the image portion of the optimum composition is specified, for example, information indicating the position of the image portion (trimming instruction information) is output to the trimming processing block 91.
In response to the input of the trimming instruction information as described above, the trimming processing block 91 executes image processing for extracting an image portion indicated by the trimming instruction information from the input reproduction captured image data. The extracted image portion is output as one independent image data. This is the edited captured image data.
With such a configuration, for example, as image data editing processing, trimming is automatically performed in which image data having a content obtained by extracting a portion having an optimal structure from the image content of the original image data is newly obtained. Become. Such an editing function may be adopted, for example, as an image data editing application installed in a personal computer or an image editing function in an application for managing image data.

FIG. 22 shows an example of a configuration in which the composition determination of the present invention is applied to an imaging apparatus such as a digital still camera.
Here, captured image data obtained by imaging by an imaging unit (not shown) is input to the subject detection / composition determination processing block 101 and the file creation processing block 103 in the imaging apparatus 100. In this case, the captured image data input into the imaging apparatus 100 is captured image data that should be stored in a storage medium in accordance with a release operation, for example, and is not illustrated here. The signal is generated based on an image signal obtained by image capturing in the image capturing unit.
First, in the subject detection / composition determination processing block 101, subject detection is performed on the input captured image data, and the optimum composition is determined based on the detection information. Specifically, for example, as in the case of FIG. 21, it is only necessary to obtain information specifying an image portion having an optimal composition in the entire screen of the input captured image data. Then, information representing the determination result for the optimum composition obtained in this way is output to the metadata creation processing block 102.
In the metadata creation processing block 102, metadata (composition editing metadata) including information necessary for obtaining an optimum composition from corresponding captured image data is created based on the input information, and a file creation processing block 103 is created. Is output. The contents of the composition editing metadata include, for example, position information that can indicate where the image area portion to be trimmed is on the screen as the corresponding captured image data.
In the imaging apparatus 100 shown in this figure, captured image data is recorded on a storage medium so as to be managed as a still image file in a predetermined format. In response to this, the file creation processing block 103 converts (creates) the captured image data into a still image file format.
First, the file creation processing block 103 performs image compression encoding corresponding to the image file format on the input captured image data, and creates a file body portion composed of the captured image data. At the same time, the composition editing metadata input from the metadata creation processing block 102 is stored in a predetermined storage position to create data portions such as a header and an additional information block. Then, a still image file is created from the file body part, header, additional information block, and the like, and this is output. As a result, as shown in the figure, a still image file to be recorded on the storage medium has a structure including metadata (composition editing metadata) together with captured image data.

FIG. 23 shows a configuration example of an editing apparatus that edits a still image file created by the apparatus of FIG.
The editing apparatus 110 shown in the drawing takes in still image file data and first inputs it to the metadata separation processing block 111. The metadata separation processing block 111 separates captured image data and metadata corresponding to the file body portion from the data of the still image file. The metadata obtained by the separation is output to the metadata analysis processing block 112, and the captured image data is output to the trimming processing block 113.

The metadata analysis processing block 112 is a part that executes processing for analyzing the captured metadata. As composition analysis metadata, for the composition editing metadata, an image region to be trimmed for the corresponding captured image data is specified at least from the information for obtaining the optimum composition as the content. Then, trimming instruction information for instructing trimming of the specified image area is output to the trimming processing block 113.
The trimming processing block 113 indicates the trimming instruction information input from the metadata separation processing block 112 from the captured image data input from the metadata separation processing block 111 side, similarly to the trimming processing block 91 of FIG. Image processing for extracting the image portion is executed, and the extracted image portion is output as edited captured image data which is one independent image data.

  According to the system including the imaging device and the editing device shown in FIGS. 22 and 23, original still image data (captured image data) obtained by, for example, photographing can be stored as it is without being processed. Thus, editing can be performed to extract an image having the optimum composition from the original still image data by using metadata. Further, the extraction image portion corresponding to such an optimal composition is automatically determined.

FIG. 24 shows an example in which the present invention is applied to an imaging apparatus capable of capturing and recording moving images as a video camera or the like.
Moving image data is input to the imaging device 120 shown in this figure. This moving image data is generated based on an imaging signal obtained by imaging by an imaging unit assumed to be included in the same imaging device 120, for example. This moving image data is input to the subject detection / composition determination processing block 122 and the file creation / recording processing block 124 in the imaging apparatus 120.
In this case, the subject detection / composition determination processing block 122 determines whether the composition of the input moving image data is acceptable. For example, the subject detection / composition determination processing block 122 holds parameters (good composition-corresponding parameters) regarding what kind of composition is supposed to be good in advance. The parameters include the occupancy rate of the individual subject in the screen set as appropriate according to the number of detected individual subjects, the inter-subject distance K, and the like. The subject detection / composition determination processing block 122 For example, composition determination is performed on the input moving image data, for example, regarding what composition is continuously being formed (for example, composition parameters such as an actual occupancy rate of the individual subject in the moving image data, an inter-subject distance K). In addition, the composition parameter of the moving image data obtained as a result of this determination is compared with the above-described good composition parameter. If the composition parameter of the moving image data has a certain degree of similarity to the good composition correspondence parameter, it is determined that the composition is good, and if the similarity is below a certain value, the composition is good. It is determined that there is no.
When the subject detection / composition determination processing block 122 determines that a good composition is obtained for the moving image data as described above, the subject detection / composition determination processing block 122 performs the above processing on the moving image data. Information (good composition image section instruction information) indicating where the image section (good composition image section) determined to have obtained a good composition is output. The good composition image section instruction information) is, for example, information indicating a start position and an end position as a good composition image section in the moving image data.

In this case, the metadata creation processing block 123 generates various required metadata for the moving image data recorded as a file on the storage medium by the moving image recording processing block 124 described below. In addition, when the good composition image section instruction information is input from the subject detection / composition determination processing block 122 as described above, it is confirmed that the image section indicated by the input good composition image section instruction information is a good composition. The generated metadata is generated and output to the moving image recording processing block 124.
The moving image recording processing block 124 executes control for recording the input moving image data on a storage medium so as to be managed as a moving image file in a predetermined format. When metadata is output from the metadata creation processing block 123, control is performed so that the metadata is recorded so as to be included in the metadata accompanying the moving image file. To do.
As a result, as shown in the figure, the moving image file recorded on the storage medium includes the moving image data obtained by imaging accompanied by metadata indicating an image section in which a good composition is obtained. Will have.
It should be noted that the image section in which a good composition is obtained, which is indicated by the metadata as described above, may be an image section based on a moving image having a certain time width, or extracted from the moving image data. It may be based on a still image. Also, in place of the above metadata, moving image data or still image data of an image section in which a good composition is obtained is generated, and this is added to secondary image data still image data attached to the moving image file. A configuration of recording as a file (or a file independent of a moving image file) is also conceivable.
Also, as shown in FIG. 24, in the configuration in which the imaging apparatus 120 includes the subject detection / composition determination processing block 122, the moving image determined to be a good composition image section by the subject detection / composition determination processing block 122. It is also conceivable that only the image section is recorded as a moving image file. Furthermore, a configuration may be considered in which image data corresponding to an image section determined to have a good composition by the subject detection / composition determination processing block 122 is output to an external device via a data interface or the like.

FIG. 25 shows an example in which the present invention is applied to a printing apparatus that performs printing.
In this case, the printing apparatus 130 captures image data (still image) having the image content to be printed. The captured data includes the trimming processing block 131 and the subject detection / composition. Input to the determination processing block 132.
First, the subject detection / composition determination processing block 132 executes subject detection processing / composition determination processing similar to the subject detection / composition determination processing block 92 of FIG. Is executed, and trimming instruction information having contents corresponding to the processing result is generated and output to the trimming processing block 131.
The trimming processing block 131 executes image processing for extracting the image portion indicated by the trimming instruction information from the input image data in the same manner as the trimming processing block 91 of FIG. Then, the extracted image portion data is output to the print control processing block 133 as print image data.
The print control processing block 133 executes control for operating a printing mechanism (not shown) using the input print image data.
With such an operation, depending on the printing apparatus 130, the image portion that is said to have the optimum composition is automatically extracted from the image content of the input image data and printed as a single image. It will be.

The example shown in FIG. 26 is suitable for application to, for example, an apparatus and system for storing a large number of still image files and providing services using these still image files.
A large number of still image files are stored in the storage unit 141.
The subject detection / composition determination processing block 142 takes in a still image file stored in the storage unit 141 at a predetermined timing, and extracts still image data stored in the file main body. Then, for this still image data, for example, processing similar to the subject detection / composition determination processing block 101 in FIG. 22 is executed to obtain information indicating the determination result for the optimal composition, and this information is used as metadata creation processing. Output to block 143.

  Based on the input information, the metadata creation processing block 143 creates metadata (composition editing metadata) in the same manner as the metadata creation processing block 102 of FIG. In this case, the created metadata is registered in the metadata table stored in the storage unit 141. This metadata table is an information unit formed by storing metadata so as to show a correspondence relationship with still image data stored in the same storage unit 141. That is, depending on the metadata table, metadata (composition editing metadata) and a still image file subjected to subject detection processing and composition determination processing by the subject detection / composition determination processing block 101 in order to create this metadata Correspondence with is shown.

  Then, for example, a still image file stored in the storage unit 141 is output in response to a request for an external still image file (for example, if it is a server, a still image file is downloaded in response to a download request from a client). When the still image file output processing block 144 searches for and imports the requested still image file from the storage unit 141, the metadata corresponding to the searched still image file (composition editing metadata) Data) is also retrieved from the metadata table.

The still image file output processing block 144 includes at least functional blocks corresponding to, for example, the metadata analysis processing block 112 and the trimming processing block 113 shown in FIG.
In the still image file output processing block 144, the internal metadata generation processing block analyzes the captured metadata to obtain trimming instruction information. Then, trimming according to the trimming instruction information is executed on the still image data stored in the captured still image file by the same internal trimming processing block. Then, the image portion obtained by the trimming is newly generated as one still image data, and this is output.

The system configuration shown in FIG. 26 can be applied to various services.
For example, one can be applied to a photo print service via a network. That is, the user uploads image data (still image file) desired to be printed (printed) to the print service server via the network. In the server, the still image file uploaded in this way is stored in the storage unit 141, and metadata corresponding to this file is also created and registered in the metadata table. When the image is actually printed out, the still image file output processing block 144 outputs the still image data extracted from the optimum composition as image data for printing. In other words, depending on this service, when a photo print is requested, a print that has been corrected to an optimal composition is sent.

  For example, it can be applied to a server such as a blog. The storage unit 141 stores image data uploaded together with blog text data. Thereby, for example, an image obtained by extracting the optimum composition from the image data uploaded by the user can be pasted on the blog page.

  The examples described with reference to FIGS. 17 to 26 are only a part, and other devices, systems, application software, and the like to which the composition determination according to the present invention can be applied are also conceivable.

Further, in the description of the embodiments so far, it is assumed that the subject (individual subject) is a person. For example, an object such as an animal or a plant, In this case, it is conceivable to apply the present invention.
Further, the image data to be subject to object detection should not be limited to only those obtained from imaging (captured image data), for example, images having image contents such as pictures or design images. It is also possible to target data.
Further, the composition (optimum composition) determined under the present invention is not necessarily limited to that determined based only on the three-division method. For example, a composition setting method based on the golden rate is known along with the three-division method, but there is no particular problem in adopting other methods such as the golden rate. Furthermore, the composition is not limited to the generally good composition such as the above three-division method or the golden rate. For example, even if the composition is generally not good, depending on the composition setting, it may be considered that the user may find it interesting or feel good. Therefore, the composition (optimum composition) determined under the present invention may be arbitrarily set in consideration of practicality and entertainment properties, and is not particularly limited in practice.

  DESCRIPTION OF SYMBOLS 1 Digital still camera, 2 Shutter button, 3 Lens part, 10 Head, 21 Optical system, 22 Image sensor, 23 A / D converter, 24 Signal processing part, 25 Encoding / decoding part, 26 Media controller, 27 Control part, 28 ROM, 29 RAM, 30 Flash memory, 31 Operation section, 32 Display driver, 33 Display section, 34 Pan head compatible communication section, 40 Memory card, 51 Control section, 52 Communication section, 53 Bread mechanism section, 54 Bread motor , 55 Pan drive unit, 56 Tilt mechanism unit, 57 Tilt motor, 58 Tilt drive unit, 61 Subject detection processing block, 62 Composition control processing block, 63 Communication control processing block, SBJ (SBJ0-n) Individual subject, 71 Communication control processing block, 72 Pan / tilt control Physical block, 73 Subject detection processing block, 74 Composition control processing block, 75 Imaging unit, 81, 92, 101, 122, 132, 142 Subject detection / composition determination processing block, 82 Notification control processing block, 83 Display unit, 84 Release Control processing block, 91/131 trimming processing block, 102/123/143 metadata creation processing block, 103 file creation processing block, 111 metadata separation processing block, 112 metadata analysis processing block, 113 trimming processing block, 124 file creation Recording processing block, 133 print control processing block, 141 storage unit, 144 still image file output processing block

Claims (15)

Subject detection means for detecting the presence of a specific subject in the image based on the image data;
Composition determining means for determining the composition according to the number of detected subjects that are the subjects detected by the subject detecting means;
With
The composition determination means is
The composition determination means is
A composition determination device that determines whether the rectangular image is portrait or landscape based on the distance between subjects located at both left and right ends when the number of detected subjects is equal to or greater than a predetermined value . When the number of detected subjects is equal to or greater than a predetermined value and the composition determination means is a predetermined value equal to or greater than 2, the composition determination means determines the rectangular image based on the distance between the subjects located at the left and right ends. It is designed to determine whether it is portrait or landscape.
The composition determination apparatus according to claim 1 . The composition determination means is
Composition determination device according to claim 1 or 2 can be set the maximum number of the object of interest of configuration diagram control. The composition determination device
The composition determination apparatus according to claim 3 , wherein among the detected subjects, the composition control target is set in descending order of the size of the detected subject based on the set upper limit number. The composition determination means is
If the number of subjects is 1,
The composition determination apparatus according to claim 1 , wherein the rectangular image is vertically long. The composition determination means is
When the number of subjects is 2,
The composition determination apparatus according to claim 2 , wherein the rectangular image is vertically long or horizontally long based on a relationship between a distance between the subjects and a predetermined threshold. The composition determination means is
If the number of subjects is 3,
The composition determination apparatus according to claim 2 , wherein the rectangular image is horizontally long. The composition determination means is
When the number of detected subjects is equal to or greater than a predetermined value, the rectangular image is displayed either vertically or horizontally based on the distance between subjects located at both left and right ends and the distance between subjects located at both top and bottom ends. To determine whether or not
The composition determination apparatus according to claim 1. When the number of detected subjects is equal to or greater than a predetermined value and the composition determination means has a predetermined value equal to or greater than 2, the distance between subjects located at both left and right ends and the distance between subjects located at both upper and lower ends Based on the above, it is determined whether the rectangular image is vertical or horizontal,
The composition determination apparatus according to claim 8 . The composition determination apparatus according to claim 1, further comprising mechanism control means for controlling the mechanism so that the composition determined by the composition determination means is obtained. The composition determination apparatus according to claim 1, further comprising notification control means for notifying a user when the determined composition is reached. The composition determination means is
Change the composition determination method according to the number of detected subjects,
In case the number of the detected subject is 2 or more, the distance between the object located in the left and right ends, the ratio of the length of the image horizontal direction, in claim 1 using a different value depending on the number of the detected subject The composition determination device described. The composition determination means is
The composition determination apparatus according to claim 12 , wherein the ratio is increased as the number of detected subjects increases. A subject detection procedure for detecting the presence of a specific subject in an image based on image data;
A composition determination procedure for determining the composition according to the number of detected subjects that are the subjects detected by the subject detection procedure;
Run
The composition determination procedure is as follows:
A composition determination method for determining whether the rectangular image is portrait or landscape based on the distance between subjects located at the left and right ends when the number of detected subjects is a predetermined value or more . A subject detection procedure for detecting the presence of a specific subject in an image based on image data;
A composition determination procedure for determining the composition according to the number of detected subjects that are the subjects detected by the subject detection procedure;
Is executed by the composition determination device,
The composition determination procedure is as follows:
A program for determining whether the rectangular image is portrait or landscape based on the distance between subjects located at the left and right ends when the number of detected subjects is equal to or greater than a predetermined value .

Images