JP2018201101A - Imaging apparatus and control method thereof, program, and storage medium - Google Patents

Abstract

To provide a trimming technique giving preferable impression to a user, without increasing workload of the user, by automatically performing such trimming that the situation and atmosphere of a photographic scene remain in the composition, and connection of the front and the back can be understood, in the continuous shooting image.SOLUTION: An imaging apparatus includes image acquisition means for acquiring temporally continuous images, subject detection means for detecting a main subject from the image, tracking detection means for tracking and detecting the main subject in the second and subsequent continuous images, subject region determination means for determining the subject region based on the main subject information, and composition determination means for determining a trimming composition based on the main subject information and the subject region. At least the trimming size and the trimming aspect to be determined by the composition means are determined in the first image out of the temporally continuous images, and the trimming positions in the second and subsequent temporary continuous images are determined by the tracking detection subject information.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a trimming control technique for image data.

  Conventionally, when taking a picture with an imaging device such as a digital camera, the photographer shoots with an awareness of the composition, but it is particularly difficult for beginners of photography to determine a suitable composition by instantly judging the subject and scene, It takes skill to instantly determine a desirable composition that can be sympathized with many people. Further, when trimming continuously shot images, the trimmed images are often reproduced continuously to check the flow of the scene, so it is desirable to have a composition that shows the connection between the front and back.

  Patent Document 1 is a technique for detecting a change in facial expression of a subject by detecting the same face from a plurality of images taken continuously and performing trimming so that the position and size of the same face are aligned.

Japanese Patent No. 5424827

  However, the prior art disclosed in the above-mentioned patent documents does not determine a suitable composition by judging a subject or a scene, and does not consider trimming that can understand the connection of scenes of continuous shot images. .

  Accordingly, the present invention has been made in view of the above problems, and an object of the present invention is to enable automatic trimming so that the situation and atmosphere of the shooting scene remain in the composition and the connection between the front and rear can be recognized in the continuous shot image. Thus, it is to realize a trimming technique that gives a suitable impression to the user without increasing the work burden on the user.

The image processing apparatus of the present invention includes an image acquisition unit that acquires temporally continuous images;
Subject detection means for detecting a main subject from the image;
Tracking detection means for tracking and detecting the main subject in the second and subsequent images of the temporally continuous image;
Subject region determining means for determining a subject region based on the main subject information;
Composition determining means for determining a trimming composition based on the main subject information and the subject region;
With
At least the trimming size and the trimming aspect determined by the composition means are determined for the first image of the temporally continuous image, and the trimming position is the tracking for the second and subsequent images of the temporally continuous image. It is characterized by being determined by detected subject information.

  According to the present invention, in continuous image trimming, it is possible to easily obtain a trimmed image having a composition with continuity before and after without being affected by the movement of the subject and the variation in detection of the subject.

It is a block diagram which shows the structure of the imaging device which concerns on this embodiment. It is a flowchart which shows the image shooting operation | movement which concerns on this embodiment. It is a flowchart which shows the auto trimming operation | movement which concerns on this embodiment. It is a figure for demonstrating the three division method. It is a figure which shows the change of the to-be-photographed object and trimming frame which concern on this embodiment. It is a flowchart which shows the auto trimming operation | movement which concerns on 2nd Embodiment.

  Hereinafter, embodiments for carrying out the present invention will be described in detail. The embodiment described below is an example for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment. Moreover, you may comprise combining suitably one part of each embodiment mentioned later.

<Device configuration>
First, the main configuration of the imaging apparatus according to the present embodiment will be described with reference to FIG. In FIG. 1, reference numeral 100 denotes an image processing apparatus. Reference numeral 10 denotes a photographing lens. Reference numeral 12 denotes a shutter having an aperture function. An image sensor 14 converts an optical image of a subject into an electric signal. Reference numeral 16 denotes an A / D converter that converts an analog signal output from the image sensor 14 into a digital signal.

  A timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, and is controlled by the memory control circuit 22 and the system control circuit 50. . An image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control circuit 22. Further, the image processing circuit 20 performs predetermined calculation processing using image data generated by imaging, and supplies the calculation result to the system control circuit 50.

  Based on the calculation result, the system control circuit 50 controls the exposure control unit 40 and the distance measurement control unit 42 to perform TTL (through-the-lens) AF (autofocus) processing, AE (automatic exposure). Processing, EF (flash pre-emission) processing is realized. Further, the image processing circuit 20 also performs TTL AWB (auto white balance) processing using image data generated by imaging.

  A memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression circuit 32. The data output from the A / D converter 16 is supplied to the memory control circuit 22 via the image processing circuit 20 or directly to the memory control circuit 22, and thereby written in the image display memory 24 or the memory 30. It is.

  Reference numeral 24 denotes an image display memory. Reference numeral 26 denotes a D / A converter. An image display unit 28 includes a TFT-LCD or the like. The display image data written in the image display memory 24 is supplied to the image display unit 28 via the D / A converter 26, whereby an image is displayed on the image display unit 28. An electronic viewfinder function can be realized by sequentially displaying images related to imaging on the image display unit 28. In addition, a playback display function and various display functions can be used by using the image display unit 28. Further, the image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control circuit 50. When the display is turned off, the power consumption of the imaging apparatus 100 can be significantly reduced. it can.

  A memory 30 stores captured still images and moving images. The memory 30 has a storage capacity sufficient to store a predetermined number of still images and a moving image for a predetermined time. By using this memory 30, a large number of images can be stored at high speed even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot. The memory 30 can also be used as a work area for the system control circuit 50.

  A compression / decompression circuit 32 compresses or decompresses image data by adaptive discrete cosine transform (ADCT) or the like. The compression / decompression circuit 32 reads an image stored in the memory 30, performs a compression process or an expansion process, and writes the processed data to the memory 30.

  An exposure control unit 40 controls the shutter 12 having an aperture function, and can perform flash light control in cooperation with the flash 48. A distance measuring control unit 42 controls focusing of the photographing lens 10. The exposure control unit 40 and the distance measurement control unit 42 are controlled by the system control circuit 50 based on a calculation result obtained by the image processing circuit 20 calculating image data generated by imaging. Thereby, TTL exposure control and distance measurement control are realized.

  A zoom control unit 44 controls zooming of the photographing lens 10. A barrier control unit 46 controls the operation of the barrier (protection unit) 102. Reference numeral 48 denotes a strobe, which has a function of projecting AF auxiliary light for brightly illuminating the subject and facilitating focusing. The strobe 48 may be a single unit, or a plurality of strobes may be provided, such as a xenon tube for flash emission and an LED for minute long-second emission, and each may be switched and controlled during main emission.

  Reference numeral 50 denotes a system control circuit that controls the entire imaging apparatus 100. A memory 52 stores constants, variables, programs, and the like for the operation of the system control circuit 50. The memory 52 also stores a program diagram used in AE. The program diagram is a table that defines the relationship between the aperture value for the exposure value and the control value for the shutter speed.

  Reference numeral 54 denotes a display unit such as a liquid crystal display device or a speaker, which displays an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program by the system control circuit 50. The display unit 54 is installed in a single or a plurality of positions near the operation unit of the imaging apparatus 100 so that the display unit 54 can be visually recognized. A part of the display unit 54 is installed in the optical viewfinder 104.

  Among the display contents of the display unit 54, what is displayed on the LCD or the like includes single shot / continuous shooting display, self-timer display, compression rate display, number of recorded pixels, number of recorded pixels, number of remaining images, shutter There are speed display, aperture value display, etc. In addition, exposure compensation display, flash display, red-eye reduction display, macro shooting display, buzzer setting display, clock battery level display, battery level display, error display, information display with multiple digits, and attachment / detachment of the recording medium 200 There are status display, communication I / F operation display, date / time display, and the like. Among the display contents of the display unit 54, what is displayed in the optical viewfinder 104 includes in-focus display, camera shake warning display, flash charge display, shutter speed display, aperture value display, exposure correction display, and the like.

  Reference numeral 56 denotes an electrically erasable / recordable non-volatile memory, such as an EEPROM. Reference numerals 60, 62, 64, and 66 are operation units for the user to instruct the system control circuit 50 to perform various operations. For example, a switch or a dial, a touch panel, pointing by eye-gaze detection, a voice recognition device, or a combination of one or more Consists of. More specifically, 60 is a mode dial switch, which is turned off, automatic exposure mode such as program AE mode, fully automatic mode, manual exposure mode, sleeping face shooting mode, panoramic shooting mode, playback mode, multi-screen playback / erase Each mode such as a mode and a PC connection mode can be switched.

  Reference numeral 62 denotes a shutter switch SW1, which is turned on during operation of a shutter button (not shown), and performs AF (auto focus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, EF (flash pre-flash) processing, and the like. Instruct the start of operation.

  A shutter switch SW2 64 is turned on when the operation of the shutter button is completed, and an exposure process of writing image data into the memory 30 via the A / D converter 16 and the memory control circuit 22 with the signal read from the image sensor 14 being performed. Start operation. Further, a series of development processes using operations in the image processing circuit 20 and the memory control circuit 22, a recording process in which the image data is read from the memory 30, compressed by the compression / decompression circuit 32, and written to the recording medium 200. The operation start of the process is instructed.

  An operation unit 66 includes various buttons and a touch panel. The operation unit 66 includes a menu button, a set button, a macro button, an image display on / off button, a multi-screen playback page break button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, menu movement + (plus) Button, menu shift- (minus) button, playback image shift + (plus) button, playback image- (minus) button, shooting image quality selection button, exposure correction button, date / time setting button, and the like.

  Reference numeral 80 denotes a power supply control unit, which includes a battery detection circuit, a DC-DC converter, a switch circuit that switches blocks to be energized, and the like. The power supply control unit 80 detects the presence / absence of a battery, the type of battery, and the remaining battery level, controls the DC-DC converter based on the detection result and the instruction of the system control circuit 50, and requires the necessary voltage. It is supplied to each part including the recording medium for a period. 82 and 84 are connectors. A power source 86 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like.

  Reference numerals 90 and 94 denote interfaces with recording media such as memory cards and hard disks. Reference numerals 92 and 96 denote connectors for connecting to a recording medium such as a memory card or a hard disk. A recording medium attachment / detachment detection unit 98 detects whether or not the recording medium 200 or 210 is attached to the connectors 92 and 96.

  In the present embodiment, it is assumed that there are two interfaces and connectors for attaching the recording medium. Of course, the interface and the connector for attaching the recording medium may have a single or a plurality of systems, any number of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard. The interface and the connector may be configured using a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like based on a standard.

  In addition, when the interfaces 90 and 94 and the connectors 92 and 96 are configured using a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like, a LAN card, a modem card, a USB card, IEEE1394. Image data and management information attached to image data are transferred to and from peripheral devices such as other computers and printers by connecting various communication cards such as cards, P1284 cards, SCSI cards, and PHS communication cards. I can meet each other.

  Reference numeral 102 denotes protection means that is a barrier that prevents the imaging unit from being soiled or damaged by covering the imaging unit including the photographing lens 10 of the imaging device 100.

  Reference numeral 104 denotes an optical viewfinder, which can take an image using only the optical viewfinder without using the electronic viewfinder function of the image display unit 28. In the optical viewfinder 104, some functions of the display unit 54, for example, a focus display, a camera shake warning display, a flash charge display, a shutter speed display, an aperture value display, an exposure correction display, and the like are installed.

A communication unit 110 has various communication functions such as RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, and wireless communication.
Reference numeral 112 denotes a connector for connecting the imaging apparatus 100 to another device by the communication unit 110 or an antenna in the case of wireless communication.

  Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 202 composed of a semiconductor memory, a magnetic disk, and the like, an interface 204 with the imaging device 100, and a connector 206 that connects to the imaging device 100.

  Reference numeral 210 denotes a recording medium such as a memory card or a hard disk. The recording medium 210 includes a recording unit 212 composed of a semiconductor memory, a magnetic disk, and the like, an interface 214 with the imaging device 100, and a connector 216 that connects to the imaging device 100.

<Description of operation>
An image capturing operation by the imaging apparatus of the present embodiment will be described. The processing described here is realized by the system control circuit 50 developing and executing a program recorded in the nonvolatile memory 56 in a work area of the volatile memory 52 such as a RAM.

  The system control circuit 50 performs various shooting preparation operations when the power supply 86 is turned on in the mode in which the mode dial 60 can shoot, and starts displaying a through image on the image display unit 28 when the shooting preparation is completed. Here, the through image is an image captured by the image sensor 14 for display as a finder function before and after taking a still image. In the through image display state, the data sequentially written to the image display memory 24 via the image sensor 14, the A / D converter 16, the image processing circuit 20, and the memory control circuit 22 are stored in the memory control circuit 22 and the D / A. The images are sequentially displayed on the image display unit 28 via the converter 26. Thereby, an electronic viewfinder (EVF) function is realized.

  Next, an image capturing operation in the through image display state will be described with reference to FIG.

  In FIG. 2, step S201 is repeated until the shutter switch SW1 is pressed. If the shutter switch SW1 is pressed, the process proceeds to step S202.

  The system control circuit 50 performs a distance measurement process to focus the photographing lens 10 on the subject, performs a photometry process, and determines an aperture value and a shutter time (step S202).

  In step S203, if the shutter switch SW1 is released without pressing the shutter switch SW2, the process returns to step S201. In step S204, when the shutter switch SW2 is pressed, the system control circuit 50 passes through the image sensor 14, the A / D converter 16, the image processing circuit 20, the memory control circuit 22, or directly from the A / D converter. Through the memory control circuit 22, the image data written in the memory 30 is read out using the exposure processing for writing the image data taken in the memory 30 and the memory control circuit 22 and the image processing circuit 20 as necessary. A photographing process including a developing process for performing various processes is executed (step S205). In step S206, auto-trimming processing is performed based on the captured image and subject information. Details will be described later.

  In step S207, if continuous shooting continues, the process returns to step S205 to perform shooting, and if there is no continuous shooting instruction, the process ends.

  Next, an auto trimming process for automatically trimming an image input in the image processing circuit 20 will be described with reference to FIG.

  First, in step S301, the process branches depending on whether the photographed image is the second or subsequent continuous shot or not. The continuous shooting referred to here does not have to be shooting in the continuous shooting mode, but is continuous shooting in time, and is continuous shooting within a predetermined time. If it is not the second and subsequent shots, subject detection is first performed in step S302. If the subject is a person, the region of the subject can be detected by detecting the face. A known method may be used for face detection. For example, the image processing circuit 20 extracts a shape corresponding to the constituent elements of the face area such as the eyes, nose and mouth from the input image data, and calculates the face area from the existing positional relationship of the eyes, nose and mouth. . A known method may be used for subject detection when the subject is not a person. For example, the input image data is divided into regions based on color and brightness, and subject-like weight is weighted according to the size of the divided regions and the distance from the screen center. Then, an area where the weight is a certain level or more is treated as a subject.

  In step S303, a subject area is calculated. For example, when a face can be detected, a body region is estimated from the position, orientation, and size of the face, and the region including the face and body may be used as the subject region, or means for acquiring distance information for each region of the image. It is also possible to use a region including a region equidistant from the main face or object as the subject region. The distance information acquisition method may be an existing one. For example, the focus position is moved, and an area having the highest contrast is defined as an equidistant group. Further, when the background is analyzed and there is an important subject in the background, the background important subject is also included in the subject area, thereby making it possible to create a composition that gives a better impression to the user.

  In step S304, the trimming aspect is determined. The aspect may be determined from, for example, an aspect that can be set by the camera based on the aspect ratio of the subject, or may be determined to be longer in the traveling direction when the subject is moving.

  In step S305, the trimming size is determined. The size may be determined by, for example, selecting the smallest size that can enclose the entire subject area from the image output sizes that can be set by the camera, or by determining the size based on the face size of the main subject. Also good. Furthermore, the detection error tends to be larger when the subject is not a person than when the subject is a person, so when the subject is other than a person, the trimming size is set larger to absorb the detection error. You may make it change trimming size according to.

  In step S306, the trimming position is determined. The position may be determined by, for example, a so-called Hinomaru composition in which the main subject is the center of the image, or if the main subject is moving, a wide space may be created in the traveling direction of the main subject. It is also possible to apply the three-division method to the main subject. As shown in FIG. 4, the three-division method means that at the four intersections P1 to P4 of the dividing lines obtained by drawing two lines so as to divide each of the horizontal and vertical directions of the field angle to be divided into three. If an important element such as the center of gravity of the subject is placed, a balanced and stable composition can be obtained.

  If it is the second and subsequent shots in step S301, the subject or subject area detected in the first continuous shot is tracked in step S307, and the process proceeds to step S306. Existing methods may be used for subject tracking. For example, a partial image of the detected subject is stored as a template, and template matching is performed. A point having a high degree of coincidence of pixel values with the template in the vicinity of the subject coordinates of the past image in the continuous image is defined as a tracking region.

  An example of composition determination from step S301 to step S307 will be described with reference to FIG. In FIG. 5, the first, second, and third images in continuous shooting are represented by (a), (b), and (c), respectively. In (a), the main subject T1 is far away, and the same subject T1 as in (b) and (c) is gradually approaching. In addition, the trimming ranges in (a), (b), and (c) are denoted by W1, W2, and W3, respectively.

  In the case of the first continuous shooting shown in (a), the subject is detected in step S302, the subject area is calculated in step S303, and the trimming aspect, size, and position are calculated in steps S303 to S306 based on the subject area. However, in the case of the second and subsequent continuous shots shown in (b) and (c), subject tracking is performed in step S307, only the trimming position is determined in step S306, and trimming for the second and subsequent continuous shots is performed. Use the same aspect and size as the first continuous shot. That is, W1, W2, and W3 have the same aspect and size, but only the position changes in accordance with the subject tracking result in step S307.

  As an advantage of this, first, since the subject area calculation in step S303 is eliminated, the processing such as distance information and background analysis described above is eliminated as an example, so that the processing time is reduced and the influence on the continuous shooting frame speed is almost eliminated. be able to. Furthermore, by making the trimming aspect and trimming size the same, continuous cropped images can be given continuity without being affected by subject movement or subject detection variation, so only trimmed images are displayed. It is possible to reduce the stress on the user when displaying one after another on the unit 28.

  Further, the trimming composition may be changed according to the ratio of the subject size to the trimming size. For example, when the ratio of the subject size to the trimming size is sufficiently small as shown in (a), a composition for creating a space in the advancing direction of the subject or a composition in which the three-division method is applied to the subject, (b) and (c) As described above, as the subject size occupying the trimming size increases, the subject moves closer to the center of the composition so that the subject does not protrude from the composition as much as possible.

  Subsequently, when the composition of trimming is determined, in step S308, it is determined whether the composition can be trimmed, such as whether it is within the original image. If trimming is possible, the system control unit 50 has written to the memory 30. The photographed image data is read out, and the image data is trimmed by the image processing circuit 20 according to the composition set in step S306, and the trimmed image data created by trimming is written in the memory 30 (step S309).

  Finally, the system control circuit 50 reads the image data written in the memory 30 and performs various image processing using the memory control circuit 22 and the image processing circuit 20 as necessary, and the compression / decompression circuit 32. After performing the image compression process according to the mode set using the recording process, a recording process for writing image data to the recording medium 200 or 210 is executed (S310).

  If the subject detection fails in step S302, or if the subject tracking fails in step S307, the auto-trimming process does not have to be performed, but continuous shooting continues. In this case, the number of continuous shots may be reset from step S205, and subject detection, subject area calculation, and trimming composition determination may be performed again.

<Modification>
A modification of the above-described embodiment will be described with reference to FIG. Steps S301 to S310 are the same as those in the above-described embodiment, and step S601 is newly added in the modification. The changes will be described below.

  When it is determined in step S301 that the second and subsequent shots are taken, the process proceeds to step S601, where it is determined whether or not the subject ratio in the trimming size is within a predetermined range. The process proceeds to step S307, but if it is outside the predetermined range, the process proceeds to step S302. More specifically, assuming that the predetermined range is 20% to 80%, for example, when the captured image size is 100, when the subject size is less than 20 or exceeds 80, the process proceeds to step S602. . Here, the size may be simply the area, or the length of the long side in the vertical and horizontal directions.

  In the case of passing through step S601, the trimming aspect may be determined again in step S304, or the aspect may be the same as the aspect determined in step S303 in order to reduce the change in the angle of view as much as possible.

  Further, the predetermined range in step S601 may be varied according to at least one of the type of subject, the shooting mode, and distance information to the subject. For example, when the subject is a child, when the shooting mode is sport mode, or when the distance to the subject is short, the predetermined range is set to 10% to 90% in consideration of the possibility of the subject moving quickly, Easy to avoid changing the trimming size.

  In this way, in the modified example, continuous shooting trimmed images can be given continuity without being affected by subject movement or subject detection variation, so that only the trimmed images are displayed one after another on the image display unit 28. In addition to reducing the user's stress, adaptively changing the trimming size according to the proportion of the subject in the trimming size allows the subject to protrude from the trimming angle of view even when the subject moves, and the subject becomes smaller It is possible to prevent the composition from becoming too low in balance.

<Other embodiments>
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program code. It is processing to do. In this case, the program and the storage medium storing the program constitute the present invention.

  10: photographing lens, 12: shutter, 14: image sensor, 16: A / D converter, 18: timing generation circuit, 20: image processing circuit, 22: memory control circuit, 24: image display memory, 26: D / A converter, 28: image display unit, 30: memory, 32: image compression / decompression circuit, 40: exposure control unit, 42: distance measurement control unit, 44: zoom control unit, 46: barrier control unit, 48: strobe 50: System control circuit, 52: Memory, 54: Display unit, 56: Non-volatile memory, 60: Mode dial switch, 62: Shutter switch SW1, 64: Shutter switch SW2, 66: Operation unit, 80: Power control unit , 82: connector, 84: connector, 86: power supply, 90: interface, 92: connector, 94: interface, 96: connector, 98: recording medium De-detection unit, 100: Image processing apparatus, 102: Protection means, 104: Optical finder, 110: Communication unit, 112: Connector (or antenna), 200: Recording medium, 202: Recording unit, 204: Interface, 206: Connector 210: Recording medium 212: Recording unit 214: Interface 216: Connector

Claims (8)

Image acquisition means for acquiring temporally continuous images;
Subject detection means for detecting a main subject from the image;
Tracking detection means for tracking and detecting the main subject in the second and subsequent images of the temporally continuous image;
Subject region determining means for determining a subject region based on the main subject information;
Composition determining means for determining a trimming composition based on the main subject information and the subject region;
With
At least the trimming size and the trimming aspect determined by the composition means are determined for the first image of the temporally continuous image, and the trimming position is the tracking for the second and subsequent images of the temporally continuous image. An image processing apparatus characterized by being determined by detected subject information. The image processing apparatus according to claim 1, wherein the time-sequential images have a predetermined short time interval from one image capture to the next image capture. The distance information acquiring unit for acquiring distance information for each area of the image, and the subject region determining unit determines based on the main subject information and the distance information. Image processing apparatus. The image processing apparatus according to claim 3, wherein the subject area determination unit does not use the distance information for the second and subsequent images of the temporally continuous images. 5. The composition determination unit according to claim 1, wherein the composition determination unit redoes the subject detection, the subject area calculation, and the trimming composition determination when the subject detection or the tracking detection cannot be performed. Image processing device. The image processing apparatus according to claim 1, wherein the composition determination unit determines the composition according to a ratio of a subject size to a trimming size. 7. The image processing apparatus according to claim 1, further comprising trimming size updating means, wherein at least the trimming size is updated when the ratio of the subject size to the trimming size exceeds a predetermined range. . 8. The trimming size update unit makes the predetermined range variable according to at least one of a subject type, a shooting mode, and distance information to the subject. Image processing apparatus.