JP5754308B2 - Electronic camera, image processing apparatus, and image processing program - Google Patents

Description

  The present invention relates to an electronic camera, an image processing apparatus, and an image processing program.

  Conventionally, after a plurality of images are generated and recorded on a recording medium, the image is evaluated from the viewpoint of exposure, contrast, blur, etc., and an image whose evaluation is lower than a predetermined standard is deleted from the above-described recording medium Is known (see, for example, Patent Document 1).

  In the invention of Patent Document 1, image evaluation processing is executed by exposure condition evaluation, contrast evaluation, blur, focus blur evaluation, and the like.

JP 2006-50494 A

  By the way, it is known that the state of the main subject for the photographer or the viewer greatly affects the image evaluation described above. Therefore, in the above-described image evaluation, it is conceivable to detect a main subject region and use it for image evaluation.

  However, if the main subject is a moving subject or a part of the main subject is hidden, such as when there is a physical change in the main subject between multiple images in time series, However, even if the main subject region is detected and image evaluation is performed, the scale in the evaluation varies, so that there is a problem that accurate evaluation cannot be performed.

  The present invention has been made in view of the above-described problem, and an object of the present invention is to suppress scale variation in evaluation even when a main subject changes when evaluating a plurality of images.

  The electronic camera of the present invention includes an imaging unit that images a subject and generates a plurality of temporally continuous images, a first detection unit that detects a main subject region for each of the plurality of images, and the first Based on the plurality of images in which the main subject area is detected by one detection unit, a second detection unit that detects a portion where the main subject areas overlap as a second main subject area, and an image of the main subject area A first evaluation value and a second evaluation value based on the image of the second main subject area are calculated, and the plurality of images are evaluated based on at least one of the first evaluation value and the second evaluation value. And an evaluation unit for performing.

  The evaluation unit performs the evaluation again based on the evaluation value of one of the first evaluation value and the second evaluation value, and then performs the evaluation again based on the other evaluation value. Also good.

  Further, the evaluation unit may perform the evaluation by performing a predetermined weighting on the first evaluation value and the second evaluation value.

  Further, each of the plurality of images includes a recognition unit that performs scene recognition, and the evaluation unit sets at least one of the first evaluation value and the second evaluation value based on a recognition result by the recognition unit. The evaluation method based on the above may be determined.

  Further, the second detection unit extracts feature points in the main subject region for a plurality of images in which the main subject region is detected by the first detection unit, and based on the feature points, After the main subject area is deformed, an overlapping portion may be detected as the second main subject area.

  In addition, when detecting the main subject region for each of the plurality of images, the first detection unit is temporally continuous with the image if the main subject region cannot be detected in an arbitrary image. The main subject area in the arbitrary image may be detected based on the main subject area detected in at least one image.

  Moreover, you may provide the deletion part which deletes one part image from the several image recorded on the recording medium based on the evaluation result by the said evaluation part.

  In addition, when the first detection unit detects the main subject region for each of the plurality of images, and the number of images in which the main subject region cannot be detected continues more than a predetermined number, The detection of the main subject area is stopped, and the evaluation unit may set the first evaluation value relating to a plurality of images in which the main subject area cannot be detected as a predetermined value indicating that the main subject area cannot be detected. .

  Further, the first detection unit detects the main subject region at each of the plurality of images at a time-series end portion of the plurality of temporally continuous images when detecting the main subject region. If there are more than a predetermined number of images that cannot be detected, the detection of the main subject area in those images is stopped, and the evaluation unit performs the first evaluation on a plurality of images in which the main subject area cannot be detected. The value may be a predetermined value indicating that the main subject area cannot be detected.

  The image processing apparatus according to the present invention includes an acquisition unit that acquires a plurality of images as a processing target image, a first detection unit that detects a main subject region for each of the plurality of images acquired by the acquisition unit, A second detection unit that detects, as a second main subject region, a portion where the main subject region overlaps among a plurality of images in which the main subject region has been detected by the first detection unit; and an image of the main subject region And a second evaluation value based on the image of the second main subject region, and based on at least one of the first evaluation value and the second evaluation value, by the acquisition unit An evaluation unit that evaluates the acquired plurality of images, and a selection unit that selects some of the plurality of images acquired by the acquisition unit based on an evaluation result by the evaluation unit. .

  Note that a recording unit that records the plurality of images acquired by the pre-acquisition unit on a recording medium and a part of the images selected by the selection unit among the plurality of images recorded on the recording medium are deleted. A deletion unit, wherein the acquisition unit acquires a plurality of images captured and generated continuously in time as the plurality of images, and the first detection unit is recorded on the recording medium The main subject area may be detected for each of a plurality of images.

  An image processing program of the present invention is an image processing program for realizing image processing on a processing target image by a computer, wherein an acquisition step of acquiring a plurality of images as the processing target image, and the acquisition step For each of the plurality of acquired images, the main subject region overlaps between a first detection step for detecting a main subject region and a plurality of images for which the main subject region has been detected in the first detection step. Calculating a second detection step for detecting a portion as a second main subject region, a first evaluation value based on the image of the main subject region, and a second evaluation value based on the image of the second main subject region; The plurality of images acquired in the acquisition step based on at least one of the first evaluation value and the second evaluation value. Comprising an evaluation step of the evaluation, based on the evaluation result in the evaluation step, among the plurality of images obtained in the obtaining step, a selection step of selecting a portion of the image.

  According to the present invention, when evaluating a plurality of images, even when the main subject changes, it is possible to suppress variation in the scale in the evaluation.

1 is a block diagram illustrating a configuration example of an electronic camera according to an embodiment. 6 is a flowchart showing the operation of the electronic camera in one embodiment. Schematic explaining the flow of processing in one embodiment The figure explaining the detection of the main subject area in one embodiment Another figure explaining detection of a main subject area in one embodiment Another figure explaining detection of a main subject area in one embodiment Another figure explaining detection of the 2nd main subject field in one embodiment

  FIG. 1 is a block diagram illustrating a configuration example of an electronic camera 10 according to an embodiment. The electronic camera 10 includes an imaging lens 11, a shutter 12, an imaging device 13, an AFE 14, an image processing unit 15, a first memory 16, a second memory 17, a recording I / F 18, a CPU 19, and an operation. A unit 20 and a bus 21 are provided. Here, the image processing unit 15, the first memory 16, the second memory 17, the recording I / F 18, and the CPU 19 are respectively connected via a bus 21. Further, the shutter 11, the image sensor 13, the AFE 14, and the operation unit 20 are each connected to the CPU 19.

  The shutter 12 is a mechanical shutter disposed between the photographing lens 11 and the image sensor 13.

  The imaging element 13 is an imaging device that captures an image formed by the imaging lens 11. The image pickup device 13 of one embodiment is configured by, for example, a CMOS solid-state image pickup device that can read an image signal of an arbitrary light receiving device by random access. Note that the output of the image sensor 13 is connected to the AFE 14.

  The AFE 14 is an analog front end circuit that performs analog signal processing on the output of the image sensor 13. The AFE 14 performs correlated double sampling, image signal gain adjustment, and A / D conversion of the image signal. The output of the AFE 14 is connected to the image processing unit 15. The CPU 19 adjusts the imaging sensitivity corresponding to the ISO sensitivity by adjusting the gain of the image signal by the AFE 14.

  The image processing unit 15 performs various types of image processing (color interpolation processing, gradation conversion processing, white balance adjustment, etc.) on the digital image signal output from the AFE 14.

  The first memory 16 temporarily stores image data in the pre-process and post-process of image processing. For example, the first memory 16 is configured by an SDRAM that is a volatile storage medium. The second memory 17 is a non-volatile memory that stores a program executed by the CPU 19 and a data table indicating a correspondence relationship between the S / N ratio and the imaging sensitivity value in each image signal.

  The recording I / F 18 has a connector for connecting a nonvolatile storage medium 22. The recording I / F 18 executes writing / reading of main image data, which will be described later, with respect to the storage medium 22 connected to the connector. The storage medium 22 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 22.

  The CPU 19 is a processor that comprehensively controls the operation of the electronic camera 10. For example, the CPU 19 drives the imaging device 13 in response to a user's imaging instruction input during operation in a shooting mode for shooting a subject, and records the main image with recording in the nonvolatile storage medium 22. An imaging process is executed.

  In addition, the CPU 19 in the shooting mode executes a known automatic exposure calculation prior to capturing the main image, and sets the imaging sensitivity that is one parameter of the imaging conditions. Further, the CPU 19 may set the above-described imaging sensitivity based on a user input.

  The operation unit 20 includes a plurality of switches that accept user operations. The operation unit 20 includes, for example, a release button that receives an instruction to capture a main image, a cross-shaped cursor key, a determination button, and the like.

  The electronic camera 10 includes a so-called best shot mode in which a plurality of images are continuously captured, the generated plurality of images are temporarily recorded, and then an image evaluation is performed to delete an image with a low evaluation. This mode may be set based on a user operation through the operation unit 20 or may be automatically set by the CPU 19. In the best shot mode, the user gives a shooting instruction by operating the release button of the operation unit 20 once.

  Next, an operation example of the electronic camera 10 in the best shot mode will be described with reference to a flowchart of FIG. 2 and a schematic diagram of FIG. The step numbers (S101 to S113) in FIGS. 2 and 3 correspond to each other.

  In step S <b> 101, the CPU 19 determines whether or not a shooting instruction has been given by the user. The CPU 19 waits until a shooting instruction is given by the user via the operation unit 20, and proceeds to step S102 when the shooting instruction is issued.

  In step S <b> 102, the CPU 19 controls each unit to capture the nth image. In the first step S102, n = 1. Hereinafter, an image generated by the n-th shooting is referred to as Img [n]. For example, the image generated by the first shooting is Img [1].

  In step S103, the CPU 19 records the nth image generated by the imaging in step S102 on the storage medium 22 via the recording I / F 18. Note that the CPU 19 may record in the first memory 16 instead of recording in the storage medium 22.

  In step S104, the CPU 19 determines whether or not the processing from step S102 to step S103 has been completed for N sheets. If the CPU 19 determines that N sheets have not been completed, the process proceeds to step S105. If the CPU 19 determines that N sheets have been completed, the process proceeds to step S106 described below. N is a predetermined number of shots (for example, N = 20). This number may be fixed, may be determined automatically by the CPU 19, or may be determined by a user operation via the operation unit 20.

  In step S105, the CPU 19 sets n = n + 1 and returns to step S102. That is, the CPU 19 repeats N times of photographing (step S102) and records N images (step S103).

  In step S <b> 106, the CPU 19 detects the main subject area S [n] for each of the N images recorded on the storage medium 22. Hereinafter, the main subject area detected for the nth image is referred to as a main subject area S [n]. For example, the main subject area detected for the first image is the main subject area S [1]. Note that the detection of the main subject area is performed in the same manner as in the known technique based on, for example, luminance information, focus information, contrast information, edge information, and the like.

  However, if the main subject area S [n] cannot be properly detected, the CPU 19 performs the following processing.

  (1) A case where the main subject region S [X] cannot be detected in an arbitrary image Img [X].

  For example, as shown in FIG. 4, when the main subject region S [X] cannot be detected in the image Img [X], the CPU 19 detects it in at least one image temporally continuous with the image Img [X]. The main subject region S [X] is detected by performing interpolation processing based on the main subject region (for example, the main subject region S [X-1] and the main subject region S [X + 1]). At this time, as at least one image temporally continuous with the image Img [X], the main subject region (for example, for example, detected in the temporally continuous image on the front side or the continuous image on the back side) Based on the main subject region S [X−1] or the main subject region S [X + 1]), the main subject region S [X] may be detected by performing interpolation processing from only one direction.

  Further, the number of images before and after being used for interpolation may be any number. For example, the CPU 19 may perform an interpolation process based on two images of the main subject region S [X-1] and the main subject region S [X-2], or the main subject region S [X + 1] and the main subject region S [X + 1] Interpolation processing may be performed based on two images of the main subject area S [X + 2]. Further, the CPU 19 is based on, for example, four images of a main subject region S [X-1], a main subject region S [X-2], a main subject region S [X + 1], and a main subject region S [X + 2]. Interpolation processing may be performed.

  A method other than interpolation may be used.

  (2) A case where a predetermined number (for example, three) or more of images in which the main subject area S [n] cannot be detected continues.

  For example, as shown in FIG. 5, when the main subject area S [n] cannot be detected in three images from the images Img [X−1] to Img [X + 1], the CPU 19 determines that the main subject is a frame of the target image. It is determined that the subject has gone out, and detection of the main subject region S [n] in these three images is stopped.

  (3) Among N images, a predetermined number (for example, three) or more of images in which the main subject region S [n] cannot be detected continue at a time-series end portion (first or last portion). If.

  For example, as shown in FIG. 6, the main subject region S [n] in the three images from the images Img [1] to Img [3], which are the end portions on the first side in time series, among the N images. Cannot be detected, the CPU 19 determines that the main subject has moved out of the frame of the target image as in (2) described above, and stops detecting the main subject region S [n] in these three images.

  (4) The main subject area S [n] cannot be detected in all N images.

  For each of the N images, the CPU 19 sets the entire screen as the main subject area S [n], or sets the fixed area (for example, the center area) as the main subject area S [n].

  In the above (1) to (3), in order to simplify the processing, when the main subject region S [X] cannot be detected in an arbitrary image Img [X], the main subject region S [n] is selected. Of the detected images, the main subject region S [n] in the image closest in time to the arbitrary image Img [X] may be set as the main subject region S [X] in the arbitrary image Img [X].

  In the above (1) to (4), the case where the main subject region S [n] cannot be appropriately detected has been described as an example. However, the detected main subject region S [n] The same processing may be performed when there is a doubt about reliability. For example, when the position and size of the main subject region S [X] detected in an arbitrary image Img [X] and the main subject region S [n] detected in consecutive images are significantly different. Has a high possibility of erroneous detection and the reliability of the main subject area S [X] is doubtful. In such a case, the CPU 19 may perform the same processing as (1) to (4) described above.

  In step S107, the CPU 19 calculates a first evaluation value E [n] for each of the N images recorded in the storage medium 22 based on the main subject area S [n] detected in step S104. Hereinafter, the evaluation value calculated for the nth image is referred to as a first evaluation value E [n]. For example, the evaluation value calculated for the first image is the first evaluation value E [1]. The evaluation value is calculated in the same manner as in the known technique based on, for example, luminance information, focus information, contrast information, edge information, and the pixel value in the main subject region S [n].

  However, as described in (2) and (3) of step S104, the CPU 19 detects the first evaluation value E [n] related to the image for which the detection of the main subject area has been stopped. It is set to a predetermined value indicating that it cannot be performed. The value indicating that the main subject region cannot be detected may be any value, for example, the minimum value that can be taken by the first evaluation value E [n]. Also, a relatively low value among the values that can be taken by the first evaluation value E [n] may be a value indicating that the main subject region cannot be detected.

  In the detection of the main subject area S [n] described in step S106, when the main subject area S [n] is detected by performing an interpolation process or the like, the main subject area S is not performed without performing the interpolation process. The evaluation value E [n] may be adjusted so that the evaluation value E [n] is lower than when [n] is detected.

  In step S108, the CPU 19 detects the second main subject area Sc. The second main subject area Sc corresponds to a portion where the main subject area S [N] overlaps the main subject area S [1] detected in each of the N images described in step S106. For example, when N = 20, the portion where the main subject region S [20] overlaps the main subject region S [1] detected in step S106 is the second main subject region Sc.

  FIG. 7 illustrates the second main subject area Sc. In FIG. 7, for the sake of simplicity, the second main subject region Sc is illustrated as a portion where the main subject region S [X−1] and the main subject region S [X + 1] overlap. Further, in FIG. 7, the second main subject region Sc is shown in an enlarged manner for easy understanding. As shown in FIG. 7, the main subject region S [X−1] to the main subject region S [X + 1] have different shapes according to the movement of the main subject. For example, in the main subject region S [X + 1], both the person portion that is the original main subject and the background yacht portion are detected as the main subject region. On the other hand, the second main subject area Sc is an overlapping portion of the main subject area S [X−1] to the main subject area S [X + 1], and therefore includes only the portion of the person who is the original main subject.

  Note that the CPU 19 performs, for example, an alignment process on the main subject area S [1] to the main subject area S [N] detected in step S106, and then performs an overlay process to obtain a second main subject area Sc. Is detected. At this time, the CPU 19 extracts a feature point in the main subject region with respect to the main subject region S [N] from the main subject region S [1], and based on the feature point, the main subject region S [1] extracts the main subject. The overlapping portions may be detected as the second main subject region Sc after each of the regions S [N] is deformed. As the deformation process, for example, a rotation process, an enlargement / reduction process, a distortion process (non-linear distortion process), and the like can be considered as in the known technique.

  Further, the CPU 19 may detect not the entire main subject region S [1] to the main subject region S [N] but a portion where the main subject regions based on some images overlap as the second main subject region Sc. good. For example, a portion of the N images described in step S106 where the main subject region S [n] detected in every other image overlaps may be detected as the second main subject region Sc. Further, the main subject region S [1] to the main subject region S [N] may be relatively compared, and those having a high degree of similarity may be excluded from the target for detecting the overlapping portion. In any case, the process of detecting the second main subject region Sc can be simplified by reducing the number of main subject regions S [n] that are targets for detecting overlapping portions.

In addition, when the CPU 19 cannot appropriately detect the second main subject region Sc, the CPU 19 may perform the following processing.
When the portion where the main subject region S [N] overlaps from the main subject region S [1] cannot be detected appropriately, the portion where the main subject regions overlap based on all the images (N images described in step S106) Instead, a portion where the main subject areas overlap based on some images (for example, about 80% of the N images described in step S106) is detected as the second main subject area Sc. May be.
If the portion where the main subject region S [N] overlaps the main subject region S [1] cannot be detected appropriately, the N images described in step S106 are time-series or the main subject region S [N]. It is divided into a plurality of groups according to the change amount of Then, for each group, a portion where the main subject regions based on the images included in the group overlap may be detected as the second main subject region Sc. In this case, the second main subject region Sc having a different shape for each of the plurality of groups described above is detected. Furthermore, as described above, one second main subject region Sc may be detected by combining a plurality of second main subject regions Sc detected for each group (addition, selection, etc.).
When the area of the second main subject area Sc is very small compared to the areas of the main subject area S [1] to the main subject area S [N], the detection of the second main subject area Sc is stopped. Alternatively, instead of a portion where the main subject regions overlap based on all the images (N images described in step S106), some of the images (for example, among the N images described in step S106) , A portion where the main subject areas overlap based on about 80% of the number of images) may be detected as the second main subject area Sc, or the above-described grouping may be performed. When the detection of the second main subject region Sc is stopped, the CPU 19 calculates the second evaluation value Ec [n for each image when calculating the second evaluation value Ec [n] in step S109 described later. ] Is a predetermined value indicating that the second main subject region cannot be detected (see the first evaluation value E [n] in step S107).

  In step S109, the CPU 19 calculates a second evaluation value Ec [n] for each of the N images recorded in the storage medium 22 based on the second main subject area Sc detected in step S108. Hereinafter, the second evaluation value calculated for the nth image is referred to as a second evaluation value Ec [n]. For example, the second evaluation value calculated for the first image is the second evaluation value Ec [1]. The second evaluation value is calculated in the same manner as the first evaluation value E [n] described in step S107, for example, luminance information, focus information, contrast information, edge information, etc., and pixels in the second main subject region Sc. Based on the value, it is carried out in the same manner as in the known art.

  In step S110, the CPU 19 performs scene recognition. The CPU 19 performs scene recognition based on the N images recorded on the storage medium 22 in the same manner as in the known technique. This scene recognition is for estimating whether or not the main subject changes, and how it changes. For example, in a scene such as a portrait, it can be estimated that the main subject has little change. Further, for example, in the scene illustrated in FIG. 7, it can be estimated that the main subject itself moves while changing.

  In step S111, the CPU 19 for each of the N images recorded on the storage medium 22, the first evaluation value E [n] calculated in step S107 and the second evaluation value Ec [n] calculated in step S109. The image is evaluated based on at least one of the above.

  There are the following methods for image evaluation.

  (1) After performing the evaluation based on the first evaluation value E [n], the evaluation is performed based on the second evaluation value Ec [n].

  By performing the evaluation in this order, an image with good reflection in the main subject region is selected based on the first evaluation value E [n] suitable for absolute evaluation, and further, the second evaluation suitable for relative evaluation. Based on the value Ec [n], an image with relatively good reflection can be selected.

  This selection method is useful in a scene where there is little change in the main subject itself, such as a portrait.

  (2) After evaluating based on the second evaluation value Ec [n], evaluation is performed based on the first evaluation value E [n].

  By performing the evaluation in this order, an image with relatively good reflection is selected based on the second evaluation value Ec [n] suitable for relative evaluation, and further, the first evaluation value suitable for absolute evaluation Based on E [n], an image with good reflection in the main subject region can be selected.

  This selection method is useful in a scene in which the main subject itself moves while changing as illustrated in FIG.

  (3) Evaluation is performed after predetermined weighting is applied to the first evaluation value E [n] and the second evaluation value Ec [n].

  By performing the evaluation by such a method, the first evaluation value E [n] suitable for the absolute evaluation and the second evaluation value Ec [n] suitable for the relative evaluation are used in a well-balanced manner. be able to. In a scene such as a portrait where there is little change in the main subject itself, the weight of the first evaluation value E [n] suitable for absolute evaluation is increased, and the main subject itself as illustrated in FIG. 7 changes. In a moving scene, the weight of the second evaluation value Ec [n] suitable for relative evaluation is preferably increased.

  (4) Evaluation is performed based on either the first evaluation value E [n] or the second evaluation value Ec [n].

  By performing the evaluation based only on the first evaluation value E [n], it is possible to evaluate the image based on the absolute evaluation. By performing the evaluation based only on the second evaluation value Ec [n], the image based on the relative evaluation can be obtained. Can be evaluated.

  The CPU 19 performs evaluation by any of the methods (1) to (4) described above, and calculates an evaluation value. The evaluation value is calculated in the same manner as in the known technique. It should be noted that which of the above methods (1) to (4) is used may be selected by the CPU 19 or may be selected by a user operation via the operation unit 20. When selecting automatically by the CPU 19, the CPU 19 performs selection based on the result of the scene recognition performed in step S110, for example.

  In step S112, the CPU 19 determines an image to be deleted. Based on the evaluation value calculated in step S111, the CPU 19 selects a predetermined number (for example, 15 images) of images from the lowest evaluation as images to be deleted. By making such a selection, an image with a high evaluation can be selectively left. The number of images to be deleted may be fixed, may be determined automatically by the CPU 19, or may be determined by a user operation via the operation unit 20. Also good.

  In step S113, the CPU 19 deletes the image determined in step S112 from the recording medium 22, and ends the series of processes.

  In step S103, when an image generated by shooting is recorded in the first memory 16, the CPU 19 deletes the image from the first memory and stores the remaining image in the storage medium 22 via the recording I / F 18. Record.

  As described above, according to the present embodiment, the subject is imaged, a plurality of temporally continuous images are generated, the main subject region is detected for each of the generated plurality of images, and the main subject is detected. A portion where the main subject region overlaps is detected as a second main subject region for the plurality of images for which the region has been detected. Furthermore. A first evaluation value based on the image of the main subject area and a second evaluation value based on the image of the second main subject area are calculated, and based on at least one of the first evaluation value and the second evaluation value, the recording medium A plurality of images recorded in the image are evaluated. Therefore, even when the main subject changes when evaluating a plurality of images, it is possible to suppress variation in the scale in the evaluation.

  In particular, according to the present embodiment, when a main subject cannot be detected from some images in a series of images taken continuously in time series, or when the main subject is a moving subject and the main subject itself is physically When a moving subject rotates (when a moving subject rotates, a part of the main subject is out of frame, or a protrusion amount changes, a passerby crosses in front of the main subject and a part of the main subject is hidden) Etc.) can be evaluated on substantially the same scale. As a result of such evaluation, a suitable image can be selected and deleted, so that an optimal image can be left in the best shot mode.

  In the present embodiment, after taking N images (step S102) and recording (step S103), the main subject region S [n] is detected (step S106) and the first evaluation value E [n] is calculated. Although an example of performing (Step S107) has been shown, the present invention is not limited to this example. For example, immediately after taking N images (step S102), detection of the main subject region S [n] (step S106) and calculation of the first evaluation value E [n] (step S107) are started. good.

  In the present embodiment, as an example of determining that the main subject has moved outside the frame of the target image in the detection of the main subject region S [n] (step S106), the detection target of the main subject region S [n] In this example, the determination is based on the same number of images (3) when the image is at the end of the time series and when the image is not at the end of the time series. However, the CPU 19 determines the reference number for determining that the main subject has moved out of the frame of the target image when the image to be detected in the main subject region S [n] is at the end of the time series and It may be changed depending on whether it is not in the department. For example, the CPU 19 determines that the main subject has moved out of the frame of the target image when two images in which the main subject region S [n] cannot be detected are consecutive at the end of the time series, and the main subject region S [n ] Is not at the end of the time series, it is determined that the main subject has moved out of the frame of the target image when three images in which the main subject region S [n] cannot be detected are consecutive. Also good.

  Further, for example, the CPU 19 determines that the main subject has moved out of the frame of the target image when four images in which the main subject region S [n] cannot be detected are consecutive at the end of the time series, and the main subject region S When the detection target image of [n] is not at the end of the time series, it is determined that the main subject has moved out of the frame of the target image when three images in which the main subject region S [n] cannot be detected are consecutive. May be.

  Note that the above-mentioned “predetermined number as a reference for determining that the main subject has moved out of the frame of the target image” may be set based on a user operation via the operation unit 20 or automatically by the CPU 19. It may be set. When the CPU 19 automatically sets, an appropriate number may be set according to the type of shooting mode and shooting conditions set at the time of imaging.

  In the present embodiment, an example in which scene recognition is performed in step S110 has been described. However, scene recognition is not necessarily performed.

  In the present embodiment, an example in which the series of processes shown in the flowchart of FIG. 2 is performed in the best shot mode is shown, but the present invention is not limited to this example. For example, the present invention can be similarly applied when generating a plurality of images in the continuous shooting mode. Moreover, you may perform the process after step S106 for the some image selected by the user.

  In the above-described embodiment, the example in which the technique of the present invention is implemented in the electronic camera 10 illustrated in FIG. 1 has been described. However, the present invention is not limited to this. For example, the present invention can be similarly applied to an electronic camera having a configuration other than the electronic camera 10 shown in FIG.

  Further, the image processing apparatus described in the above embodiment may be realized as software by a computer and an image processing program. In this case, what is necessary is just to make it the structure which implement | achieves part or all of the process after step S106 demonstrated with the flowchart of FIG. 2 with a computer.

  Further, in addition to the computer described above, the present invention can be similarly applied to an electronic device (for example, a photo viewer, a digital photo frame, a photo printing apparatus, etc.) having a digital image reproduction display function and a retouch function. it can. In addition, the electronic camera 10 described in the above-described embodiment may be mounted as a camera module such as a mobile phone terminal.

  In the above-described embodiment, an example in which each process is realized by software has been described. However, it is a matter of course that each process may be realized by hardware using an ASIC or the like.

DESCRIPTION OF SYMBOLS 10 ... Electronic camera, 13 ... Image sensor, 14 ... AFE, 16 ... 1st memory, 17 ... 2nd memory, 19 ... CPU, 22 ... Storage medium

Claims (12)

An imaging unit that images a subject and generates a plurality of temporally continuous images;
A first detector for detecting a main subject region for each of the plurality of images;
A second detection unit that detects, as a second main subject region, a portion where the main subject region overlaps a plurality of images in which the main subject region is detected by the first detection unit;
A first evaluation value based on the image of the main subject area and a second evaluation value based on the image of the second main subject area are calculated, and based on at least one of the first evaluation value and the second evaluation value An electronic camera comprising: an evaluation unit that evaluates the plurality of images. The electronic camera according to claim 1,
The evaluation unit performs the evaluation again based on the other evaluation value after performing the evaluation based on one of the first evaluation value and the second evaluation value. And an electronic camera. The electronic camera according to claim 1,
The evaluation unit performs the evaluation by performing a predetermined weighting on the first evaluation value and the second evaluation value. The electronic camera according to claim 1,
A recognition unit that performs scene recognition for each of the plurality of images,
The evaluation unit determines the evaluation method based on at least one of the first evaluation value and the second evaluation value based on a recognition result by the recognition unit. The electronic camera according to claim 1,
The second detection unit extracts feature points in the main subject region for a plurality of images in which the main subject region has been detected by the first detection unit, and based on the feature points, the main subject An electronic camera, wherein an overlapping portion is detected as the second main subject area after the area is deformed. The electronic camera according to claim 1,
When detecting the main subject region for each of the plurality of images, if the main subject region cannot be detected in an arbitrary image, the first detection unit is at least one temporally continuous with the image. An electronic camera, wherein the main subject region in the arbitrary image is detected based on the main subject region detected in a single image. The electronic camera according to any one of claims 1 to 6,
An electronic camera comprising: a deletion unit that deletes some of the plurality of images recorded on the recording medium based on the evaluation result by the evaluation unit. The electronic camera according to claim 6 or 7,
When detecting the main subject region for each of the plurality of images, the first detection unit, when a predetermined number or more of images in which the main subject region cannot be detected continues, Stop detecting the subject area,
The electronic camera according to claim 1, wherein the evaluation unit sets the first evaluation value relating to a plurality of images in which the main subject area cannot be detected as a predetermined value indicating that the main subject area cannot be detected. The electronic camera according to claim 6 or 7,
When detecting the main subject region for each of the plurality of images, the first detection unit detects the main subject region at a time-series end portion of the plurality of temporally continuous images. If there are more than a predetermined number of images that cannot be detected, stop the detection of the main subject area in those images,
The electronic camera according to claim 1, wherein the evaluation unit sets the first evaluation value relating to a plurality of images in which the main subject area cannot be detected as a predetermined value indicating that the main subject area cannot be detected. An acquisition unit that acquires a plurality of images as a processing target image;
A first detection unit that detects a main subject region for each of the plurality of images acquired by the acquisition unit;
A second detection unit that detects, as a second main subject region, a portion where the main subject region overlaps a plurality of images in which the main subject region is detected by the first detection unit;
A first evaluation value based on the image of the main subject area and a second evaluation value based on the image of the second main subject area are calculated, and based on at least one of the first evaluation value and the second evaluation value An evaluation unit that evaluates the plurality of images acquired by the acquisition unit;
An image processing apparatus comprising: a selection unit that selects a part of the plurality of images acquired by the acquisition unit based on an evaluation result by the evaluation unit. The image processing apparatus according to claim 10.
A recording unit that records the plurality of images acquired by the pre-acquisition unit on a recording medium;
A deletion unit that deletes the part of the images selected by the selection unit among the plurality of images recorded on the recording medium;
The acquisition unit acquires a plurality of images captured and generated continuously in time as the plurality of images,
The image processing apparatus, wherein the first detection unit detects the main subject region for each of the plurality of images recorded on the recording medium. An image processing program for realizing image processing on an image to be processed by a computer,
An acquisition step of acquiring a plurality of images as the processing target image;
A first detection step of detecting a main subject region for each of the plurality of images acquired in the acquisition step;
A second detection step of detecting, as a second main subject region, a portion where the main subject region overlaps a plurality of images in which the main subject region has been detected in the first detection step;
A first evaluation value based on the image of the main subject area and a second evaluation value based on the image of the second main subject area are calculated, and based on at least one of the first evaluation value and the second evaluation value An evaluation step for evaluating the plurality of images acquired in the acquisition step;
An image processing program comprising: a selection step of selecting some of the plurality of images acquired in the acquisition step based on an evaluation result in the evaluation step.

Images