JP2013013023A - Electronic camera, image processor and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform suitable evaluation even if a main subject changes when a plurality of images are evaluated.SOLUTION: An electronic camera includes: an imaging section for imaging a subject and generating a plurality of continuous images; a detection section for detecting a main subject region on each of the plurality of images generated by the imaging section and detecting the main subject region in an arbitrary image on the basis of the main subject region detected in at least one image which temporally continues to the arbitrary image when the main subject region cannot be detected in the arbitrary image; and an evaluation section for calculating an evaluation value based on the image of the main subject region and evaluating the plurality of images generated by the imaging section on the basis of the calculated evaluation value.

Description

  The present invention relates to an electronic camera, an image processing apparatus, and an image processing program.

  Conventionally, after a plurality of images are generated and recorded on a recording medium, the image is evaluated from the viewpoint of exposure, contrast, blur, etc., and an image whose evaluation is lower than a predetermined standard is deleted from the above-described recording medium Is known (see, for example, Patent Document 1).

  In the invention of Patent Document 1, image evaluation processing is executed by exposure condition evaluation, contrast evaluation, blur, focus blur evaluation, and the like.

JP 2006-50494 A

  By the way, it is known that the state of the main subject for the photographer or the viewer greatly affects the image evaluation described above. Therefore, in the above-described image evaluation, it is conceivable to detect a main subject region and use it for image evaluation.

  However, when the main subject is a moving subject or when a part of the main subject is hidden, the main subject region is detected in some of the images that are continuous in time series. Problems may arise. In such a case, in some of the images described above, there is a problem that the main subject region cannot be detected and image evaluation cannot be performed, and a suitable evaluation cannot be performed.

  The present invention has been made in view of the above problems, and an object of the present invention is to perform a suitable evaluation even when a main subject changes when evaluating a plurality of images.

  The electronic camera of the present invention captures a subject and generates a plurality of continuous images, and performs detection processing of a main subject region for each of the plurality of images generated by the imaging unit. If the main subject area cannot be detected in the image of the image, the main subject area in the arbitrary image is detected based on the main subject area detected in at least one image temporally continuous with the image. A detection unit that performs processing; and an evaluation unit that calculates an evaluation value based on the image of the main subject region and evaluates the plurality of images generated by the imaging unit based on the calculated evaluation value.

  Note that the detection unit performs detection processing of the main subject region for each of the plurality of images generated by the imaging unit, and when the number of images in which the main subject region cannot be detected continues for a predetermined number or more. Stops the detection processing of the main subject region in those images, and the evaluation unit indicates that the evaluation value regarding a plurality of images in which the main subject region cannot be detected cannot be detected by the main subject region. The value of

  In addition, when the detection process of the main subject region is performed on each of the plurality of images generated by the imaging unit, the detection unit includes time-series end portions of the plurality of temporally continuous images. If the predetermined number of images in which the main subject area cannot be detected continues, the main subject area detection process in those images is stopped, and the evaluation unit detects a plurality of images in which the main subject area cannot be detected. The evaluation value relating to the image may be a predetermined value indicating that the main subject area cannot be detected.

  In addition, a recording unit that records the plurality of images generated by the imaging unit on a recording medium, and a part of the plurality of images recorded on the recording medium based on an evaluation result by the evaluation unit You may provide the deletion part to delete.

  The image processing apparatus according to the present invention performs a main subject region detection process on each of the acquisition unit that acquires a plurality of images as the processing target image, and the plurality of images acquired by the acquisition unit. When the main subject area cannot be detected in the image, the main subject area detection process in the arbitrary image is performed based on the main subject area detected in at least one image temporally continuous with the image. A evaluation unit that calculates an evaluation value based on the image of the main subject region, and evaluates the plurality of images acquired by the acquisition unit based on the calculated evaluation value, and the evaluation unit And a selection unit that selects a part of the plurality of images acquired by the acquisition unit based on the evaluation result by the acquisition unit.

  Note that a recording unit that records the plurality of images acquired by the pre-acquisition unit on a recording medium and a part of the images selected by the selection unit among the plurality of images recorded on the recording medium are deleted. A deletion unit, wherein the acquisition unit acquires a plurality of images captured and generated continuously as the plurality of images, and the detection unit is configured to each of the plurality of images recorded on the recording medium. For the above, the main subject area detection process may be performed.

  An image processing program of the present invention is an image processing program for realizing image processing on a processing target image by a computer, wherein an acquisition step of acquiring a plurality of images as the processing target image, and the acquisition step For each of the plurality of acquired images, the main subject region is detected, and if the main subject region cannot be detected in an arbitrary image, it is detected in at least one image temporally continuous with the image. A detection step for performing detection processing of the main subject region in the arbitrary image based on the main subject region that has been performed, an evaluation value based on the image of the main subject region is calculated, and based on the calculated evaluation value An evaluation step for evaluating the plurality of images acquired in the acquisition step; Based on the evaluation result in the evaluation step, among the plurality of images acquired in the acquiring step, and a selection step of selecting a portion of the image.

  According to the present invention, when evaluating a plurality of images, a suitable evaluation can be performed even when the main subject changes.

1 is a block diagram illustrating a configuration example of an electronic camera according to an embodiment. 6 is a flowchart showing the operation of the electronic camera in one embodiment. Schematic explaining the flow of processing in one embodiment The figure explaining the detection of the main subject area in one embodiment Another figure explaining detection of a main subject area in one embodiment Another figure explaining detection of a main subject area in one embodiment

  FIG. 1 is a block diagram illustrating a configuration example of an electronic camera 10 according to an embodiment. The electronic camera 10 includes an imaging lens 11, a shutter 12, an imaging device 13, an AFE 14, an image processing unit 15, a first memory 16, a second memory 17, a recording I / F 18, a CPU 19, and an operation. A unit 20 and a bus 21 are provided. Here, the image processing unit 15, the first memory 16, the second memory 17, the recording I / F 18, and the CPU 19 are respectively connected via a bus 21. Further, the shutter 11, the image sensor 13, the AFE 14, and the operation unit 20 are each connected to the CPU 19.

  The shutter 12 is a mechanical shutter disposed between the photographing lens 11 and the image sensor 13.

  The imaging element 13 is an imaging device that captures an image formed by the imaging lens 11. The image pickup device 13 of one embodiment is configured by, for example, a CMOS solid-state image pickup device that can read an image signal of an arbitrary light receiving device by random access. Note that the output of the image sensor 13 is connected to the AFE 14.

  The AFE 14 is an analog front end circuit that performs analog signal processing on the output of the image sensor 13. The AFE 14 performs correlated double sampling, image signal gain adjustment, and A / D conversion of the image signal. The output of the AFE 14 is connected to the image processing unit 15. The CPU 19 adjusts the imaging sensitivity corresponding to the ISO sensitivity by adjusting the gain of the image signal by the AFE 14.

  The image processing unit 15 performs various types of image processing (color interpolation processing, gradation conversion processing, white balance adjustment, etc.) on the digital image signal output from the AFE 14.

  The first memory 16 temporarily stores image data in the pre-process and post-process of image processing. For example, the first memory 16 is configured by an SDRAM that is a volatile storage medium. The second memory 17 is a non-volatile memory that stores a program executed by the CPU 19 and a data table indicating a correspondence relationship between the S / N ratio and the imaging sensitivity value in each image signal.

  The recording I / F 18 has a connector for connecting a nonvolatile storage medium 22. The recording I / F 18 executes writing / reading of main image data, which will be described later, with respect to the storage medium 22 connected to the connector. The storage medium 22 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 22.

  The CPU 19 is a processor that comprehensively controls the operation of the electronic camera 10. For example, the CPU 19 drives the imaging device 13 in response to a user's imaging instruction input during operation in a shooting mode for shooting a subject, and records the main image with recording in the nonvolatile storage medium 22. An imaging process is executed.

  In addition, the CPU 19 in the shooting mode executes a known automatic exposure calculation prior to capturing the main image, and sets the imaging sensitivity that is one parameter of the imaging conditions. Further, the CPU 19 may set the above-described imaging sensitivity based on a user input.

  The operation unit 20 includes a plurality of switches that accept user operations. The operation unit 20 includes, for example, a release button that receives an instruction to capture a main image, a cross-shaped cursor key, a determination button, and the like.

  The electronic camera 10 includes a so-called best shot mode in which a plurality of images are continuously captured, the generated plurality of images are temporarily recorded, and then an image evaluation is performed to delete an image with a low evaluation. This mode may be set based on a user operation through the operation unit 20 or may be automatically set by the CPU 19. In the best shot mode, the user gives a shooting instruction by operating the release button of the operation unit 20 once.

  Next, an operation example of the electronic camera 10 in the best shot mode will be described with reference to a flowchart of FIG. 2 and a schematic diagram of FIG. The step numbers (S101 to S110) in FIGS. 2 and 3 correspond to each other.

  In step S <b> 101, the CPU 19 determines whether or not a shooting instruction has been given by the user. The CPU 19 waits until a shooting instruction is given by the user via the operation unit 20, and proceeds to step S102 when the shooting instruction is issued.

  In step S <b> 102, the CPU 19 controls each unit to capture the nth image. In the first step S102, n = 1. Hereinafter, an image generated by the n-th shooting is referred to as Img [n]. For example, the image generated by the first shooting is Img [1].

  In step S103, the CPU 19 records the nth image generated by the imaging in step S102 on the storage medium 22 via the recording I / F 18. Note that the CPU 19 may record in the first memory 16 instead of recording in the storage medium 22.

  In step S104, the CPU 19 determines whether or not the processing from step S102 to step S103 has been completed for N sheets. If the CPU 19 determines that N sheets have not been completed, the process proceeds to step S105. If the CPU 19 determines that N sheets have been completed, the process proceeds to step S106 described below. N is a predetermined number of shots (for example, N = 20). This number may be fixed, may be determined automatically by the CPU 19, or may be determined by a user operation via the operation unit 20.

  In step S105, the CPU 19 sets n = n + 1 and returns to step S102. That is, the CPU 19 repeats N times of photographing (step S102) and records N images (step S103).

  In step S <b> 106, the CPU 19 detects the main subject area S [n] for each of the N images recorded on the storage medium 22. Hereinafter, the main subject area detected for the nth image is referred to as a main subject area S [n]. For example, the main subject area detected for the first image is the main subject area S [1]. Note that the detection of the main subject area is performed in the same manner as in the known technique based on, for example, luminance information, focus information, contrast information, edge information, and the like.

  However, if the main subject area S [n] cannot be properly detected, the CPU 19 performs the following processing.

  (1) A case where the main subject region S [X] cannot be detected in an arbitrary image Img [X].

  For example, as shown in FIG. 4, when the main subject region S [X] cannot be detected in the image Img [X], the CPU 19 detects it in at least one image temporally continuous with the image Img [X]. The main subject region S [X] is detected by performing interpolation processing based on the main subject region (for example, the main subject region S [X-1] and the main subject region S [X + 1]). At this time, as at least one image temporally continuous with the image Img [X], the main subject region (for example, for example, detected in the temporally continuous image on the front side or the continuous image on the back side) Based on the main subject region S [X−1] or the main subject region S [X + 1]), the main subject region S [X] may be detected by performing interpolation processing from only one direction.

  Further, the number of images before and after being used for interpolation may be any number. For example, the CPU 19 may perform an interpolation process based on two images of the main subject region S [X-1] and the main subject region S [X-2], or the main subject region S [X + 1] and the main subject region S [X + 1] Interpolation processing may be performed based on two images of the main subject area S [X + 2]. Further, the CPU 19 is based on, for example, four images of a main subject region S [X-1], a main subject region S [X-2], a main subject region S [X + 1], and a main subject region S [X + 2]. Interpolation processing may be performed.

  A method other than interpolation may be used.

  (2) A case where a predetermined number (for example, three) or more of images in which the main subject area S [n] cannot be detected continues.

  For example, as shown in FIG. 5, when the main subject area S [n] cannot be detected in three images from the images Img [X−1] to Img [X + 1], the CPU 19 determines that the main subject is a frame of the target image. It is determined that the subject has gone out, and detection of the main subject region S [n] in these three images is stopped.

  (3) Among N images, a predetermined number (for example, three) or more of images in which the main subject region S [n] cannot be detected continue at a time-series end portion (first or last portion). If.

  For example, as shown in FIG. 6, the main subject region S [n] in the three images from the images Img [1] to Img [3], which are the end portions on the first side in time series, among the N images. Cannot be detected, the CPU 19 determines that the main subject has moved out of the frame of the target image, and stops detecting the main subject region S [n] in these three images, as in (2) above. .

  (4) The main subject area S [n] cannot be detected in all N images.

  For each of the N images, the CPU 19 sets the entire screen as the main subject area S [n], or sets the fixed area (for example, the center area) as the main subject area S [n].

  In the above (1) to (3), in order to simplify the processing, when the main subject region S [X] cannot be detected in an arbitrary image Img [X], the main subject region S [n] is selected. Of the detected images, the main subject region S [n] in the image closest in time to the arbitrary image Img [X] may be set as the main subject region S [X] in the arbitrary image Img [X].

  In the above (1) to (4), the case where the main subject region S [n] cannot be appropriately detected has been described as an example. However, the detected main subject region S [n] The same processing may be performed when there is a doubt about reliability. For example, when the position and size of the main subject region S [X] detected in an arbitrary image Img [X] and the main subject region S [n] detected in consecutive images are significantly different. Has a high possibility of erroneous detection and the reliability of the main subject region S [X] is doubtful. In such a case, the CPU 19 may perform the same processing as (1) to (4) described above.

  In step S107, the CPU 19 calculates an evaluation value E [n] for each of the N images recorded in the storage medium 22 based on the main subject area S [n] detected in step S104. Hereinafter, the evaluation value calculated for the nth image is referred to as an evaluation value E [n]. For example, the evaluation value calculated for the first image is the evaluation value E [1]. The evaluation value is calculated in the same manner as in the known technique based on, for example, luminance information, focus information, contrast information, edge information, and the pixel value in the main subject region S [n].

  However, as described in (2) and (3) of step S104, the CPU 19 cannot detect the evaluation value E [n] relating to the image for which the detection of the main subject area is stopped from the main subject area. Is a predetermined value. The value indicating that the main subject area cannot be detected may be any value, for example, the minimum value that can be taken by the evaluation value E [n]. Also, a relatively low value among the possible values of the evaluation value E [n] may be a value indicating that the main subject area cannot be detected.

  In the detection of the main subject area S [n] described in step S106, when the main subject area S [n] is detected by performing an interpolation process or the like, the main subject area S is not performed without performing the interpolation process. The evaluation value E [n] may be adjusted so that the evaluation value E [n] is lower than when [n] is detected.

  In step S108, the CPU 19 evaluates the image for each of the N images recorded in the storage medium 22 based on the evaluation value E [n] calculated in step S107, and calculates an evaluation value. The evaluation value is calculated in the same manner as in the known technique.

  In step S109, the CPU 19 determines an image to be deleted. Based on the evaluation value calculated in step S108, the CPU 19 selects a predetermined number (for example, 15) of images from the lowest evaluation as images to be deleted. By making such a selection, an image with a high evaluation can be selectively left. The number of images to be deleted may be fixed, may be determined automatically by the CPU 19, or may be determined by a user operation via the operation unit 20. Also good.

  In step S110, the CPU 19 deletes the image determined in step S109 from the recording medium 22, and ends the series of processes.

  In step S103, when an image generated by shooting is recorded in the first memory 16, the CPU 19 deletes the image from the first memory and stores the remaining image in the storage medium 22 via the recording I / F 18. Record.

  As described above, according to the present embodiment, a subject is imaged, a plurality of temporally continuous images are generated, and the generated plurality of images are recorded on a recording medium. Then, for each of the plurality of images recorded on the recording medium, the main subject area is detected, and if the main subject area cannot be detected in an arbitrary image, at least one image that is temporally continuous with the image. Based on the main subject area detected in step 1, the main subject area in the above-described arbitrary image is detected. Further, an evaluation value based on the image of the main subject area is calculated, a plurality of images recorded on the recording medium are evaluated based on the calculated evaluation value, and a plurality of images recorded on the recording medium are calculated based on the evaluation result. Delete some of the images. Therefore, when evaluating a plurality of images, a favorable evaluation can be performed even when the main subject changes. As a result of such evaluation, a suitable image can be selected and deleted, so that an optimal image can be left in the best shot mode.

  In the present embodiment, after taking N images (step S102) and recording (step S103), detection of the main subject region S [n] (step S106) and calculation of the evaluation value E [n] (steps) Although an example of performing S107) has been shown, the present invention is not limited to this example. For example, detection of the main subject region S [n] (step S106) and calculation of the evaluation value E [n] (step S107) may be started immediately after taking N images (step S102).

  In the present embodiment, as an example of determining that the main subject has moved outside the frame of the target image in the detection of the main subject region S [n] (step S106), the detection target of the main subject region S [n] In this example, the determination is based on the same number of images (3) when the image is at the end of the time series and when the image is not at the end of the time series. However, the CPU 19 determines the reference number for determining that the main subject has moved out of the frame of the target image when the image to be detected in the main subject region S [n] is at the end of the time series and It may be changed depending on whether it is not in the department. For example, the CPU 19 determines that the main subject has moved out of the frame of the target image when two images in which the main subject region S [n] cannot be detected are consecutive at the end of the time series, and the main subject region S [n ] Is not at the end of the time series, it is determined that the main subject has moved out of the frame of the target image when three images in which the main subject region S [n] cannot be detected are consecutive. Also good.

  Further, for example, the CPU 19 determines that the main subject has moved out of the frame of the target image when four images in which the main subject region S [n] cannot be detected are consecutive at the end of the time series, and the main subject region S When the detection target image of [n] is not at the end of the time series, it is determined that the main subject has moved out of the frame of the target image when three images in which the main subject region S [n] cannot be detected are consecutive. May be.

  Note that the above-mentioned “predetermined number as a reference for determining that the main subject has moved out of the frame of the target image” may be set based on a user operation via the operation unit 20 or automatically by the CPU 19. It may be set. When the CPU 19 automatically sets, an appropriate number may be set according to the type of shooting mode and shooting conditions set at the time of imaging.

  In the present embodiment, an example in which the series of processes shown in the flowchart of FIG. 2 is performed in the best shot mode is shown, but the present invention is not limited to this example. For example, the present invention can be similarly applied when generating a plurality of images in the continuous shooting mode. Moreover, you may perform the process after step S106 for the some image selected by the user.

  In the above-described embodiment, the example in which the technique of the present invention is implemented in the electronic camera 10 illustrated in FIG. 1 has been described. However, the present invention is not limited to this. For example, the present invention can be similarly applied to an electronic camera having a configuration other than the electronic camera 10 illustrated in FIG.

Further, the image processing apparatus described in the above embodiment may be realized as software by a computer and an image processing program. In this case, what is necessary is just to make it the structure which implement | achieves part or all of the process after step S106 demonstrated with the flowchart of FIG. 2 with a computer. Or
Further, in addition to the computer described above, the present invention can be similarly applied to an electronic device (for example, a photo viewer, a digital photo frame, a photo printing apparatus, etc.) having a digital image reproduction display function and a retouch function. it can. In addition, the electronic camera 10 described in the above-described embodiment may be mounted as a camera module such as a mobile phone terminal.

  In the above-described embodiment, an example in which each process is realized by software has been described. However, it is a matter of course that each process may be realized by hardware using an ASIC or the like.

DESCRIPTION OF SYMBOLS 10 ... Electronic camera, 13 ... Image sensor, 14 ... AFE, 16 ... 1st memory, 17 ... 2nd memory, 19 ... CPU, 22 ... Storage medium

Claims (7)

An imaging unit that images a subject and generates a plurality of continuous images;
For each of the plurality of images generated by the imaging unit, a main subject region detection process is performed, and when the main subject region cannot be detected in an arbitrary image, at least one image temporally continuous with the image A detection unit that performs detection processing of the main subject region in the arbitrary image based on the main subject region detected in the image of
An electronic camera comprising: an evaluation unit that calculates an evaluation value based on an image of the main subject region, and evaluates the plurality of images generated by the imaging unit based on the calculated evaluation value . The electronic camera according to claim 1,
The detection unit, when performing detection processing of the main subject region for each of the plurality of images generated by the imaging unit, if the number of images that cannot detect the main subject region continues for a predetermined number of times, Stop the detection process of the main subject area in those images,
The electronic camera according to claim 1, wherein the evaluation unit sets the evaluation value relating to a plurality of images in which the main subject area cannot be detected as a predetermined value indicating that the main subject area cannot be detected. The electronic camera according to claim 1,
For each of the plurality of images generated by the imaging unit, when the detection process of the main subject region is performed, the detection unit, at the time-series end of the plurality of temporally continuous images, If there are more than a predetermined number of images in which the main subject area cannot be detected, the main subject area detection process in those images is stopped,
The electronic camera according to claim 1, wherein the evaluation unit sets the evaluation value relating to a plurality of images in which the main subject area cannot be detected as a predetermined value indicating that the main subject area cannot be detected. The electronic camera according to any one of claims 1 to 3,
A recording unit that records the plurality of images generated by the imaging unit on a recording medium;
An electronic camera, comprising: a deletion unit that deletes some of the plurality of images recorded on the recording medium based on an evaluation result by the evaluation unit. An acquisition unit that acquires a plurality of images as a processing target image;
For each of the plurality of images acquired by the acquisition unit, a main subject region detection process is performed, and if the main subject region cannot be detected in an arbitrary image, at least one image that is temporally continuous with the image A detection unit that performs detection processing of the main subject region in the arbitrary image based on the main subject region detected in the image of
An evaluation unit that calculates an evaluation value based on the image of the main subject region, and that evaluates the plurality of images acquired by the acquisition unit based on the calculated evaluation value;
An image processing apparatus comprising: a selection unit that selects a part of the plurality of images acquired by the acquisition unit based on an evaluation result by the evaluation unit. The image processing apparatus according to claim 5.
A recording unit that records the plurality of images acquired by the previous acquisition unit on a recording medium;
A deletion unit that deletes the part of the images selected by the selection unit among the plurality of images recorded on the recording medium;
The acquisition unit acquires a plurality of images captured and generated continuously as the plurality of images,
The image processing apparatus, wherein the detection unit performs the main subject area detection process for each of the plurality of images recorded on the recording medium. An image processing program for realizing image processing on an image to be processed by a computer,
An acquisition step of acquiring a plurality of images as the processing target image;
For each of the plurality of images acquired in the acquisition step, a main subject region detection process is performed, and when the main subject region cannot be detected in an arbitrary image, at least one image temporally continuous with the image A detection step of performing detection processing of the main subject region in the arbitrary image based on the main subject region detected in the image of
An evaluation value based on the image of the main subject area, and an evaluation step for evaluating the plurality of images acquired in the acquisition step based on the calculated evaluation value;
An image processing program comprising: a selection step of selecting some of the plurality of images acquired in the acquisition step based on an evaluation result in the evaluation step.

Images