JP2018121346A - Image processing apparatus, imaging apparatus, and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract a specific region accurately according to a subject.SOLUTION: An image processing apparatus includes an area detector for detecting multiple subject areas included in an image by color information or brightness information, a calculating section for calculating area changes of the subject areas detected in the area detector, by using multiple images including the subject areas, and an extracting section for extracting a specific area from the multiple subject areas detected by the area detector by the area changes of the subject areas calculated by the calculating section. The extracting section extracts the subject area as the specific based on the area changes of the subject areas calculated by the calculating section, out of the multiple subject areas detected by the area detector.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing device, an imaging device, and an image processing program.

  Conventionally, various techniques relating to focus adjustment have been considered. For example, the invention of Patent Document 1 discloses an imaging apparatus that performs focus adjustment using a subject extraction technique. In the invention of Patent Document 1, it is possible to optimally perform automatic focus adjustment on a subject even when the subject moves in the camera direction by deforming the AF distance measurement frame in accordance with the amount of change in the subject shape. Yes.

JP 2009-069748 A

  By the way, when there are a plurality of subjects in the object scene, a specific region (for example, a subject region or a region of interest) that the user is photographing and a specific region that is a background are in the optical axis direction of the imaging apparatus. There are cases where images are overlapped. In such a case, the specific area as the background is extracted, and the specific area desirable for the user may not necessarily be extracted.

  The present invention has been made in view of the above points, and an object of the present invention is to accurately extract a specific area according to a subject.

  An image processing apparatus according to the present invention includes a region detection unit that detects a plurality of subject regions included in an image based on color information or luminance information, and an area change of the subject region detected by the region detection unit includes the subject region. A calculation unit that calculates using a plurality of images, and an extraction unit that extracts a specific region from the plurality of subject areas detected by the region detection unit based on a change in the area of the subject region calculated by the calculation unit, The extraction unit extracts a subject region as the specific region based on a change in the area of the subject region calculated by the calculation unit among the plurality of subject regions detected by the region detection unit.

  The extraction unit sets priorities for extracting the specific region with respect to the plurality of subject regions detected by the region detection unit based on the area change of the subject region calculated by the calculation unit. May be.

  In addition, the extraction unit sets the priority to be low for a subject region having a large area change of the subject region calculated by the calculation unit among the plurality of subject regions detected by the region detection unit. May be.

  Moreover, the determination part which determines whether the said specific area extracted by the said extraction part is continuously detected using the some image in which the said specific area is included is provided.

  The calculation unit further includes an operation member that changes the number of images used for calculating the area change of the subject region detected by the region detection unit.

  An imaging apparatus according to the present invention includes an imaging device that captures an image of a subject and any of the image processing devices described above, and the calculation unit calculates an area change of the subject region detected by the region detection unit. The number of images used for is changed according to the frame rate of the image sensor.

  The image pickup apparatus of the present invention includes an image pickup device that picks up a subject via an optical system and any one of the image processing devices described above, and the calculation unit detects the subject region detected by the region detection unit. The number of images used for calculating the area change is changed by the zoom magnification of the optical system.

  The image processing program of the present invention allows a computer to detect an area detection procedure for detecting a plurality of subject areas included in an image based on color information or luminance information, and an area change of the subject area detected by the area detection procedure. A calculation procedure for calculating using a plurality of images including a region, and an extraction procedure for extracting a specific region from the plurality of subject areas detected by the region detection procedure based on an area change of the subject region calculated by the calculation procedure The extraction procedure uses the subject region as the specific region based on the area change of the subject region calculated by the calculation procedure among the plurality of subject regions detected by the region detection procedure. Extract.

  According to the present invention, it is possible to accurately extract a specific area according to a subject.

2 is a block diagram illustrating configurations of a lens barrel 10, an imaging device 20, and a storage medium 40. FIG. It is a figure explaining the conventional mask extraction process. It is a flowchart which shows operation | movement of CPU26 at the time of automatic detection mode execution of this embodiment. It is a figure explaining calculation of a change characteristic. It is a figure explaining a mask extraction process.

  Hereinafter, embodiments will be described in detail with reference to the drawings.

  In the present embodiment, an apparatus having the lens barrel 10, the imaging device 20, and the storage medium 40 as shown in FIG. 1 will be described as an example.

  The imaging device 20 captures an optical image incident from the lens barrel 10. The obtained image is stored in the storage medium 40 as a still image or a moving image.

  The lens barrel 10 includes a focus adjustment lens (hereinafter referred to as an “AF (Auto Focus) lens”) 11, a lens driving unit 12, an AF encoder 13, and a lens barrel control unit 14. The lens barrel 10 may be detachably connected to the imaging device 20 or may be integrated with the imaging device 20.

  The imaging device 20 includes an imaging unit 21, an image processing device 22, a display unit 23, a buffer memory unit 24, a storage unit 25, a CPU 26, an operation unit 27, and a communication unit 28. The imaging unit 21 includes an imaging element 29 and an A / D (Analog / Digital) conversion unit 30. The imaging unit 21 includes an imaging element (not shown), and is controlled by the CPU 26 according to set imaging conditions (for example, an aperture value, an exposure value, etc.).

  In the lens barrel 10, the AF lens 11 is driven by the lens driving unit 12, and guides an optical image to a light receiving surface (photoelectric conversion surface) of an imaging element (not shown) in the imaging unit 21 of the imaging device 20. The AF encoder 13 detects the movement of the AF lens 11 and outputs a signal corresponding to the movement amount of the AF lens 11 to the lens barrel control unit 14. Here, the signal corresponding to the movement amount of the AF lens 11 may be, for example, a sine wave signal whose phase changes according to the movement amount of the AF lens 11.

  The lens barrel control unit 14 controls the lens driving unit 12 in accordance with a drive control signal input from the CPU 26 of the imaging device 20. Here, the drive control signal is a control signal for driving the AF lens 11 in the optical axis direction. The lens barrel control unit 14 changes, for example, the number of steps of the pulse voltage output to the lens driving unit 12 according to the drive control signal. Further, the lens barrel control unit 14 outputs the position (focus position) of the AF lens 11 in the lens barrel 10 to the CPU 26 of the imaging device 20 based on a signal corresponding to the movement amount of the AF lens 11. Here, the lens barrel control unit 14 integrates, for example, signals according to the movement amount of the AF lens 11 according to the movement direction of the AF lens 11, thereby moving the AF lens 11 in the lens barrel 10 ( Lens position) may be calculated. The lens driving unit 12 drives the AF lens 11 according to the control of the lens barrel control unit 14 and moves the AF lens 11 in the optical axis direction within the lens barrel 10.

  In the imaging device 20, the imaging unit 21 may include, for example, an imaging element that converts an optical image formed on the photoelectric conversion surface by the lens barrel 10 (optical system) into a digital signal and outputs the digital signal. good. In addition, for example, the imaging unit 21 includes an imaging device and an A / D conversion unit, and the imaging device converts an optical image formed on the photoelectric conversion surface by the lens barrel 10 (optical system) into an electrical signal and converts it into an A signal. The A / D converter may output to the / D converter, and the A / D converter may digitize the electrical signal converted by the image sensor and output the digital signal. Note that the imaging element is configured by a photoelectric conversion element such as a CMOS (Complementary Metal Oxide Semiconductor), for example. Further, the image sensor may convert an optical image into an electric signal for a partial region of the photoelectric conversion surface (image clipping). Further, the imaging unit 21 outputs an image obtained when a shooting instruction from the user is received via the operation unit 27 to the storage medium 40 via the communication unit 28. On the other hand, the imaging unit 21 outputs continuously obtained images to the buffer memory unit 24 and the display unit 23 as through images in a state before receiving a shooting instruction from the user via the operation unit 27.

  The image processing device 22 performs image processing on the image temporarily stored in the buffer memory unit 24 based on the image processing conditions stored in the storage unit 25. The image after image processing is stored in the storage medium 40 via the communication unit 28. Further, the image processing device 22 performs mask extraction processing on the image temporarily stored in the buffer memory unit 24 (details will be described later). Information about the result of the mask extraction process is output to the CPU 26 and stored in the storage unit 25, the storage medium 40, and the like.

  The display unit 23 is, for example, a liquid crystal display, and displays an image generated by the imaging unit 21, an operation screen, and the like. The buffer memory unit 24 temporarily stores the image generated by the imaging unit 21. The storage unit 25 stores imaging conditions, determination conditions referred to by the CPU 26 in various determinations, and the like.

  The CPU 26 includes an area detection unit 31, a characteristic calculation unit 32, and a control unit 33. The CPU 26 acquires necessary information from the image processing device 22, the storage unit 25, and the like, and based on the acquired information, Centrally control each part of. Specific operations of each part will be described later. For control by the CPU 26, focus adjustment (AF) setting, exposure adjustment (AE) setting, white balance adjustment (WB) setting, flash emission amount change setting, subject tracking setting, various shooting mode settings, etc. , Various image processing settings, various display settings, brightness optimization settings linked to zoom magnification, and the like. In addition, the CPU 26 monitors the operation state of the operation unit 27 and outputs image data to the display unit 23.

  The operation unit 27 includes, for example, a power switch, a shutter button, a multi-selector (cross key), or other operation keys. The operation unit 27 receives a user operation input when operated by the user, and outputs a signal corresponding to the operation input to the CPU 26. Output to.

  The communication unit 28 is connected to a removable storage medium 40 such as a card memory, and writes, reads, or deletes information (image data, area information, etc.) to the storage medium 40.

  The storage medium 40 is a storage unit that is detachably connected to the imaging device 20 and stores information (image data, area information, and the like). Note that the storage medium 40 may be integrated with the imaging device 20.

  The imaging device 20 includes an automatic detection mode for automatically detecting a specific area in addition to a normal mode for detecting a specific area (for example, a main subject area or a region of interest) based on focus adjustment information during shooting. The automatic detection mode is a mode in which a specific region is automatically and continuously detected based on a through image for composition confirmation and the like, and each unit is controlled based on information on the detected specific region. This automatic detection mode may be set by a user operation via the operation unit 27, or may be automatically set by the CPU.

  Here, a case to be solved by the present embodiment will be described with reference to FIG. 2A, FIG. 2B, and FIG. 2C are images generated continuously in time. In FIGS. 2A to 2C, a person who is a main subject exists on the near side of the object scene, and a bus exists on the back side of the object scene. The position of the person on the front side changes in the screen due to framing by the user, and the bus on the back side moves from the left side to the right side of the screen. In such a case, when conventional automatic detection is performed, as shown in FIG. 2A, the bus that is the background (background) may be divided by the person who is the foreground and extracted as a plurality of specific areas. . In the case of FIG. 2A, for example, four specific areas m1 to m4 are extracted, and the specific area m2 is detected as the main subject area. However, in the case of FIG. 2B, for example, three specific areas m1 to m3 may be extracted, and the specific area m1 may be detected as the main subject area. This is because, in the example of FIG. 2B, the specific area m1 corresponding to a part of the background bus is closer to the center than the specific area m2 corresponding to the foreground person. As described above, in order to solve the problem that the subject that is originally the background is extracted as the main subject region and the specific region desirable for the user is not necessarily extracted, in the present embodiment, the subject is accurately identified according to the subject. A process intended to extract a region is executed.

  The operation of the CPU 26 when executing the automatic detection mode for detecting the main subject area as the specific area will be described below with reference to the flowchart of FIG.

  In step S <b> 101, the CPU 26 controls the imaging unit 21 to start acquiring a through image. The acquired image information of the through image is temporarily stored in the buffer memory unit 24. This through image is continuously generated at a predetermined time interval. The acquisition of the through image by the CPU 26 is sequentially performed sequentially in time.

  In step S102, the CPU 26 controls the image processing device 22 to perform normal image processing. Normal image processing includes white balance adjustment, interpolation processing, color tone correction processing, gradation conversion processing, and the like. Since the specific method of each process is the same as that of a well-known technique, description is abbreviate | omitted. The image processing device 22 acquires the image data of the target image from the buffer memory unit 24, performs image processing, and then outputs the image data to the buffer memory unit 24 again.

  In step S <b> 103, the CPU 26 performs mask extraction processing by the area detection unit 31. The mask extraction process is a technique for calculating a feature amount in an image and detecting a main subject region based on a similar feature amount (mask). For example, the area detection unit 31 obtains an evaluation value from a feature amount in an image, and performs mask extraction by obtaining a continuous area of evaluation values that can be regarded as identical or identical. Examples of the mask include a pure color mask, a color mask, and a luminance mask. Any method of mask extraction may be used. In addition, since a specific method for extracting a mask is the same as that of a known technique, description thereof is omitted.

  Note that the image processing apparatus 22 creates a plurality of types of masks, excludes unnecessary masks from the created types of masks, and determines priorities for the remaining masks. The priority indicates a priority when determining a mask that is actually used for extracting a subject region from a plurality of types of masks. The priority is generally determined such that the priority of the mask related to color information is relatively high. Then, the image processing apparatus 22 obtains a mask having a certain evaluation value or higher while giving priority to each mask having a high priority in accordance with these priorities, and uses the mask as a mask for extracting a subject area. In step S103, the image processing apparatus 22 extracts N masks in descending order of evaluation values. N described above is a predetermined number (for example, N = 5), and may be appropriately changed based on a user operation via the operation unit 27. The image processing device 22 extracts subject areas based on the selected N masks, and proceeds to step S104 described later. In the following, it is assumed that N = 5 described above, and the image processing apparatus 22 extracts five masks (masks M1 to M5). In the subsequent mask extraction process, the five masks (masks M1 to M5) are basically extracted continuously. For these five masks (masks M1 to M5), the extraction conditions may be set loosely to facilitate continuous extraction.

  In step S104, the CPU 26 determines whether or not the subject area has been extracted. If the CPU 26 determines that one or more subject areas have been extracted, the process proceeds to step S105. On the other hand, if it is determined that no subject area can be extracted, the CPU 26 returns to step S102, and performs the processing from step S102 onward for the image of the next frame.

  In step S105, the CPU 26 records mask information. The CPU 26 records information about the result of the mask extraction process executed in step S103 in the buffer memory unit 24, the storage unit 25, and the like as mask information. The mask information includes information such as the extraction conditions (color classification, mask classification, etc.) of the mask extraction processing in step S103, the position of the mask, the size and shape of the mask, and the rank of each mask based on the evaluation value.

  In step S106, the CPU 26 determines whether or not the same mask has been extracted continuously n times (n frames) or more. If the CPU 26 determines that the same mask has been extracted n times or more in succession, it is assumed that the same subject area is continuously detected, and the process proceeds to step S107 described later. On the other hand, when determining that the same mask has not been extracted n times or more in succession, the CPU 26 returns to step S102 described above, and performs the processing from step S102 onward for the next through image.

  This determination is made for each of the five masks described in step S103. However, it is not always necessary to extract the same mask n times or more consecutively for all the masks. Of the five masks described in step S103, the same mask is continuously applied n times or more for at least one mask. Should be extracted. The case where the same mask is extracted is a case where masks of substantially the same color and size are extracted at substantially the same position in a plurality of frames. For this determination, the mask information described in step S105 may be used. In any case, when the amount of change between frames is within a specified range, it can be determined that they are the same.

  Note that n described above is a predetermined threshold (for example, n = 7), and is appropriately determined based on the frame rate at the time of image capturing by the image capturing unit 21, the zoom magnification in the lens barrel 10, the user operation via the operation unit 27, and the like. It may be changeable. Further, the tendency of mask extraction performed in the past (the type of extracted mask, the appearance frequency, etc.) is obtained based on the mask information described in step S105, and the value of n described above is appropriately set according to this tendency. It may be changeable. For example, for a mask extracted at a high frequency in the past, the value of n may be changed to a smaller value.

  Further, the CPU 26 may be configured to record the contents of the determination made in step S106 in addition to the mask information described in step S105, and to use it at the time of processing related to subsequent frames.

  Further, the CPU 26 may be configured to determine whether or not the mask extraction process is stably performed by a method other than that described above. For example, the determination may be performed in consideration of whether the same mask has been extracted continuously until the previous time. With this configuration, even if temporary unextraction occurs only this time, if the same mask is extracted in the next frame without invalidating the previous accumulation, the previous stable state is maintained. Can be considered. For example, contrary to the case illustrated in FIG. 2, when a person exists on the back side and a car or a train passes through the front side, it can be handled as temporary unextraction.

  In step S <b> 107, the CPU 26 calculates a change characteristic of the area of each mask by the characteristic calculation unit 32. The characteristic calculation unit 32 obtains a change in the area of each mask within a predetermined time in a plurality of frames. FIG. 4 is a graph showing the relationship between the frame number (time) and the mask area. The vertical axis in FIG. 4 indicates the mask area, and the area increases as it goes upward. The horizontal axis in FIG. 4 indicates the frame number (time), and the frame number increases toward the right, and time elapses. As shown in FIG. 4, the characteristic calculation unit 32 obtains a change in the mask area as a change characteristic for each of the five masks (masks M1 to M5) described in step S103.

  In step S108, the CPU 26 determines whether or not the main subject region has been extracted by the control unit 33 based on the change characteristic calculated in step S107. If the CPU 26 determines that the main subject area has been extracted by the control unit 33, the process proceeds to step S109. On the other hand, if the control unit 33 determines that the main subject region has not been extracted, the CPU 26 returns to step S102 described above and performs the processing from step S102 onward for the next through image.

  Whether or not the main subject region has been extracted can be determined based on the change characteristics calculated in step S107 based on a change in mask area between a predetermined reference frame (reference image) and a processing target frame (image). Alternatively, the determination can be made based on a change in mask area between a frame (image) to be processed and a frame (image) immediately before (or acquired before the frame to be processed). For example, the reference frame may be a frame at a predetermined time, or the first frame of n times when it is determined in step S106 that the same mask has been extracted n times (n frames) or more. There may be.

  For example, with respect to an arbitrary mask, the control unit 33 sets the priority of the mask to be lower as the change in area is larger. That is, the control unit 33 makes it difficult to extract the mask as the main subject region or prevent the mask from being extracted as the main subject region as the change in the area is larger. As described with reference to FIG. 2, the area of the background (background) tends to have a large change in area. Therefore, in the example of FIG. 4, since the change in the area of the mask M1 is large, it is determined that it is not the main subject area but the background (background) area.

  Conversely, the control unit 33 sets a higher priority for an arbitrary mask as the change in the area is smaller. That is, the smaller the change in area, the easier it is for the control unit 33 to extract the mask as the main subject region, or to extract the mask as the main subject region. As described with reference to FIG. 2, the main subject region tends to have a small (or no) change in area. Therefore, in the example of FIG. 4, since the change in the area of the mask M2 and the mask M4 is small, it is determined that it is not the background (background) area but the main subject area.

  Note that the control unit 33 may make such a determination using any factor. For example, the control unit 33 may statistically evaluate the mask area by obtaining an average value or a median value, and the area ratio or a change in the distance of the mask from the center of the screen may be used as a factor. Further, for example, the control unit 33 may use an area change deviation or a mask distance deviation from the center of the screen as a factor.

  The determination in step S108 described above will be described with reference to FIG. In FIG. 5, Fa to Fe show examples when the conventional mask extraction processing is performed on images of frame numbers 1, 10, 20, 30, and 40, respectively, and FA to FE in FIG. In the following, an example in which the mask extraction processing of this embodiment is performed on the images of frame numbers 1, 10, 20, 30, and 40 will be shown. In Fa to Fe and FA to FE in FIG. 5, the five masks (masks M1 to M5) (parts) described in step S103 are extracted. Note that the mask M2 and the mask M5 are regions corresponding to the original main subject, and the other masks are regions corresponding to the background (background). In Fa and Fb of FIG. 5, the mask M2 that is an area corresponding to the main subject is correctly extracted as the main subject area. The same applies to FA and FB in FIG. However, in Fc of FIG. 5, the mask M1, which is an area corresponding to the background (background), is erroneously extracted as the main subject area. On the other hand, in the FC of FIG. 5, the priority of the mask M1 is set low by the above-described determination, so the mask M1 that is an area corresponding to the background (background) is not erroneously extracted as the main subject area. . Then, in Fd and Fe of FIG. 5, the mask M5 that is a region corresponding to the main subject newly entering the screen is correctly extracted as the main subject region. The same applies to FD and FE in FIG.

  When the control unit 33 performs the above-described processing, the CPU 26 determines whether or not the main subject area has been extracted.

  In step S <b> 109, the CPU 26 sets a main subject area by the control unit 33. The CPU 26 sets an area corresponding to the mask determined as the main subject in step S108 as the main subject area. This main subject area can be used for a shooting preparation operation.

  In step S110, the CPU 26 determines whether or not a shooting instruction has been issued. If the CPU 26 determines that a shooting instruction has been given, the process proceeds to step S111. On the other hand, if it is determined that the shooting instruction is not issued even after a predetermined time has elapsed, the CPU 26 returns to step S102 described above, and performs the processing from step S102 onward for the next through image. The photographing instruction is performed by a user operation via the shutter button of the operation unit 27. This user operation may be a so-called half shutter or all shutter.

  In step S111, the CPU 26 controls each unit to perform shooting. Note that processing such as AE, AF, and WB is performed based on information on the main subject region set in step S109.

  In step S112, the CPU 26 records the image data of the image generated by the shooting executed in step S111 on the storage medium 40 via the communication unit 28, and ends the series of processes.

  As described above, according to the present embodiment, a plurality of image data generated sequentially in time are sequentially acquired, a plurality of image feature amounts indicated by the acquired image data are calculated, and a plurality of features are calculated. An area detection unit that detects a specific area (for example, a subject area or an attention area) based on the amount, and a feature amount calculated for at least two images among the images indicated by the plurality of acquired image data And a characteristic calculation unit that obtains a change characteristic of the feature quantity, and determines priority in the plurality of feature quantities based on the change characteristic, and performs detection of a specific area by the area detection unit based on the determined priority. . Therefore, with the above-described configuration, it is possible to prevent an area of the subject that is the background (background) from being erroneously extracted as the specific area, and accurately extract the specific area according to the subject.

  Further, according to the present embodiment, the specific region is continuously detected by the region detection unit by comparing the specific regions detected in at least two images among the images indicated by the plurality of image data. If it is determined whether or not the specific area is continuously detected, a change characteristic is obtained by the characteristic calculation unit, and the specific area is detected by the area detection unit based on the priority. Therefore, the change of the subject can be grasped by the change characteristic, and the specific area can be extracted according to the change of the subject.

  Further, according to the present embodiment, a change characteristic related to the area of the specific area detected for at least two images among images indicated by a plurality of image data is obtained, and the change characteristic is determined based on the obtained change characteristic. The priority of the feature amount used for detection of the specific area that is the calculation target is determined, and the specific area is detected by the area detection unit based on the determined priority. Therefore, the change of the subject can be grasped more accurately by the change characteristic regarding the area.

  Also, according to the present embodiment, based on the change characteristics, the priority is set lower as the area change is larger, and the specific area is detected by the area detector. Therefore, the larger the change in the area, the more the area is determined to be the background (background), and the subject area that is the background (background) can be prevented from being erroneously extracted as the specific area. .

  Further, according to the present embodiment, the area change rate is obtained based on the change characteristic, and when the change rate is larger than a predetermined threshold, it is used to detect the specific region that is the target of the change characteristic calculation. The priority of the feature amount is set lower than when the rate of change is smaller than a predetermined threshold, and the specific region is detected by the region detection unit. Therefore, as the area change rate increases, it is determined that the area is a background (background) area, and the subject area that is the background (background) is prevented from being erroneously extracted as a specific area. it can.

  In the above embodiment, an example in which the priority is determined according to the change characteristic has been shown. However, as a special example, even a mask whose priority is lowered according to the change characteristic is continuously extracted. The mask may be extracted as a specific area. That is, it is determined whether or not the specific area is continuously detected by the area detection unit by comparing the specific areas detected in at least two images among the images indicated by the plurality of image data. If it is determined that the specific area is continuously detected, the priority of the feature amount used for detecting the specific area continuously detected is set higher than when the specific area is not continuously detected. Thus, the specific area is detected by the area detecting unit. Therefore, when a specific area is continuously detected, it can be determined that the specific area is a user-desired area, and an accurate specific area can be extracted.

  In the above embodiment, when the zoom magnification in the lens barrel 10 is changed during a series of processes, the comparison of the areas is unnecessary. Therefore, in such a case, it is preferable to reset the processing or appropriately convert the area according to the zoom magnification.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  In the above embodiment, an example in which a series of processing is performed based on a through image for composition confirmation has been described, but the present invention is not limited to this example. For example, the present invention can be similarly applied to a case where a live view image for composition confirmation generated in a single-lens reflex camera or the like is targeted. Further, the present invention can be similarly applied to a case where a moving image recorded in the storage medium 40 or the like is a target.

  In the above embodiment, an example in which a series of processing is performed for all frames has been described, but the present invention is not limited to this example. For example, a plurality of images generated intermittently in time may be targeted. Specifically, a plurality of images subjected to frame thinning as appropriate may be targeted. In this case, a series of processes in the above embodiment may be performed on a plurality of thinned images, and all the images may be displayed. By performing such processing, the processing load can be reduced.

  Further, the image processing described in the above-described embodiment may be realized by software by a “computer system” having a computer and an image processing program. In this case, a part or all of the processing of the flowchart described in the embodiment may be configured to be executed by the computer system. For example, part or all of the processing from step S102 to step S109 in FIG. 3 may be executed by a computer. By adopting such a configuration, it is possible to perform the same processing as in the above-described embodiment.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a www system is used. The computer-readable recording medium is a writable nonvolatile memory such as a flexible disk, a magneto-optical disk, a ROM, or a flash memory, a portable medium such as a CD-ROM, and a storage such as a hard disk built in the computer system. Refers to the device.

  Further, the computer-readable recording medium is a volatile memory (for example, DRAM (Dynamic Random Access) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Memory)) that holds a program for a certain period of time is also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 20 ... Imaging device, 21 ... Imaging part, 22 ... Image processing apparatus, 23 ... Display part, 26 ... CPU

Claims (8)

An area detection unit for detecting a plurality of object areas included in an image based on color information or luminance information;
A calculation unit for calculating an area change of the subject region detected by the region detection unit using a plurality of images including the subject region;
An extraction unit that extracts a specific region from the plurality of subject regions detected by the region detection unit based on a change in the area of the subject region calculated by the calculation unit;
The extraction unit is an image processing apparatus that extracts a subject region as the specific region based on an area change of the subject region calculated by the calculation unit among the plurality of subject regions detected by the region detection unit. The image processing apparatus according to claim 1.
The extraction unit is configured to perform image processing for setting a priority for extracting the specific region with respect to a plurality of the subject regions detected by the region detection unit based on an area change of the subject region calculated by the calculation unit. apparatus. The image processing apparatus according to claim 2,
The extraction unit sets the priority to be low for a subject region in which the area change of the subject region calculated by the calculation unit is large among the plurality of subject regions detected by the region detection unit. apparatus. The image processing apparatus according to any one of claims 1 to 3,
An image processing apparatus provided with the determination part which determines whether the said specific area extracted by the said extraction part is continuously detected using the some image in which the said specific area is included. In the image processing device according to any one of claims 1 to 4,
An image processing apparatus comprising: an operation member that changes the number of images used for calculating an area change of the subject region detected by the region detection unit by the calculation unit. An image sensor for imaging a subject;
An image processing device according to any one of claims 1 to 5,
The image pickup apparatus, wherein the calculation unit changes the number of images used for calculating the area change of the subject region detected by the region detection unit according to a frame rate of the image pickup element. An image sensor for imaging a subject via an optical system;
An image processing device according to any one of claims 1 to 5,
The image pickup apparatus, wherein the calculation unit changes the number of images used for calculating an area change of the subject region detected by the region detection unit according to a zoom magnification of the optical system. On the computer,
An area detection procedure for detecting a plurality of object areas included in an image based on color information or luminance information;
A calculation procedure for calculating an area change of the subject region detected by the region detection procedure using a plurality of images including the subject region;
An extraction procedure for extracting a specific region from a plurality of the subject regions detected by the region detection procedure based on an area change of the subject region calculated by the calculation procedure;
The extraction procedure is an image processing program for extracting a subject region as the specific region based on an area change of the subject region calculated by the calculation procedure among the plurality of subject regions detected by the region detection procedure.

Images