JP2013098746A - Imaging apparatus, imaging method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform various types of control relating to imaging by the concurrent use of a subject region by subject detection and a tracked region by tracking.SOLUTION: An imaging unit 124 generates image data by taking an image of a subject. A subject detector 170 identifies a subject region with respect to the obtained image data. A tracking unit 174 identifies a tracked region as a tracking target with respect to the obtained image data. A distance determination unit 176 determines whether the identified subject region is included within a predetermined distance from the identified tracked region. If the identified subject region is included within the predetermined distance, a region integration unit 178 integrates information on the subject region to information on the identified tracked region. When the region integration unit 178 integrates the information on the subject region to the information on the identified tracked region, a controller controls imaging by use of the subject region.

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program for performing various control related to imaging using a predetermined area specified from image data obtained by imaging a subject.

  2. Description of the Related Art In recent years, face detection technology for detecting a human face in a captured image has been adopted in an imaging apparatus such as a digital still camera or a digital video camera. Such an imaging apparatus performs control such as autofocus and automatic exposure based on face information (luminance, position, etc.) detected by the face detection technique.

  There has been proposed a technique for extending this face detection technique, detecting a body area (body detection) based on the face area where the face is detected, and performing autofocus based on the face area and the body area (for example, Patent Document 1). According to such a technique, the imaging apparatus can perform autofocus with high accuracy by taking a wide target area for distance measurement.

  Also, in an imaging apparatus such as a digital still camera or a digital video camera, a so-called tracking technique for tracking a subject to be imaged is conventionally used. For example, in a tracking technique that uses color information of captured image data, the imaging device first specifies a color pattern of a tracking area within a frame of the captured image data. Next, with respect to the frame next to the specified frame of the tracking area, the color pattern of the area in the vicinity of the coordinates of the tracking area of the previous frame is specified, and the color patterns of both specified frames are compared. Then, an area that is considered to be similar in color to the tracking area is regarded as a new tracking area in the next frame. Thereafter, the imaging apparatus performs control based on information (luminance, position, etc.) of the image in the tracking area while repeating the same processing for each frame.

JP 2008-76786 A

  By the way, for example, when a pet such as a cat or a dog is imaged, the above-described control such as autofocus and automatic exposure can be performed by using a face detection technique for the face. However, even if the technique of Patent Document 1 is used, it is difficult for the pet to detect the body because the position of the body relative to the face and the shape of the body are difficult to specify. Furthermore, it is not easy for the photographer to keep the pet in a state in which the face faces the imaging device. A predetermined area is specified from image data obtained by imaging the pet, and the specified area is used. Therefore, it has been difficult to perform control such as autofocus and automatic exposure.

  Here, if the tracking technique described above is used in combination with the face detection technique, even if a face is detected and a face is turned away from the imaging apparatus and the face is lost (lost), the imaging apparatus The subject can be continuously tracked by the color pattern and control can be performed. On the other hand, when the subject being tracked through such processing turns the face toward the imaging device again, a new face is detected. At this time, the imaging apparatus recognizes the detected face, which should be the same subject, and the subject that has been tracked as different subjects. In this case, the detected face information should be referred to, but only the tracking area information is referred to, and there is a possibility that appropriate control may not be performed.

  Therefore, in view of such problems, the present invention provides an imaging apparatus, an imaging method, and a program capable of performing various types of control related to imaging by appropriately using a subject area by subject detection and a tracking area by tracking appropriately. The purpose is that.

  In order to solve the above-described problem, an imaging apparatus including an imaging unit that images a subject and generates image data according to the present invention is generated with a subject detection unit that identifies a subject region for the generated image data. For the image data, a tracking unit that identifies a tracking region that is a tracking target, a distance determination unit that determines whether or not the identified subject region is included within a predetermined distance from the identified tracking region, and a specified When the subject area is included within a predetermined distance, an area integration unit that integrates information about the subject area into information about the identified tracking area, and information about the subject area is integrated into the tracking area information by the area integration unit, And a control unit that controls imaging using the subject area.

  When the information related to the subject area is integrated into the tracking area information by the area integration unit, the control unit may perform autofocus or automatic exposure control for the subject area.

  A display unit that displays an image based on the image data is further provided, and when the information related to the subject area is integrated into the tracking area information by the area integration unit, the control unit displays an indicator on the display unit by superimposing only the subject area. Further control may be performed.

  The distance determination unit may derive the size of the specified subject area and set the predetermined distance in proportion to the size of the subject area.

  In order to solve the above problems, an imaging method of the present invention images a subject, generates image data, identifies a subject region for the generated image data, and sets a tracking region that is a tracking target for the generated image data. It is determined whether or not the specified subject region is included within a predetermined distance from the specified tracking region, and when the specified subject region is included within the predetermined distance, information regarding the specified tracking region is related to the subject region. The information is integrated, and when the information on the subject area is integrated with the information on the specified subject area, the imaging is controlled using the subject area.

  In order to solve the above problems, a program according to the present invention includes a computer that processes image data generated by imaging a subject, a subject detection unit that identifies a subject area for the generated image data, and a generation A tracking unit that identifies a tracking area that is a tracking target, a distance determination unit that determines whether or not the identified subject area is included within a predetermined distance from the identified tracking area, When the determined subject area is included within a predetermined distance, an area integration unit that integrates information about the subject area into information about the specified tracking area, and information about the subject area is integrated into the tracking area information by the area integration unit And a control unit that controls imaging using the subject area.

  According to the present invention, it is possible to perform various types of control related to imaging by appropriately using a subject area based on subject detection and a tracking area based on tracking.

It is a functional block diagram showing a schematic function of an imaging device. It is the external view which showed an example of the imaging device. It is explanatory drawing for demonstrating area | region integration processing. It is explanatory drawing for demonstrating area | region integration processing. It is the flowchart which showed the whole flow of the imaging method.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  The imaging apparatus 100 according to the present embodiment, when subject detection and tracking are used together, performs autofocusing when a plurality of areas are specified for image data of the same subject by subject detection and tracking. It is possible to appropriately perform various controls relating to imaging such as, automatic exposure, or superimposition of an index on image data. Hereinafter, such an imaging apparatus 100 will be described in detail.

(Imaging device 100)
FIG. 1 is a functional block diagram illustrating a schematic function of the imaging apparatus 100, and FIG. 2 is an external view illustrating an example of the imaging apparatus 100. The imaging apparatus 100 includes a display unit 120, an operation unit 122, an imaging unit 124, a data processing unit 126, an image storage unit 128, an image processing unit 130, a recording / playback I / F unit 132, and an image output. Unit 134, main storage unit 136, and central control unit 138. The operation unit 122, the data processing unit 126, the image storage unit 128, the image processing unit 130, the recording / playback I / F unit 132, the image output unit 134, the main storage unit 136, the central control unit 138, and a drive described later. The control unit 154 is connected via the system bus 140.

  The display unit 120 includes a liquid crystal display, an organic EL (Electro Luminescence) display, and the like, and displays an image based on display image data (view image data) (hereinafter simply referred to as “display image data”). To do). The photographer can check the captured image data by referring to the display image displayed on the display unit 120, and can capture the subject at a desired position and size.

  In addition, a touch sensor that is a part of the operation unit 122 is disposed on the surface of the display unit 120, and position information touched by the photographer is input to the central control unit 138 as an output of the operation unit 122.

  The operation unit 122 includes a touch sensor, an operation key including a release switch, a cross key, a joystick, and the like, and receives an operation input such as switching of an imaging mode from a photographer. In this imaging mode, for example, in addition to a mode for adapting to an environment such as outdoors, indoors, and night scenes, AF (automatic focus) and AE (automatic exposure) control are performed on a detected human face as a processing target with priority. “Human face priority AF & AE”, “Touch tracking AF & AE” that performs AF and AE control with the touch position of the photographer's touch sensor preferentially as a processing target, AF and AE with the detected pet face as a priority processing target A mode in which the priority order of AF & AE such as “pet face priority AF & AE” for controlling is determined is included.

  The imaging unit 124 images a subject through the imaging lens 124a and generates image data. Specifically, the imaging unit 124 includes an IR cut filter 140 that blocks near-infrared light, a zoom lens 142 for realizing a zoom function, a focus lens 144 used for focus adjustment, an aperture 146 used for exposure adjustment, and an imaging lens. An image sensor (imaging circuit) 148 composed of a CCD (Charge Coupled Devices) or the like that photoelectrically converts light such as a subject image incident through 124a to generate an image signal; an amplifier 150 that amplifies the image signal from the image sensor 148; An A / D converter 152 that converts the amplified image signal into image data that is digital data, a drive control unit 154 that controls the drive of the zoom lens 142, the focus lens 144, and the aperture 146, and a drive circuit 156 thereof are included. Composed.

  The data processing unit 126 performs processing such as forming luminance data and color difference data on the input image data, and transmits the processed data to the image storage unit 128. Further, the data processing unit 126 derives the average luminance of the image data and outputs it to the central control unit 138.

  The image storage unit 128 includes a buffer memory such as SDRAM (Synchronous-DRAM), and temporarily stores image data. The image data stored in the image storage unit 128 is referred to by the image processing unit 130, the central control unit 138, and the like.

  The image processing unit 130 acquires image data from the image storage unit 128, performs encoding processing in a format such as MPEG-2, MPEG-4, or MPEG-4 / AVC, and records data that is recording data. Generate. Further, the image processing unit 130 reduces the image data acquired from the image storage unit 128 in accordance with a control signal from the display control unit 182 to be described later, generates display image data, and outputs the display image data to the display unit 120. On the other hand, when reproducing recorded data recorded on an arbitrary recording medium 160 described later, the image processing unit 130 executes a process of decoding the recorded data.

  The recording / reproducing I / F unit 132 records the recording data (data stream) generated by the image processing unit 130 on an arbitrary recording medium 160. As the arbitrary recording medium 160, a medium that does not require a power source such as a DVD or a BD, or a medium that requires a power source such as a RAM, an EEPROM, a nonvolatile RAM, a flash memory, or an HDD can be applied. Further, the recording / reproducing I / F unit 132 reads out the recording data from an arbitrary recording medium 160 and outputs it to the image processing unit 130.

  The image output unit 134 outputs the image data from the image storage unit 128 and the image processing unit 130 to an external device 162 such as a liquid crystal display device or a projector, for example.

  The main storage unit 136 is configured by a buffer memory such as an SDRAM, and stores a program processed by the central control unit 138. The main storage unit 136 stores image data used for subject detection processing and tracking processing described later.

  However, when the tracking process is started, the image data stored in the image storage unit 128 is transferred to the first area and the second area of the main storage unit 136. In the subject detection process, the image data stored in the first area is used, and in the tracking process, the image data stored in the second area is used.

  Here, the reason that the same image data is stored in each of the first area and the second area is that the subject detection process takes longer than the tracking process. For example, the subject detection process takes about 5 frame periods, while the tracking process ends within one frame period. Therefore, in the subject detection process, image data is stored once every five frames. When both the time required for the subject detection process and the tracking process ends within one frame period, the first area and the second area of the main storage unit 136 may not be stored.

  The central control unit 138 manages and controls the entire imaging apparatus 100 by a semiconductor integrated circuit including a central processing unit (CPU), a ROM in which programs are stored, a RAM as a work area, and the like. Perform the operation. The central control unit 138 also functions as a subject detection unit 170, a color distribution extraction unit 172, a tracking unit 174, a distance determination unit 176, a region integration unit 178, an imaging control unit 180, and a display control unit 182.

  The subject detection unit 170 detects a subject (for example, a part other than the face of a person or a pet or a face) from the luminance data for the image data stored in the first area of the main storage unit 136. In the present embodiment, the subject detection unit 170 gives an example of detecting the face of a pet such as a cat or dog as the subject.

  Here, for example, the subject detection unit 170 detects a face by template matching using a reference template corresponding to the contour of the entire face. However, the subject detection unit 170 uses a plurality of discriminators generated by template matching based on facial components (eyes, nose, ears, etc.) and the Adaboost algorithm as disclosed in JP 2010-2960 A. Any existing face detection technique such as a technique used in cascade connection may be used. In any case, the subject detection unit 170 identifies face information indicating the position, size, inclination, etc. of the detected face and stores it in the main storage unit 136.

  The color distribution extraction unit 172 converts the luminance data and color difference data of the face center position and its peripheral region, out of the image data stored in the first region of the main storage unit 136, into RGB data, and obtains the color distribution. . In this embodiment, the outer edge of the peripheral area is, for example, a rectangular area composed of 11 pixels in the horizontal direction and 11 pixels in the vertical direction. However, the shape of the peripheral area is not limited to a rectangle, and may be a polygon. It may be circular or any other shape and size.

  In the present embodiment, the color distribution is indicated by a color histogram. The color histogram is derived as follows. That is, each pixel of the target image data is divided into spaces when the RGB color space is divided into N parts (for example, N = 16) perpendicular to the respective axes (R axis, G axis, B axis). It is determined which of the color ranges indicated by. The color histogram is derived by counting the number of pixels included in each color range and associating the color range with the number of pixels included in the color range. However, N is a predetermined natural number. Further, the color histogram may be derived by changing the number of divisions perpendicular to each axis for each axis.

  The color distribution extraction unit 172 may obtain color distribution by converting image data into HSV (Hue Saturation Value) data instead of RGB data. In this case, the color histogram is derived in the same manner as the RGB data for the color range indicated by each divided space when the N space is divided perpendicularly to each axis (H axis, S axis, V axis) in the HSV color space. The

  Then, the color distribution extraction unit 172 stores the derived color distribution information (the color histogram and the position information of the image data from which the color histogram is derived) in the main storage unit 136 in association with the subject area from which the color distribution is derived.

  The tracking unit 174 specifies a tracking region that is a tracking target by comparing color histograms with respect to image data stored in the second region of the main storage unit 136.

  Specifically, the tracking unit 174 first reads out information on the color distribution stored in the main storage unit 136. Then, the tracking unit 174 derives a color histogram from the image data of the comparison region near the position included in the read color distribution information.

  Thereafter, the tracking unit 174 derives a histogram intersection between the derived color histogram and the color histogram included in the read color distribution information.

  Here, the histogram intersection compares the number of pixels in the same color range for two color histograms, extracts only the smaller number of pixels, and takes the sum of the number of extracted pixels for all color ranges. It is a thing. As the color distribution of the image data to be compared is similar, the value of the histogram intersection increases.

  Next, the tracking unit 174 moves the comparison region by one pixel in the horizontal direction and the vertical direction within a predetermined range, and derives a histogram intersection at each position. Then, the tracking unit 174 considers that the subject has moved to a position where the value of the histogram intersection is the largest, and stores the position information in the main storage unit 136.

  With the configuration in which the tracking area is specified by comparing the color histograms, the tracking unit 174 is difficult to specify due to the change in the shape of the subject. For example, the body of the pet can be tracked.

  However, the tracking process is not limited to the histogram intersection using the color histogram, and any existing technique for comparing the similarity of the feature quantities of the color distribution may be used. Alternatively, a technique for comparing the similarity of single colors may be used, or a technique combining color information and shape information may be used.

…（数式１） The distance determination unit 176 determines whether or not the subject region specified by the subject detection unit 170 is included within a predetermined distance from the tracking region specified by the tracking unit 174. Here, the distance between the tracking area and the subject area is derived from the position of each area in the image data. For example, when there is a subject area at the position of the horizontal direction Px1 pixel and the vertical direction Py1 pixel from the upper left end in the screen in the image data, and the tracking area is at the position of the horizontal direction Px2 pixel and the vertical direction Py2 pixel, The distance L, which is the distance of the subject area, is derived by the following formula 1. However, Px1, Px2, Py1, and Py2 are integers.

... (Formula 1)

  Here, the position from which the distance is derived may be the center in each of the tracking area and the subject area, or may be the position where the distance between the two areas is the shortest. The starting point on the subject area side may be the center of the face or the center position of the left and right eyes. In any case, the starting point of each of the tracking area and the subject area is determined according to a predetermined criterion.

  When the subject area is included within a predetermined distance from the tracking area, the area integration unit 178 integrates information related to the subject area into information related to the tracking area. Such integration processing will be described in detail later.

  The imaging control unit 180 performs focus control as control during imaging. First, the imaging control unit 180 reads out image data of the subject area or the tracking area from the main storage unit 136. Then, the imaging control unit 180 performs automatic focus adjustment by the contrast method using the read image data. The contrast method is a technique for performing focusing using the fact that there is a correlation between the degree of blurring of an image on the image sensor 148 and the contrast, and the contrast becomes maximum when the image is focused.

  Here, the magnitude of the contrast is evaluated by the ratio of the high frequency component included in the image data. The imaging control unit 180 extracts a high frequency component by applying a BPF (band pass filter) to luminance data in the read image data, and uses the integral value of the extracted absolute value of the high frequency component as an evaluation value. Auto-focus calculation is performed to maximize. Then, the imaging control unit 180 outputs a control signal to the drive control unit 154 according to the calculation result, and controls the drive circuit 156 to perform the focusing operation.

  In addition, the imaging control unit 180 performs exposure adjustment control as control during imaging. First, the imaging control unit 180 reads out image data of the subject area or the tracking area from the main storage unit 136. Then, the imaging control unit 180 derives an average value for the luminance data included in the image data, and outputs a control signal to the drive control unit 154 so that the average value is included in a predetermined range. Let the amount adjust.

  The display control unit 182 causes the image processing unit 130 to generate display image data and causes the display unit 120 to display the display image data. At this time, the display control unit 182 superimposes an index indicating the subject area and the tracking area as control during imaging.

  Next, the integration process from the identification of the subject area and the tracking area and the subsequent control by the imaging apparatus 100 described above will be described in detail.

(Area identification and integration processing)
3 and 4 are explanatory diagrams for explaining the region integration processing. At the start of imaging, as shown in FIG. 3A, when the cat A faces the imaging device 100, the subject detection unit 170 detects the face of the cat A and identifies the subject area. At this time, the tracking area is not specified, and the processing by the distance determination unit 176 and the region integration unit 178 is not performed.

  The display control unit 182 superimposes a size index (here, a frame 190) determined in proportion to the size of the face on the specified subject area on the display unit 120 in a superimposed manner. Let

  After a subject area corresponding to the face of cat A is identified in a certain frame X, when several frames have passed, the result of the next subject identifying process is derived. As a result of the next subject specifying process, when a subject region is specified within a predetermined distance from the subject region specified in frame X, the subject specifying unit 170 tracks the same subject (the face of cat A). Judge.

  On the other hand, if the subject area is not specified within a predetermined distance from the subject area specified in frame X as a result of the next subject specifying process, the subject specifying unit 170 determines that the target has been lost (lost determination). processing). Lost means losing sight of the object. Here, the object indicates the subject area itself or a subject corresponding to the subject area (for example, the face of cat A).

  Then, a tracking process for a subject corresponding to the trigger is started by a predetermined trigger. Here, the predetermined trigger is, for example, a touch operation on the touch sensor (operation unit 122) at a position corresponding to the frame 190, detection of one or more faces, or a detected face is registered in advance. For example, it is determined that the face is identical.

  In addition, when a touch operation is performed at a position without the touch sensor frame 190, tracking processing is started with image data corresponding to an area of a predetermined size centered on the touched position as a tracking target. Also good.

  When the tracking process is started at a predetermined trigger, the subject detection unit 170 continues the subject detection process using the image data stored in the first area, while the color distribution extraction unit 172 and the tracking unit 174 Tracking processing is performed using the image data stored in the two areas.

  However, the tracking area specified by the tracking process is associated with the subject area that is the identification source of the tracking area as indicating the same subject. Then, the display control unit 182 does not superimpose the index on the tracking region until the subject region associated with the tracking region is lost, and the imaging control unit 180 associates the tracking region with the tracking region in the autofocus and exposure adjustment processing. The image data of the selected subject area is used. However, the subject region and the tracking region are approximately equal until the subject region is lost.

  Then, after several frames, as shown in FIG. 3B, it is assumed that the cat A turns away from the imaging apparatus 100 and the subject specifying unit 170 has lost the subject area. The display control unit 182 causes the display unit 120 to display a frame 192 that is an index indicating the tracking region on the display image data, when the subject region that is the identification source of the tracking region is lost. At this time, when the image data targeted for tracking processing is a face, the size of the frame 192 is determined according to the size of the face.

  Further, the size of the frame 192 may be set larger as the area of the comparison region in which the value of the histogram intersection is equal to or greater than a predetermined threshold is larger in the tracking region.

  Subsequently, it is assumed that after several frames have passed while the frame 192 that is an index indicating the tracking area is displayed, the cat A turns its face toward the imaging apparatus 100 again. At this time, when the frame 190 is simply superimposed on the face of the cat A detected by the subject detection unit 170, as shown in FIG. 4A, two indices (frame 190, frame 192) for the same cat A Will be superimposed.

  Therefore, in the present embodiment, region integration processing is performed. First, the distance determination unit 176 determines whether or not the subject area is included within a predetermined distance from the tracking area. When the subject area is included within the predetermined distance from the tracking area as shown in FIG. 4A, the distance determination unit 176 considers that the tracking area and the subject area are the same subject, and the tracking area. Is associated with the subject area. FIG. 4A shows an example of a predetermined distance range with a broken-line circle S. FIG.

  However, the distance determination unit 176 derives the size of the subject area and sets the predetermined distance in proportion to the size of the subject area. In the present embodiment, the predetermined distance is, for example, three times the size of the subject area. Here, the size of the subject area (here, the size of the cat's face) is, for example, the diameter of the subject area. However, as the size of the subject area, for example, the square root of the area of the subject area may be used, or the maximum width in the horizontal direction or the vertical direction of the subject area may be used. Here, the predetermined distance is three times the size of the subject area, but is preferably a value according to the characteristics of the subject, such as two times, four times, and five times.

  With this configuration, the distance determination unit 176 can perform selection of subject areas to be integrated with high accuracy based on a determination criterion corresponding to the size of the subject.

  As shown in FIG. 4B, in the same frame, a plurality of subject areas (a frame 190 superimposed on the face of cat A in FIG. 4B, a frame 196 superimposed on the face of cat B, a cat, When the distance determination unit 176 identifies the subject area closest to the tracking area, the distance determination unit 176 derives the predetermined distance, for example, the derived size. Is set to 3 times. Then, when there is no subject area within a predetermined distance from the tracking area, the distance determination unit 176 sets a predetermined distance and determines whether or not the subject area next to the tracking area is within the predetermined distance. FIG. 4B shows an example of a predetermined distance range with a broken-line circle S.

  Thus, the distance determination unit 176 sequentially performs determination processing from the subject area close to the tracking area until it is determined that the subject area is within the predetermined distance or all the subject areas are not within the predetermined distance. Do. In the case of FIG. 4B, it is assumed that the closest subject area is within a predetermined distance.

  In addition, by setting the range in which the subject detection unit 170 detects the subject to be within a predetermined range from the tracking region, for example, the number of subject regions that are subject to determination processing by the distance determination unit 176 is suppressed, The processing load may be reduced.

  Furthermore, the distance determination unit 176 may set the predetermined distance as a predetermined fixed value. With this configuration, the distance determination unit 176 can perform selection of subject areas to be integrated with a low processing load.

  When the subject area is included within a predetermined distance from the tracking area, the area integration unit 178 integrates information regarding the subject area into information regarding the tracking area. If it is not included within the predetermined distance, the region integration unit 178 does not perform any particular processing.

  Here, “integrating information” means, for example, overwriting the position information of the subject area on the position information of the tracking area. By overwriting the position information of the subject area on the position information of the tracking area, the tracking unit 174 can resume the tracking process from the position of the subject area, and the tracking area is close to the subject even if the subject area is lost again later. There is an increased possibility that the body and the like of the subject can be specified as the tracking region indicating the position.

  In addition, as a specific process of “integrating information”, an association process between a subject area and a tracking area is also included. The area integration unit 178 associates a subject area that was within a predetermined distance from the tracking area with the tracking area.

  Further, as a specific process of “integrating information”, in addition to overwriting the position information of the image data, the color distribution information may be derived again based on the luminance data and color difference data of the subject area. Good. With this configuration, the tracking unit 174 can improve the accuracy of subsequent tracking processing.

  When the information regarding the subject area is integrated into the tracking area information by the area integration unit 178, the display control unit 182 associates the display image data with the tracking area without superimposing an index on the tracking area. The indicator is superimposed on only the subject area and displayed on the display unit 120. That is, as shown in FIG. 3C, the frame 194 is superimposed only on the face of the cat A. At this time, the size of the frame 194 depends on the size of the detected subject area. Further, the tracking information is updated with the size of the frame 194.

  Similarly, when a plurality of subject areas are specified in the same frame shown in FIG. 4B, the display control unit 182 also integrates information related to the subject area into the tracking area information by the region integration unit 178. As shown in FIG. 4C, for cat A, a frame 194 is superimposed only on the face.

  As described above, the imaging apparatus 100 can avoid a situation in which a plurality of indices are displayed for the same subject on the display unit 120, and the operability of the touch operation to the touch sensor and the subject of the display unit 120 can be avoided. Visibility can be improved.

  In addition, when the information related to the subject area is integrated into the tracking area information by the area integration unit 178, the imaging control unit 180 controls autofocus or automatic exposure for the subject area associated with the tracking area. That is, the imaging control unit 180 refers to the image data of the subject area (cat A's face) that is the target of the information integration process on the tracking area (cat A's torso), and controls autofocus and automatic exposure. Do. With this configuration, the imaging control unit 180 can appropriately perform imaging control according to the face of the subject.

  After the region integration processing by the region integration unit 178, the tracking unit 174 resumes the tracking processing from the region indicated by the position information of the image data of the tracking region where the information is integrated. That is, the tracking unit 174 performs tracking processing on the subject area and its vicinity.

  Further, as shown in FIG. 3D, when the cat A is faced away from the imaging device 100 and the face of another cat B is reflected, the subject corresponding to the tracking area and the new face of the cat B is displayed. Since the cat A and the cat B can be identified as different subjects if the region is separated from the predetermined distance, the display control unit 182 displays the frame 192 indicating the tracking region of the cat A and the subject region of the cat B. The frame 196 shown is superimposed on the display image data and displayed on the display unit 120.

  As described above, the imaging apparatus 100 according to the present embodiment regards the subject area and the tracking area included within a predetermined distance as the same subject, and integrates information about each. Therefore, for example, imaging control such as autofocus and automatic exposure can be appropriately performed, and visibility and operability can be improved by integrating indexes displayed on the display unit 120.

  Also provided are a program that causes a computer to function as the imaging apparatus 100 and a recording medium such as a computer-readable flexible disk, magneto-optical disk, ROM, EPROM, EEPROM, CD, DVD, or BD on which the program is recorded. . Here, the program refers to data processing means described in an arbitrary language or description method.

(Imaging method)
Next, a series of flow of an imaging method for imaging a subject using the above-described imaging device 100 will be described. FIG. 5 is a flowchart showing the overall flow of the imaging method. Such processing is repeatedly executed at a predetermined frame period during imaging in an imaging mode in which subject identification processing and tracking processing are performed in parallel.

  As shown in FIG. 5, the imaging unit 124 images a subject and generates one frame of image data (S200). The image data is stored in the image storage unit 128 (S202).

  The central control unit 138 determines whether or not the subject area specifying process for the image data of the previous frame is continuing (S204). If it is continuing (YES in S204), the process proceeds to the tracking process determination (S216).

  When the subject area specifying process for the image data of the previous frame has been completed (NO in S204), the color distribution extraction unit 172 determines whether or not the subject area has been specified, that is, the subject area is specified in the subject area specifying process. It is determined whether there is image data that can be specified as (S206). If not specified (NO in S206), the process proceeds to the lost determination process (S210).

  When the subject area is specified (YES in S206), the color distribution extraction unit 172 derives the color distribution of the specified subject area and stores it in the main storage unit 136 in association with the subject area of the derivation source (S208). ).

  Subsequently, the subject specifying unit 170 performs a lost determination process (S210). In the lost determination process, as described above, when a subject area is specified within a predetermined distance as a result of the next subject specifying process for a subject area specified in a certain frame X as a result of the subject specifying process, The subject specifying unit 170 determines that the same subject is being tracked, and if not specified, the subject specifying unit 170 determines that the target has been lost.

  Then, the central control unit 138 stores the new one-frame image data stored in the image storage unit 128 in the first area of the main storage unit 136 (S212). The subject detection unit 170 starts subject identification processing (S214).

  Then, the central control unit 138 determines whether or not the tracking process has started (S216). If the tracking process has not started (NO in S216), the central control unit 138 can, for example, perform a touch operation on the touch sensor. It is determined whether or not a predetermined opportunity has occurred (S218). If it has not occurred (NO in S218), the process is terminated.

  When a predetermined trigger occurs (YES in S218), the tracking unit 174 starts tracking processing (S220). Then, the process proceeds to the imaging control process (S238).

  When tracking processing has already started (YES in S216), the central control unit 138 stores one frame of image data stored in the image storage unit 128 in the second area of the main storage unit 136 (S222). The tracking unit 174 performs tracking processing continued from the previous frame (S224). At this time, when the tracking area is specified by the tracking process, the tracking area is associated with the subject area which is the identification source of the tracking area as indicating the same subject.

  Subsequently, the central control unit 138 determines whether or not it is determined that the subject area associated with the tracking area is lost in the lost determination processing step S210 (S226). If not lost (NO in S226), the central control unit 138 sets the subject area as a target for index superposition or imaging control (S228). Then, the process proceeds to the imaging control process (S236).

  When the subject area is lost (YES in S226), the distance determination unit 176 determines whether the subject area is included within a predetermined distance from the tracking area (S230). If not included (NO in S230), the central control unit 138 sets the tracking area as a target for index superposition or imaging control (S232). Then, the process proceeds to the imaging control process (S236).

  When the subject area is included within the predetermined distance from the tracking area (YES in S230), the area integration unit 178 overwrites the position information of the subject area with the position information of the tracking area or within the predetermined distance from the tracking area. Information integration processing is performed such as associating the subject area with the tracking area (S234).

  Then, the imaging control unit 180 performs autofocus and automatic exposure processing on the set control target (S236), and the display control unit 182 displays the display image data on which the index is superimposed. It is displayed on 120 (S238).

  Even with the imaging method using the imaging apparatus 100 described above, imaging control such as autofocus and automatic exposure can be appropriately performed, and visibility and operability can be improved by integrating indexes displayed on the display unit 120. It becomes.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  Note that each step of the imaging method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or a subroutine.

  The present invention can be used in an imaging apparatus, an imaging method, and a program that perform various controls related to imaging using a predetermined area specified from image data obtained by imaging a subject.

DESCRIPTION OF SYMBOLS 100 ... Imaging device 120 ... Display part 124 ... Imaging part 170 ... Subject detection part 174 ... Tracking part 176 ... Distance judgment part 178 ... Area integration part 180 ... Imaging control part 182 ... Display control part

Claims (6)

An imaging device including an imaging unit that images a subject and generates image data,
A subject detection unit that identifies a subject region for the generated image data;
For the generated image data, a tracking unit that identifies a tracking region that is a tracking target;
A distance determination unit that determines whether or not the specified subject area is included within a predetermined distance from the specified tracking area;
An area integration unit that integrates information related to the subject area into information related to the specified tracking area when the specified subject area is included within the predetermined distance;
When information regarding the subject region is integrated into the tracking region information by the region integration unit, a control unit that performs imaging control using the subject region;
An imaging apparatus comprising:   2. The information processing apparatus according to claim 1, wherein when the information related to the subject area is integrated into the tracking area information by the area integration unit, the control unit controls autofocus or automatic exposure for the subject area. The imaging device described. A display unit for displaying an image based on the image data;
When the information related to the subject area is integrated into the tracking area information by the area integration unit, the control unit further performs control to superimpose an index only on the subject area and display the information on the display unit. The imaging apparatus according to claim 1 or 2.   The distance determination unit derives the size of the specified subject area, and sets the predetermined distance in proportion to the size of the subject area. The imaging device according to item. Capture the subject, generate image data,
For the generated image data, specify a subject area,
For the generated image data, specify a tracking area to be tracked,
Determining whether the identified subject area is included within a predetermined distance from the identified tracking area;
When the specified subject area is included within the predetermined distance, information related to the subject area is integrated into information related to the specified tracking area;
An imaging method, wherein when information related to the subject area is integrated into the information related to the specified subject area, imaging is controlled using the subject area. A computer that processes image data generated by imaging a subject,
A subject detection unit that identifies a subject region for the generated image data;
For the generated image data, a tracking unit that identifies a tracking region that is a tracking target;
A distance determination unit that determines whether or not the specified subject area is included within a predetermined distance from the specified tracking area;
An area integration unit that integrates information related to the subject area into information related to the specified tracking area when the specified subject area is included within the predetermined distance;
When information regarding the subject region is integrated into the tracking region information by the region integration unit, a control unit that performs imaging control using the subject region;
Program to make it function.

Images