JP2010193293A - Imaging apparatus, control method thereof and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent a frame to be displayed from being fluctuated even if a subject detection result is deviated by a motion or the like of a subject. <P>SOLUTION: An imaging apparatus includes: an imaging device 3 composed of a plurality of pixels; a first resize means 6 for converting the size of an image signal captured by the imaging device 3 into an image size for subject detection; a subject detecting means 7 for detecting subject information from the image for subject detection resized by the first resize means 6; a second resize means 10 for converting the size of the image signal captured by the imaging device 3 into a display image size; and a subject detection frame display control means 11 for generating information of a frame to be superimposed on a display image based on the subject information detected by the subject detecting means 7, the image size for subject detection resized by the first resize means 6 and the display image size resized by the second resize means 10. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus such as a video camera that performs subject detection, a control method thereof, and a program.

  In recent years, in an imaging apparatus, a technique for detecting the position and size of a specific subject such as a human face by an image recognition algorithm has been put into practical use, and is used for, for example, AF (autofocus) control. In other words, if a distance measurement frame that matches the position and size of a specific subject is set, an AF evaluation value is generated and AF control is performed, focus control focusing on the specific subject can be performed.

  At the same time, the user can be informed that the specific subject has been detected by superimposing a frame on the image displayed on the liquid crystal monitor to indicate how the specific subject is currently detected. A specific subject is recognized (for example, Patent Document 1).

JP 2003-107335 A

  In conventional imaging apparatuses, the size of an image for subject detection is often different from the size of an image displayed on a liquid crystal monitor. For this reason, if the frame is superimposed on the display image using the result of subject detection as it is, the size of the frame will not match. Therefore, the result of subject detection is converted according to the size of the display image, and a frame is superimposed.

  However, if the frame size is converted to the size of the display image and the frame is superimposed, if the subject detection result slightly shifts, the shift is the ratio between the size of the subject detection image and the size of the display image. Will be doubled and superimposed.

  FIG. 9 shows how the difference in the result of subject detection is doubled. FIG. 9A is an image of a pixel size on a 28 × 28 sensor, where the subject is located in black.

  FIG. 9B is a diagram showing a result of subject detection after reducing the image of FIG. 9A to 1/3 for subject detection. Hp and Vp are horizontal and vertical position data of the subject, and Hw and Vw are horizontal and vertical size data of the subject.

  FIG. 9C is a diagram illustrating a result of superimposing a resized frame by reducing the image of FIG. 9A to 1/2 for a display image. The resizing of the frame is calculated using, for example, the calculations shown in the following formulas (1) to (4).

  Hp ′ and Vp ′ are horizontal and vertical position data of the frame after resizing, and Hw ′ and Vw ′ are horizontal and vertical size data of the frame after resizing. H1 and V1 are the horizontal and vertical image sizes of the subject detection image, and H2 and V2 indicate the horizontal and vertical image sizes of the display image. In the drawing, values after the decimal point generated during the Hp ′ and Vp ′ calculations are rounded down, and values after the decimal point generated during the Hw ′ and Vw ′ calculations are rounded up.

  On the other hand, FIG. 9D is an image of a pixel size on a 28 × 28 sensor, where a subject is painted in black, and is one frame after FIG. 9A. In the image, the subject has moved one pixel to the left due to the movement of the subject.

  FIG. 9E is a diagram showing the result of subject detection after reducing the image of FIG. 9D to 1/3 for subject detection. Hp and Vp are horizontal and vertical position data of the subject, and Hw and Vw are horizontal and vertical size data of the subject.

  FIG. 9F is a diagram illustrating a result of superimposing a resized frame by reducing the image of FIG. 9A to 1/2 for a display image.

  Here, when FIGS. 9 (a), (b), and (c) are compared with FIGS. 9 (d), (e), and (f), in the images ((a) and (d)) on the sensor. Since the subject has moved, there is a difference between the subject detection results ((b) and (e)). As a result, the position and size deviation of the frame to be superimposed on the display image is doubled ((c) and (f)), and there is a problem that the frame fluctuates for each frame when the images are continuous. In particular, when the ratio of the display image size to the subject detection image size is large, the position and size of the frame to be superimposed on the display image change greatly, and there is a problem that the frame varies greatly from frame to frame. .

  The present invention has been made in view of the above situation, and an object of the present invention is to prevent a displayed frame from fluctuating even when a subject detection result shifts due to a subject's movement or the like.

An image pickup apparatus according to the present invention includes an image pickup device including a plurality of pixels, a first resizing unit that converts a size of an image signal acquired by the image pickup device into a subject detection image size, and the first resizing unit. Subject detection means for detecting subject information from the resized subject detection image, second resizing means for converting the size of the image signal acquired by the image sensor into a display image size, and detection by the subject detection means. Subject that generates information on a frame to be superimposed on the display image based on the subject information, the subject detection image size resized by the first resizing unit, and the display image size resized by the second resizing unit And a detection frame control means.
An imaging apparatus control method according to the present invention is an imaging apparatus control method including an imaging device including a plurality of pixels, and the step of converting the size of an image signal acquired by the imaging device into a subject detection image size. Detecting subject information from the subject detection image, converting the size of the image signal acquired by the image sensor into a display image size, the detected subject information, and the subject detection image size. And generating information on a frame to be superimposed on the display image based on the display image size.
The program of the present invention is a program for controlling an image pickup apparatus including an image pickup device composed of a plurality of pixels, and a process for converting the size of an image signal acquired by the image pickup device into an object detection image size. , Processing for detecting subject information from the subject detection image, processing for converting the size of an image signal acquired by the image sensor into a display image size, the detected subject information, the subject detection image size, and Based on the display image size, a computer is caused to execute processing for generating information on a frame to be superimposed on the display image.

  According to the present invention, it is possible to prevent the displayed frame from fluctuating even when the subject detection result is shifted due to the movement of the subject.

It is a block diagram showing an example of composition of an imaging device concerning a 1st embodiment. It is a block diagram which shows the structural example of the to-be-detected frame display control means in 1st Embodiment. It is a figure for demonstrating the effect of 1st Embodiment. 6 is a flowchart illustrating a control operation of a subject detection frame display control unit according to the first embodiment. It is a block diagram which shows the structural example of the imaging device which concerns on 2nd Embodiment. It is a block diagram which shows the structural example of the to-be-detected frame display control means in 1st Embodiment. 6 is a flowchart illustrating a control operation of a subject detection frame display control unit according to the first embodiment. It is a characteristic view which shows the example of a characteristic of electronic zoom magnification, (alpha), and (beta). It is a figure which shows a mode that the shift | offset | difference of a to-be-photographed object detection result is doubled.

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.
(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment to which the present invention is applied. In FIG. 1, reference numeral 1 denotes a lens, and reference numeral 2 denotes lens driving means for driving the lens 1. Reference numeral 3 denotes an image sensor composed of a plurality of pixels, and 4 denotes an image sensor driving means for driving the image sensor 3. Reference numeral 5 denotes an analog front end unit (referred to as AFE) including a CDS circuit and an A / D converter.

  Reference numeral 7 denotes a subject detection means. Reference numeral 6 denotes first resizing means for converting the image signal acquired by the image sensor 3 and output from the AFE 5 into the size of the subject detection image used by the subject detection means 7. Reference numeral 8 denotes an AF evaluation value generation unit that generates an AF evaluation value based on the image signal from the AFE 5 and the subject detection information from the subject detection unit 7.

  Reference numeral 9 denotes camera signal processing means. Reference numeral 10 denotes a second resizing unit that converts an image signal acquired by the image pickup device 3 and output from the camera signal processing unit 9 into an image size to be displayed on the display device 16.

  Reference numeral 11 denotes subject detection frame display control means which functions as subject detection frame control means in the present invention. An image superimposing unit 12 superimposes the frame from the subject detection frame display control unit 11 on the image signal from the second resizing unit 10. Reference numeral 14 denotes an image recording medium that can be detached from the imaging apparatus. Reference numeral 13 denotes recording processing means for recording the image signal subjected to the inter-frame predictive encoding process on the recording medium 14.

  Reference numeral 16 denotes a display device that displays an image signal subjected to camera signal processing. Reference numeral 15 denotes display processing means for displaying an image on the display device 16. 17 is a bus. A system control unit 18 controls the entire image pickup apparatus.

  A shooting operation in the imaging apparatus configured as described above will be described. First, the lens 1 is driven by a driving pulse from the lens driving means 2 controlled by the system control unit 18 to form a subject image on the image sensor 3. The image sensor 3 captures an image using a drive pulse from the image sensor driving unit 4 controlled by the system control unit 18.

  The analog image signal output from the image pickup device 3 is removed from the clock synchronization noise by the CDS circuit inside the AFE 5 and converted into a digital image signal by the A / D converter inside the AFE 5. The digital image signal from the AFE 5 is converted into an image signal of luminance color difference by the camera signal processing means 9.

  The digital image signal from the AFE 5 is converted by the first resizing unit 6 into an image signal having a size suitable for subject detection by the subject detecting unit 7. The subject detection means 7 detects a subject from the image signal converted by the first resizing means 6, and uses subject position data and subject size data as AF evaluation value generation means 8 and a bus 17 as subject detection information. To the system control unit 18. The AF evaluation value generation unit 8 generates an AF evaluation value using the subject detection information from the subject detection unit 7 and the image signal from the AFE 5.

  The second resizing means 10 converts the image signal from the camera signal processing means 9 into a display image size and outputs it to the image superimposing means 12, converts it into a recording image size, and outputs it to the recording processing means 13. .

  The subject detection frame display control unit 11 determines the frame to be superimposed on the display image from the subject information detected by the subject detection unit 7 and the ratio of the display image size to the subject detection image size (hereinafter referred to as “size ratio”). As information, frame position data, frame size data, and frame width data are calculated.

  The image superimposing unit 12 superimposes a frame based on the frame information from the subject detection frame display control unit 11 on the image signal from the second resizing unit 10.

  The display processing means 15 converts the image signal from the image superimposing means 12 into a signal suitable for the display device 16 (for example, an NTSC analog signal or the like) and displays it on the display device 16. The recording processing unit 13 compresses and encodes the image signal from the camera signal processing unit 9, converts it into data suitable for the recording medium 14 (for example, file system data having a hierarchical structure), and records the data on the recording medium 14.

  The system control unit 18 controls the lens driving unit 2, the image sensor driving unit 4, the camera signal processing unit 9, the recording processing unit 13, and the display processing unit 15.

  Next, the configuration of the subject detection frame display control unit 11 will be described with reference to FIG. Reference numeral 101 denotes frame resizing means, which uses the size ratio from the system control unit 18 to calculate the position and size of the frame to be superimposed on the display image using, for example, the calculations shown in the above-described equations (1) to (4). To do.

  Reference numeral 102 denotes a frame width calculation unit that calculates a frame width according to the size ratio from the system control unit 18 and outputs the frame width to the image superimposing unit 12.

  Here, for example, when the ratio between the display image size and the subject detection image size is 10: 1, if the subject detection result is shifted by 1 pixel, the frame resizing result is shifted by 10 pixels. On the other hand, when the ratio between the display image size and the subject detection image size is 1:10, even if the subject detection result is shifted by one pixel, the frame resizing result is shifted by only 0.1 pixel. Therefore, the frame width calculation means 102 sets the frame width using, for example, calculations shown in the following equations (5) and (6).

  Here, Wh and Wv are the horizontal and vertical widths of the display frame, H1 and V1 are the horizontal and vertical image sizes of the subject detection image, and H2 and V2 are the horizontal and vertical images of the display image. Indicates size. In addition, α and β represent the number of pixels that are allowed to vary. For example, when one pixel deviation of the subject detection result is allowed, by setting α = 1, one pixel deviation of the subject detection result is allowed. The frame width is set.

  FIG. 3 shows a state in which the frame width is set using the calculations shown in equations (5) and (6). 3 (a), (b), (d), and (e) are the same as FIGS. 9 (a), (b), (d), and (e), and FIGS. 3 (c) and (f) As shown in FIG. 9, the width of the frame is overlapped with two pixels with respect to (c) and (f) of FIG.

  As described above, by setting the width of the frame in accordance with the ratio of the display image size to the subject detection image size, fine fluctuation of the frame does not occur. That is, by performing control to increase the width of the frame as the display image size becomes larger than the image size for subject detection, it is possible to absorb minute fluctuations in the frame.

  Next, the control operation of the subject detection frame display control unit 11 will be described with reference to the flowchart of FIG. First, in step S101, position data and size data are acquired as a subject detection result from the subject detection means 7, and the process proceeds to step S102. In step S102, the position data and size data of the frame to be displayed are calculated using, for example, the calculations shown in the above formulas (1) to (4).

  In step S103, the width of the currently displayed frame is compared with the frame position data and size data calculated in step S102, and the process proceeds to step S104. In step S104, the width of the currently displayed frame is compared with the frame position data and size data calculated in step S102. As a result of the comparison, if the position data and size data of the frame calculated in step S102 exceed the width of the currently displayed frame, the process proceeds to step S105. If not, the process proceeds to step S108. Will not be updated.

  In step S105, the frame position data and size data calculated in step S102 are processed so as to match the pixel position of the display image by performing processing such as truncation of decimals, and the process proceeds to step S106. Next, in step S106, the width of the frame is calculated using, for example, the calculations shown in the above-described equations (5) and (6).

  In step S107, the frame position, size, and width calculated in steps S105 and S106 are output to the image superimposing means 12, and the frame is updated.

  As described above, by providing the frame with a width, it is possible to prevent the frame from fluctuating for each image even when the subject detection result is shifted due to a factor such as the movement of the subject.

(Second Embodiment)
Next, a second embodiment will be described. The imaging apparatus according to the second embodiment includes electronic zoom control means 19 as shown in FIG. The configuration other than the electronic zoom control means 19 is substantially the same as that in the first embodiment, and the same reference numerals are given to the same configurations, and detailed description thereof is omitted.

  The second resizing means 10 converts the image signal from the camera signal processing means 9 into a display image size and a recording image size, and performs electronic zoom processing according to the electronic zoom magnification information from the electronic zoom control means 19. Output to the image superimposing means 12 and the recording processing means 13.

  Next, the configuration of the subject detection frame display control unit 11 will be described with reference to FIG. Reference numeral 103 denotes frame resizing means, which uses the size ratio from the system control unit 18 and the electronic zoom magnification to calculate the position and size of the frame to be superimposed on the display image, for example, by the following equations (7) to (10). Use to calculate.

  A frame width calculation unit 104 calculates the frame width according to the size ratio and the electronic zoom magnification from the system control unit 18 and outputs the frame width to the image superimposing unit 12.

  Here, for example, when the ratio between the display image size and the subject detection image size is 10: 1 and the magnification of the electronic zoom is double, if the subject detection result is shifted by one pixel, the frame resizing result is 20 Pixel shift will occur. On the other hand, when the ratio between the display image size and the subject detection image size is 1:10 and the magnification of the electronic zoom is double, even if the subject detection result is shifted by one pixel, the frame resizing result is 0. .05 pixels will be displaced. Therefore, the frame width calculation unit 104 sets the frame width using, for example, calculations shown in the following equations (11) and (12).

  Here, Wh ′ and Wv ′ are the horizontal and vertical widths of the frame to be displayed in consideration of the electronic zoom magnification, and H1 and V1 are the horizontal and vertical image sizes of the subject detection image, and H2 and V2 Indicates the horizontal and vertical image sizes of the display image. In addition, α and β represent the number of pixels that allow variation. In other words, in addition to the display image size becoming larger than the subject detection image size, fine control of the frame can be absorbed by performing control to increase the frame width as the electronic zoom magnification increases. it can.

  Next, the control operation of the subject detection frame display control unit 11 will be described with reference to the flowchart of FIG. First, in step S201, position data and size data are acquired as a subject detection result from the subject detection means 7, and the process proceeds to step S202. Next, in step S202, the electronic zoom magnification is obtained from the electronic zoom control means 19, and the process proceeds to step S203. Next, in step S203, for example, the position data and size data of the frame to be displayed are calculated using the calculations shown in the above equations (7) to (10).

  In step S204, the width of the currently displayed frame is compared with the position data and size data of the frame calculated in step S203, and the process proceeds to step S205. In step S205, the width of the currently displayed frame is compared with the frame position data and size data calculated in step S203. As a result of the comparison, if the position data and size data of the frame calculated in step S203 exceed the width of the currently displayed frame, the process proceeds to step S206. If not, the process proceeds to step S209. Will not be updated.

  In step S206, processing such as truncation of the fractional number is added to the frame position data and size data calculated in step S203 so as to match the pixel position of the display image, and the process proceeds to step S207. Next, in step S207, for example, the width of the frame is calculated using the calculations shown in the above-described equations (11) and (12).

  In step S208, the frame position, size, and width calculated in steps S206 and S207 are output to the image superimposing unit 12 to update the frame.

  In this embodiment, the electronic zoom process is applied only to the display image or the recording image, and the electronic zoom process is not applied to the subject detection image. Zoom processing may be added. In that case, instead of considering the electronic zoom magnification as in the above-described equations (11) and (12), in the above-described equations (5) and (6), α and β are, for example, the characteristic lines shown in FIG. By controlling in this way, the same effect can be obtained.

  As described above, by controlling the width of the frame according to the magnification of the electronic zoom, even if the deviation of the subject detection result due to the subject's movement or the like is increased by the magnification of the electronic zoom, for each image It is possible to prevent the frame from fluctuating finely.

  The object of the present invention can also be achieved by supplying a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus. In this case, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, the functions of the above-described embodiments are not limited to being realized by executing the program code read by the computer. For example, an OS (basic system or operating system) running on the computer performs part or all of the actual processing based on an instruction of the program code, and the functions of the above-described embodiments are realized by the processing. May be.

  Further, the program code read from the storage medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. In this case, after being written in the memory, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the function of the above-described embodiment is performed by the processing. Is realized.

DESCRIPTION OF SYMBOLS 1 ... Lens 2 ... Lens drive means 3 ... Imaging element 4 ... Imaging element drive means 5 ... AFE
6. First resizing means 7 Subject detecting means 8 AF evaluation value generating means 9 Camera signal processing means 10 Second resizing means 11 Subject detection display control means 12 Image superposition Means 13 ... Recording processing means 14 ... Recording medium 15 ... Display processing means 16 ... Display device 17 ... Bus 18 ... System controller 19 ... Electronic zoom control means

Claims (7)

An image sensor comprising a plurality of pixels;
First resizing means for converting the size of the image signal acquired by the image sensor into a subject detection image size;
Subject detection means for detecting subject information from the subject detection image resized by the first resizing means;
Second resizing means for converting the size of the image signal acquired by the image sensor into a display image size;
Based on the subject information detected by the subject detection means, the subject detection image size resized by the first resizing means, and the display image size resized by the second resizing means, it is superimposed on the display image. An imaging apparatus comprising: subject detection frame control means for generating frame information.   The imaging apparatus according to claim 1, wherein the subject detection frame control unit sets the width of the frame based on a ratio of the display image size to the subject detection image size.   The imaging apparatus according to claim 2, wherein the subject detection frame control unit increases the width of the frame when the ratio is large. An electronic zoom control means for giving an electronic zoom magnification to the second resizing means;
4. The subject detection frame control unit further generates information on a frame to be superimposed on the display image based on an electronic zoom magnification from the electronic zoom control unit. The imaging device described in 1. The subject detection means detects subject position data and size data from the subject detection image as the subject information,
The subject detection frame control means does not update the frame when the updated position data and size data of the subject do not exceed the frame superimposed on the display image. Item 5. The imaging device according to any one of Items 1 to 4. A method for controlling an imaging apparatus including an imaging device including a plurality of pixels,
Converting the size of the image signal acquired by the image sensor into a subject detection image size;
Detecting subject information from the subject detection image;
Converting the size of the image signal acquired by the imaging device into a display image size;
And a step of generating information of a frame to be superimposed on the display image based on the detected subject information, the subject detection image size, and the display image size. A program for controlling an image pickup apparatus including an image pickup device including a plurality of pixels,
A process of converting the size of the image signal acquired by the image sensor into a subject detection image size;
Processing for detecting subject information from the subject detection image;
A process of converting the size of the image signal acquired by the image sensor into a display image size;
A program for causing a computer to execute processing for generating information on a frame to be superimposed on the display image based on the detected subject information, the subject detection image size, and the display image size.

Images