JP4894708B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus having a moving image shooting function.

2. Description of the Related Art Conventionally, an electronic camera having a moving image shooting function that can generate still image data during moving image shooting is known. As an example, Patent Document 1 discloses a configuration in which identification information is added to a moving image frame when a still image shooting instruction is given during moving image shooting, and a frame with the identification information is output when a still image is read. ing.
JP 2006-42241 A

  By the way, in the prior art, only one frame of a moving image is output as a still image. For this reason, in the frame output as a still image, the main subject is not necessarily photographed with a preferable composition, and there is room for improvement in that the performance as a still image is often inferior.

  The present invention is to solve the above-mentioned problems of the prior art. An object of the present invention is to provide means capable of acquiring a still image having a more preferable composition at the time of moving image shooting.

An imaging apparatus according to a first invention includes an imaging device, a memory, a moving image data generation unit, a recording interface, an input unit, a subject recognition unit, and a trimming processing unit. The imaging device images a subject at predetermined intervals and outputs data of each image. The memory temporarily holds image data. The moving image data generation unit generates moving image data from a plurality of images. The recording interface records moving image data on a storage medium. The input unit accepts a still image recording instruction input. The subject recognition unit detects a subject to be recognized from the image. The trimming processing unit trims a partial area including a subject to be recognized from an image captured at the time of inputting an instruction, and generates data of a trimmed image in which the subject to be recognized is arranged with a predetermined composition. The trimming processing unit switches between a first process for trimming an image vertically and a second process for trimming an image horizontally based on the amount of margin from the edge of the image to the subject to be recognized.

  In a second aspect based on the first aspect, the resolution of the image is set higher than the resolution of one frame of the moving image data.

  In a third aspect based on the first or second aspect, the trimming processing unit determines a trimming position so that a subject to be recognized is positioned at an intersection of vertical and horizontal lines dividing the trimmed image by the golden ratio. To do.

  In a fourth aspect based on any one of the first to third aspects, the trimming processing unit determines a trimming range based on a size of a subject to be recognized in an image.

  According to the present invention, it is possible to acquire a trimmed image in which a partial area including a subject to be recognized is trimmed from an image in response to a still image recording instruction input during moving image shooting, and the recognition target is arranged in a predetermined composition.

  FIG. 1 is a block diagram showing the configuration of the electronic camera of this embodiment. The electronic camera of this embodiment has a still image shooting function and a moving image shooting function.

  The electronic camera includes an image pickup optical system 11, an image pickup element 12, an image pickup element drive circuit 13, a signal processing circuit 14, a first frame memory 15, a second frame memory 16, a moving image data generation unit 17, A RAM 18, a still image data generation unit 19, a recording I / F 20, a monitor 21, an input unit 22, a CPU 23 and a bus 24 are included. Here, the moving image data generation unit 17, the still image data generation unit 19, the recording I / F 20, the monitor 21, and the CPU 23 are connected via a bus 24. The image sensor driving circuit 13 and the input unit 22 are each connected to the CPU 23.

  The imaging optical system 11 includes a plurality of lens groups including a zoom lens and a focusing lens. For simplicity, the imaging optical system 11 is illustrated as a single lens in FIG.

  The image sensor 12 photoelectrically converts a subject image by a light beam that has passed through the imaging optical system 11 to generate an analog image signal. The output of the image sensor 12 is connected to a signal processing circuit 14. Note that the image sensor 12 of the present embodiment is composed of a CMOS sensor that can read out an image of 3264 × 2448 pixels at a frame rate of 60 fps.

  The image sensor drive circuit 13 supplies timing pulses to the image sensor 12 based on instructions from the CPU 23. The charge accumulation time of the image sensor 12 and the signal readout timing are adjusted by the timing pulse of the image sensor drive circuit 13.

  The signal processing circuit 14 includes therein a front analog front end circuit (AFE) that performs analog signal processing on the output of the image sensor 12, and a subsequent digital front end circuit (DFE) that performs digital signal processing. ing. The AFE of the signal processing circuit 14 performs correlated double sampling, image signal gain adjustment, and A / D conversion of the image signal. The DFE of the signal processing circuit 14 performs digital data clamp correction, dark current component offset correction, and the like. The output of the signal processing circuit 14 is connected to the first frame memory 15 and the second frame memory 16, respectively.

  The first frame memory 15 and the second frame memory 16 are buffer memories that temporarily hold image data generated during moving image shooting. The first frame memory 15 is connected to the moving image data generation unit 17 and is used to generate moving image data. The second frame memory 16 is connected to a still image data generation unit 19 and is used to generate trimmed image data, which will be described later.

  The moving image data generation unit 17 is an ASIC that performs image processing relating to a moving image. The moving image data generation unit 17 executes various image processing (color interpolation processing, gradation conversion processing, contour enhancement processing, white balance adjustment, etc.) on the image data for one frame in the first frame memory 15. To do. The moving image data generation unit 17 executes resolution conversion processing for each frame and compression processing (for example, Motion-JPEG, Motion-JPEG2000, MPEG-2, etc.) in a predetermined image format, Generate. The moving image data generation unit 17 also generates display image data on the monitor 21.

  The RAM 18 is a non-volatile storage medium that stores feature information used when subject recognition is performed, and is connected to the still image data generation unit 19. Note that in the electronic camera of this embodiment, it is possible to register any object including a human, an animal, a building, a vehicle, or the like as a subject to be recognized.

  Here, the feature information is composed of image data of a subject as an example. The feature information may be data indicating parameters such as the contour component, brightness, color difference, and contrast ratio of the captured image. Furthermore, when the subject to be recognized is a human face, the position of the facial feature point, the relative distance of each feature point, and the like can be used as the feature information.

  The still image data generation unit 19 is an ASIC that performs image processing related to a still image. The still image data generation unit 19 executes various types of image processing and compression processing on image data for one frame in the first frame memory 15. The still image data generation unit 19 functions as a subject recognition unit 25 and a trimming processing unit 26.

  The subject recognition unit 25 recognizes a main subject included in the image based on feature information prepared in advance in the RAM 18. As an example, the subject recognition unit 25 executes matching processing for analyzing the subject of the image based on the feature information. At this time, the subject recognizing unit 25 executes the matching process by paying attention to the commonality of patterns such as the luminance component, color difference component, contour component, and contrast ratio of the image. Next, the subject recognizing unit 25 obtains the similarity between the subject of the image and the feature information based on the result of the matching process. Then, the subject recognition unit 25 determines that a subject corresponding to the feature information exists in the image when the similarity is equal to or greater than the threshold. Here, the subject recognizing unit 25 can also perform person authentication with the electronic camera by registering the feature information of each face in the RAM 18 in advance. The subject recognition unit 25 can also perform a known face detection process for detecting a human face area without performing personal authentication.

  Based on the output of the subject recognition unit 25, the trimming processing unit 26 performs a trimming process for cutting out a partial region including a recognition target subject from the image. Note that the trimming processing unit 26 can switch between a first process for trimming an image vertically and a second process for trimming an image horizontally.

  Two connectors for connecting the storage medium 27 are formed in the recording I / F 20. The recording I / F 20 executes data writing / reading with respect to the storage medium 27 connected to the connector. In particular, the recording I / F 20 of the present embodiment can be simultaneously written into the moving image recording storage medium 27 and the still image recording storage medium 27. The storage medium 27 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 27.

  The monitor 21 displays a screen instructed by the CPU 23. For example, the monitor 21 functions as an electronic viewfinder by displaying a moving image of an image captured during moving image recording in real time. Further, the monitor 21 can display a menu screen that allows input in GUI (Graphical User Interface) format. Here, on the above menu screen, the user can specify items such as (1) designation of a subject to be recognized, (2) designation of resolution of a still image, and (3) designation of composition of a still image to the CPU 23. Can be set. Note that the CPU 23 executes all processes related to input and display on the menu screen.

  The input unit 22 includes, for example, a cross-shaped cursor key, a determination button, a capture button, and the like. The input unit 22 receives various inputs of the electronic camera from the user. For example, the input unit 22 receives an input operation on the menu screen, an operation mode switching operation, and the like. In particular, the capture button receives an input of a still image capturing timing from the user during moving image shooting.

  The CPU 23 is a processor that comprehensively controls each unit of the electronic camera. As an example, the CPU 23 executes AF calculation, AE calculation, and the like by a known means at the time of shooting.

  Hereinafter, with reference to the flowchart of FIG. 2, an operation example of the electronic camera at the time of moving image shooting in the present embodiment will be described. In the example of FIG. 2, the description will be made on the assumption that the first process for trimming an image vertically is selected by default.

  Step 101: When receiving a moving image shooting start instruction input from the user, the CPU 23 drives the image sensor 12 to start moving image shooting.

  Specifically, the image sensor 12 outputs an image of 3264 × 2448 pixels at a frame rate of 60 fps at the time of moving image shooting. The output from the image sensor 12 passes through the signal processing circuit 14 in a pipeline manner and is buffered in the first frame memory 15 and the second frame memory 16, respectively. The moving image data generation unit 17 generates moving image data conforming to a predetermined moving image format from the image data in the first frame memory 15. At this time, the moving image data generation unit 17 adjusts the resolution of one frame of the moving image to 1920 × 1080 pixels by cutting the top and bottom of the image so that the aspect ratio is 16: 9 and converting the resolution of the image. . The moving image data is finally recorded on the moving image storage medium 27 via the recording I / F 20.

  At the time of moving image shooting, an image captured by the image sensor 12 is displayed on the monitor 21 under the control of the CPU 23. FIG. 3 is a diagram illustrating an example of the display screen of the monitor 21 during moving image shooting. On the display screen of the monitor 21 of the present embodiment, the entire image (aspect ratio 4: 3) captured by the image sensor 12 and the image range of the moving image data (aspect ratio 16: 9) are shown. Accordingly, the user can determine the shooting timing of the still image based on the entire image while confirming the range of the moving image on the display screen.

  Step 102: The CPU 23 determines whether or not there is an input of still image shooting using the capture button. When there is the input (YES side), the process proceeds to S103. On the other hand, if there is no input (NO side), the process proceeds to S112.

  Step 103: In this case, the CPU 23 starts generating still image data. Then, the CPU 23 designates a frame (3264 × 2448 pixels) captured when the capture button is pressed among the images stored in the second frame memory 16 as a processing target image.

  Step 104: The subject recognition unit 25 executes subject recognition processing for the processing target image (S103). In the present embodiment, the following description is given on the assumption that the face of a specific person is authenticated as the subject to be recognized.

  Step 105: In S104, the CPU 23 determines whether or not a recognition target subject has been detected from the processing target image (S103). If the subject to be recognized is detected (YES side), the process proceeds to S106. On the other hand, if the subject to be recognized cannot be detected (NO side), the process proceeds to S111.

  Step 106: The trimming processing unit 26 determines the size of the trimming range of the processing target image (S103) based on the size of the recognition target subject. At this time, when the ratio of the size of the subject to be recognized and the trimming range is designated in advance, the trimming processing unit 26 designates the size of the trimming range according to the ratio.

  As an example, when a person is a recognition target and in the first processing for trimming an image vertically long, the trimming processing unit 26 determines the size of the trimming range as follows. In general, it is empirically known that when the main objects in the screen are arranged in accordance with the so-called golden ratio of about 1: 1.618, the composition brings a sense of stability to the screen. In particular, in the case of a portrait of a person, the size of the face is also an element for determining the composition.

Therefore, the trimming processing unit 26 according to the present embodiment determines the Y value so that the height (Y _F ) of the subject's face is 38.2% with respect to the vertical dimension (Y) of the trimming range. . That is, if the face height (Y _F ) is 1, the vertical dimension (Y) of the trimming range is 2.618 times (1 / 0.382). Thereafter, the trimming processing unit 26 obtains the lateral dimension (X) of the trimming range from the value of Y according to a predetermined aspect ratio (4: 3, 3: 2, 16: 9, etc.) (see FIG. 4).

  Step 107: The trimming processing unit 26 determines a trimming range of the processing target image (S103) so that the recognition target subject is contained in a predetermined composition. As an example, the trimming processing unit 26 determines the trimming range so that the subject to be recognized is positioned at the intersection of the vertical and horizontal lines dividing the trimmed image by the golden ratio.

  With reference to FIG. 4, the case of the first process in which a person is a recognition target and an image is cropped vertically will be specifically described. The trimming processing unit 26 is a line in which the center point of the face (for example, the position between the eyebrows and the position of the center of gravity of the face region) divides the trimmed image vertically and horizontally at a ratio of about 1: 1.618 (that is, X and Y). The trimming range is determined so as to be positioned on the intersections of the lines that divide each of the lines into 38.2% and 61.8%. In the example of FIG. 4, the composition is set so that the center of the face is located at the coordinates (0.382X, 0.382Y) starting from the upper left of the trimmed image.

  Step 108: In the composition of the trimming range in S107, the trimming processing unit 26 can secure a margin amount from the edge of the processing target image to the recognition target subject (that is, whether the trimming range fits in the processing target image). ) Determine whether or not. If the above requirement is satisfied (YES side), the process proceeds to S110. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S109.

  In the electronic camera of the present embodiment, still image shooting is performed as necessary while shooting a movie. Therefore, when the main subject of movie shooting is different from the authentication target, the authentication target subject is located in the corner of the image. There are also many. In actual shooting, there may be a case where the image is not captured to the periphery of the subject to be recognized due to the restriction between the focal length of the lens and the distance to the subject. For example, with a non-wide-angle lens, it is necessary to secure a distance to the subject of several meters to several tens of meters in order to sufficiently capture the surroundings of a subject such as a person. In many cases, a sufficient distance to the subject cannot be secured.

  In such a case, the trimming range may protrude from the processing target image, and if trimming is performed as it is, a part of the image is cut and the appearance is deteriorated. Therefore, the trimming processing unit 26 in S108 determines whether or not the trimming range fits within the processing target image.

  Specifically, the trimming processing unit 26 obtains a margin amount to be secured around the recognition target subject from the composition of the trimming range (the position of the subject to be authenticated) and the size of the recognition target subject. Next, the trimming processing unit 26 obtains the length (number of pixels) from the recognition target subject to the edge of the image in the processing target image. Then, the trimming processing unit 26 determines that the trimming range is within the image to be processed when the length is equal to or larger than the margin amount.

  As an example, consider the case of the first processing in which a person is a recognition target and an image is cropped vertically with reference to FIG. In this case, the margin amount from the upper end of the person's face to the upper end of the screen corresponds to 19.1% of the vertical dimension (Y) of the trimming range. Therefore, the trimming processing unit 26 uses 19.1Y as a threshold amount as the margin amount on the upper end side of the screen, and determines whether or not the length from the upper end of the human face to the upper end of the image in the processing target image is equal to or greater than the threshold value. Of course, the trimming processing unit 26 performs the same determination for the lower side of the screen and the left and right sides of the screen.

  Step 109: The trimming processing unit 26 switches to the second process of trimming the image horizontally and resets the trimming range. Also in this case, the trimming processing unit 26 determines the trimming range so that the subject to be recognized is positioned at the intersection of the vertical and horizontal lines dividing the horizontally long trimmed image by the golden ratio (see FIG. 5).

  If sufficient margins cannot be secured around the recognition target even after switching to the second process in S109, the CPU 23 may shift the process to S111 (the flowchart in FIG. 2 illustrates this case). (Omitted).

  Step 110: The trimming processor 26 trims the image to be processed (S103) within the trimming range determined in S107 or S109, and generates still image data. At this time, the still image data generation unit 19 may perform resolution conversion on the trimmed image to adjust it to a predetermined number of pixels. Thereafter, the still image data is recorded in the still image storage medium 27 via the recording I / F 20. In addition, after the process of S110, it transfers to S112.

  Step 111: The still image data generation unit 19 executes image resolution conversion without performing trimming, and generates still image data. That is, in this case, a low-resolution image having the same composition as the processing target image (S103) is still image data. Thereafter, the still image data is recorded in the still image storage medium 27 via the recording I / F 20.

  Step 112: The CPU 23 determines whether or not a moving image shooting end input has been received from the user. When the input is present (YES side), the CPU 23 ends the moving image shooting. On the other hand, if there is no input (NO side), the process returns to S102 and the CPU 23 repeats the above operation. This is the end of the description of the flowchart of FIG.

  Hereinafter, operational effects of the electronic camera of the present embodiment will be described. In the electronic camera of this embodiment, when a capture button is pressed during moving image shooting, a partial area of an image including a recognition target subject can be trimmed to obtain a still image in which the main subject is arranged in a predetermined composition. . In particular, in the present embodiment, the position and size of the subject are automatically determined based on the golden ratio. Therefore, the user can easily obtain a still image with a preferred composition without performing complicated operations.

  In the electronic camera of this embodiment, a still image is generated by performing trimming from an image having a resolution higher than one frame of moving image data. Therefore, it is also possible to acquire a trimmed image having a resolution equal to or higher than that of the moving image.

  Furthermore, in the electronic camera according to the present embodiment, the trimming range is switched when the trimming range does not fit in the image to be processed, so that the possibility that a still image with a preferred composition can be acquired is greatly improved.

(Supplementary items of the embodiment)
(1) In the above embodiment, an example in which the subject to be recognized is a person has been described. However, the subject to be recognized is not limited to a person, and may be any object such as an animal, a building, or a vehicle. it can. Of course, only face detection may be performed by the electronic camera as subject recognition, and face authentication may not be performed.

  (2) The position of the main subject in the trimmed image is not limited to the example in the above embodiment. For example, in FIG. 4, the upper left corner of the trimmed image is the starting point, and any of the intersections of coordinates (0.618X, 0.382Y), (0.382X, 0.618Y), (0.618X, 0.618Y). A composition in which a subject to be recognized may be arranged.

  (3) In the above embodiment, when the second process for trimming an image in landscape orientation is selected as an initial setting, and the image is trimmed in landscape orientation, the trimming range does not fit in the processing target image, and the first process is switched. You may do it.

  It should be noted that the present invention can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

The block diagram which shows the structure of the electronic camera of this embodiment Flowchart for explaining an operation example at the time of moving image shooting of the electronic camera in the present embodiment The figure which shows an example of the display screen of the monitor at the time of video recording Explanatory drawing showing the composition of the subject in the cropped image Explanatory drawing showing switching of trimming range

Explanation of symbols

DESCRIPTION OF SYMBOLS 12 ... Imaging device 15, 16 ... Frame memory, 17 ... Moving image data generation part, 18 ... RAM, 19 ... Still image data generation part, 20 ... Recording I / F, 22 ... Input part, 23 ... CPU, 25 ... Subject recognition unit, 26 ... trimming processing unit, 27 ... storage medium

Claims (4)

An image sensor that images a subject at predetermined intervals and outputs data of each image;
A memory for temporarily storing the image data;
A moving image data generation unit that generates moving image data from a plurality of the images;
A recording interface for recording the moving image data in a storage medium;
An input unit for receiving still image recording instruction input;
A subject recognition unit for detecting a subject to be recognized from the image;
The instruction trimmed partial region including the subject of the recognition target from the captured the image when the input, with the object of the recognition target generates data arranged trimming image in a predetermined composition, of the image A trimming processing unit that switches between a first process for trimming the image vertically and a second process for trimming the image horizontally based on a margin amount from an end to the subject to be recognized ;
An imaging apparatus comprising: The imaging device according to claim 1,
An image pickup apparatus, wherein the resolution of the image is set higher than the resolution of one frame of the moving image data. In the imaging device according to claim 1 or 2,
The image pickup apparatus, wherein the trimming processing unit determines the trimming position so that the subject to be recognized is positioned on an intersection of vertical and horizontal lines that divide the trimmed image by a golden ratio. In the imaging device according to any one of claims 1 to 3,
The imaging apparatus, wherein the trimming processing unit determines the trimming range based on a size of a subject to be recognized in the image .

Images