JP2010147651A - Image pickup apparatus and image reproduction apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image reproduction apparatus or an image pickup apparatus which can perform illuminance adjustment and white balance adjustment suitable for an image in an extracted photographed body region at a time of image pickup or image reproduction. <P>SOLUTION: The image pickup apparatus 1 performs the illuminance adjustment and the white balance adjustment for a cut out image signal used for display of the specified photographed body region based on color information of the specified photographed body region out of color information recorded beforehand. Moreover, the image pickup apparatus cuts out the specified photographed object region from all image signals at a time of the image pickup, and performs the illuminance adjustment and the white balance adjustment of the cut out image signal independently from the all image signals. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

The present invention relates to an imaging apparatus that can cut out and record only a specific subject area from a captured image, and an image playback apparatus that can cut out and reproduce only a specific subject area from a playback image. The present invention also relates to an imaging apparatus or an image reproducing apparatus that can perform luminance adjustment and white balance adjustment on a specific subject area.

  2. Description of the Related Art Conventionally, when taking a moving image of a subject (such as a person) of interest using a moving image shooting device such as a digital video camera or a digital still camera capable of shooting a movie, the photographer does not miss the subject and miss the shooting. In addition to checking the subject on the monitor, it was necessary to change the shooting direction and zoom magnification by operating the moving image shooting device according to the movement of the subject.

  Therefore, the photographer must concentrate on shooting, and it is difficult for the photographer to shoot while communicating with the subject of interest or to pay attention to other things. It was.

  Therefore, in order to eliminate such difficulties, the following moving image photographing devices and moving image reproducing devices are being proposed. In other words, when shooting a movie, the photographer shoots at a wide angle as much as possible so that the photographer does not fail to shoot the target subject. Extracting the subject area that the photographer is interested in from the moving image and recording it separately from the moving image by wide-angle shooting, or when reproducing the captured moving image and displaying it on the monitor, There has been proposed a moving image playback apparatus that can extract and display only a target subject area from a moving image.

  However, if the adjustment of the brightness and white balance for the moving image of the target subject area obtained by such a moving image photographing device or moving image reproducing device is performed in the same way as for the moving image obtained by wide-angle photographing, it becomes unnatural. There is a problem that it may end up.

    For example, it is assumed that the shooting composition at the time of wide-angle shooting is a person against the background of a white building using sunlight in the daytime as a light source, and the person is in the shade. In such a case, the moving image shot at a wide angle is very bright overall, so the moving image shooting device is adjusted so that the luminance is low, and the white balance that occupies most of the image is white. Adjust the building so that it is displayed in a certain whiteness.

    However, if the same brightness adjustment or white balance adjustment as that of a moving image by wide-angle shooting is performed on a moving image of the subject area of interest (in this case, it corresponds to a shaded area where a person is present), a very unnatural moving image is obtained. There is a problem of becoming.

    In addition to the above-described moving image shooting, the same problem can occur when shooting a still image.

Japanese Patent Application Laid-Open No. 2004-228561 discloses a method in which white balance is appropriately performed even during electronic zooming. According to this method, first, the entire region of one reference image data is divided into a plurality of areas, and feature detection is performed in each of the areas. Then, an area to be electronically zoomed is selected from the entire area of the image data, and a white balance process is performed using only the feature detection results of the areas included in the electronic zoom area, and the feature detection results in all areas are used. A technique for performing white balance control in the electronic zoom region is disclosed. However, this method cannot solve the above problem.
JP2004-64676

Accordingly, the present invention has been made in view of the above problems, and when reproducing a captured moving image or still image, it is possible to extract and reproduce a subject area of interest from the moving image or still image. An image reproducing apparatus capable of performing appropriate brightness adjustment and white balance adjustment on the extracted moving image or still image of the subject area, and at the time of moving image or still image shooting, An imaging apparatus capable of extracting a subject area of interest from an image of the entire subject to be photographed by wide-angle shooting and recording it separately from the entire image by wide-angle photographing, and is suitable for the image of the extracted subject area An object of the present invention is to provide an image pickup apparatus capable of performing brightness adjustment and white balance adjustment.

An imaging apparatus according to a first invention of the present application includes an imaging unit that acquires an image signal to be imaged by imaging, a color information deriving unit that derives color information from the image signal acquired by the imaging unit, and the image signal. Compression processing means for applying a compression process to the compressed image signal, and association means for associating and recording the compressed image signal generated by the compression processing means and the overall color information derived by the color information deriving means And the color information deriving means divides the image signal acquired by the photographing means into a plurality of small areas, derives color information of each small area as small area color information, and all the small areas Color information is used as the overall color information.

According to the first invention of the present application, the imaging apparatus divides the compressed image signal into a plurality of small areas, derives small area color information before compression processing of each small area, and converts all the small area color information into the entire color. Information is recorded in association with the compressed image signal. Therefore, in the case of extracting and reproducing a specific subject area from the decompressed image signal of the compressed image signal, luminance adjustment and white balance adjustment are performed on the specific subject area using small area color information corresponding to the specific subject area. Can be applied. Thereby, the brightness adjustment and the white balance adjustment for the specific subject area can be performed independently of the brightness adjustment and the white balance adjustment for the entire image signal to be photographed, and an adjustment suitable for the specific subject area is possible.

An image reproducing apparatus according to a second invention of the present application includes a compressed first image signal generated by performing compression processing on a first image signal acquired by photographing, and a first image associated with the compressed first image signal. The whole color information of the signal is generated by the acquisition means for acquiring from the recording medium, the expansion means for performing the expansion process on the compressed first image signal acquired by the acquisition means to generate the first image signal for display, and the expansion means First reproducing means for reproducing the displayed first image signal for display, and generating means for extracting a specific subject area from the first image signal for display generated by the decompressing means to generate a second image signal for display. A white balance adjusting unit that performs a white balance adjusting process on the second image signal for display generated by the generating unit based on the color information, and the second reproducing unit includes a white balance adjusting unit. That luminance adjustment process and the white balance adjustment process by the white balance adjusting means, characterized in that for reproducing the second image signal for display that has been subjected.

  According to the image reproducing device of the third invention of the present application, the acquisition unit acquires the compressed first image signal and the entire color information of the first image signal associated with the compressed first image signal from the recording medium.

  Therefore, the white balance adjusting means can perform luminance adjustment processing on the display second image signal using the color information of the small area corresponding to the display second image signal in the entire color information.

  Therefore, the image reproducing apparatus according to the second invention of the present application can perform white balance adjustment processing on the second image signal for display of the specific subject area independently of the first image signal for display. White balance adjustment suitable for the region can be performed.

According to a third aspect of the present invention, there is provided an image reproduction device including a compressed first image signal generated by performing a compression process on a first image signal acquired by photographing, and a first image associated with the compressed first image signal. The color information of the signal is generated by an acquisition unit that acquires the color information from the recording medium, an expansion unit that performs a decompression process on the compressed first image signal acquired by the acquisition unit, and generates a first image signal for display. First reproduction means for reproducing the displayed first image signal, generation means for extracting a specific subject area from the first display image signal generated by the decompression means, and generating a second display image signal; Luminance adjustment means for performing luminance adjustment processing on the second image signal for display generated by the generation means based on the information, and second reproduction for reproducing the second image signal for display subjected to luminance adjustment processing by the luminance adjustment means With means That.
According to the image reproducing device of the third invention of the present application, the acquisition unit acquires the compressed first image signal and the color information of the first image signal associated with the compressed first image signal from the recording medium.
Therefore, the luminance adjusting means can perform the luminance adjustment processing on the display second image signal using the color information of the small area corresponding to the display second image signal among the color information.

  Therefore, the image reproducing device according to the third invention of the present application can perform the brightness adjustment process on the second image signal for display of the specific subject area independently from the first image signal for display. The brightness adjustment suitable for can be performed.

In the image reproducing device according to the fourth invention of the present application, the image reproducing device according to the third invention further performs white balance adjustment processing on the second image signal for display generated by the generating means based on the color information. An adjustment unit is provided, and the second reproduction unit reproduces the second image signal for display subjected to the luminance adjustment processing by the luminance adjustment processing unit and the white balance adjustment processing by the white balance adjustment unit.

  According to the image reproducing apparatus of the fourth invention of the present application, the white balance adjusting means performs white balance adjustment processing on the second image signal for display using color information corresponding to the second image signal for display among the color information. Apply.

  Therefore, the image reproducing apparatus according to the fourth aspect of the present invention can perform the brightness adjustment process and the white balance adjustment process on the display second image signal of the specific subject area independently of the display first image signal. Thus, brightness adjustment and white balance adjustment suitable for a specific subject area can be performed.

An imaging apparatus according to a fifth invention of the present application is an imaging unit that acquires a first image signal to be imaged by imaging, and a white balance adjustment process, a color synchronization process, and a compression for the first image signal acquired by the imaging unit. Image processing means for generating a compressed first image signal by processing, a generation means for generating a second image signal by extracting a specific subject region from the first image signal, and a second image signal generated by the generation means A luminance adjustment unit that performs luminance adjustment processing based on color information included in the second image signal; and a white balance adjustment processing that is performed on the second image signal generated by the generation unit based on color information included in the second image signal. White balance adjusting means for performing, color compression processing and compression processing on the second image signal after brightness adjustment and white balance adjustment, and generating a compressed second image signal; And recording means for recording the compressed second image signal generated by the generated compressed first image signal and the compression processing unit by the image processing means.

According to the imaging device of the fifth invention of the present application, the second image signal can be acquired at the time of photographing, and the luminance adjustment process and the white balance adjustment are performed on the second image signal based on the color information held by the second image signal. Processing can be performed.
Therefore, the brightness adjustment unit and the white balance adjustment unit can perform the brightness adjustment process and the white balance adjustment process on the second image signal independently of the first image signal. Thereby, brightness adjustment and white balance adjustment suitable for the specific subject area can be performed.

  According to the present invention, when playing back a captured moving image or still image, the image reproducing device is capable of extracting and playing back a subject area of interest from the moving image or still image. Image reproduction apparatus capable of performing appropriate brightness adjustment and white balance adjustment on a moving image or still image, and an image of the entire subject to be photographed by wide-angle shooting at the time of moving image or still image shooting An imaging apparatus capable of extracting a subject area of interest and recording it separately from an entire image obtained by wide-angle shooting, and performing appropriate brightness adjustment and white balance adjustment on the extracted image of the subject area An image pickup device capable of being provided can be provided.

The significance or effect of the present invention will become more apparent from the following description of embodiments.

  However, the following embodiment is merely one embodiment of the present invention, and the meaning of the term of the present invention or each constituent element is not limited to that described in the following embodiment. Absent.

<First Embodiment>
A first embodiment of the present invention will be described with reference to the drawings. In the following description, an imaging apparatus such as a digital still camera or a digital video camera will be described as an example. The imaging device is only required to be capable of at least either moving image shooting or still image shooting and can reproduce a shot image.

  In the present application, “imaging” and “imaging” are used synonymously.

************************************************** ***********************
Description of overall configuration
************************************************** ***********************

FIG. 1 is a block diagram illustrating an outline of an internal configuration of the imaging apparatus 1 according to the first embodiment.

  In FIG. 1, an imaging apparatus 1 includes a zoom lens that can freely change a focal length within a certain range, a focus lens that forms an optical image of a subject on an imaging element, a diaphragm that controls exposure, and incident light. An image signal that is an analog electric signal output from the image pickup unit 2 and an image pickup unit 2 including an image pickup element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor that converts the signal into an analog electric signal at a frame period. An AFE (Analog Front End) 3 that amplifies and converts the amplified image signal to a digital image signal and outputs it, and a memory 4 that temporarily records the digital image signal output from the AFE 3 are provided. .

  The imaging apparatus 1 includes an AF evaluation value calculation unit 5 that calculates an AF (Automatic Focus) evaluation value from an image signal output from the memory 4, an AE evaluation value calculation unit 6 that calculates an AE (Automatic Exposure) evaluation value, An AWB (Automatic White Balance) evaluation value and an AWB evaluation value calculation unit 7 that calculates color information included in the image signal, and an image signal output from the memory 4 based on the AWB evaluation value from the AWB evaluation value calculation unit 7 A WB adjustment unit 8 that performs white balance adjustment is provided.

  The imaging apparatus 1 includes a color synchronization processing unit 9 that performs color synchronization processing (also referred to as color separation processing) by performing color interpolation processing on the image signal output from the WB adjustment unit 8, and the image signal is stationary. If the image is taken, JPEG (Joint Photographic Experts Group) method, and if it is a moving image, MPEG (Moving Picture Experts Group) method is used to compress and encode the image signal. A compression processing unit 10 that generates a signal, an association processing unit 11 that generates and outputs an extended compressed image signal by associating the compressed image signal with the color information output from the AWB evaluation value calculation unit 7, and an association processing unit 11 Are provided with a memory 12 for temporarily storing the extended compressed image signal output from the recording medium 13 and a recording medium 13 for recording the extended compressed image signal via the memory 12. As the memory 5 and the memory 12, for example, a commonly used memory such as a video random access memory (VRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), or a synchronous DRAM (SDRAM). Is used. Further, as the recording medium 13, for example, an internal recording medium built in the imaging apparatus 1 such as a flash memory, or an external recording that can be freely attached to and detached from the imaging apparatus 1 such as an SD memory card or a Memory Stick (registered trademark). Medium is used.

  The imaging apparatus 1 includes an image reproduction processing unit 14 that reads an extended compressed image signal from a recording medium 13, performs an expansion process, and outputs a display image signal. As will be described later, the image reproduction processing unit 14 also generates and outputs a display cut-out image signal obtained by cutting out a specific subject area from the display image signal.

  The imaging device 1 includes a display image signal output from the image reproduction processing unit 14, an image based on the display cut-out image signal, or the color synchronization processing unit 9 when the imaging device 1 is in a shooting standby state (preview mode). The image display unit 15 displays a so-called preview image based on the image signal output from the image signal in real time.

  Furthermore, the imaging apparatus 1 receives an operation unit 16 that inputs an instruction from a user including a shutter button for shooting, an instruction input from the operation unit 16, and the driving operation of each block in the imaging apparatus 1 ( For example, a CPU (Central Processing Unit) 17 that controls lens driving, aperture aperture adjustment, and the like, and each program for controlling the operation of each block by the CPU 17 and data generated when the program is executed Is provided with a memory (not shown) for temporary storage.

  FIG. 2 is a block diagram illustrating an outline of the internal configuration of the imaging unit 2.

  The imaging unit 2 includes a plurality of lenses including a zoom lens 21 that changes the focal length within a certain range and adjusts the shooting magnification of the subject, and a focus lens 22 that focuses on the subject. Optical system 23, a diaphragm 24 that adjusts the amount of light transmitted through the optical system 23, an image sensor 25 such as a CCD or CMOS sensor, movement of the zoom lens 21 and focus lens 22, and adjustment of the aperture amount of the diaphragm 24. A driver 26 composed of a motor or the like for realizing the above is provided. The arrangement of the zoom lens 21 and the focus lens 22 may be reversed.

************************************************** **********************
Explanation of definition of image sensor 25, color filter, and image signal
************************************************** ***********************
The image sensor 25 is composed of a plurality of light receiving pixels (for example, hundreds of thousands to tens of millions of pixels) arranged in a grid pattern at equal intervals in the vertical and horizontal directions. A color filter is disposed in front of each light receiving pixel.

  FIG. 3A is a diagram showing a partial region of the image sensor 25 and shows 4 × 4 = 16 light receiving pixels. Each light receiving pixel converts incident light into an electrical signal and outputs the electrical signal. This electric signal corresponds to the pixel signal of the light receiving pixel.

    FIG. 3B is a diagram showing a part of the arrangement of the color filters arranged on the front surface of the image sensor 25, and shows an arrangement of 4 × 4 = 16 color filters. The arrangement shown in FIG. 3B is generally called a Bayer arrangement. The types of color filters are a red filter (R filter) that transmits only a red component (R component) of incident light, a green filter (G filter) that transmits only a green component (G component) of light, and light. There are three types of blue filters (B filters) that transmit only the blue component (B component).

FIG. 3C shows an array of pixel signals output from 4 × 4 = 16 light receiving pixels which are partial areas of the image sensor 25.

  The color filter F1 (B filter) in FIG. 3B is arranged in front of the light receiving pixel P1 in FIG. 3A, and the pixel signal S1 in FIG. 3C is a pixel signal output from the light receiving pixel P1. is there. Accordingly, the pixel signal S1 is a B component pixel signal of light that passes through the color filter F1 and enters the light receiving pixel P1.

  Between the color filters F2 to 16 in FIG. 3B, the light receiving pixels P2 to 16 in FIG. 3A, and the pixel signals S2 to 16 in FIG. A correspondence similar to that of the pixel signal S1 exists.

Therefore, in the imaging apparatus 1,
Each light receiving pixel of the image sensor 25 having the color filter disposed in front converts the R, G, or B component of the incident light into a pixel signal and outputs it.

In the following, pixel signals of R, G, and B components of light output from the light receiving pixels corresponding to the R, G, and B filters are respectively referred to as “R signal”, “G signal”, and “B signal”. Describe. A plurality of collections of R, G, and B signals are referred to as image signals.

  The image signal before being input to the WB adjustment unit 8 is also called so-called RAW data.

  Accordingly, the memory 4 stores the RAW data output from the image pickup device 25 that is amplified and digitized as necessary.

  In the following, it is assumed that the image signal is configured by arranging a plurality of pixel signals in a grid pattern at equal intervals in the vertical direction and the horizontal direction, as shown in FIG.

In addition, the color synchronization processing unit 9 performs a color interpolation process on the image signal composed of the R, G, and B signals, so that a set of (R signal, G signal) output from each light receiving pixel of the image sensor 25. , B signal).
That is, before color synchronization processing, the pixel signal from each light receiving pixel is one of R, G, and B signals. In the following, a set of two pixel signals is output. In the image signal after color synchronization, a plurality of (R, G, B) sets are arranged in the vertical and horizontal directions as shown in FIG. Are arranged in a grid pattern at regular intervals.

The compression processing unit 10 converts the image signal composed of the (R, G, B) set output from the color synchronization processing unit 9 into a signal composed of the (Y, Cb, Cr) set in the course of the compression processing. To do. Here, Y represents a luminance signal, and Cb and Cr represent color difference signals.
In this way, the compression processing unit 10 converts the image signal from the color synchronization processing unit 9 into a signal composed of the luminance signal (Y) and the color difference signals (Cb, Cr) in the course of the compression processing, and then performs the compression processing. Apply.

  Hereinafter, such a signal converted into the YCbCr format is referred to as a display image signal.

Therefore,
The compressed image signal output from the compression processing unit 10 is obtained by compressing the display image signal.

In the following, the display image signal is configured by arranging a plurality of (Y, Cb, Cr) sets in a grid pattern at equal intervals in the vertical and horizontal directions as shown in FIG. Shall.

  In FIGS. 3A and 3B, one R, G, or B filter is arranged in one-to-one correspondence with one light receiving pixel, but one R, G, or B filter is used. Can be arranged so as to correspond to a plurality of light receiving elements.

************************************************** ***********************
Description of AF evaluation value calculation unit 5
************************************************** ***********************

The AF evaluation value calculation unit 5 divides the image signal output from the memory 4 into n (n is a positive integer) small areas and m (m ≦ n) small pixels at preset positions. An area composed of areas is set as an AF evaluation area. A plurality of R, G, and B signals are arranged in each small region.

  The AF evaluation value calculation unit 5 performs HPF (High Pass Filter) processing on the image signal from the AF evaluation region to extract a high frequency component, and outputs the high frequency component to the CPU 17 as an AF evaluation value.

  FIG. 4 shows a diagram in which the image signal from the memory 4 is divided into, for example, 8 × 8 = 64 small areas of a predetermined size.

  In FIG. 4, an area 100 is the entire area of the image signal composed of 64 small areas, an area 101 is a partial area of the area 100 composed of 4 × 4 = 16 small areas, and an area 102 is an area from the area 100. The area excluding 101 is shown.

The AF evaluation value calculation unit 5 divides the image signal from the memory 4 into small areas like the area 100 in FIG. 4 and sets the area 101 as an AF evaluation area. Then, an HPF process with a cutoff frequency of 600 KHz, for example, is performed on a signal consisting only of the G signal among the image signals in the AF evaluation area, and the absolute value of the integrated value of the G signal after the HPF process is calculated. The AF evaluation value calculator 5 outputs the calculated value to the CPU 17 as an AF evaluation value.

  The CPU 17 adjusts the position of the focus lens 22 via the driver 26 shown in FIG. 2 based on the AF evaluation value output from the AF evaluation value calculation unit 5 so as to maximize the AF evaluation value by a so-called hill climbing method. To do. As a result, the imaging apparatus 1 can acquire an image signal focused on the subject included in the image of the AF evaluation area.

************************************************** ***********************
Explanation of AE evaluation value calculation unit 6
************************************************** ***********************
Similar to the AF evaluation value calculation unit 5, the AE evaluation value calculation unit 6 divides the image signal output from the memory 4 into n (n is a positive integer) number of small regions, and at a predetermined position. An area composed of m (m ≦ n) small areas is defined as an AE evaluation area.

  The AE evaluation value calculation unit 6 first integrates R, G, and B signals included in each small region for each of n (n is a positive integer), and based on the calculated integrated value, The luminance signal of each of the n small areas is calculated by the following equation (1).

Y = 0.299Rt + 0.587Gt + 0.114Bt (1)
Rt, Gt, and Bt indicate the integrated values of the R, G, and B signals in each small region.

  The AE evaluation value calculation unit 6 multiplies the luminance signal of each small area belonging to the AE evaluation area by the weight w1, and multiplies the luminance signal of each small area belonging to the area other than the AE evaluation area by the weight w2. In this way, the absolute value of the integrated value of the luminance signal of each of the n small areas multiplied by the weight w1 or w2 is calculated, and the calculated value is output to the CPU 17 and the AWB evaluation value calculation unit 7 as an AE evaluation value.

  In FIG. 4, for example, the AE evaluation value calculation unit 6 divides the image signal from the memory 4 into 64 small areas like the area 100 and sets an area 101 including 16 small areas as the AE evaluation area. .

  The AE evaluation value calculation unit 6 calculates the luminance signal Y of each small region according to the above formula (1) based on Rt, Gt, and Bt of each of the 64 small regions of the region 100.

  Next, the weight w1 is multiplied to each luminance signal of the small area (16) belonging to the AE evaluation area, and the weight w2 is assigned to each luminance signal of the small area (48) belonging to the area 102 other than the AE evaluation area. Multiply Then, the absolute value of the integrated value of the 64 luminance signals multiplied by the weight w1 or w2 is calculated in this way, and this is output as an AE evaluation value to the CPU 17 and the AWB evaluation value calculation unit 7.

  The CPU 17 adjusts the gain (gain) in the diaphragm 24 or the AFE 3 via the driver 26 so that the AE evaluation value output from the AE evaluation value calculation unit 6 approaches a predetermined target value.

  Here, for example, by setting the weight w1 of the AE evaluation area and the weight w2 of the area C as w1 = 1.0 and w2 = 0.2, the brightness (luminance) of the AE evaluation area is emphasized. Adjustments will be made.

************************************************** ***********************
Description of AWB evaluation value calculation unit 7 and WB adjustment unit 8
************************************************** ***********************
Similar to the AF evaluation value calculation unit 5 or the AE evaluation value calculation unit 6, the AWB evaluation value calculation unit 7 divides the image signal output from the memory 4 into n (n is a positive integer) small regions. To divide. Then, the integrated values Rt, Gt, and Bt of the R, G, and B signals are calculated for each small region, and the ratio of Rt and Gt, Rt / Gt, and the ratio Bt / Gt of Bt and Gt are calculated. Based on the Bt / Gt and Rt / Gt of each of the n small areas, and the AE evaluation value from the AE evaluation value calculator 6, the AWB evaluation value calculation unit 7 determines what light source the image signal of each small area is. It is estimated whether the image signal is captured and acquired below.

  Then, based on the estimated frequency of each light source, the light source at the time of photographing is determined, and the white balance adjustment value for each of the R, G, and B signals corresponding to the preset light source is used as the AWB evaluation value. Output to the WB adjustment unit 8.

  The AWB evaluation value calculation unit 7 also combines the integrated values Rt, Gt, and Bt of the R, G, and B signals calculated for each of the n small areas together with information such as coordinates that specify each small area. Color information is output to the association processing unit 11.

  In FIG. 4, the AWB evaluation value calculation unit 8 divides the image signal from the memory 4 into 8 × 8 = 64 small regions like a region 100, for example.

  Then, Rt / Gt and Bt / Gt are calculated for each of the 64 small regions as described above.

  For example, if the AE evaluation value is equal to or greater than a predetermined value and Bt / Gt = 0.6 and Rt / Gt = 1.2 in a certain small area, the AWB evaluation value calculation unit 7 may display an image from the small area. The signal is presumed to be an image signal taken and acquired under “daylight white fluorescent light”.

  The AWB evaluation value calculation unit 7 uses the estimated light source as a reference light source, and calculates a white balance adjustment value for white balance adjustment for each of the R, G, and B signals so that the color of the reference light source becomes a predetermined color. Calculate and output to the WB adjustment unit 8.

  The AWB evaluation value calculation unit 7 combines Rt, Gt, and Bt for the small area and the coordinates of the small area as color information of the small area, and associates the color information of all 64 small areas with the association processing unit. 11 to output.

  The WB adjustment unit 8 multiplies each of the R, G, and B signals of the image signal from the memory 4 by the white balance adjustment value output from the AWB evaluation value calculation unit 7, thereby obtaining the image signal from the memory 4. White balance adjustment of R, G, and B signals is performed. Then, the image signal after white balance adjustment is output to the color synchronization processing unit 9.

************************************************** ***********************
Explanation of the compression processing unit 10
************************************************** ***********************
The compression processing unit 10 performs compression processing by the JPEG method when the image signal output from the color synchronization processing unit 9 is based on a still image, and by the MPEG method when it is based on a moving image.

  JPEG and MPEG image compression algorithms convert an image signal into a display image signal in YCbCr format, and reduce the data capacity of the display image signal from the information (Cb, Cr) regarding the color from the image signal. The compression processing is performed by thinning out the part. For this reason, even if an image signal compressed by the JPEG method and the MPEG method is expanded to generate a display image signal, a part of the color information has been lost.

  Therefore, in order to compensate for such lost color information, the imaging apparatus 1 associates the compressed image signal generated by the compression process with the color information that the compressed image signal had before compression and before color synchronization. Record.

************************************************** ***********************
Explanation of the association processing unit 11
************************************************** ***********************
The association processing unit 11 stores the compressed image signal from the compression processing unit 10 and the color information from the AWB evaluation value calculation unit 7 in the memory 12 in association with each other and records them in the recording medium 13.

  The association processing unit 11 can employ, for example, a combining process that combines the compressed image signal and color information to generate one extended compressed image signal.

  FIG. 5A shows the configuration of the extended compressed image signal output from the association processing unit 11.

  In FIG. 5A, the extended compressed image signal 200 includes header information 201, a compressed image signal 202 output from the compression processing unit 10, and color information 203 output from the AWB evaluation value calculation unit 7. .

  The header information 201 includes information related to the data structure of the extended compressed image signal 200, information related to the compressed image signal 202, information related to the color information 203, and the like.

  The information regarding the data structure of the extended compressed image signal 200 includes information such as the data size of each of the header information 201, the compressed image signal 202, and the color information 203, the arrangement within the extended compressed image signal 200, and the like.

  The information related to the compressed image signal 202 includes: compression condition, ISO (International Organization for Standardization) sensitivity at the time of shooting, focal length, shooting condition information including light source information used as a reference at the time of white balance adjustment, before compression and before color synchronization. The image signal is divided into n small areas, and information such as the size of the small areas is included.

  Information relating to the color information 203 includes information specifying each small area in the image signal before compression and color synchronization divided into n small areas, and R, G, and B signals in each small area. It consists of each integrated value.

  The memory 12 stores the expanded compressed image signal 200 generated by the association processing unit 11, and the expanded compressed image signal 200 is recorded on the recording medium 13 as image data.

************************************************** ***********************
Description of the overall configuration of the image reproduction processing unit 14
************************************************** ***********************
When a control signal instructing image reproduction is input from the CPU 17, the image reproduction processing unit 14 reproduces an extended compressed image signal 200 (FIG. 5A) of a moving image or a still image stored in the recording medium 13. To do.

  FIG. 6 is a block diagram showing an outline of the configuration of the image reproduction processing unit 14.

  The image reproduction processing unit 14 includes an expansion processing unit 30 that expands the compressed image signal 202 included in the extended compressed image signal 200 stored in the recording medium 13 to generate a display image signal 602 described later. The decompression processing unit 30 also generates an extended display image signal 300 described later from the expanded compressed image signal 200 and outputs it to the multiplexer 36, the specific subject detection unit 31, and the image cutout processing unit 32 described later. The display image signal 302 expanded by the expansion processing unit 30 and the image signal before compression processing and after color synchronization are different in data format and data size, but both represent the same image.

  FIG. 5B shows the data structure of the extended display image signal 300 output from the expansion processing unit 30. The extended display image signal 300 includes header information 301, a display image signal 302 expanded by the expansion processing unit 30, and color information 203 output from the AWB evaluation value calculation unit 7. The header information 301 is obtained by appropriately updating the header information 201 of the extended compressed image signal 200 in FIG.

The image reproduction processing unit 14 also includes a specific subject detection unit 31 that detects a specific subject region including a specific subject from the display image signal 302 expanded by the expansion processing unit 30, and a specific subject detected by the specific subject detection unit 31. An area is cut out from the display image signal 302 to generate a display cut-out image signal 402 described later, and an image cut-out processing unit 32 that generates and outputs an extended display cut-out image signal 400 is provided.

FIG. 5C shows a data structure of the extended display cutout image signal 400 output from the image cutout processing unit 32. The extended display cutout image signal 400 includes header information 401, a display cutout image signal 402 cut out by the image cutout processing unit 32, and color information 203 output from the AWB evaluation value calculation unit 7.

  The header information 401 is obtained by appropriately updating the header information 301 of the extended display image signal 300 in FIG.

The image reproduction processing unit 14 also includes a luminance adjustment unit 33 that adjusts the brightness (luminance) of the display cutout image signal 402, a WB adjustment unit 34 that adjusts the white balance of the display cutout image signal 402, and a display cutout. An enlargement processing unit 35 that performs an enlargement process on the display cut-out image signal 402 so that the image signal 402 is displayed as an image of an appropriate size by the image display unit 15, and an extended display unit that is output from the expansion processing unit 30 It comprises a multiplexer 36 that selects either the image signal 300 or the expanded display cut-out image signal 400 output from the image enlargement processing unit 35 and outputs it to the image display unit 15.

************************************************** ***********************
Description of the specific subject detection unit 31
************************************************** ***********************
The specific subject detection unit 31 refers to the number of divided regions of the image signal before compression and before color synchronization and the size of the small region after the division included in the header information 301 of the extended display image signal 300, and displays the display image signal. Similarly to the AF evaluation value calculation unit 5, the AE evaluation value calculation unit 6, or the AWB evaluation value calculation unit 7, 302 is divided into n small regions.

  The specific subject detection unit 31 detects a specific subject from the display image signal 302 and sets a specific subject region including the specific subject as a rectangular region including a plurality of small regions. At this time, the specific subject detection unit 31 sets the specific subject region so that the enlargement ratio in the vertical direction and the horizontal direction of the enlargement processing by the image enlargement processing unit 35 described later is equal.

  The specific subject detection unit 31 outputs information for specifying a small region constituting the specific subject region to the image cutout processing unit 32 described later.

FIG. 7 is a diagram for explaining detection of a specific subject and setting of a specific subject area by the specific subject detection unit 31. FIG. 7 shows an image of the display image signal 302, and further shows a case where the specific subject is a specific person.

  The specific subject detection unit 31 refers to the number of divisions of the image signal before compression and color synchronization included in the header information 301 of the extended display image signal 300 and 8 × 8 = 64, and the size of the small region. The image signal 302 is divided into an area 500 composed of 8 × 8 = 64 small areas.

The specific subject detection unit 31 detects the face portion 502 of the specific person 501 from the area 500,
From the detected size of the face portion 502, the size of the body portion 503 that follows the face is estimated. Then, an area 504 of the specific person 501 composed of a plurality of small areas is set. In FIG. 7, the area 504 includes 5 (vertical) × 5 (horizontal) = 25 small areas including the specific person 501.

Next, the specific subject detection unit 31
The specific person area (specific subject area) including the area 504 of the specific person 501 is set so that the detected area 504 of the specific person 501 has the same enlargement ratio in the vertical direction and the horizontal direction in the enlargement processing by the image enlargement processing unit 35 described later. ) 505 is set.

  FIG. 7 shows a case where the image enlargement processing unit 35 described later performs enlargement processing on the specific person area 505 so that the specific person area 505 has the same size as the area 500.

  Therefore, the specific person area 505 is composed of 5 × 5 = 25 small areas, and the area 500 composed of 8 × 8 = 64 small areas is an area that is uniformly reduced in the vertical and horizontal directions.

  The specific subject detection unit 31 outputs the coordinates of each small region to the image cutout processing unit 32 as information for specifying each of the 25 small regions constituting the specific person region 505.

  Various methods are known as a method for detecting the face portion of a person included in the display image signal 302, and the specific subject detection unit 31 can employ any method. Typically, for example, in FIG. 7, a display image signal and a predetermined image size (configured in units of small regions) of the region of interest (not necessarily configured in units of small regions) set in the display image signal 302. The display image signal for the face portion 502 of the specific person 501 registered in advance is determined to determine the similarity between both display image signals, and attention is paid based on the similarity. It is detected whether or not a face is included in the area (whether or not the target area is a face area). In the display image signal 302, the region of interest is shifted in the horizontal direction or the vertical direction by one pixel. Then, the display image signal in the region of interest after the shift is compared with the display image signal of the face portion 502 of the specific person 501, and the similarity between both display image signals is determined again. Detection is performed. Thus, the region of interest is updated and set while being shifted pixel by pixel from the upper left to the lower right of the display image signal 302, for example. Further, the display image signal 302 is reduced at a certain rate, and the same face detection process is performed on the reduced display image signal 302. By repeating such processing, a face of any size can be detected from the display image signal 302.

************************************************** ***********************
Description of the image cutout unit 32
************************************************** ***********************
The image cutout unit 32 extracts the specific subject area from the display image signal 302 in FIG. 5B based on the information specifying each of the plurality of small areas constituting the specific subject area output from the specific subject detection unit 31. Cutout and display cutout image signal 402 is generated, and header information 401 is generated by adding information specifying a small area constituting the specific subject area to header information 301. Then, an extended display clipped image signal 400 including the header information 401, the display clipped image signal 402, and the color information 203 is generated and output to the luminance adjustment unit 33.

In FIG. 7, when the coordinates of each of the 100 small areas constituting the specific person area 505 are output from the specific subject detection unit 31, the image clipping unit 32 forms the specific person area 505 from the area 500. 5 × 5 = 25 small regions are cut out to generate a cut-out image signal 402 for display, and information for identifying the 25 small regions constituting the cut-out image signal 402 for display is included in the header information 301. The coordinates are added to form header information 401. Then, an extended display clipped image signal 400 including the header information 401, the display clipped image signal 402, and the color information 203 is generated and output to the luminance adjustment unit 33.

************************************************** ***********************
Description of the brightness adjustment unit 33
************************************************** ***********************
As described above, the header information 401 of the extended display cutout image signal 400 (FIG. 5C) includes information for specifying each small area constituting the display cutout image signal 402.

    In addition, the color information 203 includes information for specifying each small area of the display image signal 302 before compression and color synchronization, and Rt, Gt, and Bt of each small area.

    Therefore, the luminance adjustment unit 33 extracts Rt, Gt, and Bt of each small region from the color information 203 based on the information specifying each small region constituting the cutout image signal for display 402.

  The luminance adjustment unit 33 calculates the luminance signal Y of each small region for each small region constituting the display cut-out image signal 402 by the above equation (1).

  The luminance adjustment unit 33 calculates the integrated value (sum) of the luminance signals Y of each small region as a luminance evaluation value, and also adjusts the luminance of the display cutout image signal 402 so that the luminance evaluation value approaches a predetermined target value. Adjust by image processing.

  Then, the luminance adjustment unit 33 outputs the expanded display cutout image signal 400 including the calculated luminance evaluation value and the display cutout image signal 402 after the luminance adjustment to the WB adjustment unit 34.

************************************************** ***********************
Explanation of WB adjustment unit 34 and image enlargement processing unit 35
************************************************** ***********************
The WB adjustment unit 34 performs white balance adjustment on the display cut-out image signal 402 that has been subjected to the luminance adjustment processing from the luminance adjustment unit 33.

  Similar to the luminance adjustment unit 33, the WB adjustment unit 34 extracts Rt, Gt, and Bt of each small area constituting the cut-out image signal 402 for display from the color information 203, and also Rt / Gt and Bt / Gt. Is calculated.

  Then, the WB adjustment unit 34 determines under what light source each small region was shot based on Bt / Gt and Rt / Gt of each small region and the luminance evaluation value output from the luminance adjustment unit 33. In addition to the estimation, the light source of the display cut-out image signal 402 is determined based on the estimated frequency of the light source.

  Then, the WB adjustment unit 34 multiplies each of the R, G, and B signals of the display cutout image signal 402 by a white balance adjustment value for each of the R, G, and B signals corresponding to the preset light source. As a result, the white balance of the display cut-out image signal 402 is adjusted.

  The WB adjustment unit 34 outputs the expanded display cut-out image signal 400 including the display cut-out image signal 402 after white balance adjustment to the image enlargement processing unit 35.

  The image enlargement processing unit 35 performs an enlargement process on the display cut-out image signal 402 from the WB adjustment unit 34 so as to conform to a predetermined format such as the NTSC format, and the display for which the enlargement process has been performed. The extended display cutout image signal 400 including the cutout image signal 402 is output to the multiplexer 36.

************************************************** ***********************
Description of multiplexer 36 and image display unit 15
************************************************** ***********************
The multiplexer 36 is a cutout image for extended display that has been subjected to luminance adjustment, white balance adjustment, and enlargement processing in response to an input of a control signal for instructing cutout image reproduction from the CPU 16 or a control signal for instructing whole image reproduction. Either the signal 400 or the extended display image signal 300 is selected and output to the image display unit 15.

    The image display unit 15 outputs the cutout image signal 402 for display from the extended cutout image signal 400 or the extended display image signal 300 that has been subjected to the luminance adjustment, white balance adjustment, and enlargement processing output from the multiplexer 36. The display image signal 302 is extracted and displayed.

************************************************** ***********************
Description of the overall operation of the imaging apparatus 1 in the shooting mode and playback mode
************************************************** ***********************
The imaging apparatus 1 has two operation modes, a shooting mode and a playback mode. When the operation mode of the image capturing apparatus 1 is the image capturing mode and the user does not perform an image recording operation (recording operation) of a moving image or a still image by the user from the operation unit 16 (preview mode), the image capturing apparatus 1 includes the image capturing unit 2. 1 outputs an image signal for a preview image in a frame cycle, performs a necessary process in each block of FIG. 1, and displays an image display unit for a preview image based on the image signal for the preview image output from the color synchronization processing unit 9 15 is displayed. At this time, the image signal for the preview image is not recorded on the recording medium 13.

  When a moving image recording operation is performed by the user from the operation unit 16 in the shooting mode, the imaging device 1 outputs an image signal for recording a moving image at a frame period from the imaging unit 2, and in each block of FIG. Necessary processing is performed to generate an extended compressed image signal 200, which is sequentially recorded on the recording medium 13. In addition, a moving image based on an image signal for moving image recording output from the color synchronization processing unit 9 is displayed on the image display unit 15.

  When a still image recording operation is performed in the shooting mode, the imaging device 1 outputs an image signal for recording a still image from the imaging unit 2 and performs necessary processing in each block of FIG. The signal 200 is generated and recorded on the recording medium 13. Further, a still image based on an image signal for recording a still image output from the color synchronization processing unit 9 is displayed on the display unit 15.

The imaging device 1 expands the compressed image signal 202 (FIG. 5A) stored in the recording medium 13 when a reproduction operation of the entire area of the moving image or still image is performed from the operation unit 16 in the reproduction mode. The image is expanded by the processing unit 30 to be a display image signal 302, and an image based on this is displayed on the image display unit 15. Note that the entire area of the moving image or still image is subjected to brightness adjustment and white balance adjustment at the time of shooting.

  On the other hand, when the reproduction operation of the specific subject area is performed from the operation unit 16, the imaging apparatus 1 performs brightness adjustment and display on the display cutout image signal 402 based on the color information 203 of the extended display cutout image signal 400. White balance adjustment is performed, and an image based on the white balance adjustment is displayed on the image display unit 15.

************************************************** ***********************
Flow chart of shooting operation
************************************************** ***********************
Next, the operation of the imaging apparatus 1 when a moving image or still image recording operation is performed from the operation unit 16 (FIG. 1) will be described. FIG. 8 is a flowchart showing a flow of a series of processes when the imaging apparatus 1 performs a recording operation.

  In step S <b> 100, when the user performs power-on and shooting mode setting operations on the operation unit 16, the imaging apparatus 1 is activated and enters a shooting standby state (preview mode).

  In step S <b> 101, the imaging device 1 acquires an image signal for a preview image from the imaging unit 2 at a frame period, and stores it in the memory 4 via the AFE 3.

  In step S102, each of the AF evaluation value calculation unit 5 and the AE evaluation value calculation unit 6 calculates an AF evaluation value and an AE evaluation value from the image signal stored in the memory 4, and outputs them to the CPU 17. The CPU 17 adjusts the imaging unit 2 based on the AF evaluation value and the AE evaluation value.

  The AWB evaluation value calculation unit 7 divides the image signal stored in the memory 4 into n small regions, and calculates integrated values Rt, Gt, and Bt of the R, G, and B signals in each small region. Then, the light source of each small region is estimated based on Rt, Gt, and Bt of each small region and the AE evaluation value, and the light source having the highest frequency among the estimated light sources is used as the light source of the entire image signal. Then, the preset white balance adjustment value for the light source is output to the WB adjustment unit 8. The WB adjustment unit 8 performs white balance adjustment processing on the image signal from the memory 4 based on the white balance adjustment value output from the AWB evaluation value calculation unit 7.

  In step S <b> 103, the color synchronization processing unit 9 generates R, G, and B signals output from each light receiving pixel of the image sensor 25 (FIG. 2) by color interpolation processing, and outputs them to the compression processing unit 10.

  In step 104, the image display unit 15 displays an image based on the image signal output from the color synchronization processing unit 9.

  In step S105, the CPU 17 determines whether or not an image recording operation has been performed from the operation unit 16, and if it is determined that an image recording operation has been performed, the imaging apparatus 1 shifts from the preview mode to the image recording mode, and step The process proceeds to S106. If it is determined that the image recording operation has not been performed, the imaging apparatus 1 returns to step S101 and continues the preview mode.

  In step S <b> 106, the imaging apparatus 1 acquires a recording image signal from the imaging unit 2 at a frame period, and stores it in the memory 4 via the AFE 3.

  In step S107, the CPU 17 adjusts the imaging unit 2 and the like based on the AF evaluation value and the AE evaluation value as in step S102.

  In addition, the AWB evaluation value calculation unit 7 derives a white balance adjustment value and outputs the white balance adjustment value to the WB adjustment unit 8 as in step S102.

  The WB adjustment unit 8 performs white balance processing on the image signal from the memory 4 based on the white balance adjustment value output from the AWB evaluation value calculation unit 7.

  The AWB evaluation value calculation unit 7 further combines the integrated values Rt, Gt, and Bt of the R, G, and B signals in each of the n small areas of the image signal and the information that specifies each small area together with the color information. And output to the association processing unit 11.

  In step S <b> 108, the color synchronization processing unit 9 generates R, G, and B signals output from each light receiving pixel of the image sensor 25 (FIG. 2) by color interpolation processing, and outputs them to the compression processing unit 10.

  The compression processing unit 10 changes the data format of the image signal output from the color synchronization processing unit 9 to generate a display image signal. Then, the compressed image signal is generated by performing compression processing on the display image signal, and is output to the association processing unit 11.

  In step S <b> 109, the association processing unit 11 combines the compressed image signal and the color information from the AWB evaluation value calculation unit 7 to generate an extended compressed image signal 200 (FIG. 5A) and stores it in the memory 12. At the same time, it is also recorded on the recording medium 13.

  In step S110, when the CPU 17 determines whether or not an image recording end operation (recording end operation) has been performed from the operation unit 16, and determines that the image recording end operation has been performed, the imaging apparatus 1 switches from the image recording mode to the preview mode. The process proceeds to step S111. If it is determined that the image recording end operation has not been performed, the process returns to step S106 to continue the image recording mode.

  In step S111, the CPU 17 determines whether or not an operation for ending the shooting mode has been performed from the operation unit 16, and when the ending operation has been performed, the shooting is ended and a series of processes is ended. If the end operation has not been performed, the process returns to step S101 to continue the preview mode.

************************************************** ***********************
Flow chart of playback operation
************************************************** ***********************
Next, the operation of the imaging apparatus 1 when a moving image or still image reproduction operation is performed from the operation unit 16 (FIG. 1) will be described. FIG. 9 is a flowchart showing a flow of a series of processes when the imaging apparatus 1 performs an image reproduction operation.

  In step S200, when a power-on operation and a reproduction mode setting operation are performed on the operation unit 16, the imaging device 40 is activated and enters a reproduction standby state.

  In step S201, the CPU 17 determines whether or not an image playback operation has been performed from the operation unit 16, and if it is determined that an image playback operation has been performed, the process proceeds to step S202, and if it is determined that no image playback operation has been performed, Continue playback standby.

  In step S <b> 202, the imaging apparatus 1 reads image data (extended compressed image signal 200 (FIG. 5A)) recorded on the recording medium 13 into the memory 12.

  In step S203, the expansion processing unit 30 (FIG. 6) performs an expansion process on the extended compressed image signal 200 read into the memory 12 to generate an extended display image signal 300 (FIG. 5B). The data is output to the specific subject detection unit 31 and the image cutout processing unit 32.

  In step S204, the CPU 17 determines whether or not an operation for reproducing the specific subject area has been performed from the operation unit 16. If the CPU 17 determines that an operation for reproducing the specific subject area has been performed, the process proceeds to step S206. If it is determined that the operation for reproducing is not performed, that is, the operation for reproducing the entire area is performed, the process proceeds to step S205.

  In step S206, the image cutout processing unit 32 cuts out the specific subject region based on the information specifying the specific subject region output from the specific subject detection unit 31, generates the extended display cutout image signal 400, and adjusts the brightness. To the unit 33.

  In step S206, the luminance adjustment unit 33 performs luminance adjustment on the display cut-out image signal 402 based on the color information 203 included in the extended display cut-out image signal 400 output from the image cut-out processing unit 32, thereby adjusting the luminance. The expanded cutout image signal 400 including the cutout display image signal 402 subjected to the above is output to the WB adjustment unit 34.

  The WB adjustment unit 34 performs white balance adjustment of the display cutout image signal 402 based on the color information 203 included in the extended display cutout image signal 400 output from the luminance adjustment unit 33, and the white balance adjustment is performed. The expanded display cut-out image signal 400 including the display cut-out image signal 402 is output to the image enlargement processing unit 35.

  In step S208, the enlargement processing unit 35 conforms to a predetermined format such as the NTSC format for the display cutout image signal 402 included in the extended display cutout image signal 400 output from the WB adjustment unit 35. Is subjected to enlargement processing and output to the multiplexer 36.

  The multiplexer 36 selects the extended display cut-out image signal 400 output from the image enlargement processing unit 35 and outputs the selected image to the image display unit 15.

  The image display unit 15 extracts an enlarged display cut-out image signal 402 from the extended display cut-out image signal 400 and displays an image corresponding thereto.

  In step S <b> 205, the multiplexer 36 selects the extended display image signal 300 output from the decompression processing unit 30 and outputs it to the image display unit 15.

  The image display unit 15 extracts the display image signal 302 from the extended display image signal 300 and displays an image corresponding thereto.

  In step S209, the CPU 17 determines whether or not the playback end operation has been performed from the operation unit 16, and if it is determined that the playback end operation has been performed, the process proceeds to step S210, and if it is determined that the playback end operation has not been performed, Returning to step S202, the reproduction mode is continued.

  In step S210, the CPU 17 determines whether or not the end operation of the reproduction mode has been performed from the operation unit 16, and when it is determined that the end operation has been performed, the series of processing ends. If it is determined that the ending operation has not been performed, the process proceeds to step S211.

  In step S211, when the next image data (expanded compressed image signal 200 (FIG. 5A)) is designated from the operation unit 16, the imaging apparatus 1 returns to step S202 to reproduce the designated image data. .

  As described above, the imaging apparatus 1 extracts color information from the image signal output from the memory 4 at the time of shooting. The color information and the compressed image signal of the image signal from which the color information is extracted are recorded together as an extended compressed image signal 200 on a recording medium.

  The imaging device 1 generates a display image signal 302 by expanding the compressed image signal 202 included in the extended compressed image signal 200 recorded on the recording medium, reproduces and displays an image of the entire area, and displays an image for display. Only the specific subject area is cut out from the signal 302 to generate a display cut-out image signal 402, and the image, that is, the image of the specific subject area can be reproduced and displayed.

  In this case, the imaging apparatus 1 can perform brightness adjustment and white balance adjustment on the image of the specific subject area based on the color information of the specific subject area in the color information recorded in advance.

  Thereby, the imaging device 1 can perform the brightness adjustment and the white balance adjustment for the image of the specific subject area independently of the brightness adjustment and the white balance adjustment for the image of the entire area.

************************************************** ***********************
Modified example
************************************************** ***********************
In the first embodiment described above, the integrated values Rt, Gt, and Bt of the R, G, and B signals calculated for each of the n small areas in the image signal output from the memory 4 and the information that specifies each small area. Is combined with the color information, but is not limited to this.

  For example, the color information may be a luminance value of each of n areas calculated from the above equation (1) based on Rt, Gt, and Bt, a combination of information for specifying each small area, and the like.

  In the first embodiment, the image signal output from the memory 4 is divided into n (n is a positive integer) small areas. When the number n of divided areas is increased, the data size of the color information is increased, so that the capacity of the memory 12 and the recording medium 13 must be increased. However, since the size of the small area is reduced when the number n of divided areas is increased, the luminance adjusting unit 34 can set the area more finely when performing the luminance adjustment by setting the area to be weighted. The brightness of the clipped image can be adjusted well.

  In addition, since the WB adjustment unit 34 can determine the light source of the cut-out image signal with high accuracy, the white balance adjustment of the cut-out image can be performed with high accuracy.

  Therefore, in the imaging device 1, the number n of divided areas may be made variable according to the image quality required for the clipped image based on the clipped image signal for display.

  In the first embodiment, as one of the association processes, a combination process of the compressed image signal output from the compression processing unit 10 and the color information output from the AWB evaluation value calculation unit 7 is employed. However, it is not limited to this.

  For example, association information indicating that they are related to each other may be added to the header area of the compressed image signal and the corresponding color information color information header area. In this way, even if the compressed image signal and the color information are separately stored or recorded in different areas of the memory 12 or the recording medium 13, it is possible to detect one from the other.

  In the first embodiment, the specific subject detection unit 31 sets the specific subject region as a rectangular region. However, the specific subject region is not limited thereto, and may be set as a circle or an ellipse.

************************************************** ***********************
Second embodiment
************************************************** ***********************
<Second Embodiment>

Next, a second embodiment of the present invention will be described with reference to the drawings.

************************************************** ***********************
Description of overall configuration
************************************************** ***********************

FIG. 10 is a block diagram showing an outline of the internal configuration of the imaging apparatus 40 according to the second embodiment of the present invention.

In FIG. 10, the parts denoted by the same numbers as those in FIG. 1 are parts having the same functions as those in the first embodiment and performing the same operations. Therefore, description of the function and operation is omitted.

  In FIG. 10, the imaging device 40 includes an imaging unit 2, an AFE 3, a memory 4, an AF evaluation value calculation unit 5, an AE evaluation value calculation unit 6, a WB adjustment unit 8, a color synchronization processing unit 9, a compression processing unit 10, and a memory. 12, a recording medium 13, an image display unit 15, an operation unit 16, and a CPU 17.

  The imaging apparatus 40 includes an AWB evaluation value calculation unit 41 that has the same function as the AWB evaluation value calculation unit 7 in the first embodiment and performs substantially the same operation. The difference in operation is that the AWB evaluation value calculation unit 41 does not output color information for each of the n small areas to the association processing unit 43 described later.

  The imaging device 40 cuts out a specific subject area from an image signal from the memory 4 (hereinafter referred to as a whole image signal) and adjusts the luminance of the cut out image signal (hereinafter referred to as a cut-out image signal). A specific subject cutout unit 42 that performs processing, white balance adjustment processing, color synchronization processing, and compression processing is provided.

  The specific subject cutout unit 42 outputs a compressed cutout image signal (hereinafter referred to as a compressed cutout image signal) to the association processing unit 43 described later.

  The imaging device 40 associates a compressed image signal (hereinafter, referred to as a compressed total image signal) of all image signals output from the compression processing unit 10 with a compressed clipped image signal output from the specific subject clipping unit 42. An association processing unit 43 stored in the memory 12 is provided. The compressed whole image signal and the compressed cut-out image signal subjected to the association processing by the association processing unit 43 are recorded on the recording medium 13 via the memory 12.

  The imaging device 40 reads a compressed full image signal or a compressed cut-out image signal from the recording medium 13 and performs a decompression process or the like to display a whole image signal for display (hereinafter referred to as a whole display image signal for display) or for display. An image reproduction processing unit 44 that generates a cutout image signal (hereinafter referred to as a display cutout image signal) and outputs the cutout image signal to the image display unit 15 is provided.

************************************************** ***********************
Description of specific subject cutout unit 42
************************************************** ***********************
FIG. 11 is a block diagram illustrating an outline of the internal configuration of the specific subject cutout unit 42.
The specific subject cutout unit 42 detects a specific subject from all image signals from the memory 4, sets a specific subject region including the specific subject, and outputs position information from the specific subject detection unit 50 and the specific subject detection unit 50. Based on the output position information of the specific subject area, the specific subject area is cut out from all the image signals from the memory 4, and the cutout image output from the image cutout processing unit 51 that generates the cutout image signal. The luminance adjustment unit 52 that performs luminance adjustment processing on the signal, the WB adjustment unit 53 that performs white balance adjustment processing on the cut-out image signal output from the luminance adjustment unit 52, and the cut-out image signal output from the WB adjustment unit 53 by color interpolation Color synchronization processing unit 54 that performs color synchronization processing, and a cutout image output from the color synchronization processing unit 54 The compression processing to the item, and a compression processing unit 55 for generating a compressed cut-out image signal.





************************************************** ***********************
Description of the specific subject detection unit 50
************************************************** ***********************
The specific subject detection unit 50 detects a specific subject from all the image signals from the memory 4 and sets a specific subject region of a predetermined size including the specific subject as a rectangular region. At this time, the specific subject detection unit 50 sets the specific subject region so that the enlargement ratio in the vertical direction and the horizontal direction of the enlargement processing by the image enlargement processing unit 65 described later is equal.

  Then, the specific subject detection unit 50 outputs the set position information of the specific subject region to the image cutout processing unit 51 described later.

  FIG. 12 is a diagram for explaining the detection of the specific subject and the setting of the specific subject region by the specific subject detection unit 50. FIG. 12 shows images of all image signals from the memory 4, and further shows a case where the specific subject is a specific person.

  The specific subject detection unit 50 detects the face portion 602 of the specific person 601 from the entire image signal region 600, and estimates the size of the body portion 603 that follows the face from the size of the detected face portion 602. To do. Then, the area 604 of the specific person 601 is set. Next, the specific subject detection unit 50 determines the area of the specific person 601 so that the detected area 604 of the specific person 601 has the same vertical and horizontal enlargement ratios in the enlargement process performed by the image enlargement processing unit 65 described later. A specific person area (specific subject area) 605 including 604 is set.

  FIG. 12 illustrates a case where the image enlargement processing unit 65 described later performs enlargement processing on the specific person area 605 so that the specific person area 605 has the same size as the area 600 of all the image signals.

  Therefore, the specific person region 605 is a region obtained by uniformly reducing the region 600 of all image signals in the vertical direction and the horizontal direction.

  The specific subject detection unit 50 outputs, for example, the coordinates of the four vertices of the specific person region 605 to the image cutout processing unit 51 as the position information of the specific person region 605.

  As a method for detecting the face portion of a person included in the entire image signal 600, the method used by the specific subject detection unit 31 of the first embodiment can be employed.

************************************************** ***********************
Explanation of image cutout processing unit 51
************************************************** ***********************
The image cutout processing unit 51 cuts out the specific subject region from all the image signals from the memory 4 based on the position information of the specific subject region output from the specific subject detection unit 50, and generates a cutout image signal.

  In FIG. 12, the image cutout processing unit 51 cuts out the specific person region 605 based on the coordinates of the four vertices of the specific person region 605 output from the specific subject detection unit 50, and supplies the cutout image signal to the luminance adjustment unit 52. Output.

  Hereinafter, the specific person region 605 is referred to as a cut-out image signal 605.

************************************************** ***********************
Description of the brightness adjustment unit 52
************************************************** ***********************
The brightness adjustment unit 52 divides the cut-out image signal into p (p is a positive integer) minute regions by the same process as the AE evaluation value calculation unit 6 of the first embodiment. Then, an area composed of q (q ≦ p) minute areas at a preset position is set as a luminance evaluation area.

  The brightness adjustment unit 52 calculates a brightness signal for each of the p micro areas, and multiplies the brightness signal of each micro area belonging to the brightness evaluation area by a weight w3 to thereby each micro area belonging to an area other than the brightness evaluation area Is multiplied by a weight w4.

  In this way, the absolute value of the integrated value of the luminance signals of the p minute regions multiplied by the weight w3 or w4 is calculated as the luminance evaluation value.

  The luminance adjustment unit 52 performs luminance adjustment processing on the clipped image signal so that the calculated luminance evaluation value becomes a predetermined target value, and outputs it to the WB adjustment unit 53.

    FIG. 13 is a diagram showing a cut-out image signal 605 (specific person region 605) cut out by the image cut-out processing unit 51 from the region 600 of all image signals in FIG.

  In FIG. 13, the luminance adjustment unit 52 divides the cut-out image signal 605 into 8 × 8 = 64 minute regions and evaluates the luminance of a region 606 including minute regions 8 (vertical) × 3 (horizontal) = 24. This is an area.

  The luminance adjustment unit 52 calculates the luminance signal Y for each of the 64 minute regions.

  Next, the weight w3 is multiplied by the luminance signal of the minute area (24) belonging to the luminance evaluation area, and the weight w4 is multiplied by the luminance signal of each small area (40) belonging to the area other than the luminance evaluation area. . Then, the absolute value of the integrated value of the 64 luminance signals multiplied by the weight w3 or w4 is calculated in this way, and this is used as the luminance evaluation value.

  The brightness adjustment unit 52 adjusts the brightness of the cut-out image signal so that the brightness evaluation value approaches a predetermined target value, and outputs the brightness evaluation value to the WB adjustment unit 53.

************************************************** ***********************
Explanation of WB adjustment unit 53
************************************************** ***********************
Similar to the luminance adjustment unit 52, the WB adjustment unit 53 divides the clipped image signal into p (p is a positive integer) minute regions.

  Then, the integrated values Rs, Gs, and Bs of the R, G, and B signals are calculated for each minute region by the same processing as the AWB evaluation value calculation unit 7 in the first embodiment. Then, a ratio between Rs and Gs, Rs / Gs, a ratio between Bs and Gs, and Bs / Gs are calculated. Based on the Bs / Gs and Rs / Gs of each of the p small areas, and the luminance evaluation value from the luminance adjustment section 53, the WB adjustment unit 53 takes an image signal under what light source. It is estimated whether it is an acquired image signal.

  Based on the estimated power of the light source, the light source at the time of shooting is determined, and white balance adjustment values for each of the R, G, and B signals corresponding to the preset light source are derived.

  The WB adjustment unit 53 multiplies each of the R, G, and B signals of the clipped image signal by the white balance adjustment value, and outputs the multiplied clipped image signal to the color synchronization processing unit 54.

  In FIG. 13, for example, the WB adjustment unit 53 divides the cut-out image signal 605 into 8 × 8 = 64 minute regions and calculates Rs / Gs and Bs / Gs.

  The WB adjustment unit 53 estimates the light source of each minute region based on the calculated Rs / Gs, Bs / Gs, and the luminance evaluation value, and cuts out the light source having the highest frequency among the estimated light sources, and the light source of the image signal 605 And

The WB adjustment unit 53 multiplies each of the R, G, and B signals of the cutout image signal 605 by the white balance adjustment value for each of the determined R, G, and B signals for the determined light source, The cut-out image signal 605 is output to the color synchronization processing unit 54.
Note that the number of divided areas p of the cut-out image signal may be equal to the number of divided areas n of all the image signals in the AE evaluation value calculation unit 6 of the first embodiment, and the number of divided areas q of the luminance evaluation area may be equal. And the m number of divided areas of the AE evaluation area in the AE evaluation value calculation unit 6 of the first embodiment may be equal. The small area that is the divided area of the entire image signal and the minute area that is the divided area of the cut-out image signal may have the same size.

  When the number p of divided areas is increased, the size of the minute area is reduced, so that the brightness adjustment unit 52 can set the brightness evaluation area more finely, and thus can accurately adjust the brightness of the cut-out image signal.

  Further, since the WB adjustment unit 53 can determine the light source of the cut-out image signal with high accuracy, white balance adjustment can be performed with high accuracy.

  Therefore, in the imaging device 40, the number of divided areas p may be variable in accordance with the image quality required for the clipped image based on the clipped image signal.

************************************************** ***********************
Description of color synchronization processing unit 54 and compression processing unit 55
************************************************** ***********************
The color synchronization processing unit 54 performs color synchronization processing on the cut-out image signal output from the WB adjustment unit 53 by the same processing as the color synchronization processing unit 9 in the first embodiment, and outputs it to the compression processing unit 55. To do.

  The compression processing unit 55 generates a compressed cutout image signal by performing compression processing on the cutout image signal that has been subjected to the color synchronization processing by the same processing as the compression processing unit 10 in the first embodiment, and generates an association cutout image signal. Output to 43.

************************************************** ***********************
Explanation of the association processing unit 43
************************************************** ***********************
The association processing unit 43 stores the compressed whole image signal output from the compression processing unit 10 and the compressed cutout image signal output from the specific subject cutout unit 42 in association with each other in the memory 12 and records them in the recording medium 13. .

  For example, the association processing unit 43 combines the compressed all image signal and the compressed cutout image signal into one, or the compressed whole image signal and the compressed cutout image signal are associated with each other. The process etc. which add the correlation information which shows to both can be employ | adopted.

    Note that there is a relationship between the compressed full image signal and the compressed cutout image signal that are associated with each other, that is, the cutout image signal based on the compressed cutout image signal is cut out from the full image signal based on the compressed full image signal. Exists.

************************************************** ***********************
Description of the overall configuration of the image reproduction processing unit 44
************************************************** ***********************
As described above, the compressed cut-out image signal and the compressed whole image signal are recorded on the recording medium 13 in association with each other.

  The user of the imaging device 40 selects an image of a cut-out image signal (hereinafter referred to as a cut-out image) or an image of all image signals (hereinafter referred to as a full image) by operating the operation unit 16. Can be played.

  When the control signal instructing the reproduction of the entire image or the cutout image is input from the CPU 17, the image reproduction processing unit 44 compresses the compressed full image signal or the compressed cutout image signal of the moving image or the still image stored in the recording medium 13. Are subjected to decompression processing and the like.

  FIG. 14 is a block diagram showing an outline of the configuration of the image reproduction processing unit 44.

  The image reproduction processing unit 44 selects and outputs either the compressed full image signal or the compressed cut-out image signal from the recording medium 13 via the memory 12, and the compressed full image signal is output from the image selection processing unit 61. When it is output, the compressed all image signal is decompressed, and when the compressed cutout image signal is output from the decompression processing unit 62 and the image selection processing unit 61 for generating the entire display image signal, the compressed full image signal is decompressed. The processing is performed, and the expansion processing unit 63 that generates the display cut-out image signal, and the display cut-out image signal output from the expansion processing unit 63 is subjected to enlargement processing so as to conform to a display format such as the NTSC format, for example. All display image signals output from the enlargement processing unit 64 and the expansion processing unit 62, or enlarged display cut-out images output from the enlargement processing unit 64 And a multiplexer 65 which selects and outputs one of the signals.

************************************************** ***********************
Explanation of the image selection processing unit 61
************************************************** ***********************
When the image selection processing unit 61 receives an instruction to reproduce all images from the CPU 17 via the operation unit 16, the image selection processing unit 61 reads the compressed all image signals recorded on the recording medium 13 via the memory 12 and outputs them to the expansion processing unit 62. To do.

  When an instruction to reproduce a clipped image is received from the CPU 17 via the operation unit 16, the compressed clipped image signal recorded on the recording medium 13 is read via the memory 12 and output to the decompression processing unit 63.

  When the image selection processing unit 61 receives the instruction to reproduce the clipped image while sequentially reading out the compressed all image signals from the memory 12 and outputting them to the decompression processing unit 62, the compression processing associated with the compressed all image signals is received. While switching to readout of the cutout image signal, the read out compressed cutout image signal is output to the decompression processing unit 63. The same applies to the reverse case.

  When the association process by the association processing unit 43 is a process of combining the compressed whole image signal and the compressed clipped image signal to generate a compressed image signal, the image processing selection unit 61 performs the combined compression in accordance with an instruction from the CPU 17. Select one of the image signals and output.

  In the case where the association process is a process of adding common association information to the compressed all-image signal and the compressed clipped image signal, for example, when an instruction to reproduce the clipped image is received during the reproduction of all images, the image selection processing unit 61 A compressed cut-out image signal having the same association information as the association information added to the image signal is read from the recording medium 13 and output to the expansion processing unit 63.

************************************************** ***********************
Description of multiplexer 65
************************************************** ***********************
When the multiplexer 65 receives an instruction to reproduce all images from the CPU 17 via the operation unit 16, the multiplexer 65 outputs a display all image signal output from the expansion processing unit 62.

  On the other hand, when an instruction for a cut-out image is received, an enlarged display cut-out image signal output from the enlargement processing unit 64 is output.

************************************************** ***********************
Description of overall operation of imaging device 40 in shooting mode and playback mode
************************************************** ***********************
Similar to the imaging device 1 of the first embodiment, the imaging device 40 has two operation modes: a shooting mode and a playback mode. When the operation mode of the imaging device 40 is the shooting mode and no image recording operation of a moving image or a still image is performed from the operation unit 16 (preview mode), the imaging device 40 previews from the imaging unit 2 at a frame period. The entire image signal for the image is output, the necessary processing is performed in each block of FIG. 10, and a preview image based on the entire image signal for the preview image output from the color synchronization processing unit 9 is displayed on the image display unit 15. To do.

  In the preview mode, the imaging device 40 does not record all image signals for preview images on the recording medium 13. The specific subject region cutout process by the specific subject cutout unit 42 is not performed on all image signals for the preview image output from the memory 4.

  When a moving image recording operation is performed by the user from the operation unit 16 in the shooting mode, the imaging device 40 outputs all image signals for recording a moving image at a frame period from the imaging unit 2, and each block of FIG. The necessary processing is performed to generate a compressed whole image signal, which is sequentially recorded on the recording medium 13. At this time, a moving image based on the entire image signal for moving image recording output from the color synchronization processing unit 9 is displayed on the image display unit 15.

  In addition, the imaging device 40 uses the specific subject cutout unit 42 to cut out the specific subject region from all the image signals and generates a cutout image signal. Then, the clipped image signal is subjected to brightness adjustment, white balance adjustment, color synchronization processing, and compression processing separately from the entire image signal, thereby generating a compressed clipped image signal and recording it on the recording medium 13.

  When the image capturing apparatus 40 records the compressed full image signal and the compressed cut-out image signal on the recording medium 13, the image capturing apparatus 40 records both of them in association with each other.

  In addition, when the still image recording operation is performed in the shooting mode, the imaging device 40 generates the compressed full image signal and the compressed cutout signal in the same manner as when the moving image recording operation is performed, and associates them with each other. To the recording medium 13. At this time, a still image based on all image signals for recording a still image output from the color synchronization processing unit 9 is displayed on the image display unit 15.

When the imaging device 40 is in the reproduction mode, the user can instruct the entire image reproduction or the cut-out image reproduction from the operation unit 16.

  When the entire image reproduction operation is performed from the operation unit 16, the imaging device 40 reads the compressed all image signal stored in the recording medium 13 and displays the entire image based on this on the image display unit 15.

  When a cut-out image reproduction operation is performed from the operation unit 16, the imaging device 40 reads a compressed cut-out image signal stored in the recording medium 13 and displays a cut-out image based on the compressed cut-out image signal on the image display unit 15.

  For example, when the image capture device 40 is displaying and reproducing all images on the image display unit 15 and a cut-out image reproduction operation is performed from the operation unit 16, the image capture device 40 changes to the reproduction display of all images. Then, the compressed cutout image signal associated with the compressed whole image signal of the whole image is read from the recording medium 13, and the cutout image based on this is reproduced and displayed.

  When the compressed clipped image signal and the compressed whole image signal are combined, the compressed clipped image signal is selected, and the clipped image based on this is reproduced and displayed.

  Note that the luminance adjustment and white balance adjustment for the clipped image have already been performed at the time of shooting, and unlike the first embodiment, the luminance adjustment and white balance adjustment for the clipped image are not performed during reproduction.

************************************************** ***********************
Flow chart of photographing operation of imaging device 40
************************************************** ***********************
Next, the operation of the imaging device 40 when a moving image or still image recording operation is performed from the operation unit 16 (FIG. 1) will be described. FIG. 15 is a flowchart illustrating a flow of a series of processes when the imaging device 40 performs a recording operation.

  In step S300, when the user performs power-on and shooting mode setting operations on the operation unit 16, the imaging device 40 is activated and enters a shooting standby state (preview mode).

  In step S <b> 301, the imaging device 40 acquires an image signal for a preview image from the imaging unit 2 at a frame period and stores it in the memory 4.

  In step S <b> 302, each of the AF evaluation value calculation unit 5 and the AE evaluation value calculation unit 6 calculates an AF evaluation value and an AE evaluation value from all the image signals from the memory 4 and outputs them to the CPU 17. The CPU 17 adjusts the imaging unit 2 based on the AF evaluation value and the AE evaluation value.

  The AWB evaluation value calculation unit 41 divides all image signals from the memory 4 into n small regions, and calculates integrated values Rt, Gt, and Bt of R, G, and B signals in each small region. Then, the light source of each small region is estimated based on Rt, Gt, and Bt of each small region, and the AE evaluation value, and the light source having the highest frequency among the estimated light sources is set as the light source of all image signals. Then, the preset white balance adjustment value for the light source is output to the WB adjustment unit 8. The WB adjustment unit 8 performs white balance processing on all the image signals from the memory 4 based on the white balance adjustment value output from the AWB evaluation value calculation unit 41.

In step S303, the color synchronization processing unit 9 generates R, G, and B signals output from each light receiving pixel of the image sensor 25 (FIG. 2) by color interpolation processing, and outputs the signals to the compression processing unit 10 step S304. Then, the image display unit 15 displays an image based on the image signal output from the color synchronization processing unit 9.

  In step S305, the CPU 17 determines whether or not an image recording operation has been performed from the operation unit 16, and if it is determined that an image recording operation has been performed, the imaging device 40 transitions from the preview mode to the image recording mode, and step The process proceeds to S306. If it is determined that the image recording operation has not been performed, the imaging device 40 returns to step S301 and continues the preview mode.

  In step S306, the imaging device 40 acquires the recording image signal from the imaging unit 2 at a frame period, and stores it in the memory 4 via the AFE 3.

  In step S307, the CPU 17 adjusts the imaging unit 2 based on the AF evaluation value and the AE evaluation value.

  In step S308, the specific subject cutout processing unit 42 generates a cutout image signal by cutting out the specific subject region from all the image signals. Then, this is subjected to compression processing to generate a compressed cutout image signal and output it to the association processing unit 43.

  In step S309, the AWB evaluation value calculation unit 41 derives a white balance adjustment value and outputs it to the WB adjustment unit 8 as in step S302. The WB adjustment unit 8 performs white balance adjustment processing on all image signals from the memory 4 based on the white balance adjustment value output from the AWB evaluation value calculation unit 41.

  In addition, the color synchronization processing unit 9 performs color interpolation processing on all image signals output from the WB adjustment unit 8, and outputs R, G, and B signals output from each light receiving pixel of the image sensor 25 (FIG. 2). Generate and output to the compression processing unit 10.

  The compression processing unit 10 generates the entire display image signal by changing the data format of the image signal output from the color synchronization processing unit 9, performs compression processing on the entire display image signal, and compresses the entire compressed image. A signal is generated and output to the association processing unit 43.

  In step S <b> 310, the association processing unit 43 associates the compressed whole image signal from the compression processing unit 10 with the compressed cut-out image signal from the specific subject cut-out unit 42, stores it in the memory 12, and records it in the recording medium 13.

  In step S311, the CPU 17 determines whether an image recording end operation (recording end operation) has been performed from the operation unit 16, and when determining that the image recording end operation has been performed, the imaging apparatus 1 starts from the image recording mode. The process proceeds to the preview mode, and the process proceeds to step S312. If it is determined that the image recording end operation has not been performed, the process returns to step S306 to continue the image recording mode.

  In step S312, the CPU 17 determines whether or not an operation for ending the shooting mode has been performed from the operation unit 16, and when the ending operation has been performed, the shooting is ended and a series of processes is ended. If the end operation has not been performed, the process returns to step S301 to continue the preview mode.

************************************************** ***********************
Flowchart of cut-out process in S308
************************************************** ***********************
Next, the operation of the cutout process in step S308 in FIG. 15 will be described. FIG. 16 is a flowchart illustrating a flow of a series of processes when the specific subject clipping unit 42 of the imaging device 40 performs the clipping process.

  In step S <b> 400, the specific subject detection unit 50 detects the specific subject from all image signals from the memory 4, sets a specific subject region, and outputs the position information to the image cutout unit 51.

  In step S401, the image cutout processing unit 51 cuts out the specific subject region from all the image signals from the memory 4 based on the position information of the specific subject region from the specific subject detection unit 50, and the luminance adjustment unit 52 as a cutout image signal. Output to.

  In step S402, the luminance adjustment unit 52 divides the cut-out image signal output from the image cut-out processing unit 51 into a plurality of minute regions, and calculates a luminance signal for each minute region.

  Then, the calculated integrated value of the luminance signal of each minute region is used as a luminance evaluation value, and luminance adjustment is performed so that this value approaches a predetermined target value. In addition, the luminance adjustment unit 52 outputs the calculated luminance evaluation value to the WB adjustment unit 53.

  In step S403, the WB adjustment unit 53 divides the cut-out image signal output from the luminance adjustment unit 52 into a plurality of minute regions in the same manner as the luminance adjustment unit 52, and each of the R, G, and B signals for each minute region. Calculate the integrated value.

  Then, based on the integrated value and the luminance evaluation value from the luminance adjusting unit 52, the light source of each minute region is estimated. The light source with the highest frequency of the estimated light source is set as the light source of the cut-out image signal, and the white balance adjustment values for the R, G, and B signals set in advance for the determined light source are set as R, G, And the B signal are multiplied, and the clipped image signal after multiplication is output to the color synchronization processing unit 54.

  In step S <b> 404, the cut-out image signal output from the WB adjustment unit 53 is subjected to color synchronization processing and output to the compression processing unit 55.

    The compression processing unit 55 performs compression processing on the cutout image signal that has been subjected to the color synchronization processing, generates a compressed cutout image signal, and outputs the compressed cutout image signal to the association processing unit 43.

************************************************** ***********************
Flow chart of playback operation of imaging device 40
************************************************** ***********************
Next, an operation of the imaging device 40 when a moving image or still image reproduction operation is performed from the operation unit 16 (FIG. 1) will be described. FIG. 17 is a flowchart showing a flow of a series of processes when the imaging device 40 performs an image reproduction operation.

  In step S500, when a power-on operation and a reproduction mode setting operation are performed on the operation unit 16, the imaging device 40 is activated and enters a reproduction standby state.

  In step S501, the CPU 17 determines whether or not an image playback operation has been performed from the operation unit 16, and if it is determined that an image playback operation has been performed, the process proceeds to step S502, and if it is determined that no image playback operation has been performed, Continue playback standby.

  In step S502, the CPU 17 determines whether or not the operation for reproducing the cut-out image has been performed from the operation unit 16. If the CPU 17 determines that the operation for reproducing the cut-out image has been performed, the process proceeds to step S503. If it is determined that all the images have been reproduced, the process proceeds to step S506.

  In step S <b> 503, the image selection processing unit 61 of the imaging device 40 reads the compressed cut-out image signal recorded on the recording medium 13 into the memory 12 and outputs it to the expansion processing unit 63.

  If the imaging device 40 has reproduced all images last time, the compressed cutout image signal associated with all image signals corresponding to all images is extracted from the recording medium 13 and read into the memory 12, and the decompression processing unit To 63.

  If the association processing by the association processing unit 43 is a synthesis process that combines the compressed all image signal and the compressed cut-out image signal into one, the image selection processing unit 61 has already been read into the memory 12. A compressed cutout image signal is selected and output.

  In step S <b> 504, the decompression processing unit 63 (FIG. 14) performs decompression processing on the compressed cutout image signal from the image selection processing unit 61, generates a display cutout image signal, and outputs the display cutout image signal to the enlargement processing unit 64.

  The enlargement processing unit 64 subjects the display cut-out image signal from the decompression processing unit 63 to enlargement processing so as to conform to a predetermined format such as the NTSC format, and outputs the result to the multiplexer 65.

  In step S <b> 505, the multiplexer 65 selects the cut-out image signal for display that has been subjected to the enlargement process output from the enlargement processing unit 64, and outputs it to the image display unit 15. Then, the image display unit 15 displays a cutout image based on the display cutout image signal output from the multiplexer 65.

  In step S <b> 506, the image selection processing unit 61 of the imaging device 40 reads the compressed total image signal recorded on the recording medium 13 into the memory 12 and outputs it to the expansion processing unit 62. .

  If the image capturing apparatus 40 was playing a clipped image last time, the compressed full image signal associated with the clipped image signal corresponding to the clipped image is extracted from the recording medium 13 and read into the memory 12, and the decompression processing unit To 62.

  If the association processing by the association processing unit 43 is a synthesis process that combines the compressed all image signal and the compressed cut-out image signal into one, the image selection processing unit 61 has already been read into the memory 12. Select and output all compressed image signals.

  In step S <b> 507, the decompression processing unit 62 (FIG. 14) performs decompression processing on the compressed all image signals from the image selection processing unit 61, generates a display all image signal, and outputs it to the multiplexer 65.

  In step S508, the image display unit 15 displays all images based on all image signals output from the multiplexer 65.

  In step S509, the CPU 17 determines whether or not the playback end operation has been performed from the operation unit 16, and if it is determined that the playback end operation has been performed, the process proceeds to step S510, and if it is determined that the playback end operation has not been performed, Returning to step S502, reproduction is continued.

  In step S510, the CPU 17 determines whether or not an operation for ending the playback mode has been performed from the operation unit 16, and when it is determined that the ending operation has been performed, the series of processing ends. If it is determined that the ending operation has not been performed, the process proceeds to step S511.

  In step S511, when the next image data is designated from the operation unit 16, the imaging device 40 returns to step S502 to reproduce the designated image data.

  As described above, according to the second embodiment of the present invention, the imaging device 40 cuts out a specific subject area from all image signals acquired by the imaging unit 2 at the time of shooting, and generates a cut-out image signal. Then, the luminance adjustment and the white balance adjustment are performed on all the image signals, and the luminance adjustment and the white balance adjustment of the cut-out image signal are performed independently of the entire image signals.

  Thereby, brightness adjustment and white balance adjustment can be performed on all images based on the color information included in all image signals, and brightness on the basis of color information included only in the cut-out image signal for cut-out images. Adjustment and white balance adjustment can be performed. Thereby, the brightness adjustment and the white balance adjustment suitable for the entire image can be performed on all the images, and the brightness adjustment and the white balance adjustment suitable for the clipped image can be performed on the cut out images independently.

  The imaging device 40 also records the compressed whole image signal and the compressed cut-out image signal in the recording medium 13 in association with each other.

  As a result, during playback of all images, when switching to playback of a clipped image cut out from the entire image, or vice versa, switching can be performed without ending the playback process.

************************************************** ***********************
Modified example
************************************************** ***********************
In the second embodiment, the specific subject detection unit 50 sets the specific subject region as a rectangular region from all the image signals, but is not limited thereto, and may be set as a circle or an ellipse.

  The image device 1 or the imaging device 40 in the first and second embodiments can be realized by hardware or a combination of hardware and software.

In particular, the image processing of each block executed in the imaging device 1 and the imaging device 40 of FIG. It can be realized by software or a combination of hardware and software.

1 is an overall configuration diagram of an imaging apparatus 1 according to a first embodiment of the present invention. It is an internal block diagram of the imaging part 2 of FIG. It is a figure which shows the example of the arrangement | sequence of a color filter, a light reception pixel, a pixel signal, and an image signal. It is a figure which shows the example of the area | region division of an image signal. It is an example of the data structure of the signal produced | generated by each of the correlation process part 11 of FIG. 1, the expansion | extension process part 30 of FIG. It is an internal block diagram of the image reproduction process part 14 of FIG. It is a figure which shows the operation example of the specific subject detection by the specific subject detection part 31 of FIG. 3 is a flowchart illustrating an example of an imaging procedure by the imaging apparatus 1 of FIG. 4 is a flowchart illustrating an example of a reproduction procedure of an extended compressed image signal recorded on a recording medium 13 by the imaging apparatus 1 of FIG. It is a whole block diagram of the imaging device 40 which concerns on 2nd Embodiment of this invention. It is an internal block diagram of the specific subject cutout part 42 of FIG. FIG. 12 is a diagram illustrating an operation example of specific subject detection by the specific subject detection unit in FIG. 11. It is a figure which shows the operation example of the brightness | luminance adjustment and white balance adjustment by the brightness | luminance adjustment part 52 and the WB adjustment part 53 of FIG. It is an internal block diagram of the image reproduction process part 44 of FIG. It is a flowchart which shows the example of an imaging | photography procedure by the imaging device 40 of FIG. 11 is a flowchart illustrating an example of a procedure of cutout processing by a specific subject cutout unit in FIG. 10. 11 is a flowchart illustrating an example of a reproduction procedure of a compressed full image signal or a compressed cut-out image signal recorded on the recording medium 13 by the imaging device 40 of FIG.

Explanation of symbols

6 AE evaluation value calculation unit 7 AWB evaluation value calculation unit 8 WB adjustment unit 14 Image reproduction processing unit 31 Specific subject detection unit 32 Image segmentation processing unit 33 Brightness adjustment unit 34 WB adjustment unit

Claims (5)

Photographing means for obtaining an image signal to be photographed by photographing;
Color information deriving means for dividing the image signal acquired by the photographing means into a plurality of small areas and deriving color information of each small area;
Compression processing means for performing compression processing on the image signal to generate a compressed image signal;
An image pickup apparatus comprising: an association unit configured to associate and record the compressed image signal generated by the compression processing unit and the color information derived by the color information deriving unit.




A compressed first image signal generated by performing a compression process on the first image signal acquired by photographing, and color information of the first image signal associated with the compressed first image signal are recorded from a recording medium. Acquisition means for acquiring;
Decompression means for subjecting the compressed first image signal acquired by the acquisition means to expansion processing to generate a display first image signal;
First reproduction means for reproducing the first image signal for display generated by the expansion means;
Generating means for extracting a specific subject area from the display first image signal generated by the decompression means and generating a display second image signal;
A white balance adjusting unit that performs a white balance adjustment process on the second image signal for display generated by the generating unit based on the color information;
The image reproduction apparatus, wherein the second reproduction means reproduces the display second image signal that has been subjected to the luminance adjustment processing by the luminance adjustment processing means and the white balance adjustment processing by the white balance adjustment means.
A compressed first image signal generated by performing a compression process on the first image signal obtained by photographing, and color information of the first image signal associated with the compressed first image signal are recorded from a recording medium. Acquisition means for acquiring;
Decompression means for performing decompression processing on the compressed first image signal acquired by the acquisition means to generate a first image signal for display;
First reproduction means for reproducing the first image signal for display generated by the expansion means;
Generating means for extracting a specific subject area from the display first image signal generated by the expansion means and generating a display second image signal;
Brightness adjusting means for performing brightness adjustment processing on the second image signal for display generated by the generating means based on the color information;
Second reproduction means for reproducing the second image signal for display subjected to luminance adjustment processing by the luminance adjustment means;
An image reproducing apparatus comprising: A white balance adjusting unit that performs a white balance adjustment process on the second image signal for display generated by the generating unit based on the color information;
The said 2nd reproduction | regeneration means reproduces | regenerates the said 2nd image signal for a display by which the brightness adjustment process by the said brightness adjustment process means and the white balance adjustment process by the said white balance adjustment means were performed. The image reproducing device described. Photographing means for obtaining a first image signal to be photographed by photographing;
Image processing means for generating a compressed first image signal by performing white balance adjustment processing, color synchronization processing, and compression processing on the first image signal acquired by the photographing means;
Generating means for extracting a specific subject area from the first image signal and generating a second image signal;
Brightness adjusting means for performing brightness adjustment processing on the second image signal generated by the generating means based on color information included in the second image signal;
White balance adjustment means for performing white balance adjustment processing on the second image signal generated by the generation means based on color information included in the second image signal;
Compression processing means for performing color synchronization processing and compression processing on the second image signal after brightness adjustment and white balance adjustment to generate a compressed second image signal;
Recording means for recording the compressed first image signal generated by the image processing means and the compressed second image signal generated by the compression processing means;
An imaging apparatus comprising: