JP6210898B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for changing the hue and saturation of an image.

  Some digital cameras on the market in recent years have a function of obtaining an image with a special effect by changing the color tone or gradation of the image or adding noise or blur to the image. Such special effects can also be obtained by executing software in a device other than a digital camera, such as a smartphone or a personal computer (PC).

  As an example of a technique for obtaining a special effect, JP2013-175960A discloses a luminance component image extracted from original image data, and the extracted luminance component image is colored into a sepia color to generate a sepia tone image. A technique is described in which a generated sepia-tone image is combined with an original image to give a special effect that leaves a feeling similar to that of an actually printed photograph, that is, a sense of reality.

  By the way, a color change such as fading due to secular change or tanning by sunlight (hereinafter referred to as “sunburn” as appropriate for this color change) may occur in a printed photograph.

JP 2013-175960 A

  In the case where the effect of sunburn as described above is to be realized in an image taken with a digital camera, using the technique described in Japanese Patent Application Laid-Open No. 2013-175960, a sunburned texture is realized to some extent. be able to.

  However, the actual sunburn of the printed photograph is not only uniform in the overall color change (for example, a color change in which unevenness such as only a part of the color changes occurs), but also occurs for each photo. Whereas the color change is different, the image obtained by the technique described in the above publication is always an image with a certain effect added, so how the sunburn effect is applied. It is thought that the user can easily predict it and it will be unexpected and boring.

  Therefore, as a countermeasure, it may be possible to apply a different tanning effect for each photo using random numbers. In this case, for example, images obtained by continuous shooting are similar to each other or related to each other. Since separate effects are applied to images and the like, there is a possibility that the user feels uncomfortable when viewing and becomes a function with low convenience.

  The present invention has been made in view of the above circumstances, and it is possible to change the color of an image that is not random but highly convenient, while the result cannot be easily predicted and there is room for unexpectedness. An object is to provide a processing device, an image processing method, and a program.

An image processing apparatus according to an aspect of the present invention is configured to calculate a representative value corresponding to a partial area based on pixel values extracted from luminance signals of a plurality of pixels constituting the partial area of the image. And an analysis unit for obtaining a representative value arrangement based on the calculated representative values and the arrangement of the partial areas, and a gain corresponding to each pixel constituting the image based on the representative value arrangement. A gain map generating unit that obtains a gain map in which a region where the gain value is maximum corresponds to a partial region other than the partial region where the representative value is maximum; and And a color change processing unit that changes at least one of hue and saturation of the image.

An image processing method according to an aspect of the present invention is to calculate a representative value corresponding to a partial area based on pixel values extracted from luminance signals of a plurality of pixels constituting the partial area of the image. And an analysis step for obtaining a representative value arrangement based on the calculated representative values and a plurality of partial area arrangements, and a gain corresponding to each pixel constituting the image based on the representative value arrangement A gain map generating step for obtaining a gain map in which a region where the gain value is maximum corresponds to a partial region other than the partial region where the representative value is maximum; and And a color change processing step for changing at least one of hue and saturation of the image.

A program according to an aspect of the present invention is a program for causing a computer to perform image processing, and causes a computer to perform image processing on a partial area based on pixel values extracted from luminance signals of a plurality of pixels constituting the partial area of the image. A corresponding representative value is calculated for a plurality of partial areas, and an analysis step for obtaining a representative value arrangement based on the calculated representative values and a plurality of the partial area arrangements; and the representative value arrangement And a gain map for giving a gain value corresponding to each pixel constituting the image, such that a region where the gain value is maximum corresponds to a partial region other than the partial region where the representative value is maximum. A gain map generation step for obtaining a correct gain map, and a color change processing step for changing at least one of hue and saturation of the image based on the gain map. When a program for execution.

  According to the image processing apparatus, the image processing method, and the program of the present invention, it is possible to change the color of an image that is not random but highly convenient, while the result cannot be easily predicted and there is room for unexpectedness. It becomes possible.

1 is a block diagram illustrating a configuration of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 2 is a block diagram showing a configuration of a special image processing unit in the first embodiment. FIG. 3 is a diagram illustrating an example of an image represented by image data stored in an SDRAM that is an analysis target by an analysis unit in the first embodiment. The figure which shows the 1st-4th partial area | region of the object which calculates a representative value in the image of the said Embodiment 1. FIG. In the said Embodiment 1, the figure which shows arrangement | positioning of the representative value obtained by the analysis part based on arrangement | positioning and a representative value of the 1st-4th partial area | region. In the said Embodiment 1, the figure which shows arrangement | positioning of the representative value produced by changing the arrangement | positioning of FIG. 5 by the gain map production | generation part. The figure which shows the gain map calculated by the gain map production | generation part based on arrangement | positioning of FIG. 6 in the said Embodiment 1. FIG. The figure which shows the 1st-6th partial area | region of the object which calculates a representative value in the image of the said Embodiment 1. FIG. In the said Embodiment 1, the figure which shows arrangement | positioning of the representative value obtained by the analysis part based on arrangement | positioning and a representative value of the 1st-6th partial area | region. In the said Embodiment 1, the figure which shows arrangement | positioning of the representative value produced by changing the arrangement | positioning of FIG. 9 by the gain map production | generation part. The figure which shows the gain map computed by the gain map production | generation part based on arrangement | positioning of FIG. 10 in the said Embodiment 1. FIG. 3 is a flowchart showing a flow of main processing in the imaging apparatus according to the first embodiment. 3 is a flowchart showing a flow of image processing performed by an image processing unit of the imaging apparatus according to the first embodiment. 3 is a flowchart illustrating color change processing performed by a color change processing unit of the imaging apparatus according to the first embodiment. 6 is a chart showing an example of a basic gain map stored in advance in the imaging apparatus according to the second embodiment of the present invention. The figure which shows the basic gain map selected by the gain map production | generation part in the said Embodiment 2. FIG. The figure which shows the gain map produced by interpolating the basic gain map shown in FIG. 16 in the said Embodiment 2. FIG. FIG. 10 is a diagram showing a modification of the basic gain map stored in advance in the imaging apparatus in the second embodiment. FIG. 10 is a diagram illustrating an example of a correction coefficient map stored in advance by an imaging apparatus in Embodiment 3 of the present invention. The figure which shows the example of the correction gain map obtained by correct | amending a gain map based on the correction coefficient map in the said Embodiment 3. FIG.

Embodiments of the present invention will be described below with reference to the drawings.
[Embodiment 1]

  1 to 14 show Embodiment 1 of the present invention, and FIG. 1 is a block diagram showing a configuration of an imaging apparatus.

  In this embodiment, the image processing apparatus is applied to an imaging apparatus, and the imaging apparatus is configured as a digital camera as a specific example.

  That is, as shown in FIG. 1, this imaging apparatus is configured by connecting an interchangeable lens 1 and a camera body 2 via an interface (I / F) 3 so that they can communicate with each other.

  Next, the interchangeable lens 1 is detachably attached to the camera body 2 via, for example, a lens mount, and an electrical contact formed on the lens mount (electricity provided on the interchangeable lens 1 side). The interface 3 is composed of contacts and electrical contacts provided on the camera body 2 side. Thus, the interchangeable lens 1 can communicate with the camera body 2 via the interface 3.

  The interchangeable lens 1 includes a lens 11, a diaphragm 12, a driver 13, a microcomputer 14, and a flash memory 15.

  The lens 11 is a photographic optical system for forming an optical image of a subject on an image sensor 23 described later of the camera body 2.

  The diaphragm 12 is an optical diaphragm that controls the passage range of the light beam that passes through the lens 11.

  The driver 13 adjusts the focus position by driving the lens 11 based on a command from the microcomputer 14 and further changes the focal length when the lens 11 is an electric zoom lens or the like. In addition, the driver 13 drives the diaphragm 12 based on a command from the microcomputer 14 to change the aperture diameter. By driving the diaphragm 12, the brightness of the optical image of the subject changes, and the size of the blur also changes.

  The microcomputer 14 is a so-called lens-side computer, and is connected to the driver 13, the flash memory 15, and the interface 3. The microcomputer 14 communicates with a microcomputer 41, which is a main body computer, which will be described later, via the interface 3, receives an instruction from the microcomputer 41, and reads / writes information stored in the flash memory 15 And the driver 13 is controlled. Further, the microcomputer 14 transmits various information regarding the interchangeable lens 1 to the microcomputer 41.

  The flash memory 15 is a storage medium that stores a control program executed by the microcomputer 14 and various types of information regarding the interchangeable lens 1 in a nonvolatile manner.

  The interface 3 connects the microcomputer 14 of the interchangeable lens 1 and the microcomputer 41 of the camera body 2 so as to be capable of bidirectional communication.

  Subsequently, the camera body 2 includes a shutter 21, an imaging unit 22, a RAW reduction processing unit 26, an image processing unit 27, an AE processing unit 30, an AF processing unit 31, an image compression / decompression unit 32, and an LCD. The driver 33, the LCD 34, a memory interface (memory I / F) 35, a recording medium 36, an SDRAM 37, an operation unit 38, a flash memory 39, a bus 40, and a microcomputer 41 are provided.

  The shutter 21 controls the time required for the light flux from the lens 11 to reach the image sensor 23, and is, for example, a mechanical shutter configured to run a shutter curtain. However, an element shutter may be used instead of the mechanical shutter, or a mechanical shutter and an element shutter may be used in combination. The shutter 21 is driven by a command from the microcomputer 41 to control the time during which the light beam reaches the image sensor 23, that is, the exposure time of the subject by the image sensor 23.

  The imaging unit 22 captures a subject and acquires image data, and includes an imaging element 23, an analog processing unit 24, and an analog / digital conversion unit (A / D conversion unit) 25.

  The image sensor 23 photoelectrically converts an optical image of a subject formed through the lens 11 and the diaphragm 12 to generate an analog image signal. The image sensor 23 is configured as a single-plate image sensor in which a Bayer array color filter is arranged in front of a plurality of pixels arranged in the vertical direction and the horizontal direction, for example. Needless to say, the image sensor 23 is not limited to a single-plate image sensor, and may be, for example, a stacked image sensor that separates color components in the substrate thickness direction.

  The analog processing unit 24 shapes the waveform of the analog image signal read from the image sensor 23 while reducing reset noise and the like, and further increases the gain so that the target brightness is obtained.

  The A / D conversion unit 25 converts the analog image signal output from the analog processing unit 24 into a digital image signal (hereinafter referred to as image data). The image data output from the A / D converter 25 is so-called RAW image data.

  The RAW reduction processing unit 26 reduces the size (number of pixels) of the RAW image data as necessary. The RAW reduction processing unit 26 reduces the RAW image data reduced by, for example, thinning, adding pixels, or interpolating the RAW image data. Generate. The RAW reduction processing unit 26 may reduce the image data output from the A / D conversion unit 25, but may reduce the RAW image data temporarily stored in the SDRAM 37.

  The RAW image data output from the A / D conversion unit 25 (or the reduced RAW image data output from the RAW reduction processing unit 26) is transferred via the bus 40 and temporarily stored in the SDRAM 37.

  The image processing unit 27 performs various types of image processing on the RAW image data (or reduced RAW image data, hereinafter not shown), and includes a basic image processing unit 28, a special image processing unit 29, It is equipped with.

  The basic image processing unit 28 performs basic image processing on the RAW image data. Examples of the image processing performed by the basic image processing unit 28 include optical black (OB) subtraction processing, white balance (WB) processing, synchronization processing, color reproduction processing, luminance change processing, edge enhancement processing, noise reduction processing, and the like. Is mentioned. The image data after each processing is performed by the basic image processing unit 28 is stored in the SDRAM 37 again.

  The special image processing unit 29 performs image processing that does not correspond to the basic image processing described above, for example, processing related to image processing. The special image processing unit 29 in the present embodiment performs processing including color change processing for changing at least one of hue and saturation of an image. Specifically, the special image processing unit 29 according to the present embodiment gives the image data color change as in the case where the printed photograph fades due to secular change or sunburns due to sunlight as the color change processing. Process. The image data processed by the special image processing unit 29 is stored in the SDRAM 37.

  The AE processing unit 30 calculates subject brightness based on the RAW image data. The subject brightness calculated here is used for automatic exposure (AE) control, that is, control of the diaphragm 12, control of the shutter 21, control of the exposure timing of the image sensor 23, and the like. Note that RAW image data or the like is used here as data for calculating the subject brightness, but instead, a dedicated photometric sensor is provided in the imaging apparatus and data obtained from the photometric sensor is used. It doesn't matter if you do.

  The AF processing unit 31 extracts high-frequency component data from the RAW image data, and acquires a focus evaluation value by AF (autofocus) integration processing. The focus evaluation value acquired here is used for AF driving of the lens 11. Needless to say, the AF is not limited to such contrast AF. For example, a phase difference AF may be performed using a dedicated AF sensor (or an AF pixel on the image sensor 23).

  The image compression / decompression unit 32 reads the image data from the SDRAM 37 when recording the image data, and the read image data is, for example, a JPEG compression method for a still image, a JPEG compression method or an H for a moving image. . The compressed image data is temporarily stored in the SDRAM 37 according to a compression method such as the H.264 compression method. The compressed image data stored in the SDRAM 37 in this way is arranged as recording data by adding a header necessary for composing a file by the microcomputer 41. Based on the control of the microcomputer 41, the prepared recording data is recorded on the recording medium 36 via the memory I / F 35. In the imaging apparatus according to the present embodiment, the number of pixels of an image to be recorded on the recording medium 36 can be selected and set from, for example, a plurality of options.

  The image compression / decompression unit 32 also decompresses the read image data. That is, when a recorded image is played back, for example, a JPEG file is read from the recording medium 36 via the memory I / F 35 based on the control of the microcomputer 41 and temporarily stored in the SDRAM 37. The image compression / decompression unit 32 reads the JPEG image data stored in the SDRAM 37, expands the read JPEG image data according to the JPEG expansion method, and stores the expanded image data in the SDRAM 37.

  The LCD driver 33 reads the image data stored in the SDRAM 37, converts the read image data into a video signal, controls the drive of the LCD 34, and causes the LCD 34 to display an image based on the video signal. The image display performed by the LCD driver 33 includes a rec view display that displays image data immediately after shooting for a short time, a playback display of a JPEG file recorded on the recording medium 36, a moving image display in a live view display, and the like. included. When live view display or REC view display is performed using the above-described reduced RAW image data, the processing time may be shortened and the processing load may be reduced.

  The LCD 34 is a panel capable of displaying an image such as a liquid crystal panel or an organic EL panel. The LCD 34 displays an image and various types of information related to the imaging apparatus by the drive control of the LCD driver 33 as described above.

  As described above, the memory I / F 35 writes information such as image data to the recording medium 36 and reads information such as image data from the recording medium 36.

  The recording medium 36 stores information such as image data in a non-volatile manner, and is composed of, for example, a memory card that can be attached to and detached from the camera body 2. However, the recording medium 36 is not limited to a memory card, and may be a disk-shaped recording medium, a recording medium fixed to the camera body 2, or a recording medium at a remote position connected via a communication line. It may be any other suitable recording medium. Thus, the recording medium 36 does not have to have a configuration unique to the imaging apparatus.

  The SDRAM 37 is a storage unit that temporarily stores various data such as the above-described RAW image data or image data processed by the image processing unit 27, the image compression / decompression unit 32, and the like.

  The operation unit 38 is for performing various operation inputs to the image pickup apparatus, such as a power button for turning on / off the power supply of the image pickup apparatus, for example, a 1st (first) release for instructing start of image capturing. Release button consisting of a two-stage operation button having a switch and a 2nd (second) release switch, a playback button for playing back a recorded image, a menu button for setting an imaging device, and a moving image It includes an operation button such as a moving image button for shooting, a cross key used for an item selection operation, an OK button used for a selection item determination operation, and the like. When the operation unit 38 is operated, a signal corresponding to the operation content is output to the microcomputer 41.

  The flash memory 39 is a storage medium that stores processing programs executed by the microcomputer 41 and various types of information related to the imaging apparatus in a nonvolatile manner. Here, the information stored in the flash memory 39 includes, for example, various parameters necessary for the operation of the imaging apparatus such as a white balance gain according to the white balance mode, and a model number and a manufacturing number for specifying the imaging apparatus. Are some examples. Information stored in the flash memory 39 is read by the microcomputer 41.

  The bus 40 is a transfer path for transferring various data and control signals generated at a certain location in the imaging apparatus to another location in the imaging apparatus. The bus 40 in this embodiment includes an A / D conversion unit 25, a RAW reduction processing unit 26, an image processing unit 27, an AE processing unit 30, an AF processing unit 31, an image compression / decompression unit 32, and an LCD driver. 33, a memory I / F 35, an SDRAM 37, and a microcomputer 41.

  The microcomputer 41 controls each part in the camera body 2 and transmits a command to the microcomputer 14 via the interface 3 to control the interchangeable lens 1. The microcomputer 41 controls the image pickup apparatus in an integrated manner. Part. When the user performs an operation input from the operation unit 38, the microcomputer 41 reads parameters necessary for processing from the flash memory 39 according to a processing program stored in the flash memory 39, and performs various sequences according to the operation content. Execute.

  Next, FIG. 2 is a block diagram illustrating a configuration of the special image processing unit 29.

  The special image processing unit 29 includes an analysis unit 45, a gain map generation unit 46, and a color change processing unit 47.

  The analysis unit 45 calculates a representative value corresponding to the partial region based on the pixel values of the pixels constituting the partial region of the image, and calculates the plurality of representative values and the plurality of partial regions. The representative value arrangement is obtained based on the arrangement of

  The gain map generation unit 46 is a gain map that gives a gain value corresponding to each pixel constituting the image based on the arrangement of the representative values obtained by the analysis unit 45, and the region where the gain value is maximum is the representative. A gain map corresponding to a partial region other than the partial region having the maximum value is obtained. Here, the gain map generator 46 obtains a gain map in which the spatial distribution of gain values changes smoothly. Furthermore, the gain map generation unit 46 has a distribution in which the histogram of gain values on the gain map is flattened (however, it does not mean that the gain map is completely flat, and that the bias is relatively small). It is preferable to obtain a large gain map, that is, a bias map with a relatively small bias such as a bias with many pixels with a small gain value or a bias with many pixels with a large gain value.

  Specifically, the gain map generation unit 46 of the present embodiment arranges a plurality of representative values at a plurality of predetermined pixel positions on the gain map so as to vary the arrangement of the representative values, and arbitrarily selects from the predetermined pixel positions. The gain map is calculated by interpolating the gain value at an arbitrary pixel position based on the distance to the pixel position. Here, the gain map calculated by the gain map generator 46 is stored in the SDRAM 37.

  The color change processing unit 47 changes at least one of hue and saturation of the image based on the gain map output from the gain map generation unit 46.

Specifically, if the pixel position in the image is (x, y) and at least one of the components of the color space representing the image is represented by C, it depends on the pixel position. The predetermined constant is C_cnt, the component before the color change by the color change processing unit 47 at the pixel position (x, y) is C_in (x, y), the component after the color change is C_out (x, y), and the gain If the gain value of the map is G (x, y), the color change processing unit 47, for example,
C_out (x, y) = C_in (x, y) + C_cnt × G (x, y)
By performing this calculation for all pixel positions (x, y), the color of the image is changed. Therefore, the larger the gain value G (x, y), the larger the amount of color change (On the other hand, if the gain value G (x, y) is 0, the color is not changed).

  An example of gain map creation processing performed by the special image processing unit 29 will be described with reference to FIGS.

  FIG. 3 is a diagram illustrating an example of an image P represented by image data stored in the SDRAM 37, which is an object of analysis by the analysis unit 45.

  The image data stored in the SDRAM 37 is, for example, RAW image data output from the imaging unit 22 or read from the recording medium 36, or reduced RAW image data generated by the RAW reduction processing unit 26 or read from the recording medium 36. Any one of image data that has undergone basic image processing by the basic image processing unit 28, image data that has been read from the recording medium 36 and expanded by the image compression / decompression unit 32, and the like. For example, the main subject OBJ is shown in the center of the image P shown in FIG. 3, and the upper left side of the background of the main subject OBJ is a portion with low luminance, and the lower right side is a portion with high luminance. (Here, it is difficult to express smooth gradation on the drawing, but it does not look like that, but the gradation changes smoothly from the low-brightness area on the upper left to the high-luminance area on the lower right. The same shall apply hereinafter).

  The analysis unit 45 extracts a luminance signal of the image P as shown in FIG. 3 (or a luminance equivalent signal that can be approximately regarded as a luminance signal, and so on). Then, the analysis unit 45 divides the image P related to the extracted luminance signal into, for example, 1/8 each of the vertical and horizontal directions, out of a total of 64 partial regions obtained by the division, for example, the first partial region R1 at the upper left corner, The second partial region R2 in the upper right corner, the third partial region R3 in the lower left corner, and the fourth partial region R4 in the lower right corner are set as partial regions for which representative values are calculated. FIG. 4 is a diagram illustrating first to fourth partial regions R1 to R4 for which representative values are calculated in the image P.

  Then, the analysis unit 45 calculates and calculates an average pixel value for each of the first to fourth partial regions R1 to R4 by integrating the pixel values in the partial region and dividing by the number of pixels in the partial region. The average pixel value is set as representative values V1 to V4 of the partial regions R1 to R4. It is assumed that the representative values V1 to V4 obtained here are, for example, V4> V2> V3> V1.

  Although the average pixel value of the partial region is used as the representative value here, the median value of the pixel values of the partial region may be used as the representative value, and for example, the maximum value of the pixel values of the partial region may be used as the representative value. More generally, the representative value calculation method is not limited to these.

  FIG. 5 is a diagram illustrating the arrangement of representative values obtained by the analysis unit 45 based on the arrangement of the first to fourth partial regions R1 to R4 and the representative values V1 to V4.

  In the 2 × 2 arrangement of representative values shown in FIG. 5, the representative value V1 of the first partial region R1 at the upper left corner is arranged at the upper left, the representative value V2 of the second partial region R2 at the upper right corner is arranged at the upper right, and the lower left The representative value V3 of the third partial region R3 at the corner is arranged at the lower left, and the representative value V4 of the fourth partial region R4 at the lower right corner is arranged at the lower right.

  The gain map generation unit 46 is one of the partial regions R1 to R3 corresponding to the representative values V1 to V3 other than the maximum representative value V4 based on the arrangement of the representative values as shown in FIG. The representative values are arranged differently so that the maximum representative value V4 is allocated. The reason for performing such processing is to obtain a higher color change effect. That is, the fourth partial region R4 in which the maximum representative value V4 is obtained is a region with high luminance. When the luminance is high, the color expression range becomes narrower as it approaches white (color saturation state). This is because the effect obtained even if the change process is performed is low. Specifically, the gain map generation unit 46 rotates the representative value arrangement shown in FIG. 5 by 90 degrees counterclockwise, for example. FIG. 6 is a diagram showing the arrangement of representative values created by the gain map generation unit 46 by changing the arrangement of FIG.

  Next, the gain map generation unit 46 performs normalization so that the representative value becomes a value within a predetermined range in the changed representative value arrangement shown in FIG. In this normalization, for example, the maximum representative value V4 is set to 1, the minimum representative value V1 is set to 0, and the other representative values V2 and V3 are in the range of more than 0 and less than 1 depending on the respective values. It is processing to become. With this normalization, the gain value histogram in the generated gain map may have a wider dynamic range of gain values than the gain value histogram in the gain map generated without this normalization. (For example, when the dynamic range that a pixel value can take is 0 to 1023, the minimum representative value is 10, and the maximum representative value is 1000, 0 to 1023 is simply converted to 0 to 1. The minimum representative value is about 0.01 and the maximum representative value is about 0.98. However, when the normalization of this embodiment is used, the minimum representative value is 0.0 and the maximum representative value is 1.0. To be). However, since the magnitude of the change amount in the color change process can be changed by changing the magnitude of the constant value multiplied by the gain value (the predetermined constant C_cnt described above), in that case This normalization process can be omitted.

  Subsequently, the gain map generation unit 46 generates a gain map that gives a gain value corresponding to each pixel constituting the image, based on the arrangement of the representative values after change (or after normalization). This gain map is applied to, for example, an image that has undergone basic processing by the basic image processing unit 28, that is, a pixel configuration map that has a one-to-one correspondence with the pixel configuration of the image that has undergone basic processing. Create

  Specifically, as shown in FIG. 6, the upper left corner representative value V2 is the gain value of the upper left corner pixel of the gain map, the upper right corner representative value V4 is the gain value of the upper right corner pixel of the gain map, and the lower left corner representative value. Based on the distance from these four predetermined pixel positions to any pixel position, where V1 is the gain value of the pixel in the lower left corner of the gain map and the representative value V3 of the lower right corner is the gain value of the pixel in the lower right corner of the gain map The gain map is calculated by interpolating (for example, linear interpolation) the gain value at an arbitrary pixel position. Here, the reason why the gain map is calculated based on the distance is that it is preferable to create a gain map with a smooth gradation. However, since there are various interpolation methods, other appropriate maps are used. Of course, the method may be used. The calculated gain map is stored in the SDRAM 37, for example.

  FIG. 7 is a diagram showing the gain map GM calculated by the gain map generator 46 based on the arrangement of FIG.

  The gain map GM shown in FIG. 7 is calculated based on the changed representative value arrangement shown in FIG. 6, and therefore, the gain value in the lower left part is low and the gain value in the upper right part is high. The other portions have intermediate gain values corresponding to the distances from the four corners that are predetermined pixel positions.

  Another example of gain map creation processing performed by the special image processing unit 29 will be described with reference to FIGS.

  In the example shown in FIG. 4, the 2 × 2 arrangement of the representative values shown in FIG. 5 is obtained based on the first to fourth partial regions R1 to R4 of the four corners, but here, as shown in FIG. A 3 × 2 arrangement of representative values is obtained by using partial regions R5 and R6 at intermediate positions along the long side direction of the image P having a rectangular shape. As described above, the number of partial regions for which the representative value is calculated is not limited to four, and may be any appropriate plural number.

  FIG. 8 is a diagram illustrating first to sixth partial regions R1 to R6 for which representative values are calculated in the image P.

  The fifth partial region R5 is located in the middle between the first partial region R1 and the second partial region R2, and the sixth partial region R6 is located in the middle between the third partial region R3 and the fourth partial region R4. Yes. The analysis unit 45 calculates not only the representative values V1 to V4 as described above, but also calculates the representative value V5 from the fifth partial region R5 and the representative value V6 from the sixth partial region R6. And the analysis part 45 acquires arrangement | positioning of a representative value as shown in FIG.

  FIG. 9 is a diagram illustrating the arrangement of representative values obtained by the analysis unit 45 based on the arrangement of the first to sixth partial regions R1 to R6 and the representative values V1 to V6.

  In the example shown in FIG. 9, the representative value V5 is, for example, an intermediate value between the representative value V1 and the representative value V2, and the representative value V6 is, for example, an intermediate value between the representative value V3 and the representative value V4.

  For example, the gain map generation unit 46 rotates the representative value arrangement shown in FIG. 9 by 135 degrees clockwise, for example. FIG. 10 is a diagram illustrating the arrangement of representative values created by the gain map generation unit 46 by changing the arrangement of FIG. 9. Furthermore, the gain map generation unit 46 performs normalization of the representative values as described above in the arrangement of the changed representative values shown in FIG. 10 as necessary.

  As described above, the process of changing the arrangement of the representative values is not limited to the rotation in units of 90 degrees, and may be performed by rotation at an arbitrary angle. The rotation direction and rotation angle at this time are determined based on, for example, the positional relationship between the maximum representative value and the representative value adjacent to the maximum representative value and the relationship between the values of the representative values. (See also Embodiment 2 described later).

  Furthermore, the change of the representative value is not limited to the rotation, but may be performed by reversing the mirror symmetry, and one set of representative values may be exchanged once or two different combinations. You may perform by performing more than once.

  Subsequently, the gain map generation unit 46 creates a gain map GM as shown in FIG. 11 based on the arrangement of the representative values after change (or after normalization). FIG. 11 is a diagram showing the gain map GM calculated by the gain map generator 46 based on the arrangement of FIG.

  Since the gain map GM shown in FIG. 11 is calculated based on the changed arrangement of the representative values shown in FIG. 10, the gain value is distributed higher on the left side and lower on the right side.

  As described above, the number of partial areas for which the representative value is calculated may be an appropriate plural number, and the method for changing the arrangement of the representative value is not limited to a specific method. The distribution may be such that the gain value in any part in the peripheral part is higher than in the central part. This is because when the gain value in the center is high (that is, the color change in the center is large), the color of the whole image may appear unnatural and prevent such an image from appearing. Because.

  For example, when reduced RAW image data is used as the image data for creating the gain map GM as shown in FIG. 7 or FIG. 11, the processing is performed because the number of pixels is smaller than that of the RAW image data. It can be performed at high speed and with low load. Further, if the size of the partial area set in the reduced RAW image data (the number of pixels included in the partial area) is the same as the size of the partial area set in the unreduced RAW image data, it occupies the image. Since the ratio of the size of the partial area is larger than that in the case of unreduced RAW image data, it is less susceptible to noise and slight changes in the angle of view, and an advantage that a stable gain map GM can be created. There is.

  Further, when using the reduced RAW image data or the RAW image data, the pixel data corresponding to the most sensitive filter in the color filter of the image sensor 23 is extracted and used as a luminance-corresponding signal. The processing can be performed at a lower load than when a luminance signal is generated from the pixel data corresponding to the filter and used.

  When the gain map GM is created as described above, the color change processing unit 47 uses at least the hue and saturation of the image subjected to the basic image processing using the gain value G at each pixel position of the gain map GM. Change one.

  Specifically, the case where the image that has undergone the basic image processing is an RGB image, that is, the color space that represents the image is the RGB color space, and the components C related to hue or saturation are R, G, and B will be described. To do.

As described above for the component C, the pixel position is (x, y), and the pixel values of the respective color components before color change by the color change processing unit 47 are R_in (x, y), G_in (x, y), B_in (x, y), pixel values of each color component after color change are R_out (x, y), G_out (x, y), B_out (x, y), and predetermined constants that do not depend on the pixel position for each color component. When the gain value corresponding to the pixel position in R_cnt, G_cnt, B_cnt and gain map GM is G (x, y), the color change processing unit 47
R_out (x, y) = R_in (x, y) + R_cnt × G (x, y)
G_out (x, y) = G_in (x, y) + G_cnt × G (x, y)
B_out (x, y) = B_in (x, y) + B_cnt × G (x, y)
The color change process is performed by performing the above calculation.

In order for this color change process to be a process that changes at least one of hue and saturation,
R_cnt ≠ G_cnt
G_cnt ≠ B_cnt
B_cnt ≠ R_cnt
It is necessary that at least one of the above holds.

An example of the value of such a predetermined constant is
(R_cnt, G_cnt, B_cnt) = (25, -15, -10)
It is. This constant value is a value for adding magenta color to the image. Accordingly, constants (R_cnt, G_cnt, B_cnt) may be appropriately selected according to the color to be added to the original image. This constant may be changed in units of frames or may be changed according to the shooting scene.

  Here, the RGB color space is exemplified as a color space for representing an image, but various colors such as a YCbCr color space, a YUV color space, a YPbPr color space, an L * a * b * color space, and an HSV color space are used. Similar color change processing can be performed in the space. For example, since the YCbCr color space, YUV color space, YPbPr color space, or L * a * b * color space are all divided into luminance components and color difference components, at least one of the color difference components is used. The processing as described above may be performed (the luminance component may or may not be performed). In the case of the HSV color space, the above-described process may be performed on at least one of the hue H and the saturation S (the brightness V may or may not be performed). .

  Next, processing of the imaging apparatus will be described with reference to FIGS.

  First, FIG. 12 is a flowchart showing a flow of main processing in the imaging apparatus. The process shown in FIG. 12 is performed based on the control of the microcomputer 41.

  When the power button of the operation unit 38 is turned on to turn on the image pickup apparatus, this main process is started, and the image pickup apparatus is first initialized (step S11). In this initialization, a recording flag indicating whether or not a moving image is being recorded is also reset to off.

  Next, the microcomputer 41 determines whether or not the playback button of the operation unit 38 has been operated (step S12).

  If the playback button is operated here, playback / editing processing is performed (step S13). In this reproduction / editing process, a list of files recorded on the recording medium 36 is displayed, a selection operation from the user is waited for, the selected file is reproduced, or the selected image is edited. It is.

  If the playback button has not been operated in step S12, or if the process in step S13 has been performed, it is determined whether the menu button of the operation unit 38 has been operated and camera settings relating to the imaging device have been selected. (Step S14).

When the camera setting is selected here, a menu for changing the camera setting is displayed on the LCD 34, and a user operation for changing the camera setting is awaited from the operation unit 38. Here are some examples of camera settings:
Finish: Natural, Vivid, Flat, Monotone Effect: Sunburn effect Still image recording mode: JPEG recording, TIFF recording,
JPEG + RAW recording, RAW recording Movie recording mode: Motion JPEG, H.264 H.264
Image quality: Fine, normal, etc., but not limited to these.

  When a user operation is performed, camera settings are performed according to the operation content (step S15). In this camera setting, when the tanning effect, which is a color changing process that reproduces the aging of the printed photograph and fading due to sunlight as described above, is turned on, which shade of tanning effect is further obtained. The user may be able to select and set as desired.

  If no camera setting is selected in step S14 or if the process of step S15 is performed, it is determined whether or not the moving image button of the operation unit 38 has been operated (step S16).

  Here, when the moving image button is operated, first, the recording flag is reversed (that is, turned off if turned on, turned on if turned off) (step S17), and the recording flag is turned on. It is determined whether or not the moving image is being recorded based on whether it is off or not (step S18).

  If the moving image is being recorded, the recording is started, so that a moving image file is generated and prepared so that image data can be recorded (step S19).

  If the movie is not being recorded, the recording ends, so the number of frames is recorded in the header of the moving image file so that it can be played back as a moving image file, and the file writing is completed. The moving image file is closed (step S20).

  If the moving image button is not operated in step S16, or if the processing in step S19 or step S20 is performed, it is determined whether or not moving image recording is performed based on the recording flag as in step S18 ( Step S21).

  If the moving image is not being recorded, it is determined whether or not the release button has transitioned from the OFF state to the 1st release on state, which is the first-stage pressing state (so-called half-pressed state) (step S22).

  When transitioning to the state where the first release is on, automatic exposure (AE) control for capturing a still image is performed by the AE processing unit 30 and automatic focus control (AF) is performed at AF at the transition timing. This is performed by the processing unit 31 (step S23). Thus, after the 1st release button is pressed, so-called AE lock and AF lock are performed.

  On the other hand, if the transition to the first release on state has not been made in step S22, it is determined whether or not the release button is in the second release on state (the so-called full press state), which is the second-stage press state. (Step S24).

  Here, when the 2nd release is in the on state, a series of photographing processing is performed in which the diaphragm 12 is driven, the shutter 21 is opened, and exposure is performed by the image sensor 23 (step S25).

  Subsequently, image processing as will be described later with reference to FIG. 13 is performed on the image obtained by photographing (step S26).

  Thereafter, the still image that has been subjected to image processing is displayed on the LCD 34 as, for example, a rec view, and is recorded on the recording medium 36 as a still image file in a camera-set format (step S27). As a specific example, when JPEG recording is set as the still image recording mode, JPEG compression is performed in the YC422 format to create and record a JPEG file. If TIFF recording is set as the still image recording mode, it is converted into RGB data to create a TIFF file in the RGB format and perform recording. Further, if it is set to perform RAW recording, the RAW image obtained by shooting in step S25 is recorded as a RAW file.

  If the moving image is being recorded in step S21, or if the 2nd release is not turned on in step S24, the shutter 21 is opened and automatic exposure (AE) control for moving image shooting or live view is performed. Is performed by the AE processing unit 30 (step S28), and one frame (or one field, etc.) image is captured by the electronic shutter (step S29).

  Image processing as described later with reference to FIG. 13 is performed on the image thus captured (step S30), and live view display is performed on the LCD 34 (step S31).

  Next, it is determined whether or not the moving image is being recorded (step S32). If the moving image is being recorded, the frame image obtained by shooting is compressed by the image compression / decompression unit 32 to the recording medium 36. Recording (addition) is performed (step S33).

  Whether the process of step S23, step S27, or step S33 is performed, or if the moving image is not being recorded in step S32, it is determined whether or not the power button is turned off (step S34).

  If the power button has not been turned off, the process returns to step S12 to repeat the above-described processing. If the power button has been turned off, the main process is terminated.

  Next, FIG. 13 is a flowchart illustrating a flow of image processing performed by the image processing unit 27 of the imaging apparatus.

  Upon entering this process, an OB subtraction process is performed to subtract the optical black value from the image data (step S41).

  Subsequently, for example, R gain and B gain corresponding to the white balance mode set in advance by the user are read from the flash memory 39 of the camera body with respect to image data having a Bayer array, and the read R gain is set to R pixel. White balance (WB) correction is performed by multiplying the value by the B gain and the B pixel value, respectively (step S42). When auto white balance is selected as the white balance mode, R gain and B gain are calculated by analyzing captured image data, and white balance correction is performed using the calculated R gain and B gain. Do.

  Then, from the image data of the Bayer array in which only one color component of the RGB components per pixel exists, the color components that do not exist in the pixel of interest are interpolated and obtained from the surrounding pixels, so that all the pixels are RGB three color components. Is synchronized (step S43).

  Further, the color matrix coefficient corresponding to the set white balance mode is read from the flash memory 39 of the main body and multiplied by the image data, and the color is corrected so as to achieve an appropriate color reproduction according to the hue and saturation. Thus, color reproduction processing is performed (step S44).

  Next, after each RGB data is gamma-converted and then color-converted to YCbCr data, the luminance data Y is further gamma-converted to perform luminance change processing (step S45).

  Subsequently, an edge component is extracted from the image data by a band pass filter, and an edge enhancement process is performed by multiplying the extracted edge component by a coefficient corresponding to the degree of edge enhancement and adding it to the image data (step S46). ).

  In addition, the image data is subjected to frequency decomposition, and a noise reduction process for reducing noise is performed by performing a coring process or the like according to the frequency (step S47).

  The processing in steps S41 to S47 described above is basic image processing performed by the basic image processing unit 28.

  Subsequently, it is determined whether or not the sunburn effect is set to ON by input setting from the operation unit 38 (step S48).

  Here, when the sunburn effect is set to ON, the special image processing unit 29 performs the processes of steps S49 to S51 described below.

  That is, the analysis unit 45 extracts a luminance signal (or luminance equivalent signal) from the image P as shown in FIG. 3, sets a partial region as shown in FIG. 4 (or FIG. 8), and FIG. Alternatively, the arrangement of representative values as shown in FIG. 9) is calculated (step S49).

  Next, the gain map generator 46 changes the arrangement as shown in FIG. 6 (or FIG. 10) based on the arrangement of the representative values as shown in FIG. 5 (or FIG. 9). A gain map GM as shown in FIG. 7 (or FIG. 11) is created based on the arrangement of the representative values (step S50).

  Thereafter, the color change processing unit 47 performs color change processing as will be described later with reference to FIG. 14 (step S51).

  When the process of step S51 is performed or when the effect of the sunburn effect is not on in step S48, the process returns from this process.

  Next, FIG. 14 is a flowchart illustrating color change processing performed by the color change processing unit 47 of the imaging apparatus.

  Upon entering this process, the color change processing unit 47 performs pixel image processing on the pixel position (x, y) according to the pixel arrangement order, for example, from the image data that has been subjected to basic image processing and stored in the SDRAM 37. A value is acquired (step S61).

  Next, when the acquired pixel is not expressed in the RGB color space, it is converted into a pixel value (R_in (x, y), G_in (x, y), B_in (x, y)) in the RGB color space. (Step S62).

  Then, the gain value G (x, y) at the same pixel position (x, y) as the pixel from which the pixel value has been acquired in step S61 is acquired from the gain map GM stored in the SDRAM 37 (step S63).

  Subsequently, by multiplying predetermined constants R_cnt, G_cnt, and B_cnt by a gain value G (x, y), offsets (R_cnt × G (x, y), G_cnt × G (x , Y), B_cnt × G (x, y)) is calculated (step S64).

  Further, offsets (R_cnt × G (x, y), G_cnt × G (x, y), B_cnt × G) are added to the pixel values (R_in (x, y), G_in (x, y), B_in (x, y)). (X, y)) is added to perform the color change process (step S65).

  In addition, color space conversion opposite to that in step S62 is performed as necessary to return to the original color space representation (step S66).

  Thereafter, it is determined whether or not the processing for all the pixels has been completed (step S67). If the processing for all the pixels has not been completed, the process returns to step S61 to perform the same processing for the next pixel position. When processing for all pixels is completed, the process returns from this processing.

  According to the first embodiment, since the gain map is created based on the pixel values of the pixels constituting the partial region of the image and at least one of the hue and saturation of the image is changed, it is not random. However, different color changes can be performed for each image, and a color-changed image that cannot be predicted easily and has room for unexpectedness can be obtained. And since the pixel value of the image is used and it is not a random process, the correlation between frames in continuous shooting or moving images can be maintained to some extent, and it is possible to avoid an unnatural image group. it can. Therefore, it is possible to change the color of an image with high convenience.

  In addition, since the gain map is created so that the area where the gain value is the maximum corresponds to the partial area other than the partial area where the representative value is the maximum, the area where the large color change is performed has a color saturation luminance. It is possible to avoid a high area, and the color change is more effective.

  Furthermore, since a gain map is obtained in which the spatial distribution of gain values changes smoothly, the color does not change abruptly between adjacent pixels. Also, since the gain map is generated so that the histogram of gain values has a flattened distribution, the color of the entire image is strongly changed almost uniformly, or there is almost no color change of the entire image. Can be avoided. In this way, it is possible to obtain a color-change image having a natural texture in which color change occurs smoothly and a portion where the color change is relatively large and a portion where the color change is relatively well distributed.

  A plurality of representative values having different arrangements are arranged at a plurality of predetermined pixel positions on the gain map, and a gain value at an arbitrary pixel position is interpolated based on a distance from the predetermined pixel position to the arbitrary pixel position. As a result, it is possible to obtain a color-changed image in which the hue is changed with a natural gradation without causing an unnatural step or the like in the hue change.

In addition, since the color-changed image is calculated by adding an offset obtained by multiplying the gain value by a predetermined constant to the image, it is suitable for calculation processing by a computer, and high-speed processing is possible. At this time, by selecting a constant to multiply the gain value, it is possible to change the color of the desired hue, and not only for obtaining an image that looks sunburned, but also for various other uses. It becomes possible.
[Embodiment 2]

  FIGS. 15 to 18 show the second embodiment of the present invention, and FIG. 15 is a chart showing an example of a basic gain map stored in advance in the imaging apparatus.

  In the second embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals and the description thereof is omitted as appropriate, and only different points will be mainly described.

  In the present embodiment, a basic gain map indicating a basic gain value distribution is stored in advance in, for example, the flash memory 39 (for example, in the manufacturing process of the imaging device).

  As described above, the number of pixels (pixel configuration) of an image to be recorded on the recording medium 36 can be selected and set from a plurality of options. However, it is not realistic to prepare a large number of basic gain maps in advance for all recordable pixel configurations because the flash memory 39 needs a large capacity. Therefore, it is assumed that the basic gain map is configured as data having a pixel configuration equal to or less than the maximum number of pixels of the recorded image, for example, a pixel configuration that is equal to or smaller than the minimum number of pixels of the recorded image. With such a configuration, the storage capacity of the flash memory 39 can be reduced.

  In the example shown in FIG. 15, a basic gain map BGM1 that forms a smooth gradation from a high gain value (for example, gain value 1) in the upper right corner toward a lower gain value (for example, gain value 0) in the lower left corner, A basic gain map BGM2 that forms a smooth gradation from a high gain value (example: 1) to a low gain value (example: 0) in the upper left corner, and a lower upper right corner from a gain value that is high in the lower left corner (example: 1) A basic gain map BGM3 that forms a smooth gradation toward a gain value (example: 0) and a smooth gradation from a high gain value (example: 1) in the upper left corner to a lower gain value (example: 0) in the lower right corner The basic gain map BGM4 forming the following is stored in the flash memory 39.

  In these basic gain maps BGM1 to BGM4, the spatial distribution of gain values changes smoothly, and the gain value histogram becomes a flattened distribution (that is, identification of high gain values, low gain values, etc.). The gain map does not have a large or small number of pixels of the gain value.

  In such a configuration, the analysis unit 45 analyzes the image data to obtain, for example, the arrangement of representative values as shown in FIG. 5, as in the first embodiment described above.

  The gain map generator 46 selects a basic gain map to be used from the basic gain maps BGM1 to BGM4 stored in the flash memory 39 based on the arrangement of the representative values obtained by the analyzer 45. The selection at this time is performed as follows, for example.

  As described above, the high luminance region is close to white (color saturation state) and the color expression range is narrow. On the other hand, since the low luminance region is close to black, the color expression range is similarly narrow. In general, the color expression range is wide in the region where the luminance is moderate.

  Therefore, the gain map generation unit 46, for example, from the representative values V1 to V4 obtained as shown in FIG. 5, for example, the dynamic range of the representative value (that is, the dynamic range of the pixel value, for example, 10 bits) The representative value closest to the median value (for example, 512) of 0 to 1023) is selected.

  In the example shown in FIG. 5, it is assumed that the representative value closest to the median value of the dynamic range is V2.

  Then, the gain map generation unit 46 selects a basic gain map BGM1 as shown in FIG. 16 so that a high gain value is distributed in the upper right which is the arrangement of the representative value V2. FIG. 16 is a diagram showing the basic gain map BGM1 selected by the gain map generator 46. As shown in FIG.

  As described above, the selected basic gain map BGM1 often has a pixel configuration smaller than that of the image data to be subjected to color change processing. Therefore, the gain map generation unit 46 generates a gain map GM having the same pixel configuration as that of the image data to be subjected to the color change process by interpolating (for example, linear interpolation) the basic gain map BGM1. FIG. 17 is a diagram showing the gain map GM created by interpolating the basic gain map BGM1 shown in FIG.

  Since the basic gain map BGM1 of the interpolation source is a distribution in which the spatial distribution of gain values changes smoothly and the gain value histogram is flattened as described above, the gain map GM created by linear interpolation or the like is also used. The spatial distribution of gain values changes smoothly, and the gain value histogram is flattened.

  FIG. 18 is a diagram illustrating a modification of the basic gain map stored in advance in the imaging apparatus.

  In the example shown in FIG. 15, the basic gain maps BGM1 to BGM4 are stored in the flash memory 39, but these have a rotational symmetry of 90 degrees. Therefore, in the example shown in FIG. 18, only one basic gain map out of the basic gain maps BGM1 to BGM4, for example, the basic gain map BGM corresponding to the basic gain map BGM4 is stored in the flash memory 39, and the others. When the basic gain maps BGM1 to BGM3 are required, the basic gain map BGM is obtained by rotating.

  By adopting such a configuration, the storage capacity of the basic gain map in the flash memory 39 can be reduced to, for example, 1/4 compared with the example shown in FIG. Further, rotating the basic gain map BGM in which gain values are arranged in the row direction and the column direction in units of 90 degrees can be easily executed by changing the reading start position and the reading order of the basic gain map BGM. There is also an advantage that the processing load hardly increases.

  In the above description, the basic gain map indicating the basic gain value distribution is stored in the flash memory 39 in advance, but instead of this, mathematical formulas and algorithms that can create a gain map are stored in the flash memory 39 in advance. The gain map may be created by selecting a mathematical formula or algorithm to be used based on the representative value arrangement. Since the gain map can often be expressed by a relatively simple mathematical expression or the like, the storage capacity of the flash memory 39 can be further reduced by adopting such a configuration.

According to the second embodiment, the same effects as those of the first embodiment described above are obtained, the spatial distribution of gain values changes smoothly, and the gain value histogram is a flattened distribution. Since the gain map GM is generated based on the gain map, it is possible to avoid a process in which the entire image is strongly and uniformly color-changed or there is almost no color change of the entire image. In this way, it is possible to obtain a color-change image having a natural texture in which color change occurs smoothly and a portion where the color change is relatively large and a portion where the color change is relatively well distributed.
[Embodiment 3]

  FIGS. 19 and 20 show Embodiment 3 of the present invention, and FIG. 19 is a diagram showing an example of a correction coefficient map stored in advance by the imaging apparatus.

  In the third embodiment, the same parts as those in the first and second embodiments are denoted by the same reference numerals and the description thereof is omitted as appropriate, and only different points will be mainly described.

  In the present embodiment, a correction coefficient map CCM indicating the distribution of correction coefficients for correcting the gain map GM created as described in the first embodiment is stored in the flash memory 39 in advance (for example, an imaging device). In the manufacturing process). As shown in FIG. 19, the correction coefficient map CCM has the smallest value at a specific pixel position (for example, the central portion of the image), and has a larger value as the distance from the specific pixel position (for example, the closer to the periphery of the image). The distribution of correction coefficients. Further, it is assumed that the correction coefficient map CCM is normalized so that the range that the correction coefficient can take is 0 or more and 1 or less in order to simplify the calculation.

  Note that a part of the correction coefficient map CCM may be cut out or the value may be changed according to the subject. For example, the main subject may be identified, and a correction coefficient map CCM in which the correction coefficient of the main subject portion becomes smaller may be generated by partially cutting out or changing the value.

  Similar to the basic gain map of the second embodiment described above, the correction coefficient map CCM has a pixel configuration equal to or smaller than the minimum pixel number of the recorded image, for example, a pixel configuration equal to or smaller than the maximum pixel number of the recorded image. It is preferable to configure as data and perform interpolation or the like according to the pixel configuration of the image data to be subjected to color change processing. With this configuration, the storage capacity of the flash memory 39 can be reduced as described above.

  In such a configuration, the analysis unit 45 analyzes the image data to obtain, for example, the arrangement of representative values as shown in FIG. 5, as in the first embodiment described above.

  Further, the gain map generation unit 46 performs the interpolation process after changing the arrangement of the representative values obtained by the analysis unit 45 as shown in FIG. 6, for example, as shown in FIG. 7. The point of calculating GM is also the same as in the first embodiment.

  In the present embodiment, the gain map generator 46 further corrects the gain map GM created as shown in FIG. 7, for example.

  That is, the gain map generation unit 46 reads out the correction coefficient map CCM as shown in FIG. 19 from the flash memory 39, and the pixel configuration of the image data for which the pixel configuration of the read correction coefficient map CCM is the target of color change processing ( If it is different from the pixel configuration of the gain map GM), the correction coefficient map CCM having the same pixel configuration as that of the gain map GM is obtained by interpolating the correction coefficient map CCM (for example, linear interpolation).

  Next, the gain map generation unit 46 multiplies each gain value on the gain map GM by a correction coefficient at the same pixel position on the correction coefficient map CCM, thereby correcting a correction gain map CGM as shown in FIG. Is generated. FIG. 20 shows an example of the correction gain map CGM obtained by correcting the gain map GM based on the correction coefficient map CCM.

  The color change processing unit 47 then changes at least one of the hue and saturation of the image as described above based on the correction gain map CGM.

  In the case of the configuration of the second embodiment described above, the basic gain map corrected by the correction coefficient map CCM shown in FIG. 19 may be stored in the flash memory 39 in advance. It is not necessary to store the correction coefficient map CCM separately.

  According to the third embodiment, the same effects as those of the first and second embodiments described above can be obtained, and the color change processing is performed so that the specific pixel position has the smallest value and the distance from the specific pixel position increases. Since the correction is performed based on the correction gain map CGM corrected by the correction coefficient to be a value, for example, the color change of the main subject OBJ or the like can be suppressed, and an image having an impression that the entire image is covered with color Therefore, a preferable color-change image can be obtained.

  Although the image processing apparatus has been mainly described above, an image processing method that performs the same processing as the image processing apparatus may be used, or a program for controlling the computer to perform the same processing as the image processing apparatus, It may be a non-temporary recording medium readable by a computer for recording the program.

  Further, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope in the implementation stage. In addition, various aspects of the invention can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components from all the components shown by embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined. Thus, it goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

DESCRIPTION OF SYMBOLS 1 ... Interchangeable lens 2 ... Camera body 3 ... Interface (I / F)
DESCRIPTION OF SYMBOLS 11 ... Lens 12 ... Aperture 13 ... Driver 14 ... Microcomputer 15 ... Flash memory 21 ... Shutter 22 ... Imaging part 23 ... Imaging element 24 ... Analog processing part 25 ... A / D conversion part 26 ... Raw reduction processing part 27 ... Image processing Unit 28 ... Basic image processing unit 29 ... Special image processing unit 30 ... AE processing unit 31 ... AF processing unit 32 ... Image compression / decompression unit 33 ... LCD driver 34 ... LCD
35 ... Memory interface (memory I / F)
36 ... Recording medium 37 ... SDRAM
38 ... Operation part 39 ... Flash memory 40 ... Bus 41 ... Microcomputer (control part)
45 ... Analysis unit 46 ... Gain map generation unit 47 ... Color change processing unit

Claims (7)

The representative value corresponding to the partial area is calculated based on the pixel value extracted from the luminance signal of the plurality of pixels constituting the partial area of the image, and the calculated representative value and the plurality of representative values are calculated. An analysis unit for obtaining an arrangement of representative values based on the arrangement of the partial areas of
A gain map that gives a gain value corresponding to each pixel constituting the image based on the arrangement of the representative value, wherein a region where the gain value is maximum is a portion other than the partial region where the representative value is maximum A gain map generator for obtaining a gain map corresponding to the region;
A color change processing unit that changes at least one of hue and saturation of the image based on the gain map;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the gain map generation unit obtains the gain map such that a spatial distribution of gain values on the gain map changes smoothly.   The gain map generation unit arranges a plurality of representative values at a plurality of predetermined pixel positions on the gain map so as to vary the arrangement of the representative values, and from the predetermined pixel position to an arbitrary pixel position. The image processing apparatus according to claim 2, wherein the gain map is calculated by interpolating a gain value at the arbitrary pixel position based on the distance of the image. The gain map generation unit multiplies each gain value on the gain map by a correction coefficient that has a smallest specific pixel position and a larger value as the distance from the specific pixel position increases. Generate a corrected gain map,
The image processing apparatus according to claim 2, wherein the color change processing unit changes at least one of hue and saturation of the image based on the corrected gain map. If the pixel position in the image is (x, y) and at least one of the components of the color space expressing the image is represented by C, C is a predetermined value that does not depend on the pixel position. , C_cnt, the component before color change by the color change processing unit at the pixel position (x, y) is C_in (x, y), the component after color change is C_out (x, y), and the gain map If the gain value is G (x, y), the color change processing unit
C_out (x, y) = C_in (x, y) + C_cnt × G (x, y)
The image processing apparatus according to claim 1, wherein the color of the image is changed by performing the above calculation on all pixel positions (x, y). The representative value corresponding to the partial area is calculated based on the pixel value extracted from the luminance signal of the plurality of pixels constituting the partial area of the image, and the calculated representative value and the plurality of representative values are calculated. An analysis step for obtaining an arrangement of representative values based on the arrangement of the partial areas of
A gain map that gives a gain value corresponding to each pixel constituting the image based on the arrangement of the representative value, wherein a region where the gain value is maximum is a portion other than the partial region where the representative value is maximum A gain map generation step for obtaining a gain map corresponding to the region;
A color change processing step for changing at least one of hue and saturation of the image based on the gain map;
An image processing method comprising: A program for causing a computer to perform image processing,
On the computer,
The representative value corresponding to the partial area is calculated based on the pixel value extracted from the luminance signal of the plurality of pixels constituting the partial area of the image, and the calculated representative value and the plurality of representative values are calculated. An analysis step for obtaining an arrangement of representative values based on the arrangement of the partial areas of
A gain map that gives a gain value corresponding to each pixel constituting the image based on the arrangement of the representative value, wherein a region where the gain value is maximum is a portion other than the partial region where the representative value is maximum A gain map generation step for obtaining a gain map corresponding to the region;
A color change processing step for changing at least one of hue and saturation of the image based on the gain map;
A program for running