JP4998009B2 - Image processing apparatus and imaging apparatus - Google Patents

Description

The present invention relates to images processing instrumentation 置及 beauty imaging device.

  Some digital cameras have a noise removal function (see Patent Document 1). In recent years, noise removal processing uses a spatial digital filter to remove noise while comparing individual pixels with surrounding pixels, so there is little possibility of erroneously removing portions that are not noise on a captured image. Further, if the diameter of the spatial digital filter is set to be somewhat large, a higher noise removal effect can be obtained.

However, since such noise removal requires a large amount of computation, it is necessary to store the image data of the photographed image in the frame memory when performing noise removal processing on the photographed image. For this reason, noise removal processing is generally executed after other image processing that is pipeline processing.
Japanese Patent Laid-Open No. 4-119770

However, in the contour enhancement process, which is one step of the pipeline process, the impression given to the captured image is opposite to the noise removal process. Therefore, the effect of the noise removal process is hindered due to the contour enhancement process, or the noise removal process is the cause. There is a possibility that the effect of the edge enhancement process may be hindered.
For this reason, the signal processing apparatus described in Patent Literature 1 limits the process performed on the captured image to only one of the contour enhancement process and the noise removal process.

In recent digital cameras and the like, the intensity of the noise removal process is set to be high in advance on the assumption that the effect of the noise removal process is hindered .

The image processing apparatus according to claim 1, an input unit for inputting image data, a memory for storing the image data input by the input unit, a bus connected to the memory, and a bus connected to the bus. In addition, an interpolation color processing unit that performs pixel interpolation processing and color conversion processing on image data, and a pipeline connection between the interpolation color processing unit and the interpolation color processing unit that do not pass through the bus are possible. An edge enhancement processing unit that applies enhancement processing to image data, a noise removal processing unit that is connected to the bus and applies noise removal processing to image data, and a compression processing unit that is connected to the bus and applies compression processing to image data When the imaging sensitivity of the image data input by the input unit is the first imaging sensitivity, the image data stored in the memory is converted to the interpolated color process via the bus. The image data is subjected to the pixel interpolation process and the color conversion process performed by the interpolation color processing unit to generate image data that has undergone interpolation color processing, and the image data that has undergone interpolation color processing is subjected to the interpolation color processing. To the memory via the bus, the interpolated color processed image data stored in the memory is transmitted to the noise removal processing unit via the bus, and the noise removal processing unit Noise removal processing is performed on the interpolated color processed image data to generate noise removal processed image data, the noise removal processed image data is transmitted from the noise removal processing unit to the contour enhancement processing unit, and the contour The contour enhancement processing by the enhancement processing unit is performed on the noise-removed image data to generate the contour enhancement processed image data, and the contour Tone-processed image data is transmitted from the contour enhancement processing unit to the memory via the bus, and the contour-enhanced image data stored in the memory is transmitted to the compression processing unit via the bus. And when the imaging sensitivity of the image data input by the input unit is a second imaging sensitivity lower than the first imaging sensitivity, the image data stored in the memory is transferred to the bus. The image data is transmitted to the interpolated color processing unit, and the interpolated color processing unit performs the pixel interpolation processing and the color conversion processing on the image data to generate interpolated color processed image data and connect the pipeline. Thus, the interpolated color processed image data is transmitted from the interpolated color processing unit to the contour emphasizing processing unit without going through the bus, and the contour by the contour emphasizing processing unit Applying enhancement processing to the interpolated color processed image data to generate contour enhancement processed image data, transmitting the contour enhancement processed image data from the contour enhancement processing unit to the memory via the bus, By transmitting the edge enhanced image data stored in the memory to the compression processing unit via the bus, the noise by the noise removal processing unit is added to the image data input by the input unit. And a control unit that controls the removal processing not to be performed.

The imaging device according to claim 7 includes an imaging unit that captures an image of a subject and generates image data, and the image processing device according to any one of claims 1 to 6. .

According to the present invention, and an object thereof is to provide an image processing apparatus and an image pickup apparatus capable of performing noise removal processing and the edge enhancement processing effectively to the image.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described. This embodiment is an embodiment of a digital camera according to the present invention.
FIG. 1 is a block diagram of a digital camera. As shown in FIG. 1, the digital camera 10 includes a photographing lens 11, an image sensor 12, an amplifier 13A, an A / D converter (A / D) 13B, a signal processing circuit 14, a timing generator (TG) 15, and a frame memory 16. , CPU 17, interpolation / color processing circuit 18, contour enhancement processing circuit 19, noise removal processing circuit 20, compression processing circuit 21, display circuit 22, rear monitor 23, card interface (card I / F) 24, operation buttons 25, etc. The card interface 22 is provided with a card memory 24A.

  Among these, the frame memory 16, the interpolation / color processing circuit 18, the contour enhancement process 19, the noise removal processing circuit 20, the compression processing circuit 21, the display circuit 22, and the card interface 24 are connected to the CPU 17 via a bus. The interpolation / color processing circuit 18, the edge enhancement processing circuit 19, and the noise removal processing circuit 20 that are image processing circuits are composed of different modules. However, the interpolation / color processing circuit 18 and the contour emphasis processing 19 can be pipeline-connected as necessary. For this reason, a buffer memory (not shown) is provided between the interpolation / color processing circuit 18 and the contour enhancement processing circuit 19 to compensate for the difference in processing speed between the two.

  The user of the above digital camera 10 can input various instructions to the CPU 17 via the operation buttons 25. The instructions include instructions for switching the photographing sensitivity of the digital camera 10 and photographing instructions. When a shooting sensitivity switching instruction is input, the CPU 17 adjusts the gain of the amplifier 13A. Here, for the sake of simplicity, only two types of imaging sensitivity that can be designated by the user are “high” and “low”. When the shooting sensitivity specified by the user is “high”, the gain of the amplifier 13A is set to a large value, and when the shooting sensitivity specified by the user is “low”, the gain of the amplifier 13A is set to a small value. Is done. The contents of this setting (gain value) are saved until the user next switches the shooting sensitivity.

  FIG. 2 is an operation flowchart of the CPU 17 during photographing. As shown in FIG. 2, when a shooting instruction is input from the user (step S10 YES), the CPU 17 drives the timing generator 15 to start shooting (step S11). At this time, the timing generator 15 synchronously drives the image sensor 12, the amplifier 13A, the A / D converter 13B, and the signal processing circuit 14 to acquire image data of the captured image. The image data is stored in the frame memory 16.

Subsequently, the CPU 17 sends the image data in the frame memory 16 to the interpolation / color processing circuit 18 and gives an instruction to start the processing to the interpolation / color processing circuit 18 (step S12). It is determined whether or not it is “high” (step S13).
When the photographing sensitivity being set is “low” (NO in step S13), the CPU 17 pipeline-connects the interpolation / color processing circuit 18 and the contour enhancement processing circuit 19, and not only the interpolation / color processing circuit 18. A process start instruction is also given to the contour emphasis processing circuit 19 (step S14).

  At this time, the image data is subjected to pixel interpolation processing and color conversion processing in this order in the interpolation / color processing circuit 18, and the processed image data (YUV format image data) is subjected to the contour enhancement processing circuit 19. In FIG. Therefore, the pixel interpolation processing, color conversion processing, and contour enhancement processing at this time are pipeline processing. The CPU 17 stores the processed image data in the frame memory 16 (step S15).

Subsequently, the CPU 17 determines whether or not the contour enhancement processing has been executed (step S16). In this case, it is determined that the processing has been executed (YES in step S16), and the CPU 17 compresses the image data in the frame memory 16 into a compression processing circuit. 21 and gives an instruction to start processing to the compression processing circuit 21 (step S18).
At this time, the compression processing circuit 21 compresses the image data, and the processed image data (compressed code data such as JPEG format) is written to the card memory 24A via the card interface 24 (step S19). ).

  On the other hand, when the photographing sensitivity being set is “high” (YES in step S13), the CPU 17 cancels the connection between the interpolation / color processing circuit 18 and the contour enhancement process 19 in the interpolation / color processing circuit 18. Processing is performed on the image data, and the processed image data (YUV format image data) is stored in the frame memory 16 without being sent to the contour enhancement processing circuit 19 (step S15). That is, at this time, the outline emphasis process is bypassed.

Subsequently, the CPU 17 determines whether or not the contour emphasis processing has been executed (step S16). In this case, it is determined that the contour enhancement processing has not been executed (step S16 NO), and the CPU 17 converts the image data in the frame memory 16 into noise. The signal is sent to the removal processing circuit 20, and a start instruction is given to the noise removal processing circuit 20 (step S17).
At this time, the noise removal processing circuit 20 performs noise removal processing on the image data. This noise removal process is a spatial digital filter process using a spatial digital filter (such as an epsilon filter) having a diameter larger than a certain diameter. Unlike the low-pass filter process, such a noise removal process can remove noise while distinguishing it from a picture.

Further, the CPU 17 sends the image data after the noise removal processing to the contour enhancement processing circuit 19 and gives an instruction to start the processing to the contour enhancement processing circuit 19 (step S14). The CPU 17 stores the image data after the contour enhancement process in the frame memory 16 (step S15).
Subsequently, the CPU 17 determines whether or not the contour enhancement processing has been executed (step S16). In this case, it is determined that the processing has been executed (YES in step S16), and the CPU 17 compresses the image data in the frame memory 16 into a compression processing circuit. 21 and gives an instruction to start processing to the compression processing circuit 21 (step S18). The compressed image data (compressed code data such as JPEG format) is written into the card memory 24A via the card interface 24 (step S19).

FIG. 3 compares the path of image data at the time of the above shooting between the case where the shooting sensitivity being set is “low” (NO in step S13) and the case where it is “high” (YES in step S13). It is a figure to do.
When the photographing sensitivity is “low”, the image data is stored in the frame memory 16, the interpolation / color processing circuit 18, the contour enhancement processing circuit 19, the frame memory 16, and the compression processing circuit 21 as shown in FIG. Go through in order. At this time, the processing by the interpolation / color processing circuit 18 (interpolation / color processing) and the contour enhancement processing by the contour enhancement processing circuit 19 are bypass processing.

  When the photographing sensitivity is “low”, it can be considered that noise is not superimposed on the image data (see symbol “a” in FIG. 3), and thus sufficiently good image data can be obtained only by performing the edge enhancement process (see FIG. 3). (See symbol b in FIG. 3). Good image data is image data in which noise is suppressed and the outline of a pattern is enhanced. Note that reference symbols a, b, c, and d in FIG. 3 indicate the state of the image data in the process (only a certain line on a certain captured image), the horizontal direction is the spatial direction, and the vertical direction is the luminance. Direction.

  On the other hand, when the photographing sensitivity is “high”, as shown in FIG. 3B, the image data is stored in the frame memory 16, the interpolation / color processing circuit 18, the frame memory 16, the noise removal processing circuit 20, and the contour enhancement. It goes through the processing circuit 19, the frame memory 16, and the compression processing circuit 21 in order. At this time, the connection between the interpolation / color processing circuit 18 and the contour enhancement processing circuit 19 is released, and the frame memory 16 and the noise removal processing circuit 20 are sequentially inserted into the path between them.

When the photographing sensitivity is “high”, noise is superimposed on the image data (see FIG. 3, symbol c), and thus noise can be suppressed through the noise removal process (see FIG. 3, symbol d). Further, the outline of the pattern is emphasized through the outline enhancement process (see e in FIG. 3). That is, both the effect of the noise removal process and the effect of the edge enhancement process are obtained.
As described above, the reason why both the effect of the noise removal process and the effect of the edge enhancement process are obtained is the execution order of the noise removal process and the edge enhancement process. For comparison, an example in which the execution order is opposite to that in the digital camera 10 is shown in FIG.

  In the example shown in FIG. 4, the object of the contour emphasis process is image data in a state where noise is superimposed (see symbol a in FIG. 4), so that not only the contour of the pattern but also the contour of the noise is emphasized (see FIG. 4 reference b). Further, when the noise removal process is performed, not only the noise of the image data but also the contour of the pattern is suppressed (see reference numeral c in FIG. 4), so that the final photographed image becomes a so-called “smooth image”.

As described above, according to the digital camera 10, only the former of the edge enhancement processing and noise removal processing is applied to the image data when the imaging sensitivity is “low”, and both are applied to the image data when the imaging sensitivity is “high”. Is done. The order of processing when both are performed is the order of noise removal processing and contour enhancement processing (see FIG. 3B).
According to this order, the noise removal process does not hinder the effect of the edge enhancement process, and the edge enhancement process does not hinder the effect of the noise removal process. Therefore, both noise removal processing and contour enhancement processing are effectively performed on the image data.

  Even if the strength of the noise enhancement processing is increased, the effect of the contour enhancement processing is not diminished. Even if the strength of the contour enhancement processing is increased, the effect of the noise removal processing is not diminished. Therefore, in the present digital camera 10, the intensity of the noise removal process and the intensity of the edge enhancement process may be set to the necessary minimum heights. Therefore, noise removal processing and contour enhancement processing are each efficiently performed on the image data.

Note that the CPU 17 of the present embodiment performs on / off of the noise removal function of the digital camera 10 according to the shooting sensitivity being set, but may be performed according to a user's instruction to turn on / off the noise removal function. .
In addition, the CPU 17 of the present embodiment sets the noise removal processing strength constant, but the noise removal processing strength may be switched according to the imaging sensitivity being set. In that case, the higher the shooting sensitivity during setting, the higher the intensity may be set. Further, the CPU 17 may switch the intensity of the noise removal process according to an intensity switching instruction from the user.

Further, the CPU 17 of the present embodiment sets the strength of the contour enhancement processing to be constant, but the strength of the contour enhancement processing may be switched according to an instruction from the user. Even in such a case, the CPU 17 does not need to link the strength of the noise enhancement processing with the strength of the contour enhancement processing, and may set both of them independently.
Further, in the present embodiment, the digital camera to which the present invention is applied has been described. However, the present invention is not limited to a printer having an image processing function or an image storage function having an image processing function. It is also possible to apply to other devices.

  Moreover, you may make a computer perform a part or all of the image processing (especially FIG. 2 step S12-S17) of this embodiment. In this case, an image processing program in which the image processing procedure is written is installed in the computer. The image processing program is installed in the computer via a recording medium such as a CD-ROM or the Internet.

1 is a configuration diagram of a digital camera 10. FIG. 6 is an operation flowchart of the CPU 17 at the time of shooting. It is a figure which compares the path | route of the image data at the time of imaging | photography in the case where the imaging sensitivity in setting is "low", and the case where it is "high". It is a figure which shows the example when the execution order of a noise removal process and an outline emphasis process is opposite to that in this digital camera 10.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Digital camera 10, 11 ... Shooting lens, 12 ... Image sensor, 13A ... Amplifier, 13B ... A / D converter (A / D), 14 ... Signal processing circuit, 15 ... Timing generator (TG), 16 ... Frame Memory, 17 ... CPU, 18 ... Interpolation / color processing circuit, 19 ... Outline emphasis processing circuit, 20 ... Noise removal processing circuit, 21 ... Compression processing circuit, 22 ... Display circuit, 23 ... Back monitor, 24 ... Card interface (card) I / F) 24, 25 ... operation buttons, 24A ... card memory

Claims (7)

An input unit for inputting image data;
A memory for storing the image data input by the input unit;
A bus connected to the memory;
An interpolation color processing unit connected to the bus and performing image interpolation processing and color conversion processing on image data;
Connected to the bus and capable of pipeline connection with the interpolation color processing unit without passing through the bus, an edge emphasis processing unit that performs image emphasis processing on image data,
A noise removal processing unit connected to the bus and applying noise removal processing to the image data;
A compression processing unit connected to the bus and performing compression processing on the image data;
When the imaging sensitivity of the image data input by the input unit is the first imaging sensitivity, the image data stored in the memory is transmitted to the interpolation color processing unit via the bus, Interpolation color processed image data is generated by performing the pixel interpolation processing and the color conversion processing by the interpolation color processing unit on the image data, and the interpolation color processed image data is transmitted from the interpolation color processing unit via the bus. The interpolation color processed image data stored in the memory is transmitted to the noise removal processing unit via the bus, and the noise removal processing by the noise removal processing unit is performed by the interpolation. Applying the color processed image data to generate noise-removed image data, and applying the noise-removed image data from the noise removal processing unit to the contour enhancement The contour enhancement processing unit performs the contour enhancement processing on the noise-removed image data to generate the contour enhancement processed image data, and the contour enhancement processed image data is converted to the contour enhancement processing. Sending from the processing unit to the memory via the bus, controlling to transmit the edge enhancement processed image data stored in the memory to the compression processing unit via the bus,
When the imaging sensitivity of the image data input by the input unit is a second imaging sensitivity that is lower than the first imaging sensitivity, the image data stored in the memory is interpolated via the bus. The image data is transmitted to a color processing unit, and the pixel interpolation processing and the color conversion processing by the interpolation color processing unit are performed on the image data to generate image data that has undergone interpolation color processing. The interpolated color processed image data is transmitted from the interpolated color processing unit to the contour emphasizing processing unit without going through, and the contour emphasizing processing by the contour emphasizing processing unit is performed on the interpolated color processed image data. Emphasized image data is generated, and the contour emphasized image data is transmitted from the contour emphasizing processing unit to the memory via the bus, and stored in the memory. By transmitting the contour-enhanced processed image data to the compression processing unit via the bus, the noise removal processing by the noise removal processing unit is performed on the image data input by the input unit. A control unit for controlling the
An image processing apparatus. The image processing apparatus according to claim 1.
The control unit controls to record the compressed image data subjected to the compression processing by the compression processing unit on a recording medium.
An image processing apparatus. The image processing apparatus according to claim 1 or 2,
The control unit switches whether to perform the noise removal processing by the noise removal processing unit on the image data according to an instruction from a user.
An image processing apparatus. The image processing apparatus according to any one of claims 1 to 3,
The control unit switches the intensity of the noise removal processing by the noise removal processing unit according to the photographing sensitivity of the image data.
An image processing apparatus. In the image processing device according to any one of claims 1 to 4,
The control unit switches the intensity of the noise removal processing by the noise removal processing unit in accordance with an instruction from a user.
An image processing apparatus. The image processing apparatus according to any one of claims 1 to 5,
The control unit sets the strength of the noise removal processing by the noise removal processing unit and the strength of the contour enhancement processing by the contour enhancement processing unit independently for the image data input by the input unit. thing
An image processing apparatus. An imaging unit that images a subject and generates image data;
An image processing apparatus according to any one of claims 1 to 6.
An imaging apparatus characterized by the above.

Images