KR102368222B1 - Image processing apparatus, image processing method, and photographing apparatus - Google Patents

Abstract

An image processing apparatus is disclosed. The apparatus includes an image reduction unit, a moiré determination unit, and a moiré removal unit. The image reduction unit acquires an image signal output from a sensor array in which a plurality of light receiving elements are arranged in a two-dimensional Bayer array, reduces the image signal, and then develops the image signal to generate reduced image data. The moire determination unit determines an area in which moire occurs with respect to the reduced image data generated by the image reduction unit. The moire removal unit removes moiré from the original image data obtained by developing the image signal according to the determination result of the moiré generation region by the moiré determination unit.

Description

Image processing apparatus, image processing method, and photographing apparatus TECHNICAL FIELD

The present invention relates to an image processing apparatus, an image processing method, and a photographing apparatus.

The image sensor (ImageSensor) used to capture still images or moving pictures in photographing devices such as digital still cameras (Digital 　 still cameras) and digital video cameras (Digital 　 Cameras) includes a Bayer array CMOS image sensor or CCD. An image sensor is mainly used.

The Bayer arrangement is an arrangement in which four pixels including two green pixels, one red pixel, and one blue pixel are configured as a pair. The Bayer arrangement was developed to reduce the design of a complex optical system and to eliminate pixel shift in relation to the original light spectrum and RGB light reception by a separate sensor. This Bayer arrangement utilizes the characteristic that the human eye is insensitive to red or blue resolution and is sensitive to green or black and white.

When generating an image using the YCbCr color space or RGB color model from the signal output from the image sensor of the Bayer array (hereinafter referred to as “development”), it is due to the irradiation of the image with a period shorter than the existence period of each color Pulsed color (color moiré) may occur. A technique for removing this pulsed color (color moiré) is being developed (see, for example, Japanese Patent Application Laid-Open No. 2005-210218 as Patent Document 1 and Japanese Patent Application Laid-Open No. 2010-4396 as Patent Document 2, etc.) .

(Patent Document 1) Japanese Patent Laid-Open No. 2005-210218 (Patent Document 2) Japanese Patent Laid-Open No. 2010-4396

The technique disclosed in Japanese Patent Laid-Open No. 2005-210218 (hereinafter referred to as Patent Document 1) is to reduce color moiré by changing the Bayer arrangement. In addition, the technique disclosed in Japanese Patent Application Laid-Open No. 2010-4396 (hereinafter referred to as Patent Document 2) uses a temporally different color interpolation image to reduce noise by a three-dimensional noise reduction method to achieve color moiré. is to suppress

However, there was a problem in that when the data of the entire Bayer array was used for development, no technique was effective in reducing color moiré.

The problem of the background art as described above is the content that the inventor possessed for the derivation of the present invention or acquired during the derivation of the present invention, and it cannot be said that the content was known to the general public before the filing of the present invention.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image processing apparatus, an image processing method, and a photographing apparatus capable of effectively removing color moiré when developed using data of the entire Bayer array.

An image processing apparatus according to a first aspect of the present invention includes an image reduction unit, a moiré determination unit, and a moire removal unit.

The image reduction unit acquires an image signal output from a sensor array in which a plurality of light receiving elements that receive light through an optical system and photoelectrically convert it are arranged in a two-dimensional Bayer array, reduce the image signal, and then develop and reduce the image signal Create image data.

The moire determination unit determines an area in which moire occurs with respect to the reduced image data generated by the image reduction unit.

The moiré removal unit removes moiré from the original image data obtained by developing the image signal according to a result of the determination of the moiré generation region by the moiré determination unit.

The moiré determining unit may determine an area in which a difference occurs in the repeating patterns of the first color and the second color of the reduced image data as the moiré generating area.

The moiré determination unit may divide the reduced image data into predetermined areas and, if there is an area in which a difference between the repeating pattern of the first color and the second color occurs, the area may be determined as a moiré generation area. there is.

The moiré determination unit may compare a certain area with another area and determine the area as a moiré generating area if the difference between the repeating patterns of the first color and the second color is more prominent than the other area.

The first color may be a color having the largest number of pixels in the Bayer arrangement, and the second color may be a color generated from a color other than the first color in the Bayer arrangement.

The first color may be green, and the second color may be magenta.

The image processing apparatus may include: a storage unit for storing the original image data and the reduced image data; and a storage control unit for controlling reading and writing of the original image data and the reduced image data to the storage unit.

An image processing method as a second aspect of the present invention includes an image reduction step, a moiré determination step, and a moire removal step.

In the image reduction step, an image signal output from a sensor array in which a plurality of light receiving elements that receive light through an optical system and photoelectrically convert light is obtained is acquired, and the image signal is reduced and then developed Reduced image data is generated.

In the moire determination step, a region in which moire occurs is determined in the reduced image data generated in the image reduction step.

In the moiré removal step, moiré is removed from the original image data obtained by developing the video signal according to the determination result of the moiré generation region by the moiré determination step.

A photographing apparatus as a third aspect of the present invention includes a photographing unit, an image reduction unit, a moiré determining unit, and a moiré removing unit.

The photographing unit includes a sensor array in which a plurality of light receiving elements that receive light through an optical system and photoelectrically convert the light beam are arranged in a two-dimensional Bayer arrangement.

The image reduction unit generates reduced image data by reducing and developing the image signal output from the photographing unit.

The moire determination unit determines an area in which moiré occurs with respect to the reduced image data generated by the image reduction unit.

The moiré removal unit removes moiré from the original image data obtained by developing the image signal according to a result of the determination of the moiré generation region by the moiré determination unit.

According to the image processing apparatus, the image processing method, and the photographing apparatus according to the embodiments of the present invention, image data is reduced, and color moiré is determined on the reduced image data. Accordingly, color moiré that appears when the data of the entire Bayer array is used for development can be effectively removed.

1 is a view showing a Bayer arrangement.
2 is a diagram illustrating an example of an image serving as a photographing source.
FIG. 3 is a view showing an example of an image in which color moiré occurs when the image shown in FIG. 2 is captured.
FIG. 4 is a diagram showing a state in which the color moiré of cyan and orange occurs in a Bayer arrangement.
FIG. 5 is a diagram showing a state in which the color moiré of cyan and orange occurs in a Bayer arrangement.
FIG. 6 is a diagram showing the occurrence of green and magenta color moiré in a Bayer arrangement.
7 is a diagram showing an example of the configuration of the photographing apparatus 100 according to an embodiment of the present invention.
8 is a diagram showing an example of the configuration of the interpolation processing unit 106 included in the photographing apparatus 100 according to an embodiment of the present invention.
9 is a flowchart illustrating an example of an operation of the photographing apparatus 100 according to an embodiment of the present invention.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. In addition, in the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and thus redundant descriptions are omitted.

<1. Background>

Before describing the preferred embodiment of the present invention in detail, the background leading to the embodiment of the present invention will be described.

As described above, in a photographing apparatus such as a digital camera or a digital video camera, a Bayer array CMOS image sensor or a CCD image sensor is mainly used as a sensor used for photographing a still image or a moving image.

1 is a view showing a Bayer arrangement. The Bayer arrangement is an arrangement in which four pixels of one red (R) and one blue (B) are paired with respect to two green (G) pixels. The Bayer arrangement as shown in FIG. 1 was developed to reduce the design of a complex optical system and to eliminate pixel shift, which was originally used to receive RGB light by a separate image sensor by splitting light.

The Bayer arrangement shown in FIG. 1 utilizes the characteristic that the human eye is insensitive to red or blue resolution and is sensitive to green or black and white. In addition, for convenience, the green (G) pixel in the red (R) row is referred to as Gr, and the green (G) pixel in the blue (B) row is referred to as Gb.

Since the Bayer arrangement consists of a pair of 4 pixels of 1 red (R) and 1 blue (B) for 2 green (G) pixels, green is the number of horizontal pixels of the image sensor in the horizontal and vertical directions and It has the same band as the number of vertical pixels, but is reduced to half of the horizontal and vertical in the band in the 45 ^o direction of inclination. Accordingly, the luminance signal has a small band in the oblique direction. In addition, since the Bayer arrangement consists of a pair of 4 pixels of 1 red (R) and 1 blue (B) for 2 green (G) pixels, the blue and red regions are only half of green.

In order to obtain a normal image with such a Bayer array image sensor, conventionally, for example, a filter that does not resolve lines thinner than the basic 4-pixel thickness of the Bayer array is mounted on the image sensor to blur each color. thing has been done

However, when such a filter is mounted on an image sensor, red and blue thin lines are not output from the image sensor, and green, which is a major component of luminance, is also not output from the sensor if it is thinner than 2-pixel width.

And, of course, mounting such a filter on the image sensor causes cost increase and also causes image resolution to be lowered compared to the actual number of pixels. As a result, the necessity of excluding the filter and satisfying both resolution improvement and cost reduction is argued, and products aiming to satisfy both resolution improvement and cost reduction are also sold.

However, when an image sensor excluding a filter is used, when generating (developing) an image using the YCbCr color space or RGB color model from the signal output from the image sensor of the Bayer array, it is shorter than the existence period of each color. Pulsed color (color moiré) resulting from periodic image irradiation may occur.

When the image sensor excluding the filter is used, for example, a red image of a shorter period than the red existence period of the image sensor is irradiated. Irradiating a red image with a shorter period than the red existence period of the image sensor causes aliasing for red.

This can be said to be the same not only for red, but also for blue in which only one pixel exists among four pixels. In addition, irradiating a vivid green image in an oblique direction also for green causes oblique aliasing with respect to green. This aliasing becomes color moiré that is generated when an image using the YCbCr color space or RGB color model is generated (developed) from the signal output from the image sensor of the Bayer array.

The biggest problem is the moire of green and magenta that occurs when thin lines in the oblique direction are developed. This indicates an oblique disturbance. This green and magenta moiré phenomenon occurs because there are very thin lines in the oblique direction in the captured image.

2 is a diagram illustrating an example of an image serving as a photographing source. And FIG. 3 is a view showing an example of an image in which color moiré occurs when the image shown in FIG. 2 is captured.

When an image in which very thin white and black lines are mixed as shown in FIG. 2 is taken, colors other than white and black do not exist in the original image, but as shown in FIG. 3 , a swirling pulse color (color moiré) may occur. Reference numeral 11 in Fig. 3 denotes a portion where cyan and orange swirl-shaped pulse colors (color moiré) are generated. Reference numeral 12 in Fig. 3 denotes a portion in which green and magenta swirl-shaped pulse colors (color moiré) are generated.

The color moiré of cyan and orange generated in the portion indicated by reference numeral 11 in FIG. 3 is generated to reproduce a vertical or horizontal thin line. 4 and 5 are views showing the occurrence of cyan and orange color moiré in a Bayer arrangement.

As shown in FIGS. 4 and 5 , when the color moiré of cyan and orange occurs, it can be seen that the green part is changed at the boundary between cyan and orange. Therefore, it can be determined whether or not the color moiré of cyan and orange occurs by taking the correlation with green. In addition, when the color moiré of cyan and orange occurs, for example, the color moiré may be removed by connecting a portion where the color moiré occurs with white.

On the other hand, green and magenta color moiré generated in the portion indicated by reference numeral 12 in FIG. 3 is generated to reproduce the thin line in the oblique direction. FIG. 6 is a diagram showing the occurrence of green and magenta color moiré in a Bayer arrangement.

However, in the case of this green and magenta color moiré, the occurrence of color moiré cannot be determined by correlating with green like that of cyan and orange color moiré. This is because, unlike the color moiré of cyan and orange, as shown in FIG. 6 , it cannot be said that green is always attached.

The green and magenta color moiré can be removed by the following processing in terms of software.

The probability of appearance of the repeating pattern of green and magenta in a natural image is very low, and the probability that it is a broad and perfectly constant frequency on an image taken of a thin line in the oblique direction is very low. Therefore, it is possible to observe a wide range of images, and if there is a repeating pattern of green and magenta, it is actually possible to determine that part as a black-and-white thin line in the oblique direction. In the case of software processing, the situation is determined using, for example, a space of 128 × 128 pixels or a large one of 512 × 512 pixels.

In this way, it is easy to determine the color moiré of green and magenta in software and remove the color moiré of green and magenta. However, it is very difficult to perform the same processing in terms of hardware.

Therefore, the present inventors earnestly studied a technique for removing color moiré, in particular, a technique for effectively removing color moiré of green and magenta in which the occurrence of color moiré cannot be determined by taking a correlation with green. As a result, the present inventor reduces image data obtained from a digital image signal, determines the occurrence of color moiré with respect to the reduced image data, and removes color moiré according to the determination result, as described below. came to invent

In the above, the background leading to the embodiment of the present invention has been described. Next, an embodiment of the present invention will be described in detail.

<2. One embodiment of the present invention>

[2.1. Example of function configuration]

First, an example of the configuration of a photographing apparatus according to an embodiment of the present invention will be described in detail. 7 is a diagram showing an example of the configuration of the photographing apparatus 100 according to an embodiment of the present invention. Hereinafter, an example of the configuration of the photographing apparatus 100 according to an embodiment of the present invention will be described with reference to FIG. 7 .

7 , the photographing apparatus 100 according to an embodiment of the present invention includes a photographing unit 102 , an interpolation processing unit 106 , a codec unit 108 , a communication unit 110 , and a bus ( 112), a control unit 114, and DDR   SDRAM (Double   Data   Rate   Synchronous   Dynamic Random Access Memory 116).

The photographing unit 102 includes a plurality of lens groups including a zoom lens or a focus lens, an image sensor, and the like. The photographing unit 102 photoelectrically converts a subject image formed on the light-receiving surface of the image sensor, and performs signal processing such as predetermined noise removal processing, sensitivity adjustment processing, analog/digital conversion (A/D conversion), and the like to obtain a digital image. generate a signal

The image sensor installed in the photographing unit 102 may be a CCD image sensor or a CMOS image sensor. In addition, a Bayer array primary color filter is installed on the light receiving surface of the image sensor installed in the photographing unit 102 to capture a color image. The digital image signal generated by the photographing unit 102 is stored in the DDR SDRAM 116 through the bus 112 .

In the present embodiment, when generating the digital image signal, the photographing unit 102 generates reduced image data reduced to a predetermined size in addition to the image signal that has not been reduced, and a bus 112 for the reduced image data. ) through the DDR　SDRAM (116). For example, the photographing unit 102 performs pixel addition, generates reduced image data in which the horizontal and vertical directions are reduced by 1/4, and stores the reduced image data in the DDR SDRAM 116 . This reduced image data is used for moiré determination processing in the interpolation processing unit 106, which will be described later.

The interpolation processing unit 106 performs development processing by performing interpolation processing, for example, CFA (Color Filter Array) interpolation processing, on the digital image signal generated by the photographing unit 102 and stored in the DDR SDRAM 116 . The image data obtained by performing the development process by the interpolation processing unit 106 is stored in the DDR SDRAM 116 via the bus 112 .

In the present embodiment, the interpolation processing unit 106 reduces the digital image signal generated by the photographing unit 102 and stored in the DDR SDRAM 116 to a predetermined size before the digital image signal is developed. Then, moiré determination processing is performed on the reduced image data generated by developing. The color moiré phenomenon is not reduced even if the image is reduced, and since the information is condensed rather than the size of the original image when the image is reduced, the color moiré effect is more pronounced and changes more rapidly.

When an image of a thin line in which color moiré occurs is photographed by a device including an image sensor in which pixels are Bayer, there is no correlation between red, blue, and green. Therefore, even when each signal of R, Gb, Gr, and B is added (Binning), the phenomenon of color moiré does not disappear, and the position of occurrence of color moiré when viewed as a whole is the same in the original image and the reduced image.

The present embodiment performs moiré determination processing on the reduced image data for the interpolation processing unit 106 using this aspect. By performing moiré determination processing on the reduced image data, the photographing apparatus 100 according to an embodiment of the present invention provides 16 times the space even with the same amount of memory if the image is reduced by 1/4 in the vertical and horizontal directions. can be observed to determine whether color moiré has occurred.

The codec unit 108 performs encoding processing on the image data generated by the interpolation processing unit 106 or decoding processing of the encoded image data. Encoding processing and decoding processing are collectively referred to as codec processing. The video data processed by the codec by the codec unit 108 is stored in the DDR SDRAM 116 through the bus 112 .

The communication unit 110 executes a process for transmitting information to an external device and a process for receiving information from an external device. For example, the communication unit 110 transmits image data encoded by the codec unit 108 to an external device.

The control unit 114 is configured of, for example, a CPU (Central Processing Unit) and the like, and controls the operation of each unit of the photographing apparatus 100 .

In addition, although not shown in FIG. 1 , the photographing apparatus 100 includes a shutter button for instructing the user to shoot an image with respect to the photographing unit 102 , an operation button for accepting a user's operation on the photographing apparatus 100 , A recording medium for storing the image data generated by the interpolation processing unit 106 may be provided. The recording medium for preserving the image data generated by the interpolation processing unit 106 may be detachable from the photographing apparatus 100 .

Above, an example of the configuration of the photographing apparatus 100 according to an embodiment of the present invention has been described with reference to FIG. 7 . Next, an example of a detailed configuration of the interpolation processing unit 106 included in the photographing apparatus 100 according to an embodiment of the present invention will be described.

8 is a diagram showing an example of a detailed configuration of the interpolation processing unit 106 included in the photographing apparatus 100 according to an embodiment of the present invention. Hereinafter, an example of a detailed configuration of the interpolation processing unit 106 included in the photographing apparatus 100 according to an embodiment of the present invention will be described with reference to FIG. 8 .

As shown in FIG. 8 , the interpolation processing unit 106 includes an RDMA (Read Direct Memory Access) controller 121 , a buffer memory 122 , a memory control unit 123 , an image generation unit 124 , and a moire determination unit 125 . ), a switch unit 126 , and a Write Direct Memory Access (WDMA) controller 127 .

The RDMA controller 121 reads the digital image signal generated by the photographing unit 102 or the reduced image data generated by reducing the digital image signal to a predetermined size and then developing it from the DDR SDRAM 116 through the bus 112 . . When the digital image signal or reduced image data generated by the photographing unit 102 is read from the DDR SDRAM 116 through the bus 112 , the RDMA controller 121 stores the read digital image signal or reduced image data in the buffer memory 122 . ) is stored in

In the present embodiment, the interpolation processing unit 106 includes a mode (referred to as mode A for convenience) for developing the digital image signal generated by the photographing unit 102, and the digital image signal generated by the photographing unit 102 It operates by switching between two operation modes (referred to as mode B for convenience) in which color moiré judgment processing is performed on reduced image data obtained by developing after reducing the image to a predetermined size.

When the interpolation processing unit 106 operates in mode A, the RDMA controller 121 reads the digital image signal generated by the photographing unit 102 from the DDR SDRAM 116 through the bus 112 .

On the other hand, when the interpolation processing unit 106 operates in mode B, the RDMA controller 121 converts the reduced image data obtained by reducing the digital image signal generated by the photographing unit 102 to a predetermined size and then developing the reduced image data to the DDR SDRAM 116 . ) through the bus 112.

The buffer memory 122 is constituted of, for example, a memory such as SRAM (Static Random Access Memory). The buffer memory 122 stores a digital image signal or reduced image data transmitted from the RDMA controller 121 . The digital image signal or reduced image data stored in the buffer memory 122 is sequentially read by the memory controller 123 .

In this embodiment, the buffer memory 122 functions as a line memory when storing digital image signals. Also, the buffer memory 122 functions as a memory in which each line of the line memory is further divided into a plurality of blocks when storing reduced image data.

When the interpolation processing unit 106 operates in mode A, the buffer memory 122 stores the digital image signal transmitted from the RDMA controller 121 . Meanwhile, when the interpolation processing unit 106 operates in mode B, the buffer memory 122 stores reduced image data transmitted from the RDMA controller 121 .

The memory controller 123 controls the buffer memory 122 to read data stored in the buffer memory 122 , that is, a digital image signal or reduced image data. In the present embodiment, when a digital image signal is stored in the buffer memory 122 , the memory controller 123 reads the digital image signal from the buffer memory 122 according to address information instructed by the image generator 124 . . In addition, when reduced image data is stored in the buffer memory 122 , the memory controller 123 reads the digital image signal from the buffer memory 122 according to the address information instructed by the moire determining unit 125 .

When the interpolation processing unit 106 operates in mode A, the memory control unit 123 reads data stored in the buffer memory 122 according to an instruction from the image generation unit 124 . Meanwhile, when the interpolation processing unit 106 operates in mode B, the memory control unit 123 reads data stored in the buffer memory 122 according to an instruction from the moire determination unit 125 .

The image generator 124 performs a developing process (a process of generating image data) by performing interpolation between pixels on the digital image signal read through the memory controller 123 . In the present embodiment, the image generator 124 performs the developing process by performing color filter array (CFA) interpolation process on the digital image signal.

Also, the image generating unit 124 performs interpolation while removing color moiré according to a result of color moiré determination processing for determining an area in which color moiré occurs by the moiré determination unit 125, which will be described later. When color moiré is removed, the image generating unit 124 interpolates, for example, a portion in which color moiré occurs while connecting a white color. This method of removing color moiré is not limited to a specific method, and the image generating unit 124 may effectively remove color moiré according to the result of the color moiré determination processing performed by the moire determination unit 125 .

When instructing the memory controller 123 to read the digital image signal from the buffer memory 122 , the image generator 124 instructs address information to be read in units of a plurality of lines and output to the image generator 124 . do. The image generator 124 generates image data by repeating interpolation processing on the digital image signal read in units of a plurality of lines until one image is completed. The image generator 124 transmits the image data generated by the developing process to the WDMA controller 127 .

The moiré determination unit 125 performs color moiré determination processing for determining an area in which color moiré occurs on the reduced image data stored in the buffer memory 122 and read through the memory control unit 123 . The moire determination unit 125 instructs the memory control unit 123 to read the reduced image data from the buffer memory 122 in order to perform color moiré determination processing on all of the reduced image data, the reduced image data The address information is instructed so that the whole is read and output to the moire determination unit 125 .

Then, the moiré determination unit 125 subdivides the color moiré, that is, a portion in which a pattern in which two specific colors appear repeatedly, with respect to the entire reduced image data transmitted from the memory control unit 123, and subdivides the buffer memory 122 . It is determined for each area of the reduced image data stored in the .

The moiré determination unit 125 determines whether or not color moiré occurs in each region of the reduced image data, but in this embodiment, when determining whether or not color moiré occurs, the probability of color moiré is determined in a plurality of steps, yes For example, it may be determined as 10 steps from 1 to 10. In this case, the one with a larger number may be made into a thing with a high probability of color moirail.

The moire determination unit 125 transmits the color moiré determination processing result for each area of the reduced image data to the WDMA controller 127 in association with information identifying each area and the color moiré determination processing result.

The switch unit 126 connects the WDMA controller 127 to the image generator 124 when the interpolation processing unit 106 operates in mode A, and when the interpolation processing unit 106 operates in mode A, the moire plate A conversion operation connecting the WDMA controller 127 to the government 125 is performed.

The WDMA controller 127 sends the image data generated by the developing process of the image generator 124 and the result of the color moiré judgment process by the moiré judgment part 125 to the DDR SDRAM 116 via the bus 112 . Save.

When the interpolation processing unit 106 operates in mode A, the WDMA controller 127 stores the image data generated by the developing process of the image generation unit 124 in the DDR SDRAM 116 .

On the other hand, when the interpolation processing unit 106 operates in mode B, the WDMA controller 127 stores the color moiré determination processing result by the moire determination unit 125 in the DDR SDRAM 116 .

The interpolation processing unit 106 included in the photographing apparatus 100 according to an embodiment of the present invention has a configuration as shown in FIG. 8, and thus determines the occurrence of color moiré in each area of the reduced image data, and , the development process may be performed while color moiré is removed according to the determination result of the occurrence of color moiré, and image data from which color moiré has been removed may be generated.

An example of a detailed configuration of the interpolation processing unit 106 included in the photographing apparatus 100 according to an embodiment of the present invention has been described with reference to FIG. 8 . Next, an example of the operation of the photographing apparatus 100 according to an embodiment of the present invention will be described.

[2.2. Operation example]

9 is a flowchart illustrating an example of an operation of the photographing apparatus 100 according to an embodiment of the present invention. As shown in Fig. 9, color moiré determination processing is performed on reduced image data obtained by developing a digital image signal after reduction, and according to the color moiré determination processing result, color moiré is removed while being the basis of reduced image data. This is an example of an operation of the photographing apparatus 100 when developing a digital image signal. Hereinafter, an example of the operation of the photographing apparatus 100 according to an embodiment of the present invention will be described with reference to FIG. 9 .

The photographing apparatus 100 first reduces the digital image signal to be developed and then executes color moiré determination processing on reduced image data obtained by developing, and determines an area in which color moiré occurs. The photographing apparatus 100 reads out the reduced image data stored in the DDR SDRAM 116 (step S101), divides the read reduced image data into a plurality of rectangular regions of the same size, and color moiré for each region It is determined whether or not it occurs (step S102). The read processing in step S101 is executed, for example, by the RDMA controller 121 , and the determination processing in step S102 is executed by, for example, the moire determination unit 125 .

The photographing apparatus 100 subdivides and stores the reduced image data in the buffer memory 122 for color moiré, that is, a portion in which a pattern in which two specific colors appear repeatedly, with respect to the entire reduced image data read from the DDR SDRAM 116 . It is determined for each area of the image data. The photographing apparatus 100 determines whether or not color moiré occurs in each area of the reduced image data read from the DDR SDRAM 116 . The photographing apparatus 100 according to the present embodiment may determine the probability of color moiré in a plurality of steps, for example, 10 steps of 1 to 10, when determining whether or not color moiré occurs. In this case, the one with a larger number can also be made into the thing with a high probability of color moirail.

When determining color moiré for a certain area, the photographing apparatus 100 may determine whether or not color moiré is present by comparing with other areas, particularly adjacent areas. For example, when a repeating pattern of green and magenta is found as a result of observing a region having reduced image data, the photographing apparatus 100 determines that a repeating pattern of green and magenta is not found in an adjacent region. It can also be determined that the repeating pattern of magenta and magenta is highly likely to be a color moiré.

When the photographing apparatus 100 determines whether or not color moiré occurs for each area in the reduced image data, the information identifying the area and the determination result of the area are correlated and recorded in the DDR SDRAM 116 (step S103). The write processing in step S103 is executed by, for example, the WDMA controller 127.

For example, the region identification information increases the value toward the right by setting the upper-left region of the reduced image data to 0, moves down one stage when reaching the region at the rightmost end, and returns a value from the left end to the right. is increased, and this may be repeated until the lower right region is reached.

Next, the photographing apparatus 100 reads the digital image signal, which is the basis of the reduced image data on which the color moiré determination processing has been performed, and the color moiré determination processing result from the DDR SDRAM 116 (step S104). The read processing in step S104 is executed by the RDMA controller 121, for example.

Then, the photographing apparatus 100 executes a developing process of generating image data by interpolation between pixels while removing color moiré according to the color moiré determination processing result with respect to the digital image signal read from the DDR SDRAM 116 . do (step S105). The process of step S105 is executed by, for example, the image generating unit 124 .

For example, when it is determined that color moiré occurs in the upper left region of the reduced image data, the photographing apparatus 100 removes color moiré from the region corresponding to the region in the digital image signal and interpolation of The photographing apparatus 100 removes color moiré by, for example, performing interpolation processing while connecting portions in which color moiré has occurred in white. Here, the method for removing such color moiré is not limited to a specific method.

The photographing apparatus 100 according to an embodiment of the present invention determines the occurrence of color moiré in each region of the reduced image data by executing a series of operations as shown in FIG. 9 , and determines the occurrence of color moiré According to the result, the development process is performed while color moiré is removed, and image data from which color moiré is removed may be generated.

<3. Summary>

As described above, according to an embodiment of the present invention, the occurrence of color moiré is determined for each region of the reduced image data, and the development process is performed while color moiré is removed according to the determination result of the color moiré, and color moiré is performed. Image data from which is removed may be generated.

The photographing apparatus 100 according to an embodiment of the present invention determines the occurrence of color moiré in each area of the reduced image data, so that it is possible to reliably determine the occurrence of color moiré in hardware even when the memory capacity is small. there is.

Each step in the processing executed by the apparatus in this specification is not necessarily time-series processed according to the order described as a flowchart or sequence diagram. For example, each step in the processing executed by each apparatus may be processed in an order different from the order described as a flowchart, or may be processed in parallel.

In addition, in order to exhibit the functions equivalent to the configuration of each device described above to hardware (not shown) such as CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory) embedded in the device of the present specification. A computer program may also be written. In addition, a storage medium in which the computer program is stored may be provided. In addition, by configuring each functional block shown in the functional block diagram in hardware, a series of processing can be realized in hardware.

In the above, preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to these examples. It is clear that those of ordinary skill in the technical field to which the present invention pertains can derive examples of various changes or modifications within the scope of the technical idea described in the claims. It is understood to be within the technical scope of the present invention.

The present invention is highly likely to be used in image processing apparatuses having various functions.

100 shooting devices
102 Cinematography
106 interpolation processing unit
108 codec part
110 Ministry of Communications
112 bus
114 control
116 SDRAM
121 RDMA controller
122 buffer memory
123 memory control
124 image generator
125 Moiré Judgment Unit
126 switch
127 WDMA controller

Claims (6)

A plurality of light receiving elements that receive light through an optical system and photoelectrically convert it acquire an image signal output from a sensor array arranged in a two-dimensional Bayer array, reduce the image signal, and then develop it to generate reduced image data image reduction unit;
a moire determination unit for determining an area in which moire occurs with respect to the reduced image data generated by the image reduction unit; and
and a moiré removing unit that removes moiré from original image data obtained by developing the image signal according to a result of the determination of the moiré generation region by the moiré determining unit. delete delete delete A plurality of light receiving elements that receive light through an optical system and photoelectrically convert it acquire an image signal output from a sensor array arranged in a two-dimensional Bayer array, reduce the image signal, and then develop it to generate reduced image data image reduction step;
a moire determination step of determining an area in which moire occurs with respect to the reduced image data generated in the image reduction step; and
and a moiré removal step of removing moiré from the original image data obtained by developing the image signal according to the determination result of the moiré generation region by the moiré determination step. ◈Claim 6 was abandoned when paying the registration fee.◈ a photographing unit including a sensor array in which a plurality of light receiving elements that receive light through an optical system and photoelectrically convert them are arranged in a two-dimensional Bayer array;
an image reduction unit for generating reduced image data by reducing and developing the image signal output from the photographing unit;
a moiré determination unit for determining an area in which moiré occurs with respect to the reduced image data generated by the image reduction unit; and
and a moiré removing unit that removes moiré from original image data obtained by developing the image signal according to a determination result of the moiré generation region by the moiré determining unit.

Images