JP2017005644A - Image processing apparatus, image processing method and imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of removing color moire effectively, when developing data of the entire Bayer array, as they are, by reducing image data, and determining the color moire for the reduced image data.SOLUTION: An image processing apparatus includes: an image reduction unit for generating reduced image data by reducing and then developing image signals, outputted from a sensor array where a plurality of light-receiving elements for receiving a light beam via an optical system and converting photoelectrically are placed two-dimensionally in Bayer arrangement; a moire determination unit for determining a region where moire is occurring from the reduced image data generated in the image reduction unit; and a moire removal unit for removing moire from original image data obtained by developing the image signal by using the results of determination of a region where moire is occurring, made by the moire determination unit.SELECTED DRAWING: Figure 8

Description

  The present invention relates to an image processing device, an image processing method, and an imaging device.

  A Bayer array CMOS image sensor is used as an image sensor (Image Sensor) used to capture still images and moving images in an imaging apparatus such as a digital still camera or a digital video camera. Alternatively, a CCD image sensor is mainly used.

  The Bayer arrangement is an arrangement in which four pixels of one red pixel and one blue pixel are combined with two green pixels. The Bayer array was originally developed to reduce the complexity of the design of the optical system and eliminate pixel shifts, while originally separating light and receiving RGB with another sensor. This Bayer array uses the characteristic that the human eye is insensitive to red and blue resolutions and sensitive to green and black and white.

  When generating (developing) an image in the YCbCr color space or RGB color model from signals output from a Bayer array image sensor, false colors (color moiré) are generated due to irradiation of finer images than the period of each color. There is a case. A technique for removing the false color (color moire) has been developed (see, for example, Patent Documents 1 and 2).

Japanese Patent Laying-Open No. 2005-210218 JP 2010-4396 A

  The technique disclosed in Patent Document 1 is intended to reduce color moire by changing the Bayer arrangement. The technique disclosed in Patent Document 2 suppresses color moire by using temporally different color interpolation images when noise is reduced by the three-dimensional noise reduction method. However, each of the techniques has a problem in that it is not effective in reducing color moiré when developed as it is from data of the entire Bayer array.

  Accordingly, the present invention has been made in view of the above problems, and an object of the present invention is to reduce the image data and perform color moiré determination on the reduced image data, so that the entire Bayer array can be determined. It is an object of the present invention to provide a new and improved image processing apparatus, image processing method, and imaging apparatus capable of effectively removing color moire when developed as it is from data.

  In order to solve the above problems, according to an aspect of the present invention, a plurality of light receiving elements that receive a light beam through an optical system and perform photoelectric conversion are output from a sensor array that is two-dimensionally arranged in a Bayer array. An image reduction unit that obtains an image signal, develops the image signal after reduction, and generates reduced image data, and a moire determination unit that determines a region where moire is generated from the reduced image data generated by the image reduction unit And a moire removing unit that removes moire from the original image data obtained by developing the image signal using the determination result of the area where moire is generated by the moire determining unit. An image processing apparatus is provided.

  The moire determination unit may determine a region where a moire is generated at a place where a difference in the repeated pattern between the first color and the second color occurs in the reduced image data.

  The moiré determination unit divides the reduced image data into predetermined areas, and if there is a place where a repeated pattern difference between the first color and the second color occurs, moiré is generated in the area. It may be determined that the region is a

  The moire determination unit compares a certain area with another area, and if a difference in the repetitive pattern between the first color and the second color appears in the area, moire occurs in the area. It may be determined that the area is.

  The first color may be a color having the largest number of pixels in the Bayer array, and the second color may be a color generated from a color other than the first color in the Bayer array.

  The first color may be green and the second color may be magenta.

  The image processing apparatus further includes a storage unit that stores the original image data and the reduced image data, and a storage control unit that controls reading and writing of the original image data and the reduced image data with respect to the storage unit. Good.

  In order to solve the above problem, according to another aspect of the present invention, a sensor array in which a plurality of light receiving elements that receive and photoelectrically convert a light beam via an optical system is arranged in a two-dimensional Bayer array. An image reduction step of acquiring an output image signal, developing the image signal after reduction and generating reduced image data, and determining a region where moire is generated from the reduced image data generated in the image reduction step And a moire removal step of removing moire from the original image data obtained by developing the image signal using a determination result of a region where moire is generated by the moire determination step. An image processing method is provided.

  In order to solve the above problem, according to another aspect of the present invention, there is provided a sensor array in which a plurality of light receiving elements that receive a light beam through an optical system and perform photoelectric conversion are arranged in a two-dimensional manner in a Bayer array. An image capturing unit, an image reducing unit that generates reduced image data by developing after reducing an image signal output from the image capturing unit, and a region where moire is generated from the reduced image data generated by the image reducing unit. A moire determination unit for determining, and a moire removal unit for removing moire from the original image data obtained by developing the image signal using the determination result of the area where moire is generated by the moire determination unit. An imaging apparatus is provided.

  As described above, according to the present invention, image data is reduced, and color moiré is determined for the reduced image data, thereby effectively removing color moiré when developed as it is from the entire Bayer array data. A new and improved image processing apparatus, image processing method, and imaging apparatus that can be performed can be provided.

It is explanatory drawing which shows a Bayer arrangement | sequence. It is explanatory drawing which shows the example of the image used as the image origin. It is explanatory drawing which shows the example of the image which color moiré generate | occur | produced when the image shown in FIG. 2 was imaged. It is explanatory drawing which shows a mode that the color moire of cyan and orange has occurred in the Bayer arrangement. It is explanatory drawing which shows a mode that the color moire of cyan and orange has occurred in the Bayer arrangement. It is explanatory drawing which shows a mode that the color moire of green and magenta has occurred with the Bayer arrangement. It is explanatory drawing which shows the function structural example of the imaging device 100 which concerns on one Embodiment of this invention. It is explanatory drawing which shows the function structural example of the interpolation process part 106 contained in the imaging device 100 which concerns on one Embodiment of this invention. 5 is a flowchart illustrating an operation example of the imaging apparatus 100 according to an embodiment of the present invention.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

<1. Background>
Before describing the preferred embodiment of the present invention in detail, the background leading to the embodiment of the present invention will be described.

  As described above, a Bayer array CMOS image sensor or a CCD image sensor is mainly used as a sensor used to capture a still image or a moving image in an imaging apparatus such as a digital still camera or a digital video camera.

  FIG. 1 is an explanatory diagram showing a Bayer array. The Bayer array is an array in which four pixels of one pixel of red (R) and one pixel of blue (B) are combined with two pixels of green (G). The original Bayer as shown in FIG. 1 has been developed to reduce the complexity of the optical system design and eliminate pixel shifts from what originally separated light and received RGB by another image sensor. Is an array. The Bayer array shown in FIG. 1 uses the characteristic that the human eye is insensitive to red and blue resolutions and sensitive to green and black and white. For convenience, a green (G) pixel in the red (R) row is referred to as Gr, and a green (G) pixel in the blue (B) row is referred to as Gb.

  Since the Bayer array has a configuration in which four pixels of one pixel of red (R) and one pixel of blue (B) are combined with two pixels of green (G), green is horizontal and vertical. Although it has a band equal to the number of horizontal pixels and the number of vertical pixels of the image sensor, the band in the direction of 45 degrees is reduced to half of the horizontal and vertical. Therefore, the luminance signal has a small band in the oblique direction. In addition, since the Bayer array has a configuration in which four pixels of red (R) 1 pixel and blue (B) 1 pixel are combined with 2 green (G) pixels, the blue and red regions are There is only half of green.

  In order to obtain a normal image with this Bayer array image sensor, conventionally, for example, a filter that does not resolve lines finer than the basic four pixels of the Bayer array is mounted on the image sensor to blur each color. Has been done.

  However, when such a filter is mounted on the image sensor, fine lines of red and blue are not output from the image sensor, and green, which is a main component of luminance, is not output from the sensor.

  Of course, mounting such a filter in an image sensor causes an increase in cost, and also causes a reduction in the resolution of an image expressed than the actual number of pixels. As a result, the necessity of eliminating this filter and satisfying the improvement in resolution and the cost reduction at the same time is screamed, and products aiming to satisfy the improvement in resolution and the cost reduction at the same time are on the market.

  However, when an image sensor that excludes a filter is used, when an image based on the YCbCr color space or the RGB color model is generated (developed) from a signal output from the image sensor in the Bayer array, an image that is finer than the existence cycle of each color is generated. False colors (color moire) due to irradiation may occur.

  When an image sensor excluding the filter is used, for example, a red image finer than the red existence period of the image sensor is irradiated. Irradiation of a red image that is finer than the red existence period of the image sensor is a factor that causes folding of red.

  The same can be said for not only red but also blue that exists only in one of four pixels. Also, for green, the fact that a fine green image is irradiated in an oblique direction is a factor that causes folding in the oblique direction with respect to green. This aliasing becomes a color moire that can be generated when an image is generated (developed) in the YCbCr color space or RGB color model from the signal output from the Bayer array image sensor.

  The biggest problem is the moire of green and magenta that occurs when developing oblique thin lines. This represents an obstruction in the diagonal direction. This phenomenon of green and magenta moire occurs because there are very thin lines in the oblique direction in the captured image.

  FIG. 2 is an explanatory diagram illustrating an example of an image to be captured. FIG. 3 is an explanatory diagram illustrating an example of an image in which color moire occurs when the image illustrated in FIG. 2 is captured.

  When an image with very thin white and black lines as shown in FIG. 2 is captured, the original image has no color other than white and black, but as shown in FIG. False color (color moire) may occur. Reference numeral 11 in FIG. 3 indicates a place where a cyan and orange spiral false color (color moire) is generated. Reference numeral 12 in FIG. 3 indicates a place where a green and magenta spiral false color (color moire) has occurred.

  The cyan and orange color moire generated in the place indicated by reference numeral 11 in FIG. 3 is generated in an attempt to reproduce a vertical or horizontal thin line. FIG. 4 and FIG. 5 are explanatory diagrams showing a state in which the color moire of cyan and orange is generated in a Bayer arrangement.

  As shown in FIGS. 4 and 5, when cyan and orange color moire occurs, it can be seen that the green location changes at the boundary between cyan and orange. Accordingly, it is possible to determine whether or not cyan and orange color moire has occurred by correlating with green. When cyan and orange color moire occurs, it is possible to remove the color moire by, for example, connecting the places where the color moire occurs with white.

  On the other hand, the green and magenta color moire generated at the location indicated by reference numeral 12 in FIG. 3 is generated in an attempt to reproduce a slanting thin line. FIG. 6 is an explanatory diagram showing a state in which green and magenta color moire occurs in a Bayer array.

  However, the color moiré of green and magenta cannot be determined by correlating with green like the color moiré of cyan and orange. This is because, unlike cyan and orange color moire, it is not always green as shown in FIG.

  The color moire of green and magenta can be removed by software as follows. The appearance probability of green and magenta repetitive patterns in natural images is very low, and it is very unlikely that they will be in a wide range and a completely constant frequency on an image obtained by imaging oblique thin lines. When an image is observed and there are repeated green and magenta patterns, it is actually possible to determine that the location is a black and white thin line in an oblique direction. In the case of processing by software, the above situation is determined using a space of, for example, 128 pixels × 128 pixels or a larger one, 512 pixels × 512 pixels.

  As described above, it is easy for software to determine the color moire between green and magenta and to remove the color moire between green and magenta. However, it is extremely difficult to perform the same processing in terms of hardware.

  Therefore, the present inventor has a technique for removing color moiré, particularly for effectively removing green and magenta color moiré that cannot be determined by correlation with green. We conducted an intensive study on the technology. As a result, as described below, the inventor reduces the image data obtained from the digital image signal, determines the occurrence of color moiré for the reduced image data, and reduces the color moiré based on the determination result. It came to devise the technology which can be removed.

  The background that led to the embodiment of the present invention has been described above. Subsequently, an embodiment of the present invention will be described in detail.

<2. One Embodiment of the Present Invention>
[2.1. Functional configuration example]
First, a functional configuration example of an imaging apparatus according to an embodiment of the present invention will be described in detail. FIG. 7 is an explanatory diagram illustrating a functional configuration example of the imaging apparatus 100 according to an embodiment of the present invention. Hereinafter, a functional configuration example of the imaging apparatus 100 according to an embodiment of the present invention will be described with reference to FIG.

  As illustrated in FIG. 7, the imaging apparatus 100 according to an embodiment of the present invention includes an imaging unit 102, an interpolation processing unit 106, a codec unit 108, a communication unit 110, a bus 112, and a control unit 114. DDR SDRAM (Double Data Rate Synchronous Random Access Memory) 116.

  The imaging unit 102 includes a plurality of lens groups including a zoom lens (Zoom Lens) and a focus lens (Focus Lens), an image sensor, and the like. The imaging unit 102 photoelectrically converts a subject image formed on the light receiving surface of the image sensor, and performs signal processing such as predetermined noise removal processing, sensitivity adjustment processing, analog / digital conversion (A / D conversion), and the like. A digital image signal is generated.

  The image sensor provided in the imaging unit 102 may be a CCD image sensor or a CMOS image sensor. In addition, a Bayer array primary color filter is provided on the light receiving surface of the image sensor provided in the imaging unit 102 to capture a color image. The digital image signal generated by the imaging unit 102 is stored in the DDR SDRAM 116 via the bus 112.

  In the present embodiment, when generating the digital image signal, the imaging unit 102 generates reduced image data reduced to a predetermined size in addition to the image signal that has not been reduced, and the bus 112 is also used for reduced image data. And stored in the DDR SDRAM 116. For example, the imaging unit 102 performs pixel addition, generates reduced image data in which the horizontal direction and the vertical direction are reduced to ¼, and stores the reduced image data in the DDR SDRAM 116. This reduced image data is used for moire determination processing in the interpolation processing unit 106 described later.

  The interpolation processing unit 106 performs development processing by performing interpolation processing, for example, CFA (Color Filter Array) interpolation processing, on the digital image signal generated by the imaging unit 102 and stored in the DDR SDRAM 116. Image data obtained by performing development processing by the interpolation processing unit 106 is stored in the DDR SDRAM 116 via the bus 112.

  In this embodiment, the interpolation processing unit 106 develops after reducing the digital image signal generated by the imaging unit 102 and stored in the DDR SDRAM 116 to a predetermined size before performing development processing on the digital image signal. Thus, the moire determination process is performed on the reduced image data generated. The color moire phenomenon is not reduced even when the image is reduced, and when the image is reduced, the information is condensed rather than the size of the original image, so that there is an aspect in which the change becomes more conspicuous and conspicuous.

  When a thin line image in which color moiré occurs is picked up by an apparatus including an image sensor in which pixels are arranged in a Bayer array, there is no correlation between red, blue, and green, so that each R, Gb, Gr, B signal Even when added to (Binning), the phenomenon of color moire does not disappear, and the occurrence position of color moire when viewed in the entire image is the same position in the original image and the reduced image.

  In the present embodiment, the moire determination process is performed on the reduced image data in the interpolation processing unit 106 using such an aspect. By performing the moiré determination process on the reduced image data, the imaging apparatus 100 according to an embodiment of the present invention can reduce the vertical direction and the horizontal direction to ¼ even if the image has the same memory amount. It is possible to determine the occurrence of color moire by observing double space.

  The codec unit 108 performs encoding processing on the image data generated by the interpolation processing unit 106 and decoding processing of the encoded image data. Encoding processing and decoding processing are collectively referred to as codec processing. Image data subjected to codec processing by the codec unit 108 is stored in the DDR SDRAM 116 via the bus 112.

  The communication unit 110 executes processing for transmitting information to an external device and processing for receiving information from an external device. For example, the communication unit 110 transmits the image data encoded by the codec unit 108 to an external device.

  The control unit 114 is configured by a CPU (Central Processing Unit), for example, and controls the operation of each unit of the imaging apparatus 100.

  Although not shown in FIG. 1, the imaging device 100 includes a shutter button for a user to instruct the imaging unit 102 to capture an image, an operation button for receiving a user operation on the imaging device 100, and an interpolation processing unit 106. A recording medium for storing the generated image data may be provided. The recording medium for storing the image data generated by the interpolation processing unit 106 may be removable from the imaging apparatus 100.

  The functional configuration example of the imaging apparatus 100 according to the embodiment of the present invention has been described above with reference to FIG. Subsequently, a detailed functional configuration example of the interpolation processing unit 106 included in the imaging apparatus 100 according to an embodiment of the present invention will be described.

  FIG. 8 is an explanatory diagram illustrating a detailed functional configuration example of the interpolation processing unit 106 included in the imaging apparatus 100 according to an embodiment of the present invention. Hereinafter, a detailed functional configuration example of the interpolation processing unit 106 included in the imaging apparatus 100 according to an embodiment of the present invention will be described with reference to FIG.

  As illustrated in FIG. 8, the interpolation processing unit 106 includes an RDMA (Read Direct Memory Access) controller 121, a buffer memory 122, a memory control unit 123, an image generation unit 124, a moire determination unit 125, and a switch unit. 126 and a WDMA (Write Direct Memory Access) controller 127.

  The RDMA controller 121 reads the digital image signal generated by the imaging unit 102 and reduced image data generated by developing the digital image signal after reducing the digital image signal to a predetermined size from the DDR SDRAM 116 via the bus 112. When the RDMA controller 121 reads the digital image signal or reduced image data generated by the imaging unit 102 from the DDR SDRAM 116 via the bus 112, the RDMA controller 121 stores the read digital image signal or reduced image data in the buffer memory 122.

  In the present embodiment, the interpolation processing unit 106 has a mode for developing the digital image signal generated by the imaging unit 102 (referred to as mode A for convenience) and the digital image signal generated by the imaging unit 102 with a predetermined size. The operation is performed by switching between two operation modes: a mode in which color moire determination processing is performed on reduced image data obtained by development after being reduced to (referred to as mode B for convenience).

  When the interpolation processing unit 106 operates in mode A, the RDMA controller 121 reads the digital image signal generated by the imaging unit 102 from the DDR SDRAM 116 via the bus 112.

  On the other hand, when the interpolation processing unit 106 operates in the mode B, the RDMA controller 121 buses reduced image data obtained by developing after reducing the digital image signal generated by the imaging unit 102 to a predetermined size from the DDR SDRAM 116. Read via 112.

  The buffer memory 122 is configured by a memory such as SRAM (Static Random Access Memory), for example. The buffer memory 122 stores digital image signals and reduced image data sent from the RDMA controller 121. Digital image signals and reduced image data stored in the buffer memory 122 are sequentially read out by the memory control unit 123.

  In the present embodiment, the buffer memory 122 functions as a line memory when storing a digital image signal. The buffer memory 122 functions as a memory in a state where each line of the line memory is further divided into a plurality of blocks when storing the reduced image data.

  When the interpolation processing unit 106 operates in mode A, the buffer memory 122 stores the digital image signal sent from the RDMA controller 121. On the other hand, when the interpolation processing unit 106 operates in mode B, the buffer memory 122 stores the reduced image data sent from the RDMA controller 121.

  The memory control unit 123 controls reading of data stored in the buffer memory 122, that is, digital image signals and reduced image data. In the present embodiment, when a digital image signal is stored in the buffer memory 122, the memory control unit 123 reads the digital image signal from the buffer memory 122 based on the address information instructed from the image generation unit 124. When reduced image data is stored in the buffer memory 122, the memory control unit 123 reads a digital image signal from the buffer memory 122 based on the address information instructed from the moire determination unit 125.

  When the interpolation processing unit 106 operates in mode A, the memory control unit 123 reads data stored in the buffer memory 122 based on an instruction from the image generation unit 124. On the other hand, when the interpolation processing unit 106 operates in mode B, the memory control unit 123 reads out data stored in the buffer memory 122 based on an instruction from the moire determination unit 125.

  The image generation unit 124 performs development processing (processing for generating image data) by performing inter-pixel interpolation processing on the digital image signal stored in the buffer memory 122 and read from the memory control unit 123. In the present embodiment, the image generation unit 124 performs development processing by performing CFA interpolation processing on the digital image signal.

  The image generation unit 124 performs interpolation processing while removing color moire based on the result of color moire determination processing for determining a region where color moire is generated by a moire determination unit 125 described later. When removing the color moire, the image generation unit 124 performs an interpolation process while connecting the places where the color moire occurs, for example, with white. The color moire removal method is not limited to a specific method, and the image generation unit 124 can effectively remove the color moire based on the result of the color moire determination processing by the moire determination unit 125. Become.

  When the image generation unit 124 instructs the memory control unit 123 to read out the digital image signal from the buffer memory 122, the image generation unit 124 instructs the address information to read out and output to the image generation unit 124 in units of a plurality of lines. The image generation unit 124 generates image data by repeating interpolation processing for digital image signals read in units of a plurality of lines until one image is completed. The image generation unit 124 sends the image data generated by the development process to the WDMA controller 127.

  The moiré determination unit 125 performs color moiré determination processing for determining an area where color moiré is generated on the reduced image data stored in the buffer memory 122 and read from the memory control unit 123. Since the moire determination unit 125 performs color moire determination processing on the entire reduced image data, when the memory control unit 123 is instructed to read the reduced image data from the buffer memory 122, the entire reduced image data is processed. The address information is instructed to be read and output to the moire determination unit 125.

  The moire determination unit 125 subdivides the entire reduced image data sent from the memory control unit 123 into the buffer memory 122 by subdividing a color moire, that is, a place where a pattern in which two specific colors appear repeatedly exists. Is determined for each area of the reduced image data stored in.

  The moire determination unit 125 determines whether or not color moiré occurs for each area of the reduced image data. In this embodiment, when determining whether or not color moiré occurs, the moiré determination unit 125 is likely to have color moiré. May be determined in a plurality of stages, for example, 10 stages from 1 to 10. In this case, the higher the number, the higher the probability of color moire.

  The moire determination unit 125 sends the result of the color moire determination process for each area of the reduced image data to the WDMA controller 127 in association with the information for identifying each area and the result of the color moire determination process.

  The switch unit 126 connects the image generation unit 124 and the WDMA controller 127 when the interpolation processing unit 106 operates in mode A, and the moire determination unit 125 and WDMA when the interpolation processing unit 106 operates in mode A. A switching operation for connecting the controller 127 is performed.

  The WDMA controller 127 stores the image data generated by the development processing of the image generation unit 124 and the result of the color moire determination processing by the moire determination unit 125 in the DDR SDRAM 116 through the bus 112.

  When the interpolation processing unit 106 operates in mode A, the WDMA controller 127 stores the image data generated by the development processing of the image generation unit 124 in the DDR SDRAM 116.

  On the other hand, when the interpolation processing unit 106 operates in mode B, the WDMA controller 127 stores the result of the color moire determination process by the moire determination unit 125 in the DDR SDRAM 116.

  The interpolation processing unit 106 included in the imaging apparatus 100 according to the embodiment of the present invention has the configuration illustrated in FIG. 8, thereby determining the occurrence of color moire for each area of the reduced image data. Based on the determination result of occurrence of color moire, development processing can be performed while removing color moire, and image data from which color moire has been removed can be generated.

  The detailed functional configuration example of the interpolation processing unit 106 included in the imaging apparatus 100 according to the embodiment of the present invention has been described above with reference to FIG. Subsequently, an operation example of the imaging apparatus 100 according to an embodiment of the present invention will be described.

[2.2. Example of operation]
FIG. 9 is a flowchart illustrating an operation example of the imaging apparatus 100 according to an embodiment of the present invention. FIG. 9 shows that color moire determination processing is performed on the reduced image data obtained by developing after reducing the digital image signal, and reduction is performed while removing color moire based on the result of the color moire determination processing. It is an operation example of the imaging apparatus 100 when performing development processing on a digital image signal that is the basis of image data. Hereinafter, an operation example of the imaging apparatus 100 according to an embodiment of the present invention will be described with reference to FIG.

  The imaging apparatus 100 first performs color moire determination processing on reduced image data obtained by developing after reducing a digital image signal to be developed, and determines an area where color moire has occurred. The imaging apparatus 100 reads the reduced image data stored in the DDR SDRAM 116 (step S101), divides the read reduced image data into a plurality of rectangular areas having the same size, and color moiré occurs in each area. It is determined whether or not (step S102). The read process in step S101 is executed by, for example, the RDMA controller 121, and the determination process in step S102 is executed by, for example, the moire determination unit 125.

  The image capturing apparatus 100 stores the color moire, that is, a place where a pattern in which specific two colors appear repeatedly exists in the entire reduced image data output from the DDR SDRAM 116 and stores it in the buffer memory 122. The determination is made for each area of the reduced image data. The image capturing apparatus 100 determines whether or not color moiré occurs for each area of the reduced image data output from the DDR SDRAM 116. In this embodiment, when determining whether or not color moiré occurs, the image capturing apparatus 100 performs color moiré. The certainty may be determined in a plurality of stages, for example, 10 stages from 1 to 10. In this case, the higher the number, the higher the probability of color moire.

  When determining the color moire for a certain area, the imaging apparatus 100 may determine whether the color moire is based on a comparison with another area, particularly an adjacent area. For example, as a result of observing a certain area of the reduced image data, the imaging apparatus 100 finds a repeated pattern of green and magenta. If the repeated pattern of green and magenta is not found in the adjacent area, You may determine with the possibility of a color moire having a high repetition pattern of green and magenta.

  When the imaging apparatus 100 determines whether or not color moiré is generated for each area in the reduced image data, the information identifying the area is associated with the determination result of the area and is recorded in the DDR SDRAM 116 (step S103). ). The recording process in step S103 is executed by the WDMA controller 127, for example.

  The information for identifying the area is, for example, the value in the upper left area of the reduced image data is set to 0, the value increases in the right direction, and when reaching the rightmost area, it moves down by one step and moves from the left end to the right direction. The value increases again, and this may be repeated until the lower right region is reached.

  Subsequently, the imaging apparatus 100 reads the digital image signal that is the basis of the reduced image data subjected to the color moire determination process and the result of the color moire determination process from the DDR SDRAM 116 (step S104). The read processing in step S104 is executed by, for example, the RDMA controller 121.

  Then, the imaging apparatus 100 executes development processing for generating image data by interpolating between pixels while removing color moire based on the result of the color moire determination processing on the digital image signal read from the DDR SDRAM 116 ( Step S105). For example, the image generation unit 124 executes the processing in step S105.

  For example, when it is determined that a color moire has occurred in the upper left area of the reduced image data, the imaging apparatus 100 removes the color moire from the area corresponding to the area in the digital image signal. Interpolate between. The imaging apparatus 100 removes the color moire by performing interpolation processing while connecting the places where the color moire is generated with white, for example, but the method for removing the color moire is not limited to a specific method.

  The imaging apparatus 100 according to an embodiment of the present invention determines the occurrence of color moire for each area of the reduced image data by executing a series of operations as shown in FIG. Based on the determination result, development processing is performed while removing color moire, and image data from which color moire has been removed can be generated.

<3. Summary>
As described above, according to an embodiment of the present invention, the occurrence of color moire is determined for each area of the reduced image data, and development is performed while removing color moire based on the determination result of occurrence of color moire. An imaging apparatus 100 that can perform processing and generate image data from which color moire has been removed is provided.

  The imaging apparatus 100 according to an embodiment of the present invention determines the occurrence of color moiré for each area of the reduced image data, so that even when the memory capacity is small, the occurrence of color moiré in hardware. Can be determined accurately.

  Each step in the processing executed by the apparatus in the present specification does not necessarily have to be processed in time series in the order described as a flow chart or a sequence diagram. For example, each step in the processing executed by each device may be processed in an order different from the order described as the flowchart, or may be processed in parallel.

  In addition, functions (e.g., a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) incorporated in the device in this specification have functions equivalent to the configuration of each device described above. ) Can be created. A storage medium storing the computer program can also be provided. Moreover, a series of processes can also be realized by hardware by configuring each functional block shown in the functional block diagram with hardware.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

100: Imaging device 102: Imaging unit 106: Interpolation processing unit 108: Codec unit 110: Communication unit 112: Bus 114: Control unit 116: SDRAM
121: RDMA controller 122: buffer memory 123: memory control unit 124: image generation unit 125: moire determination unit 126: switch unit 127: WDMA controller

Claims (20)

An image signal output from a sensor array in which a plurality of light receiving elements that receive a light beam via an optical system and perform photoelectric conversion are arranged in a two-dimensional Bayer array is acquired, and developed after reduction of the image signal to reduce a reduced image An image reduction unit for generating data;
A moire determination unit that determines a region where moire is generated from the reduced image data generated by the image reduction unit;
A moire removing unit that removes moire from the original image data obtained by developing the image signal using a determination result of a region where moire is generated by the moire determining unit;
An image processing apparatus comprising:   2. The moiré determination unit determines a place where a difference in the repetitive pattern between the first color and the second color in the reduced image data is generated as an area where moiré is generated. An image processing apparatus according to 1.   The moiré determination unit divides the reduced image data into predetermined areas, and if there is a place where a repeated pattern difference between the first color and the second color occurs, moiré is generated in the area. The image processing apparatus according to claim 2, wherein the image processing apparatus is determined to be a region that is in a closed state.   The moire determination unit compares a certain area with another area, and if a difference in the repetitive pattern between the first color and the second color appears from the other area, moire occurs in the area. The image processing apparatus according to claim 3, wherein the image processing apparatus is determined to be a region in which the image processing is performed.   The first color is a color having the largest number of pixels in the Bayer array, and the second color is a color generated from a color other than the first color in the Bayer array. The image processing apparatus according to claim 2.   The image processing apparatus according to claim 5, wherein the first color is green and the second color is magenta. A storage unit for storing the original image data and the reduced image data;
A storage control unit that controls reading and writing of the original image data and the reduced image data with respect to the storage unit;
The image processing apparatus according to claim 1, further comprising: An image signal output from a sensor array in which a plurality of light receiving elements that receive a light beam via an optical system and perform photoelectric conversion are arranged in a two-dimensional Bayer array is acquired, and developed after reduction of the image signal to reduce a reduced image An image reduction step for generating data;
A moire determination step for determining a region where moire is generated from the reduced image data generated in the image reduction step;
A moire removal step of removing moire from the original image data obtained by developing the image signal using the determination result of the area where moire occurs in the moire determination step;
An image processing method comprising:   9. The moiré determination step determines a place where a difference in the repetitive pattern between the first color and the second color in the reduced image data is generated as an area where moiré is generated. An image processing method described in 1.   The moire determination step divides the reduced image data into predetermined areas, and if there is a place where a repeated pattern difference between the first color and the second color occurs, moire occurs in the area. The image processing method according to claim 9, wherein the image processing area is determined to be a region that is present.   In the moire determination step, when a certain area is compared with another area and a difference in the repetitive pattern between the first color and the second color appears from the other area, moire is generated in the area. The image processing method according to claim 10, wherein the image processing area is determined to be an area.   The first color is a color having the largest number of pixels in the Bayer array, and the second color is a color generated from a color other than the first color in the Bayer array. The image processing method according to any one of claims 9 to 11.   The image processing method according to claim 12, wherein the first color is green and the second color is magenta. An imaging unit including a sensor array in which a plurality of light receiving elements that receive and photoelectrically convert a light beam through an optical system are arranged in a two-dimensional Bayer array;
An image reduction unit that develops after reduction of the image signal output from the imaging unit and generates reduced image data;
A moire determination unit that determines a region where moire is generated from the reduced image data generated by the image reduction unit;
A moire removing unit that removes moire from the original image data obtained by developing the image signal using a determination result of a region where moire is generated by the moire determining unit;
An imaging apparatus comprising:   15. The moiré determination unit determines a place where a difference in the repeated pattern between the first color and the second color in the reduced image data is generated as an area where moiré is generated. The imaging device described in 1.   The moiré determination unit divides the reduced image data into predetermined areas, and if there is a place where a repeated pattern difference between the first color and the second color occurs, moiré is generated in the area. The image pickup apparatus according to claim 15, wherein the image pickup apparatus is determined to be a region in which the image pickup is performed.   The moire determination unit compares a certain area with another area, and if a difference in the repetitive pattern between the first color and the second color appears from the other area, moire occurs in the area. The image pickup apparatus according to claim 16, wherein the image pickup apparatus is determined to be a region in which the image pickup is performed.   The first color is a color having the largest number of pixels in the Bayer array, and the second color is a color generated from a color other than the first color in the Bayer array. The imaging device according to any one of claims 15 to 17.   The imaging apparatus according to claim 18, wherein the first color is green and the second color is magenta. A storage unit for storing the original image data and the reduced image data;
A storage control unit that controls reading and writing of the original image data and the reduced image data with respect to the storage unit;
The imaging apparatus according to claim 14, further comprising: