JP2024087384A - Image Processing Module - Google Patents

Abstract

When performing filter processing, the processing efficiency of the filter processing is improved without wasting extra time required for processing pixels in an overlap portion.
[Solution] In an image processing module comprising a delay buffer that stores pixel values of at least H×2N+2M+1 pixels from among the pixels of an input image, a filter input pixel selection means that selects (2M+1)×(2N+1) pixel values, and a filter calculation means that performs the filter processing using the pixel values selected by the filter input pixel selection means and outputs the output pixel value corresponding to a pixel of interest, the filter input pixel selection means complements pixel values of pixels outside the input image and inputs them to the filter processing means, and after pixel values of H×2N+2M+1 pixels are stored in the delay buffer, the filter input pixel selection means outputs the output pixel value one pixel at a time for each pixel of the input image that is input.
[Selection diagram] Figure 9

Description

This disclosure relates to an image processing module that performs filtering of an input image using hardware.

In image processing, filtering refers to the process of updating the pixel value of a pixel of interest by referring to the pixel of interest and its surrounding pixels, with the pixel being the center. When filtering image signals, image data has a finite area, and there have traditionally been various issues with processing edge points such as the left and right edges of an image. As shown in Figure 1, when the pixel of interest is near the top, bottom, left, or right edge of the input image, some of the input pixels required for filtering are outside the input image, as shown by the shaded areas. As a result, there is a lack of data required for filtering, and correct processing cannot be performed. Pixels that are outside the input image, as shown by the shaded areas, among the pixels within the range of filtering, are hereafter referred to as "overlap".

JP 2005-348245 A

Patent Document 1 discloses a method of performing filter processing by interpolating the pixels of the overlapping portion. However, this method requires time to input the pixels of the complemented overlapping portion, which reduces the processing efficiency of the filter processing.

One embodiment of the present disclosure is an image processing module that inputs an input image of width H pixels one pixel at a time in raster order, inputs pixel values of width (2M+1) pixels x height (2N+1) pixels (H, M, and N are natural numbers), performs filter processing using a central pixel as a pixel of interest, and outputs output pixel values one pixel at a time, comprising a delay buffer that stores pixel values of at least H x 2N + 2M + 1 pixels from among the pixels of the input image, a filter input pixel selection means that selects (2M + 1) x (2N + 1) pixel values, and performs the filter processing using the pixel values selected by the filter input pixel selection means, and selects a pixel value for the pixel of interest. and a filter calculation means for outputting the output pixel value corresponding to the pixel value. When the pixel of interest is within M pixels from the left or right end of the input image, the filter input pixel selection means selects from the delay buffer the pixel values of pixels within the input image from among a group of (2M+1) x (2N+1) pixels input to the filter calculation means, complements the pixel values of pixels outside the input image, and inputs them to the filter processing means. After pixel values of H x 2N + 2M + 1 pixels are stored in the delay buffer, the output pixel value is output one pixel at a time for each pixel input of the input image.

According to the present disclosure, when performing a filter process that complements pixel values of the overlapping portion, it is possible to improve the processing efficiency of the filter process without wasting extra time on processing the pixels of the overlapping portion.

FIG. 13 is a diagram showing a “margin” in filter processing for an input image. FIG. 1 is a diagram illustrating a configuration of a general filter processing module. FIG. 1 is a diagram showing the relationship between the sizes of an input image and an output image in general filter processing. 13A and 13B are diagrams illustrating inputs and timings of processing execution in filtering processing when pixels in an overlap portion are not complemented. FIG. 13 is a diagram showing the configuration of a filter processing module to which an end point extension means is applied. 11A and 11B are diagrams illustrating a method for complementing pixel values of an overlapping portion. 13A and 13B are diagrams illustrating inputs and timings of execution of filtering processing when pixel values of an overlapping portion are complemented. 1 is a block diagram showing a configuration of a copying machine incorporating an image processing module according to an embodiment of the present disclosure. FIG. 2 is a diagram illustrating a configuration of an image processing module according to the present embodiment. 11A and 11B are diagrams illustrating timings of input and execution of filter processing in the present embodiment. FIG. 13 is a diagram showing the relationship of "margins" when an input image is divided into a plurality of bands and filtered;

Below, an embodiment of the present disclosure will be described in detail with reference to the drawings. First, an overview of typical input image filtering will be provided.

Figure 2 shows the configuration of a typical filter processing module. Filter processing module 30 is composed of sub-modules: delay buffer 301, filter input pixel selection unit 302, and filter calculation unit 303. We will explain an example in which a total of (2M+1) x (2M+1) pixels (M is a natural number), consisting of 2M+1 pixels in the line direction (X-axis direction) and 2M+1 pixels in the column direction (Y-axis direction) with the pixel of interest at the center, are referenced as the target range for filter processing.

The delay buffer 301 is a submodule that temporarily stores each pixel value input to the filter processing module 30 in raster order, and is composed of a flip-flop circuit, a register, a memory, or the like. For example, image data of the input image shown in FIG. 3(a) is input to the delay buffer 301.

The filter input pixel selection unit 302 selects and reads out the pixel values of the (2M+1) pixels wide by (2M+1) pixels high that are to be subjected to the filter process from among the multiple pixels stored in the delay buffer 301. The filter input pixel selection unit 302 inputs the read-out pixel values all at once to the downstream filter calculation unit 303.

The filter calculation unit 303 receives as input each pixel value selected by the filter input pixel selection unit 302, performs a filter processing calculation with the center pixel as the pixel of interest, and outputs the results pixel by pixel. However, the time required for the filter processing calculation for one pixel is set to be shorter than the time from when a pixel is input to the filter processing module 30 until the next pixel is input.

As shown in FIG. 3(b), the width of the output image output from the filter processing module 30 is H pixels. Because the target range of the filter processing is (2M+1)×(2M+1) pixels, the width of the input image required is (H+2M) pixels. Therefore, image data with a width of (H+2M) pixels is input to the filter processing module 30 in raster order. On the other hand, for the input image shown in FIG. 3(a), the output image is output with the gray portion shown in FIG. 3(b) (M pixels above, below, left and right) corresponding to the overlap portion removed.

Figure 4 shows the timing of input and processing execution of filter processing when the pixels in the overlapping portion are not complemented. The input image is input to the filter processing module 30 in raster order, one pixel at a time, from the top left, as shown by the thick arrow in Figure 4(a). Therefore, filter processing cannot be executed until the (2M+1) x (2M+1) pixels to be the subject of filter processing of the first pixel of interest are complete. Specifically, if the XY coordinates of the top left pixel of the input image are (0,0), filter processing cannot be executed until the pixel at coordinates (2M,2M) is input, as shown in Figure 4(b). At this time, all input pixels from the input pixel at coordinates (0,0) to the coordinates (2M,2M) (pixels in the shaded area in Figure 4(b)) must be stored in the delay buffer 301. In other words, the delay buffer 301 needs a capacity to store (2M x (H + 2M) + 2M + 1) pixel values.

As shown in FIG. 4(c), the pixel of interest is shifted by one pixel at a time in raster order to perform filter processing. At this time, the filter input pixel selection unit 302 similarly selects (2M+1)×(2M+1) pixels to be filtered from the group of pixels stored in the delay buffer 301. As shown in FIG. 4(d), the filter processing result of one pixel is output each time a pixel is input until the pixel at the coordinates (H+2M-1, 2M) on the right edge of the input image is input. The delay buffer 301 erases the oldest pixel each time and stores the newly input pixel value. Therefore, the delay buffer 301 must always store (2M×(H+2M)+2M+1) pixel values, as shown in the shaded areas in FIG. 4(c) and (d).

Furthermore, when pixels of the next line of the input image are input, filter processing cannot be performed until the 2M+1th pixel of that line is input, as shown in FIG. 4(e). After the 2M+1th pixel is input, the filter processing result of one pixel is output for each pixel input, as shown in FIG. 4(f).

The X-coordinate values of the input pixels to each column of the filter calculation unit 303 for each of the cases in Figures 4(a) to (f) are shown in Figures 4(b1) to (d1) and (f1).

Thus, the width of one line of the input image is (H+2M) pixels, while the width of the output image is H pixels. Similarly, the width of a column of the output image output from the filter processing module 30 (total number of lines in the input image) is also (H+2M) pixels, since the range of pixels subject to filter processing is (2M+1) x (2M+1) pixels. Therefore, ignoring the sub-scanning direction, the output rate is H/(H+2M) times the input rate.

Next, we will explain an example of a method for performing filter processing by complementing pixels in the overlap area outside the input image when an input image requires an overlap area, as described above.

Figure 5 shows the configuration of a filter processing module to which an end point extension means is applied. The filter processing module 30a comprises a delay buffer 301, a filter input pixel selection unit 302, and a filter calculation unit 303. In addition, a left end point extension means 305 that extends image data to the left of the left end point of the input image, and a right end point extension means 304 that extends image data to the right of the right end point of the input image are connected in tandem to the input side of the delay buffer 301. Furthermore, an end point detection means 306 that instructs the execution of an end point extension operation is connected to the right end point extension means 304 and the left end point extension means 305.

When an input image with a width of H pixels is input one pixel at a time in raster order to the filter processing module 30a, the left and right endpoints of each line are detected by the endpoint detection means 306. When the left endpoint is detected, the left endpoint is extended to the left by M pixels by the left endpoint extension means 305, and when the right endpoint is detected, the right endpoint is extended to the right by M pixels by the right endpoint extension means 304, and these are stored in the delay buffer 301. Therefore, when the width of the input image is H pixels, an image with a width of H+2M pixels is stored in the delay buffer 301.

With reference to FIG. 6, a method for complementing pixel values of the overlapping portion will be described. Here, the range of pixels to be filtered is 2M+1=5. This is an example in which the right end point extension means 304 complements pixel values of the overlapping portion when performing filter processing on 5×5 pixels centered on the pixel of interest m. In the first example, as shown in FIG. 6(a), when the pixel of interest m is located at the right end of the input image, the right two columns of the 5×5 pixels to be filtered, that is, the 5×2 pixels shown in gray, become the pixels of the overlapping portion. Therefore, the right end point extension means 304 complements the pixel values of the two columns of pixels of the overlapping portion as the right end pixels (c, h, m, r, w) of the same line of the input image.

Similarly, if the pixel of interest m is located one column inside from the right edge of the input image, the one column to the right becomes a pixel of the overlap portion. Therefore, as shown in FIG. 6(b), the right edge point extension means 304 complements the pixel values of the pixels of one column of the overlap portion as the pixel (d, i, n, s, x) at the right edge of the same line of the input image.

When complementing pixel values of the overlap portion, the right end point extension means 304 can also complement using pixel values of pixels in a column at the end of the input image and in a mirror image position, as shown in FIG. 6(c) as a second example. That is, the right end point extension means 304 extends the first column of the overlap portion with the right end pixel (c, h, m, r, w), and complements the second column of the overlap portion with the second pixel from the right end (b, g, l, q, v). When the filter process is for enhancing edges, the second example shown in FIG. 6(c) can produce better results than the first example shown in FIGS. 6(a) and (b).

If the pixel of interest is at the left end of the input image, the left end point extension means 305 extends the end point in the same way as the right end point, and the pixels in the overlapping part are complemented. The input image input to the filter processing module 30a has the pixels in the overlapping part complemented by the left end point extension means 305 and the right end point extension means 304, and all columns are processed in raster order. As shown in FIG. 6(d), the width of one line of the input image is increased by the amount by which the left and right end points are extended, and input to the delay buffer 301. In this way, even if the pixel of interest is at the edge of the input image, filter processing can be performed within a range of 5 x 5 pixels by complementing the pixel value of the overlapping part.

The processing rate in the filter processing module 30a will be described. Here, the filter processing will be described by taking as an example a filter processing that references (2M+1)×(2M+1) pixels (M is a natural number) with the pixel of interest at the center. When an input image with a width of H pixels is input to the filter processing module 30a one pixel at a time in raster order, the end points are extended to the left and right by M pixels by the right end point extension means 304 and the left end point extension means 305. According to the first example of complementing the pixel values of the overlapping portion described above, for each of the y-th line (y=0, 1, 2, ...) of the input image, the pixel value of the pixel at the left end coordinate (0, y) of the line is input M times. Next, H pixels of one line of the input image are input, and the pixel value of the pixel at the right end coordinate (H-1, y) of the line is input M times. Similar processing is repeated for subsequent lines, so that H+2M pixels with the end points extended to the left and right for each line are input to the delay buffer 301.

Figure 7 shows the timing of input and processing execution of the filter processing when pixel values of the overlapping portion are interpolated. In the filter processing module 30a, as shown in Figure 7(a), an image with a width of H+2M pixels, in which M columns of pixels are added to the left and right of the input image, is processed as a new input image. Therefore, the timing of input and processing execution shown in Figure 7(b) is the same as the timing of input and processing execution shown in Figure 4(b). Similarly, the timing of input and processing execution in Figures 7(c) to (f) is the same as the timing of input and processing execution in Figures 4(c) to (f). The X coordinate values of the input pixels to each column of the filter calculation unit 303 shown in Figures 7(b1) to (d1) and (f1) are also the same as those in Figures 4(b1) to (d1) and (f1). Therefore, if the sub-scanning direction is ignored, the output rate is H/(H+2M) times the input rate.

As described above, when the processing shown in FIG. 4 is performed using the filter processing module 30 shown in FIG. 2, the output rate is H/(H+2M) times the input rate, which is slower than the input rate. The reason for this slowness is that during filter processing, pixels that are not included in the pixels of the output image must also be input, and it takes time to input those pixels. Furthermore, the ratio of the output rate to the input rate decreases as the number of main scanning pixels H of the input image decreases. On the other hand, when the processing shown in FIG. 7 is performed using the filter processing module 30a shown in FIG. 5, the output rate is also H/(H+2M) times the input rate, which is slower than the input rate.

Therefore, in this embodiment, the filter processing result of the pixel of interest including the right-end overlap portion is output, and the pixel value of the left-end overlap portion is complemented. This makes it possible to improve the processing efficiency of the filter processing without wasting extra time on processing the pixels of the overlap portion.

First Embodiment
8 shows the configuration of a copying machine incorporating an image processing module according to an embodiment of the present disclosure. In this embodiment, a digital copying machine is taken as an example, but the technology of the present disclosure can be applied to various other types of image processing devices that process two-dimensional images, such as video display devices. A typical digital copying machine includes a scanner unit, a printer unit, a controller ASIC, a main memory, a communication interface, and a mechanism unit. Each of these units and their internal structure will be described below.

The controller chip 11000 controls the entire copier. It is often implemented in the form of an ASIC (Application Specific Integrated Circuit) that integrates many internal modules described below into one chip. The scanner unit 11001 optically scans an original to obtain digital image data. It is controlled by a scanner control unit 119 (described below) in the controller chip 11000. The scanner unit 11001 has a lighting unit and a line sensor (not shown) inside, and transfers the read result to the controller chip 11000 as an analog signal in pixel units. The main memory 11002 temporarily stores image data and various control data. The main memory 11002 is generally composed of a DRAM (Dynamic Random Access Memory). The communication interface 11003 communicates between the copier and the outside. Specifically, a wireless LAN, a USB (Universal Serial Bus), etc. are applied. A typical digital copier has multiple types of communication interfaces and exchanges control commands and image data with the outside.

The print carriage head 11004 of the printer unit forms an image on the surface of the paper, which is a recording material, by means of ejecting ink or the like, based on the image data output from the controller chip 11000. The print carriage head 11004 is scanned in the main scanning direction by the carriage head transport unit 110056 (described below) over the paper, which is transported in the sub-scanning direction by the paper transport unit 110055 (described below), and ink is ejected at the same time.

The mechanism unit 11005 controls the mechanical operations within the copier using output signals from the controller chip 11000, detects the status within the copier, and notifies the controller chip 11000 of the information. The mechanism unit 11005 includes the following submodules: The document transport unit (auto document feeder) 110051 transports the document to be scanned. The scanner unit transport unit 110052 moves the scanner unit 11001 when the document is read by the flatbed scanner. The document position detection unit 110053 detects the presence of the document and the end of the document when the document is read by the auto document feeder 110051, and sends this to the controller chip 11000.

The operation panel 110054 is a panel that allows the user to manually operate the copier.

The paper transport unit 110055 transports the paper to be printed by the print carriage head 11004. The carriage head transport unit 110056 operates the print carriage head 11004 in a direction perpendicular to the paper transport direction. The paper position detection unit 110057 detects the position of the paper being transported by the paper transport unit 110055 and sends this information to the controller chip 11000.

Next, the internal configuration of the controller chip 11000 will be described. The bus 111 connects each processing unit within the controller chip 11000.

The scanner image processing unit 112 performs various image processing on the digital image data obtained by the scanner unit 11001. The scanner image processing unit 112 includes the following three sub-modules. The shading processing module 1121 performs shading processing on the image data obtained by the scanner unit 11001. The filter processing module (A) 1122 performs processing such as edge emphasis on the image data after shading processing. The magnification processing module 1123 performs digital magnification processing according to the magnification setting specified by the user. These sub-modules process each pixel of the input image in raster order in a pipeline manner simultaneously within the scanner image processing unit 112.

The memory controller 113 is connected to the bus 111 and controls data exchange with the main memory 11002. The interface (I/F) 114 exchanges data between the controller chip 11000 and the communication interface 11003 and mechanism unit 11005 outside the chip.

The rendering processing unit 115 expands compressed image data that the copier receives from the outside via the communication interface 11003, and converts it into uncompressed bitmap image data by performing rendering processing of PDL (page description language) data. However, it is also possible to perform the above-mentioned rendering processing by the CPU 116 without installing this rendering processing unit as hardware.

The CPU (central control unit) 116 controls the entire device via the bus 111.

The printer image processing unit 117 performs processing such as color conversion and quantization on the bitmap image data processed by the scanner image processing unit 112 and the rendering processing unit 115, and outputs the data to the printer control unit 118. The printer image processing unit 117 includes the following three submodules. The filter processing module (B) 1171 performs a different type of filter processing on the input digital image data from the filter processing module (A) 1122. The filter processing module (B) 1171 performs filter processing on the output image data of the rendering processing unit 115, such as thinning the edges of the output image, taking into account ink bleeding and the like. The color conversion module 1172 converts image data represented by luminance (RGB) into multi-value density data corresponding to ink colors such as CMYK, using processes such as matrix calculations and 3D-LUT (lookup table). The quantization module 1173 converts the image data converted into multi-value data corresponding to ink colors by the color conversion module 1172 into quantized data that can be output by the printer unit, using known methods such as dithering and error diffusion. The processing results of the printer image processing unit 117 are output to the main memory 11002.

The printer control unit 118 reads the quantized data output from the printer image processing unit 117 from the main memory 11002 and transmits it to the print carriage head 11004. The scanner control unit 119 controls the scanner unit 11001 according to commands from the CPU 116 and the state detected by the mechanism unit 11005. The A/D conversion unit 120 A/D converts the pixel-by-pixel analog signal output from the scanner unit 11001 into digital image data and writes it to the main memory 11002.

In this embodiment, the internal data flow of the copier is as follows:
Scanner unit 11001
→ A/D conversion unit 120 → Main memory 11002
→Scanner image processing unit 112 →Main memory 11002
→Printer image processing unit 117 →Main memory 11002
→Printer control unit 118
→ Print carriage head 11004
When printing PDL data received from outside the copier,
Communication interface 11003 → main memory 11002
→Rendering processing unit 115 →Main memory 11002
→Printer image processing unit 117 →Main memory 11002
→Printer control unit 118
→ Print carriage head 11004
It becomes.

In the example of the copier described above, the filter processing operation in this embodiment can be performed by either filter processing module (A) 1122 or filter processing module (B) 1171. When copying, filter processing is performed by filter processing module (A) 1122, and when PDL printing, filter processing is performed by filter processing module (B) 1171. Note that the filter processing operation may be performed in the rendering processing unit 115. Furthermore, the filter processing of this embodiment can be applied not only to the copier of this embodiment, but also to any device that performs hardware filter processing on two-dimensional images.

Figure 9 shows the configuration of the image processing module of this embodiment. A case will be described in which the image processing module is applied to the filter processing module (A) 1122 in the above-mentioned copier. The filter processing module (B) 1171 has a similar configuration. The filter processing module (A) 1122 is composed of the sub-modules of a delay buffer 301, a filter input pixel selection unit 302, and a filter calculation unit 303.

The delay buffer 301 is a submodule that temporarily stores each pixel value input to the filter processing module (A) 1122 in raster order, and is composed of a flip-flop circuit, a register, or a memory.

The filter input pixel selection unit 302 selects and reads out the pixel values of the (2M+1) pixels wide by (2M+1) pixels high that are to be subjected to the filter process from among the multiple pixels stored in the delay buffer 301. The filter input pixel selection unit 302 inputs the read pixel values all at once to the downstream filter calculation unit 303.

The filter calculation unit 303 receives as input each pixel value selected by the filter input pixel selection unit 302, performs a filter processing calculation with the center pixel as the pixel of interest, and outputs the results pixel by pixel. However, the time required for the filter processing calculation for one pixel is set to be shorter than the time from inputting one pixel in the filter processing module (A) 1122 to inputting the next pixel.

In the first embodiment, the above-mentioned end point extension means is not used, and the filter input pixel selection unit 302 performs processing to compensate for the pixel values of the overlapping portion. Details will be described with reference to FIG. 9.

The delay buffer 301 comprises H memories 3011[0] to 3011[H-1] (H is the pixel width of the input image) that hold only the most recent 2M+1 pixels of the pixel values of the input image in FIFO format for each X coordinate value in the input image. In addition, the delay buffer 301 comprises a multiplexer 3010 that selects which of the H memories 3011 the input pixel value will be held in, based on the X coordinate value of the input image. The pixel value of a pixel whose X coordinate value in the input image is x is held in memory 3011[x] for the most recent 2M+1 pixels. In each of memories 3011[x], when there is a new input pixel beyond 2M+1, the oldest pixel is erased and the new pixel is stored. In other words, when the pixel values from line 0 to line 2M of the input image are input, the pixel values of 2M+1 pixels (coordinates (x,0) to (x,2M)) for one column are stored in memory 3011[x] for each X coordinate value.

In addition, the delay buffer 301 does not store pixel values of the input image, but instead includes a fixed value register 3012 that can set 2M+1 predetermined fixed values. The fixed value register 3012 will be described later in the third embodiment.

The filter input pixel selection unit 302 selects pixel values of (2M+1) x (2M+1) pixels to be input to the filter calculation unit from H memories 3011 or fixed value registers 3012 in the delay buffer 301. The filter input pixel selection unit 302 has (2M+1) selectors 3021 for this selection. Selector 3021[0] selects pixel values in the target range (coordinates (0,0) to (2M,0)) of the 0th line of filter processing from memory 3011[0] to memory 3011[2M] of the delay buffer 301. Similarly, selector 3021[2M] selects pixel values in the target range (coordinates (0,2M) to (2M,2M)) of the 2nd M line of filter processing from memory 3011[0] to memory 3011[2M]. In this way, all (2M+1) x (2M+1) pixel values in the target range (coordinates of the pixel of interest: (M, M)) with coordinates (0, 0) as the upper left pixel are input to the filter calculation unit 303.

The filter calculation unit 303 performs filtering on the (2M+1) pixel value groups (the total number of pixels is (2M+1) x (2M+1)) selected by the (2M+1) selectors 3021, and outputs one pixel value as the processing result corresponding to the pixel of interest. Here, the pixel value groups of the 2M+1 columns (line 0 to line 2M) in the range to be filtered are pixel values selected and read out by the selector 3021 from the 2M+1 memories 3011 of the delay buffer 301.

Since the memory 3011 is represented as H separate memories, the total capacity of the memory 3011 is an amount that can store (2M+1) pixels x the width H pixels of the input image. It is also possible to combine the H memories into one memory and have the filter input pixel selection unit 302 select (2M+1) pixel values for each column for (2M+1) columns.

Figure 10 shows the timing of input and processing execution of the filter processing in this embodiment. The input image is input to the filter processing module (A) 1122 one pixel at a time in raster order from the top left as shown by the thick arrow in Figure 10 (a). The filter input pixel selection unit 302 selects a memory 3011 corresponding to each column (X coordinate value) of the input image from the delay buffer 301, and inputs the 2M + 1 pixel values stored there to the filter calculation unit 303. In Figure 10, the target range of the filter processing is shown by a thin dashed rectangle. If the target range includes the overlap part (gray part) (Figures 10 (b), (c), (f), (g), and (h)), the input pixel value of the overlap part is replaced with the pixel value in the input image. Specifically, the pixels of the overlap part are complemented by selecting the same memory 3011 in some of the (2M + 1) selectors 3021.

The selection algorithm is explained in detail below. Here, as in the first example of the method for supplementing pixel values in the overlapping portion described above, the overlapping portion is supplemented with the same pixel value as the pixels at the left and right ends of the input image. The coordinates of the input pixel in the input image are (X, Y) (0≦X≦H-1, 0≦Y).

(Step 1) When Y<2M or X<M, i.e., in the state prior to FIG. 10(b), if the pixels in the range subject to filtering are not aligned in the sub-scanning direction: do not perform filter calculation.

(Step 2) When Y≧2M and M≦X≦2M-1, i.e., the state of FIG. 10(b) and (c): When 0≦k≦X, the selector 3021[k] of the filter input pixel selection unit 302 selects the memory 3011[0]. When X<k≦2M, the memory 3011[X-M+k] is selected and filter calculation processing is performed.

This selection means that the pixels in the left overlapping part of the input image (the gray parts in Figures 10(b) and (c)) are complemented with the pixel value of the pixel at the left end of the same line in the input image.

(Step 3) When Y≧2M and 2M≦X≦H-1, i.e., the state shown in FIG. 10(d) and (e): Selector 3021[k] (0≦k≦2M) selects memory 3011[X-2M+k] and performs filter calculation processing.

In this state, to perform filter processing, the pixel value of the oldest input pixel (X-2M, Y-2M) corresponding to the top left pixel of the range targeted for filter processing is required. In other words, the delay buffer 301 needs a capacity to store input pixels from the oldest to the newest. This capacity corresponds to the area of the shaded areas in Figures 10(d) and (e), and is (2MH+2M+1) pixels. This is because, as is clear from Figure 10(e), the pixel values of the topmost line from column 0 to column (H-2M-2) are not required.

(Step 4) When Y≧2M+1 and 0≦X≦M-1, i.e., the state of FIG. 10(f) and (g): When 0≦k≦X, selector 3021[k] (0≦k≦2M) selects memory 3011[H-2M-1]. When X<k≦2M, selects memory 3011[H-1] and performs filter calculation processing.

This selection means that the pixels in the right overlap part of the input image (the gray parts in Figures 10(f) and (g)) are complemented with the pixel value of the pixel at the right end of the same line in the input image. It is important to note that the pixel value of the most recent input pixel, specifically the pixel value of the pixel at the right end of the next line, is not used in the filter calculation. In other words, while M pixels (left overlap width) from the left end of the input image raster are being input one pixel at a time, the right overlap part is complemented. Processing time is saved by performing the processing of the left overlap part while performing the filter calculation of the complemented pixels and outputting the output pixel values one pixel at a time.

(Step 5) When Y ≥ 2M+1 and M ≤ X ≤ 2M-1, i.e., the state of FIG. 10(h): The same as the process in (Step 2) above. After this, the processes in (Step 2) to (Step 4) are repeated.

In this way, when inputting one pixel at a time from the input image to the filter processing module 1122(A) in raster order, a group of input pixels to the filter calculation unit 303 is selected so that the pixels in the overlapping portion are complemented according to the coordinate value of the input pixel. As a result, after the process of (2) above, it is possible to obtain an output pixel value of one pixel every time one pixel is input. Therefore, compared to when filter processing is performed using an end point extension means, it is possible to prevent a decrease in processing speed due to complemented pixels.

The X-coordinate values of the input pixels to each column of the filter calculation unit 303 for each of the cases in Fig. 10(b) to (h) are shown in Fig. 10(b1) to (h1). Here, "column r" refers to the pixel group whose X-coordinate in the input image is r, and is stored in memory 3011[r].

When the process transitions from (Step 4) to (Step 5) above, that is, when the coordinates of the input pixel change from X=M-1 to X=M, the input pixel group to the filter calculation unit 303 moves from the right end to the left end of the input image. At this time, the range of Y coordinate values changes from 0 to 2M to 1 to 2M+1.

In the description of this embodiment, the target range of the filter process is (2M+1) x (2M+1) pixels in both the vertical and horizontal directions, but the same processing can be performed even if the number of pixels in the vertical and horizontal directions are different. When the target range of the filter process is (2M+1) x (2M+1) pixels, the total required capacity of the memory 3011 is (H x 2M + 2M + 1) pixels, as described above. If the target range of the filter process is (2M+1) x (2N+1) pixels, the total required capacity of the memory 3011 is (H x 2N + 2M + 1) pixels.

According to the first embodiment, the filter input pixel selection unit 302 selects a group of pixels in the delay buffer 301 while complementing the pixels in the overlap portion according to the coordinates of the input pixels that are input one by one, and inputs the selected group to the filter calculation unit 303. As a result, after the processing in step 2 above, one pixel of output can be obtained for each pixel input, so that output can be performed at the same rate as the input rate. Therefore, it is possible to prevent a decrease in processing speed due to complementing pixels in the overlap portion.

Second Embodiment
In the first embodiment, the pixels at the left and right ends of the input image are extended as pixels to be complemented in the left and right overlapping parts. However, when the filter process is a process for emphasizing edges, it is known that better results can be obtained by using the second example of the method for complementing pixel values of the overlapping parts, as described above.

In this case, the processing of (step 1) and (step 3) is the same as in the first embodiment, and the processing of (step 2) and (step 4) is as follows.

(Step 2) When Y≧2M and M≦X≦2M-1, that is, the state of FIG. 10(b) and (c): When 0≦k≦X, memory 3011[M-1-k] is selected, and when X<k≦2M, memory 3011[X-M+k] is selected, and filter calculation processing is performed. At this time, the X coordinate value of the input pixel to each column of filter calculation unit 303 is shown in FIG. 10(b2) and (c2).

(Step 4) When Y≧2M+1 and 0≦X≦M-1, that is, the state of FIG. 10(f) and (g): When 0≦k≦X, memory 3011[H-M-1+k] is selected, and when X<k≦2M, memory 3011[H+M-k] is selected, and filter calculation processing is performed. At this time, the X coordinate value of the input pixel to each column of filter calculation unit 303 is shown in FIG. 10(f2) and (g2).

The processing rate is the same as in the first embodiment. In the second embodiment, when processing to emphasize edges, the image quality of the filter processing result at the edge of the input image is improved compared to the first embodiment.

Third Embodiment
In the first and second embodiments, the pixels to be complemented on the left and right of the filter process are selected from within the input image. Depending on the content of the filter process, a predetermined fixed value that does not depend on the pixels of the input image, for example, a pixel value corresponding to white or black, may be used.

In this case, the above fixed values are set in advance for all pixels in the fixed value register 3012, and steps 2 and 4 in the first embodiment are carried out as follows.

(Step 2) When Y≧2M and M≦X≦2M-1, i.e., the state of FIG. 10(b) and (c): When 0≦k≦X, select fixed value register 3012, and when X<k≦2M, select memory 3011[X-M+k] to perform filter calculation processing.

(Step 4) When Y≧2M+1 and 0≦X≦M-1, i.e., when 0≦k≦X, as in the state of FIG. 10(f) and (g), select memory 3011[H-M-1+k], and when X<k≦2M, select fixed value register 3012 to perform filter calculation processing.

The processing rate is the same as in the first embodiment. In the third embodiment, the image quality of the filter processing result at the ends of the input image is lower than in the first and second embodiments, but the circuit configuration is simpler, so filter processing can be realized at low cost.

Fourth Embodiment
11 shows the relationship of "margins" when an input image is divided into multiple bands and filtered. After dividing an input image into multiple bands, each band is input to a filter processing module one pixel at a time in vertical raster order for processing. In this case, the filter processing module can have a configuration similar to that of the above embodiment, except that the input order is changed from (horizontal) raster order to column (vertical raster) order.

One of the advantages of this vertical raster order processing method is that it significantly reduces the amount of delay buffer required to store past input pixels in processes such as filtering that require reference to pixels surrounding the pixel of interest.

As shown in Figure 11, the filter processing module adds the amount equivalent to the overlap pixels (gray areas) above and below to the height of the output band and reads it as one band from the input image. Next, the filter processing module removes the overlap areas and outputs the result. In terms of the operation of the filter processing module, this is the same as the method of compensating for pixel values in the overlap areas shown in Figure 4, but with the vertical and horizontal sides swapped. The width H of the output image in Figure 4 is the output band height in Figure 11. In this case, the output rate relative to the input rate is H/(H+2M) times, and if the width H is made small to reduce the amount of delay buffer required, the output rate will be significantly smaller than the input rate.

Therefore, the filter processing module of the first embodiment is applied to reverse the lines and columns, and the pixel values of the pixels outside the input band are complemented in the filter input pixel selection unit 302. Furthermore, by making the height of the input band the same as the output band, H lines, without including the overlap, it is possible to output at the same output rate as the input rate.

Other Embodiments
The above-described embodiment does not limit the contents of the present disclosure, and not all of the combinations of features described in the present embodiment are necessarily essential to the solution of the present disclosure. Furthermore, the relative arrangements, shapes, and the like of the components described in the present embodiment are merely examples, and are not intended to limit the contents of the present disclosure.

The disclosure of this embodiment also includes the following configuration.
Configuration 1)
An image processing module which receives an input image with a width of H pixels, pixel by pixel, in raster order, receives pixel values of a width of (2M+1) pixels by a height of (2N+1) pixels (H, M, and N are natural numbers), executes a filter process using a central pixel as a pixel of interest, and outputs output pixel values,
a delay buffer for storing pixel values of at least H×2N+2M+1 pixels among the pixels of the input image;
an input pixel selection means for selecting (2M+1)×(2N+1) pixel values;
a filter calculation means for performing the filtering process using the pixel value selected by the input pixel selection means and outputting the output pixel value corresponding to the pixel of interest,
an input pixel selection means for selecting from the delay buffer, when the pixel of interest is within M pixels from the left or right end of the input image, a pixel value of a pixel within the input image from a group of (2M+1)×(2N+1) pixels to be input to the filter calculation means, complementing pixel values of pixels outside the input image and inputting the complemented pixel values to the filter calculation means, and outputting the output pixel value one pixel at a time for each pixel of the input image input after pixel values of H×2N+2M+1 pixels have been stored in the delay buffer.

Configuration 2)
The input pixel selection means
When the coordinates of the pixel of the input image are (X, Y) (0≦X≦H−1, 0≦Y),
When the coordinates of the pixel of interest are Y≧2M and M≦X≦2M−1, the pixel values of pixels located outside the left edge of the input image are complemented;
When the coordinates of the pixel of interest are Y≧2M+1 and 0≦X≦M−1, the pixel values of pixels outside the right end of the input image are complemented.
The image processing module according to configuration 1.

Configuration 3)
An image processing module which receives an input image of a line of height H, pixel by pixel, in vertical raster order, receives pixel values of width (2N+1) pixels by height (2M+1) pixels (H, M, and N are natural numbers), executes a filter process using a central pixel as a pixel of interest, and outputs output pixel values pixel by pixel,
a delay buffer for storing pixel values of at least H×2N+2M+1 pixels among the pixels of the input image;
an input pixel selection means for selecting (2N+1)×(2M+1) pixel values;
a filter calculation means for performing the filtering process using the pixel value selected by the input pixel selection means and outputting the output pixel value corresponding to the pixel of interest,
an input pixel selection means for selecting from the delay buffer, when the pixel of interest is within M pixels from the left or right end of the input image, a pixel value of a pixel within the input image from a group of (2N+1)×(2M+1) pixels to be input to the filter calculation means, complementing pixel values of pixels outside the input image and inputting the complemented pixel values to the filter calculation means, and outputting the output pixel value one pixel at a time for each pixel of the input image input after pixel values of H×2N+2M+1 pixels have been stored in the delay buffer.

Configuration 4)
The input pixel selection means
When the coordinates of a pixel in the input image are (X, Y) (0≦X, 0≦Y≦H−1),
When the coordinates of the pixel of interest are X≧2M and M≦Y≦2M−1, the pixel values of pixels located outside the left edge of the input image are complemented;
When the coordinates of the pixel of interest are X≧2M+1 and 0≦Y≦M−1, the pixel value of a pixel outside the right end of the input image is complemented.
4. The image processing module according to claim 3.

Configuration 5)
The input pixel selection means
selecting from the delay buffer a pixel value of a pixel at the left end of the input image to complement the pixel value of a pixel located outside the left end of the input image, and selecting from the delay buffer a pixel value of a pixel at the right end of the input image to complement the pixel value of a pixel located outside the right end of the input image;
5. The image processing module according to any one of configurations 1 to 4.

Configuration 6)
The input pixel selection means
a pixel value in the input image that is located in a mirror image position with respect to the left end of the input image is selected and complemented as a pixel value of a pixel located outside the left end of the input image, and a pixel value in the input image that is located in a mirror image position with respect to the right end of the input image is selected and complemented as a pixel value of a pixel located outside the right end of the input image;
5. The image processing module according to any one of configurations 1 to 4.

Configuration 7)
The input pixel selection means
A pixel value of a pixel outside the input image is interpolated using a predetermined fixed value.
5. The image processing module according to any one of configurations 1 to 4.

Configuration 8)
An image processing module which inputs an input image with a width of H pixels one pixel at a time in raster order, inputs pixel values of a width of (2M+1) pixels by a height of (2N+1) pixels (H, M, and N are natural numbers), performs filter processing using a central pixel as a pixel of interest, and outputs output pixel values one pixel at a time, further comprising: a delay buffer which stores pixel values of at least H×2N+2M+1 pixels among the pixels of the input image; selects (2M+1)×(2N+1) pixel values from the delay buffer; performs the filter processing using the selected pixel values; and outputs the output pixel value corresponding to the pixel of interest,
When the coordinates of the pixel of the input image are (X, Y) (0≦X≦H−1, 0≦Y),
when the coordinates of the pixel of interest are Y≧2M and M≦X≦2M−1, complementing pixel values of pixels located outside the left edge of the input image;
selecting a pixel value of a pixel in the input image when the coordinates of the pixel of interest are Y≧2M and 2M≦X≦H−1;
and when the coordinates of the pixel of interest are Y≧2M+1 and 0≦X≦M−1, complementing pixel values of pixels outside the right end of the input image, and after pixel values of H×2N+2M+1 pixels have been stored in the delay buffer, outputting the output pixel value one pixel at a time each time one pixel of the input image is input.

Configuration 9)
An image processing module which inputs an input image of a height H line one pixel at a time in vertical raster order, inputs pixel values of a width of (2N+1) pixels by a height of (2M+1) pixels (H, M, and N are natural numbers), performs filter processing using a central pixel as a pixel of interest, and outputs output pixel values one pixel at a time, the image processing method comprising the steps of: selecting (2M+1)×(2N+1) pixel values from a delay buffer which stores pixel values of at least H×2N+2M+1 pixels out of the pixels of the input image; performing the filter processing using the selected pixel values; and outputting the output pixel value corresponding to the pixel of interest,
When the coordinates of a pixel in the input image are (X, Y) (0≦X, 0≦Y≦H−1),
When the coordinates of the pixel of interest are X≧2M and M≦Y≦2M−1, complementing pixel values of pixels located outside the left edge of the input image;
selecting a pixel value of a pixel in the input image when the coordinates of the pixel of interest are X≧2M and 2M≦Y≦H−1;
and when the coordinates of the pixel of interest are X≧2M+1 and 0≦Y≦M−1, complementing a pixel value of a pixel outside the right end of the input image, and after pixel values of H×2N+2M+1 pixels have been stored in the delay buffer, outputting the output pixel value one pixel at a time every time one pixel of the input image is input.

1122 Filter processing module 301 Delay buffer 302 Filter input pixel selection unit 303 Filter calculation unit 3010 Multiplexer 3011 Memory 3012 Fixed value register 3021 Selector

Claims (9)

An image processing module which receives an input image with a width of H pixels, pixel by pixel, in raster order, receives pixel values of a width of (2M+1) pixels by a height of (2N+1) pixels (H, M, and N are natural numbers), executes a filter process using a central pixel as a pixel of interest, and outputs output pixel values,
a delay buffer for storing pixel values of at least H×2N+2M+1 pixels among the pixels of the input image;
an input pixel selection means for selecting (2M+1)×(2N+1) pixel values;
a filter calculation means for performing the filtering process using the pixel value selected by the input pixel selection means and outputting the output pixel value corresponding to the pixel of interest,
an input pixel selection means for selecting from the delay buffer, when the pixel of interest is within M pixels from the left or right end of the input image, a pixel value of a pixel within the input image from a group of (2M+1)×(2N+1) pixels to be input to the filter calculation means, complementing pixel values of pixels outside the input image and inputting the complemented pixel values to the filter calculation means, and outputting the output pixel value one pixel at a time for each pixel of the input image input after pixel values of H×2N+2M+1 pixels have been stored in the delay buffer. The input pixel selection means
When the coordinates of the pixel of the input image are (X, Y) (0≦X≦H−1, 0≦Y),
When the coordinates of the pixel of interest are Y≧2M and M≦X≦2M−1, the pixel values of pixels located outside the left edge of the input image are complemented;
When the coordinates of the pixel of interest are Y≧2M+1 and 0≦X≦M−1, the pixel values of pixels outside the right end of the input image are complemented.
2. The image processing module according to claim 1. The input pixel selection means
selecting from the delay buffer a pixel value of a pixel at the left end of the input image to complement the pixel value of a pixel located outside the left end of the input image, and selecting from the delay buffer a pixel value of a pixel at the right end of the input image to complement the pixel value of a pixel located outside the right end of the input image;
3. An image processing module according to claim 1 or 2. The input pixel selection means
a pixel value in the input image that is located in a mirror image position with respect to the left end of the input image is selected and complemented as a pixel value of a pixel located outside the left end of the input image, and a pixel value in the input image that is located in a mirror image position with respect to the right end of the input image is selected and complemented as a pixel value of a pixel located outside the right end of the input image;
3. An image processing module according to claim 1 or 2. The input pixel selection means
A pixel value of a pixel outside the input image is interpolated using a predetermined fixed value.
3. An image processing module according to claim 1 or 2. An image processing module which receives an input image of a line of height H, pixel by pixel, in vertical raster order, receives pixel values of width (2N+1) pixels by height (2M+1) pixels (H, M, and N are natural numbers), executes a filter process using a central pixel as a pixel of interest, and outputs output pixel values pixel by pixel,
a delay buffer for storing pixel values of at least H×2N+2M+1 pixels among the pixels of the input image;
an input pixel selection means for selecting (2N+1)×(2M+1) pixel values;
a filter calculation means for performing the filtering process using the pixel value selected by the input pixel selection means and outputting the output pixel value corresponding to the pixel of interest,
an input pixel selection means for selecting from the delay buffer, when the pixel of interest is within M pixels from the left or right end of the input image, a pixel value of a pixel within the input image from a group of (2N+1)×(2M+1) pixels to be input to the filter calculation means, complementing pixel values of pixels outside the input image and inputting the complemented pixel values to the filter calculation means, and outputting the output pixel value one pixel at a time for each pixel of the input image input after pixel values of H×2N+2M+1 pixels have been stored in the delay buffer. The input pixel selection means
When the coordinates of a pixel in the input image are (X, Y) (0≦X, 0≦Y≦H−1),
When the coordinates of the pixel of interest are X≧2M and M≦Y≦2M−1, the pixel values of pixels located outside the left edge of the input image are complemented;
When the coordinates of the pixel of interest are X≧2M+1 and 0≦Y≦M−1, the pixel value of a pixel outside the right end of the input image is complemented.
7. An image processing module according to claim 6. An image processing module which inputs an input image with a width of H pixels one pixel at a time in raster order, inputs pixel values of a width of (2M+1) pixels by a height of (2N+1) pixels (H, M, and N are natural numbers), performs filter processing using a central pixel as a pixel of interest, and outputs output pixel values one pixel at a time, further comprising: a delay buffer which stores pixel values of at least H×2N+2M+1 pixels among the pixels of the input image; selects (2M+1)×(2N+1) pixel values from the delay buffer; performs the filter processing using the selected pixel values; and outputs the output pixel value corresponding to the pixel of interest,
When the coordinates of the pixel of the input image are (X, Y) (0≦X≦H−1, 0≦Y),
when the coordinates of the pixel of interest are Y≧2M and M≦X≦2M−1, complementing pixel values of pixels located outside the left edge of the input image;
selecting a pixel value of a pixel in the input image when the coordinates of the pixel of interest are Y≧2M and 2M≦X≦H−1;
and when the coordinates of the pixel of interest are Y≧2M+1 and 0≦X≦M−1, complementing pixel values of pixels outside the right end of the input image, and after pixel values of H×2N+2M+1 pixels have been stored in the delay buffer, outputting the output pixel value one pixel at a time each time one pixel of the input image is input. An image processing module which inputs an input image of a height H line one pixel at a time in vertical raster order, inputs pixel values of a width of (2N+1) pixels by a height of (2M+1) pixels (H, M, and N are natural numbers), performs filter processing using a central pixel as a pixel of interest, and outputs output pixel values one pixel at a time, the image processing method comprising the steps of: selecting (2M+1)×(2N+1) pixel values from a delay buffer which stores pixel values of at least H×2N+2M+1 pixels out of the pixels of the input image; performing the filter processing using the selected pixel values; and outputting the output pixel value corresponding to the pixel of interest,
When the coordinates of a pixel in the input image are (X, Y) (0≦X, 0≦Y≦H−1),
When the coordinates of the pixel of interest are X≧2M and M≦Y≦2M−1, complementing pixel values of pixels located outside the left edge of the input image;
selecting a pixel value of a pixel in the input image when the coordinates of the pixel of interest are X≧2M and 2M≦Y≦H−1;
and when the coordinates of the pixel of interest are X≧2M+1 and 0≦Y≦M−1, complementing a pixel value of a pixel outside the right end of the input image, and after pixel values of H×2N+2M+1 pixels have been stored in the delay buffer, outputting the output pixel value one pixel at a time every time one pixel of the input image is input.

Images