KR20090121667A - Sub-sampling system for real-time image processing - Google Patents

Abstract

PURPOSE: A sub sampling system for a real time image process is provided to process various image processing within a short time by reducing the image size to be processed. CONSTITUTION: The sub sampling system(100) for the real time image process comprises a window generating unit, window operation unit, pixel interpolation unit, and sub-sampling image output unit. The window generating unit(120) assigns the register having the storage corresponded to a plurality of sub sampling rates. The window generating unit assigns the internal memory corresponded to the original image and plurality of sub sampling rates. The window operation unit(130) successively stores a part of the original image in the internal memory by the pixel basis. The window operation unit successively stores pixel data belonging to the moving window corresponded to the domain in which sub-sampling operates among pixel data stored in the internal memory in the register. The window operation unit successively transfers the moving window. The pixel interpolation unit(210) produces the interpolation pixel corresponding to a plurality of sub sampling rates by using the pixel data stored in the register.

Description

Sub-sampling system for real-time image processing

The present invention relates to a system and method for real-time image processing, and more particularly to a sub-sampling system and method for real-time image processing.

Until recently, a number of subsampling systems and methods have been proposed. Subsampling refers to image processing that converts an original image consisting of pixels of a certain size (eg 640 * 480) to a smaller size (eg 320 * 240).

The subsampling technique is very useful in real time image processing. For real-time image processing, the computation time according to the image processing should be reduced because the computation time is reduced when the subsampling is performed, the size of the image that is the object of the image processing becomes smaller.

Meanwhile, a number of hardware or software systems for real time image processing have been proposed. Among these various systems, Korean Patent No. 10-0523848, published October 26, 2005, proposes a hardware system and method for processing an image based on a window.

Specifically, Patent No. 10-0523848 proposes a real-time window-based image processing structure (RWIPS) having a plurality of window buffers implemented with registers and a plurality of line buffers implemented with dual port RAM. The original image is stored in the line buffer in pixel units, and the register stores pixel data belonging to a specific window among images designated in the line buffer.

The conventionally proposed subsampling technique has a problem that the subsampling efficiency is not reduced or the subsampling rate is not changed.

The present invention has been proposed to improve the conventionally proposed technique, and an object of the present invention is to propose a subsampling technique with improved efficiency.

It is yet another object of the present invention to propose a western sampling technique that simultaneously supports multiple subsampling rates.

The present invention relates to a system for performing subsampling according to a plurality of sub-sampling rates on an original image, wherein the registers having storage spaces corresponding to the plurality of sub-sampling rates are allocated, and the original image and the plurality of sub-sampling rates are assigned. A window generator for allocating an internal memory corresponding to the sub-sampling rate of the apparatus; A portion of the original image is sequentially stored in the internal memory in units of pixels, and pixel data belonging to a moving window corresponding to a region in which subsampling is performed among pixel data stored in the internal memory is sequentially stored in the register. A window operator which sequentially moves the moving window; A pixel interpolation unit configured to calculate a plurality of interpolation pixels corresponding to each of the plurality of sub-sampling rates using the pixel data stored in the register; And a sub-sampling image output unit configured to output the interpolation pixel.

The apparatus may further include a sync control signal generator configured to generate a sync signal for restoring the interpolation pixel to an image.

Preferably, the sync signal includes a frame sync signal, a line sync signal, and a pixel sync signal.

Advantageously, the size of said register is determined in accordance with the minimum subsampling rate of said plurality of subsampling rates.

Preferably, the moving window moves in a row direction of pixel data stored in the internal memory.

Advantageously, said internal memory comprises dual port RAM of a size corresponding to a size of a row of said original image and a minimum subsampling rate of said plurality of subsampling rates.

According to the present invention, there is an advantage of performing subsampling in real time.

When the subsampling is performed according to the present invention, the size of an image to be processed is reduced, so that various image processing can be completed in a short time.

In addition, the present invention can be simply implemented in hardware, it can be easily applied to robot vision, intelligent camera, industrial sites, etc.

Specific operations, features, and effects of the present invention will be further embodied by one embodiment of the present invention described below. Hereinafter, with reference to the accompanying drawings will be described an embodiment of the present invention.

1 is a block diagram of a sub-sampling system according to the present embodiment. As shown. The system 100 of FIG. 1 includes a subsampling coefficient input unit 110, a window generator 120, a window calculator 130, a pixel interpolator 210, a synchronous control signal generator 220, and a subsampled image output. The unit 310 is included.

The subsampling coefficient input unit 110 receives a subsampling coefficient from an external source. The system 100 of FIG. 1 performs subsampling according to the subsampling coefficients. The subsampling coefficient represents the number of subsampling rates. Specifically, when the subsampling coefficient is determined to be '4', subsampling is performed according to four types of subsampling rates.

The subsampling rate represents a ratio between the size of the original image and the size of the subsampled image. For example, if the original image consists of 100 * 100 pixels, half-subsampling results in an image of 50 * 50 pixels.

The system according to this embodiment supports multiple subsampling rates. The lowest of the plurality of sub-sampling rates is called the minimum sub-sampling rate. For example, among the 1/1, 1/2, 1/3, and 1/4 sub-sampling rates, 1/4 rate is the minimum sub-sampling rate.

The subsampling coefficient corresponds to the minimum subsampling rate. For example, if the subsampling coefficient is determined to be '4', the minimum subsampling rate is determined to be 1/4. As a result, when the subsampling coefficient is determined as '4', subsampling is performed according to four subsampling rates of 1/1, 1/2, 1/3, and 1/4.

The window generator 120 allocates register and memory resources for subsampling. Specifically, a register having a storage space corresponding to a plurality of sub sampling rates is allocated, and an internal memory corresponding to the original image and the plurality of sub sampling rates is allocated.

The register may be implemented by flip-flop. As described above, the size of the register preferably corresponds to a plurality of sub-sampling rates. Specifically, when the minimum sub-sampling rate is 1 / N, the size of the register is preferably determined to be N * N.

The internal memory may include a dual port RAM. The size of the dual port RAM is preferably determined according to the size of the row of the original image and the minimum sub-sampling rate. For example, when the size of the original image is m * k and the minimum sub-sampling rate is 1 / N, the size of the dual port RAM is preferably m * (N-1).

The window operator 130 processes an image operation on an original image input from the outside. In detail, the window calculator 130 obtains an original image and stores some of the original image in the internal memory and the register. The window operator 130 stores pixel data belonging to the moving window in the register and moves the moving window.

The pixel interpolator 210 is a module that performs subsampling on pixel data stored in the register. In detail, the pixel interpolator 210 accesses the register to perform interpolation of the pixel data. The pixel interpolator 210 may perform interpolation corresponding to each of a plurality of sub-sampling rates.

The sync control signal generator 220 generates a sync signal for restoring an interpolation pixel generated by the pixel interpolator 210. In order to restore the pixel data to an image, a synchronization signal indicating frame synchronization, line synchronization and pixel synchronization is required. The sync control signal generator 220 may generate a plurality of sync signals corresponding to each of a plurality of sub-sampling rates.

The sub-sampled image output unit 310 outputs an interpolation pixel generated by the pixel interpolator 210 and a synchronization signal generated by the synchronization control signal generator 220. Preferably, the sub-sampled image output unit 310 may combine and output an interpolation pixel and a synchronization signal corresponding to a specific sub-sampling rate. A plurality of interpolation pixels and sync signals corresponding to one sub sampling rate may be referred to as a sub sampling output signal. For example, a plurality of interpolation pixels (ie, 1/3 interpolation pixels) generated according to 1/3 sub-sampling rate and a sync signal corresponding to the 1/3 sub-sampling rate correspond to 1/3 sub-sampling rate. Subsampling output signal. Since the system 100 of FIG. 1 supports a plurality of sub sampling rates, the sub sampling image output unit 310 may generate a plurality of sub sampling output signals.

Hereinafter, the operation in each block introduced in FIG. 1 will be described in more detail.

2 is a diagram illustrating a method of storing an original image in an internal memory and a register when an original image is input from the outside.

As shown, the internal memory may include a line buffer. The line buffer may be implemented with the above-described dual port RAM. In addition, as described above, when the size of the original image is m * k and the minimum sub-sampling rate is 1 / N, the size of the line buffer may be m * (N-1). In addition, as described above, the size of the register preferably corresponds to a plurality of sub-sampling rates. That is, when the minimum sub-sampling rate is 1 / N, it may be determined as N * N as shown.

The line buffer sequentially stores the original image in pixels. Meanwhile, among the pixel data stored in the internal memory, pixel data belonging to a moving window is stored in a register.

The moving window represents an area where subsampling is performed. If subsampling is performed on a register other than the internal memory of the pixel data belonging to the moving window, fast image processing is possible. In detail, storing pixel data in a readily accessible register enables real-time processing of these pixels, and enables real-time image processing because simultaneous access to all pixel values belonging to the moving window is possible if necessary.

3A to 3M illustrate a technique in which an original image is stored in an internal memory and pixel data stored in the internal memory is stored in a register.

3A to 3M show a case where the subsampling coefficient is three. Thus, subsampling at 1/1, 1/2, and 1/3 rates is performed. Also, since the minimum sub-sampling rate is 1/3, the size of the register is determined to be 3 * 3.

3A is an example of an original image having a size of 10 * 6. As shown, pixel data from 1 to 10 form the first horizontal line of the original image. Since the size of the original image is 10 * 6 and the minimum sub-sampling rate is 1/3, the line buffer included in the internal memory has a size of 10 * (3-1).

As shown in FIG. 3B, the first horizontal line (pixel data of pixels 1 to 10) of the original image is input to the line buffer in units of pixels.

First, the result of inputting the pixel data 1 into the line buffer and the register is shown in FIG. 3C. 2, 3, and 4 pixel data are sequentially input to the line buffer and the register, and the results are shown in FIGS. 3D to 3F.

The result of inputting the pixel data of the tenth pixel of the last horizontal line of the original image is shown in FIG. 3G. When the input to the first horizontal line of the original image is finished, the input to the second horizontal line (pixel data of 11 to 20 pixels) is performed. In this case, the data stored in the first line of the line buffer is stored in the register while moving to the second line of the line buffer. Referring to FIG. 3H, it can be seen that the pixel data 1 is moved from the first line to the second line of the line buffer and stored in the register. The 12th and 13th pixel data are sequentially input to the line buffer and the register, and the results are shown in FIGS. 3I to 3J.

The result of inputting the 20th pixel data of the last data of the second horizontal line of the original image is shown in FIG. 3K. When the input to the second horizontal line of the original image is finished, the input to the third horizontal line (pixel data of pixels 21 to 30) is performed. The pixel data of 21 and 22 are sequentially input to the line buffer and the register, and the result is shown in FIGS. 3L to 3M.

Inputting the pixel data into the internal memory and the register according to the method of FIG. 3 is equivalent to moving a 3 * 3 sized moving window by forming a moving window in the original image.

That is, if the window operation unit 130 repeats the operation of FIG. 3, as shown in FIG. 4, the moving window having a size of 3 * 3 is moved in the row direction of the original image. When the moving window moves to the right end in the row direction, the moving window is moved to the right in the row direction again for the next horizontal line.

The pixel interpolator 210 performs interpolation using pixel data belonging to the moving window. For example, if the minimum sub-sampling rate is 1/4, the 4 * 4, 3 * 3, and 2 * 2 sized pixel data are simultaneously accessed and interpolated using a moving window of 4 * 4 size. You can output the result.

5 illustrates an example of calculating interpolation pixels using a moving window having a size of 4 * 4. In order to calculate a quarter interpolation pixel, pixel data 1, 2, 3, 4, 11, 12, 13, 14, 21, 22, 23, 24, 31, 32, 33, and 34 of FIG. 5 must be interpolated. do. In addition, in order to calculate 1/3 interpolation pixels, pixel data of 1, 2, 3, 11, 12, 13, 21, 22, and 23 of FIG. 5 should be interpolated. In order to calculate 1/2 interpolation pixels, pixel data of 1, 2, 11, and 12 of FIG. 5 should be interpolated.

Since the pixel data of 1, 2, 3, 4, 11, 12, 13, 14, 21, 22, 23, 24, 31, 32, 33, and 34 are located in an area corresponding to the moving window, the 16 pixel data Is already stored in the register by the window operator 130. On the other hand, since the data stored in the register can be accessed simultaneously, each interpolation pixel can be calculated in parallel. That is, the pixel interpolator 210 may approach the register and output interpolated pixels subsampled at 1/4 rates, and may output subsampled interpolated pixels at 1/3 and 1/2 rates.

When the moving window moves in the row direction or the column direction, it is not necessary to perform interpolation according to all sub-sampling rates every time the moving window is moved. For example, for a quarter rate subsampling, it is sufficient that interpolation is performed each time the moving window moves by four pixels in the row or column direction. That is, when the subsampling rate is 1/4, one quarter interpolation pixel is generated every time the moving window is moved by four pixels.

The pixel interpolation unit 210 may use various methods such as a bilinear interpolation method, a closest value using method, and a third-order line interpolation method.

Since 1/2 to 1 / N interpolation pixels output by the pixel interpolation unit 210 do not include position information of each pixel, in order to restore an image, a synchronization signal indicating frame synchronization, line synchronization, and pixel synchronization is provided. It is preferred to be included.

The synchronization control signal generator 220 generates a synchronization signal for reconstructing the interpolation pixels calculated by the pixel interpolator 210 into an image.

An example of each synchronization signal is shown in FIG.

First, the pixel sync signal is a signal for distinguishing each pixel.

Also, the line sync signal is a signal that distinguishes the horizontal lines of the original image. For example, for a 640 * 480 image, the line sync signal is used to distinguish 480 horizontal lines.

In addition, the frame synchronization signal distinguishes each frame. For example, it is used to distinguish between a 640 * 480 image displayed at t time and a 640 * 480 image displayed at t + 1 time.

7 shows an example of changing the frequency of the synchronization signal generated by the synchronization control signal generator. When subsampling is performed at a rate of 1 / N, the frequency of the pixel synchronizing signal is controlled to 1 / N of the frequency of the pixel synchronizing signal for the original pixel. In addition, the frequency of the line synchronization signal is also controlled to 1 / N of the original frequency. On the other hand, the frame sync signal does not need to be changed, so it is controlled to be equal to the original frequency.

The sub-sampled image output unit 310 outputs an output signal according to each sub-sampling rate. For example, for a 1/4 rate subsampling, a 1/4 interpolation pixel using a 4x4 moving window, a pixel sync signal corresponding to a 1/4 rate subsampling rate, a line sync signal, and a frame sync signal Output

8 is an example of an original image. Performing subsampling with the subsampling coefficient set to '4' for the original image of FIG. 8, results in the result of FIG. 9.

The first picture of Figure 9 is an image subsampled at 1/1 rate, i.e. the same as the original output.

The second picture of Fig. 9 is the output subsampled at 1/2 rate by this embodiment, the third picture is the output subsampled at 1/3 rate by this embodiment, and the fourth picture is this embodiment. Is subsampled at 1/4 rate.

Since the present embodiment is based on the register as described above, parallel processing is possible. Accordingly, sub-sampled images having various sizes can be generated within a short time by simultaneously outputting interpolation pixels corresponding to the sub-sampling rate. Such subsampling may be usefully used to obtain a foreground image from a background image.

Although specific numerical values or specific data have been presented in the above-described embodiment, this is merely for describing the present embodiment in detail, and the scope of the present invention is not limited to the specific numerical values or specific data presented through the present embodiment. On the other hand, those skilled in the art having ordinary knowledge of the present invention will be capable of various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes and additions should be regarded as belonging to the following claims.

Since the present invention relates to a subsampling technique usefully used in an image processing system, it will be reasonable that the industrial applicability is recognized.

1 is a block diagram of a sub-sampling system according to the present embodiment.

2 is a diagram illustrating a method of storing an original image in an internal memory and a register.

3A to 3M illustrate a method of storing an original image in an internal memory and a register.

4 is a block diagram illustrating a moving window having a size of 3 * 3.

5 illustrates an example of calculating interpolation pixels using a moving window having a size of 4 * 4.

6 is an example of a synchronization signal used in this embodiment.

7 shows an example of changing the frequency of the synchronization signal generated by the synchronization control signal generator.

8 and 9 show subsampling results according to the present embodiment.

Claims (11)

In a system for performing subsampling according to a plurality of subsampling rates for an original image, A window generator for allocating a register having a storage space corresponding to the plurality of sub-sampling rates and allocating an internal memory corresponding to the original image and the plurality of sub-sampling rates; A portion of the original image is sequentially stored in the internal memory in units of pixels, and pixel data belonging to a moving window corresponding to a region in which subsampling is performed among pixel data stored in the internal memory is sequentially stored in the register. A window operator which sequentially moves the moving window; A pixel interpolation unit configured to calculate an interpolation pixel corresponding to each of the plurality of sub-sampling rates using the pixel data stored in the register; And A subsampling image output unit configured to output the interpolation pixel The method of claim 1, And a synchronization control signal generator configured to generate a synchronization signal for restoring the interpolation pixel to an image. The sub-sampling image output unit outputs the synchronization signal together with the interpolation pixel. The method of claim 2, The sync signal includes a frame sync signal, a line sync signal, and a pixel sync signal. The method of claim 3, The synchronous control signal generation unit controls the frequency of the line synchronous signal and the pixel synchronous signal according to the plurality of sub-sampling rates. The method of claim 1, The size of the storage space of the register is determined according to the minimum sub-sampling rate of the plurality of sub-sampling rate The method of claim 5, The amount of storage space of the register is. A subsampling system, characterized in that N * N when the minimum subsampling rate is 1 / N The method of claim 1, The moving window moves in a row direction of pixel data stored in the internal memory. The method of claim 1, The internal memory includes a dual port RAM having a size corresponding to a size of a horizontal line of the original image and a minimum subsampling rate among the plurality of subsampling rates. A method of performing subsampling according to a plurality of subsampling rates on an original image, the method comprising: Allocating a register having a storage space corresponding to the plurality of sub-sampling rates and an internal memory corresponding to the original image and the plurality of sub-sampling rates; A second step of sequentially storing a portion of the original image in the internal memory in units of pixels; A third step of sequentially storing pixel data belonging to a moving window corresponding to an area in which subsampling is performed among pixel data stored in the internal memory, in the register; Calculating an interpolation pixel according to each of the plurality of sub-sampling rates using pixel data stored in the register; A fifth step of repeatedly moving the moving window and repeating the third and fourth steps; And And a sixth step of outputting an interpolation pixel calculated through the fifth step. The method of claim 9, The intersampling pixel output through the sixth step is output together with a synchronization signal corresponding to the plurality of subsampling rates. The method of claim 10, The sync signal includes a frame sync signal, a line sync signal, and a pixel sync signal.

Images