RU142379U1 - IMAGE PROCESSING DEVICE - Google Patents

Abstract

1. An image processing device comprising an input controller connected to an integral image computing unit connected to an output controller, wherein the input controller is configured to receive an initial bitmap image as a stream of pixels from an image source, with the possibility of preliminary image processing and with the possibility of transmission it to the integrated image calculation unit, which, as the pixel brightness of the original raster image, is received from the input controller The image located near the current pixel of the original raster image is configured to calculate the brightness value of the current pixel of the integrated image and the brightness value of the current pixel of the integrated image of the square of the brightness of the pixels of the original raster image, and with the possibility of transmitting the calculated brightness values of the current pixels of the integrated image and the integral image of the square of brightness to a memory controller, which is configured to write calculated values to memory luminance pixels integral current image and the luminance of a square integral image, the image formation in the integral memory and the integral image yarkosti.2 square. The device according to claim 1, characterized in that the image source is selected from a set of image sources containing a CMOS sensor, a video camera, a memory controller. The device according to claim 1, characterized in that the input controller is configured to pre-process the image using one of the image processing methods selected from

Description

The utility model relates to the field of image data processing, namely to image processing devices, and can be used in video surveillance systems and video analytics.

Recently, there has been an active development of technologies and algorithms that allow us to analyze the video stream. Many video analytics algorithms use a pre-processed image, in particular an integrated image, to speed up the work.

On the other hand, an increase in the resolution and frame rate of the video stream requires an increase in the amount of equipment necessary for processing and storing video image data, increasing the processing speed, and increasing the throughput of the data transmission paths between the processing modules and the external memory.

The current level of development of VLSI technology (in addition to large integrated circuits) does not allow placing enough memory on the chip to store full-size frames of high-resolution video. In addition, the bandwidth of modern external memory interfaces is insufficient for the application of modern algorithms for processing high-resolution video images at high speed.

To solve the aforementioned problems, the claimed utility model employs the technology of processing video data “in-pass” (without writing the original image into memory and then reading from it for processing), using small memory buffers, which allows us to calculate the integral image and the image of squares of brightness pixels of the original image and optimize the operation of video analytics algorithms, such as positioning objects on the stage and others.

Closest to the claimed utility model is the image processing device described in patent CN 103236034 A, in which the integrated image is calculated by dividing the original image into blocks, followed by processing of these blocks. This device is selected as a prototype of the claimed utility model.

The disadvantage of the prototype image processing device is that it places high demands on the memory bandwidth, since it works on a principle from memory to memory, that is, it first reads the original image from memory, and then writes the processed image back to memory. In addition, the prototype device must be stored in the memory of the original image, which requires a large amount of memory.

The objective of the claimed utility model is to create an image processing device that requires less bandwidth and less memory by processing the original image “in-pass”, that is, without writing the original image to memory and then reading the image from memory to perform processing.

The problem is solved by creating an image processing device containing an input controller connected to an integral image calculation unit connected to an output controller, in which the input controller is configured to receive an initial raster image in the form of a pixel stream from an image source, with the possibility of preliminary image processing and with the possibility of transmitting it to the integrated image calculation unit, which, as I receive it from the input data controller, the brightness of the pixels of the original raster image located near the current pixel of the original raster image, is configured to calculate the brightness value of the current pixel of the integrated image and the brightness value of the current pixel of the integrated image of the square of the pixel brightness of the original raster image, and with the possibility of transmitting the calculated brightness values of the current pixels of the integrated image and integral image of the square of brightness in the memory controller, which is made with possibly Tew memory write current calculated luminance values of pixels of the integral image and brightness of the image of a square of the integral, the integral formation in the image memory and a square integral image brightness.

In a preferred embodiment of the device, the image source is selected from a set of image sources containing a CMOS sensor, a video camera, a memory controller.

In a preferred embodiment of the device, the input controller is configured to pre-process the image using one of the image processing methods selected from a set of methods containing a Bayer mask demo, white balance adjustment, geometric correction.

In a preferred embodiment of the device, the integrated image calculation unit is configured to calculate a brightness value of a current pixel of the integrated image and a brightness value of a current pixel of a square of brightness, while packing the calculated pixel brightness values into 64-bit words.

In a preferred embodiment of the device, the integrated image calculation unit is configured to calculate a brightness value of a current pixel of an integrated image and a brightness value of a current image pixel of a square of brightness, while packing the calculated pixel brightness values into 64-bit words, the packaging method being selected by the user in two ways, in the first of which the lower 29 bits of the integrated image and the lower 35 bits of the image of squares of brightness are packed, in the second m of which packaged older 29-bit integral image and older 35-bit square image brightness.

In a preferred embodiment of the device, as luminance data of pixels of the original raster image located above and to the left of the current pixel of the original raster image arrives from the input controller, the integral image calculation unit is configured to calculate the brightness value of the current pixel of the integrated image with coordinates (i, j), where i is the horizontal coordinate, j is the vertical coordinate, and the brightness of all pixels of the original raster image is summed up, located ennyh above and to the left of the current pixel of the original raster image with coordinates i, j according to the expression

Sum = ST + SC [i] + P [i, j],

where ST is the value of the memory register into which the sum of the brightnesses of all pixels of the original raster image located above and to the left of the current pixel is recorded; SC [i] is the value of the memory register into which the sum of the brightnesses of all pixels of the original raster image located in column i above the current pixel is recorded; P [i, j] - brightness of the current pixel.

In a preferred embodiment of the device, as the brightness data of the pixels of the original raster image located above and to the left of the current pixel of the original raster image arrives from the input controller, the integrated image calculation unit is configured to calculate the brightness value of the current pixel of the integrated image of the square of brightness with coordinates (i, j ), where i is the horizontal coordinate, j is the vertical coordinate, the squares of the brightnesses of all pixels of the original raster from images located above and to the left of the current pixel of the original raster image with coordinates i, j according to the expression

Sum2 = STD + SCD [i] + P2,

where STD is the value of the memory register into which the sum of the squared brightnesses of all pixels of the original raster image located above and to the left of the current pixel is recorded; SCD [i] is the value of the memory register into which the sum of the squared brightnesses of all the pixels of the original raster image located in column i above the current pixel is recorded; P2 = P [i, j] * P [i, j] is the square of the brightness of the current pixel P [i, j] of the original raster image.

For a better understanding of the claimed utility model, the following is a detailed description with the corresponding graphic materials.

FIG. 1. The structural diagram of the image processing device, made according to the utility model.

Items:

1 - input controller;

2 - block calculation of the integrated image;

3 - memory controller.

FIG. 2. The calculation scheme of the integrated image, made according to the utility model.

The claimed image processing device comprises an input controller 1, which is connected to the integral image computing unit 2, which is connected to the output controller 3. The claimed device can be part of a more complex image processing path and in general works as follows (Fig. 1-2). An input raster video image from an external source (CMOS sensor, video camera, transmission line) is fed to the input of the input controller 1. The bit depth of the original image data is 8 bits per pixel for a monochrome image resolution of a maximum of 2048 lines of 2048 pixels. Image pre-processing is performed by the input controller 1 and the data of the original pre-processed image are transmitted to the logic circuits of the integrated image calculation unit 2, which calculate the integrated image and the integrated image of the square of brightness. The resulting integrated image and the integral image of the square of brightness are packed into 64 bit data words one word per pixel and output to the output controller 3, by means of which the integral image is placed in the lower 29 bits in the memory, and the integral image of the square in the older 35 bits in the memory brightness.

Consider in more detail an embodiment of the claimed image processing device (Fig. 1-2). The initial raster image (pixel stream) is received from the image source (CMOS sensor, video camera, memory controller) by the input controller 1 (Input_Ctrl), then the data is transmitted to the integral image calculation unit 2 (Integral_Img), the calculation result of which is transmitted to the memory controller 3 (Outpu_Ctrl).

An integrated image is an image in which the brightness of each pixel is the sum of the brightnesses of all pixels of the original image lying above and to the left of this pixel of the integrated image:

The brightness calculation of each pixel of the integrated image by means of the integrated image calculation unit 2 is performed by the area (Fig. 2.) limited by the current pixel (vertical hatching) and the upper left corner of the original image (pixel coordinates in the upper left corner (0,0)), by summing the brightness of pixels in a given area.

For calculation, a memory is used in which the sums of the brightnesses of pixels of the original image are recorded in columns to the previous row (highlighted by slanted shading), inclusively, and the sums of the brightnesses of pixels of the original image up to the current pixel in the row (highlighted by horizontal shading).

The integral image calculation unit 2 uses an iterative algorithm.

Before starting the processing of the original image, all memory registers used for calculation are reset to zero: ST register 30 bits; an array of SC 2048 elements of 19 bits; TMP temporary data register 38 bit.

The SC temporary data array contains the sum of the brightnesses of the pixels of the original image in columns.

To accumulate the sum of the brightness of the pixels of the original image in the region, the ST register is used, in which the sum of the brightness values of all pixels of the original image located above and to the left of the current pixel of the original image is recorded.

Before starting processing a new line of the original image, the ST register is reset.

The brightness value of the current pixel of the integrated image with coordinates (i, j), where i is the horizontal coordinate, j is the vertical coordinate is calculated as the sum of the brightness of the pixels of the original image by the integrated image calculation unit 2 according to the following formula:

Sum = ST + SC [i] + P [i, j],

where P [i, j] is the brightness of the current pixel of the original image with coordinates (i, j). Then overwrite the current value of the sum of the column in the array: TMP = SC [i] + P [i, j], SC [i] = TMP, overwrite the register value: TMP = ST + SC [i], ST = TMP and go to calculating the brightness value of the next pixel of the integrated image.

To calculate the brightness value of the current pixel of the integrated image, the square of the brightness is performed by a similar method. The difference is only in the capacity of the used memory registers: register STD 38 bits; an array of SCD 2048 elements of 27 bits.

The square of the brightness of the current pixel of the original image: P2 = P [i, j] * P [i, j].

Sum of squares: Sum2 = STD + SCD [i] + P2.

The sum of the squares in the column: TMP = SCD [i] + P2, SCD [i] = TMP.

Sum of squares in a row: TMP = STD + SCD [i], STD = TMP.

The least significant bits are discarded to record the brightness values of the current pixel of the integrated image and the integral image of the square of brightness in a 64-bit memory register.

If the bit depth of the integrated image data does not fit in 29 bits, then, depending on the settings, the lower or upper 29 bits are taken.

If the bit depth of the integral image of the square of the brightness does not fit in 35 bits, then the lower or the highest 35 bits are taken depending on the settings.

The data is packed into 64 bits, the integral image data occupies the lower 29 bits of 64 in the memory, the square image brightness data occupies the highest 35 bits of 64 in the memory.

The methods described above for calculating the brightness value of the current pixel of the integrated image and the integral image of the square of brightness can save 34% of the memory compared to the calculation method without any optimization and align the data on the border of the 64 bit word.

The claimed utility model allows you to process the image “in the pass” in the presence of hardware limitations on the amount of internal memory on the chip and the bandwidth of the external memory interface, reducing the amount of data transmitted through the transmission channel of the integral image calculation unit - memory, reducing the requirements on the amount of internal memory of the device and allowing to increase the image processing speed by video analytics algorithms.

Although the embodiment of the utility model described above was set forth to illustrate the present utility model, it is clear to those skilled in the art that various modifications, additions and replacements are possible without departing from the scope and meaning of the present utility model disclosed in the attached utility model formula.

Claims (7)

1. An image processing device comprising an input controller connected to an integral image computing unit connected to an output controller, wherein the input controller is configured to receive an initial bitmap image as a stream of pixels from an image source, with the possibility of preliminary image processing and with the possibility of transmission it to the integrated image calculation unit, which, as the pixel brightness of the original raster image, is received from the input controller The image located near the current pixel of the original raster image is configured to calculate the brightness value of the current pixel of the integrated image and the brightness value of the current pixel of the integrated image of the square of the brightness of the pixels of the original raster image, and with the possibility of transmitting the calculated brightness values of the current pixels of the integrated image and the integral image of the square of brightness to a memory controller, which is configured to write calculated values to memory luminance pixels integral current image and the luminance of a square integral image, the image formation in the integral memory and the integral square image brightness. 2. The device according to claim 1, characterized in that the image source is selected from a set of image sources containing a CMOS sensor, a video camera, a memory controller. 3. The device according to p. 1, characterized in that the input controller is configured to pre-process the image using one of the image processing methods selected from a set of methods containing the Bayer mask demosaics, white balance adjustment, and geometric correction. 4. The device according to p. 1, characterized in that the integrated image calculation unit is configured to calculate a brightness value of a current pixel of the integrated image and a brightness value of a current image pixel of a square of brightness, while packing the calculated pixel brightness values into words of 64 bits in length. 5. The device according to claim 4, characterized in that the integrated image calculation unit is configured to calculate a brightness value of a current pixel of the integrated image and a brightness value of a current image pixel of a square of brightness, while packing the calculated pixel brightness values into words of 64 bits in length, the method Packages are selected by the user from two methods, in the first of which the lower 29 bits of the integrated image and the lower 35 bits of the image of squares of brightness are packed, in the second of which pakovyvayut older 29-bit integral image and older 35-bit square image brightness. 6. The device according to claim 1, characterized in that, as the brightness of the pixels of the original bitmap located above and to the left of the current pixel of the original bitmap arrives from the input controller, the integral image calculation unit is configured to calculate the brightness value of the current pixel of the integrated image with coordinates (i, j), where i is the horizontal coordinate, j is the vertical coordinate, and the brightness of all pixels of the original bitmap located above and evee current pixel of the original bitmap with coordinates i, j according to the expression Sum = ST + SC [i] + P [i, j], where ST is the value of the memory register into which the sum of the brightnesses of all pixels of the original raster image located above and to the left of the current pixel is recorded; SC [i] is the value of the memory register into which the sum of the brightnesses of all pixels of the original raster image located in column i above the current pixel is recorded; P [i, j] - brightness of the current pixel. 7. The device according to claim 1, characterized in that, as the brightness of the pixels of the original bitmap located above and to the left of the current pixel of the original bitmap arrives from the input controller, the integral image calculation unit is configured to calculate the brightness value of the current pixel of the square integral image brightness with coordinates (i, j), where i is the horizontal coordinate, j is the vertical coordinate, sum the squares of brightness of all pixels of the original raster image, ra a position above and to the left of the current pixel of the original raster image with coordinates i, j according to the expression Sum2 = STD + SCD [i] + P2, where STD is the value of the memory register into which the sum of the squared brightnesses of all pixels of the original raster image located above and to the left of the current pixel is recorded; SCD [i] is the value of the memory register into which the sum of the squared brightnesses of all the pixels of the original raster image located in column i above the current pixel is recorded; P2 = P [i, j] · P [i, j] is the square of the brightness of the current pixel P [i, j] of the original raster image.

Images