JP2014170986A - Image processing apparatus and parameter storage method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To store a parameter in a parameter memory with the minimum required number of memory accesses.SOLUTION: An image processing apparatus 10 includes: an image input unit 1 for inputting an image; an image memory 2 for storing the image inputted by the image input unit 1; a parameter compression unit 5 for compressing a parameter to be used in image processing on the image stored in the image memory 2; a parameter memory 6 for storing the parameter compressed by the parameter compression unit 5; a parameter decompression unit 7 for decompressing the parameter stored in the parameter memory 6; an image processing unit 4 for performing image processing on the image stored in the image memory 2 using the parameter decompressed by the parameter decompression unit 7; and an image output unit 3 for outputting the image that has undergone the image processing by the image processing unit 4.

Description

  The present invention relates to an image processing apparatus and a parameter storage method thereof, and more particularly to an image processing apparatus and a parameter storage method thereof for deforming the shapes of input still images and moving images.

  Conventionally, it is known to change the shape of an image in a projection device such as a projector or a HUD (Head Up Display). For example, as shown in FIG. 10A, when the input image Ii is projected as it is due to the installation angle of the projection apparatus 100, the screen shape to be projected, and lens distortion, the projection image Ip may be distorted. Therefore, as shown in FIG. 10B, an image processing apparatus 10 has been proposed that eliminates this distortion by applying electronic signal processing to the input image Ii (see Patent Document 1).

  In order to transform an image in such an image processing apparatus 10, the following procedure is performed. First, the image for one screen input from the image input unit 1 is stored in the image memory 2 in the order of scanning lines. Next, when the image stored in the image memory 2 is read, pixel data reading control is performed in consideration of deformation, not the scanning line order. This read control is performed along the deformation processing parameter P representing the deformation shape. When the distortion shape can be expressed by a mathematical expression such as a trapezoid, the deformation processing parameter P can be sequentially calculated and generated. On the other hand, when the shape of the distortion cannot be expressed by a mathematical expression, a method is used in which the deformation processing parameter P obtained in advance is stored in one memory for each pixel and is referred to. In order to realize this, a parameter memory 6 for storing the deformation processing parameter P is required separately from the image memory 2. The image subjected to the deformation process is output from the image output unit 3 and input to the projection apparatus 100.

JP 2012-19315 A

  By the way, the memory area of the parameter memory 6 described in Patent Document 1 is 48 bits. The 48 bits represent a memory bandwidth necessary for the parameter memory 6 to store the parameter. When the memory area of the parameter memory 6 is increased to 48 bits or more, it is possible to expect a significant improvement in calculation processing speed and processing capacity, but the cost increases. In other words, the increase in the memory area cannot be increased any more due to cost constraints. Therefore, in the image processing apparatus 10 described in Patent Document 1, when a new function is added, the current memory bandwidth is exceeded, and the parameters cannot be efficiently stored in the parameter memory 6. Since the excess is stored in the parameter memory 6 at the next memory access, there is a problem that the number of memory accesses increases as a result.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an image processing apparatus capable of storing parameters in the parameter memory with the minimum number of memory accesses and a parameter storage method thereof. It is.

  In order to solve the above problems, an image processing apparatus according to an embodiment of the present invention includes an image input unit that inputs an image, an image memory that stores an image input by the image input unit, and a storage in the image memory. A parameter compression unit for compressing parameters used for image processing performed on the processed image, a parameter memory for storing parameters compressed by the parameter compression unit, and a parameter decoding unit for decoding parameters stored in the parameter memory An image processing unit that performs image processing on the image stored in the image memory using the parameters decoded by the parameter decoding unit, and an image that outputs an image subjected to image processing by the image processing unit And an output unit.

  Here, the parameter compression unit obtains a relative value by taking a difference between the two parameters represented by the absolute value as a pair, and calculates the relative value of the parameter represented by the relative value and the parameter represented by the absolute value. Pairs may be simultaneously transferred to the parameter memory.

  The parameter compression unit may take a difference between the odd-numbered column parameter and the adjacent even-numbered column parameter, and convert the even-numbered column parameter from an absolute value to a relative value.

  The parameter decoding unit simultaneously reads a pair of a parameter represented by a relative value and a parameter represented by an absolute value from the parameter memory, and converts the parameter represented by the relative value into an absolute value. Also good.

  The parameter decoding unit may convert the parameter of the adjacent even column from the relative value to the absolute value based on the parameter of the odd column.

  The parameter may be numerical data having information on two-dimensional coordinate values.

  In order to solve the above problems, a parameter storage method according to an embodiment of the present invention is a parameter storage method for storing parameters in an image processing apparatus, the step of inputting an image, and the input image is stored in an image memory. And storing the compressed parameter in the parameter memory, and compressing the parameter used in the image processing performed on the image stored in the image memory, and storing the compressed parameter in the parameter memory.

  According to the present invention, it is possible to provide an image processing apparatus capable of storing parameters in the parameter memory with the minimum necessary number of memory accesses, and a parameter storage method thereof.

1 is a schematic block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. 1 is a detailed block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. It is explanatory drawing of the method of storing a parameter in the parameter memory of the image processing apparatus which concerns on a comparative example. It is explanatory drawing of the method of storing a parameter in the parameter memory of the image processing apparatus which concerns on a comparative example. It is explanatory drawing of the method of storing a parameter in the parameter memory of the image processing apparatus which concerns on embodiment of this invention, (a) Comparative example, (b) Before compression, (c) After compression, (d) At the time of storage, ( e) After decryption. It is a typical block diagram of the data which encoded the input image in the image processing apparatus which concerns on embodiment of this invention. FIGS. 7A and 7B are explanatory diagrams of a method for creating the parameters of the input image shown in FIG. It is explanatory drawing of the method of carrying out differential compression of the parameter produced by the production method shown by FIG. 7, and storing it in a parameter memory as a 2nd parameter, (a) Before converting into a relative value, (b) Relative value After converting to. It is explanatory drawing of the example of application of the image processing apparatus which concerns on embodiment of this invention, and is a schematic plan view which shows the mode of the label test | inspection of a drink bottle. It is explanatory drawing of the projection image by the conventional projector, (a) When not deform | transforming an image, (b) When transforming an image.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Configuration of image processing apparatus)
FIG. 1 is a schematic block diagram showing a configuration of an image processing apparatus 10 according to an embodiment of the present invention. The image processing apparatus 10 is an apparatus that performs image processing on an input image (still image and moving image) and outputs the image. An image input unit 1, an image memory 2, an image output unit 3, and an image processing unit 4. A parameter compression unit 5, a parameter memory 6, and a parameter decoding unit 7. The image input unit 1 is an input port for inputting an image. The image memory 2 stores the image input by the image input unit 1. The parameter compression unit 5 compresses parameters used for image processing applied to the image stored in the image memory 2. The parameter memory 6 stores the parameters compressed by the parameter compression unit 5. The parameter decoding unit 7 decodes the parameters stored in the parameter memory 6. The image processing unit 4 performs image processing on the image stored in the image memory 2 using the parameters decoded by the parameter decoding unit 7. The image output unit 3 is an output port or the like that outputs an image subjected to image processing by the image processing unit 4.

  For example, the parameter compression unit 5 obtains a relative value by taking a difference between a pair of two parameters represented by an absolute value, and a pair of the parameter represented by the relative value and the parameter represented by the absolute value. Are simultaneously transferred to the parameter memory 6. Specifically, the difference between the parameter of the odd-numbered column and the parameter of the even-numbered column adjacent thereto is taken, and the parameter of the even-numbered column is converted from an absolute value to a relative value.

  The parameter decoding unit 7 simultaneously reads a parameter pair represented by a relative value and a parameter represented by an absolute value from the parameter memory 6 and converts the parameter represented by the relative value into an absolute value. Specifically, based on the odd-numbered column parameters, the adjacent even-numbered column parameters are converted from relative values to absolute values.

  As described above, the image processing apparatus 10 according to the embodiment of the present invention uses differential compression when storing parameters necessary for image processing in the parameter memory 6. By creating a parameter by differentially compressing the absolute value represented by two two-dimensional coordinate values by relative expression, the memory bandwidth required for parameter transfer can be halved and stored in the parameter memory 6. .

  FIG. 2 is a detailed block diagram showing the configuration of the image processing apparatus 10 according to the embodiment of the present invention. FIFOs (First-In First-Out) 11 and 12 are buses having a function of transferring stored data. Arbiter 13 has a function of selecting a bus to be operated among FIFOs 11 and 12. When the Arbiter 13 selects the FIFO 11, the image is transferred from the image input unit 1 to the image memory 2 through the FIFO 11 and the FIFO 14. On the other hand, when the Arbiter 13 selects the FIFO 12, the image is transferred from the image memory 2 to the image output unit 3 through the FIFO 14 and the FIFO 12. When an instruction regarding deformation processing is input from an external deformation software or the like, the deformation circuit 15 performs parameter creation calculation including a differential compression function, and stores the parameter in the parameter memory 6 through the FIFO 16. When starting the deformation of the image, parameters are read from the parameter memory 6 through the FIFO 17. The conversion circuit 18 and the modification table generation circuit 19 convert the read parameters into a modification table and transfer them to the work memory 21 through the FIFO 20. The deformation table is a table for determining the degree of deformation of the output image. The work memory 21 is a memory used when performing deformation processing. An image is read from the image memory 2 to the work memory 21 and a deformation table is read from the parameter memory 6 to the work memory 21 to perform deformation processing. Various other processes can also be performed using the work memory 21. Finally, the deformed image is transferred through the FIFO 14 and the FIFO 12 and output from the image output unit 3.

(Comparative example: Parameter storage method)
3 and 4 are explanatory diagrams of a method for storing parameters in the parameter memory 6 of the image processing apparatus 10 according to the comparative example. For example, the image processing apparatus 10 according to the comparative example encodes the image input from the image input unit 1 and sets the parameter P1 using the memory bandwidth of 32 bits in the memory area (48 bits) of the parameter memory 6. Store. The parameter P1 is a set of numerical values based on a two-dimensional coordinate value (X, Y) composed of 16 bits of X coordinates and 16 bits of Y coordinates. As shown in this figure, in the comparative example, there is an unused area that is not used for storing the parameter P1. Therefore, as shown in FIG. 4, when the second parameter P2 is created and stored in the unused area, the parameter P2 exceeds the remaining memory bandwidth, and the parameters P1 and P2 are efficiently stored in the parameter memory 6. Cannot be stored.

(This embodiment: Parameter storage method)
FIG. 5 is an explanatory diagram of a method for storing parameters in the parameter memory 6 of the image processing apparatus 10 according to the embodiment of the present invention. In the comparative example, as shown in FIG. 5A, one parameter P1 is stored in the parameter memory 6 by one memory access. On the other hand, in the present embodiment, as shown in FIGS. 5B and 5C, the second parameter P2 is compressed to the parameter P2a based on the first parameter P1. Therefore, as shown in FIG. 5D, two parameters P1 and P2a can be stored in the parameter memory 6 by one memory access. Specifically, the first parameter P1 is stored in a 32-bit memory area from 0 bits to 31 bits, and the second parameter P2a is stored in a 16-bit memory area from 32 bits to 47 bits. The The two parameters P1 and P2a stored in this way are read from the parameter memory 6 by one memory access. Then, as shown in FIG. 5E, the second parameter P2a is decoded and converted into the original 32-bit parameter P2.

(Operation example)
Next, an operation example in which the deformation circuit 15 of the image processing apparatus 10 according to the embodiment of the present invention creates a parameter using differential compression and stores it in an unused area of the parameter memory 6 will be described.

  FIG. 6 is a schematic configuration diagram of data obtained by encoding an input image in the image processing apparatus 10 according to the embodiment of the present invention. As shown in this figure, the data obtained by encoding the input image are all represented by a two-dimensional coordinate graph. The two-dimensional coordinate graph is composed of a rectangular range from the origin (0, 0) of the input image to 16 (16, 8) in the X-axis direction and 8 in the Y-axis direction. This range is counted as one block, and parameters are created by repeating the number of blocks corresponding to the resolution of the input image.

  FIG. 7 is an explanatory diagram of a method for creating the parameters of the input image shown in FIG. Specifically, FIG. 7A is a two-dimensional coordinate graph in which the coordinates of one block are expressed by absolute values. This absolute value representation also interpolates the decimal values between the coordinates necessary for deformation. FIG. 7B shows a scanning path of coordinates followed when creating a parameter. As shown in this figure, first, the origin A1 (0, 0), the point A2 (1, 0), the point A3 (2, 0),... When the point A17 (16, 0) is reached and the 17th parameter is created, the next scan starts from the point B1 (0, 1). In other words, the scan for parameter creation follows a path in which the block moves from the left end of the block to the right end of the block, and then moves downward from the block and moves again from the left end of the block to the right end of the block. In this way, the parameters are created by sequentially scanning the bottom block and the right end. When the scanning of one block is completed, the process moves to the next block, and parameters are created in the same manner.

  FIG. 8 is an explanatory diagram of a method of differentially compressing the parameter created by the creation method shown in FIG. 7 and storing it in the parameter memory 6 as the second parameter. First, as shown in FIG. 8A, the input image is encoded and expressed as a two-dimensional coordinate graph in which all values are absolute. Here, numbers are assigned from the left in the scanning order. A column composed of (0, 0) (0, 1) (0, 2) (0, 3) (0, 4)... Is called a first column (odd column), and (1, 0) (1, 1) A column composed of (1,2) (1,3) (1,4)... Is called a second column (even column). In the same manner, the third column (odd column), the fourth column (even column), the fifth column (odd column),.

  Therefore, in the present embodiment, as shown in FIG. 8B, the coordinates of the even-numbered columns are converted into relative values by relative expression. That is, a difference is taken between the absolute values of consecutive odd columns and the adjacent even columns, and the coordinates of the even columns are converted into relative values by relative expression. The coordinates of odd-numbered columns are used for creating the first parameter as absolute values, and the coordinates of even-numbered columns are converted to relative values (+1, 0) and used for creating the second parameter. Yes. Since the relative value (+1, 0) is a relative expression based on the absolute value, it can have information on the absolute value without returning to the absolute value. Further, since the Y coordinate of the relative value (+1, 0) is always 0, the memory bandwidth necessary for parameter transfer can be stored in the parameter memory 6 by half that of the comparative example.

  By the differential compression having these two effects, two parameters P1 and P2 can be stored in the parameter memory 6 by one memory access. That is, the first parameter P1 is created with an odd-numbered absolute value and stored in the parameter memory 6 using a memory bandwidth of 32 bits. Also, a second parameter P2 is created with the even-numbered relative value, and stored in the parameter memory 6 using the 16-bit memory bandwidth of the unused area. As a result, the parameters can be filled so that there is no space in the 48-bit memory area of the parameter memory 6, so that the memory usage rate is improved. This operation is repeated for the number of pixels to complete the deformation processing parameters for the output image.

  Next, an operation example in which the parameters stored in the parameter memory 6 are transferred to the work memory 21 through the conversion circuit 18 and the transformation table generation circuit 19 will be described. First, the conversion circuit 18 reads a parameter from the parameter memory 6 and converts the 16-bit parameter subjected to differential compression into the original 32-bit parameter. For example, the absolute value coordinate (0, 0) and the relative value coordinate (+1, 0) of the pair can be converted into the absolute value coordinate (1, 0) based on the absolute value coordinate (0, 0). Next, the deformation table generation circuit 19 creates a deformation table based on the parameters converted by the conversion circuit 18 and transfers the deformation table to the work memory 21. The modification table is a table in which parameters after decoding are stored, but the table configuration is not particularly limited. Thereafter, the image is read from the image memory 2 to the work memory 21, the image thus read is deformed based on the deformation table, and the deformed image is transferred to the image output unit 3.

(Application example)
The image processing apparatus 10 according to the embodiment of the present invention can be applied to an apparatus that deforms the shapes of input still images and moving images, such as lens distortion correction and projector keystone correction. For example, the present invention may be applied to a projection device such as a projector or a HUD, or may be applied to an inspection camera used in a factory or the like as described below.

  FIG. 9 is a schematic plan view showing a state of label inspection of the beverage bottle 32. A plurality of beverage bottles 32 are conveyed by the belt conveyor 31. A label is affixed to the body of the beverage bottle 32. This label is obtained by printing a product display or the like with ink on a transparent substrate film. A plurality of inspection cameras 33 are installed at predetermined positions on the belt conveyor 31. The inspection camera 33 images a label affixed to the surface of the beverage bottle 32, and automatically inspects the gap between the labels and the display content of the label based on the captured image. Since the surface of the beverage bottle 32 is not flat, it is desirable to apply the image processing apparatus 10 according to the embodiment of the present invention to the inspection camera 33 and deform the captured image when inspecting the label.

  As described above, according to the present embodiment, parameters can be stored in the parameter memory with the minimum number of necessary memory accesses. That is, in order to relax the restriction on the memory area, the parameters are stored using differential compression. The differential compression has the effect of compressing the difference between successive data of the parameters to be stored, so that it has the effect of halving the memory bandwidth required for storing the parameters, and is a comparative example with one memory access. There is an effect that the parameter memory 6 stores an amount of parameters that is twice as large as the above.

  The points of this embodiment are summarized as follows.

(1) The parameter memory 6 stores parameters for performing image processing for each pixel unit. The parameter is represented by numerical data having two-dimensional coordinate value information. This two-dimensional coordinate value is an absolute value. Parameters that are continuous input signals can be assigned numbers such as first, second, third, etc. in order from the first parameter. The relative value is obtained by taking a difference between the consecutive odd and even columns as one pair. The relative value obtained by taking the difference has the same two-dimensional coordinate value information as the absolute value.

(2) The relative value obtained by taking the difference is stored in the parameter memory 6 as the second parameter. In the case of the comparative example, only one parameter could be stored in the parameter memory 6 due to memory area restrictions. However, since the capacity of the parameter that is a relative value is half of the capacity of the parameter that is an absolute value, the memory area required for storing in the parameter memory 6 is also half that of the comparative example. The parameter compressed in this way can be stored in the parameter memory 6 at the same time as storing the first parameter. That is, two parameters can be stored in the parameter memory 6 by one memory access using differential compression.

  Here, the deformation process is exemplified as the image process. However, the present invention can be applied to all image processes that require the parameters to be stored in the parameter memory with the minimum number of memory accesses. Of course, it is possible to simultaneously perform a plurality of types of image processing with the minimum number of memory accesses.

  In addition, the present invention can be realized not only as the image processing apparatus 10 but also as a parameter storage method that uses a characteristic processing unit included in the image processing apparatus 10 as a step, or the steps can be performed by a computer. It can also be realized as a program to be executed. It goes without saying that such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

DESCRIPTION OF SYMBOLS 1 ... Image input part 2 ... Image memory 3 ... Image output part 4 ... Image processing part 5 ... Parameter compression part 6 ... Parameter memory 7 ... Parameter decoding part 10 ... Image processing apparatus

Claims (7)

An image input unit for inputting an image;
An image memory for storing an image input by the image input unit;
A parameter compression unit for compressing parameters used for image processing applied to the image stored in the image memory;
A parameter memory for storing parameters compressed by the parameter compression unit;
A parameter decoding unit for decoding the parameters stored in the parameter memory;
An image processing unit that performs image processing on an image stored in the image memory using the parameters decoded by the parameter decoding unit;
An image output unit that outputs an image subjected to image processing by the image processing unit;
An image processing apparatus comprising:   The parameter compressing unit obtains a relative value by taking a difference between two parameters represented by absolute values as a pair, and simultaneously pairs the parameter represented by the relative value and the parameter represented by the absolute value. The image processing apparatus according to claim 1, wherein the image processing apparatus transfers the parameter memory.   The image processing apparatus according to claim 2, wherein the parameter compression unit takes a difference between an odd-numbered column parameter and an adjacent even-numbered column parameter, and converts the even-numbered column parameter from an absolute value to a relative value.   The parameter decoding unit simultaneously reads a pair of a parameter represented by a relative value and a parameter represented by an absolute value from the parameter memory, and converts the parameter represented by the relative value into an absolute value. The image processing apparatus according to claim 1.   The image processing apparatus according to claim 4, wherein the parameter decoding unit converts a parameter of an adjacent even-numbered column from a relative value to an absolute value based on an odd-numbered column parameter.   The image processing apparatus according to claim 1, wherein the parameter is numerical data having information of a two-dimensional coordinate value. A parameter storage method for storing parameters in an image processing apparatus,
Inputting an image;
Storing the input image in an image memory;
Compressing parameters used for image processing applied to the image stored in the image memory;
Storing the compressed parameters in a parameter memory;
A parameter storage method comprising: