JPH09231349A - Information processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processor capable of suitably executing the exchange/integration of a processing instruction string by decomposing geometric transformation based upon different symmetry. SOLUTION: An instruction string for commanding picture processing is stored in an instruction storing part 1. An instruction changing part 4 successively extracts geometric transformation instructions stored in the storing part 1 and an instruction symmetry judging part 2 judges symmetry for the geometric transformation of connected processing instructions. A transformation decomposing part 3 decomposes a geometric transformation instruction based upon the judged result. An instruction changing part 4 changes the instruction string stored in the storing part 1 based upon the decomposed instruction or exchanges the order of instructions in the instruction string.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus provided with affine transformation and other image processing functions and providing various image processing functions by combining them.

[0002]

2. Description of the Related Art A two-dimensional affine transformation L is an Ax with respect to x by a second-order invertible matrix A and a two-dimensional vector b.
It is a conversion on a plane that corresponds to + b. In the following, especially b =
The case of 0 will be described, and in order to simplify the expression, the affine transformation L and the matrix A that represents the affine transformation L are often confused and used.

Enlargement, reduction, rotation, reflection,
Applying an affine transformation including translation and a combination thereof is a basic function in an information processing device,
Many methods have been proposed for efficiently executing the affine transformation. Some of these methods decompose a given affine transformation into products of special types of transformations. For example,

[Equation 1] It is well known that the rotation is decomposed into the product of three oblique axis transformations and executed. In addition, Japanese Patent Publication No. 4-812
30 and Japanese Patent Publication No. 4-81231, the rotation is decomposed into a product of two oblique axis transformations and one enlargement / reduction (hereinafter referred to as enlargement / reduction).

[Equation 2] Has been proposed. Alternatively, Japanese Patent Laid-Open No. 1-131971
As described in Japanese Patent Publication No.
A method of decomposing into 0 ° rotation and a method of decomposing into three oblique axis conversions as described in Japanese Patent Publication No. 2-14750 are also proposed. Furthermore, JP-A-61-151
In Japanese Patent No. 783, the decomposition of the affine transformation, which is not limited to rotation, into the product of two oblique axis transformations and one scaling.

(Equation 3) Has also been proposed. The main focus of the decomposition of these affine transformations is to express the transformations as a combination of processings such as oblique axis transformation and expansion / contraction that do not cost much operation, thereby performing a single affine transformation processing at high speed.

On the other hand, in Japanese Laid-Open Patent Publication No. 05-207266, a plurality of processing commands to be applied to an image are stored, their processing orders are exchanged if necessary, and similar processing is integrated or unnecessary processing is omitted. It describes a method for improving the efficiency of processing by doing so. According to this, for example, the coordinate conversion L ₁ L ₂ L _{3 is} called without sequentially executing a plurality of commands of the same type of processing such as (coordinate conversion L ₁ → coordinate conversion L ₂ → coordinate conversion L ₃ ). Replaced with a single integrated processing run. As a result, the processing time can be shortened, and the number of geometric conversions of the essentially irreversible digital image can be reduced to improve the quality of the processed image. Other techniques for integrating or rearranging such processing include, for example, Japanese Patent Publication No. 3-78668 and Japanese Patent Laid-Open No. 4-78668.
For example, there are those described in Japanese Patent No. 273387.

In order to improve the efficiency of processing by exchanging the processing order, integrating similar processing, or omitting unnecessary processing, which is proposed in Japanese Patent Laid-Open No. 5-207266, not only a single affine conversion but also a plurality of affine conversions, For example, image processing other than affine transformation, such as filter processing by convolution, appears and modifies the sequence of these processing instructions.

For example, if the image f is filtered with the kernel g, and the coordinates are transformed by the matrix L, the image f
Is transformed with a matrix L, then g is subjected to coordinate transformation with the matrix L, and the value is multiplied by the absolute value of the determinant of L, which is the same as the one applied with a filter, and (g * f) L = | Det L | gL * fL. Here, g * f is a notation that represents the convolution of the image f with the kernel g, and fL is the notation that represents the coordinate transformation of the image f with the matrix L. This equation represents the exchange relationship between the filter and the coordinate transformation. By using such an exchange relation,
For example, a series of processes such as (coordinate conversion → filter → coordinate conversion) is replaced so that processes of the same kind as (coordinate conversion → coordinate conversion → filter) are connected, and further integration of processes is attempted. You can However, in this case, side effects may occur due to the core of the filter being changed by the order exchange. That is, for example, the nucleus g
Is represented by an array of size 3 × 3, and if the size of the array for representing gU is changed to 5 × 5 by rotating it, the processing time of the filter increases.

Although it is not desirable to have rotational symmetry in a general image, the kernel of a filter can often assume geometrical symmetry. In the above equation, for example, L is | det L | =
If 1, then (g * f) L = gL * fL, and if L is a rotation and g has rotational symmetry such as a Laplacian filter or a smoothing filter, then (g * f) L = It becomes g * fL. The filter having rotational symmetry is not limited to the linear convolution as described above, and the same applies to a morphological filter whose constituent elements have rotational symmetry.

Therefore, for example, g is isotropic,
If L can be decomposed into L = UN as the product of the rotation U and the oblique axis conversion N, then (g * f) L = gN * fL, and no rotation is performed on g, only the oblique axis conversion. You can see that it is good to apply.

Usually, the core of the filter is represented as a two-dimensional array, and if the core is rotated, the size of the array representing the core is increased and the processing time is increased. Also,
Coordinate transformation of the kernel causes resampling errors and degrades processing quality. Since the array representing the kernel is much smaller than the array representing the image, the error associated with the coordinate conversion becomes relatively large. Further, if the number of non-zero components in the array increases due to the coordinate conversion to the kernel, the processing time also increases. These problems can be avoided by replacing the coordinate transformations to the kernel with simple ones.

Alternatively, when Φ is a process for obtaining the maximum spatial frequency of the image and Φ is performed after the coordinate conversion by the matrix L is performed on the image f, L is the product of the rotation U and the expansion / contraction A and is decomposed into L = UA. If it is possible, Φ (fL) = Φ (fA) holds, and it is understood that the original image f need only be scaled without being rotated.

As described above, in many kinds of processing for giving the feature quantity of an image, many kinds of processing have geometrical symmetry,
If the coordinate conversion is performed before the feature amount extraction, it is often possible to replace the coordinate conversion with a simpler one without changing the result.

Under such circumstances, many image processings have symmetry with respect to a subgroup of the transformation group formed by coordinate transformation. By utilizing this symmetry, the affine transformation performed without changing the result can be replaced with a simpler one, so that the processing speed and quality can be improved. For that purpose, it is necessary to divide the affine transformation according to the typical symmetry and decompose it. However, in the conventional method of disassembling and executing the affine transformation, it is not possible because it does not consider disassembling the functions separately.

Further, when the efficiency of the processing is improved by integrating the similar processing proposed in Japanese Patent Laid-Open No. 5-207266 or omitting unnecessary processing, for example, (enlargement / reduction D → rotation U) As in the affine transformation DU, a specialized coordinate transformation such as scaling or rotation is once reduced to a general affine transformation, which may cause deterioration in processing quality and reduction in processing speed.

FIG. 25 is a conceptual diagram showing a conventional affine conversion process. As shown in FIG. 25, the normal affine transformation process is performed on a digital image by extending the digital image f into an image f ′ on a continuous area by designating some interpolation method, and by coordinate transformation L. This is represented by the process of pulling back, performing sampling again, and obtaining a digital image g. FIG. 25 is a so-called commutative diagram, in which the vertices of a graph represent a set, the arrows connecting the vertices represent a mapping, and when there are a plurality of rows of arrow lines connecting two vertices in the diagram, they are connected. The mapping defined by does not depend on the selection of the arrow sequence. Z is a rational integer ring and R is a real number field. The digital image is
If it has coordinates on a plane and has a density for each point, the digital image has a correspondence relationship between the points on the plane and the density. for that reason,
The digital image f can be regarded as a bounded real-valued function defined on a rational integer point in the plane. Considered in this way, the extension f ′ to the entire plane can be regarded as a real-valued function on the plane satisfying f ′ · i = f. Here, there is arbitrariness in the selection of the extension f ′, and this arbitrariness corresponds to the difference in interpolation methods such as nearest neighbor interpolation and linear interpolation.

Here, the image f'may not satisfy the assumption of the sampling theorem at the time of resampling due to the coordinate transformation L. For example, it is typical that L is a combination of reduction and rotation. On the other hand, in the reduction processing, an algorithm different from the conceptual diagram of FIG. 25 in which the action of the low-pass filter such as the projection method is added is general. Therefore, if a low-pass filter can be applied in the reduction processing, In some cases, it is more appropriate not to integrate such processes and execute them as an affine transformation.

Although it may not always be a good idea to mechanically expand such specialized processing to general affine transformation, it is necessary to divide the affine transformation that has been integrated once again into functions and decompose them. This makes it possible to efficiently use a specialized affine transformation algorithm while omitting redundant repetitive processing.

For the above reasons, the decomposition of the coordinate transformation from the viewpoint of symmetry or invariance is caused by exchanging or integrating the processing order for a plurality of coordinate transformations and image processing other than the coordinate transformations. It is effective for efficiency improvement.

However, in the conventional decomposition of the affine transformation, the focus has been only on improving the efficiency of a single affine transformation process, so that the efficiency of a plurality of such affine transformations and other processes is improved as a whole. For this purpose, decomposition by transformation symmetry has not been attempted.

[0019]

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned circumstances, and it is possible to appropriately perform exchange integration of processing instruction sequences by decomposing affine transformations based on different symmetries. An object of the present invention is to provide an information processing device that can be used.

[0020]

According to a first aspect of the present invention, in an information processing apparatus, an instruction storage unit for storing an instruction sequence consisting of a plurality of image processing instructions, and a geometry of each instruction in the instruction sequence. Instruction symmetry determining means for determining symmetry regarding conversion, geometric transformation instruction decomposing means for decomposing an instruction for geometric transformation stored in the instruction storing means, and an instruction sequence stored in the instruction storing means are changed. The instruction changing means changes the instruction sequence by utilizing decomposition of the conversion instruction by the geometric conversion instruction means based on the symmetry determined by the instruction symmetry determining means. It is what

According to a second aspect of the present invention, in the information processing apparatus according to the first aspect, the instruction symmetry determination means is
The symmetry with respect to the positive scalar multiplication or the symmetry with respect to the coordinate transformation in which the absolute value of the determinant is 1 is determined, and the geometric transformation instruction decomposition means performs the coordinate transformation by the positive scalar multiplication and the absolute value of the determinant. Is decomposed into a product with a coordinate transformation of 1.

According to a third aspect of the present invention, in the information processing apparatus according to the first aspect, the instruction symmetry determination means is
Determining symmetry about rotation, reflection, oblique axis transformation or unimodular diagonal transformation, the geometric transformation command decomposition means transforms coordinate transformation whose absolute value of determinant is 1 into rotation or reflection; It is characterized by decomposing into a product of the diagonal axis transformation and the unimodular diagonal transformation.

According to a fourth aspect of the present invention, in the information processing apparatus according to the first aspect, the instruction changing means is capable of exchanging a processing order between a linear filter or a morphological filter and coordinate conversion. It is what

[0024]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram showing an example of a system including an embodiment of an information processing apparatus of the present invention. In the figure, 1 is an instruction storage unit, 2 is a symmetry storage unit, 3 is a conversion decomposition unit, 4 is an instruction changing unit, 5 is an image storage unit, 6 is an image processing unit, 7 is an image input unit, and 8 is an instruction input. Reference numeral 9 is an image display unit, 10 is an image output unit, 11 is a control unit, and 12 is an information processing unit.

The image storage unit 5 stores the image input from the image input unit 7. The image processing unit 6 processes the image stored in the image storage unit 5 according to the image processing command stored in the command storage unit 1.

The image input unit 7 is composed of a scanner or the like, and inputs image data and stores it in the image storage unit 5. FIG.
FIG. 3 is a block diagram showing an example of an image input unit. In the figure,
21 is a CCD sensor, 22 is an optical system, 23 is an optical system drive circuit, 24 is an A / D converter, 5 is a buffer memory, 26
Is an image data transmission circuit, and 27 is a control circuit. The image of the original is read by the optical system driving circuit 23 by the CCD sensor
The original or the optical system 2 in the direction perpendicular to the scanning direction of
2. The CCD sensor 21 is moved and the image of the entire original is read. The image read by the CCD sensor 21 is
The analog signal is converted into a digital signal by the A / D converter 24 and temporarily stored in the buffer memory 25. The image data stored in the buffer memory 25 is encoded and transmitted to the image storage unit 6 via the image data transmission circuit 26. The control circuit 27 receives a signal from the control unit and controls the entire image input unit 7.

The instruction input unit 8 is composed of, for example, a keyboard and a pointing device such as a mouse. The operator gives an image processing command from the instruction input unit 8 while referring to the image data displayed on the display unit 9. The given image processing command is stored in the command storage unit 1.

The display unit 9 is composed of, for example, a display device such as a display, and displays at least the images before and after the processing stored in the image storage unit 5. The control unit 11 controls each unit.

The image output unit 10 is composed of a printer, an interface to a network and other systems, and the like.
The image stored in the image storage unit 5 or the image processed by the image processing unit 6 is output. FIG. 3 shows the image output unit 10.
FIG. 4 is a block diagram showing an example of the above. In the figure, 31 is an image data receiving circuit, 32 is a laser driver, 33 is a laser oscillator, 34 is a synchronizing circuit, 35 is a motor drive circuit, 36 is a polygon mirror, 37 is a photosensitive drum, and 38 is a paper feed system drive circuit. , 39 are control circuits. Here, the image output unit 1
As one of 0, a laser beam printer is shown.
The image data transmitted is the image data receiving circuit 31.
The laser light is output from the laser driver 32 and the laser oscillator 33 while being synchronized by the synchronizing circuit 34. In addition, the synchronization circuit 34 is the motor drive circuit 3
5 is controlled to drive / stop the motor for rotating the polygon mirror 6. Further, the synchronization of the laser oscillator 33 and the polygon mirror 36 is controlled by the synchronizing circuit 34 based on the laser light detected by the beam detector, and a latent image is formed on the photosensitive drum 39. The synchronizing circuit 34 is controlled by a control circuit 39 which controls the entire printer. The control circuit 39 further includes a paper feed system drive circuit 38.
In addition to controlling, the control unit 11 receives a command to perform a series of controls. Since the laser beam printer is already well known, detailed description thereof will be omitted here.

The information processing unit 12 has an instruction storage unit 1, a symmetry determination unit 2, a conversion decomposition unit 3, and an instruction changing unit 4.
The command storage unit 1 holds image processing command commands. The symmetry determination unit 2 determines the attribute of symmetry with respect to the geometric transformation of each of the image processing instructions stored in the instruction storage unit 1, corresponding to each image processing instruction. The conversion decomposition unit 3 decomposes the geometric conversion instruction based on the symmetry of the instruction.

The command changing unit 4 creates a command sequence stored in the command storage unit 1 and having a function equivalent to that of the original image processing command. FIG. 4 is an explanatory diagram showing the concept of the instruction change processing by the instruction change unit in the embodiment of the information processing apparatus of the present invention. How to create a new sequence of instructions in the instruction changing unit 4 is, for example, as shown in FIG.
As shown in (2), processing for integrating processing instructions of the same type into one processing instruction, such as integrating two coordinate conversions into one coordinate conversion, is performed. Further, as shown in FIG. 4 (B), processing for omitting redundant processing instructions is performed, such as omitting coordinate conversion that represents identity conversion. FIG.
As shown in (C), the processing order may be exchanged, for example, the processing order of the filter processing and the coordinate conversion may be exchanged. Further, the instruction changing unit 4 inquires of the symmetry determination unit 2 whether or not the instruction held in the instruction storage unit 1 has symmetry regarding geometric transformation, and based on the symmetry of the instruction, the geometric transformation instruction connected to it. Is decomposed by the conversion decomposition unit 3.

An example of the decomposition processing of the geometric conversion instruction performed by the conversion decomposition unit 3 will be described below. It is assumed that the transformation indicated by the geometric transformation instruction is represented by the matrix A.
Here, the positive scalar multiplication and the decomposition of the determinant into a matrix having an absolute value of 1 are as in the following expression (1).

(Equation 4) Here, the positive scalar multiplication indicates scaling conversion with equal scaling ratios in the vertical and horizontal directions.

Also, the matrix

(Equation 5) Satisfies | det A | = | ad-bc | = 1,
Decomposition for rotation or mirroring, oblique axis transformation, unimodular diagonal transformation, if det A = 1, for example,
It becomes like the following formula (2).

(Equation 6) If det A = −1, for example, the following (3)
It looks like an expression.

(Equation 7)

Further, by using these together, a general reversible matrix

(Equation 8) On the other hand, in the case of det A> 0, it can be decomposed into positive expansion / contraction, oblique axis conversion, and rotation as shown in Expression (4).

[Equation 9] When det A <0, positive expansion / contraction, oblique axis conversion, and decomposition into reflection can be performed as shown in equation (5).

(Equation 10)

Further, the exchange relation shown in the equation (6)

[Equation 11] It is also possible to decompose the order of scaling and diagonal axis transformation using, and by considering the transposed matrix of the above equations,
Various decompositions are possible by exchanging the order of rotation (or reflection) and scaling / oblique axis conversion.

A rotation component, a reflection component, an oblique axis transformation component, a scaling component, etc. can be separated from the geometric transformation given by these decomposition methods.

The flow of processing for executing the decomposition based on the symmetry of interest from the given geometric transformation using the above matrix decomposition will be described in detail. Below is the given matrix

(Equation 12) , The matrix with symmetry of interest is

(Equation 13) , A matrix that satisfies A = A _i A _e , is simpler than A, or has a symmetry different from A _i

[Equation 14] Expressed by

FIG. 5 is a flow chart for explaining the flow of the symmetric decomposition process for the positive scalar multiplication in the conversion decomposition unit. A general invertible matrix A is a positive scalar matrix, that is, a diagonal matrix A having equal positive diagonal elements.
and _i, absolute value of the determinant will be described a flow of process of decomposing so as to represent as the product A _i A _e of the first matrix A _e. The following processing implements the matrix decomposition shown in equation (1).

The value of the matrix A is read in S41, and the coefficient p of the scalar matrix is calculated in S42. The coefficient p is (| ad
-Bc |) ^1/2 should be calculated. In S43, the calculation result of S42 is set in the scalar matrix A _i . That is, a
_i and d _i are p, and b _i and c _i are 0. Furthermore, S44
The element of the matrix A _e whose absolute value of the determinant is 1 is calculated at. That, a _e is a / p, b _e the b / p, c _e is c / p,
d _e may be d / p. Thus the matrix A becomes

(Equation 15) Is decomposed like. Here, focusing on the symmetry of the scalar multiplication, the scalar matrix is separated.

FIG. 6 is a flow chart for explaining the flow of the symmetric decomposition process for the linear conversion in which the absolute value of the determinant is 1 in the conversion decomposition unit. Here, a general invertible matrix A is represented as a product A _i A _e of a matrix A _i whose determinant has an absolute value of 1 and a positive scalar matrix, that is, a diagonal matrix A _e having equal positive diagonal components. The flow of the process of disassembling is shown. This processing is almost the same as the processing shown in FIG. 5, but the components of the matrix A _i whose absolute value of the determinant is 1 are calculated in S53, and the components of the scalar matrix A _e are calculated in S54. Thus the matrix A becomes

(Equation 16) Is decomposed like. Thus, the absolute value of the determinant is 1
It can also be separated using the symmetry for the linear transformation of.

Next, the matrix A whose absolute value of the determinant is 1 is decomposed into the product A _i A _e of the scaling A _i and the product A _e of the oblique axis transformation and rotation (or reflection). The flow of processing to be performed will be described. The following processing implements the matrix decomposition shown in equations (2) to (3).

FIG. 7 is a flow chart for explaining the flow of processing for separating the linear transform whose absolute value of the determinant is 1 in the transform decomposition unit by the symmetry regarding the scaling. The value of the matrix A is read in S61, and the intermediate results ρ and ι are calculated in S62. In S63, the calculation result of S62 is set in the diagonal matrix A _i representing the scaling. That is, a _{i is set} to 1 / ρ, d _i is set to ρ, and b _i and c _i are set to 0.

Next, the matrix A _e is set. First, S64
Then, the rotation component is calculated and set in the matrix Ae. That, a _e is -d / ρ, b _e the c / ρ, c _e is c / ρ, d
_e may be d / ρ. Further, in S65, the component of oblique axis conversion is applied to Ae and updated. That is, i _e to _e
Adding, it may be added to Iotad _e to b _e. As a result, the scaling component is set in the matrix A _i, and the component made up of the product of the remaining oblique axis transformation and the rotation is set in the matrix Ae. As a result, the scaling component can be separated.

FIG. 8 is a flow chart for explaining the flow of processing for separating the linear transformation whose absolute value of the determinant is 1 in the transformation decomposition section by the symmetry about the rotation. Each step is almost the same as in FIG. 7, but here, S7
In step 3, the rotation component is set in the matrix A _i , and S74 and S75
Then, the matrix A _e is set with the component consisting of the product of the remaining oblique axis transformation and the scaling. As a result, the rotation component can be separated.

FIG. 9 is a flow chart for explaining the flow of processing for separating the linear transformation whose absolute value of the determinant is 1 in the transformation decomposition section by the symmetry about the oblique axis transformation. Each step is almost the same as that in FIG. 7 described above, but here, the oblique axis transformation component is set in the matrix A _i in S83, and the component formed by the product of the residual rotation and scaling is added in the matrix A _e in S84. Set. As a result, the oblique axis conversion component can be separated.

In each of the above examples, the matrix A is the matrix product A _i A
_{Although the} decomposition is performed so as to have the symmetric component A _i of interest at the head by decomposing into _e , it is also obvious that the decomposition can be performed so as to have the symmetric component at the end.

By combining the above processes into a modular form, the "scalar multiplication" and "the absolute value of the determinant is 1 for the linear transformation A represented by a general reversible matrix.
Transformation A _i with specified symmetry among symmetries such as "linear transformation of", "scaling", "rotation", "reflection", "oblique axis transformation", and the original transformation A It is possible to realize a conversion decomposition unit that decomposes into a simplified conversion A _e .

Next, by integrating the operations of the respective parts described above, the symmetry determination part 2 and the conversion decomposition part 3 are combined with the instruction changing part 4.
An example of processing by decomposition of geometric transformation instructions realized by combining with the above and combination of instructions having symmetry will be described. FIG. 10 is a flowchart showing an example of the flow of processing for changing an instruction sequence by combining geometric transformation instructions with instructions having symmetry. In S91, it is determined whether or not there is any instruction stored in the instruction storage unit 1 that has not been read yet. If there is no command that has not been read, the process ends. One instruction is read in S92. For the sake of explanation, this read instruction is designated as X. In S93, it is determined whether the instruction X is a geometric conversion instruction. If it is a geometric conversion command, the process returns to S91, and if not, the process proceeds to the next S94. S
At 94, the symmetry determination unit 2 determines whether or not the instruction X has symmetry. If there is no symmetry, the process returns to S91, and if it has symmetry, the process proceeds to the next S95. In S95, the previously determined symmetry P (for example, "rotational symmetry") is set.
In S96, it is determined whether the instruction processed immediately before the instruction X is a geometric conversion instruction. For the sake of explanation, the instruction immediately before this is represented by Y. If the instruction Y is not the geometric conversion instruction, the process returns to S91, and if it is the geometric conversion instruction, the next S97 is executed.
Proceed to. S97 based on symmetry P specified geometric transformation instruction Y in, decomposes into a geometric transformation L _e was omitted L _i from the geometric transformation L _i and instruction Y with symmetry by converting decomposition unit 3. At this time, the conversion decomposition unit 3 calls for the scalar multiplication shown in FIGS. 5 to 9 and the absolute value of the determinant shown in FIGS. It is a processing procedure. In S98, the instruction changing unit 4 changes the geometric transformation instruction L _e and the instruction X with reference to the symmetry determination unit 2.

An example of the above-mentioned operation will be described below using a specific example. Generally, F (fL) = | det L ′ | F (f) L ′ F (f) L = | det L ′ | F (fL ′) L ′ = ^t L ⁻¹ for Fourier transform and linear variable transform There is an exchange relationship. Here, f is the original image, F is the Fourier transform, and ^t L is the transposed matrix of L.

FIG. 11 is an explanatory diagram showing a concrete example of the order exchange of Fourier transform and scaling. The diagonal matrix in the figure represents the coordinate transformation represented by this diagonal matrix, that is, the scaling. In FIG. 11, step 1 is a step 2 in which the scaling instruction is decomposed into vertical shrinking and horizontal scaling.
The horizontal enlargement and the Fourier transform are exchanged in order, and the Fourier transform and the reduction are replaced.

FIG. 12 is a schematic diagram showing the result of processing by order exchange between the Fourier transform and the scaling in FIG. 12A shows the result of the process corresponding to the first procedure in FIG. 11, and FIG. 12B shows the result of the process corresponding to the procedure modified in step 1. Shows the results of processing corresponding to the procedure modified in step 2.

FIG. 13 is an explanatory diagram of order exchange between enlargement and Fourier transform. The image obtained by the Fourier transform is an image representing the frequency component of the original image. When the original image is reduced, the frequency components that can be represented by the reduced image are actually the same as those of the original image, and therefore the size of the image after the Fourier transform does not change. On the other hand, when enlarged, the amount of information in the original image does not increase,
Since only a rough image is obtained as a whole, high frequency components are removed. Therefore, the image after the Fourier transform is reduced.

By utilizing this property, when the Fourier transform is applied after the enlargement, the processing order is exchanged as shown in FIG. 13 and the enlargement processing is replaced by the reduction processing to improve the processing speed. You can That is, FIG.
As shown in FIG. 2 (C), the image to which the Fourier transform is applied is a reduced image and is not a large enlarged image as shown in FIGS. 12 (A) and 12 (B), so that the processing time can be shortened. Is. Also, the amount of memory required for processing can be reduced.

FIG. 14 is an explanatory diagram of an example of the symmetry table. The symmetry regarding the expansion of the Fourier transform described above can be held in the symmetry determination unit 2 as the symmetry table shown in FIG. 14, for example. In this symmetry table, the Fourier transform has a symmetry about scaling, and if the immediately preceding geometric transformation command is scaling, the scaling factor is checked, and if the scaling factor is greater than 1, that is, if scaling is performed, the scaling process is performed. Is changed to a reduction process in which the magnification is the reciprocal of the enlargement ratio, and the processing order with the Fourier transform is exchanged.

FIG. 15 is an explanatory diagram of an example of changing the instruction sequence by disassembling the geometric transformation when the coordinate transformation and the Fourier transformation are connected. In FIG. 15, step 1 is performed by rotating (or mirroring) the geometrical transformation, dividing it into the oblique axis transformation, and step 2; exchanging the enlargement component of the enlargement / reduction with the Fourier transform.

The process shown in FIG. 15 will be described in more detail with reference to the flowchart shown in FIG. FIG.
FIG. 16 is an explanatory diagram of changes in the contents of the instruction storage unit in the example shown in FIG. 15. As shown in FIG. 16A, the instruction storage unit 1 stores a sequence of processes of linear transformation and Fourier transformation. The instructions appearing in this instruction sequence are processed in order. The first instruction, the linear transformation, is skipped in S93,
The next instruction, the Fourier transform, is captured. In S94, the presence / absence of symmetry of the Fourier transform is checked to determine whether the symmetry determination unit 2
Contact The symmetry determination unit 2 refers to a symmetry table representing the symmetry of the instruction, for example, as shown in FIG. 14, and determines that the Fourier transform has symmetry regarding scaling. In S96, it is determined that the immediately preceding instruction is a linear conversion, and the process proceeds to S97. In S97, the conversion decomposition unit 3 separates the linear conversion into the scaling components based on the symmetry about the scaling, and the command changing unit 4 stores the command sequence shown in FIG. Change to the instruction sequence shown in FIG. In Figure 15, the linear transformation is rotated, skewed,
Although broken down into enlargements, FIGS. 16B and 16C show rotation and skew as one linear transformation. S98
Then, the instruction changing unit 4 again refers to the symmetry table as shown in FIG. 14 which represents the symmetry of the symmetry determination unit 2 to refer to the instruction sequence shown in FIG. Change to a column. In this manner, the processing of the instruction sequence stored in the instruction storage unit 1 shown in FIG. 16A is completed, and the instruction sequence changing process is completed.

Next, an example of the above operation will be described using another specific example. FIG. 17 is an explanatory diagram showing an example of changing an instruction sequence including coordinate conversion and histogram generation processing.
In FIG. 17, step 1 decomposes the geometric transformation into a transformation of which the absolute value of the determinant is 1 and the scalar multiplication. Step 2 cancels the transformation of which the absolute value of the determinant is 1 and replaces the scalar multiplication with the histogram processing. Since the histogram of the image is not affected by the position of the pixel, the geometric conversion that does not change the area, that is, the conversion in which the determinant is 1 is unnecessary. The process is simplified by omitting this unnecessary process. Further, the process of changing the area by the geometric transformation is further simplified by not performing the conversion to the image as the two-dimensional data but performing it on the graph of the result of the histogram of the one-dimensional data.

The conversion for making the area of the histogram generation processing invariant, that is, the symmetry for the linear conversion in which the absolute value of the determinant is 1 and the symmetry for the scalar multiplication are, for example, the symmetry shown in FIG. The table may be held in the symmetry determination unit 2 as a table. This symmetry table shows that the histogram generator has symmetry about the transformation that makes the area invariant, and that the geometric transformation instruction may be deleted if the previous geometric transformation instruction does not change the area. There is. Histogram generation has symmetry about scalar multiplication, and if the previous geometric transformation instruction is scalar multiplication, histogram generation is executed prior to geometric transformation, and the resulting histogram graph is displayed. This means that the process can be changed to the process of multiplying the ratio of the scalar multiplication.

The process described with reference to FIG. 17 will be described in more detail with reference to the flowchart shown in FIG. 18 is shown in FIG.
7 is an explanatory diagram of changes in the contents of the instruction storage unit in the example shown in FIG. In the instruction storage unit 1, as shown in FIG.
The sequence of processes of linear conversion and histogram generation is stored. The instructions in the instruction sequence shown in FIG. 18A are sequentially processed. The first instruction, the linear transformation, is skipped in S93 and the next instruction histogram generation is captured.
In step S94, the symmetry determination unit 2 is queried for the presence / absence of symmetry in histogram generation. The symmetry determination unit 2 refers to the symmetry table showing the symmetry of the instruction, for example, referring to the symmetry table shown in FIG. 14, and determines that the histogram generation has the symmetry regarding the conversion that does not change the area and the scalar multiplication. In S95, the area invariant transformation and the scalar multiplication are set as the symmetry P. In S96, it is determined that the immediately preceding instruction is a linear conversion, and the process proceeds to S97. In S97, the conversion decomposition unit 3 decomposes the linear conversion into the scalar multiplication and the conversion that does not change the area, based on the symmetry P, and the instruction changing unit 4 is stored in the instruction storage unit 1 as shown in FIG. The instruction shown is changed to the instruction sequence shown in FIG. In S98, the instruction changing unit 4 again refers to the symmetry table shown in FIG. 14 representing the symmetry of the symmetry determination unit 2, erases the conversion without changing the area, and exchanges the scalar multiplication and the histogram generation, The instruction sequence shown in FIG. 18 (B) is shown in FIG. 18 (C).
Change to the instruction sequence shown in. As described above, the processing of the instruction sequence stored in the instruction storage unit 1 is completed, and the instruction sequence changing process is completed.

Next, the operation when the instruction string includes a linear filter will be described. FIG. 19 is an explanatory diagram of changes in the contents of the instruction storage unit when linear filter processing is included. The instruction storage unit 1 can store a linear filter, as shown in FIG. Here, the parameter of the filter instruction is g representing the kernel of the filter (for example, a small image of about 3 × 3 pixels), and a quadratic expression representing the geometric transformation for convenience of order exchange of the filter and the geometric transformation described later. It consists of two matrices L and. The meaning of the data structure shown in FIG. 19A is that kernel g is transformed by geometric transformation L, and the kernel element is multiplied by the absolute value of the determinant of geometric transformation L | det L
Represents the convolution with | gL.

20 and 21 are explanatory views of an example of a method of realizing geometric transformation of a linear filter into a kernel. In FIG. 20, each area delimited by a thin line is an area corresponding to a pixel, and the 3 × 3 area indicated by a thick line is an area corresponding to the core of the linear filter after geometric transformation. As shown in FIG. 20, the pixel value after conversion can be determined by calculating the weighted average of the pixel values based on the area ratio of the areas delimited by the thin lines included in the areas delimited by the thick line. it can. This method is called the projection method. Alternatively, in FIG. 21, white circles on the grid points are pixel positions. The bold grid points are the pixels of the core of the linear filter. As shown in FIG. 21, when the position of the pixel corresponding to the core of the linear filter is not on the grid point, it can be obtained from the pixel values at the neighboring grid points by, for example, the linear interpolation method. By such an affine transformation method or the like, it is possible to realize the geometric transformation of the linear filter into the kernel.

If it is not necessary to perform geometric transformation on the kernel when designating the filter instruction, the matrix representing the geometric transformation to the kernel is expressed as the unit matrix id as shown in FIG. 19 (B). do it.

It is now assumed that, as shown in FIG. 19C, a sequence of filter instructions and geometric transformation instructions is stored in the image storage unit 2. At this time, the instruction changing unit 4 exchanges the order of the filter and the geometric transformation as shown in FIG. 19 (D), and further exchanges the geometric transformation with the parameter representing the geometric transformation to the kernel as shown in FIG. 19 (E). By acting
The processing order of the filter instruction and the geometric transformation instruction is exchanged.

FIG. 22 is a flow chart showing the flow of processing for exchanging the order of the filter and the geometric transformation.
The instruction changing unit 4 changes the instruction sequence stored in the instruction storage unit 1 according to the following procedure. In S101, it is determined whether there is any instruction stored in the instruction storage unit 1 that has not been read yet. If there is no command that has not been read, the process ends. In S102, one instruction is read. For the sake of explanation, this read instruction is represented by X. S103
Then, it is determined whether the instruction X is a filter instruction. If it is not a filter command, the process returns to S101, and if it is a filter command, the process proceeds to the next S104. In S104, it is determined whether there is an instruction that has not been read again. In S105, the next instruction is read. This read instruction is represented by Y.
In S106, it is determined whether the instruction Y is a geometric conversion instruction.
If it is not a geometric conversion command, the process returns to S101, and if it is a filter command, the process proceeds to S107. In S107, the instruction storage unit 1
Swap the order of the instructions corresponding to instruction X and instruction Y above.
In S108, the matrix of parameters representing the geometric transformation of the kernel of the filter instruction is multiplied by the matrix of parameters of the geometric transformation instruction Y from the right.

The processing of S107 and S108 will be described with reference to FIG. It is assumed that an instruction sequence is stored in the instruction storage unit 1 as shown in FIG. 19C, a filter instruction is fetched as the instruction X, and a linear conversion instruction is fetched as the instruction Y. At this time, in S107,
As shown in FIG. 19D, the linear transformation instruction and the filter instruction are exchanged, and in S108, the matrix L representing the geometric transformation of the kernel of the filter instruction is changed to LM. In this way, the instruction changing unit 4 can exchange the order of the filtering process and the geometric transformation by the matrix.

FIG. 23 is an explanatory diagram of changes in the contents of the instruction storage unit when the processing for integrating the geometric transformation instructions of the same kind is executed on the instruction sequence including the filter instruction. Here, the case where the linear conversion instructions are integrated is shown. That is, as shown in FIG. 23 (A), (1) Coordinate conversion L ₀ is applied (2) Core g ₀ filter is applied (3) Coordinate conversion L ₁ is applied (4) Core g ₁ filter is applied ( 5) A series of processing instructions for performing the coordinate conversion L ₂ repeatedly perform the conversion based on the exchange relationship between the filter and the coordinate conversion and the integration of the same kind of processing.

First, the fourth filter process and the fifth coordinate conversion process are exchanged, and the conversion matrix of the filter process is set to L _{2 as} shown in FIG. 23 (B). 3rd and 4th at this point
Secondly, since coordinate conversion processing, which is the same kind of processing, is continuous, these two processings are integrated as shown in FIG. Further, the second filter process and the newly generated third linear conversion process are exchanged, and the conversion matrix of the filter process is set to L ₁ L ₂ as shown in FIG. FIG.
In (D), since the first and second linear conversion processes are continuous, these are integrated as shown in FIG. 23 (E).

By the processing up to this point, an instruction sequence of (1a) applying the coordinate transformation L ₀ L ₁ L ₂ (2a) applying the filter of the kernel g ₂ (4a) applying the filter of the kernel g ₃ is obtained. However, the kernel g ₂ can be obtained by (1) applying the coordinate transformation L ₁ L ₂ to the kernel g ₀ , and (2) multiplying the filter value by | det L ₁ L ₂ |. Similarly, the kernel g ₃ is obtained by (1) applying the coordinate transformation L ₂ to the kernel g ₁ and (2) multiplying the filter value by | det L ₂ |. Since the subsequent conversion is a process of simplifying the coordinate conversion to the core of the next filter processing instruction, FIG.
3 (F) and (G) will be described later.

Next, the flow of processing for simplifying the coordinate conversion of the filter processing instructions appearing in the instruction sequence stored in the instruction storage unit 1 into the kernel will be described. FIG. 24 is a flowchart showing the flow of processing for simplifying the coordinate conversion parameter of the filter processing instruction appearing in the instruction sequence. The instruction changing unit 4 changes the instruction sequence stored in the instruction storage unit 1 according to the following procedure.

In S111, it is determined whether or not there is an instruction stored in the instruction storage unit 1 that has not been read yet. If there is no command that has not been read, the process ends. In S112, one instruction is read. For explanation,
This read instruction is represented by A. Command S in S113
Is a filter command. If it is not a filter command, the process returns to S111, and if it is a filter command, the next S114.
Proceed to. In S114, it is determined whether the coordinate conversion parameter of the filter command A is different from the unit matrix. If it is the unit matrix, the process returns to S111, and if it is not the unit matrix, S115.
Proceed to. In S115, it is determined whether the filter command A has isotropic (that is, rotation or reflection) symmetry. If the filter command A does not have isotropicity, the process returns to S111, and if it does, the process proceeds to the next S116. In S116, the matrix L of the coordinate conversion parameters is decomposed into the rotation / reflection part L _i and the expansion / contraction oblique axis conversion part L _e . In S117, it replaces the coordinate transformation parameter L in scaling-oblique transverse conversion portion L _e. Then, the process returns to S111 again.

The above processing will be described with reference to FIG. As described above, the instruction sequence shown in FIG. 23A stored in the image storage unit 1 based on the processing procedure described in FIG.
3 (E) is converted into the instruction sequence. Then, FIG.
The processing procedure described in step 1 is continuously executed. Where n ₀
And the kernel g ₁ are both assumed to be isotropic. For the sake of explanation, it is assumed that the two matrices L ₁ L ₂ and L ₂ appearing as the coordinate conversion parameters of the filter are different from the unit matrix.

The instruction changing unit 4 determines that the coordinate conversion parameters L ₁ L _{2 of the} filter processing appearing second in the instruction sequence shown in FIG. 23E stored in the instruction storage unit 1 are not the unit matrix. . Also, the kernel g _{0 of the} second filtering process
The symmetry determination unit 2 determines that is has isotropic symmetry.
Therefore, in S116 of FIG. 24, the transformation decomposition unit 3 decomposes the matrix L = L ₁ L ₂ into a product L _e of rotation or reflection L _i and expansion / contraction and oblique axis transformation to obtain L = L _i L _e . . S11
In FIG. 7, the instruction changing unit 4 is based on this decomposition.
The coordinate conversion parameters L ₁ L ₂ of the instruction sequence shown in FIG. 3 (E) are replaced with L ₃ = L _e to obtain the instruction sequence shown in FIG. 23 (F).

Similarly for the third filter processing in the instruction sequence shown in FIG. 23 (F), the conversion / decomposition unit 3 uses the matrix L =
Rotation or reflection L _{i of} L ₂ and product L _e of scaling and oblique axis conversion
To L = L _i L _e . In step S117, the instruction changing unit 4 determines the coordinate conversion parameter L based on this decomposition.
₂ is replaced with L ₄ = L _e to obtain the instruction sequence shown in FIG.

Since the instruction changing unit 4 makes the above-mentioned changes to the instruction sequence, the conversion to the kernel is limited to the scaling and the oblique axis conversion. For example, the size of the array by the rotation of the kernel shown in FIG. The increase can be avoided.

The image processing function as described above can be used in various systems that handle digital images, such as a document editor, a drawing tool, an image transmission device and a copying device.

In the above description, an image in a two-dimensional space is used for the sake of simplicity. However, in coordinate conversion in a three-dimensional or four-dimensional space including a time space such as a graphic. Can be similarly configured.
The space in which the image is mapped may be not only one-dimensional density but also a three-dimensional space including colors.

[0077]

As is apparent from the above description, according to the present invention, by decomposing the affine transformation based on different symmetries, one is processing other than coordinate transformation, When there is a process with symmetry or invariance for coordinate transformation, the symmetry can be used to change the coordinate transformation to be performed to a simpler one to shorten the processing time and improve the processing quality. . In addition, by decomposing general affine transformation into products of specialized affine transformations such as scaling and rotation, the best algorithms prepared for specialized affine transformations can be freely used to speed up processing and increase the processing speed. The quality can be improved. Furthermore, by determining the symmetry of the geometric transformation of image processing different from the geometric transformation, and subjecting the geometric transformation to decomposition based on the symmetry according to the result, the image processing instruction sequence including the geometric transformation is There are various effects such as efficient exchange integration.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of a system including an embodiment of an information processing device of the present invention.

FIG. 2 is a block diagram showing an example of an image input unit.

FIG. 3 is a block diagram illustrating an example of an image output unit.

FIG. 4 is an explanatory diagram showing the concept of instruction change processing by an instruction change unit in the embodiment of the information processing apparatus of the present invention.

FIG. 5 is a flowchart illustrating a processing flow of decomposition by positive symmetry regarding positive scalar multiplication in a conversion decomposition unit.

FIG. 6 is a flowchart illustrating a flow of a symmetric decomposition process for a linear conversion in which the absolute value of the determinant is 1 in the conversion decomposition unit.

FIG. 7 is a flowchart illustrating a flow of a process of separating a linear transform whose absolute value of a determinant is 1 in a transform decomposition unit by symmetry regarding scaling.

FIG. 8 is a flowchart illustrating a flow of a process of separating a linear transform whose absolute value of a determinant is 1 in a transform decomposition unit by symmetry about rotation.

FIG. 9 is a flowchart illustrating a flow of processing for separating a linear transformation whose absolute value of the determinant is 1 in the transformation decomposition unit with symmetry about the oblique axis transformation.

FIG. 10 is a flowchart showing an example of a flow of processing for changing an instruction sequence by combining a geometric transformation instruction with an instruction having symmetry.

FIG. 11 is an explanatory diagram showing a specific example of a Fourier transform and order exchange of scaling.

FIG. 12 is a schematic diagram showing a result of processing by order exchange between Fourier transform and scaling in FIG.

FIG. 13 is an explanatory diagram of order exchange between enlargement and Fourier transform.

FIG. 14 is an explanatory diagram of an example of a symmetry table.

FIG. 15 is an explanatory diagram of an example of changing an instruction sequence by disassembling geometric transformation when coordinate transformation and Fourier transformation are connected.

16 is an explanatory diagram of changes in the contents of the instruction storage unit in the example shown in FIG.

FIG. 17 is an explanatory diagram showing an example of changing an instruction sequence including coordinate conversion and histogram generation processing.

18 is an explanatory diagram of changes in the contents of the instruction storage unit in the example shown in FIG.

FIG. 19 is an explanatory diagram of changes in the contents of the instruction storage unit when linear filter processing is included.

FIG. 20 is an explanatory diagram of an example of a method of realizing geometric transformation of a linear filter into a kernel.

FIG. 21 is an explanatory diagram of another example of a method of realizing geometric transformation of a linear filter into a kernel.

FIG. 22 is a flowchart showing the flow of processing for performing order exchange between a filter and geometric transformation.

FIG. 23 is an explanatory diagram of changes in the contents of the instruction storage unit when a process of integrating geometric transformation commands of the same type is executed with respect to an instruction sequence including a filter instruction.

FIG. 24 is a flowchart showing the flow of processing for simplifying the coordinate conversion parameter of the filter processing instruction appearing in the instruction sequence.

FIG. 25 is a conceptual diagram showing a conventional affine conversion process.

[Explanation of symbols]

1 ... Command storage unit, 2 ... Symmetry storage unit, 3 ... Transformation decomposition unit,
4 ... Command change unit, 5 ... Image storage unit, 6 ... Image processing unit, 7
... image input section, 8 ... instruction input section, 9 ... image display section, 10
... image output unit, 11 ... control unit, 12 ... information processing unit.

Claims (4)

[Claims] 1. An instruction storage unit for storing an instruction sequence composed of a plurality of image processing instructions, an instruction symmetry determination unit for determining symmetry regarding geometric transformation of each instruction in the instruction sequence, and the instruction storage. A geometric transformation instruction decomposing means for decomposing the instruction for geometric transformation stored in the means, and an instruction changing means for changing the instruction sequence stored in the instruction storage means,
The information processing apparatus, wherein the instruction changing unit changes the instruction sequence by utilizing decomposition of the conversion instruction by the geometric conversion instruction unit based on the symmetry determined by the instruction symmetry determination unit. 2. The instruction symmetry determination means determines symmetry for positive scalar multiplication or symmetry for coordinate transformation in which the absolute value of a determinant is 1, and the geometric transformation instruction decomposition means determines coordinate transformation. The information processing apparatus according to claim 1, wherein is decomposed into a product of a positive scalar multiplication and a coordinate transformation in which the absolute value of the determinant is 1. 3. The instruction symmetry determination means determines symmetry regarding rotation, reflection, oblique axis transformation or unimodular diagonal transformation, and the geometric transformation instruction decomposition means determines absolute values of determinants. The information processing apparatus according to claim 1, wherein the coordinate transformation of 1 is decomposed into a product of rotation or reflection, oblique axis transformation, and unimodular diagonal transformation. 4. The information processing apparatus according to claim 1, wherein the instruction changing unit is capable of exchanging a processing order of a linear filter or a morphological filter and a coordinate conversion.