JPH036674A - Picture distortion correcting device - Google Patents

Abstract

PURPOSE:To correct a picture containing such distortion as distance, rotation, etc., into a desired form by setting the coordinates and the increment of a reading start point and controlling the address of the reading point of an input frame memory based on the addition arithmetic result using the set coordinates and increment. CONSTITUTION:The density value data obtained by quantizing the video image signal is stored in the coordinates corresponding to the position on an image pickup screen of an input frame memory Fi. The memories R and Rd store the reading point coordinates Xs and Ys given from the memory Fi and the increments DELTAx and DELTAy used for obtaining the next reading point coordinates based on the coordinates Xs and Ys. Then the memories R and Rd which reads successively the density value data out of the memory Fi based on the reading point coordinates obtained by the addition arithmetic using the coordinates Xs and Ys and the increments DELTAx and DELTAy to be stored. Thus it is possible to correct the distance distortion of photographed picture and the distortions of a rotary shift component, etc., and to reproduce a desired form.

Description

[Detailed Description of the Invention] [Field of Industrial Application] The present invention is suitable for capturing images of a wide range of areas such as roads or parking lots, and performing various detection and analysis processes using the reproduced images. The present invention relates to a distortion correction device.

[Conventional technology]

In general, when roads, parking lots, production lines, etc. are photographed with a television camera, and traffic flow or the number of parked cars is detected or monitored from the images, the imaged area is wide, so the image may not be visible near the television camera. The image appears larger, and the farther away it is, the smaller the image appears.

Figure 2 (a) is a view of the parking lot viewed from directly above.
The thick lines in the figure are marking lines to indicate parking areas for individual vehicles, and the arrows indicate the direction of vehicle entry.In service areas such as expressways and parking areas, diagonal lines as shown in the figure are used to improve the efficiency of crowd flow. Parking areas are often designated.

Figure (b) shows an image of the parking area E, which includes a series of parking areas ■ to ■, in the parking lot mentioned above, taken from diagonally above. The straight lines that occur and are parallel to each other are convergence points VP,, VP2
, and a rotational movement component also occurs depending on the installation position of the television camera.

When managing traffic flows and parking lots by observing images obtained by photographing roads or parking lots in this manner, distortion of the images occurs due to such perspective factors and rotational movement components.

In order to remove such distortions, it is desirable to reproduce a parking area that has been imaged in a distorted shape, for example, an apparent upper trapezoid, on the image memory and screen as a rectangular shape. For example, the imaged screen of the parking area E consisting of the parking areas ■ to ■ as shown in b) is reproduced in the image memory and on the screen as each of the parking areas ■ to ■ as a rectangular area as shown in (C) of the same figure. It is desirable to do so.

One of the conventional methods for performing such processing is to divide the imaged area into a neighboring area with less image distortion and a neighboring area with more image distortion, as shown in Figure 11, which takes an image of a road from its extension direction. For the far region with large distortion, reference points A, A2 are placed at the edges of the processing target region.
．． -, , -, , B,, B2°, and create a trapezoidal area surrounded by these reference points, for example, trapezoid AI A2
Lascous scan during reproduction is controlled so that BI B2 becomes rectangular on the reproduced image, or a trapezoidal area on the captured image is converted into a rectangular shape by known affine transformation.

Although such processing causes some deformation in the shape of objects such as cars, such deformation is not a big problem when performing traffic control or parking lot management, and image distortion can be avoided. More emphasis is placed on the effect of easing processing through correction.

[Problem to be solved by the invention]

However, when controlling the scanning movement for reading the frame memory as described above, uniform raster scanning becomes impossible, which poses an obstacle in terms of processing time and ease of processing, and affine transformation Generally, floating-point matrix operations are required when performing enlargement processing by applying methods such as The disadvantage is that the configuration is extremely complex.

The present invention makes it possible to correct distortions such as perspective and rotation components of a captured image by using simple addition calculation results, and reproduces image distortions in a desired shape such as a rectangle. The purpose is to obtain a correction device.

[Means to solve the problem]

As shown in the principle diagram of FIG. 1, there is an input frame memory Fi that stores captured image data, and a coordinate XS of a reading start point from the input frame memory.

Memories R and R that store Ys and increments Δx and Δy for obtaining the next readout point coordinates based on these coordinates, respectively;
d, the readout point coordinates obtained by an addition operation using the coordinates of the readout start point stored in the memory, and the increment; An image distortion correction device was constructed.

In another embodiment, M defines the range of scanning operations in the output frame memory.

An image distortion correction device including an N count setting section was constructed.

Note that the above address conversion means C includes multiplexers 11 and 12, adders 21 and 22, x address register 13 . It consists of a Y address register 14 and an address decoder 24.

, [Function] In order to facilitate understanding of the present invention, FIG. 1, which shows the principle of the present invention, shows that image distortion occurs when the density value data from the input frame memory Fi is transferred to the output frame memory FO. It is shown that correction is performed. Reading from this output frame memory is performed by raster scanning.

It is also assumed that density value data is stored in the input frame memory and the output frame memory in a pattern corresponding to the captured image and the image after distortion correction. and,
The addresses of these input frame memories and output frame memories are represented by coordinates on the horizontal axis and coordinates on the vertical axis, and the notation of ``coordinates'' is used instead of ``address'' as appropriate.

Note that as mentioned above, the output frame memo + JFo is shown as an example to make it easier to understand, and the density value data read from the input frame memo + JF i is directly input to an image processing processor, etc. using the converted address. It may also be supplied to a processing device.

As shown in FIG. 2(a) cited above, the density value data obtained by quantizing the video image signal of a parking area E including a plurality of parking areas ■ to ■ of a parking lot, for example, is supplied via a selector S. The input frame memo JFi is determined by the write address generated based on the horizontal and vertical synchronization signals of the video camera
is stored at the coordinates corresponding to the position on the imaging screen, but by enlarging this storage pattern on the input frame memory, the third
Shown in Figure (a).

Since the density value data of the hatched dividing line, which serves as a reference for processing according to the present invention, is captured by a fixed video camera, it is stored in a fixed pixel on the input frame memory.

From this, the coordinates of the readout start point and the increment for obtaining the next readout point coordinates used in the present invention are naturally the same when imaging the same area, and the calculations described later will adjust the readout start point coordinates and the increment for obtaining the next readout point coordinates to match the actual image. It is set above.

When the density value data stored in this output frame memory F○ is read out by raster scanning operation, the target area p1 in FIG. 3(a) corresponding to the above-mentioned parking area etc.
~p, and p1'~p in the same figure (b).

It is reproduced as a desired shape such as a rectangle as shown in the figure.

For this purpose, when transferring the density value data stored in the input frame memory F i in the pattern shown in FIG. 3(a) to the output frame memory FO, for example,
The density value data read from the read address of the input frame memory Fi determined by the address conversion means shown in FIG. 1 is stored at the corresponding address of the output frame memory.

In this figure, ○ and * correspond to pixels on the input frame memory, and this * indicates a pixel in which the density value data of the line dividing the parking area shown by hatching is stored. .

In order to convert such a pattern, the read start point coordinates (Xs, Ys ) and an address conversion means C for converting into a read address of the input frame memory using the increments Δx and Δy stored in the incremental memory Rd.

Next, the readout start point coordinates (Xs, Ys) and the increments Δx, Δy will be explained using FIG. 3.

Set the processing start point for the target area to be corrected in the image, in this example the parking area ■.This starting point ° is determined by the output frame memo 'J' in consideration of the raster scan operation described above.
At the point where density value data is first written on Fo, that is, M=1. The coordinates (
8. The point p located at 1) is determined as the processing start point for the target area (■).

As shown in FIG. 3(b), the density value data of these 91 points is stored in the output frame memory F o M=1°N=1
Points 92, 93, and 94, which are the ends of the line segment that surrounds this target area ■, are written at the origin, which is the coordinate M8 of the output frame memory Fo, on the M axis where N=1. 9
27 points, 937 points on the N axis with coordinates M-1°N=11, and points p and ' with coordinates M=8, N=11, respectively.

Therefore, when transferring the density value data from the input frame memory F1 to the output frame memory Fo, the following (1) is written to the write address (m, n) of the output frame memory Fo.
The read address (x, y) of the input frame memory Fi is calculated by performing an addition operation as shown in the equation.

x = INT (Xs (n) +Δx (n)
・(m-1) )3' = INT (Y s (n
) +△y(n) ・(m-1)) (1) Here, INT is an integer function that truncates the fraction below the decimal point, and is based on the fact that memory storage elements exist only at integer addresses. be.

This integerization function also has an operation in which the values on the coordinates do not change and remain the same unless the calculation result in the parentheses on the right side of equation (1) is carried beyond the decimal point.

Next, to explain the above formula, m is the value of the M axis of the output frame memo 1JFoO write address, and writing has already been done on the scanning line in Figure 4 from the origin (1, 1) of this memory. If the number of pixels is Z, then m = Mod (Z/Mmax) (Mod is the remainder, and Mmax is the maximum value of the M axis as shown in Figure 4, in other words, the number of pixels on one scanning line. )
It is.

Also, the N-axis value n of the write address of the output frame memory
corresponds to n shown in parentheses, and the value n is n=I NT (Z/Mmax) + 1, counting from the bottom with the M axis of output frame memory F○ being #1. is the number of the scan line.

Therefore, X s (n) and Y s (n) are the coordinates of the reading start point on the input 7 frame memory F i from which the density value data to be written to the coordinates (L n >) of the output frame memory are read. The coordinates of this output frame memory (
1, n) is called a write start point, and a point corresponding to the coordinates of the input frame memory from which density value data to be written to this write start point is read is called a read start point.

Note that the n value of this writing start point is m = 1 (
If the starting point is the origin of the coordinates, it is automatically set by the scanning operation.

Further, the increments Δx (n) and Δy (n) are the density value data to be sequentially written to the pixels on the scanning line to which the write start point belongs following the read start points X s (n) and Y s (n). In order to read sequentially, when the value of the M-axis coordinate of the write point of the output frame memory increases by 1, the N-axis and Y-axis when the read point on the input frame memory Fl moves from the read start point. indicates the increment of the coordinates. (This increment means the slope of the straight line determined by the ratio of ΔX and Δy.

) To find these increments, compare frame memo + J-
Points on the input frame memory must be determined that correspond to opposing points on the m=8 line corresponding to each write start point on the F oON axis.

In other words, since the number of pixels on line segment + p3 in Fig. 3(a) and the number of pixels on line segment 1)21)4 are generally different, the number of pixels on line segment 1) + p3 corresponds to each pixel on line segment 1) + p3. Line segment 1) A pixel on 2 ps cannot be defined as an existing pixel.

Therefore, the number of pixels on the line segment p + ps is expressed as the reference pixel number by A, and the number of pixels on the line segments p2 to p4 is expressed as the enlarged pixel number by B, and the respective X of points p1 and p31 and points p2 and p4 are Assuming that each number of pixels is approximately determined by the difference in coordinates on the axis, the ratio of the number of pixels on each X axis is A (Xp3-XI)l) Standard number of pixels
(2) becomes.

Here, if B/A= and this is the magnification rate, this magnification rate is the number of pixels on the line segment p+ p3 and the line segment pi I
) 4 to make the number of pixels the same, the line segment p2 is the X-axis increment for pixels p<;
) 3 in the X-axis direction, then by setting the increment between pixels in the X-axis direction on line segment 1) 2 ps to K, each pixel on line segment pI p3 The appearance of points on the corresponding line segment p2 p< can be created in a one-to-one ratio.

In addition, as mentioned above, by making the points on the line segment 1)2 p4 corresponding to the pixels on the line segment p+ p3 appear better, the width (number of pixels) between each corresponding point can also be approximated. It becomes possible to ask for it.

In other words, if we assume that the number of pixels between corresponding points on line segment p+ +)3 and line segment 1)2 ps can be determined by the difference in coordinates on the X axis as above, then the number of pixels between points p and p2 is The number of pixels in between is (Xp2Xp+).

Therefore, if this number of pixels is used as the reference number of pixels in equation (2) above, then point p3. The number of pixels between p4 is (Xp4-Xp3) i
:Become. Then, if the number of pixels of (XI)4-Xp3) is the number of enlarged pixels, the ratio of both (xp4
)/(Xp2Xp+) is a line segment p3 that makes the number of pixels on line segment p+ p2 and the number of pixels on line segment p3p4 the same.
This is the X-axis increment ΔX for the pixel p4.

Of course, we have taken only the line segment p3 p< as an example here, but for each corresponding point on the previously calculated line segment 1)2 p< (hereinafter referred to as a virtual point), the line segment p+ l)2 This ΔX is determined based on the number of pixels.

However, by setting each ΔX as an increment in the X direction, each line segment appears to have the same number of pixels as the reference pixel number, so ΔX is treated as an enlargement ratio here.

Furthermore, the line segment p2 corresponding to the pixel on the line segment p+ p3
By creating the points on p4 in an apparent 1:1 ratio, it is possible to define a straight line between each corresponding point. By advancing the readout point along each of these straight lines, for example, line segment p+ p2 can be changed to line segment p1'p2' in FIG. 3(b), which was a diagonal line in input frame memory F1'-. can be written as a straight line on the output frame memory Fo.

Reading the data on the input frame memo 'JFl along each of these straight lines is equivalent to increasing the x and y coordinate values with the slope of each straight line.

In other words, the slope of this straight line is ΔX and Δy in equation (1) above.
By selecting ΔX and Δy so that the relationship is ΔX/Δy, it becomes possible to read density value data along this straight line.

Therefore, if point p1 is the readout starting point for n=1 in FIG. 3(a), the X-axis increment ΔX(1) is the difference in the number of pixels on the X-axis between point p and p2, that is, the number of enlarged pixels (Xp2Xp+) is divided by the number of pixel intervals 7 (reference number of pixels) between p1' and p2' corresponding to points p1 and p2 on the output frame memory.

In addition, the Y-axis increment △y has the slope of the line segment Ill +)2 as △y(1)/Δx(1) = (1-5)/(8-15)
= 4/7, so Δy(1)=4/7xΔX(1).

In other words, these increments △X, Δy are input frame memo 1
This corresponds to a vector in the moving direction of the readout coordinates of pixels to be sequentially read out following the readout start point on JFi. however,
As can be seen from the above, the increments Δx and Δy are generally values that include decimals, so unless they are converted into integers as shown in equation (1) above, the coordinates for accessing the storage elements that actually exist in the memory will be can't get it.

Table 1 on the next page shows the write start point number and its coordinates for the example shown in Figure 3, and the X-axis coordinates Xs, Y-axis coordinates Ys, and Axis increment ΔX
and Y-axis increment Δy, and also shows the coordinates of the read point on the input frame memory corresponding to the M-axis value m of the write point on the output frame memory determined by these values. be.

Note that the read start point memory R shown in FIG. 1 contains the X-axis coordinate Xs and Y-axis coordinate YS of the read start point, and
Incremental memory Rd stores the above X-axis increment ΔX and Y-axis increment Δ
y is stored, and according to the present invention, the readout point coordinates are calculated by the addition operation of equation (1) using these values.

The first column of this Table 1 is the number of the reading start point, the ■ column is the coordinate of the writing start point, the ■ column and These are the X-axis increment and Y-axis increment from this reading start point, and the "M-coordinate value" in the column (■) indicates that the M-axis value of the write point in the output frame memory determined by these values is m=l. The coordinates (x, y) of the readout point of the input frame memory corresponding to each value of m sequentially increased from 1 to 8 by raster scanning until m=8, and according to the present invention, the readout point coordinates are Xs, As described above, it is calculated by the addition operation of equation (1) using the values of Ys, ΔX, and Δy.

In FIG. 3(a), the trajectory of the readout point on the input frame memory read out according to Table 1 is shown by a thin dotted line, and the origin coordinates (1) of the output frame memory F○ shown in FIG. , 1) becomes the first writing point by raster scanning as shown in Fig. 4, and the point shown in Fig. 4(a) is
#1 readout start point X5(1), Ys(1
) is written in the density value data at point p+'. Furthermore, this #1
It goes without saying that the coordinates of the read start point are read out from the read start point memory R.

At the next writing coordinate (2, 1), the value of the N axis is “1”
does not change, but the value of the M coordinate becomes 2, so the increment ΔX =
1.00. Coordinate value x = 8 + 1.00 = 9.0 obtained by adding Δy = 0.71 to the reading start point coordinates (8, 1) of the input frame memory. 'l
= 1 +Q.

Reading will be performed at the coordinates given by 71=1.71, but since the coordinate values are frame memory addresses, unless they are integers, there is no corresponding storage element.
For the Y-axis value 1.71, which is not an integer, the decimal part is rounded down and read from the coordinates (9, 1), so the value m = 2 of the M coordinate in row #1 of Table 1 above is indicates the coordinates (9, l) of this input frame memory.

When the write coordinates of the output frame memory become (3, 1) due to raster scanning, the value X of the X-axis of the read coordinates of the input frame memory is X=8+2XΔX-8+2=1
0, and the value y on the Y axis is V = 1 + 2X△y = 1 +
2 x O, 71 = 1 + 1.42 = 2.42, but the value of this Y axis is rounded down to 2 as in the above case.

In this way, the coordinates of the upper writing start point are (1, 1)
For example, if the input frame memory coordinates (8,
1) −(9,1)→(10,2)→(11,3) −
(12,3)→(13,4)→(14,5) −(15
, 5), the density value data for the eight pixels are sequentially read out, and since the coordinates of the M axis of the output frame memory increase by 1, these density value data are (1, 1).
→(2,1) −(3゜1)−(4,1)→(5,1)
It will be sequentially written to eight coordinates: -(6,1)→(7,1) -(8,1).

In this way, when writing is performed up to the last pixel of this scanning line, "l" is added to the value of the N axis, as is clear from the raster scan diagram in FIG. Since the value n of the axis becomes 2 and the value m of the M axis is reset to 1, the writing starts at the coordinates (1, 2) and moves to the reading start point #2. At this time Mmax=3
If this is specified, the scanning operation of m=1 to 8 in FIG. 3(b) will be executed.

As a result, the read start point coordinates of the input frame memory corresponding to the write start point coordinates (1, 2) X5 (2) =
7. Ys(2) -2 is read from the read start point memory R and the increment Δx(2> = 1°00 and Δy(2) = O167 is read from the increment memory Rd, as in the first line above. Reading is performed sequentially from the corresponding coordinates of the input frame memory, but as shown in Table 1, the input frame memory coordinates (7, 2
) from (8,2) → (9,3) → (10°4) → (1
1,4) → (12,5) → (13,6) → (14,6)
The density value data for the eight pixels are sequentially read out from (1, 2) to (2, 2) in the output frame memory FO
) − (3, 2) → (4, 2) → (5, 2) → (6, 2)
→ (7, 2) → (8, 2) are sequentially written to eight coordinates.

However, the reading start point of #3 (Xs(3).

The input frame memory coordinates (7°3
), as can be seen by counting the number of pixels on the trajectory of the readout points starting from this coordinate in Figure 3(a), there are only seven readout points, so the coordinate (7°3) Output the density value data of frame memory F o (1, “3)
and (2, 3), thereby increasing the number of writing pixels on the output frame memory to #1 and #2.
As in the case of the reading start point, eight frames are set so that a rectangular pattern is maintained on this output frame memory.

This means that the increment Δx of the X and Y axes in equation (1) above,
By selecting the numerical values of Δy as described above, an operation of reading out the same point occurs until the result of the addition operation using these values is carried beyond the decimal point. As a result, the number of read pixels on the top side becomes the same, making it possible to form a rectangular shape.

The same applies to the case of reading starting from the reading start point after #4, and in the case of the reading start point #11, there is no data from the coordinates (2, 11) and (4°12) of the input frame memory. Two pixels in the output frame memory are being written to.

As is clear from comparing Figure 3 (a) and ('b), the marking line of the parking area ■, which was stored in the input frame memory as a diagonal line, appears vertically or horizontally in the output frame memory. In addition to being stored as a straight line as shown in Figure (a), even if the readout start point is not on a straight line, it is stored as a straight line as shown in Figure (b), which causes distortion. The effect has been achieved that the parking area ■, which was stored as a rectangle in the input frame memory, is stored as a rectangle in the output frame memory.

Furthermore, as mentioned above, the increment ΔX on the N-axis corresponds to the magnification ratio, but if this increment ΔX=
When set to 0.1, that is, in the above formula (1), X-INT
In the case of (Xs(n)+0.IX' (m-1)), a carry occurs only when m increases by 11. Then, until this carry occurs, IN
Due to the truncation by the T function, the read coordinates of the N axis of the input frame memory Fi do not advance and become the same value, so the output frame memory F between 2 and 1 on the iOM axis does not advance.
As for the 0 pixel, the same data is written one after another, and in the end, the dark displacement data of one pixel on the input frame memory is expanded to 10 pixels on the output frame memory.

Conversely, if the value of ΔX becomes 1 or more, some pixels on the input frame memory will not be read out, so the pattern will be reduced on the output frame memory. By appropriately selecting the value of the increment ΔX, it is possible to enlarge or reduce the line segment on the input frame memory Fl on the output frame memory or on the reproduced image.

Here, in order to explain in detail the method of setting Δx and Δy and the method of calculating the readout point using equation (1), a calculation method will be described.

Comparing the shapes of the dividing lines in FIGS. 3(a) and (b), we find that the pattern p1' on the output frame memory in FIG. p2'p4'p3' is stored as a rectangular pattern of 8 x 11 pixels, so pl p2 1:)4 p is stored on the input frame memory.
Opposite sides of the area surrounded by 3, for example, the straight line pIp
The number of pixels on the line p21) and the line p21) must be made equal before transferring it to the output frame memory.

Therefore, if the number of pixels on straight line 1) + p3 is 11 as shown in Table 1 above, and the number of pixels on straight line 1)21)4 is 9, then each of the 11 pixels on straight line l p3 When the interval is 1, the straight line 1) corresponding to these pixels
The pixel spacing on 2 pa should be 9/11-0.82, and this straight line 21)4 is y=-X+2
Since it is 0, the reading start point (Xs(1),
The virtual points assumed on the straight line p21)4 corresponding to the 11 pixels on these straight lines p+ p3, which are Ys(1)) ~ (Xs(11), Ys(11)), are as shown in Table 2 below. Become.

Table 2 Table 3 In addition, the X coordinate X i (n) and Y coordinate Yi of the virtual point
n in parentheses in (n) is the readout start point number shown in the parentheses of the readout start point, and the readout start point (Xs(5
), Ys(5)), the virtual point C(Xi(5)
, Y1(5)).

Therefore, if we calculate the slope of the line segment connecting this readout start point and the virtual point corresponding to this readout start point, we can find the slope f'(
n) is f' (n) = (Ys(n)-Yi(n))/(Xs
(n)-xi(n)), which is shown in Table 3 below.

Next, if we expand the number of pixels on the straight lines p3 and p4 to the same number as the number of pixels on the straight line p+p2 of the input frame memory and correct the distortion, we will calculate the corresponding virtual point from each reading start point. The number of pixels in between and the increment ΔX in the X-axis direction determined based on the number of pixels are as shown in Table 4 below.

Table 4 Table 5 As shown in Table 3 above, since the slope connecting the reading start point to the corresponding virtual point is already known,
Once the increment ΔX in the axial direction is determined, the increment Δy in the Y-axis direction can be calculated using the following formula.

Δy(n) = Δx (n) ・f'(n) Increment Δy in the Y-axis direction from each reading start point obtained in this way
are shown in Table 5 below.

Note that if this increment ΔX or Δy is a number larger than 1, the pixels of the input frame memory will be read out intermittently and the reduction will be performed in the opposite way to the above case. By appropriately selecting the filter, the pattern on the input frame memory can be transformed so as to obtain the desired shape on the output frame memory.

Furthermore, although the pixels at coordinates (8, 3), (10, 6), etc. of the input frame memory are not read out in FIG. 3(a), the present invention can be applied to parking lot management, etc. In some cases, it is not necessary to target a small object where density value data is stored in only one pixel, and similarly, even if there is some deformation in the shape of the subject, changes in density value can be detected macroscopically. There is no practical problem when capturing an image, and the effect of being able to quickly process the distortion of a captured image and correct it to a rectangular shape that is easy to raster scan using simple calculations as described above is extremely large.

In addition, in the above explanation, only the processing of the parking area ■ in Figure 2 etc.] was described, but the parking areas ■ to ■ can also be treated as one area and the same processing as ■ The image can be reproduced as shown in FIG. 2(C). In addition, if the reproduced parking areas of ■ to ■ are the same, the coordinates (15, 6) to (7, 14) in Figure 3 (a)
The above parking area ■
The same processing as above may be performed, and in this case, the coordinates (15°6) of the input frame memory are set as the reading start point, and the data is written to the coordinates (9, 1) of the output frame memory, for example. In the above table, #12 is set as the writing start point n-
1, m = 9, and calculated or actual values as ΔX and Δy. Store these data in the read start point memory and incremental memory, respectively, and set the scanning operation (write) range. may be set in advance.

In addition, in the above explanation, in order to make the explanation easier, Figure 2 (
Although we focused only on the parking area E on one side shown in a), when processing the parking areas on both sides, the next writing start point in the same output frame memory as above, for example, n=21. m=1)
and the coordinates of the read start point on input frame memo 1JFi corresponding to each write start point, and the increment ΔX from this read start point.

It will be easily understood that by storing Δy in memory, it is possible to correct the surface area.

〔Example〕

FIG. 7 shows an embodiment of an image distortion correction device according to the present invention. Note that the M and N count setting sections 30 indicated by dotted lines will be described later.

A video signal captured by a video camera passes through a low-pass filter 1 for preventing aliasing noise, and then is separated into a luminance signal and a synchronization signal by an image signal separation circuit 2 and a synchronization separation circuit 3, and the luminance signal is
The signal is quantized by the D converter 4 and supplied to the density value data input terminal of the input frame memory 5, and the synchronization signal is supplied to the address counter 6 and timing signal generator 7.

This address counter 6 counts the horizontal synchronizing signal and vertical synchronizing signal of this synchronizing signal, and counts the horizontal synchronizing signal and vertical synchronizing signal of this synchronizing signal,
By calculating the write address to and supplying this calculated address to the address input terminal of the input frame memory 5 via the selector 8, the above A is written to this address.
The quantized density value data from the D converter 4 is written in a pattern as shown in FIG. 3(a).

The timing signal generator 7 performs AD conversion for one screen and input frame memory 5 by counting the horizontal synchronization signal and vertical synchronization signal from the synchronization separation circuit 3.
When it is detected that writing to the input frame memory 5 is completed, an inhibit signal is outputted to the AD converter 4 and the address counter 6 to inhibit processing, thereby preventing writing to the input frame memory 5.

Input frame memory 5 written in this way
The read density value data is read from the output frame memory 9 and the read density value data is written to the output frame memory 9 according to the following procedure.

The timing signal generator 7 sends a reset signal to the memory address controller 15 at the same time as sending out the above prohibition signal.
At the same time, this reset signal is supplied to the ○R circuit 10.
is supplied to the first and second multiplexers 11.12, the X address register 13, and the Y address register 14 through
”.

Further, the memory address controller 15 has m=l, 7
When the origin of the coordinates is used as the starting point, as in 1==1, the clock from the clock generator described later is counted, and each time the value matches the value of Mmax and the n value is automatically carried over, the address bus is The read address is sent to the memories 16 to 19 via the memory 20, and the read addresses of these memories are sequentially advanced to obtain the pre-stored coordinates Xs, Ys and increment Δx of the read start point.
, Δy are sequentially output.

The first and second multiplexers 11.12 selectively connect the outputs of the Xs memory 16 and Ys memory 17 to the adder 21.22 by the reset signal, so that the outputs of the Xs memory 16 and Ys memory 17 are selectively connected to the adder 21.22. The stored coordinates Xs and Ys of the reading start point are respectively supplied to one input terminal of adders 21 and 22.

As mentioned above, since the X address register 13 and the Y address register 14 are reset to "0" by the reset signal from the timing signal generator 7, the outputs of these registers 13.14 of the adder 2'1.22 The value of the other input terminal connected to the terminal is “0”
Therefore, the output of this adder 21.2'2 is the output of Xs memory 16 and Ys memory 17, respectively.
, YS, and Xs and Ys of the starting point a on the input frame memory 5 are stored in the X address register 13 and the Y address register 14, respectively.

The timing signal generator 7 stops outputting the reset signal after a sufficient amount of time has elapsed to complete the above operations, thereby inputting the input terminals of the multiplexers 11 and 12 to the ΔX memory 18. Switch and connect each to the Δy memory 19 side,
Thereafter, a lock start signal is output to the clock generator 23, and the selector 8 for switching the address input of the input frame memory 5 is switched to the address decoder 24 side.

A clock generator 23 receives this clock start signal.
outputs a clock that serves as a reference for calculations by the adders 21 and 22, reading from the input frame memory 5, and writing to the output frame memory 9.

This clock signal corresponds to a raster scan address that increases the write coordinate in the M-axis direction of the output frame memory by one every clock.

Therefore, the addresses Xs and Ys of the coordinates of the read start point stored in the X address register 13 and the Y address register 14, respectively, are read out by the clock from the clock generator 23.

Then, the address decoder 24 executes integerization processing, which will be described later, and converts it into a read address of the input frame memory 5. It is supplied to the address input terminal of the input frame memory 5.

Therefore, when the enable signal from the timing controller 25 is supplied to the read enable terminal RE of this input frame memory 5, this input frame memory 5
The density value data at the reading start point is read out from.

At this time, in the address counter 26, by counting the first clock as described above, M=1, which is the first writing point of the output frame memory 9. The address of the lower left corner point, which is the origin of the MN axis of N=1, that is, the origin of the MN axis in FIG. is supplied to the write enable terminal WE of the output frame memory 9, the density value data at the reading start point is written to the lower left corner of the output frame memory 9, which is the point at coordinates (1, 1).

On the other hand, the coordinates Xs and YsO of the reading start point read out from the X address register 13 and the Y address register 14 as described above are supplied to the other input terminal of the adder 21 and 22, but at this time A signal generator 7 connects the input side of the multiplexer 11.12 to the ΔX memory 18. Since these adders 21 and 22 are switched and connected to the Δy memory 19 side, respectively, the adders 21 and 22 add the coordinate values Xs and Ys of the reading start point to the ΔX memory 18 . The increment Δx and Δy from the Δy memory 19 are added together to output (XS+ΔX) and (Ys+Δy), and the values stored in the X address register 13 and the Y address register 14 are updated by these values. It will be done.

Regarding the operations after #2 in Table 1, the memory address controller 15 outputs a reset signal to the multiplexer 11.12 and address register 13.12 via the OR circuit 10 by counting the number of clocks of Mmax.
A reset signal is supplied to the Y address register 14 to switch the multiplexer 11.12 again to the illustrated position and reset the register 13.14 to "0". The memory address controller 15 also sends read addresses to the memories 16 to 19 via the address bus 20,
Update the values of Δx, Ys, and Δy.

Then, the memory address controller 15 outputs the reset signal again by counting the next clock (Mmax + 1), and repeats the above addition operation by switching the multiplexers 11 and 12 to the ΔX memory 18 and Δy memory 19 sides. .

At this time, the address counter 26 counts the clock lv
The operation shown in Figure 4, which automatically carries up the n value using Imax, is performed, but by setting the M value and N value in advance, the operation shown in Figure 3 (
It is also possible to perform the scanning operation only in the range (8, 11) shown in b).

In addition, when setting the first writing start point shown in Table 1 to a location other than the origin of the MN axis, for example, the address counter 26
Each write start point may be stored in memory, and the write address of the next write start point may be sequentially output to the output frame memory 9 when the set M value counts the corresponding clocks. .

In this addition operation, as mentioned above, it is necessary to convert the numbers below the decimal point into integers, but as shown in FIG.
In 4, the position of the decimal point is fixed as shown in the figure, and the upper bits are allocated above the decimal point, and the lower bits are allocated below the decimal point.By adopting a fixed-point format, the above configuration can easily be used without special arithmetic processing. can be executed.

Now, assuming that the number of pixels on the screen to be corrected is 256 x 256 pixels, the maximum addresses of the input frame memory (F1Ly) This means that all addresses can be expressed.

Also, the lower bits indicate the decimal point and the increment △X,
The movement interval of the readout address determined by Δy is specified, but since the maximum number of pixels is set to 256, even if one pixel of the image capture screen is enlarged to the maximum, the limit is 256 times, so the decimal point is For the following, it is sufficient to use an 8-bit configuration that can represent 1/256, which is carried to 1 only when multiplied by 256, and this 1/256 becomes the minimum unit of the increments ΔX and Δy.

To give an example using actual numerical values, if △X = 0.1, the size of the data stored in the output frame memory 9 will be 10 times that of the size stored in the input frame memory 5. , the coordinate Xs of the X-axis of the reading start point is 100
Then, the upper bits of this register are as shown in Fig. 9(a).
The binary number “0110” representing the decimal number 100 as shown in
0100'' is stored.

Also, △X=O,l becomes 25.6 by quantizing 0 to 1.0 to 0 to 256, and 25 is obtained by rounding down the decimal part, which is 198 in hexadecimal ('000 in binary)
11001''), adder 21 or 2
When the value of ΔX=0.1 of the second lever is added to the value of the register in which the above-mentioned upper bit is stored, the result is as shown in FIG. 2(b).

As mentioned above, adding ΔX sequentially means adding 198 to the lower bits one after another, and a carry to the upper bits occurs only after 11 additions. , these upper bits define the read address of the input frame memory 5, so if the address decoder 24 uses only the upper bits of the address registers 13 and 14 as addresses, IN of the above equation (1)
This means that the same integer conversion processing as the T function is performed, so an integer conversion processing means is not required, and the hardware configuration can be simplified.

The values of the read coordinates updated in this way are read from the X address register 13 and the Y address register 14, decoded by the address decoder 24 as described above, and then supplied to the input frame memory 5 as a read address via the selector 8. Then, following the read start point, the memory element to be read is accessed and read.

When all writing for one screen to the output frame memory FO is completed in this way, the memory address controller 15 detects that writing for one screen to the output frame memory is completed by counting clocks, A one-screen end signal is output to the timing signal generator 7 to notify the end of correction, and a processing start signal is output to the MPU 29. This MPU receives the data read from the output frame memory 9 through the image processing processor 28. A predetermined process is performed on the .

Furthermore, the timing signal generator 7 that has received the one-screen end signal as described above monitors the horizontal synchronization signal and the vertical synchronization signal, and when it detects the start point of the video signal, cancels the prohibition signal and outputs the signal to the AD converter. 4 and the address counter 6 are restarted, the next image data is input into the frame memory 5, and the above-described processing is performed on the next image.

Next, M and N are indicated by dotted lines in the block diagram of the embodiment shown in FIG.
Another embodiment in which a count setting section 30 is further provided will be described.

Figure 10 shows the output frame memory F in this embodiment.
The solid line is the scanning line, and the dotted line is the return line, and after the raster scan of area ■ corresponding to parking area ■ in Figure 2 is completed, the next similar area ■ is shown. In order to determine the range of the scanning operation, the setting unit 30 uses the scanning start point of these areas as a reference and determines the end of scanning of the area. The count number U on the horizontal axis and the count number V on the vertical axis of the points are stored, and in the example shown in FIG. 1O, u=8. v=11.

Note that the address counter 26 of this embodiment has each scan start point of the scanning operation ((1, 1 in FIG. 10)
), (9, 1) ) and a counter function for counting clocks of count/count numbers U and V set in the setting section 30, and a signal that performs a scanning operation in the range of u and V' with each starting point as the starting point. Output.

A specific explanation will be given below using the value of "8+v=tt". The coordinates of the writing start point when the scanning operation for writing to the output frame memory is started are (1
, 1), and at this time, the first row #1 of the first table is read out, and the density value data from the input frame memo + JFi is read out by the aXs and YS, and the density value data is read out from the output frame memory F.

is written to the coordinates (1, 1) of

The write point moves to (2, 1) by the next clock from the clock generator 23, and the readout in the input frame memory follows the coordinates shown in the m=2 column of the #1 row in Table 1. It is performed from (9, l) and written to this writing point (2, 1), and in the same way, when the writing point becomes (8° 1),
Memory address controller 1 counting clocks
5 and address counter 26, the value m of M is 8.
It is identified that it has become

Since this value of m is equal to the M-axis value u=3 stored in the M and N count setting section 30, the memory address controller 15 checking this set value outputs a reset signal and first As explained above, this reset signal switches the selectors 11 and 12 to the positions shown in FIG.
The address register 14 is reset to "0".

At the same time, an address that increases the read address by 1 to the Xs memory 16 and Ys memory 17 is sent to the address bus 20.
These notes are output via ', 116.17, so
Move to line #2 in Table 1 and start reading from this line x5(2
), Ys(2) coordinates X5-7, Ys=2 are read from the Xs memory 16 and Ys memory 17 and set in the X address register 13 and Y address register 14, respectively.

In this embodiment, as mentioned above, when the M-axis coordinate m of the write point becomes equal to the M-axis count number U stored in the M,N count setting section 30, the next reading start point is determined. The coordinates Xs, Ys and the increments ΔX, Δy are read out.

At the next clock, the density value data read out from the input frame memory Fi is transferred to the output frame memory at the coordinates (8, 2) shown in the column m=2, which is the second readout point starting from this readout start point. is written at the coordinates (2, 2).

In this way, when the raster scan to p4 corresponding to p,', that is, the coordinates (8, 11) of the output frame memory Fo is completed, the memory address controller 1
5 and the address counter 26 identify that the M-axis count number U and the N-axis count number V set in the M and N count setting section 30 both match these coordinates (8, 11). .

At the same time, the memory address controller 15 resets the X address register 13 right and the Y address register 14 to 0'', and at the same time increases the read addresses of the Xs memory 16 and Ys memory 17 by 1. reading start point X s (12> , Y s
(12) coordinates in χS mesori 16 and Ys memory 17
Since the reading start point (X s (12) , Y
s (12)), and the read density value data is at the coordinates (9, 1) of the output frame memo 1JFo.
written to.

Then, the address counter 26 receives the write start point (9, 1
) with u'=8. The address for performing the scanning operation of v=11 is output to the output frame memory 9.

According to this second embodiment, when the M-axis value and the N-axis value match the count numbers U and V stored in the M and N count setting section 30, the movement is made to a new starting point, and Coordinates of the next reading start point χs, Ys and Δx, Δy
By reading out the image data, it becomes possible to correct the size of all the areas to be processed to be the same, and uniform processing becomes possible, which further improves the efficiency of image processing.

Therefore, on the input frame memory Fl, as shown in FIG.
As shown in the figure, the pl+ has different sizes and shapes.
pz,l)3. T) 4 The area surrounded by the four points ■ and Q+, Q2. Q3. In order to arrange the area ■ surrounded by the four points of q4 on the output frame memo + JF o with the same size and adjacent to each other as shown in the figure 0:1), the number of pixels on pI'p4' must be is the above M, and the value M of M in the N count setting section 30. Also, the number of pixels on p'p2' is the value of N. If you set the coordinates as 937 points (Mo, No) by raster scanning, the next starting point of scanning will return to q1', as shown in this figure (b). A corrected image with the same shape is obtained.

Also, the coordinates by raster scan are 937 points.
If the next starting point of scanning is set to return to ql" when MO, No. .

Note that the values of q+ and q1' are stored in advance in the address counter 26 as described above.

Furthermore, even when capturing traffic flow on a curved road as shown in Figure 6, it is possible to
), Ys(0) to It is clear that by setting Δx and Δy, the distortion can be corrected so that the curved road has a constant width in the same manner as described above.

〔Effect of the invention〕

As explained in detail above, the coordinates and increment of the read start point are set corresponding to the write address of the output frame memory, and these are used to read the input frame memory based on the result of the addition operation as described above by the adder. By controlling the address of the points, it is possible to correct an image with distortions such as perspective or rotation into a desired shape such as a rectangle.

In this way, corrections can basically be performed using only addition operations and memory read and write operations, which reduces the burden on the hardware and allows for extremely high processing speeds, allowing it to keep up with the video rate. Image correction processing can also be easily realized.

In addition, it is possible to correct the size of all target areas or select the display position, and the shape can be controlled to suit the convenience of software processing, increasing the efficiency of software processing. It is possible to achieve a special effect of significantly improving the performance.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the principle of the present invention, FIG. 2 is a diagram regarding a parking lot as an example for explaining the present invention, and FIG. 3 is a diagram showing patterns on input frame memory and output frame memory. , Fig. 4 is a diagram for explaining raster scanning applied to the output frame memory, Fig. 5 is a diagram showing a pattern on the frame memory for an imaging screen different from Fig. 3, and Fig. 6 is a diagram for explaining the raster scan applied to the output frame memory. FIG. 7 is a block diagram showing an embodiment of the present invention; FIGS. 8 and 9 are diagrams showing an example of the structure of an address register and stored data; FIG. 10 is a diagram showing a pattern on the output frame memory for explaining the operation of the second embodiment, and FIG. 11 is a diagram for explaining an example of conventional processing.

Claims (2)

[Claims] (1) An input frame memory (Fi) that stores captured image data, and coordinates (Xs,
Memories (R, Rd) that store the coordinates (Ys) and increments (Δx, Δy) for obtaining the next readout point coordinates based on these coordinates, and the coordinates and increments of the readout start point stored in the memories. An image distortion correction apparatus characterized by comprising: address conversion means (C) for sequentially reading out density value data from the input frame memory based on readout point coordinates obtained by an addition operation using the above. (2) The image distortion correction device according to claim 1, further comprising an M and N count setting section that defines a range of scanning operation in the output frame memory, and corrects a plurality of regions having different shapes into the same shape. .