JP2006338137A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor capable of correcting the distortion of an image taken through a wide-angle lens or the like without a display delay or easily composing it with other image data. <P>SOLUTION: In the image processor adapted to generate display data and an address in response to an image signal generated based on light passed through a lens and write the data in a VRAM 106, a capture controller 100 converts the image signal to the address Ad of the VRAM 106 and display data D followed by outputting. An XY coordinate generating circuit 101, a correction table 102, multipliers 103 and 104 and a corrected address generation circuit 105 correct the address Ad based on the distortion ratio of the lens. The display data D is written in the VRAM 106 based on the corrected address. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that corrects image distortion caused by a wide-angle lens, a fish-eye lens, or the like.

In a video camera device such as an in-vehicle back monitor or a TV door phone, a fish-eye lens or a wide-angle lens may be used to obtain an image with a small blind spot and a wide viewing angle. In such a video camera device, distortion of the image is generated in the peripheral portion of the image by the lens. Therefore, if the target is in the peripheral portion where the distortion is large, visual recognition becomes an obstacle. In order to avoid such a situation, in a conventional image processing apparatus, an image with distortion captured by a video camera is stored in a line memory or a VRAM (Video Random Access Memory), and then these data are read and reproduced. When doing this, distortion is corrected and a natural image is output (for example, refer to Patent Document 1, Patent Document 2, and Patent Document 3).
Japanese Patent Laid-Open No. 6-225312 JP-A-5-252513 JP-A-11-196313

  However, in the conventional image processing apparatus, since the time for distortion correction becomes insufficient when the resolution becomes high, there is a problem that the display is slow, and a distorted image is stored in the memory. In order to synthesize a captured image that needs correction and another image such as a graphic that does not need distortion correction, a complicated and expensive configuration such as a memory such as a VRAM is also provided for the image after distortion correction. There is a problem that is necessary.

  The present invention has been made in view of such circumstances, and an object of the present invention is to display an image captured by a wide-angle lens or the like without distortion after correcting distortion, and to include image data including distortion and other data. An object of the present invention is to provide an image processing apparatus that can be easily combined with image data.

  The present invention has been made to solve the above-described problem. The invention according to claim 1 receives a video signal generated based on light passing through a lens, generates display data and an address, In an image processing apparatus for writing to an image memory, a capture controller that converts the video signal into an address and display data of the image memory and outputs the data, and an address output from the capture controller is corrected based on a distortion rate of the lens An image processing apparatus comprising: a correcting unit; and a writing unit that writes the display data into the image memory based on an address corrected by the correcting unit.

  According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, the correction unit includes a first conversion unit that converts an uncorrected address output from the capture controller into coordinates on a display screen. A coordinate correction unit that corrects coordinates output from the first conversion unit based on a distortion rate of the lens, and a second conversion unit that converts the corrected coordinates into addresses of the image memory. It is characterized by that.

  According to a third aspect of the present invention, in the image processing apparatus according to the first aspect, the correction unit includes an XY coordinate generation unit that converts an address output from the capture controller into an XY coordinate with an image center as an origin. Correction coefficient output means for outputting the product of the square of the distance from the origin according to the XY coordinates output from the XY coordinate generation means and the distortion rate of the lens, the X coordinates output from the XY coordinate generation means, and the correction An X correction output means for outputting the product of the output of the coefficient output means; a Y correction output means for outputting the product of the Y coordinate output from the XY coordinate generation means and the output of the correction coefficient output means; The output address is converted into XY coordinates, the X coordinate of the XY coordinate is corrected by the output of the X correction output means, and the Y coordinate of the XY coordinate is corrected by the output of the Y correction output means. XY coordinate correction means for generating corrected XY coordinates, and corrected address generation means for converting X and Y coordinates output from the XY coordinate correction means into addresses of the image memory. And

According to a fourth aspect of the present invention, in the image processing apparatus according to the first or second aspect, a calibration table storing an address of the image memory in which display data is not written by the writing unit; Complementary processing means for calculating display data of addresses stored in the calibration table based on display data of surrounding display dots and writing to the addresses of the image memory is further provided.
According to a fifth aspect of the present invention, in the image processing apparatus according to the fourth aspect of the present invention, dummy data is assigned to the initialization unit that initializes the image memory and the address corrected by the correction unit of the image memory. It further comprises dummy data writing means for writing, and calibration means for detecting an address that has been initialized out of each address of the image memory and registering it in the calibration table.

According to the first aspect of the present invention, since the image subjected to the subsequent distortion correction at the time of capturing is recorded in the image memory, it becomes easy to combine with another image on the image memory. There is an effect. Further, even when the distortion is large, correction is performed at the time of writing to the image memory, so that there is an effect that no display delay occurs.
According to the fourth aspect of the present invention, since the complementary process for filling in the writing omission caused by the distortion correction is performed only for the addresses registered in advance in the calibration table, a natural image can be obtained with a small number of processes. is there.
According to the fifth aspect of the present invention, since it has a function of detecting a write omission address generated by distortion correction and recording it in the calibration table, even when the distortion rate is changed by lens replacement or the like. There is an effect that the calibration table can be recreated and the interpolation process can be surely performed.

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to the embodiment. This image processing apparatus converts an analog video signal output from a CCD (charge coupled device) camera into digital color data, corrects distortion based on the lens of the camera, and writes it in a VRAM (video RAM) 106. Specifically, the display position of each display dot is corrected and written in the VRAM 106. FIG. 2 shows a display panel H on which display is performed using color data in the VRAM 106. The storage slot of the VRAM 106 corresponds to the display dot of the display panel H at a ratio of 1: 1.
Prior to detailed description of the image processing apparatus, reference numerals used in the following description will be described.

Ad: Address of the VRAM 106 D: Color data written to the address Ad Xd: X coordinate of the point P before correction in the coordinate system with the point G (0, 0) at the upper left corner of the display panel H as the origin Yd: Display panel H Y coordinate of the point P before correction in the coordinate system with the point G (0, 0) at the upper left corner of X as the origin X: X in the coordinate system with the point G as the origin of the point P ′ moved by the correction of the point P Coordinate Y: Y coordinate in the coordinate system with the point G as the origin of the point P ′ moved by the correction of the point P Cx: X coordinate of the central point C of the display panel H in the coordinate system with the point G as the origin Cy: Y coordinate in the coordinate system of the center point C of the display panel H with the point G as the origin RXd: X coordinate of the point P before correction in the coordinate system with the point C of the display panel H as the origin RYd: Point of the display panel H In the coordinate system with C as the origin Y-coordinate of point P immediately before RX RX: X-coordinate of point P ′ moved by correction of point P in coordinate system with point C as origin RY: Point C of point P ′ moved by correction of point P ′ Y coordinate in the coordinate system as the origin k: distortion rate of camera lens (constant)

Next, the principle of distortion correction will be described.
First, it is known that for the display point P (Xd, Yd) including distortion based on the distortion rate k of the camera lens, the coordinates of the correct display point P ′ (X, Y) from which the distortion is removed are expressed by the following equation. ing.
X = (Xd−Cx) (1 + k · rd ² ) + Cx (1)
Y = (Yd−Cx) (1 + k · rd ² ) + Cy (2)
Here, rd ² is a value obtained by the following equation.
rd ² = RXd ² + RYd ² (3)

When the above equations (1) and (2) are transformed,
X−Cx = (Xd−Cx) (1 + k · rd ² ) (4)
Y−Cy = (Yd−Cx) (1 + k · rd ² ) (5)
The following formula is obtained, and from this formula,
RX = RXd (1 + k · rd ² ) (6)
RY = RYd (1 + k · rd ² ) (7)
The following formula is obtained, and from this formula,
RX-RXd = RXd · k · rd ² = X correction value (8)
RY−RYd = RYd · k · rd ² = Y correction value (9)
The following formula is obtained. From this equation, the following correction method is obtained.
“By obtaining rd ² from the coordinates (RXd, RYd) of the display point P before correction with the point C as the origin, and applying it to the equations (8) and (9), X and Y correction values are obtained. If the correction value is added to RXd and RYd, respectively, the coordinates (RX, RY) of the corrected display point P ′ can be obtained. ”
In the embodiment of FIG. 1, the coordinates of the display point P ′ after correction are obtained by this correction method.

  By the way, when the color data to be displayed at the display point P is displayed at the display point P ′, the color data to be displayed at the display point P is lost. Hereinafter, such a point is referred to as a black point. However, even if the display point P becomes a black point, the corrected point Q ′ obtained by correcting another point Q may be exactly the position of the point P. In this case, the point P is the black point. Is not. Therefore, the points before correction are not all black points, but there are naturally points that remain as black points. Therefore, in this embodiment, after the VRAM 106 is cleared at the time of power-on, the display position is corrected by the above-described method, dummy data is written, and the storage slot whose data is “0” at the end of writing is stored. The address is detected as a black dot address. This process is called calibration. Then, after the actual image data is written into the VRAM 106, the color data adjacent to the black point coordinates detected by the above processing is written. This process is called a complement process.

Hereinafter, the circuit of FIG. 1 will be described.
In the figure, reference numeral 100 denotes a capture controller which converts a video signal output from the CCD camera into color data D and outputs it, and generates and outputs an address Ad of the VRAM 106 in which the color data D is written. The capture controller 100 outputs a control signal for controlling each part of the apparatus. Reference numeral 101 denotes an XY coordinate generation circuit which converts the address Ad into coordinates (RXd, RYd) having the point C of the display panel H as the origin, and outputs it. Reference numeral 102 denotes a correction table, in which k · rd ² (see Expression (3)) corresponding to the coordinates (RXd, RYd) is stored in advance. Reference numeral 103 denotes a multiplier that multiplies RXd and k · rd ² (see Expression (8)). A multiplier 104 multiplies RYd and k · rd ² (see Expression (9)). Reference numeral 105 denotes a corrected address generation circuit that generates and outputs a corrected address based on the address Ad, the X correction value, and the Y correction value. Reference numeral 107 denotes a calibration circuit that detects a black spot address. Reference numeral 108 denotes a calibration table in which black spot addresses detected by the calibration circuit 107 are recorded. Reference numeral 109 denotes a complementary processing circuit for writing color data supplemented to a black dot address. A switching circuit 110 switches an address for accessing the VRAM 106 among the corrected address generation circuit 105, the calibration circuit 107, and the complementary processing circuit 109. Reference numeral 111 denotes a switching circuit that switches color data to be written to the VRAM 106 between the capture controller 100 and the complementary processing circuit 109.

  Next, the operation of the image processing apparatus having the above configuration will be described with reference to the timing chart of FIG. When the power is turned on at time t1, the apparatus performs calibration table creation processing, then performs video capture and distortion correction processing during the video signal effective period, and performs black point position complement processing during the vertical blank period of the video signal. . Thereafter, the video capture and distortion correction process and the complement process are repeated in synchronization with the input video signal.

  In the calibration table creation process, first, the VRAM 106 in step Sa1 in FIG. 4 is initialized. That is, the capture controller 100 instructs the calibration circuit 107 to initialize the VRAM by using the signal C1, and outputs the VRAM 106 initialization value “0” as the color data D to the data terminal of the VRAM 106 via the switching circuit 111. . When the calibration circuit 107 is instructed to initialize the VRAM by the signal C 1, the calibration circuit 107 first outputs the head address of the VRAM 106 to the address terminal of the VRAM 106 via the switching circuit 110. When the VRAM 106 receives the head address from the switching circuit 110 and the color data “0” from the switching circuit 111, “0” is written in the storage slot of the head address. The calibration circuit 107 then outputs the address of the second storage slot, and “0” is written to the VRAM 106 in the second storage slot. Thereafter, the calibration circuit 107 similarly repeatedly outputs until the addresses of the third, fourth,..., And last storage slots are reached, and “0” is written to all the storage slots in the VRAM 106.

Next, the writing of dummy data in step Sa2 will be described. When the initialization Sa1 of the VRAM 106 ends, the capture controller 100 outputs dummy data, for example, “1” as color data D to the data terminal of the VRAM 106 via the switching circuit 111, and outputs the head address of the VRAM 106 as the address Ad. To do. When receiving the head address from the capture controller 100, the XY coordinate generation circuit 101 converts it into RXd, RYd and outputs it. When the correction table 102 receives RXd and RYd from the XY coordinate generation circuit 101, the correction table 102 outputs corresponding k · rd ² . When the multiplier 103 receives RXd from the XY coordinate generation circuit 101 and k · rd ² from the correction table 102, the multiplier 103 outputs a value RXd · k · rd ² obtained by multiplying them, that is, an X correction value by the equation (8). To do. Similarly, the multiplier 104 outputs a Y correction value according to equation (9). When the corrected address generation circuit 105 receives the head address from the capture controller 100 and receives the X correction value and the Y correction value from the multipliers 103 and 104, it first converts the head address into RXd and RYd, and then converts it into RXd. X correction value and Y correction value are added to RYd, respectively, to obtain corrected coordinates RX and RY. The corrected address generation circuit 105 outputs the corrected address obtained by converting the corrected coordinates RX and RY into the address of the VRAM 106 to the address terminal of the VRAM 106 via the switching circuit 110. When the VRAM 106 receives the corrected address from the switching circuit 110 and the data “1” from the switching circuit 111, dummy data “1” is written in the storage slot of the corrected address of the VRAM 106. Subsequently, the capture controller 100 outputs the address of the second storage slot as the address Ad while outputting the dummy data “1” as the color data D. Then, “1” is written in the storage slot of the corrected address in the VRAM 106 in the same manner as described above. Thereafter, the capture controller 100 repeatedly outputs the addresses of the third, fourth,..., And last storage slots in the same manner, and the VRAM 106 stores dummy data in the storage slots of all corresponding corrected addresses. A certain “1” is written.

  Next, black point address detection in step Sa3 will be described. The capture controller 100 instructs the calibration circuit 107 to detect the black dot address by the signal C1. When the calibration circuit 107 receives an instruction for black spot address detection by the signal C <b> 1 from the capture controller 100, the calibration circuit 107 outputs the head address of the VRAM 106 to the VRAM 106 via the switching circuit 110. When the VRAM 106 receives the head address via the switching circuit 110, the VRAM 106 outputs the data stored in the storage slot of the head address as data RD from the output terminal. The black spot address detection circuit 107 checks the data RD, and when the data is “0”, the storage slot of the head address remains in an initialized state, so the head address is recognized as a black spot address. Register in the calibration table 108. On the other hand, when the data RD is “1”, the head address is recognized as not a black dot address, and is not registered in the calibration table 108. Subsequently, the calibration circuit 107 outputs the address of the second storage slot to the VRAM 106 via the switching circuit 110. When the VRAM 106 receives the address of the second storage slot, the VRAM 106 outputs the data stored in the second storage slot as data RD. Similarly to the case of the head address, the calibration circuit 107 registers the address of the second storage slot in the calibration table 108 when the data RD is “0”. On the other hand, when the data RD is “1”, the address of the second storage slot is not registered in the calibration table 108. Similarly, the calibration circuit 107 repeats reading until the third, fourth,..., And last storage slots are reached. As a result, the addresses of all the storage slots in which the dummy data has not been written by the dummy data writing (step Sa2), that is, all the black dot addresses are registered in the calibration table 108.

  When the calibration table creation process ends and the video signal valid period of the input video signal comes, the video capture and distortion correction process is started. In this process, the above-described dummy data writing (step Sa2) is performed except that the capture controller 100 does not continue to output the dummy data as the color data D but outputs the color data D and the address Ad corresponding to the input video signal. ) As a result, the color data after distortion correction is written in the VRAM 106.

  Next, when the vertical blank period of the input video signal is reached, a complementing process is performed. In the complement process, first, the capture controller 100 instructs the complement processing circuit 109 to execute the complement process by the signal C2. The complement processing circuit 109 reads the first black spot address (for example, “15”) recorded in the calibration table 108. Next, the complementary processing circuit 109 outputs the address of the storage slot corresponding to the display dot next to the address “15” (the address “14” of the previous storage slot) to the VRAM 106 via the switching circuit 110. . When receiving the address, the VRAM 106 outputs the color data stored in the storage slot of the address. Upon receiving the color data, the complementary processing circuit 109 outputs the black spot address “15” to the address terminal of the VRAM 106 via the switching circuit 110, and the color data is output to the VRAM 106 via the switching circuit 111. Output to the data terminal. As a result, the color data of the address “14” is written into the storage slot of the address “15” of the VRAM 106. The complement processing circuit 109 sequentially performs the same processing for all black spot addresses recorded in the calibration table 108. As a result, the color data is written to all the black spot addresses where the color data was not written in the video capture and distortion correction processing.

  In addition, in the complementing process, it was explained that the color data of the display dot next to the black dot is written as it is to the black dot, but the complement is performed with the average value of the color data of the upper, lower, left, and right display dots. It is also possible to calculate from the display dot color data.

  In the present embodiment, the calibration table creation process is described as being performed when the power is turned on, but the calibration table 108 is recorded in a non-volatile storage device such as a flash memory, and the distortion rate k changes when the lens is replaced. The calibration table creation process may be performed only at the timing. After the calibration table creation process, the video capture and distortion correction process and the calibration process have been described as being performed for each frame. However, the video capture and distortion correction is performed for each line such that the distortion correction is performed only in the horizontal direction. Processing and calibration processing may be performed. In this case, the calibration process is performed during the horizontal blank period.

  In the present embodiment, the blocks 100, 101, 102, 103, 104, 105, 107, and 109 have been described as being realized by dedicated hardware. However, these processing units include a memory and a CPU (central processing unit). ), And a program for realizing these functions may be implemented by loading the program into a memory and executing the program.

  In the present embodiment, the video camera that captures a color moving image has been described. However, a camera that captures a still image or a monochrome moving image may be the target.

  The present invention is suitable for use in a display device or the like that displays an image taken by a lens having a large distortion such as a wide-angle lens.

1 is a block diagram of an image processing apparatus according to an embodiment of the present invention. It is a figure explaining the coordinate in the display panel in the embodiment. It is a timing chart explaining operation | movement of the apparatus shown in FIG. It is a flowchart explaining the operation | movement of calibration table creation shown in FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Capture controller, 101 ... XY coordinate generation circuit, 102 ... Correction table, 103 ... Multiplier, 104 ... Multiplier, 105 ... Corrected address generation circuit, 106 ... VRAM, 107 ... Calibration circuit, 108 ... Calibration table 109 ... complementary processing circuit, 110 ... switching circuit, 111 ... switching circuit

Claims (5)

In an image processing device that receives a video signal generated based on light passing through a lens, generates display data and an address, and writes it to an image memory.
A capture controller that converts the video signal into an address and display data of the image memory and outputs the data;
Correction means for correcting an address output from the capture controller based on a distortion rate of the lens;
Writing means for writing the display data into the image memory based on the address corrected by the correcting means;
An image processing apparatus comprising: The correction means includes
First conversion means for converting an address before correction output from the capture controller into coordinates on a display screen;
Coordinate correction means for correcting coordinates output from the first conversion means based on a distortion rate of the lens;
Second conversion means for converting the corrected coordinates into addresses of the image memory;
The image processing apparatus according to claim 1, further comprising: The correction means includes
XY coordinate generation means for converting the address output by the capture controller into XY coordinates with the center of the image as the origin;
Correction coefficient output means for outputting the product of the square of the distance from the origin according to the XY coordinates output from the XY coordinate generation means and the distortion rate of the lens;
X correction output means for outputting a product of the X coordinate output from the XY coordinate generation means and the output of the correction coefficient output means;
Y correction output means for outputting a product of the Y coordinate output from the XY coordinate generation means and the output of the correction coefficient output means;
The address output from the capture controller is converted into XY coordinates, the X coordinate of the XY coordinates is corrected by the output of the X correction output means, and the Y coordinate of the XY coordinates is corrected by the output of the Y correction output means. XY coordinate correction means for generating corrected XY coordinates;
Corrected address generation means for converting X and Y coordinates output from the XY coordinate correction means into addresses of the image memory;
The image processing apparatus according to claim 1, further comprising: A calibration table storing addresses of the image memory in which display data is not written by the writing means;
Complement processing means for calculating the display data of the address stored in the calibration table based on the display data of the surrounding display dots, and writing to the address of the image memory;
The image processing apparatus according to claim 1, further comprising: Initialization means for initializing the image memory;
Dummy data writing means for writing dummy data to the address corrected by the correcting means of the image memory;
Calibration means for detecting an address that has been initialized among the addresses of the image memory and registering the address in the calibration table;
The image processing apparatus according to claim 4, further comprising:

Images