KR20120005664A - A device of transforming fish-eye image to flat image in cctv camera - Google Patents

Abstract

PURPOSE: A device of transforming a fish-eye image into a flat image in a cctv camera is provided to monitor the wide area without a dead space at the same time. CONSTITUTION: An image sensor(31) detects the information of a subject. An image processor(32) changes a video signal into video information. A frame buffer(33) provisionally stores the video information. A video processor(34) converts the video information into a planar image. A display(36) shows the planer image.

Description

Plane image converting device of fisheye image of CCTV camera {A DEVICE OF TRANSFORMING FISH-EYE IMAGE TO FLAT IMAGE IN CCTV CAMERA}

The present invention relates to a planar image converting device of a fisheye image of a CCTV camera, and more particularly, to a fisheye image of a CCTV camera for converting a curved image photographed through a fisheye lens used for wide area image capturing in a CCTV camera into a planar image. Relates to a planar image conversion device.

In general, banks, department stores, 24-hour convenience stores, public offices, etc., install and operate surveillance cameras for the safety of customers, and most of these sites are equipped with fixed cameras of independent configuration, one at each of the necessary places, and It is configured to record the video to the VCR while outputting it to the monitor in real time.

have.

However, in some applications, wide angle lenses (eg, fisheye lenses) are used to obtain wider viewing angles because conventional cameras used for such applications have limited viewing angles. In addition, the field of the visual sensor applied to the conventional camera as described above includes a camera that processes the omnidirectional image obtained by the optical system and the imaging device using software used by a computer to convert the omnidirectional image into a human-readable image such as a panoramic image. Omni-directional surveillance systems are in the spotlight.

Here, the fisheye lens as described above is rounded by all the rays (ambient light) that do not pass through the center of the screen due to severe distortion aberration, the subject is rounded away as the distance from the center of the screen Because the screen is rounded, it is uncomfortable because it comes in.

There are two methods for correcting distortion, such as a forward mapping method and an inverse mapping method. The former obtains a correction coefficient for obtaining a corrected image from a distorted image and applies it directly to the distorted image. The latter method assumes a pre-corrected image and finds out which points of the image match the distorted image.

Referring to an example of the conventional omnidirectional monitoring system 10 as shown in FIG. 1, there is provided a camera 12 using a fisheye lens 11; An imaging unit (13) for converting the optical image obtained by the camera (12) into image data; And a computer 15 for converting the image data sent from the image pickup section 13 to an image and displaying the image data on the monitor 14.

On the other hand, referring to the operation of the conventional omnidirectional monitoring system 10 as described above, first, the circular optical image obtained by the camera 12 is converted into circular image data by the imaging unit 13. The imaging section 13 then outputs the converted image data to the computer 15. The computer 15 then processes the input image data into an image that is easier to see by a human, such as a rectangular panoramic image or a perspective image, and displays it on the monitor 14.

 However, since the optical apparatus of the conventional omnidirectional monitoring system 10 is equipped with a curved mirror designed to capture 360 degrees omnidirectional, such a system is installed in a school zone or a park and monitored. Since the image captured by the curved mirror is in a distorted state, it should be corrected to a flat image.

In order to convert the curved image into a planar image, it is necessary to know the geometric shape of the coordinate transformation as shown in FIG. 2.

In order to mathematically convert an image on a spherical radius r into a plane, three-dimensional coordinates (x, y, z) of one point of the curved image 21 shown in FIG. 2 are converted to two-dimensional coordinates (a, b). To be expressed as the following formula.

Where k is the distance from the center of the sphere to three-dimensional coordinates (x, y, z), and kp is the distance from the center of the sphere to two-dimensional coordinates (a, b).

At this time, a large amount of computational algorithm has to be calculated in the process of converting the curved image 21 captured and transmitted by the fisheye lens into a planar image by the computer 15, thereby reducing the number of premies of the converted planar image. The phenomenon shown as the still image of the converted image is generated. That is, there was a problem that the motion of the image is broken.

Therefore, the object of the present invention is to solve the above problems, reduce the amount of computation of the algorithm to be calculated by the computer, implement this algorithm as a circuit and attach this circuit to the camera, the image captured by this camera on the monitor It is an object of the present invention to provide a planar image converting device for fisheye images of CCTV cameras that can be viewed immediately as a planar image.

The planar image converting device of a fisheye image of a CCTV camera according to the present invention includes an image sensor which is a kind of a photoelectric conversion device that detects the information of a projectile visible through a fisheye lens and converts it into an electrical image signal. ; An image processor for converting an image signal transmitted from the image sensor into image information; A frame buffer connected to the image processor to temporarily store the converted image information; A video processor for planar imaging of image information transmitted from the image processor; It is characterized in that it is connected to the video processor DVR for recording the flattened image or a display showing the flattened image in the video processor.

The video processor includes a color space converter for converting YCbCr422 to RGB, a target address calculator for calculating target coordinates, a memory controller for controlling the frame buffer, and a color space converter for converting RGB to YCbCr422. It is characterized by.

The target address calculator converts the fisheye image into a planar image using a planarization algorithm.

int y = 0, x = 0, xr = 480, yr = 640, cx, cy, centerx = 240, centery = 320;

float r = 240, xreal, yreal, k, k2, kp, kptr;

int m_scaleImg [720 [960];

 for (x = 0; x <xr; x ++) {xreal = x;

         for (y = 0; y <yr; y ++) {yreal = y;

k2 = (yreal-yr / 2) * (yreal-yr / 2) + (xreal-xr / 2) * (xreal-xr / 2);

if (k2 == 0) kp = 0;

else {kptr = (4 * r * r * k2) / (4 * r * r + k2);

k = sqrt (k 2);

kp = kptr / k;}

if (y <(yr / 2) || x <(xr / 2)) {cy = centery-(kp / k) * (yr / 2-y);

cx = centerx-(kp / k) * (xr / 2-x);}

  if (y> (yr / 2) || x <(xr / 2)) {cy = centery + (kp / k) * (y-yr / 2);

cx = centerx-(kp / k) * (xr / 2-x);}

if (y> (yr / 2) || x> (xr / 2)) {cy = centery + (kp / k) * (y-yr / 2);

cx = centerx + (kp / k) * (x-xr / 2);}

else

              {cy = centery-(kp / k) * (yr / 2-y);

cx = centerx + (kp / k) * (x-xr / 2);}

m_scaleImg [x] [y] = m_OpenImg [cx] [cy]; }}

It is characterized by that.

As described above, the planar image converting device of the fisheye image of the CCTV camera according to the present invention has the advantage that it is possible to simultaneously monitor a large area without blind spots by collecting the image in a wide area at the same time as a CCTV camera that the curved image is converted in real time .

1 is a configuration diagram of a conventional multi-region monitoring system.
FIG. 2 is a diagram illustrating a geometric sample of a conventional coordinate transformation and a transformed image sample. FIG.
Figure 3 is a block diagram of the entire system of the planar image conversion apparatus for fisheye image of a CCTV camera according to the present invention.
4 is a block diagram showing signal processing of the image sensor of FIG.
5 is an internal block diagram of the video processor of FIG.

Hereinafter, a planar image converting device of a fisheye image of a CCTV camera according to the present invention will be described in detail with reference to the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, terms to be described below are terms defined in consideration of functions in the present invention, which may vary according to a client's or operator's intention or custom. Therefore, the definition should be based on the contents throughout this specification.

Like numbers refer to like elements throughout the drawings.

FIG. 3 is a block diagram illustrating an entire system for converting a fisheye image of a CCTV camera according to the present invention. FIG. 4 is a block diagram illustrating signal processing of the image sensor of FIG. 3, and FIG. 5 is a video processor of FIG. 3. Is an internal block diagram of.

Hereinafter, the planar image converting device of a fisheye image of a CCTV camera according to the present invention converts a curved image obtained from a fisheye lens into a planar image on a computer, and the following novel method can reduce the amount of computation of the algorithm to be processed in the computer. Uses a computerized flattening algorithm.

int y = 0, x = 0, xr = 480, yr = 640, cx, cy, centerx = 240, centery = 320;

float r = 240, xreal, yreal, k, k2, kp, kptr;

int m_scaleImg [720 [960];

 for (x = 0; x <xr; x ++) {xreal = x;

         for (y = 0; y <yr; y ++) {yreal = y;

k2 = (yreal-yr / 2) * (yreal-yr / 2) + (xreal-xr / 2) * (xreal-xr / 2);

if (k2 == 0) kp = 0;

else {kptr = (4 * r * r * k2) / (4 * r * r + k2);

k = sqrt (k 2);

kp = kptr / k;}

if (y <(yr / 2) || x <(xr / 2)) {cy = centery-(kp / k) * (yr / 2-y);

cx = centerx-(kp / k) * (xr / 2-x);}

  if (y> (yr / 2) || x <(xr / 2)) {cy = centery + (kp / k) * (y-yr / 2);

cx = centerx-(kp / k) * (xr / 2-x);}

if (y> (yr / 2) || x> (xr / 2)) {cy = centery + (kp / k) * (y-yr / 2);

cx = centerx + (kp / k) * (x-xr / 2);}

else

              {cy = centery-(kp / k) * (yr / 2-y);

cx = centerx + (kp / k) * (x-xr / 2);}

m_scaleImg [x] [y] = m_OpenImg [cx] [cy]; }}

The result of performing the FPGA circuitization to implement such a planarization algorithm in hardware is the entire system of the planar image converting device of the fisheye image of the CCTV camera according to the present invention as shown in FIG.

Referring to FIG. 3, the planar image converting device 30 of a fisheye image of a CCTV camera according to the present invention detects the information of a projectile visible through a fisheye lens 11 to detect an electric image signal. An image sensor 31, which is a type of photoelectric conversion device for converting a photoelectric device, an image processor 32 for converting an image signal transmitted from the image sensor 31 into image information, and connected to the image processor 32. Frame buffer 33 for temporarily storing the converted image information, a video processor 34 for planarizing the image information transmitted from the image processor 32, the video processor 34, DVR 35 for recording the planarized image or the display 36 representing the planarized image in the video processor 34.

The format from the image sensor 31 is 8-bit YCbCr422 for SD (normal definition) resolution and 16-bit YCbCr422 for HD resolution (high resolution), and the bit depth of the input video signal is the image. It may vary depending on the performance of the sensor 31. In addition, the frame buffer 33 is generally configured using a DRAM, which may be a storage means capable of storing a large amount of image data such as SDR SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM.

Referring now to FIG. 4, which shows in more detail the image sensor 31 shown in FIG. 3, the image sensor 31 includes two-wire serial interface registers 41, an image. Signal processor 42, timing controller 43, column decoder 44, ADC 45, CDS 46, pixel array 690 512, 47, Bias / ADC controller 48, raw decoder 49 Is made of.

Here, the digital control signal transmitted from the timing controller 43 is transmitted to the column decoder 44 and the row decoder 49, and the analog control signal is transmitted to the Bias / ADC controller 48.

In addition, the pixel array of the image sensor 31 is 690ｘ512, the output signal of the image sensor 32 is NTSC or PAL signal, the maximum frame rate is 30fps and the format of the output signal is CCIR656, 8-bit YCbCr422, 8-bit RGB565 You can select from among 9-bit RGB Bayer and 9-bit Mono. In addition, since the operation of the image sensor 31 is well known, a detailed description thereof will be omitted.

The image signal transmitted from the image sensor 31 is converted into the image information via the image processor 32, and the converted image information is imaged in a plane by the video processor 34, which is the video processor 34 Referring to FIG. 5, which shows an internal block diagram of the video processor 34, the video processor 34 includes a color space converter 51-1 for converting YCbCr422 to RGB, a target address calculator 52 for calculating coordinates of a target; And a memory controller 53 for controlling the frame buffer 33, and a color space converter 51-2 for converting RGB into YCbCr422.

Here, the CIR656 or 8-bit YCbCr422 video signal is input from the camera 12. The video signal is composed of one luminance signal Y per pixel and one color signal per two pixels. In order to image, the color space converter 51-1 converts the CIR656 or 8-bit YCbCr422 video signal into an RGB format color signal having independent pixel information per pixel. .

In addition, the target address calculator 52 calculates coordinates necessary for converting the fisheye image into the planar image by the planarization algorithm as described above, and uses the coordinate data calculated by the planarization algorithm and uses the frame buffer 33. A read address address of the frame buffer 33 required to read the fisheye image received from the camera 12 stored in the C) is generated.

In addition, the signal data of the fisheye image input from the camera 12 is converted into an RGB format through the color space converter 51-1 and stored in the frame buffer memory. The memory controller 53 stores the input image data. Write to the frame buffer 33 in units of frames; The target address calculator (52) writes the result value calculated by the planarization algorithm into the frame buffer (33); The address of the image data to be read among the input image data stored in the frame buffer 33 in units of frames is determined by using the result value of the flattening algorithm stored in the frame buffer 33, and the image data of the corresponding address is determined. Read function.

In addition, since the inputtable video signal format of a conventional video decoder IC is YCbCr422 or CCIR656 format, when the RGB video signal is input, the video signal cannot be encoded normally, and as a result, an abnormal video signal is output. The space converter 51-2 converts RGB pixel data output from the frame buffer 33 into YCbCr422 pixel data by the flattening algorithm.

The analog video signal output from the video processor 34 is used as an input video signal of a DVR or a video display (TV, monitor, etc.) which is an image storage device.

The planar image converting device of a fisheye image of a CCTV camera according to the present invention is a CCTV camera that converts a curved image in real time to simultaneously collect the image in a wide area can simultaneously monitor a large area without blind spots.

Although CIR656 or 8-bit YCbCr422 is used as an image signal input from the camera in the embodiment of the present invention, 8-bit RGB565, 9-bit RGB Bayer, 9-bit Mono, etc. may be used.

In addition, in the embodiment of the present invention, one or two frame buffers are used, but the number of such frame buffers is not limited.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Various changes, modifications or adjustments to the example will be possible. Therefore, the scope of protection of the present invention should be construed as including all changes, modifications, and adjustments that fall within the spirit of the technical idea of the present invention.

10: conventional omnidirectional monitoring system 10: fisheye lens
12: camera 13: imaging unit
14: Monitor 15: Computer
30: Planar video conversion device of fisheye video of CCTV camera
31: image sensor 32: image processor
33: frame buffer 34: video processor
35: DVR 36: Display
41: 2-wire serial interface register
42: image signal processor 43: timing controller
44: column decoder 45: ADC
46: CDS 47: pixel array
48: Bias / ADC Control Unit 49: Low Decoder
51-1, 51-2: color space converter
52: target address calculator 52 53: memory controller

Claims (3)

An image sensor which is a type of photoelectric conversion device that detects information of a wide viewing object through a fisheye lens and converts the information into an electrical image signal;
An image processor for converting an image signal transmitted from the image sensor into image information;
A frame buffer connected to the image processor to temporarily store the converted image information;
A video processor for planar imaging of image information transmitted from the image processor;
A device for converting a fisheye image of a CCTV camera, comprising: a DVR for recording a planarized image connected to the video processor or a display representing a planarized image in the video processor.
2. The video processor of claim 1, wherein the video processor comprises: a color space converter for converting YCbCr422 to RGB, a target address calculator for calculating coordinates of a target, a memory controller for controlling the frame buffer, and a color for converting RGB to YCbCr422. Planar image converting device for fisheye image of a CCTV camera, characterized in that consisting of a space converter. The planarization algorithm of claim 3, wherein the target address calculator converts the fisheye image into a planar image using a planarization algorithm.
int y = 0, x = 0, xr = 480, yr = 640, cx, cy, centerx = 240, centery = 320;
float r = 240, xreal, yreal, k, k2, kp, kptr;
int m_scaleImg [720 [960];
for (x = 0; x <xr; x ++) {xreal = x;
for (y = 0; y <yr; y ++) {yreal = y;
k2 = (yreal-yr / 2) * (yreal-yr / 2) + (xreal-xr / 2) * (xreal-xr / 2);
if (k2 == 0) kp = 0;
else {kptr = (4 * r * r * k2) / (4 * r * r + k2);
k = sqrt (k 2);
kp = kptr / k;}
if (y <(yr / 2) || x <(xr / 2)) {cy = centery-(kp / k) * (yr / 2-y);
cx = centerx-(kp / k) * (xr / 2-x);}
if (y> (yr / 2) || x <(xr / 2)) {cy = centery + (kp / k) * (y-yr / 2);
cx = centerx-(kp / k) * (xr / 2-x);}
if (y> (yr / 2) || x> (xr / 2)) {cy = centery + (kp / k) * (y-yr / 2);
cx = centerx + (kp / k) * (x-xr / 2);}
else
{cy = centery-(kp / k) * (yr / 2-y);
cx = centerx + (kp / k) * (x-xr / 2);}
m_scaleImg [x] [y] = m_OpenImg [cx] [cy]; The plane image conversion apparatus of the fisheye image of a CCTV camera, characterized in that the}}.

Images