KR20140137485A - System for multi channel display to use a fish-eye lens - Google Patents

Abstract

The present invention relates to a multichannel display system using a fisheye lens which comprises: an imaging device which has a fisheye lens mounted therein and captures an object; an image conversion control device which stores the object captured by the imaging device as a fisheye image and converts the fisheye image into a front image or a panorama image; and a monitor which displays the front image or the panorama image, wherein the image conversion control device displays at least one of the front image and the panorama image in the monitor. The multichannel display system of the present invention converts the fisheye image into the front image by correcting a range corresponding to a predetermined ratio in the fisheye image using a fisheye correction algorithm.

Description

SYSTEM FOR MULTI CHANNEL DISPLAY TO USE A FISH-EYE LENS

The present invention relates to a system for converting a fisheye image picked up by an image pickup means equipped with a fisheye lens into a planar image and more particularly to a system for converting a planar image obtained by restoring a fisheye image into a panorama, And a multi-channel display system using a fisheye lens capable of displaying the panorama image and the front side image on a monitor in multiple channels.

A fish-eye lens is an ultra-wide angle lens with a square angle (camera angle) of more than 180 °. Since the spherical surface is distorted, a wide range of images is formed.

This fisheye lens is the name given to the fact that when the light is refracted when entering the water, the fish has a 180 ° field of view when it is seen in the water, it is mainly used in the field of meteorology, It is used to record the cloud amount.

In recent years, fisheye lenses have been used in surveillance cameras such as CCTVs and automobile black boxes, thereby enabling a wider range of surveillance to be secured and reducing the angle of view (blind spot, range not visible at any angle).

At this time, the image formed on the fisheye lens is distorted by the distortion of the original image so that all the light rays (for example, ambient light) not passing through the center of the screen are distorted to form an image, The object is distorted into a round shape.

On the other hand, in recent years, due to the development of technology, in order to convert a fisheye image distorted by a fisheye lens into a planar image, it is corrected using software or a correction algorithm of a distorted image.

As a result, the fish-eye image distorted by the fish-eye lens can be easily seen with the naked eye.

A conventional system for correcting a distorted fisheye image related to the above-described system for correcting a fisheye image is configured as shown in Fig.

1 schematically shows an apparatus for correcting a conventional fisheye image.

1, includes an imaging unit 10 using a fisheye lens, a conversion unit 20 for converting an optical image obtained by the imaging unit 10 into image data, A computer 30 for correcting the image data of the image pickup device 20 into a plane image, and an outputting means 40 for outputting the plane image corrected by the computer 30.

FIG. 2 of the accompanying drawings shows a geometric shape of a conventional coordinate transformation and a transformed image sample. The transformed three-dimensional coordinate of a distorted image captured by a fish-eye lens is corrected to a plane image, FIG.

In order to correct the fisheye image by the fisheye lens, it is necessary to know the geometrical shape of the coordinate transformation. The coordinate (x, y, z) of the image in the sphere having the mathematical radius r as shown in Fig. Dimensional coordinates (x ', y'). This is expressed by the following equation.

Here, k is the distance from the center of the spherical surface to the three-dimensional coordinate (x, y, z), and kp is the distance from the center of the spherical surface to the two-dimensional coordinate (x ', y').

In such an arithmetic operation process, there is a problem that it is troublesome to separately calculate a value corresponding to each of the arithmetic operation processes, thereby complicating the arithmetic operation process.

In this case, since the calculation process is complicated, not only the time of the correction process is prolonged, but also the number of frames of the planar image is decreased and the buffering delay phenomenon of the image may occur and the real time monitoring becomes difficult due to the buffering delay phenomenon of the image .

In addition, although the above-described equations are mainly used in the calculation for correcting the fisheye image, the super-wide-angle fisheye lens has a problem that a correct image can not be obtained with a simple fisheye lens correction algorithm because a square angle is 180 degrees or more in a viewing angle range .

Accordingly, in picking up a fish-eye image, techniques for obtaining images of a higher image quality and a wider viewing angle are being developed by selectively capturing a plurality of images or selectively displaying one fisheye image separately.

On the other hand, in relation to a technique for displaying a fisheye image, Japanese Patent Application Laid-Open No. 10-2009-0044163 discloses a camera device for rearward surveillance, a control method thereof, and a vehicle including the same.

The technique described above is for enhancing the safety of the driver, stability of the driver, and convenience by installing an ultra-wide angle camera having a viewing angle of 130 to 190 degrees on the rear of the automobile to selectively monitor the left and right sides of the vehicle.

An ultra-wide-angle camera for selectively photographing the rear left and right sides including the rear or rear of the vehicle by adjusting the viewing angle by being installed at the rear portion of the vehicle; Adjusting means for selectively adjusting a viewing angle of the ultra-wide angle camera; And a display means installed in the interior of the vehicle for informing the driver of the image photographed by the ultra-wide angle camera.

However, the above-described technique displays an image through selective shooting. In order to display multiple images, the back side and the rear side including the rear side or the rear side are selectively photographed, and this is a troublesome There is a problem, and a direction must also be manipulated by the user, and in particular, there is a problem that an accident during operation can cause a greater accident.

Accordingly, it is possible to increase the image quality and the viewing angle of the image by obtaining the foreground and the entire image at one time by dividing the fisheye image into a range obtained by dividing the fisheye image into a front side image and a range of acquiring the panorama image, The development of technology that is required

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a panoramic image and a fisheye image in which a plane image is transformed into a panoramic image and a fisheye image, The present invention provides a multi-channel display system using a fisheye lens capable of displaying a panoramic image and a front side image on a monitor in a multi-channel manner.

An object of the present invention is to provide a multi-channel display system using a fisheye lens capable of displaying a flat image with a reduced number of frames through a fisheye correction algorithm that shortens the calculation time required for converting a fisheye image to a plane image.

In order to solve the above problems, the present invention provides an imaging apparatus comprising: an imaging device for mounting a fish-eye lens and imaging an object; An image conversion control device for storing an object photographed through an image pickup device as a fisheye image and converting the fisheye image into a front side image or a panorama image; And a monitor for displaying a front side image or a panorama image, wherein the image conversion control device displays at least one of a front side image or a panorama image on a monitor.

The present invention also provides a technical feature of restoring a fisheye image to a panorama and outputting it as a panorama image, and correcting a pixel corresponding to a predetermined range in a fisheye image and outputting it as a front side image.

The present invention can convert a fisheye image to a front side image by correcting a fisheye image using a fisheye correction algorithm,

As a technical feature.

The present invention has the effect of shortening the calculation time for correction since the fish-eye image captured by the fish-eye lens is corrected through the fish-eye correction algorithm.

Accordingly, it is an object of the present invention to prevent the number of frames of an image from being reduced, and to prevent a buffering delay phenomenon from occurring in a corrected image.

Further, the present invention can display a panoramic image converted by converting a fisheye image into a panorama image and a front side image transformed by correcting the range of a predetermined ratio in the fisheye image and displaying the same at the same time. It has a widening effect.

In addition, since the planar image can be obtained by dividing the planar image into a frontal image or a panoramic image, a planar image having an improved resolution can be obtained.

Further, the present invention has an advantage that a wider field of view can be ensured due to the imaging means provided with a fish-eye lens.

FIG. 1 schematically shows a conventional fisheye image correction apparatus.
Figure 2 shows a geometric form of a conventional coordinate transform and a transformed image sample.
FIG. 3 is a block diagram of a multi-channel display system using a fisheye lens system according to the present invention. FIG. 3 is a diagram illustrating a front side image a and a panorama image b transformed into a panorama by converting a full- FIG. 1 shows an example of a monitor for displaying a display screen.
FIG. 4 is a view showing a front side image (a) obtained by correcting a range of a fisheye image of a fisheye image of a multichannel display system using a fisheye lens according to the present invention, and a panorama image (b) Lt; RTI ID = 0.0 > a < / RTI >
FIG. 5 is a schematic view illustrating a fisheye image reconstructed into a panorama image and converted into a panorama image in a multi-channel display system using a fisheye lens according to the present invention.
6 shows a configuration of a multi-channel display system using a fish-eye lens according to the present invention.
FIG. 7 is a schematic view illustrating a geometric shape of a coordinate transformation and a transformed image sample for correcting and transforming a range corresponding to a certain ratio of a fisheye image to a frontal image in a multi-channel display system using a fisheye lens according to the present invention will be.

The terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary meanings and the inventor can properly define the concept of the term to describe its invention in the best possible way And should be construed in accordance with the principles and meanings and concepts consistent with the technical idea of the present invention.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents And variations are possible.

Before describing the present invention with reference to the accompanying drawings, it should be noted that the present invention is not described or specifically described with respect to a known configuration that can be easily added by a person skilled in the art, Sounds good.

In other words, the present invention can mainly describe a configuration (for example, an imaging means, an image conversion control device, and a monitor) for achieving the object according to the present invention. It is omitted.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for converting a fisheye image picked up by an image pickup means equipped with a fisheye lens into a planar image, and more particularly, a fisheye correction algorithm can be used to shorten the calculation time of correction.

Therefore, the number of frames of the planar image can be reduced, thereby preventing the occurrence of the buffering delay phenomenon in the planar image.

Here, the planar image means an image obtained by converting camera coordinates of an object of a third-dimensional object captured through an image pickup means equipped with a fish-eye lens into coordinates of a plane coordinate system existing on a two-dimensional plane and converting the coordinate into a plane image .

In addition, the above-described plane image can be divided into a panoramic image obtained by restoring a fisheye image into a panorama, and a front side image transformed by correcting a pixel range corresponding to a predetermined ratio in the fisheye image.

At this time, the pixel range corresponding to a certain ratio in the fisheye image is a range of pixels corresponding to the predetermined range among the pixels constituting the fisheye image.

Depending on other design conditions, it may also be a predetermined range around the fisheye image.

Here, the predetermined range is a range corresponding to the front portion of the image pickup means, and the range can be divided into pixels, but the direction value of the vector may be set according to another design.

The panorama image means a planar image obtained by restoring a fisheye image into a panorama and converting it.

A multichannel display system using a fisheye lens according to the present invention is a system for displaying a panorama image converted from a fisheye image into a panorama and correcting a range corresponding to a predetermined ratio of the fisheye image, So that a wider field of view can be provided to the user.

The display of the corrected and reconstructed fisheye image through the monitor will now be described with reference to FIG.

FIG. 3 is a view showing a front side image (a) obtained by correcting and converting a range of a fisheye image of a multi-channel display system using a fisheye lens according to the present invention and a panorama image (b) obtained by converting the entire fisheye image into a panorama, FIG. 1 shows an example of a monitor for displaying a display screen.

An image picked up by the image pickup means having a fisheye lens can be formed as an image as shown in Fig.

When the image is formed as shown in FIG. 3, the image is more distorted as the focal point moves away from the center of the fisheye image.

Such a fisheye image can be converted into a front side image by correcting a predetermined ratio of the image as described above, and a certain ratio of the fisheye image is a range of pixels corresponding to a preset range among pixels constituting a fisheye image.

Also, the predetermined range to be corrected in the fisheye image is a pixel within a range corresponding to a predetermined range of pixels constituting the fisheye image as described above, or a range set based on the center of the fisheye image.

For example, in the case of the fisheye image shown in FIG. 3, pixels corresponding to the set range from the center of the fisheye image are separated from the pixels constituting the fisheye image, and the range including the divided pixels is corrected.

In other words, as shown in Fig. 3 (a), the front side image converted by correcting a certain ratio of the fisheye image and the panorama image converted into the panorama image by transforming the entire fisheye image as shown in Fig. 3 (b) And displays it on the monitor 300 in multiple channels.

At this time, the display on the monitor 300 can display both the corrected front side image and the restored panorama image, and display the screen selected by the user's selection.

In addition, in the case of a fish-eye image picked up by a fish-eye lens, an image can be formed with an image of 360 degrees, which will be described with reference to FIG.

FIG. 4 is a view showing a front side image (a) obtained by correcting and converting a range of a fisheye image of a multi-channel display system using a fisheye lens according to the present invention and a panorama image (b) obtained by converting the entire fisheye image into a panorama, Lt; RTI ID = 0.0 > a < / RTI >

4, it is possible to convert a predetermined ratio of the image into a front side image even in the case of a fisheye image formed with an image of 360 degrees.

However, since the center of the fisheye image can not necessarily be the front side image, unlike the case of FIG. 3, the range of the predetermined ratio is corrected and converted into the front side image.

Also in this case, it is needless to say that the fisheye image can be converted into the panorama image by restoring the fisheye image into the panorama image.

In this case as well, the corrected frontal image and the restored panoramic image can be displayed on the monitor.

In other words, in accordance with FIG. 3 or FIG. 4 of the accompanying drawings, a fisheye image captured by the image pickup apparatus 100 equipped with a fisheye lens is a type of a fisheye lens (diagonal fisheye lens, Etc.), the fisheye image is converted into a corrected frontal image or a reconstructed panoramic image, and the frontal image and / or panoramic image is displayed on the monitor as described above.

Also, displaying the frontal and / or panoramic images on the monitor may include displaying the frontal and panoramic images on the monitor in multiple channels, or displaying at least one of the frontal or panoramic images on the monitor .

The restoration of a fisheye image into a panorama and conversion into a panorama image will be described with reference to FIG.

FIG. 5 is a schematic view of a multi-channel display system using a fisheye lens according to the present invention, in which a fisheye image is converted into a panorama image and converted into a panorama image.

Here, restoration of a fisheye image into a panorama image uses a conventional restoration method, and a detailed description thereof will be omitted, and will be briefly described below with reference to the accompanying drawings.

An image formed by a fish-eye lens is spread as a plane image around a certain reference point as shown in Fig. 5 (a) (Fig. 5 (b)).

Since the distorted fisheye image is spread as a plane image, distortion occurs in the plane image, and the resolution of the plane image in which the distortion is present is adjusted to restore the plane image free from distortion (FIG. 5 (c) ).

The reconstructed planar image is a panoramic image. At this time, the reconstruction of the panoramic image may be performed through a conventional equation, or may be performed by software or hardware of a computer.

The configuration of a multi-channel display system using such a fisheye lens will be described with reference to FIG.

FIG. 6 shows a configuration of a multi-channel display system using a fish-eye lens according to the present invention, and is divided into an image sensing apparatus 100, an image conversion control apparatus 200, and a monitor 300 according to the attached FIG.

The imaging apparatus 100 is sufficient for the imaging apparatus 100 in which a fish-eye lens is mounted as an apparatus for imaging an object.

At this time, the fish-eye lens preferably has an angle of 180 degrees or more.

In order to achieve the object of the present invention, the image conversion control apparatus 200 performs control for converting a fisheye image captured by the image pickup apparatus 100 into a front side image or a panorama image, 210, an image correction module 210, an image restoration module 230, and an output control module 240.

The image storage module 210 stores an image captured by the image capturing apparatus 100.

Here, the image is stored in RGB (Red, Green, Blue) by a CMOS image sensor, and the conventional method stores the image. Therefore, the image may be stored by a method other than a CMOS image sensor.

The image correction module 220 corrects a range of a predetermined ratio of the stored fisheye images in the image storage module 210, and outputs the front side images.

At this time, a certain ratio of the fisheye image may preferably be a range of pixels as described above.

In other words, the frontal image means the converted plane image by correcting the range corresponding to a certain ratio in the fisheye image.

The image correction module 220 includes a range setting unit and a range correction unit.

The range setting unit sets a portion corresponding to a certain ratio of the fisheye image to a range to be corrected.

At this time, the range setting unit is not limited to the type of the fisheye lens (as described above, diagonal fisheye lens, original fisheye lens, and 360 ° fisheye lens, etc.) as described above.

This is because the range corresponding to a certain ratio is corrected in the fisheye image and acquired as the front side image, so that a certain range can be set and corrected irrespective of the fisheye image formed depending on the type of the fisheye lens.

More preferably, in the case of a fisheye image having a quadrangle image, it is also possible to set the range from the center of the fisheye image to a certain range to be corrected with a certain ratio.

This is because the center of the image may be the center of the image sensing means 100, as in the case of a general image, because a fish-eye image having an image of a square (in the case of a diagonal fisheye lens and a fisheye fisheye lens)

Further, in the case of a fish-eye image (when picked up by a 360 ° fisheye lens) distorted by an image of 360 ° according to other design conditions, pixels in a range corresponding to a certain ratio in the fisheye image are corrected, . ≪ / RTI >

The range correcting unit corrects the range set by the range setting unit using the fish-eye correcting algorithm. In order to correct the range corresponding to a certain ratio of the fish-eye image, it is necessary to understand the mathematical expression of the geometric form, 7, it is as follows.

FIG. 7 is a schematic view showing a geometric shape of a coordinate transformation and a transformed image sample for correcting and transforming a predetermined range into a front side image in a multi-channel display system using a fisheye lens according to the present invention.

7, the object b imaged to have three-dimensional coordinates on the spherical surface having the radius R has three-dimensional coordinates of (X, Y, Z, not shown) according to the three-dimensional coordinate system, Dimensional coordinates of (x, y).

The conversion of the world coordinate system of the object into the three-dimensional coordinates of the camera and conversion of the two-dimensional coordinates of the plane image into the two-dimensional coordinates of the camera uses conventional parameter conversion, and a detailed description thereof will be omitted.

7, r denotes a distance from the center of the spherical surface to the two-dimensional coordinate of the distorted plane image (hereinafter referred to as a distance r), r 'denotes a distance from the center of the spherical plane to the plane image (Hereinafter referred to as a distance r ').

The fuzzy correction algorithm according to the present invention calculates the distance r 'based on the distance r and calculates the distance r' based on the calculated distance r, the distance r ' (x ', y') of the plane image whose distortion is corrected based on the coordinates (x, y) of the plane image.

The mathematical expression for this is described as follows.

Here, the distance r and the distance r 'are calculated from the distance from the center of the spherical surface to the two-dimensional coordinates of the distorted plane image and the distance from the center of the spherical plane to the two- .

Also, R represents the radius of the spherical surface, x represents the x-axis coordinate of the plane image distorted in the plane coordinate system, and y represents the y-axis coordinate of the plane image distorted in the plane coordinate system.

In addition,? Represents the angle between the center of the spherical surface and the object (b) (refer to FIG. 7), x 'represents the x-axis of the plane image corrected for distortion in the plane coordinate system, y' Axis of the corrected planar image.

The fish-eye correction algorithm according to the present invention consists of calculating the distance r as shown in the attached equation (1) and calculating the distance r 'based on the calculated distance r.

Dimensional coordinate (x, y) corrected based on the distance r and the distance r 'calculated by the above-mentioned equation (1) x ', y') can be calculated.

The correction algorithm of the distorted fisheye image thus obtained has the effect of shortening the calculation time since the distance r or the distance r 'is calculated in association with the two-dimensional coordinates (x, y) on the plane image .

Accordingly, there is an advantage that the number of frames of the planar image is reduced and buffering delay phenomenon of the image, that is, image disconnection can be prevented.

The image restoration module 230 restores a fisheye image captured through the fisheye lens into a panorama image.

In other words, the panoramic image is a planar image converted by restoring a fisheye image into a panorama.

In addition, restoration of the panoramic image can be performed as shown in FIG. 5, which uses conventional panorama restoration, and a detailed description thereof will be omitted.

The output control module 240 displays the corrected and restored image on the monitor 300 by the image correction module 220 and the image restoration module 230 described above.

At this time, the image is displayed on the monitor 300, and the corrected frontal image and the restored panoramic image are both displayed, thereby providing the user with multiple channels.

Also, according to other design conditions, a corrected frontal image or a restored panoramic image may be selectively provided according to a user's selection. In this case, the image selection module may further include an image selection module (not shown).

The monitor 300 displays the plane image corrected and restored by the image conversion control apparatus.

More preferably, the monitor 300 may include at least two or more display means.

This is because display means for providing the front side image and the panorama image at the same time should be provided and display means for selectively providing one image may be further provided according to another design condition.

As described above, the multi-channel display system using the fisheye lens according to the present invention has the effect of shortening the calculation time for the correction because the fisheye image captured by the fisheye lens is corrected through the fisheye correction algorithm.

Further, the present invention can display a panoramic image converted by converting a fisheye image into a panorama image and a front side image transformed by correcting the range of a predetermined ratio in the fisheye image and displaying the same at the same time. It has a widening effect.

In addition, since the planar image can be obtained by dividing the planar image into a frontal image or a panoramic image, a planar image having an improved resolution can be obtained.

3 to 7 are merely the main points of the present invention. As various designs can be made within the technical scope of the present invention, the present invention is limited to the configurations of Figs. 3 to 7 It is self-evident.

100: Image pickup device 200: Image conversion control device
210: image storage module 220: image correction module
230: image restoration module 240: output control module
300: Monitor

Claims (4)

An image pickup device for mounting a fish-eye lens and picking up an image of an object;
An image conversion control device that stores an object captured through the image pickup device as a fisheye image and converts the fisheye image into a front side image or a panorama image; And
And a monitor for displaying the front side image or the panorama image,
Wherein the image conversion control device comprises:
Wherein at least one of the front side image or the panorama image is displayed on a monitor.
The method according to claim 1,
The front-
Wherein the image is a transformed image obtained by correcting a range of a ratio of the fisheye image.
The method according to claim 1,
The panoramic image may include:
Wherein the fisheye image is a transformed image obtained by restoring the fisheye image into a panorama.
The method according to claim 1,
Wherein the image conversion control device comprises:
Wherein a fisheye image is converted into a front face image through a fisheye correction algorithm using the following equation.

(Where r is the distance from the center of the spherical surface to the two-dimensional coordinates of the distorted planar image, r 'is the distance from the center of the spherical surface to the two-dimensional coordinates of the distorted planar image, R is the radius of the spherical surface, Y is the coordinate of the y-axis of the plane image distorted in the plane coordinate system,? Is the angle between the center of the spherical surface and the object (b), and x 'is the distortion of the plane image in the plane coordinate system The x-axis of the planar image, y 'is the y-axis of the plane image whose distortion is corrected in the plane coordinate system).

Images