KR101011704B1 - Apparatus and method for processing video signal to generate wide viewing image - Google Patents

Abstract

PURPOSE: An image processing device producing an image of wide angle and a method thereof are provided to perform image conversion for obtaining a single wide angle image by one lookup processing process at once. CONSTITUTION: A mirror(114,115) is installed in between each plurality of cameras(111~113). A receiving unit(12) receives a plurality of capture images of a plurality of cameras. An image processing unit(14) revises a distortion in a plurality of images through a lookup table. An image processing unit projects a phase of a subject in a plurality of images on a plane. An image processing unit connects a plurality of images to a single wide angle image. An output unit(15) outputs an output image.

Description

Image processing apparatus and method for generating wide-angle image {APPARATUS AND METHOD FOR PROCESSING VIDEO SIGNAL TO GENERATE WIDE VIEWING IMAGE}

The present invention relates to an image processing apparatus and method for generating a wide-angle image.

In general, a wide-angle camera device capable of acquiring a wide range of images at present is largely mechanically a system using a PTZ (Pan / Tilt / Zoom) camera, a system using a camera with a fisheye lens, and a linear image sensor. Such as a panoramic camera system to obtain a 360 ° image.

A fisheye lens intentionally creates a cylindrical shape, and maintains uniform brightness and sharpness over the entire angle of view of 180 degrees or more. The subject of the lens center point is extremely large and the surroundings are very small. That is, the fisheye lens can acquire a wide range of images, but the acquired image at this time has a severe distortion.

A wide-angle camera device for acquiring such wide-area images is also used for a vehicle rearview device. The rear view device of the car is a camera installed at the rear of the car, and the driver checks the rear of the picture through the monitor. The field of view seen through the objective lens of a camera depends on the angle of the lens. In general, when the angle of view of the lens exceeds about 120 degrees, the distortion of the image is severe, and more lenses are required to correct this, which increases the cost.

An object of the present invention is to provide an image processing apparatus and method capable of acquiring a high resolution video image without distortion with a low cost configuration while securing a wide range of field of view.

An image processing apparatus according to an embodiment of the present invention is an image processing apparatus that receives images from a plurality of cameras and generates a wide-angle image covering a wide range of angles of view, and receives a plurality of images captured by the plurality of cameras. ; A memory unit for storing a lookup table for converting the plurality of images into a single wide-angle image; A correction process of correcting distortion included in the plurality of received images, a projection process of spreading an image of a subject included in the plurality of received images on a plane, and the plurality of received images using the lookup table An image processor configured to process an image to collectively perform a bonding process of bonding the single wide-angle image to generate an output image; And an output unit configured to output the output image to be displayed on a predetermined display.

In an image processing method according to an embodiment of the present invention, an image processing method for generating a wide-angle image covering a wide field of view by receiving images from a plurality of cameras, the method comprising: receiving a plurality of images captured by the plurality of cameras ; A correction process for correcting distortion included in the plurality of received images using a lookup table stored in a memory, a projection process of spreading images of a subject included in the plurality of received images on a plane, and the received plurality of images Generating an output image by collectively performing a bonding process of joining images into the single wide-angle image; And outputting the output image to be displayed on a predetermined display.

The image processing method according to an embodiment of the present invention described above can be recorded on a computer-readable recording medium having a program for realizing each step.

According to the present invention, it is possible to obtain a wide-angle image covering a wide range of field of view with a low-cost device configuration.

According to the present invention, image distortion is achieved by correcting distortion caused by a lens included in an image, a projection of spreading an image captured by an image sensor on a plane, and combining a plurality of images received from a plurality of cameras to obtain a single wide-angle image. By using a single lookup table, a single image processing process can be performed at a time, so that images can be output more quickly.

1 is a block diagram of an image processing apparatus according to an embodiment of the present invention.
2 is a schematic diagram illustrating a structure of a plurality of cameras according to an embodiment of the present invention.
3 is a schematic diagram illustrating an arrangement structure between a center of an angle of view and a mirror according to an embodiment of the present invention.
4 is a view comparing an image loss range according to an arrangement between a view angle center and a mirror of a camera according to an embodiment of the present invention.
5 is a view illustrating a process of combining images acquired from two cameras according to an embodiment of the present invention.
6 is a diagram illustrating a configuration of a lookup table according to an embodiment of the present invention.
7 is a schematic view illustrating a structure of a camera unit according to another embodiment of the present invention.
8 to 10 are views illustrating a process of generating an output image by combining images obtained from three cameras according to an embodiment of the present invention.
11 to 16 are diagrams illustrating a board and its arrangement used in three projection schemes according to an embodiment of the present invention.
17 is a flowchart illustrating an image processing method according to an embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an image processing apparatus 1 according to an embodiment of the present invention.

As shown in FIG. 1, an image processing apparatus 1 according to an exemplary embodiment of the present invention receives an image from a plurality of cameras 111, 112, and 113 and generates a wide-angle image covering a wide field of view. As an example, the receiver 12 receives a plurality of images captured by the plurality of cameras 111, 112, and 113, and a memory unit 13 stores a look up table for converting the plurality of images into a single wide-angle image. A correction process of correcting distortion included in the plurality of received images by using the lookup table, a projection process of spreading an image of a subject included in the plurality of received images on a plane, and the plurality of received images An image processing unit 14 for processing an image to collectively perform a bonding process for bonding a single wide-angle image into an output image, and the output image And an output unit 15 for outputting to be displayed on a predetermined display.

According to an embodiment of the present invention, the image processing apparatus 1 includes a plurality of mirrors 114 and 115, which are reflection surfaces on which one surface receives light, and a center of the angle of view is symmetrically disposed with respect to the mirrors 114 and 115. The cameras 111, 112, and 113 may further include.

That is, the image processing apparatus 1 of the present invention may be integrally provided with an image processor which processes the plurality of images to generate a wide-angle image, and a plurality of cameras that capture the images. However, the present invention is not limited thereto, and the plurality of cameras 111, 112, and 113 and the image processing module may not be integrally formed, but may be configured and used as individual modules.

Similarly, as shown in FIG. 1, the display unit 16 for displaying the generated wide-angle image may also be formed integrally with the image processing apparatus 1 of the present invention, but is not limited thereto. 16 may be provided as separate modules to be assembled and used with each other.

The plurality of cameras 111, 112, and 113 capture images at different angles of view.

2 is a schematic diagram showing a structure of a camera unit according to an embodiment of the present invention.

The camera unit illustrated in FIG. 2 includes two cameras 111 and 112 and one mirror 114 to cover a wide angle of view.

As shown in FIG. 2, a plurality of cameras 111 and 112 in which the centers of the angles of view are symmetrically arranged with respect to the mirror 114 and the mirror 114, which are reflection surfaces on which one surface receives and reflects light, are illustrated in FIG. It may include.

That is, the center 201 of the field of view covered by the first camera 111 and the center 202 of the field of view covered by the second camera 112 are disposed symmetrically with respect to the mirror 114. The distances from the centers 201 and 202 of the respective angles of view to the mirror 114 are the same.

Here, the first camera 111 includes a lens 211 and an image sensor 212. In addition, the second camera 112 also includes a lens and an image sensor to collect the light reflected from the subject to capture the image.

The image processing apparatus 1 according to an exemplary embodiment of the present invention acquires an image having a wide angle of view by using two cameras 111 and 112. For example, the image processing apparatus 1 may secure an angle of view of about 150 ° by using two cameras 111 and 112. The image processing apparatus 1 may generate a single image having a wide angle of view by processing two images acquired from the two cameras 111 and 112.

Here, the centers 201 and 202 of the angles of view of the two cameras 111 and 112 are symmetrically positioned with respect to the mirror 114. As a result, the second camera 112 obtains the reflected image according to the angle of the mirror 114. This image is the same as the image viewed from the center 201 of the angle of view of the first camera 111.

As described above, since the centers 201 and 202 of the angles of view of each of the cameras 111 and 112 are symmetrical with respect to the mirror 114, the first image acquired, that is, the image captured by the first camera 111 and the first Using the second image, that is, the image acquired by the second camera 112, an angle of view of 150 ° horizontally can be easily secured without a separate image bonding algorithm.

According to an embodiment of the present invention, the centers 201 and 202 of the angles of view of the plurality of cameras 111 and 112 may be symmetrical with respect to the center line of the mirror 114 located between one surface and the other surface of the mirror.

For example, as shown in FIG. 3, the centers of the angles of view 201 and 202 are centerlines of the mirrors positioned between one surface 302 of the mirror 114, for example, the reflective surface and the other surface 303 opposite thereto. 301 may be arranged symmetrically.

That is, the centers 201 and 202 of the angle of view are arranged based on the center line 301 positioned at the center of the one surface 302 and the other surface 303 without reference to any one surface of the mirror 114.

As a result, when combining two images to generate one wide-angle image, only the image having the same width as that of the mirror 114 is lost, thereby minimizing the image lost due to the thickness of the mirror 114. .

4 is a view comparing an image loss range according to an arrangement between a view angle center and a mirror of a camera according to an embodiment of the present invention.

As shown in (a) of FIG. 4, according to an embodiment of the present invention, the angle of view centers 201 and 202 of the camera may be symmetrically disposed with respect to the centerline 301 of the mirror 114. As such, when the angle of view centers 201 and 202 of the camera are symmetrically arranged with respect to the centerline 301 of the mirror 114, the loss of an image due to the thickness of the mirror 114 is the same width as the thickness of the mirror 114. As much as it is lost, and at a distance (e.g., tens of centimeters to several meters) between the camera and the subject that is generally considered, the missing portion may be less than one pixel.

In addition, as shown in (b) of FIG. 4, according to another embodiment of the present invention, the angle of view centers 201 and 202 of the camera may be symmetrically disposed with respect to the reflective surface 302 of the mirror 114. have. As such, when the angle of view centers 201 and 202 of the camera are symmetrically arranged with respect to the reflective surface 302 of the mirror 114, the missing portion of the image due to the thickness of the mirror 114 is shown in FIG. Is greater than the missing part of the.

In addition, as shown in (c) of FIG. 4, according to another embodiment of the present invention, the angle of view centers 201 and 202 of the camera may be the reflection surface 302 of the mirror 114 or the centerline of the mirror 114. 301 may be arranged asymmetrically rather than symmetrically.

Specifically, as shown in FIG. 4C, the distance d1 from the centerline 301 of the mirror 114 to the first angle of view center 202 located on the reflective surface 302 side of the mirror 114 and The distance d2 from the center line 301 of the mirror 114 to the second view angle center 201 located on the other surface 303 which is the opposite surface of the reflective surface 302 of the mirror 114 may be different.

According to one embodiment, the second angle of view center 201 located on the other surface 303 side of the mirror 114 is compared to the first angle of view center 202 located on the reflective surface 302 side of the mirror 114. It is disposed farther away from the centerline 301 of 114, that is, d2 may be configured larger than d1.

According to the arrangement between the view angle centers 201 and 202 of the camera as shown in FIG. 4C and the mirror 114, light blurring in the image that may occur due to the arrangement as shown in FIGS. 4A and 4B. The phenomenon can be improved.

Specifically, according to the arrangement of FIGS. 4 (a) and 4 (b), the cameras 111 and 112 are arranged because the angles of view center 201 and 202 collect light passing through the corner portions 1141 and 1142 of the mirror 114. In the captured image, an area corresponding to the edges 1141 and 1142 of the mirror 114 may be dark or poor in image quality due to blurring.

This occurs because the corners 1141 and 1142 of the mirror 114 are too close to the camera and are out of focus, and the arrangement illustrated in FIG. 4C may improve poor image quality of the image.

Referring to FIG. 4C, a camera positioned on the reflective surface 302 side of the mirror 114, that is, a camera having a first angle of view center 202, is reflected on all of the reflective surfaces 302 of the mirror 114. 4 (a) and 4 (b), which use the captured image, only the image reflected on a part of the reflective surface 302 of the mirror 114 is used as the output image.

In other words, as shown in FIG. 4 (c), the camera located on the reflective surface 302 has a predetermined distance from the edge 1142 of the mirror among the images acquired from the entire reflective surface 302 of the mirror 114. An output image is generated by excluding an image corresponding to a part of the reflective surface, that is, an image portion in which light blurring occurs due to an edge. As a result, the deterioration of the image quality due to light bleeding can be improved as compared to FIGS. 4 (a) and 4 (b) which generate an image obtained from the entire reflection surface 302 of the mirror 114 as an output image. .

Similarly, in FIG. 4C, the camera located on the other surface 303 side of the mirror 114, that is, the camera whose center of view is the second view angle center 201, is output to a portion adjacent to the edge 1141 of the mirror 114. 4 (a) and 4 (b), which generate the L, the area corresponding to the predetermined distance from the edge 1141 of the mirror 114 is excluded from the output image without generating the image as the mirror 114 Deterioration of image quality due to light bleeding caused by the edges of

As a result, the structure of FIG. 4 (c) may have a larger loss area than the structure of FIG. 4 (a). However, in the general camera operating area, the loss area is less than 1 pixel, and the corner 1141, The light bleeding caused by 1142 may be eliminated and as a result, a better image may be provided.

5 is a view illustrating a process of combining images acquired from two cameras according to an embodiment of the present invention.

FIG. 5 is an embodiment for generating one wide-angle image by acquiring image data from two image sensors provided in the two cameras 111 and 112 shown in FIG.

The receiver 12 of the image processing apparatus 1 according to an exemplary embodiment of the present invention receives original image data from two image sensors. The received original image is distorted due to lens distortion. For example, a subject that is actually straight may appear as a curve in the original image. This is a distortion of the lens of the camera is made of a spherical shape rather than an elliptical shape, the present invention uses a look-up table to correct the distortion included in the image.

In addition, the image processing apparatus 1 of the present invention performs a conversion process of combining the plurality of received images into a single wide-angle image.

Specifically, as shown in FIG. 5, the image processor 14 of the present invention combines the first image received from the first camera 111 and the second image received from the second camera 112 to form a single image. Create a wide angle image.

According to an embodiment, all or part of the first image and the second image may be collected to generate a wide-angle image. For example, in FIG. 5, the first image includes all of the 480 × 600 images in the wide angle image, but the second image includes only 320 × 600 images corresponding to some of the original images in the wide angle image.

As a result, the image processor 14 may generate an 800 × 600 wide-angle image and output an 4: 3 ratio output image.

However, the size of the generated wide-angle image does not necessarily match the sum of the sizes of the input image, that is, the first image and the second image, and according to an embodiment, the size of the generated wide-angle image may be freely set.

Further, according to the embodiment, the ratio of the image acquired by the image processing apparatus 1 of the present invention from a plurality of original images may be changed according to the embodiment, and the image processing unit may output an image in accordance with a predetermined image standard. Can be generated. For example , The output image may have a 16: 9 ratio rather than a 4: 3 ratio.

In FIG. 5, an image acquired from the first image covers an angle of view of 90 °, and an image obtained from the second image covers an angle of view of 60 °, thereby obtaining a wide-angle image covering a total angle of view of 150 °. .

The present invention provides a distortion correction process for correcting distortion generated by a lens, a bonding process of bonding the plurality of received images to a single wide-angle image, and a projection of spreading the image captured by the image sensor on a plane. ) Process is performed in batch using one lookup table.

That is, the present invention does not include a separate distortion correction lookup table for distortion correction, a separate projection lookup table for projection, and a separate lookup table for image bonding, and one lookup. It has the advantage of simplifying the image processing process into a single process by carrying out the above-described correction, projection and bonding processes all at once with only a table.

As a result, according to the present invention, the image processing time required to output a single wide-angle image using a plurality of cameras is shortened, so that image processing is performed in real time.

The lookup table used in the above process is stored in the memory unit 13 included in the image processing apparatus 1 of the present invention, and FIG. 6 is a diagram illustrating the configuration of a lookup table according to an embodiment of the present invention.

As shown in FIG. 6, one coordinate of a plurality of original images received from a plurality of cameras is mapped to one coordinate of a wide-angle image generated by image processing.

Here, in coordinates (n, x, y) constituting the left column of the lookup table, n is a component representing a camera, x is a component representing an x-axis coordinate value, and y is a component representing a y-axis coordinate value.

Preferably, n of the coordinates of the input image is an integer, x and y are real numbers, and X and Y of the coordinates of the output image are integers. However, the present invention is not limited thereto, and x and y may be configured as integers, and X and Y may be configured as real numbers.

For example, n = 1 for the first camera 111, n = 2 for the second camera 112, and the coordinates (1,3.2,2.1) of the input image are the first received from the first camera. A pixel value corresponding to the coordinate (3.2,2.1) of the image is shown, and according to the lookup table of FIG. 6, the pixel value is mapped to the coordinate (0,0) of the output image.

Similarly, the coordinates (1,4.3,2.2) represent pixel values corresponding to the coordinates (4.3,2.2) of the first image received from the first camera, which are mapped to the coordinates (1,0) of the output image.

In this way, multiple images, i.e. the first and second images, can be converted into a single wide-angle image, and the present invention provides a correction process for correcting distortion by the lens and an image captured by the image sensor through the lookup table. Perform a projection process that develops on a plane all at once.

As a result, there is no need to perform distortion correction, projection, and image combining separately, and the time required for image processing is reduced to enable real-time image output.

The look-up table described above is predetermined when the image processing apparatus 1 is manufactured and stored in the memory unit 13.

7 is a schematic view illustrating a structure of a camera unit according to another embodiment of the present invention.

Unlike the camera unit illustrated in FIG. 2, the camera unit illustrated in FIG. 7 includes three cameras 111, 112 and 113 and two mirrors 114 and 115, and each mirror 114 and 115 is disposed between the plurality of cameras 111, 112 and 113. It is provided.

An arrangement angle of the mirrors 114 and 115 may be adaptively disposed according to a distance from the centers 201, 202, and 203 of the angles of view of the cameras 111, 112, and 113 to cover a wider angle of view than the camera unit of FIG. 2.

In the camera unit illustrated in FIG. 7, one surface of the mirrors 114 and 115 is a reflective surface that receives and reflects light, and the plurality of cameras 111, 112, and 113 have symmetric centers 201, 202, and 203 of the angle of view based on the mirrors 114 and 115. Is deployed.

Specifically, the center 201 of the angle of view of the first camera 111 and the center 202 of the angle of view of the second camera 112 are symmetrically disposed with respect to the first mirror 114, and the first camera ( The center of view angle 201 of 111 and the center of view angle 203 of the third camera 113 are symmetrically disposed with respect to the second mirror 115.

In FIG. 7, the reflective surfaces of the mirrors 114 and 115 are disposed to face outwards, and similarly to FIG. 3, the centers 201, 202, and 203 of the angles of view of the plurality of cameras 111, 112, and 113 are one surface and the other surface of the mirrors 114, 115. It may be arranged to be symmetrical with respect to the centerline of the mirror located between.

That is, the centers of the angles of view of the cameras 111, 112, and 113 are not symmetrical with respect to any one side of the mirrors 114, 115, but are symmetrical with respect to the centerline of the mirror positioned at the center of one side and the other side of the mirror.

In addition, as described above with reference to FIG. 4C, the center of the angle of view may be asymmetrically disposed with respect to the center line of the mirror to remove light bleeding caused by the edge of the mirror, thereby obtaining a good image. .

The camera unit illustrated in FIG. 7 may secure an angle of view of about 180 ° by using three cameras.

8 to 10 are views illustrating a process of generating an output image by combining images obtained from three cameras according to an embodiment of the present invention.

As shown in FIG. 8, the receiver 12 receives three original images, namely, first, second and third images, from the three cameras 111, 112, and 113.

Then, the image processor 14 corrects the distortions included in the three original images by using the lookup table stored in the memory 13, and performs a projection process of spreading the images on a plane. Convert the original image into one wide-angle image.

Similarly, according to an embodiment, the image processor 14 may acquire all or part of the original image and combine it into an output image.

For example, as illustrated in FIG. 8, the image processor 14 may extract only an image corresponding to some of the first, second and third images, and combine the same to generate an output image having a predetermined size. have. The size of the output image may coincide with the sum of the sizes of the input image, but the present invention is not limited thereto, and the size of the output image may be arbitrarily set and output.

As a result, an image corresponding to an angle of view of 60 degrees from each original image is combined so that the output image can cover a total angle of view of 180 degrees.

According to an embodiment, the size of the image extracted from the original image may be changed according to the size or aspect ratio of the output image.

For example, as shown in FIG. 9, the image processor 14 extracts only a part of a predetermined size image from an image of 480 × 752 to generate one output image, and as a result, the output image is a predetermined size. It has a 16: 9 aspect ratio.

Through this image processing, the image processing unit 14 of the present invention generates a single wide-angle image as an output image, and the output unit 15 outputs the single wide-angle image to be displayed on the display unit 16.

According to another embodiment of the present invention, the image processor 14 may generate the output image by dividing the single wide-angle image into a plurality of divided images. Then, the output unit 15 may output the plurality of divided images to be displayed on the display unit 16 as an output image.

10 is a diagram illustrating a process of generating a plurality of divided images according to another exemplary embodiment of the present invention.

As illustrated in FIG. 10, the image processor 14 receives a plurality of original images, that is, first, second, and third images from the plurality of cameras 111, 112, and 113.

Then, the image processor 14 corrects distortions included in the plurality of original images by using a lookup table stored in the memory unit 13, and spreads the original images on a plane by a predetermined projection method. A plurality of images are converted into wide-angle images to generate one wide-angle image.

Then, instead of outputting the generated wide-angle image as an output image, the image processor 14 divides the generated wide-angle image into a plurality of divided images and outputs the divided images as an output image to be displayed on the display unit 16. Can be.

That is, the image displayed on the display unit 16 in FIG. 10 may be a plurality of divided images in which the wide-angle image is divided into a predetermined size, rather than one wide-angle image as shown in FIGS. 8 and 9.

The plurality of divided images may be displayed in a line on the display unit 16 to cover an angle of view of 180 ° like the one wide-angle image.

According to an embodiment, the plurality of divided images may be generated by dividing the wide-angle image into equal sizes, but the present disclosure is not limited thereto, and the plurality of divided images may be configured such that their sizes are not equal.

According to an embodiment of the present invention, a projection process performed by using the lookup table may include a perspective projection that develops an image of a subject projected on a plane onto a plane, and an image of the subject projected on a cylindrical plane on a plane It may be one of a developing cylindrical projection and a spherical projection that develops an image of a subject projected on a spherical surface in a plane.

The image projected by the perspective projection method may express the perspective of the subject, but the range of the image covered in the horizontal and vertical directions may be limited.

In contrast, the image projected by the cylindrical projection method has a wider range of images covered in the horizontal direction than the perspective projection method, but the range of the images covered in the vertical direction is limited.

On the other hand, the image projected by the spherical projection method has a wide range of images covering the horizontal and vertical direction compared to the perspective projection method.

The lookup table used in the present invention is configured such that the image is projected onto the plane according to any one of the three projections described above.

According to an embodiment, the lookup table used in the present invention may be configured to project an image according to other projection methods in addition to the three projection methods described above.

11 to 16 are diagrams illustrating a board and its arrangement used in three projection schemes according to an embodiment of the present invention.

In general, in the case of a general camera that does not cover a wide angle of view, the lookup table is configured such that the subject of FIG. 11 is displayed in the image of FIG. 12 using a perspective projection method.

Specifically, the planar board of FIG. 11 has P011, P111, P018, P118 and P065 as each corner and center point. P011 of the flat board of FIG. 11 corresponds to (50,50) which is the X and Y coordinates of P011 of FIG. 12, and P111 of the flat board of FIG. 11 is (590,50) which is the X and Y coordinates of P111 of FIG. The lookup table may be configured to correspond to.

However, in the case of a camera having a wide horizontal angle of view and a large vertical angle of view, such as a panoramic camera, the lookup table is configured such that the subject of FIG. 13 becomes an image as shown in FIG. 12 using a cylindrical projection method.

Specifically, as shown in FIG. 14, the cylindrical board of FIG. 13 may be located in front of the image processing apparatus 1 having three cameras and two mirrors. In this case, the three cameras may capture a calibration image as shown in FIG. 15.

In order to output an image as shown in FIG. 12 from the images captured by the three cameras, a part of the right image, which is a part of the third image which is an image of the third camera, reflected on the mirror is extracted, and the first image which is an image of the first camera. The center portion except for the black portion, which is the rear side of the mirror, is extracted, and the left image portion, which is the portion reflected by the mirror, is extracted from the second image, which is the image of the second camera. Then, the lookup table is configured such that each point of the extracted image corresponds to a corresponding point of the image of FIG. 12.

For example, the point 'P011' on the cylindrical board is located at coordinates (x, y) = (582.7, 414.5) in the third image, which is an image of the third camera 113. The image value of this point must be exported at output as the coordinate (X, Y) = (50, 50) of the output image.

Therefore, the real point (n, x, y) of each camera image corresponding to all integer coordinates (X, Y) constituting the output image is stored in the lookup table. N is a component meaning any one of the first to third cameras.

In practice, the image processing apparatus 1 of the present invention obtains image values for (n, x, y) points in a lookup table in real time when images are input from three cameras. And store it in the output buffer (X, Y) point. Once the image values for all (X, Y) points are found, they are output in the next frame. Thus, the present invention enables distortion correction, projection, and image bonding at once. Through this calibration step, it is possible to output the image of the wide-angle camera in real time.

In addition, when both the horizontal angle of view and the vertical angle of view are wide-ranging cameras, the look-up table may be configured such that the subject of FIG. 16 becomes the image of FIG. 12 using a spherical projection method. 16A and 16B are front and side views, respectively, of a spherical board.

Again, the spherical board as shown in FIG. 16 is located in front of the camera unit, and after extracting a part of the image captured by each camera, the coordinates of the respective points constituting the extracted image are coordinates of the planar image of FIG. By corresponding to this, the lookup table can be configured.

17 is a flowchart of an image processing method according to an embodiment of the present invention.

As shown in FIG. 17, an image processing method according to an embodiment of the present invention receives images from a plurality of cameras to generate a wide-angle image covering a wide range of view angles, and captures a plurality of images captured by the plurality of cameras. Receiving step (S11), by using a look-up table stored in the memory, a correction process for correcting the distortion included in the plurality of received images, the projection to expand the image of the subject included in the plurality of received images on the plane Performing a process and a bonding process of joining the plurality of received images into the single wide-angle image in a batch to generate an output image (S12), and outputting the output image to be displayed on a predetermined display ( S15).

In operation S11, the receiver 12 receives a plurality of images from the plurality of cameras 111, 112, and 113. Each camera captures images corresponding to different ranges of view. The image processor 14 performs image processing to receive a captured image of each camera to generate a wide-angle image.

In operation S12, a correction process of correcting the distortion included in the plurality of received images by using the lookup table stored in the memory 13 by the image processor 14, and the process of subjects included in the plurality of received images. A projection process for developing an image onto a plane and a bonding process for joining the received plurality of images into the single wide angle image are collectively performed.

As a result, the image processor 14 generates a single wide-angle image from the plurality of received images, and the generated wide-angle image covers a wide range of field of view.

Step S13 determines whether the image processing apparatus 1 of the present invention outputs a single wide-angle image as an output image and displays it on the display.

As a result, when outputting a single wide-angle image as an output image and displaying it on the display, the output unit 15 outputs the generated wide-angle image in step S15 and processes it to be displayed on the display unit 16.

When not outputting a single wide-angle image as an output image in step S13, the image processor 14 divides the single wide-angle image into a plurality of images (S14).

Then, the output unit 15 outputs the plurality of divided images and processes them to be displayed on the display unit 16.

The divided plurality of divided images may be displayed in a line on the display unit to provide an image covering a wide field of view to a user.

The image processing method of the present invention described above can be created by a computer program. And the code and code segments constituting the program can be easily inferred by a computer programmer in the art.

In addition, the written program is stored in a computer-readable recording medium (information storage medium), and read and executed by a computer to implement the method of the present invention. The recording medium may include any type of computer readable recording medium.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

1: image processing device
11: camera
12: receiver
13: memory
14: image processing unit
15: output unit
16: display unit

Claims (15)

In the image processing apparatus for receiving images from a plurality of cameras to generate a wide-angle image,
A mirror which is a reflecting surface on which one surface receives and reflects light;
A plurality of cameras having a center of an angle of view disposed on both sides of the mirror with the mirror therebetween;
A receiver configured to receive a plurality of images captured by the plurality of cameras;
A memory unit for storing a lookup table for converting the plurality of images into a single wide-angle image;
A correction process of correcting distortion included in the plurality of received images, a projection process of spreading an image of a subject included in the plurality of received images on a plane, and the plurality of received images using the lookup table An image processor which processes an image to generate an output image to perform a bonding process of bonding the single wide-angle image; And
An output unit configured to output the output image to be displayed on a predetermined display,
And a first angle of view center disposed on the reflective surface side of the mirror among centers of angles of view of the plurality of cameras is disposed closer to the mirror than a second angle of view center disposed on the other surface side of the mirror. delete delete The method of claim 1,
The image processing unit may be a perspective process for developing an image of a subject projected on a plane as a projection process, a cylindrical projection for developing an image of a subject projected on a cylindrical plane, and a projection to a spherical surface. An image processing apparatus for performing any one of spherical projection to deploy an image of a subject on a plane. The method of claim 1,
And an output image displayed on the display is the converted single wide-angle image. The method of claim 1,
And an output image displayed on the display is a plurality of divided images divided from the converted single wide-angle image. The method of claim 1,
And the image processor generates the output image in accordance with a predetermined image standard. delete A mirror which is a reflecting surface reflecting light on one surface thereof, and a plurality of cameras having a center of an angle of view disposed on both sides of the mirror with the mirror therebetween, and a reflecting surface of the mirror among centers of the angles of view of the plurality of cameras An image processing apparatus arranged to be closer to the mirror than a second angle of view center disposed on the other side of the mirror to generate a wide-angle image by receiving images from the plurality of cameras. To
Receiving a plurality of images captured by the plurality of cameras;
A correction process for correcting distortion included in the plurality of received images using a lookup table stored in a memory, a projection process of spreading images of a subject included in the plurality of received images on a plane, and the received plurality of images Generating an output image by performing a bonding process of joining the images into a single wide-angle image; And
Outputting the output image to be displayed on a predetermined display;
Image processing method comprising a. delete delete The method of claim 9, wherein the generating step,
In the projection process, a perspective projection of developing an image of a subject projected on a plane onto a plane, a cylindrical projection of developing an image of a subject projected on a cylindrical plane on a plane, and a spherical projection of deploying an image of a subject projected on a sphere on a plane Image processing method comprising the step of performing any one. The method of claim 9, wherein the output step,
And outputting the single wide-angle image as an output image displayed on the display. The method of claim 9, wherein the output step,
Outputting a plurality of divided images divided from the single wide-angle image as an output image displayed on the display. A computer-readable recording medium having recorded thereon a program for realizing each step of the method according to any one of claims 9 and 12 to 14.

Images