KR20130056605A - Apparatus and method for generating panorama image - Google Patents

Apparatus and method for generating panorama image Download PDF

Info

Publication number
KR20130056605A
KR20130056605A KR1020110122298A KR20110122298A KR20130056605A KR 20130056605 A KR20130056605 A KR 20130056605A KR 1020110122298 A KR1020110122298 A KR 1020110122298A KR 20110122298 A KR20110122298 A KR 20110122298A KR 20130056605 A KR20130056605 A KR 20130056605A
Authority
KR
South Korea
Prior art keywords
image
homography matrix
dual camera
camera
matrix value
Prior art date
Application number
KR1020110122298A
Other languages
Korean (ko)
Inventor
신현철
박경춘
홍승표
Original Assignee
한양대학교 에리카산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 한양대학교 에리카산학협력단 filed Critical 한양대학교 에리카산학협력단
Priority to KR1020110122298A priority Critical patent/KR20130056605A/en
Publication of KR20130056605A publication Critical patent/KR20130056605A/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2621—Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2622—Signal amplitude transition in the zone between image portions, e.g. soft edges
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2625—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2625—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect
    • H04N5/2627—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect for providing spin image effect, 3D stop motion effect or temporal freeze effect

Abstract

The present invention discloses an apparatus and method for generating a panoramic image. According to an embodiment of the present invention, a storage unit for storing a homography matrix value previously calculated according to the characteristics of the dual camera; An image acquisition unit configured to receive first and second images captured by the dual camera; And an image synthesizer configured to generate the panorama image by applying the stored homography matrix values to the first image and the second image.

Description

Apparatus and method for generating panoramic image {APPARATUS AND METHOD FOR GENERATING PANORAMA IMAGE}

The present invention relates to an apparatus and method for generating a panoramic image, and more particularly, to an apparatus and method for generating a panoramic image in real time using two images captured by a dual camera.

In order to overcome the limitations of a camera having a limited viewing angle, a panoramic image has been provided in recent years to provide a dual camera at a certain distance to extend the viewing angle to increase the realism.

Panoramic images are used in a wide variety of fields, for example in the case of surveillance cameras are used to secure the surveillance range of a wide field of view.

In addition, panoramic image reconstructs the surrounding environment in a vision-based intelligent vehicle safety system to compensate for the shortcomings of the single camera's field of view when recognizing vehicle extraction, vehicle recognition and tracking, and pedestrians. It is used for.

Furthermore, in the vehicle black box which is widely used in recent years, the panorama image is provided through the dual camera.

As described in Korean Patent Laid-Open Publication No. 2005-0109190 (name of the invention: a wide image generating apparatus and method using a dual camera), in order to generate a panoramic image using a dual camera, an area overlapped from an image captured by each camera is used. , Feature extraction in the overlapping region, matching, value calculation process for image correction, etc. are required, and in order to synthesize the images, one of the two images is corrected using the calculated values as described above. It is necessary to do

Recently, research is being conducted to provide panoramic images in real time not only for still images but also according to the related art. There is a problem that the sex is greatly reduced.

In order to solve the problems of the prior art as described above, an object of the present invention is to propose an apparatus and method for generating a panoramic image having real-time guarantee.

In order to achieve the above object, according to a preferred embodiment of the present invention, a panoramic image generating apparatus, comprising: a storage unit for storing a homography matrix value pre-calculated according to the characteristics of a dual camera; An image acquisition unit configured to receive first and second images captured by the dual camera; And an image synthesizer configured to generate the panorama image by applying the stored homography matrix values to the first image and the second image.

The dual camera has a fixed angle between the camera center axes, and the homography matrix value may be calculated in advance according to the angle between the fixed camera center axes.

The homography matrix value may be calculated by extracting, matching, and refining feature points of the first and second reference images captured by the dual camera.

At least one of a scale invariant feature transform (SIFT), a speed up robust feature (SURF) algorithm, and a maximum stable external regions (MSER) algorithm may be used to extract the feature points.

A random sample consensus (RANSAC) algorithm may be used for the feature point refinement.

The storage unit may store a homography matrix value calculated by an external device under the same condition as the dual camera.

According to another aspect of the present invention, a panorama image generating apparatus, comprising: an image obtaining unit receiving first and second images captured by a dual camera; Computing a homography matrix value for the synthesis of the first image and the second image by extracting, matching, and refining feature points of overlapping regions of the first image and the second image using a preset algorithm. A calculator; And a storage unit which stores the calculated homography matrix value, wherein the panorama generates a panoramic image by applying the stored homography matrix values to two images captured by the dual camera after the homography matrix value is stored. An image generating device is provided.

According to another aspect of the present invention, a method of generating a panoramic image, the method comprising: storing a homography matrix value calculated in advance according to characteristics of a dual camera; Receiving first and second images captured by the dual camera; And generating a panoramic image by applying the stored homography matrix values to the first image and the second image.

According to the present invention, since a homography matrix value for generating a panoramic image is calculated and stored in advance, there is an advantage of ensuring real time for generating the panoramic image.

1 is a diagram showing the configuration of a panoramic image generating apparatus according to an embodiment of the present invention.
2 is a view for explaining the angle between the central axis of the dual camera.
3 is a flowchart illustrating a process of calculating a homography matrix value.
4 is a view for explaining feature point purification.
5 is a diagram illustrating a configuration of a panoramic image generating apparatus according to another embodiment of the present invention.
6 is a view showing a panorama image generation result in accordance with the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, the same reference numerals will be used for the same means regardless of the reference numerals in order to facilitate the overall understanding.

1 is a diagram illustrating a configuration of an apparatus for generating a panoramic image according to an exemplary embodiment of the present invention.

As illustrated in FIG. 1, the panorama image generating apparatus according to the present embodiment may include an image acquisition unit 100, an image synthesis unit 102, and a storage unit 104.

The image acquisition unit 100 receives images captured from the first camera 110 and the second camera 112, and the image synthesis unit 102 calculates the homography matrix values pre-calculated and stored in the storage unit 104. Synthesize the captured image.

By such an image synthesis process, a panoramic image having a wide field of view is generated.

As shown in FIG. 1, the panoramic image generating apparatus according to the present embodiment may be applied to a device having two cameras, that is, a dual camera, and more preferably, a camera of the dual camera for surveillance or driving of a vehicle. The angle between the central axes can be applied to a fixed device.

For example, when dual cameras are applied to surveillance cameras, intelligent vehicle safety systems, or vehicle black boxes, the dual cameras are fixedly installed. Is maintained.

According to a preferred embodiment of the present invention, as shown in Figure 2, considering that the dual camera is maintained in a fixed state to the characteristics of the camera in the storage 104, that is, the angle between the central axis of the camera Calculate and store the homography matrix values accordingly.

The homography matrix value is a value for correcting an image captured by a dual camera as if viewed from the front, and a value that is multiplied by pixel coordinates of each image when synthesizing different images to generate a panorama image.

The calculation of the homography matrix value may be performed in the panorama image generating apparatus according to the present embodiment, but is not limited thereto and may be performed in an external device according to the characteristics of the dual camera.

In the following description, for convenience of description, the calculation of the homography matrix value is performed in an external device and then stored in the storage unit 104.

3 is a flowchart illustrating a process of calculating a homography matrix value.

Referring to FIG. 3, the external device receives a first reference image and a second reference image through a dual camera having the same characteristics as the panorama image generating apparatus (step 300).

Thereafter, overlap regions of the first and second reference images are obtained, and feature points of the overlap regions are extracted (step 302).

Extraction of feature points may be performed using at least one of a scale invariant feature transform (SIFT), a speed up robust feature (SURF) algorithm, and a maximum stable external regions (MSER) algorithm.

The feature points corresponding to each other in the first reference image and the second reference image are matched (step 304).

Next, the external device performs a feature point refinement process (step 306).

In order to calculate the homography matrix values, four pairs of feature points matched with each other are required, and the feature points that are hindered in calculating an accurate value should not be included.

Therefore, as shown in FIG. 4, the outliers that hinder the calculation of the homography matrix value are removed through the feature point refinement process.

Random sample consensus (RANSAC) algorithm may be used for feature point refinement.

After feature point purification, a homography matrix value is calculated using four pairs of feature points (step 308).

As described above, since the characteristics of the dual camera in the panoramic image generating apparatus according to the present embodiment are kept fixed, the pre-calculated homography matrix values are continuously used to generate the panoramic image by synthesizing the subsequent received images. It can be used as.

After the homography matrix value is stored, a panorama image generation process for the image received through the dual cameras 110 and 112 is performed.

The image acquisition unit 100 receives the first image and the second image captured by the dual cameras 110 and 112.

The image synthesizer 102 generates a panoramic image by applying the homography matrix values stored in the storage 104 to the first image and the second image.

In more detail, the image synthesizing unit 102 corrects each image by applying a homography matrix value to each image. In this case, the correction refers to a transformation such that the image captured from the left and right sides is viewed from the front.

Thereafter, the boundary line between the first and second images is removed to perform blending for mixing the pixels.

In the present embodiment, the blending algorithm may use a multi-blending algorithm.

According to the present embodiment, since the homography matrix value is calculated and stored in advance for a dual camera having a fixed characteristic, many processes may be omitted in the actual panorama image generation process, thereby generating a panorama image in real time. have.

Meanwhile, in the above description, the homography matrix value is calculated in advance in an external device and stored in the panorama image generating device.

According to another embodiment of the present invention, the calculation of the homography matrix value according to the characteristics of the dual camera may be performed in the panorama image generating apparatus.

5 is a diagram illustrating a configuration of a panoramic image generating apparatus according to another exemplary embodiment of the present invention.

As shown in FIG. 5, the panorama image generating apparatus according to the second embodiment of the present invention may include an image acquisition unit 500, a calculation unit 502, a storage unit 504, and an image synthesis unit 506. Can be.

Prior to generating the actual panoramic image, the panoramic image generating apparatus according to the second embodiment of the present invention calculates a homography matrix value to be used later using two images (first and second reference images) first received.

Since the process of calculating the homography matrix value has been described with reference to FIG. 3, a detailed description thereof will be omitted.

The homography matrix value calculated using the initial image is stored in the storage 504 and then applied to the image captured by the dual cameras 510 and 512.

Meanwhile, the panorama image generating apparatus according to the second embodiment may further include a monitoring unit 508 in consideration of changes in characteristics of the dual camera.

Preferably, the monitoring unit 508 may monitor whether the angle between the central axes of the first and second cameras 510 and 512 is changed.

When it is determined by the monitoring unit 508 that the angle between the central axes of the dual cameras is changed, the homography matrix value is updated.

A feature point extraction, matching, and refinement process is performed using the first and second reference images received by the dual cameras at a time point after the change of the angle between the central axes. Is updated.

The updated homography matrix value is stored in the storage unit 504 and subsequently applied to the received image to generate a panorama image according to the changed dual camera characteristic.

6 is a diagram illustrating a panorama image generated by using the panorama image generating apparatus according to the present embodiment. As shown in FIGS. 6A and 6B, a pre-stored homography matrix value is applied to each image photographed through a dual camera to generate a panoramic image as illustrated in FIG. 6C.

Preferred embodiments of the present invention described above are disclosed for purposes of illustration, and those skilled in the art will be able to make various modifications, changes, and additions within the spirit and scope of the present invention. Additions should be considered to be within the scope of the following claims.

Claims (10)

In the panoramic image generating device,
A storage unit for storing a homography matrix value calculated in advance according to the characteristics of the dual camera;
An image acquisition unit configured to receive first and second images captured by the dual camera; And
And an image synthesizer configured to generate the panorama image by applying the stored homography matrix values to the first image and the second image. The method of claim 1,
The dual camera has a fixed angle between the camera center axis, and the homography matrix value is calculated in advance according to the angle between the fixed camera center axis. The method of claim 2,
The homography matrix value is calculated by extracting, matching and refining feature points of the first and second reference images captured by the dual camera. The method of claim 3,
At least one of a scale invariant feature transform (SIFT), a speed up robust feature (SURF) algorithm, and a maximum stable external regions (MSER) algorithm is used to extract the feature points. The method of claim 3,
And a random sample consensus (RANSAC) algorithm for the feature point refinement. The method of claim 1,
Wherein,
And a homography matrix value calculated by an external device under the same condition as the dual camera. In the panoramic image generating device,
An image acquisition unit configured to receive first and second images captured by the dual camera;
Computing a homography matrix value for the synthesis of the first image and the second image by extracting, matching, and refining feature points of overlapping regions of the first image and the second image using a preset algorithm. A calculator; And
A storage unit for storing the calculated homography matrix value,
And generating a panoramic image by applying the stored homography matrix values to two images captured by the dual camera after the homography matrix values are stored. The method of claim 7, wherein
The dual camera is a panoramic image generating device that the angle between the camera center axis is fixed. The method of claim 7, wherein
Further comprising a monitoring unit for monitoring the angle between the central axis of the camera of the dual camera,
When the angle between the camera central axis is changed, the homograph matrix value is recalculated using two images received from the dual camera after the angle is changed, and the updated homography matrix value is stored in the storage unit. A panoramic image generating device for storing. In the panorama image generating method,
Storing a homography matrix value calculated in advance according to the characteristics of the dual cameras;
Receiving first and second images captured by the dual camera; And
Generating a panoramic image by applying the stored homography matrix values to the first image and the second image.
KR1020110122298A 2011-11-22 2011-11-22 Apparatus and method for generating panorama image KR20130056605A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020110122298A KR20130056605A (en) 2011-11-22 2011-11-22 Apparatus and method for generating panorama image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020110122298A KR20130056605A (en) 2011-11-22 2011-11-22 Apparatus and method for generating panorama image

Publications (1)

Publication Number Publication Date
KR20130056605A true KR20130056605A (en) 2013-05-30

Family

ID=48664593

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020110122298A KR20130056605A (en) 2011-11-22 2011-11-22 Apparatus and method for generating panorama image

Country Status (1)

Country Link
KR (1) KR20130056605A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101454803B1 (en) * 2012-12-20 2014-11-04 중앙대학교 산학협력단 Homography estimation apparatus and method
KR101482992B1 (en) * 2014-01-16 2015-01-19 한국산업기술대학교산학협력단 Black box with dual camera and method for tuning the camera location thereof
US9286656B2 (en) 2012-12-20 2016-03-15 Chung-Ang University Industry-Academy Cooperation Foundation Homography estimation apparatus and method
KR20160088090A (en) * 2015-01-15 2016-07-25 한국전자통신연구원 Apparatus and method for generating panorama image based on image quality
KR20160101762A (en) * 2015-02-17 2016-08-26 서울과학기술대학교 산학협력단 The method of auto stitching and panoramic image genertation using color histogram
KR20170114105A (en) * 2016-04-04 2017-10-13 엘지전자 주식회사 Apparatus for providing around view and Vehicle including the same
US11715178B2 (en) 2020-08-24 2023-08-01 Samsung Electronics Co., Ltd. Method and apparatus for generating image

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101454803B1 (en) * 2012-12-20 2014-11-04 중앙대학교 산학협력단 Homography estimation apparatus and method
US9286656B2 (en) 2012-12-20 2016-03-15 Chung-Ang University Industry-Academy Cooperation Foundation Homography estimation apparatus and method
KR101482992B1 (en) * 2014-01-16 2015-01-19 한국산업기술대학교산학협력단 Black box with dual camera and method for tuning the camera location thereof
KR20160088090A (en) * 2015-01-15 2016-07-25 한국전자통신연구원 Apparatus and method for generating panorama image based on image quality
KR20160101762A (en) * 2015-02-17 2016-08-26 서울과학기술대학교 산학협력단 The method of auto stitching and panoramic image genertation using color histogram
KR20170114105A (en) * 2016-04-04 2017-10-13 엘지전자 주식회사 Apparatus for providing around view and Vehicle including the same
US10366512B2 (en) 2016-04-04 2019-07-30 Lg Electronics Inc. Around view provision apparatus and vehicle including the same
US11715178B2 (en) 2020-08-24 2023-08-01 Samsung Electronics Co., Ltd. Method and apparatus for generating image

Similar Documents

Publication Publication Date Title
KR20130056605A (en) Apparatus and method for generating panorama image
CN106780550B (en) Target tracking method and electronic equipment
US9196022B2 (en) Image transformation and multi-view output systems and methods
KR101987634B1 (en) Method and Apparatus for Compensating Camera Tolerance and Around View Monitoring System Using the Same
EP3633598B1 (en) Image processing device, image processing method, and program
KR20130068243A (en) Image registration device and method thereof
CN109741241B (en) Fisheye image processing method, device, equipment and storage medium
EP3188468B1 (en) Image processing method and electronic device
JP2008102620A (en) Image processing device
US10346709B2 (en) Object detecting method and object detecting apparatus
CN110719411B (en) Panoramic all-around view image generation method of vehicle and related equipment
CN112543343A (en) Live broadcast picture processing method and device based on live broadcast with wheat and electronic equipment
JP2009288885A (en) Lane detection device, lane detection method and lane detection program
KR100948872B1 (en) Camera image correction method and apparatus
JP7081265B2 (en) Image processing equipment
CN107767326B (en) Method and device for processing object transformation in image and computing equipment
EP3480779A1 (en) Method and system for handling images
KR102029335B1 (en) Apparatus and method for correcting distored image
JP2012123750A (en) Vehicle image processor and vehicle image processing method
JP6931267B2 (en) A program, device and method for generating a display image obtained by transforming the original image based on the target image.
US20100283864A1 (en) Image processing system
US20210321049A1 (en) Image generation apparatus and image display system
US10288486B2 (en) Image processing device and method
CN115546312A (en) Method and device for correcting external parameters of camera
KR101709009B1 (en) System and method for compensating distortion of around view

Legal Events

Date Code Title Description
N231 Notification of change of applicant
A201 Request for examination
E902 Notification of reason for refusal