KR20140118062A - Rectification method of multi-view image and apparatus using the same - Google Patents

Rectification method of multi-view image and apparatus using the same Download PDF

Info

Publication number
KR20140118062A
KR20140118062A KR1020130033296A KR20130033296A KR20140118062A KR 20140118062 A KR20140118062 A KR 20140118062A KR 1020130033296 A KR1020130033296 A KR 1020130033296A KR 20130033296 A KR20130033296 A KR 20130033296A KR 20140118062 A KR20140118062 A KR 20140118062A
Authority
KR
South Korea
Prior art keywords
image
camera
images
points
correcting
Prior art date
Application number
KR1020130033296A
Other languages
Korean (ko)
Inventor
김태정
김재인
Original Assignee
인텔렉추얼디스커버리 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 인텔렉추얼디스커버리 주식회사 filed Critical 인텔렉추얼디스커버리 주식회사
Priority to KR1020130033296A priority Critical patent/KR20140118062A/en
Publication of KR20140118062A publication Critical patent/KR20140118062A/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/128—Adjusting depth or disparity

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

There is provided a method for correcting a deviation of a multi-view image for converting a multi-view image sequence without correction of vertical parallax into a stereoscopic three-dimensional image.
A method for correcting a no reference point deviation of a multi-view image according to an embodiment of the present invention includes: acquiring at least two different view images of the same sequence for the object space; Extracting a control point around the recognized objects; estimating a geometry in a coordinate system of the three-dimensional model space for each camera of the multi-view camera system based on the control points; And transforming each of the images at the different viewpoints based on the estimated geometry.

Description

A method for correcting a non-reference point deviation of a multi-view image {Rectification method of multi-view image and apparatus using the same}

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a technique for generating three-dimensional (3D) stereoscopic images.

The growth stagnation of the 3D industry due to the shortage of 3D stereoscopic contents is causing the promotion of related technology development to reduce contents production cost. In 3D stereoscopic image production, the most time and effort required is the elimination of the vertical parallax caused by the shooting process. This is a very important task because it is directly related to the competitiveness of the market.

Vertical parallax correction of unaligned stereo images, or rectification, has long been a problem in photogrammetry, so if the computer vision method is focused on fast processing speed and automation, photogrammetry will provide accuracy and precision .

More specifically, the deviation correction is a task of removing vertical parallax between two or more images of the same scene. Through this process, the epipolar lines of all the images become parallel to each other, so that all the objects in the object space projected on the images have the same vertical coordinates in the respective images. This means that it is possible to stereoscopy between images.

This is because the correction of the deviation is important for 3D contents production because it can greatly help to minimize stereoscopic fatigue experienced by viewers by eliminating the vertical parallax between images as described above.

The deviation correction method can be divided into two methods depending on whether it is automated or not. First, the manual correction method has a merit that the image conversion is performed using the camera internal and external parameters already acquired in advance, and distortion of the conversion result is small and accuracy is high.

However, since the camera parameters are required in advance in such a method, there is a problem that the photographing is stopped every time the camera's attitude or position is changed, and the camera parameters are re-measured using a camera correction pattern such as a chessboard.

On the other hand, the automatic deviation correction method is more useful than the manual method because only the correspondence between images is required. However, this automatic correction method is very dependent on the position error of the corresponding point because the geometry is estimated only by the corresponding point and the image transformation is performed through the corresponding point. Also, unlike the manual method, It is very difficult for the reference and scale of the model space formed by each camera to coincide with each other.

The mismatch problem in the model space coordinate system has a great influence on the result of the correction of the deviation, and thus causes a problem that the vertical parallax between each transformed image may not be properly removed.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a deviation correction technique capable of coherent geometric estimation of a multi-view camera system in order to solve the above-mentioned problems of the prior art.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

According to another aspect of the present invention, there is provided a method of correcting a no reference point deviation in a multi-view image, comprising: obtaining at least two different viewpoint images of the same sequence in the target space; Extracting a control point to which three-dimensional model coordinate values are assigned from the images at the different viewpoints; extracting a control point to which the three-dimensional model coordinate values are assigned based on the control points; ; And transforming each of the images at the different viewpoints based on the estimated geometry.

Wherein the step of extracting the control points comprises the steps of: selecting an arbitrary reference viewpoint image and an adjacent viewpoint image adjacent to the reference viewpoint in the images at the different viewpoints; Extracting the candidate points corresponding to each of the candidate correspondences; and applying a three-dimensional model coordinate value using a relative orientation method using a collinear conditional expression as a model for each of the candidate corresponding points; And selecting a part of the corresponding points to which the coordinate value is assigned as the control point.

Here, the collinear conditional expression has the following expression.

Figure pat00001

here,

Figure pat00002
Is a point in the model space
Figure pat00003
Dimensional image coordinate when the projection image is projected onto the adjacent viewpoint image,
Figure pat00004
The model coordinate of the camera projection center,
Figure pat00005
Is the rotation matrix of the corresponding camera with respect to the model space coordinate system, f is the focal distance of the corresponding camera,
Figure pat00006
Means the rotation element for each axis of the model coordinate system.

The step of selecting a part of the corresponding points as the control point may include selecting only the corresponding points having the similar Z-axis coordinate values of the three-dimensional model coordinates among the corresponding points as the control points. According to another embodiment of the present invention, the method may further include storing the three-dimensional model coordinate for each of the extracted control points and a corresponding chip image together.

The step of estimating the geometry for each camera may include estimating the geometry of each of the cameras constituting the multi-viewpoint camera system on the basis of the same model space using a single orientation technique . ≪ / RTI >

Wherein the converting each of the images at the different viewpoints comprises: estimating a base line having a minimum distance from a projection center position of each camera on the three-dimensional model space; And moving the center of the image plane so that the image planes at the different viewpoints with respect to the base line are perpendicular to the photographing posture.

Meanwhile, the deviation correction method according to the above-described embodiment of the present invention can be stored in a computer-readable recording medium that is manufactured as a program for execution in a computer.

As described above, unlike the conventional method, since the deviation correction operation of the multi-viewpoint camera system can be automated, the entire operation process can be simplified, and even if the posture and the position of the camera are changed during the shooting It can be instantaneously re-estimated without any additional work, and it is possible to produce various cameras in the production of 3D contents.

Further, since the geometry of each camera can be estimated based on the same reference coordinate system on the model space without a previously obtained control point (or reference point), the present invention can greatly It is expected to contribute.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a flowchart illustrating a method for correcting a reference point deviation in a multi-view image according to an exemplary embodiment of the present invention. FIG.
2 is an exemplary diagram illustrating an example of a geometric relationship between a reference image and an adjacent image in the embodiment of the present invention.
FIG. 3 is an exemplary view showing an example of selecting control points for corresponding points extracted in the embodiment of the present invention. FIG.
4 is an exemplary diagram showing an example of converting each image plane in the embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

A method of correcting a reference point deviation of a multi-view image according to an embodiment of the present invention will be described with reference to FIGS. 1 to 4. FIG. The method of FIG. 1 is directed to a method for correcting a deviation of a multi-view image for generating a three-dimensional image in a multi-viewpoint camera system that captures images at different viewpoints with respect to a target space, , I.e., step S110 of acquiring the multi-view image. In step S110, images of at least two different viewpoints of the same sequence with respect to the target space are obtained.

In step S120, control points to which three-dimensional model coordinates are assigned are extracted from the images at the different viewpoints. Here, the control point is a reference point for estimating the geometry of each camera in the multi-view camera system.

In this method, candidate points corresponding to matching are detected from a plurality of multi-view images through feature point extraction and feature point matching in an obtained multi-view image, and a corresponding point among the candidate points is selected as a control point.

In one embodiment of the present invention, the step of extracting a control point includes the steps of: selecting an arbitrary reference viewpoint image and an adjacent viewpoint image adjacent to the reference viewpoint in the images at the different viewpoints; Extracting at least one candidate correspondence point corresponding to an image among the candidate correspondence points, and selecting a corresponding point as the control point through a relative orientation technique using a collinear condition expression as a model among the candidate correspondence points.

That is, an arbitrary reference viewpoint image to be matched is selected from the plurality of multi-viewpoint images obtained, and an arbitrary adjacent viewpoint image adjacent to the reference point is selected. The feature point extraction is performed on each of the selected reference viewpoint image and the neighbor viewpoint image, and feature points matching the reference viewpoint image and the neighbor viewpoint image are extracted as candidate correspondence points through the feature point matching process. It is obvious that the candidate correspondence point may include not only the feature points of the objects belonging to the background region but also the feature points of the objects belonging to the foreground region.

  As a control point, only some corresponding points having similar characteristics to each other are selected as the control point. In one embodiment, the control points include the corresponding points, Dimensional model coordinates based on the three-dimensional model coordinates; and selecting only some corresponding points having the similar Z-axis coordinate values among the candidate corresponding points based on the three-dimensional model coordinates as control points.

As an example, some correspondence points among the candidate correspondence points may be selected as control points through a relative orientation technique using a collinear conditional expression as a model. That is, the control points are selected based on objects having similar Z-coordinate values in the object space, and they are determined through a relative orientation technique based on a collinear condition from the corresponding points extracted between the arbitrarily selected reference image and the adjacent image .

The collinear condition-based relative facial expression is achieved through a nonlinear least squares adjustment method using a collinear conditional equation expressed by Equation (1) below as a model equation.

[Equation 1]

Figure pat00007

Referring to FIG. 2 showing an example of a geometric relationship between a reference image and an adjacent image in the embodiment of the present invention,

Figure pat00008
Is a point in the model space
Figure pat00009
Dimensional image coordinate when the projection image is projected onto the adjacent viewpoint image,
Figure pat00010
The model coordinate of the camera projection center,
Figure pat00011
Is the rotation matrix of the corresponding camera with respect to the model space coordinate system, f is the focal distance of the corresponding camera,
Figure pat00012
Means a rotation element for each axis of the model coordinate system.

As a result of the relative facial expression based on Equation (1), the relative attitude angle component between the reference viewpoint image and the adjacent viewpoint image

Figure pat00013
And a position element
Figure pat00014
And the model space coordinates of the objects corresponding to the candidate corresponding points input for the three-dimensional modeling
Figure pat00015
.

As described above, the three-dimensional model coordinates are assigned to the objects included in the object space or each of the candidate corresponding points, and based on the three-dimensional model coordinates, a feature in which the Z-axis coordinate values are easily similar to each other It is possible to select specific objects or corresponding points as control points. At this time, the three-dimensional model coordinates of the extracted control points and the chip images can be stored together in a database. Therefore, even if the posture and the position of the camera change during shooting, it is possible to estimate the geometry of each camera in the coordinate system of the same model space.

In step S130, the geometry in the coordinate system of the three-dimensional model space for each camera of the multi-viewpoint camera system is estimated based on the extracted control points.

For example, the geometry of each of the cameras constituting the multi-view camera system can be individually estimated based on the same model space using a single orientation technique. In this case, the model equation is different from the equation (1)

Figure pat00016
And a position element
Figure pat00017
Is considered to be unknown.

In step S140, a deviation correction is performed to remove a vertical parallax between images at different viewpoints based on the geometric structure estimated for each camera. In order to minimize deformation of the original image with respect to all of the images, as shown in FIG. 4, the position of the center of the projection of each camera on the three-dimensional model space Estimating a base line at which a distance is minimum, moving a projection center of each camera on the baseline, and calculating a vertical position of the image plane at the different viewpoint with respect to the base line, In order to achieve the desired result.

Meanwhile, the above-described deviation correction method according to the present invention can be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording media storing data that can be decoded by a computer system. For example, there may be a ROM (Read Only Memory), a RAM (Random Access Memory), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device and the like. The computer-readable recording medium may also be distributed and executed in a computer system connected to a computer network and stored and executed as a code that can be read in a distributed manner.

 It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description, and all changes or modifications derived from the scope of the claims and their equivalents should be construed as being included within the scope of the present invention.

Claims (8)

A method for correcting a deviation of a multi-view image for generating a three-dimensional image in a multi-view camera system for shooting images at different viewpoints with respect to a target space,
Obtaining at least two or more different viewpoint images of the same sequence for the object space;
Extracting control points to which three-dimensional model coordinate values are assigned from the images at different viewpoints;
Estimating a geometry in a coordinate system of the three-dimensional model space for each camera in the multi-view camera system based on the control points; And
Transforming each of the images at the different viewpoints based on the estimated geometry
The method comprising the steps of: 2. The method of claim 1, wherein extracting the control points comprises:
Selecting an arbitrary reference viewpoint image and an adjacent viewpoint image adjacent to the reference viewpoint from the images at the different viewpoints,
Extracting at least one candidate correspondence point corresponding to the reference viewpoint image and the neighbor viewpoint image;
Assigning a three-dimensional model coordinate value to each of the candidate corresponding points using a relative orientation method using a collinear conditional equation as a model; And
And selecting a part of the candidate correspondence points to which the coordinate value is assigned as the control point
A method for correcting a no reference point deviation of a multi - view image. 3. The method of claim 2,
Wherein the collinear condition equation is the following equation.
Figure pat00018

here,
Figure pat00019
Is a point in the model space
Figure pat00020
Dimensional image coordinate when one of the adjacent viewpoint images is projected onto one of the adjacent viewpoint images,
Figure pat00021
The model coordinate of the camera projection center,
Figure pat00022
Is the rotation matrix of the corresponding camera with respect to the model space coordinate system, f is the focal distance of the corresponding camera,
Figure pat00023
Means the rotation element for each axis of the model coordinate system. 3. The method as claimed in claim 2, wherein the step of selecting, as the control point,
And selecting only the corresponding points corresponding to the Z axis coordinate values of the three-dimensional model coordinate among the candidate corresponding points as the control point
A method for correcting a no reference point deviation of a multi - view image. The method according to claim 1,
Storing coordinates of the extracted control points in the three-dimensional model coordinate system and a chip image together
The method comprising the steps of: 2. The method of claim 1, wherein estimating the geometry for each camera comprises:
And separately estimating the geometry of each of the cameras constituting the multi-view camera system on the basis of the same model space using a single orientation technique
A method for correcting a no reference point deviation of a multi - view image. 2. The method of claim 1, wherein transforming the images at the different viewpoints comprises:
Estimating a base line having a minimum distance from a position of a projection center of each camera on the three-dimensional model space;
Moving the projection center of each camera on the baseline;
And allowing the image plane of the different viewpoints to be vertical with respect to the base line
A method for correcting a no reference point deviation of a multi - view image. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the deviation correction method according to any one of claims 1 to 7.
KR1020130033296A 2013-03-28 2013-03-28 Rectification method of multi-view image and apparatus using the same KR20140118062A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020130033296A KR20140118062A (en) 2013-03-28 2013-03-28 Rectification method of multi-view image and apparatus using the same

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020130033296A KR20140118062A (en) 2013-03-28 2013-03-28 Rectification method of multi-view image and apparatus using the same

Publications (1)

Publication Number Publication Date
KR20140118062A true KR20140118062A (en) 2014-10-08

Family

ID=51990973

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020130033296A KR20140118062A (en) 2013-03-28 2013-03-28 Rectification method of multi-view image and apparatus using the same

Country Status (1)

Country Link
KR (1) KR20140118062A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020101094A1 (en) * 2018-11-16 2020-05-22 포디리플레이코리아 주식회사 Method and apparatus for displaying stereoscopic strike zone

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020101094A1 (en) * 2018-11-16 2020-05-22 포디리플레이코리아 주식회사 Method and apparatus for displaying stereoscopic strike zone
KR20200057484A (en) * 2018-11-16 2020-05-26 포디리플레이코리아 주식회사 Method and apparatus for displaying a strike zone

Similar Documents

Publication Publication Date Title
JP6426968B2 (en) INFORMATION PROCESSING APPARATUS AND METHOD THEREOF
EP2236980B1 (en) A method for determining the relative position of a first and a second imaging device and devices therefore
CN107705333B (en) Space positioning method and device based on binocular camera
US20180158198A1 (en) Multi-view rotoscope contour propagation
WO2019160032A1 (en) Three-dimensional measuring system and three-dimensional measuring method
US20120177284A1 (en) Forming 3d models using multiple images
CN107588721A (en) The measuring method and system of a kind of more sizes of part based on binocular vision
CN110176032B (en) Three-dimensional reconstruction method and device
CN112116639B (en) Image registration method and device, electronic equipment and storage medium
WO2017183470A1 (en) Three-dimensional reconstruction method
JP2018534698A (en) Method and system for large scale determination of RGBD camera attitude
JP6436743B2 (en) Image information processing apparatus, image information processing method and program, and imaging apparatus
WO2021195939A1 (en) Calibrating method for external parameters of binocular photographing device, movable platform and system
US20120162387A1 (en) Imaging parameter acquisition apparatus, imaging parameter acquisition method and storage medium
JP6936974B2 (en) Position / orientation estimation device, position / orientation estimation method and program
Patel et al. Distance measurement system using binocular stereo vision approach
Gee et al. Tsai camera calibration enhanced
JP6285686B2 (en) Parallax image generation device
CN104346614A (en) Watermelon image processing and positioning method under real scene
Kurz et al. Bundle adjustment for stereoscopic 3d
KR20140118062A (en) Rectification method of multi-view image and apparatus using the same
KR101673144B1 (en) Stereoscopic image registration method based on a partial linear method
Nozick Camera array image rectification and calibration for stereoscopic and autostereoscopic displays
KR101923581B1 (en) Normal vector extraction apparatus and method thereof based on stereo vision for hull underwater inspection using underwater robot
KR20200057929A (en) Method for rectification of stereo images captured by calibrated cameras and computer program

Legal Events

Date Code Title Description
N231 Notification of change of applicant
WITN Withdrawal due to no request for examination