JP2954319B2 - Image bonding method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial applications]

The present invention relates to image bonding in which a plurality of images obtained by projecting a spherical surface on a plane from various angles, such as a fundus photograph or a sanitary photograph, are projected onto the spherical surface or onto the spherical surface and bonded in (on) the spherical surface. The present invention relates to a method and an apparatus.

[Prior art]

2. Description of the Related Art Conventionally, a spherical surface is photographed from various angles, and the photographs are laminated on a plane to form one photograph. A typical example of this is to create a photograph of the fundus from various angles or a photograph of the ground taken from a satellite, an airplane, or the like as a screen image composed of one image corresponding to each angle. Is being done. In this case, a simple geometric transformation such as an affine transformation is performed so that the corresponding points of the photographs coincide, and the photographs are pasted on a plane. In addition, in the case of pasting satellite photos, the inside of the sphere (top)
Is also performed, but also in this case, the projection is again projected onto the plane and bonded on the plane using a projection method that best reflects the region of interest. In this way, an image of a plane is created, and the distance, area, angle, and the like of each point are measured on the image. However, since the plane image includes geometric distortion, the measurement is performed. The value will be incorrect.

[Problems to be solved by the invention]

In general, it is impossible to expand a spherical surface into a plane,
In addition, when a spherical image is projected onto a planar image, geometrical distortion occurs in the projected image. Therefore, when two-dimensional images obtained by projecting spherical images from different angles are pasted together as they are, it is difficult to perform accurate alignment due to the geometric distortion. For this reason, geometric correction by affine transformation or the like is performed for such alignment, but it is still insufficient. When a large screen is created by projecting the entire sphere or a wide range of the sphere onto a plane, the further away from the center of the projection, the greater the geometric distortion, and the distance on the plane image,
Measurement errors such as the area and the angle increase. Viewing the entire sphere or a wide range of the sphere on a single large screen is unnatural. The present invention has been made in view of the above-described conventional example. By projecting an image projected on a plane into a spherical shape again and pasting it together, geometrical distortion does not occur and pastes a plane image with high accuracy. It is an object of the present invention to provide an image bonding method and apparatus that can be combined.

[Means for Solving the Problems]

In order to achieve the above object, the image laminating apparatus of the present invention has the following configuration. Acquisition means for acquiring a plurality of plane images obtained by projecting an image on a spherical surface onto a plurality of different planes; and a projection angle for deriving a projection angle when projecting the image on the spherical surface onto the plurality of different planes. Deriving means; coordinate converting means for converting the coordinates of the plurality of planar images acquired by the acquiring means into coordinates on the spherical surface based on the projection angle derived by the projection angle deriving means; A laminating unit for laminating images obtained by projecting the plurality of planar images on the spherical surface based on the coordinates converted by the unit, and an image on the spherical surface laminated by the laminating unit in an arbitrary plane. Display means for displaying the planar image projected above. In order to achieve the above object, the image laminating method of the present invention includes the following steps. That is, an image bonding method in an image bonding apparatus for developing and bonding a spherical image to a planar image, and acquiring means for acquiring a plurality of planar images obtained by projecting an image on a spherical surface onto a plurality of different planes, Projection angle deriving means for deriving a projection angle when projecting the image on the spherical surface onto the plurality of different planes, based on the projection angle derived in the projection angle deriving step,
A coordinate transformation step of transforming the coordinates of the plurality of plane images acquired in the acquisition step into coordinates on the spherical surface, and transforming the plurality of plane images on the spherical surface based on the coordinates transformed in the coordinate transformation step. Laminating step of laminating the images projected on each other, and a display step of displaying a planar image obtained by projecting the image on the spherical surface laminated by the laminating step on an arbitrary plane. I do.

[Action]

In the above configuration, a plurality of planar images obtained by projecting an image on a spherical surface on a plurality of different planes are obtained, and a projection angle when the image on the spherical surface is projected on a plurality of different planes is derived. Based on the projection angle, the coordinates of the plurality of planar images are converted into coordinates on the spherical surface, and based on the converted coordinates, the images obtained by projecting the plurality of planar images on the spherical surface are pasted together. A planar image is displayed by projecting the image on the spherical surface thus bonded onto an arbitrary plane.

【Example】

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present embodiment, the case of a fundus image obtained by photographing the fundus will be described. However, it goes without saying that the present invention is not limited to this. <Explanation of Principle (FIG. 1)> FIG. 1 is a diagram showing projections of the fundus oculi, which best show the features of the present embodiment. In FIG. 1, reference numeral 101 denotes an eyeball, and reference numeral 102 denotes a pupil serving as a projection port when the inside of the eyeball (fundus) is projected. Ten
Each of 3,104 is the projection center of the image 107,108, and means the focus at the time of shooting. Numerals 105 and 106 denote images within the spherical surface of the eye, and 107 and 108 denote images obtained by projecting the images 105 and 106 on a plane. <Description of Spherical Image Bonding Display Apparatus> FIG. 2 is a block diagram showing a schematic configuration of the spherical image bonding display apparatus of the embodiment, and FIG. 3 is an implementation procedure in the spherical image bonding display apparatus of the embodiment. FIG. In FIG. 2, reference numeral 201 denotes a control unit (C for controlling the entire apparatus)
PU), a ROM 201a for storing a control program and various data of the CPU shown in the flowchart of FIG. 3, a RAM 201b used as a work area of the CPU 201 and temporarily storing various data, and the like. Reference numeral 202 denotes a three-dimensional input unit for inputting parameters for instructing a visual line direction, a visual field, and a projection method of an image. Reference numeral 203 denotes a high-speed operation unit, which can perform calculations for coordinate conversion described later at high speed. Reference numeral 204 denotes an image memory which stores image data input from the graphic input unit 208, and stores image data converted according to the parameters input from the three-dimensional input unit 202, and displays the image data on the CRT display 205. . 206 is a keyboard for inputting various commands and operation commands. Reference numeral 208 denotes a graphic input unit for inputting image data, 209 denotes an external storage device such as a large-capacity hard disk for storing image data and the like, and 210 denotes a printing device such as a printer or a plotter. Normally, as shown in FIG.
Are projected in the form of images 107 and 108 projected on a plane. Each of the planar images 107 and 108 is input from the graphic input unit 208. Then, the image data is stored in the image memory 204 as necessary or in an external storage device 209 such as a hard disk. <Description of Operation (FIGS. 3 to 7)> Hereinafter, the operation of the spherical image stitching display device will be described with reference to the flowchart of FIG. 3 and FIGS. 4 to 7. In the flowchart of FIG. 3, first, in step S1
The plane image (x, y) on the x, y coordinate plane is input, and the angles θ, の (FIG. 4) at the time of photographing are obtained in step S2. Hereinafter, how to determine the angles θ and Ψ will be described. FIG. 5 is a diagram for explaining how to obtain these angles θ and Ψ (actually, it is a three-dimensional space.
(Represented by a dimensional plane). That is, the angle 506 to be obtained is
When FIG. 5 shows the eyeball viewed from the horizontal direction, it is represented by Ψ,
When FIG. 5 is a view of the eyeball viewed from directly above, it is represented by θ. In FIG. 5, reference numeral 501 denotes a reference image plane serving as a reference point of the projected plane image, reference numeral 502 denotes a corresponding point on the reference image plane, and the corresponding point 502 is a point at which a point 508 on the circumference of the eyeball is projected. It is. Reference numeral 503 denotes an image plane in the eyeball to be processed, and 504 denotes an image plane 503 corresponding to the corresponding point 502 of the reference image 501.
The corresponding points above are shown. That is, the point 5 on the circumference of the eyeball 101
08 is a point where the corresponding point 504 of the processing target image 503 is projected. 505 is a reference point serving as a reference (θ = Ψ = 0), 506
Indicates an angle (θ or Ψ) to be obtained, and 507 indicates a corresponding point when the number of corresponding points of the image in the eyeball is plural. First, a corresponding point 504 is designated between an image (reference image) 501 in which a reference point 505 (a point where Ψ = θ = 0 ... the fovea in the fundus image) is captured and an image 503 to be processed. The angles Ψ and θ of the processing target image are obtained by performing geometric transformation from the correspondence. That is, in FIG. 5, the angle 506 is equal to the angle γ between the reference image 501 and the processing target image 503. When there is no point corresponding to the reference image 501 in the target image 503, the angles Ψ and θ are obtained by continuously designating the corresponding points and connecting a plurality of images. FIG. 6 is a view of the spherical surface 401 of FIG. 4 viewed from the horizontal direction, and FIG. 7 is a view of the spherical surface 401 viewed from above.
A method of obtaining an expression for coordinate conversion described later will be described. In FIG. 6, P'B = (y '+ 2R sinΨ' + MtanΨ ') cosΨ'PA = Rcosα sinβ + MtanΨ'cosΨ'EB = McosΨ' + 2R- (y '+ 2RsinΨ') sinΨ'EA = McosΨ '+ R + R cosα cosβ Here, Ψ ′ = tan ⁻¹ (sinΨ / cosΨcosθ) Also, since (P′B / PA) = (EB / EA), y ′ = R · (2R cosΨ ′ + M) ｛cosα sinβ cos
{'-Sin {' (1 + cosα cosβ)} / {M + R (cos
{'(1 + cosα cosβ) + cosα sinβ sinΨ ′)｝ y = y′cosθ cosΨ In FIG. 7, P′B = (x ′ + 2R sinθ + Mtanθ) cosθ PA = Rsinα cosβ + M tanθcosθ EB = Mcosθ + 2R− (x ′ + sin ) Sin θ EA = M cos θ + R + R cos α cos β Here, similarly to the above, since (P′B / PA) = (EB / EA), x ′ = R · (2R cos θ + M) ｛sin α cos β cos θ−s
inθ (1 + cosα cosβ)｝ / ｛M + R (cosθ (1 + cos
sα cos β) + sin α cos β sin θ)｝ x = x'cosΨ Next, the process proceeds to step S3, where all the pixels P '(x, y) of the plane image are geometrically transformed into a spherical coordinate system as shown in FIG. To the pixel P (α, β) within FIG. 4 shows this relationship, wherein 401 indicates a spherical coordinate system, 402 indicates longitude coordinates (indicated by an angle α), and 403 indicates latitude coordinates (indicated by an angle β). Reference numeral 404 denotes a pixel P ′ (α, β) obtained by projecting the pixel 408P ′ (x, y) on the plane coordinate system 405 onto the spherical coordinate system 401. Reference numeral 406 denotes an x coordinate axis on the plane coordinate system 405, and 407 denotes a y coordinate axis on the plane 405. Reference numeral 409 denotes a focus position. The coordinates P (α, β) on the spherical surface 401 and the plane image P ′ (x,
The relationship with y) is based on the mathematical formula described with reference to FIGS. 6 and 7 described above. Next, the process proceeds to step S4, where θ and Ψ are corrected by performing positioning within the spherical surface. This alignment is a measure of the misalignment of the overlapping portion on the sphere (eg, a function of the correlation coefficient).
Is minimized, and θ and Ψ are corrected by this position correction. This will be described with reference to FIG. The angle you want now 50
If 6 is α and the lengths m and n are determined as shown in the figure, mn = n cos ² α (m−2R sin α) + ｛(2R) ² sin ² α cos α
−m · 2Rsinα cosα), and obtain the angle α from m and n. This angle α is the fifth
When the figure is the top view of FIG. 4, it is θ, and the lengths m and n are represented by x coordinates. On the other hand, when FIG. 5 is a side view of FIG. 4, the angle α = Ψ, and the lengths m and n are represented by y coordinates. Next, proceeding to step S5, the angles Ψ and 求 め obtained in step S4 are obtained.
With θ, the pixels of the plane image are mapped into the spherical surface (up). Next, the process proceeds to step S6, and steps S1 to S1 are performed for all the pixels.
By repeating S4, a spherical image is completed. When a spherical image is completed in step S7, the process proceeds to step S8, where parameters for display such as a line-of-sight direction, a visual field, and a projection method are input to the three-dimensional input unit 202 or keyboard 2
Enter from 02. Then, in step S9, the image is projected on a plane using the parameters input in step S8, and a plane image is created. Then, in step S10, the plane image is displayed on the video display monitor 205. Here, steps S8 to S
By repeating step 10, a display can be made as if the eyeball was actually moved and observed as shown in FIG. In FIG. 8A, reference numeral 601 denotes an eyeball, and 602 denotes the eyeball.
This shows the viewpoint for viewing 601. Reference numeral 603 indicates an image formed on the spherical surface of the eyeball, and the arrow indicated by 604 indicates the direction in which the eyeball rotates. In FIG. 8 (B), 605 indicates the earth, and 606 indicates the viewpoint from which the earth 605 is viewed. 607 indicates an image on the spherical surface of the earth 605, and 607 and 608 indicate the rotation direction of the earth surface 605. In addition, parameters for measuring the distance, area, and angle to be measured can be specified while looking at the display, and the measurement can be calculated using the spherical image. <Other Embodiments> (FIGS. 9 and 10) Next, a case where the present embodiment is applied to an image photographed by a satellite (aviation) will be described. In FIG. 9, 701 indicates the earth, 702 indicates the projection center of the image 704, and 703 indicates the projection center of the image 705. Reference numeral 704 denotes an image on a spherical surface, and an image obtained by projecting the image on a plane is indicated by reference numeral 706. Reference numeral 705 denotes an image on a spherical surface, and an image obtained by projecting the image on a plane is indicated by 707. Here, the difference from the above-described embodiment using the fundus image is that the geometric transformation of the projection in steps S3, S4 and S6 in FIG. 3 is changed from the transformation shown in FIG. 4 to the transformation shown in FIG. . In FIG. 10, reference numeral 801 denotes a spherical coordinate system, 802 denotes longitude coordinates (α) on a spherical surface, and 803 denotes latitude coordinates (β) on a spherical surface. 805 indicates a plane coordinate system, 806 indicates its x-coordinate axis 807, and its y-coordinate axis. Reference numeral 808 denotes a pixel of the image projected on the plane, and reference numeral 809 denotes its focus. Further, in the previous embodiment, Ψ and θ of each image were obtained from the corresponding points with the reference image in FIG. 5, but in this case, 投影 and θ are obtained at the time of projection. (Even if it cannot be obtained, in the case of a satellite (aviation) image, there are many reference points (points in advance of Ψ and θ) existing in the image.) The above geometric transformation and the calculation of 求 め and θ If the method is changed, it can be realized by the same procedure as in the previous embodiment. Thus, as in the previous embodiment, a display can be provided as if the earth was actually moved and observed as shown in FIG. 8 (B). As described above, according to the present embodiment, it is possible to accurately bond spherical images without geometric distortion. By measuring the distance, the area, and the like within the spherical surface (upper), highly accurate measurement is possible. It can be observed in a natural form as if actually moving a spherical surface. By selecting a projection, an image suitable for the purpose can be obtained.

【The invention's effect】

As described above, according to the present invention, an image projected on a plane is projected onto a spherical surface again and bonded to a spherical shape, so that a planar image can be accurately pasted without generating geometric distortion. There is an effect that can be combined.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the concept of combining fundus images, FIG. 2 is a block diagram schematically showing an image combining device according to the embodiment, and FIG. 3 is a processing procedure in the image combining device according to the embodiment. FIG. 4 is a diagram illustrating a model for converting pixel coordinates in a spherical surface into coordinates on a plane, FIGS. 5 to 7 are diagrams illustrating a method of obtaining angles Ψ and θ, 8 (A) and 8 (B) are diagrams for explaining the concept when the observation position is changed, FIG. 9 is a diagram showing the concept of bonding satellite (aviation) images, and FIG. 10 is a spherical surface. FIG. 3 is a diagram illustrating a geometric model of conversion from a coordinate system to a plane coordinate system. In the figure, 201 ... CPU, 201a ... ROM, 201b ... RAM, 202 ...
.., A three-dimensional input unit, 203, a high-speed operation unit, 204, an image memory, 205, a video display monitor, 206, a keyboard.

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-1-302474 (JP, A) JP-A-62-23663 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G06T 1/00

Claims (4)

(57) [Claims] 1. An acquisition means for acquiring a plurality of planar images obtained by projecting an image on a spherical surface onto a plurality of different planes, and deriving a projection angle when projecting the image on the spherical surface onto the plurality of different planes. Projection angle deriving means, based on the projection angle derived by the projection angle deriving means, coordinate conversion means for converting the coordinates of the plurality of planar images acquired by the acquiring means into coordinates on the spherical surface, Based on the coordinates after the transformation by the coordinate transformation means, a laminating means for laminating images obtained by projecting the plurality of planar images on the sphere, and an image on the sphere laminated by the laminating means. Display means for displaying a planar image projected on an arbitrary plane; and an image pasting device. 2. The image pasting apparatus according to claim 1, wherein the pasting means performs positioning of the plurality of images when pasting the plurality of images. 3. An image stitching method in an image stitching apparatus for developing a spherical image into a two-dimensional image and stitching the two-dimensional image, and acquiring a plurality of two-dimensional images obtained by projecting an image on a spherical surface onto a plurality of different planes. Means, a projection angle deriving means for deriving a projection angle when projecting the image on the spherical surface onto the plurality of different planes, and, in the obtaining step, based on the projection angle derived in the projection angle deriving step. A coordinate conversion step of converting the coordinates of the obtained plurality of planar images into coordinates on the spherical surface, and an image obtained by projecting the plurality of planar images on the spherical surface based on the coordinates converted in the coordinate conversion step. And a display step of displaying a planar image obtained by projecting an image on the spherical surface, which is bonded in the bonding step, onto an arbitrary plane, and an image pasting step. Matching method. 4. The image pasting method according to claim 3, wherein in the pasting step, when pasting a plurality of images, the plurality of images are aligned.