JP2014021672A - Three-dimensional coordinate calculation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional coordinate calculation device for calculating a three-dimensional coordinate commonly usable in images included in a panoramic image.SOLUTION: A three-dimensional coordinate calculation device 30 comprises: a designated point acquisition unit 36 that acquires a designated point in a second photographed image and also acquires a designated point in a third photographed image, in response to an operation of an input device 20 by a user; a first three-dimensional coordinate calculation unit 37 that calculates a first three-dimensional coordinate in a first three-dimensional coordinate system based on each photographing direction of a second single-lens photographing part 11b and a third single-lens photographing part 11c, on the basis of the acquired designated points in the second and third photographed images; and a second three-dimensional coordinate calculation unit 38 that calculates a second three-dimensional coordinate of the designated points in a second three-dimensional coordinate system common in all photographing directions, on the basis of the calculated first three-dimensional coordinate and each photographing direction of the second single-lens photographing part 11b and the third single-lens photographing part 11c at the time of photographing.

Description

  The present invention relates to a three-dimensional coordinate calculation apparatus that calculates three-dimensional coordinates that can be used in common in all images included in a panoramic image.

  As a technique related to a panoramic image, a technique for generating a panoramic image by combining a plurality of images so as to be connected is known (see Patent Document 1).

JP 2010-050795 A

  Such a technique can display the inside of a building, but when calculating a three-dimensional distance between two original images of a panoramic image, the three-dimensional coordinates unique to each original image are calculated from the three-dimensional coordinates. The distance cannot be calculated, and a three-dimensional coordinate applicable to the entire panoramic image is desired.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to provide a three-dimensional coordinate calculation apparatus that calculates three-dimensional coordinates that can be commonly used in each image included in a panoramic image.

  In order to solve the above-described problem, a three-dimensional coordinate calculation apparatus according to the present invention is photographed by a plurality of first image data photographed by a first photographing device and a second photographing device that is separated from the first photographing device by a predetermined distance. A plurality of second image data, and a plurality of third image data photographed by a third photographing device that is separated from the first photographing device by a predetermined distance, and an image data storage unit that is stored in association with each photographing direction; A panoramic image data generating unit that reads the plurality of stored first image data to generate panoramic image data, an image data output unit that outputs the generated panoramic image data to a display device, and an input device by a user An image data designation acquisition unit for acquiring designation of one of the first image data in the panoramic image data in response to the operation of The output unit outputs the second image data and the third image data associated with the specified first image data to the display device, and according to an operation of the input device by a user, Acquiring a designated point in the second image data, a designated point obtaining unit for obtaining the designated point in the third image data, the designated point in the obtained second image data, and the third image data Based on the designated point, the first three-dimensional coordinates in the first three-dimensional coordinate system unique to each photographing direction with reference to the photographing direction of the first photographing device, the second photographing device, and the third photographing device. Based on the first three-dimensional coordinate calculation unit to be calculated, the calculated first three-dimensional coordinates, and the shooting directions of the second shooting device and the third shooting device at the time of shooting, Characterized in that and a second three-dimensional coordinate calculation unit for calculating a second set of three-dimensional coordinates of the specified point in common the second three-dimensional coordinate system in the shading direction.

  According to this configuration, it is possible to calculate three-dimensional coordinates that can be used in common in each image included in the panoramic image.

によって、前記指定点の前記第二の三次元座標を算出する構成であってもよい。 The three-dimensional imaging coordinates are the Z axis that is the optical axis direction of the first imaging device, the second imaging device, and the third imaging device, the X axis that is orthogonal to the Z axis, the Z axis, and the X axis. When expressed by a right-handed coordinate system orthogonal to the axis, the focal length of the second imaging device and the third imaging device is f, and the first three-dimensional coordinate of the second imaging device is ( -D / 2,0,0), the first three-dimensional coordinates of the third imaging device (d / 2,0,0), and the first three-dimensional coordinates of the designated point in the second image data. (X ₂ , y ₂ , −f), the first three-dimensional coordinates of the designated point in the third image data are (x ₃ , y ₃ , −f), and the first three-dimensional coordinates of the designated point Is (x _R , y _R , z _R ), the first three-dimensional coordinate calculation unit is
_{x R = d (x 2 +} x 3) / {2 (x 2 -x 3 + d)}
_{y R = d (y 2 +} y 3) / {2 (y 2 -y 3 + d)}
z _R = −df / {2 (x ₂ −x ₃ + d)}
To calculate the first three-dimensional coordinates of the designated point and rotate about the X axis from the reference of the photographing direction of the second photographing device and the third photographing device in the second three-dimensional coordinate system When the angle is φ, the rotation angle around the Y axis is θ, and the rotation angle around the Z axis is ψ, the second three-dimensional coordinate calculation unit is

The second three-dimensional coordinates of the designated point may be calculated by

  In order to solve the above-described problem, a three-dimensional coordinate calculation apparatus according to the present invention is photographed by a plurality of first image data photographed by a first photographing device and a second photographing device that is separated from the first photographing device by a predetermined distance. A plurality of second image data stored in association with each shooting direction; a panorama image data generation unit that reads the plurality of stored first image data and generates panorama image data; An image data output unit that outputs the generated panorama image data to a display device, and an image data specification that acquires the designation of the first image data in the panorama image data in accordance with an operation of the input device by a user An acquisition unit, and the image data output unit includes the designated first image data and the second image data associated with the first image data. A designated point acquisition unit that outputs image data to the display device, acquires a designated point in the first image data, and obtains the designated point in the second image data in accordance with an operation of the input device by a user; , Based on the designated point in the acquired first image data and the designated point in the second image data, unique to each photographing direction with reference to the photographing direction of the first photographing device and the second photographing device. A first three-dimensional coordinate calculation unit for calculating a first three-dimensional coordinate in the first three-dimensional coordinate system; the calculated first three-dimensional coordinate; the first photographing device at the time of photographing; A second three-dimensional coordinate calculation unit that calculates a second three-dimensional coordinate of the designated point in the second three-dimensional coordinate system common to all the photographing directions based on the photographing direction of the two photographing devices; Characterized in that it comprises a.

  According to this configuration, it is possible to calculate three-dimensional coordinates that can be used in common in all images included in the panoramic image.

によって、前記指定点の前記第二の三次元座標を算出する構成であってもよい。 The three-dimensional imaging coordinates are the Z axis that is the optical axis direction of the first imaging device and the second imaging device, the X axis that is orthogonal to the Z axis, and the Y axis that is orthogonal to the Z axis and the X axis. And the first three-dimensional coordinates of the first photographing device are (0, 0, 0), and f is the focal length of the first photographing device and the second photographing device. The first three-dimensional coordinates of the second imaging device are (d, 0, 0), the first three-dimensional coordinates of the designated point in the first image data are (x ₁ , y ₁ , -f), The first three-dimensional coordinates of the designated point in the second image data are (x ₂ , y ₂ , −f), and the first three-dimensional coordinates of the designated point are (x _R , y _R , z _R ). ,
When the first three-dimensional coordinate calculation unit,
_{x R = dx 1 / (x} 1 -x 2)
_{y R = dy 1 / (x} 1 -x 2)
z _R = −df / (x ₁ −x ₂ )
To calculate the first three-dimensional coordinates of the designated point and rotate about the X axis from the reference of the photographing direction of the second photographing device and the third photographing device in the second three-dimensional coordinate system When the angle is φ, the rotation angle around the Y axis is θ, and the rotation angle around the Z axis is ψ, the second three-dimensional coordinate calculation unit is

The second three-dimensional coordinates of the designated point may be calculated by

  According to the present invention, it is possible to calculate three-dimensional coordinates that can be commonly used in all images included in a panoramic image.

1 is a block diagram showing a panoramic image display system according to a first embodiment of the present invention. It is a schematic diagram for demonstrating arrangement | positioning of a board model. (A) is a figure for demonstrating the calculation principle of a 1st three-dimensional coordinate, (b) is an xz top view of (a). (A) is a figure for demonstrating the rotation around an X-axis, (b) is a figure for demonstrating the rotation around a Y-axis. (A) and (b) are figures which show the example of a screen in three-dimensional coordinate calculation and three-dimensional distance calculation. (A) and (b) are figures which show the example of a screen in three-dimensional coordinate calculation and three-dimensional distance calculation. It is a block diagram which shows the panoramic image display system which concerns on 2nd embodiment of this invention. (A) is a figure for demonstrating the calculation principle of a 1st three-dimensional coordinate, (b) is an xz top view of (a). (A) and (b) are figures which show the example of a screen in three-dimensional coordinate calculation and three-dimensional distance calculation. (A) and (b) are figures which show the example of a screen in three-dimensional coordinate calculation and three-dimensional distance calculation.

  Hereinafter, a case where the three-dimensional coordinate calculation apparatus of the present invention is applied to a panoramic image display inside a building will be described as an example with reference to the drawings as appropriate. Similar parts are denoted by the same reference numerals, and redundant description is omitted.

<First embodiment>
As shown in FIG. 1, a panoramic image display system 1A according to the first embodiment of the present invention is a system that generates and displays an omnidirectional panoramic image inside a building, for example, and includes a compound eye photographing device 10A, An input device 20, a three-dimensional coordinate calculation device 30, and a display device 40 are provided.

<Composite imaging device 10A>
The compound-eye imaging device 10A includes a first monocular imaging unit (first imaging device) 11a, a second monocular imaging unit (second imaging device) 11b, a third monocular imaging unit (third imaging device) 11c, and a monocular The drive part 12 for changing the imaging | photography direction of imaging | photography part 11a, 11b, 11c, and the control part 13 which drive-controls the monocular imaging | photography part 11a, 11b, 11c and the drive part 12 are provided.
The first monocular imaging unit 11a, the second monocular imaging unit 11b, and the third monocular imaging unit 11c are digital cameras, and the focal lengths of the second monocular imaging unit 11b and the third monocular imaging unit 11c are the same. f, the second monocular imaging unit from the left so that the optical axes of the imaging units 11a, 11b, and 11c are parallel to each other and the distance between the optical axes of the adjacent imaging units 11a to 11c is the distance d / 2. 11b, the first monocular imaging unit 11a, and the third monocular imaging unit 11c are arranged in this order, and are unitized while maintaining this arrangement. The focal length of the first monocular imaging unit 11a may be different from or the same as the focal length of the other monocular imaging units 11b and 11c.

The control unit 13 rotates the driving unit 12 to direct the monocular imaging units 11a, 11b, and 11c in a desired imaging direction, and controls the shutters of the monocular imaging units 11a, 11b, and 11c in this state to perform imaging. The captured image data relating to the captured image is output to the three-dimensional coordinate calculation device 30. In the present embodiment, the control unit 13 causes the monocular imaging units 11a, 11b, and 11c to perform imaging by rotating the compound eye imaging device 10 by a predetermined angle with the first monocular imaging unit 11a as the rotation center.
Further, the control unit 13 assigns an ID based on the shooting order (that is, the shooting direction) to the shot image data, and outputs the shot image data to which the ID is given to the three-dimensional coordinate calculation apparatus 30. Here, the 1st picked-up image data regarding the 1st picked-up image image | photographed by the 1st monocular imaging | photography part 11a, the 2nd picked-up image data regarding the 2nd picked-up image image | photographed by the 2nd monocular imaging | photography part 11b, 3rd Among the third photographed image data related to the third photographed image photographed by the monocular photographing unit 11c, those having the same photographing direction are assigned the same ID and associated with each other. Further, the second photographed image data and the third photographed image data correspond to the coordinate values in the X-axis and Y-axis directions among the coordinates of the first three-dimensional coordinate system described later in each photographed image.

<Input device 20>
The input device 20 includes a keyboard, a mouse, and the like. In response to a user operation, an input of image data designation instructions for a monocular panoramic image, designation point designations for the second and third photographed images, and the like to the three-dimensional coordinate calculation apparatus 30. Output.

<Three-dimensional coordinate calculation apparatus 30>
The three-dimensional coordinate calculation device 30 includes a CPU, a ROM, a RAM, an input / output circuit, and the like. As a functional unit, a captured image data storage unit 31, a map storage unit 32, a monocular panoramic image data generation unit 33, and the like. , Image data output unit 34, image data designation acquisition unit 35, designated point acquisition unit 36, first three-dimensional coordinate calculation unit 37, second three-dimensional coordinate calculation unit 38, and three-dimensional distance calculation unit 39.

<< Photographed Image Data Storage Unit 31 >>
The captured image data storage unit 31 receives and stores a plurality of captured image data related to a plurality of captured images output from the control unit 13.

≪Map storage unit 32≫
The map storage unit 32 stores in advance a monocular panorama map used when generating a monocular panoramic image based on a plurality of first captured images (first captured image data) of the first monocular imaging unit 11a. .

≪Monocular panoramic image data generation unit 33≫
The monocular panoramic image data generation unit 33 reads a plurality of first captured image data output from the captured image data storage unit 31, and based on the read first captured image data, monocular panoramic image data related to the monocular panoramic image. Is generated.
Specifically, the monocular panoramic image data generation unit 33 reads a monocular panoramic map 50 (see FIG. 2) stored in advance in the map storage unit 32, and the first captured image captured by the first monocular imaging unit 11a. Monocular panorama image data is generated by mapping the data to the plate model 51 of the monocular panorama map 50.

  Here, the panorama map 50 will be described with reference to FIG. As shown in FIG. 2, the panoramic map 50 is configured by arranging a plurality of plate models 51 in a spherical shape. The center of the panorama map 50 (a sphere formed by a plurality of plate models 51) corresponds to the installation position of the imaging unit 11, and each plate model 51 is a first captured image by the first monocular imaging unit 11a. It corresponds to the shooting range. That is, the number of plate models 51 is the same as the number of first captured images by the first monocular imaging unit 11a, and the corners are formed by two line segments respectively connecting the center of the panorama map 50 and the centers of the adjacent plate models 51. Corresponds to the angle difference of the shooting direction by the first monocular imaging unit 11a when the adjacent first captured images are captured at the installation position of the first monocular imaging unit 11a. The aspect ratio corresponds to the aspect ratio of each first photographed image of the first monocular photographing unit 11a, and the distance from the center of the sphere formed by the plurality of plate models 51 to the plate model 51 and the aspect ratio of the plate model 51. The size corresponds to the focal length of the first monocular imaging unit 11a and the vertical and horizontal sizes of the captured image. Each plate model 51 is assigned an ID in the same manner as the first captured image data, and the monocular panoramic image data generation unit 33 is assigned to the ID and the plate model 51 assigned to the first captured image data. On the basis of the ID, the first captured image data is mapped to the corresponding plate model 51 to generate monocular panoramic image data, and the generated monocular panoramic image data is output to the image data output unit 34. Or stored in the storage unit 32. Here, the monocular panoramic image data generation unit 33 can generate monocular panoramic image data in a predetermined range centered on the viewpoint position based on the viewpoint position instruction output from the input unit 20.

<< Image Data Output Unit 34 >>
The image data output unit 34 outputs monocular panoramic image data to the display device 40. In the present embodiment, the monocular panoramic image data output from the monocular panoramic image data generation unit 33 is output to the display device 40. Or the second captured image data and the third captured image data read from the captured image data storage unit 31 are output to the display device 40.

<< Image Data Specification Acquisition Unit 35 >>
The image data designation obtaining unit 35 obtains designation of one first photographed image data from the monocular panoramic image displayed on the display device 40 in accordance with a user operation, and performs such designation result (for example, the designated first ID of the captured image data) is output to the image data output unit 34. When the designation result is acquired, the image data output unit 34 reads out the second photographed image data and the third photographed image data corresponding to the designation result from the photographed image data storage unit 31, and the second photographed image data and the third photographed image data. Is output to the display device 40, and the second captured image and the third captured image corresponding to the designation result are displayed on the display device 40.

≪Specified point acquisition unit 36≫
The designated point acquisition unit 36 obtains the designated point instruction in the second photographed image and the designated point instruction in the third photographed image output from the input device 20. In addition, the designated point acquisition unit 35 uses the obtained designated point instruction to obtain two-dimensional coordinates (X-axis and Y-axis coordinate values) of the designated point in each captured image data stored in the captured image data storage unit 31. Is output to the first three-dimensional coordinate calculation unit 37.

≪First three-dimensional coordinate calculation unit 37≫
The first three-dimensional coordinate calculation unit 37 includes the two-dimensional coordinates of the designated point output from the designated point acquisition unit 36, the distance d between the monocular imaging units 11b and 11c, and the focal length f of the monocular imaging units 11b and 11c. And the first three-dimensional coordinates of the designated point are calculated and output to the second three-dimensional coordinate calculation unit 38. Here, the first three-dimensional coordinate is a coordinate in the first three-dimensional coordinate system unique to each photographing direction with reference to the photographing direction of the monocular photographing units 11a to 11c, and the first three-dimensional coordinate system is the photographing. Also referred to as a three-dimensional coordinate system, the first three-dimensional coordinates are also referred to as photographing three-dimensional coordinates. That is, the first three-dimensional coordinate system is common to the second and third captured images having the same shooting direction, and is different between images having different shooting directions.

(First 3D coordinate calculation principle)
Here, the calculation principle of the first three-dimensional coordinate in the first three-dimensional coordinate calculation unit 37 will be described with reference to FIGS. In the following, the arrangement direction of the monocular imaging units 11a, 11b, and 11c is the X axis, the optical axis direction of the monocular imaging units 11a, 11b, and 11c is the Z axis, and the right hand coordinate system is composed of the Y axis that is orthogonal to the X axis and the Z axis. Consider a three-dimensional coordinate space. If the first monocular imaging unit 11a is arranged at the origin in the first three-dimensional coordinate system and the distance between the second monocular imaging unit 11b on the left side and the third monocular imaging unit 11c on the right side is d, the first three-dimensional coordinate O ₁ monocular imaging unit _11a, the first three-dimensional coordinates O ₃ of the first three-dimensional coordinate O _2, third monocular photography portion 11c of the second monocular photography section 11b, It becomes as follows.
O ₁ = (0, 0, 0)
O ₂ = (− d / _2, 0, 0)
O ₃ = (d / 2, 0, 0)

The object located at the relative three-dimensional coordinates R = (x _R , y _R , z _R ) is imaged using the monocular imaging units 11b and 11c, and the object is captured by the second monocular imaging unit 11b. The two-dimensional coordinates N ₂ = (x ₂ , y ₂ ), that is, the first three-dimensional coordinates (x ₂ , y ₂ , −f), and the third monocular imaging unit 11c captures the third captured image. It is assumed that the two-dimensional coordinates N ₃ = (x ₃ , y ₃ ), that is, the first three-dimensional coordinates (x ₃ , y ₃ , −f).
Here, since the triangle RO ₂ O ₃ and the triangle RN ₂ N ₃ are similar, the following relationship is established.
_{x R = d (x 2 +} x 3) / {2 (x 2 -x 3 + d)}
_{y R = d (y 2 +} y 3) / {2 (y 2 -y 3 + d)}
z _R = −df / {2 (x ₂ −x ₃ + d)}
That is, the first three-dimensional coordinate calculation unit 37 uses the predetermined focal distance f and interval d and the value of the two-dimensional coordinates of the photographed image to obtain the first three-dimensional coordinates of the desired position. R = (x _R , y _R , z _R ) can be calculated.

«Second three-dimensional coordinate calculation unit 38»
The second three-dimensional coordinate calculation unit 38 includes the first three-dimensional coordinates of the designated point output from the first three-dimensional coordinate calculation unit 37 and the shooting directions of the monocular shooting units 11b and 11c at the time of shooting. Based on this, the second three-dimensional coordinates of the designated point are calculated and output to the three-dimensional distance calculation unit 39 and the image data output unit 34. By doing so, the calculated second three-dimensional coordinates are displayed on the display device 40 described later. Here, the second three-dimensional coordinate is a coordinate in the second three-dimensional coordinate system that is common to all photographing directions, and is integrated with the first three-dimensional coordinate system, and the second three-dimensional coordinate system. Is also called a panoramic 3D coordinate system, and the second 3D coordinate is also called a panoramic 3D coordinate system. That is, the second three-dimensional coordinate system is common to all the second captured images and the third captured images.

(Second 3D coordinate calculation principle)
Here, the calculation principle of the second three-dimensional coordinate in the second three-dimensional coordinate calculation unit 38 will be described. The second three-dimensional coordinate calculation unit 38 sets the rotation angle around the Y axis from the reference of the shooting direction of the monocular imaging units 11b and 11c in the second three-dimensional coordinate system as θ, and the rotation angle around the X axis as φ. Then, the second three-dimensional coordinate A = (x _A , y _A , z _A ) of the designated point can be calculated by the following formula.

ここで、回転角度θ，φとしては、パノラママップ５０の板モデル５１に関連付けてマップ記憶部３２に予め記憶されたものを用いることができる。

Here, as the rotation angles θ and φ, those stored in advance in the map storage unit 32 in association with the plate model 51 of the panorama map 50 can be used.

Here, as shown in FIGS. 4A and 4B, the rotation of the angle φ around the X axis (X _R axis) of the relative three-dimensional coordinate and the rotation around the Y axis (Y _R axis) of the relative three-dimensional coordinate. by applying rotation and an angle theta, to the first three-dimensional coordinate system, X _R-axis relative three-dimensional coordinates, Y _R and Z axes (Z _R axis), X of the second three-dimensional coordinate system axis _{(X a-axis),} Y axis _{(Y a-axis)} and Z-axis and _{(Z a} axis) coincides respectively. Incidentally, phi, when viewed from the positive side of the X _A-axis counterclockwise positive, negative clockwise. θ, when viewed from the positive side of Y _A-axis counterclockwise positive, negative clockwise.
That is, the rotation around the Y axis (rotation in the pan direction) and the rotation around the X axis are converted into the first three-dimensional coordinates R = (x _R , y _R , z _R ) of the designated point in the first three-dimensional coordinate system. The second three-dimensional coordinates A = (x _A , y _A , z _A ) of the designated point are calculated by performing the rotation (rotation in the tilt direction). In the present embodiment, the origin in the second three-dimensional coordinate system and the origin in the first three-dimensional coordinate system, that is, O ₁ are the same position, and the rotation angle θ is equal to the Y-axis of the second three-dimensional coordinate system. The rotation angle φ is an angle formed by the X axis of the second three-dimensional coordinate system and the X axis of the first three-dimensional coordinate system. In the present embodiment, the X-axis in the relative three-dimensional coordinate system is always maintained in the horizontal direction, so that rotation conversion around the Z-axis is not necessary.

≪Three-dimensional distance calculation unit 39≫
The three-dimensional distance calculation unit 39 acquires the second three-dimensional coordinates of the two specified points output from the second three-dimensional coordinate calculation unit 38, and based on the acquired second three-dimensional coordinates, 2 The three-dimensional distance between the two designated points is calculated and output to the image data output unit 34. Then, the image data output unit 34 can output the calculated three-dimensional distance to the display device 40 and display it on the display device 40.

<Display device 40>
The display device 40 displays a panoramic image using the panoramic image data output from the three-dimensional coordinate calculation device 30, or uses the second captured image data and the third captured image data to display the second captured image and the third captured image. Or display an image.

<Operation example>
Subsequently, regarding the operation example of the panoramic image display system 1A according to the first embodiment of the present invention, the second three-dimensional coordinates and the three-dimensional distances of the designated points S and E in the pillars included in the different first captured images are obtained. An example of calculation will be described with reference to FIGS.

  First, the control unit 13 rotates the drive unit 12 to direct the monocular imaging units 11a, 11b, and 11c in a desired imaging direction, and controls the shutters of the monocular imaging units 11a, 11b, and 11c in this state. The photographed image data relating to the photographed image is output to the three-dimensional coordinate calculation device 30.

  Subsequently, the monocular panoramic image data generation unit 33 reads a plurality of first photographed image data photographed by the first monocular photographing unit 11a from the photographed image data storage unit 31, and uses the read first photographed image data as the read first photographed image data. Based on this, monocular panoramic image data is generated and output to the image data output unit 34. The image data output unit 34 outputs the monocular panoramic image data to the display device 40, and the display device 40 displays the monocular panoramic image (see FIG. 5A).

  In this state, when the input device 20 outputs an image data designation for designating one first captured image 111 in the monocular panoramic image in accordance with the operation of the input device 20 by the user, the image data designation acquisition unit 35. Acquires the image data designation and outputs it to the image data output unit 34. The image data output unit 34 takes the second photographed image data and the third photographed image data having the same ID as the ID included in the image data designation. Read from the storage unit 31.

  The image data output unit 34 outputs the second photographed image data and the third photographed image data to the display device 40, and the display device 40 has the second photographed image 112 and the third photographed image in the same photographing direction as the first photographed image 111. The captured image 113 is displayed on the same screen (see FIG. 5B). In the present embodiment, the second photographed image 112 and the third photographed image 113 are arranged one above the other.

  In this state, when the user operates the input device 20 and designates a designated point (starting point) S1 (corresponding to the designated point S) in the second photographed image 112, the designated point instruction in the second photographed image 112 is designated as a designated point acquisition unit. 36.

Subsequently, when the user operates the input device 20 to designate a designated point (start point) S2 (corresponding to the designated point S) in the third photographed image 113, the designated point instruction in the third photographed image 113 is designated by the designated point acquisition unit 36. Is input.
When the designated point acquisition unit 36 acquires the two designated point instructions, the two-dimensional coordinates (x ₁₁ , y ₁₁ ) and the third point of the designated point S 1 in the second photographed image 112 stored in the photographed image data storage unit 31. The two-dimensional coordinates (x ₂₁ , y ₂₁ ) of the designated point S 2 in the captured image 113 are acquired and output to the first three-dimensional coordinate calculation unit 37.

Subsequently, the first three-dimensional coordinate calculation unit 37 calculates the first three-dimensional coordinates R _S = (x _RS , y _RS , z _RS ) of the designated point S based on the following formula, Output to the three-dimensional coordinate calculation unit 38.
_{x RS = d (x 11 +} x 21) / {2 (x 11 -x 21 + d)}
_{y RS = d (y 11 +} y 21) / {2 (y 11 -y 21 + d)}
_{z RS = -df / {2 (} x 11 -x 21 + d)}

  Subsequently, in a state in which the monocular panoramic image is displayed again on the display device 40 in accordance with the operation of the input device 20 by the user (see FIG. 6A), the input device 20 is the other single sheet in the monocular panoramic image. When the image data designation for designating the first captured image 121 is output, the image data designation obtaining unit 35 obtains the image data designation and outputs it to the image data output unit 34. The image data output unit 34 performs the image data designation. The second photographed image data and the third photographed image data with the same ID as the ID included in the photographed image data storage unit 31 are read out.

  The image data output unit 34 outputs the second photographed image data and the third photographed image data to the display device 40, and the display device 40 has the second photographed image 122 and the third photographed image in the same photographing direction as the first photographed image 121. The captured image 123 is displayed on the same screen (see FIG. 6B).

  In this state, when the user operates the input device 20 and designates a designated point (end point) E1 (corresponding to the end point E) in the second photographed image 122, the designated point instruction in the second photographed image 122 is designated. Is input.

Subsequently, when the user operates the input device 20 to designate a designated point (end point) E2 (corresponding to the end point E) in the third photographed image 123, a designated point instruction in the third photographed image 123 is sent to the designated point acquisition unit 36. Entered.
When the designated point acquisition unit 36 acquires the two designated point instructions, the two-dimensional coordinates (x ₁₂ , y ₁₂ ) and the third point of the designated point E1 in the second photographed image 122 stored in the photographed image data storage unit 31 are stored. The two-dimensional coordinates (x ₂₂ , y ₂₂ ) of the designated point E 2 in the captured image 123 are acquired and output to the first three-dimensional coordinate calculation unit 37.

Subsequently, the first three-dimensional coordinate calculation unit 37 calculates the relative three-dimensional coordinates R _E = (x _RE , y _RE , z _RE ) of the designated point E based on the following formula, and the second three-dimensional coordinates It outputs to the coordinate calculation part 38.
_{x RE = d (x 12 +} x 22) / {2 (x 12 -x 22 + d)}
y _RE = d (y ₁₂ + y ₂₂ ) / {2 (y ₁₂ −y ₂₂ + d)}
z _RE = −df / {2 (x ₁₂ −x ₂₂ + d)}

The second three-dimensional coordinate calculation unit 38 calculates the second three-dimensional coordinates A _S = (x _AS , y _AS , z _AS ), A _E = (x _AE , y _AE , z _AE ) is calculated and output to the three-dimensional distance calculation unit 39.

Subsequently, the three-dimensional distance calculation unit 39 calculates a three-dimensional distance D between the designated points S and E based on the following formula. Thereby, the three-dimensional distance is displayed on the display device 40.
^{_{D = {(x AE -x AS}} ) 2 + (y AE -y AS) 2 + (z AE -z AS) 2} 1/2

  The panoramic image display system 1A according to the first embodiment of the present invention can calculate second three-dimensional coordinates that can be commonly used for all images included in the panoramic image. Further, the panoramic image display system 1A can calculate a three-dimensional distance between two designated points. Further, the panoramic image display system 1 </ b> A has a second image corresponding to two captured images for acquiring specified points rather than the first captured image data corresponding to one captured image for a monocular panoramic image in one imaging direction. By setting the shooting range of each of the shot image data and the third shot image data wider, even if the vicinity of the periphery in the first shot image data is set as the specified point, the specified point is the second. Thus, the second three-dimensional coordinate and the three-dimensional distance can be reliably calculated because the captured image data and the third captured image data are surely within the photographing range.

<Second Embodiment>
Next, a panoramic image display system according to the second embodiment of the present invention will be described focusing on differences from the panoramic image display system 1A according to the first embodiment.

  As shown in FIG. 7, the panoramic image display system 1B according to the second embodiment of the present invention includes a compound eye photographing device 10B instead of the compound eye photographing device 10A. The compound eye photographing device 10B includes a first monocular photographing unit 11a and a second monocular photographing unit 11b having a focal length f, and the third monocular photographing unit 11c is omitted. The first photographed image data photographed by the first monocular photographing unit 11a in the present embodiment is used in the same manner as the first photographed image data and the second photographed image data in the first embodiment. The second captured image data captured by the second monocular imaging unit 11b is used in the same manner as the third captured image data in the first embodiment.

As shown in FIG. 8, the first monocular imaging unit 11a is arranged at the origin in the first three-dimensional coordinate system, and the distance between the left first monocular imaging unit 11a and the right second monocular imaging unit 11b. the When d, the first three-dimensional coordinate O ₁ of the first monocular photography portion _11a, the first three-dimensional coordinate O ₂ of the second monocular photography portion 11b is as follows.
O ₁ = (0, 0, 0)
O ₂ = (d, 0, 0)

An object located at relative three-dimensional coordinates R = (x _R , y _R , z _R ) is imaged using the monocular imaging units 11a and 11b, and the object is captured by the second monocular imaging unit 11a. The two-dimensional coordinates N ₁ = (x ₁ , y ₂ ), that is, the first three-dimensional coordinates (x ₁ , y ₁ , −f), and the second captured image of the second monocular imaging unit 11b. It is assumed that the _two -dimensional coordinates N ₂ = (x ₂ , y ₂ ), that is, the first three-dimensional coordinates (x ₂ , y ₂ , −f).
Here, since the triangle RO ₁ O ₂ and the triangle RN ₁ N ₂ are similar, the following relationship is established.
_{x R = dx 1 / (x} 1 -x 2)
_{y R = dy 1 / (x} 1 -x 2)
z _R = −df / (x ₁ −x ₂ )
That is, the first three-dimensional coordinate calculation unit 37 uses the predetermined focal distance f and interval d and the value of the two-dimensional coordinates of the photographed image to obtain the first three-dimensional coordinates of the desired position. R = (x _R , y _R , z _R ) can be calculated.

<Operation example>
Subsequently, regarding the operation example of the panoramic image display system 1B according to the second embodiment of the present invention, the second three-dimensional coordinates and the three-dimensional distances of the designated points S and E in the pillars included in the different first captured images are obtained. An example of calculation will be described with reference to FIGS.

  First, the control unit 13 rotates the drive unit 12 to direct the monocular imaging units 11a and 11b in a desired imaging direction, and in this state, controls the shutters of the monocular imaging units 11a and 11b to perform imaging and imaging. The captured image data relating to the image is output to the three-dimensional coordinate calculation device 30.

  Subsequently, the monocular panoramic image data generation unit 33 reads a plurality of first photographed image data photographed by the first monocular photographing unit 11a from the photographed image data storage unit 31, and uses the read first photographed image data as the read first photographed image data. Based on this, monocular panoramic image data is generated and output to the image data output unit 34. The image data output unit 34 outputs the monocular panoramic image data to the display device 40, and the display device 40 displays the monocular panoramic image (see FIG. 9A).

  In this state, when the input device 20 outputs an image data designation for designating one first captured image 111 in the monocular panoramic image in accordance with the operation of the input device 20 by the user, the image data designation acquisition unit 35. Acquires the image data designation and outputs it to the image data output unit 34. The image data output unit 34 takes the first photographed image data and the second photographed image data having the same ID as the ID included in the image data designation. Read from the storage unit 31.

  The image data output unit 34 outputs the first photographed image data and the second photographed image data to the display device 40, and the display device 40 is the second photographed in the same photographing direction as the first photographed image 111 and the first photographed image 111. The captured image 112 is displayed on the same screen (see FIG. 9B). In the present embodiment, the first captured image 111 and the second captured image 112 are arranged side by side.

  In this state, when the user operates the input device 20 and designates a designated point (start point) S1 (corresponding to the designated point S) in the first photographed image 111, the designated point instruction in the first photographed image 111 is designated as a designated point acquisition unit. 36.

Subsequently, when the user operates the input device 20 and designates a designated point (start point) S2 (corresponding to the designated point S) in the second photographed image 112, the designated point instruction in the second photographed image 112 is designated by the designated point acquisition unit 35. Is input.
When the designated point acquisition unit 36 acquires the two designated point instructions, the two-dimensional coordinates (x ₁₁ , y ₁₁ ) of the designated point S1 in the first captured image 111 and the second stored in the captured image data storage unit 31 are stored. The two-dimensional coordinates (x ₂₁ , y ₂₁ ) of the designated point S 2 in the captured image 112 are acquired and output to the first three-dimensional coordinate calculation unit 37.

Subsequently, the first three-dimensional coordinate calculation unit 37 calculates the first three-dimensional coordinates R _S = (x _RS , y _RS , z _RS ) of the designated point S based on the following formula, Output to the three-dimensional coordinate calculation unit 38.
_{x RS = dx 11 / (x} 11 -x 21)
_{y RS = dy 11 / (x} 11 -x 21)
_{z RS = -df / (x 11} -x 21)

  Subsequently, in a state in which the monocular panoramic image is displayed again on the display device 40 in accordance with the operation of the input device 20 by the user (see FIG. 10A), the input device 20 has another one of the monocular panoramic images. When the image data designation for designating the first captured image 121 is output, the image data designation obtaining unit 35 obtains the image data designation and outputs it to the image data output unit 34. The image data output unit 34 performs the image data designation. First captured image data and second captured image data having the same ID as the ID included in the captured image data storage unit 31 are read out.

  The image data output unit 34 outputs the first photographed image data and the second photographed image data to the display device 40, and the display device 40 is a second image that has the same photographing direction as the first photographed image 121 and the first photographed image 121. The captured image 122 is displayed on the same screen (see FIG. 10B).

  In this state, when the user operates the input device 20 to designate a designated point (end point) E1 (corresponding to the end point E) in the first photographed image 121, the designated point instruction in the first photographed image 121 is designated. Is input.

Subsequently, when the user operates the input device 20 to designate a designated point (end point) E2 (corresponding to the end point E) in the second photographed image 122, a designated point instruction in the third photographed image 122 is sent to the designated point acquisition unit 36. Entered.
When the designated point acquisition unit 36 acquires the two designated point instructions, the two-dimensional coordinates (x ₁₂ , y ₁₂ ) and the second point of the designated point E1 in the first captured image 121 stored in the captured image data storage unit 31 are stored. The two-dimensional coordinates (x ₂₂ , y ₂₂ ) of the designated point E2 in the captured image 122 are acquired and output to the first three-dimensional coordinate calculation unit 37.

Subsequently, the first three-dimensional coordinate calculation unit 37 calculates the relative three-dimensional coordinates R _E = (x _RE , y _RE , z _RE ) of the designated point E based on the following formula, and the second three-dimensional coordinates It outputs to the coordinate calculation part 38.
_{x RE = dx 12 / (x} 12 -x 22)
y _RE = dy ₁₂ / (x ₁₂ −x ₂₂ )
z _RE = −df / (x ₁₂ −x ₂₂ )

The second three-dimensional coordinate calculation unit 38 calculates the second three-dimensional coordinates A _S = (x _AS , y _AS , z _AS ), A _E = (x _AE , y _AE , z _AE ) is calculated and output to the three-dimensional distance calculation unit 39.

Subsequently, the three-dimensional distance calculation unit 39 calculates a three-dimensional distance D between the designated points S and E based on the following formula. Thereby, the three-dimensional distance is displayed on the display device 40.
^{_{D = {(x AE -x AS}} ) 2 + (y AE -y AS) 2 + (z AE -z AS) 2} 1/2

  The panoramic image display system 1B according to the second embodiment of the present invention can omit one monocular photographing unit as compared with the panoramic image display system 1A.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, A design change is possible suitably in the range which does not deviate from the summary of this invention. For example, the method of notifying the user of the calculated second three-dimensional coordinates and the three-dimensional distance is not limited to being displayed on the display device 40, and may be configured to utter a speaker as a notification unit. When the rotation around the Z axis (rotation angle ψ) is also taken into consideration, the second three-dimensional coordinate can be calculated by the following equation.

  Further, in the present invention, the right-handed coordinate system is adopted as the first three-dimensional coordinate system and the second three-dimensional coordinate system, but the case where the left-handed coordinate system is adopted is also included in the technical scope of the present invention.

1A, 1B Panoramic image display system 30 Three-dimensional coordinate calculation device 31 Image data storage unit 33 Monocular panoramic image data generation unit 34 Image data output unit 35 Image data designation acquisition unit 36 Designated point acquisition unit 37 First three-dimensional coordinate calculation unit 38 Second 3D coordinate calculation unit 39 3D distance calculation unit

Claims (4)

A plurality of first image data photographed by the first photographing device, a plurality of second image data photographed by a second photographing device spaced apart from the first photographing device by a predetermined distance, and a predetermined distance from the first photographing device. A plurality of third image data photographed by the separated third photographing device, and an image data storage unit stored in association with each photographing direction;
A panorama image data generation unit that reads the plurality of first image data stored to generate panorama image data;
An image data output unit for outputting the generated panoramic image data to a display device;
An image data designation acquisition unit for acquiring designation of one of the first image data in the panoramic image data in response to an operation of an input device by a user;
With
The image data output unit outputs the second image data and the third image data associated with the designated one first image data to the display device,
In accordance with the operation of the input device by the user, the designated point in the second image data is obtained, and the designated point obtaining unit that obtains the designated point in the third image data;
Shooting based on the shooting direction of the first shooting device, the second shooting device, and the third shooting device based on the specified point in the acquired second image data and the specified point in the third image data A first three-dimensional coordinate calculation unit that calculates a first three-dimensional coordinate in the first three-dimensional coordinate system unique to each direction;
Based on the calculated first three-dimensional coordinates and the photographing directions of the second photographing device and the third photographing device at the time of photographing, in the second three-dimensional coordinate system common to all photographing directions. A second three-dimensional coordinate calculation unit for calculating a second three-dimensional coordinate of the designated point;
A three-dimensional coordinate calculation apparatus comprising: The three-dimensional imaging coordinates are the Z axis that is the optical axis direction of the first imaging device, the second imaging device, and the third imaging device, the X axis that is orthogonal to the Z axis, the Z axis, and the X axis. When expressed by a right-handed coordinate system with the Y axis orthogonal to the axis,
The focal length of the second imaging device and the third imaging device is f,
The first three-dimensional coordinates of the second imaging device are (−d / 2, 0, 0),
The first three-dimensional coordinates of the third imaging device are (d / 2, 0, 0),
The first three-dimensional coordinates of the designated point in the second image data are (x ₂ , y ₂ , −f),
The first three-dimensional coordinates of the designated point in the third image data are (x ₃ , y ₃ , −f),
The first three-dimensional coordinates of the designated point are (x _R , y _R , z _R ),
When the first three-dimensional coordinate calculation unit,
_{x R = d (x 2 +} x 3) / {2 (x 2 -x 3 + d)}
_{y R = d (y 2 +} y 3) / {2 (y 2 -y 3 + d)}
z _R = −df / {2 (x ₂ −x ₃ + d)}
To calculate the first three-dimensional coordinates of the designated point,
In the second three-dimensional coordinate system, the rotation angle around the X axis from the reference of the shooting direction of the second imaging device and the third imaging device is φ, the rotation angle around the Y axis is θ, and the Z axis When the surrounding rotation angle is ψ, the second three-dimensional coordinate calculation unit is

The three-dimensional coordinate calculation apparatus according to claim 1, wherein the second three-dimensional coordinates of the designated point are calculated by: A plurality of first image data photographed by the first photographing device and a plurality of second image data photographed by a second photographing device separated by a predetermined distance from the first photographing device are stored in association with each photographing direction. An image data storage unit,
A panorama image data generation unit that reads the plurality of first image data stored to generate panorama image data;
An image data output unit for outputting the generated panoramic image data to a display device;
An image data designation acquisition unit for acquiring designation of one of the first image data in the panoramic image data in response to an operation of an input device by a user;
With
The image data output unit outputs the specified first image data and the second image data associated with the first image data to the display device,
In response to an operation of the input device by the user, a designated point in the first image data is obtained, and a designated point obtaining unit that obtains the designated point in the second image data;
Based on the designated point in the acquired first image data and the designated point in the second image data, the first unique to each photographing direction with reference to the photographing direction of the first photographing device and the second photographing device. A first three-dimensional coordinate calculator that calculates first three-dimensional coordinates in one three-dimensional coordinate system;
Based on the calculated first three-dimensional coordinates and the photographing directions of the first photographing device and the second photographing device at the time of photographing, in the second three-dimensional coordinate system common to all photographing directions. A second three-dimensional coordinate calculation unit for calculating a second three-dimensional coordinate of the designated point;
A three-dimensional coordinate calculation apparatus comprising: The three-dimensional imaging coordinates are the Z axis that is the optical axis direction of the first imaging device and the second imaging device, the X axis that is orthogonal to the Z axis, and the Y axis that is orthogonal to the Z axis and the X axis. When expressed by a right-handed coordinate system,
The focal length of the first photographing device and the second photographing device is f,
The first three-dimensional coordinates of the first photographing device are (0, 0, 0),
The first three-dimensional coordinates of the second imaging device are (d, 0, 0),
The first three-dimensional coordinates of the designated point in the first image data are (x ₁ , y ₁ , −f),
The first three-dimensional coordinates of the designated point in the second image data are (x ₂ , y ₂ , −f),
The first three-dimensional coordinates of the designated point are (x _R , y _R , z _R ),
When the first three-dimensional coordinate calculation unit,
_{x R = dx 1 / (x} 1 -x 2)
_{y R = dy 1 / (x} 1 -x 2)
z _R = −df / (x ₁ −x ₂ )
To calculate the first three-dimensional coordinates of the designated point,
In the second three-dimensional coordinate system, the rotation angle around the X axis from the reference of the shooting direction of the second imaging device and the third imaging device is φ, the rotation angle around the Y axis is θ, and the Z axis When the surrounding rotation angle is ψ, the second three-dimensional coordinate calculation unit is

The three-dimensional coordinate calculation apparatus according to claim 3, wherein the second three-dimensional coordinates of the designated point are calculated by:

Images