JP7025042B2 - Spherical image generation method, spherical image generation and display method, spherical image generation system program, and spherical image generation and display system program - Google Patents

Description

The present invention relates to an omnidirectional imaging system capable of photographing in all directions, and more specifically to a spherical imaging system capable of suppressing horizontal shake.

An omnidirectional imaging system is known that captures moving images and still images obtained by using an image pickup device and synthesizes them into an omnidirectional sphere (omnidirectional). In this system, the image pickup device is equipped with a plurality of image pickup means, and distortion correction is performed on each photographed image taken by the image pickup means, and by synthesizing these, a substantially rectangular spherical image that can be displayed on a monitor or the like. Is being generated.

When the posture of the image pickup device changes, the generated spherical image also changes according to the posture, so that the spherical image lacks visibility.

Therefore, Patent Document 1 includes a sensor that detects the tilt of the image pickup device in the vertical direction, and converts the captured image into an all-sky image corrected to global coordinates according to the tilt of the image pickup device detected by the sensor. We are proposing a system.

Japanese Unexamined Patent Publication No. 2013-214947

As described above, by converting the spherically captured image into global coordinates, a converted image in which the inclination with respect to the vertical direction is corrected is created. However, only by correcting the inclination with respect to the vertical direction, the rotation in the horizontal direction (rotation around the axis in the vertical direction) is not corrected, and the image lacks visibility. For example, when an image pickup device is attached to the user's head and a movie is shot while riding a bicycle, the tilt in the vertical direction is corrected, but the horizontal rotational sway remains, resulting in poor visibility. It will be a video.

An object of the present invention is to provide an omnidirectional imaging system capable of creating a omnidirectional image full of realism while improving visibility.

The spherical photography system according to the present invention is
Multiple imaging means to output image data and
A posture detecting means for detecting the posture information of at least one of the plurality of imaging means, and a posture detecting means.
A control means for generating an omnidirectional image in which the image data is coordinate-transformed based on the attitude information output by the attitude detecting means.
With
The control means generates an omnidirectional image by performing coordinate conversion of the image data by tilt correction in the vertical direction in global coordinates and correction of shaking in the horizontal plane in global coordinates excluding minute shaking components.

An image compositing unit that creates spherical frame data from image data captured by the plurality of imaging means, and
A correction amount calculation unit that creates correction amount data for converting the coordinates of the spherical frame data from the attitude detection means based on the attitude information of at least one image pickup means.
An association means for associating the spherical frame data with the correction amount data to obtain a spherical image,
With
The correction amount calculation unit creates correction amount data for the spherical frame data, which corrects the inclination in the vertical direction in the global coordinates and corrects the shaking in the horizontal plane in the global coordinates excluding the minute shaking component. Can be.

The attitude information is between local coordinates including x-axis, y-axis and z-axis orthogonal to each other with respect to the imaging means, and global coordinates including X-axis, Y-axis and vertical Z-axis orthogonal to each other. Information that represents the amount of rotational displacement of
The correction amount calculation unit creates a global coordinate correction amount for correcting the coordinates of the whole celestial sphere frame data to the global coordinates as correction amount data based on the attitude information, and the global coordinate correction amount is It is possible to correct the inclination of the whole celestial sphere frame data with respect to the vertical direction, and to correct only the high frequency component of the shaking for the rotation around the vertical axis.

The associating means can associate the correction amount data as header information of the spherical frame data.

The imaging means captures a moving image and
The correction amount calculation unit can create correction amount data based on the attitude information of the spherical frame data one frame before, which is input from the attitude detection means.

Further, the spherical photography system of the present invention is
With multiple imaging means
A posture detecting means for detecting the posture information of at least one of the plurality of imaging means, and a posture detecting means.
Image data that is the output of the plurality of imaging means and
A control means that generates and displays an image signal based on a posture detection signal that is an output of the posture detection means.
With
For global coordinates including the X-axis, Y-axis, and vertical Z-axis that are orthogonal to each other
A first image signal generated by rotating the plurality of image pickup means by a first angle in the X-axis direction and taking a picture.
A second image signal generated by rotating the plurality of image pickup means by the first angle in the Y-axis direction and taking a picture.
A third image signal generated by rotating the plurality of image pickup means by the first angle in the Z-axis direction and taking a picture.
When an image signal is generated and displayed by the control means from
In the control means, the first image signal and the second image signal display an image having substantially no rotational movement, and the third image signal displays an image having substantially the first angle of rotational movement. ..

According to the spherical imaging system of the present invention, the spherical image created from the image data is not only corrected in the vertical direction but also follows the rotation of the imaging means in the horizontal plane, so that it is full of realism. Since the camera shake, which is a minute vibration in the horizontal plane, is also corrected, the image can be obtained with excellent visibility.

FIG. 1 is a block diagram of the spherical photography system of the present invention. FIG. 2 is a plan view of the image pickup apparatus. FIG. 3 is a diagram showing the relationship between the local coordinates and the global coordinates. FIG. 4 is a diagram showing the relationship between the global coordinates and the direction vector. FIG. 5 is a graph showing (a) rotation angle θ _Z and (b) rotation angle θ _{Z_lpf} around the Z axis. FIG. 6 is an explanatory diagram of a procedure for creating a spherical image from a captured image.

Hereinafter, the spherical photography system 10 of the present invention will be described with reference to the drawings.

The present invention relates to the posture of the image pickup device 20 in the spherical imaging system 10 that captures an omnidirectional image (omnidirectional) and converts the obtained image into spherical frame data to obtain a spherical image. Instead, the spherical image to be created is converted according to the vertical direction, and the horizontal direction is corrected according to the posture of the image pickup device 20 according to the rotation direction, and the spherical image in the horizontal direction is blurred. Shake correction is performed to reduce (camera shake). As a result, it is possible to provide a spherical image with high visibility while being rich in presence.

FIG. 1 is a block diagram of the spherical photography system 10 showing an embodiment of the present invention, and FIG. 2 is a schematic external view of the image pickup apparatus 20. Note that FIG. 1 depicts only functional blocks related to the present invention in order to explain the present invention in an easy-to-understand manner, but these are recorded in a memory such as a CPU, RAM, ROM, flash memory, a buffer, and the like. It can be realized by connecting various programs and the like. It should be understood that these functional blocks can be realized by hardware only, software only or a combination thereof, and that these functions can be communicably connected by wireless or wired connection. Further, these functions can be realized by one device, or can be a system in which a plurality of functions such as imaging and display are individually realized by a plurality of devices.

The spherical photography system 10 includes one or a plurality of image sensors 22, 22 (imaging elements) as an imaging means. The image sensors 22 and 22 are, for example, CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensors, and optical lenses 24 and 24 are mounted on them. The optical lens 24 can be composed of a wide-angle lens, an ultra-wide-angle lens, or a plurality of lens groups, and image data of the whole celestial sphere can be acquired by the image sensors 22 and 22.

For example, when there are two image sensors 22 and 22, the optical lenses 24 and 24 employ a wide-angle lens or an ultra-wide-angle lens capable of capturing an angle of view of 180 ° or more, and as shown in FIG. 2, the image sensors 22 and 22 are used. Is arranged so that the front side and the back side of the image pickup apparatus 20 can be photographed.

The image data sequentially output from the image sensors 22 and 22 is transmitted to the control means 11. As shown in FIG. 1, the control means 11 can be configured to include, for example, an image composition unit 30, a posture calculation unit 50, a correction amount calculation unit 60, an association unit 70, a memory 80, and a display device 82.

The image data output from the image sensors 22 and 22 is transmitted to the image compositing unit 30, and spherical frame data is created by sequentially performing distortion correction and compositing. For example, the image synthesizing unit 30 corrects the distortion of the obtained image data to create spherical frame data consisting of equirectangular images. The created spherical frame data is transmitted to the association means 70 described later.

On the other hand, the posture information of the image pickup device 20, that is, the inclination or rotation of the image pickup device 20 with respect to the global coordinates is detected by the posture detection means 40 at a predetermined sampling cycle, and is calculated by the posture calculation unit 50.

For example, the attitude detecting means 40 can be an angular velocity sensor 42 and an acceleration sensor 44, as shown in FIG. Further, the posture detecting means 40 may be configured to include a direction sensor together with these sensors.

As the attitude information, as shown in FIG. 3, local coordinates including x-axis, y-axis and z-axis orthogonal to each other with respect to the image pickup apparatus 20 and X-axis, Y-axis and vertical Z-axis orthogonal to each other are used. It can be illustrated in relation to the global coordinates including. The y-axis of the local coordinates is set to coincide with the optical axis of any one of the image sensors 22, and the origin of the local coordinates is set to the center between the image sensors 22 and 22. In FIG. 3, the Z axis indicates the vertical direction (gravity direction) of the global coordinates.

When the angular velocity sensor 42 and the acceleration sensor 44 are adopted as the attitude detecting means 40, as shown in FIG. 3, the angular velocity sensor 42 refers to the image pickup device 20 with respect to the x-axis, y-axis and z-axis of the image pickup device 20. The rotational angular velocity (gx, gy, gz) around each axis of the local coordinates is output, and the acceleration sensor 44 decomposes the gravitational acceleration into each axial direction of the local coordinates of the image pickup device 20. ax, ay, az) is output.

The attitude calculation unit 50 calculates the attitude information of the image pickup apparatus 20 based on the outputs (rotational angular velocity and gravitational acceleration component) obtained from the sensors 42 and 44. The posture information can be expressed using, for example, a quaternion (quaternion) representing the posture of the image pickup apparatus 20 on the global coordinates. The quotation q can be defined below using the four variables q ₀ , q ₁ , q ₂ , q ₃ and the basis vectors i, j, k.

q = q ₀ + q ₁ i + q ₂ j + q ₃ k
i ² = j ² = k ² = ijk = -1
ij = -ji = k, jk = -kj = i, ki = -ik = j

Here, the four variables q ₀ , q ₁ , q ₂ , and q ₃ of the quarternion q can be given by the following equations.
q ₀ = cos (θ / 2)
q ₁ = nx · sin (θ / 2)
q ₂ = ny · sin (θ / 2)
q ₃ = nz · sin (θ / 2)
nx ² + ny ² + nz ² = 1

In the above equation, nx, ny, and nz are direction vectors (nx, ny, nz) on the global coordinates, and θ represents the angle of rotation around the direction vector, as shown in FIG. ..

At this time, the relationship between the posture vector p of the image pickup apparatus 20 on the local coordinates and the posture vector P on the global coordinates is defined by the following equation.
P = R ・ p
Here, R is a rotation matrix of 3 × 3, and is calculated from the quotation q by the following equation.

When the optical axis direction of one image sensor of the image pickup apparatus 20 is set to the y-axis direction of the local coordinate system, the optical axis direction vector of the image pickup apparatus 20 is represented by p = [0,1,0] ^T. At this time, assuming that the attitude quarterion of the image pickup apparatus 20 is qc, the attitude vector P in the global coordinate system of the image pickup apparatus 20 is calculated by the following equation.
P = Rc ・ p

Thereby, the posture angle (tilt and rotation angle) of the local coordinates (x, y, z) of the image pickup apparatus 20 is defined by the global coordinates (X, Y, Z). Various methods can be adopted as a method for calculating the quarternion qc of the image pickup apparatus 20 from the outputs of the angular velocity sensor 42 and the acceleration sensor 44. Further, the posture information may be calculated by other than the quarternion qc.

The posture information (quarterion qc) of the image pickup apparatus 20 calculated by the posture calculation unit 50 is transmitted to the correction amount calculation unit 60. The correction amount calculation unit 60 calculates a correction amount (quartanion qs) for correcting the coordinates of the spherical frame data to global coordinates. The correction amount calculated here completely corrects the inclination of the image pickup device 20 with respect to the Z axis indicating the vertical direction, and with respect to the shaking around the Z axis indicating the vertical direction, that is, in the horizontal plane in the global coordinates. It is a means for creating correction amount data for performing correction excluding minute shaking components.

As shown in FIG. 1, the correction amount calculation unit 60 can be composed of, for example, a delay holding unit 61, a rotation angle extraction unit 62, a low-pass filter 63, a coordinate conversion unit 64, and a coordinate multiplication unit 65.

The posture information input to the correction amount calculation unit 60 is transmitted to the coordinate multiplication unit 65 and also to the delay holding unit 61.

The delay holding unit 61 holds the quarternion qc (t-1) having the pre-sampling time t-1 for the input coordinate information.

Then, the rotation angle extraction unit 62 calculates the rotation angle θ _Z around the Z axis of the global coordinates from the quarternion qc (t-1). For example, the rotation angle θ _Z can be calculated from the Euler angles R _zxy obtained by converting the quaternion qc (t-1) into Euler angles R _zxy .

FIG. 5A shows an example of the waveform W of the calculated rotation angle θ _Z. With reference to FIG. 5A, it can be seen that the rotation angle θ _Z is a waveform in which a high frequency component is superimposed on a low frequency component. Such a waveform W is, for example, when the user attaches the image pickup device 20 to the head and takes a picture while riding a bicycle, or when the traveling direction of the bicycle is changed or the user moves the head in a horizontal plane. Is detected in.

The high-frequency component in the waveform W is a minute vibration in the horizontal plane in the global coordinates of the image pickup apparatus 20, that is, a minute vibration component such as camera shake or vibration that is not intended by the user.

On the other hand, the low frequency component in the waveform W is detected when the direction of the image pickup apparatus 20 itself is changed. That is, in the above example, it means a deliberate change in orientation detected when the user changes the traveling direction of the bicycle or moves the head in a horizontal plane.

In the present invention, in order to maintain a sense of presence, the fluctuation of the low frequency component is left to follow the direction of the image pickup apparatus 20, and the fluctuation of the high frequency component that causes a decrease in visibility is removed to obtain the coordinates around the Z axis. Make corrections.

Specifically, the rotation angle θ _Z calculated by the rotation angle extraction unit 62 is passed through a low-pass filter 63 to remove high-frequency components, and a rotation angle θ _{Z_lpf} with only low-frequency components is created. FIG. 5B shows the waveform of the rotation angle θ _{Z_lpf} obtained by passing through the low-pass filter 63.

Since the high frequency component such as camera shake is removed from the waveform w of the rotation angle θ _{Z_lpf} , the rotation angle θ _{Z_lpf} is the posture amount around the Z axis including only the intentional change in the orientation of the image pickup apparatus 20.

From the obtained rotation angle θ _{Z_lpf} , the coordinate conversion unit 64 calculates an amount of attenuation that attenuates the amount of rotation of the image pickup apparatus 20 around the Z axis. For example, as shown in the following equation, the sign of the rotation angle θ _{Z_lpf} is inverted to calculate the quarter. This quaternion is referred to as a centering quaternion qd.
qd ₀ = cos (-θ _{z_lpf} / 2)
qd ₁ = 0
qd ₂ = 0
qd ₃ = sin (-θ _{z_lpf} / 2)

Then, by applying the centering quarternion qd reflecting the rotation angle θ _{Z_lpf} to the global coordinate correction amount (quarternion qc (t)) which is the attitude information of the image pickup apparatus 20, minute vibrations in the horizontal plane of the global coordinates can be obtained. That is, it is possible to obtain correction amount data of posture information that follows the direction of the image pickup apparatus 20 while suppressing camera shake.

The above calculation is performed by the coordinate multiplication unit 65. The coordinate multiplication unit 65 multiplies the calculated centering quarternion qd by the global coordinate correction amount (quarterion qc (t)) of the current sampling time t, and the posture correction amount quarterion becomes the correction amount data applied to the spherical image. Calculate qs.
qs = qd * qc (t)
Here, * represents quaternion multiplication and is defined by the following equation using each element.
qs ₀ = qd ₀ · qc ₀ -qd ₁ · qc ₁ -qd ₂ · qc ₂ -qd ₃ · qc ₃
qs ₁ = qd ₀ · qc ₁ + qd ₁ · qc ₀ + qd ₂ · qc ₃ -qd ₃ · qc ₂
qs ₂ = qd ₀・ qc ₂ －qd ₁・ qc ₃ + qd ₂・ qc ₀ + qd ₃・ qc ₁
qs ₃ = qd ₀ · qc ₃ + qd ₁ · qc ₂ -qd ₂ · qc ₁ + qd ₃ · qc ₀

The obtained correction amount data (posture correction amount quarternion qs) is transmitted to the association means 70.

The associating means 70 associates the spherical frame data at the current sampling time t created by the image synthesizing unit 30 with the correction amount data. In the present embodiment, the correction amount data is the posture correction amount quarternion qs obtained by correcting the global coordinate correction amount at the current sampling time t by the centering quarternion qd at the presampling time t-1 as described above. The association means 70 can make these associations by writing the correction amount data in the header information of the spherical frame data, for example. The image created by this association is called a spherical image.

As shown in FIG. 1, the obtained spherical image can be transferred to a recording medium such as a memory 80 or an external storage and stored. Further, the spherical image can be directly displayed on the display device 82 as a through image.

Examples of the display device 82 include a digital camera, a smartphone, a portable electronic device such as a PDA (Personal Digital Assistant), and a monitor. In this case, the display device 82 installs software or an application for playback display, reads the correction amount data from the header information of the spherical image, applies the correction amount data to the spherical frame data, and globally. Both the conversion to the coordinates may be performed, the correction in the horizontal plane may be performed, and the spherical image may be displayed.

FIG. 6 shows an example of a spherical image obtained by applying correction amount data to spherical frame data. In FIG. 6, in order to explain the present invention in an easy-to-understand manner, two continuously captured images in time series and spherical frame data are superimposed and displayed.

6 (a) and 6 (b) show image data acquired by the image sensors 22 and 22, respectively. As shown in the figure, it can be seen that the subject 90 is shaken due to camera shake in the two captured images that are continuous in time series.

The image compositing unit 30 performs distortion correction and compositing on these image data, and creates spherical frame data such as an equirectangular image as shown in FIG. 6 (c). In FIG. 6 (c), the spherical frame data is created so that FIG. 6 (a) is at the center. With reference to the figure, it can be seen that the subject 90 in the spherical frame data is also shaken due to the shake of the captured image. Further, the horizontal line 92 of the spherical frame data is tilted due to the tilt of the image pickup device 20.

Then, the correction amount data is applied to the obtained spherical frame data, and the local coordinates of the image pickup apparatus 20 are corrected in the vertical direction based on the global coordinate correction amount (reference FIG. 6 (d)), and the vertical direction is corrected. A spherical image as shown in FIG. 6 (f) is created by performing a conversion centered on the y-axis direction of the image pickup apparatus 20 while removing minute fluctuations in the direction (reference FIG. 6 (e)). Will be done. As shown in FIG. 6 (f), in the obtained spherical image, the inclination of the horizontal line 92 is corrected and the y-axis direction of the image pickup apparatus 20 is corrected with respect to the spherical frame data (FIG. 6 (c)). It can be seen that the rotation correction in the horizontal plane as the center and the fluctuation of the high frequency component in the horizontal plane are corrected.

As a result, the obtained spherical image is not only corrected in the vertical direction, but also follows a large shaking (rotation) with a relatively small frequency in the horizontal plane of the image pickup device 20, so that it is full of realism. Since minute shaking such as camera shake in the horizontal plane is corrected, the image has excellent visibility.

If necessary, as shown by the alternate long and short dash line in FIG. 6 (f), it is also possible to process the created spherical image into an image obtained by cutting out a predetermined range.

In the above, in order to make the conversion procedure easy to understand, FIGS. 6 (c) to 6 (e) are included, but in reality, the correction amount data is added to the spherical frame data shown in FIG. 6 (c). By applying, the spherical image shown in FIG. 6 (f) is directly created.

That is, in the present invention, regardless of the posture of the image pickup means 20, the image pickup means 20 follows the rotation with respect to the Z axis, which is the rotation in the horizontal plane with respect to the global coordinates, but the rotation is not in the horizontal plane. It does not follow the rotation in the axial direction and the Y-axis direction. Therefore, the first image signal and the second image signal obtained by rotating the image pickup means 20 in the X-axis direction and the Y-axis direction by a predetermined angle (first angle) and taking a picture are displayed on the display device. Occasionally, it hardly rotates in the horizontal plane, while the third image signal obtained by rotating the image pickup means 20 by a predetermined angle (first angle) around the Z axis is rotated. It is possible to follow the image and obtain an image having a rotational movement at a substantially predetermined angle. The same applies to the images obtained by cutting out the first to third image signals.

<Other methods for calculating the amount of correction>
In the above embodiment, the attitude quaternion q is once converted to Euler angles R _zxy , the rotation amount θz around the Z axis is calculated, and the centering quaternion qd is calculated again. It doesn't matter. For example, in the component of the posture quaternion qc, the component q _z representing the rotation around the Z axis is q _z = qc ₀ ² + qc ₃ ²
When q _z is close to 1, the rotation around the Z axis becomes dominant. At this time, the centering quarternion qd is changed to qd ₀ = -k · q _z · q ₀ .
qd ₁ = 0
qd ₂ = 0
qd ₃ = (1-qd ₀ ² ) ^1/2
k is the damping coefficient (0 <k <1)
The centering quarter may be calculated as.

The above description is for the purpose of explaining the present invention, and should not be construed as limiting or limiting the scope of the invention described in the claims. In addition, each part of the present invention is not limited to the above embodiment, and it goes without saying that various modifications can be made within the technical scope described in the claims.

10 Spherical photography system 11 Control means 22 Image sensor (imaging means)
30 Image synthesis unit 40 Attitude detection means 42 Angular velocity sensor 44 Accelerometer 50 Attitude calculation unit 60 Correction amount calculation unit 62 Rotation angle extraction unit 63 Low pass filter 64 Coordinate conversion unit 65 Coordinate multiplication unit 70 Association means

Claims (8)

A posture information calculation step of detecting posture information of at least one image pickup means of a plurality of image pickup means and calculating the posture information of the image pickup means.
An omnidirectional image generation step of generating an omnidirectional image in which image data obtained from the plurality of imaging means is coordinate-transformed based on the attitude information calculated in the attitude information calculation step.
With
The spherical image generation step is based on the correction of the inclination of the image data in the vertical direction in the global coordinates and the correction of the fluctuation in the horizontal plane in the global coordinates by removing the high frequency component of the rotation angle in the horizontal plane in the global coordinates. Perform coordinate transformation to generate a spherical image,
A spherical image generation method characterized by this. An image composition step for creating spherical frame data from image data captured by the plurality of imaging means, and
A correction amount calculation step for creating correction amount data for converting the coordinates of the spherical frame data based on the posture information of at least one image pickup means calculated in the posture information calculation step.
An association step of associating the spherical frame data with the correction amount data to obtain a spherical image,
With
The correction amount calculation step corrects the inclination of the whole celestial sphere frame data in the vertical direction in the global coordinates, and the fluctuation in the horizontal plane in the global coordinates removes the high frequency component of the rotation angle in the horizontal plane in the global coordinates. Create correction amount data to perform
The spherical image generation method according to claim 1. The attitude information is between local coordinates including x-axis, y-axis and z-axis orthogonal to each other with respect to the imaging means, and global coordinates including X-axis, Y-axis and vertical Z-axis orthogonal to each other. Information that represents the amount of rotational displacement of
The correction amount calculation step creates a global coordinate correction amount for correcting the coordinates of the whole celestial sphere frame data to the global coordinates based on the attitude information, and creates the global coordinate correction amount as the correction amount data. The amount is a correction amount that corrects the inclination of the whole celestial sphere frame data in the vertical direction and removes only the high-frequency component of the rotation angle in the horizontal plane in the global coordinates for the rotation around the vertical axis. ,
The spherical image generation method according to claim 2. The association step associates the correction amount data as header information of the spherical frame data.
The spherical image generation method according to claim 2 or 3. The image data obtained from the image pickup means is a moving image and is a moving image.
The correction amount calculation step creates correction amount data based on the posture information of the spherical frame data one frame before, which is calculated in the posture information calculation step.
The spherical image generation method according to any one of claims 2 to 4. A posture information calculation step of detecting posture information of at least one image pickup means of a plurality of image pickup means and calculating the posture information of the image pickup means.
An image signal generation display step that generates and displays an image signal based on the image data obtained from the plurality of imaging means and the posture information calculated in the posture information calculation step.
With
The image signal generation display step is
For global coordinates including the X-axis, Y-axis, and vertical Z-axis that are orthogonal to each other
A first image signal generated by rotating the plurality of image pickup means by a first angle around the X axis and taking a picture.
A second image signal generated by rotating the plurality of image pickup means around the Y axis by the first angle and taking a picture.
A third image signal generated by rotating the plurality of image pickup means around the Z axis by the first angle and taking a picture.
When an image signal is generated and displayed from
The first image signal and the second image signal display an image having substantially no rotational movement, and the third image signal performs coordinate conversion by correction for removing a high frequency component of a rotation angle in a horizontal plane in global coordinates. Displaying an image with rotational movement of the first angle.
An omnidirectional image generation and display method characterized by this. An image compositing unit that obtains the image data from a plurality of image pickup means that output the image data, and an image compositing unit.
A posture calculation unit that obtains the posture information from the posture detection means that detects the posture information of at least one of the plurality of image pickup means.
As a control means of the spherical image generation system, which has
A procedure for causing the posture calculation unit to calculate posture information of at least one image pickup means of the plurality of image pickup means, and a procedure.
It is a procedure for generating an all-celestial sphere image in which the image data obtained from the plurality of imaging means is coordinate-converted based on the posture information calculated by the posture calculation unit, and the image data is global. The tilt correction for the vertical direction in the coordinates and the shaking in the horizontal plane in the global coordinates are the procedure to generate the whole celestial sphere image by performing the coordinate conversion by the correction that removes the high frequency component of the rotation angle in the horizontal plane in the global coordinates .
A program of the spherical image generation system that executes. An image compositing unit that obtains the image data from a plurality of image pickup means that output the image data, and an image compositing unit.
A posture calculation unit that obtains the posture information from the posture detection means that detects the posture information of at least one of the plurality of image pickup means.
As a control means of the spherical image generation display system, which has
A procedure for causing the posture calculation unit to calculate posture information of at least one image pickup means of the plurality of image pickup means, and a procedure.
A procedure for generating and displaying an image signal based on the image data obtained from the plurality of imaging means and the posture information calculated by the posture calculation unit.
For global coordinates including the X-axis, Y-axis, and vertical Z-axis that are orthogonal to each other
A first image signal generated by rotating the plurality of image pickup means by a first angle around the X axis and taking a picture.
A second image signal generated by rotating the plurality of image pickup means around the Y axis by the first angle and taking a picture.
A third image signal generated by rotating the plurality of image pickup means around the Z axis by the first angle and taking a picture.
When an image signal is generated and displayed from
The first image signal and the second image signal display an image having substantially no rotational movement, and the third image signal performs coordinate conversion by correction for removing a high frequency component of a rotation angle in a horizontal plane in global coordinates. The procedure for displaying an image with rotational movement at the first angle, and
A program of the spherical image generation display system that executes.

Images