JP2019045679A - Image processing apparatus, control method of image processing apparatus, image display apparatus and program - Google Patents

Abstract

To mitigate a sense of discomfort which is given to a browser due to a difference of height of an imaging position of an image before and after switching when a plurality of images having different height of the imaging position are switched to be displayed.SOLUTION: An image processing apparatus for displaying by switching a plurality of images having wide angles according to an embodiment of the present invention comprises: image acquisition means that acquires a switching source image and a switching destination image; height information acquisition means that acquires height information of imaging positions when the switching source image and the switching destination image are respectively imaged; switching effect selection means that selects switching effect of images on the basis of a difference between the height information of the imaging position of the switching source image and the height information of the imaging position of the switching destination image; and generation means that generates a display image for switching on the basis of the selected switching effect.SELECTED DRAWING: Figure 2

Description

  The present invention relates to image display processing.

  As a method of displaying an image according to the line of sight of a viewer, there is known a method of displaying an image sampled from an omnidirectional image based on the posture of the viewer on a head mounted display (hereinafter referred to as HMD). The omnidirectional image is a 360-degree panoramic image of all directions in all directions, and is acquired by an omnidirectional camera. The omnidirectional camera is also called an omnidirectional camera, and is an imaging device capable of imaging omnidirectional images at one time. Generally, when there are a plurality of omnidirectional images, the HMD displays them as a slide show while sequentially switching them.

  In addition, as a browsing method of still images recorded in a recording medium, an image reproducing apparatus having a slide show function of sequentially displaying the recorded still images, and a function of manually switching the displayed still images according to a viewer's instruction Has been proposed (see, for example, Patent Document 1). Furthermore, an image reproduction system has also been proposed that makes it possible to easily determine that the display image has been switched to the viewer when switching the display image (see, for example, Patent Document 2).

JP 2003-209809 Patent No. 4883318

  However, in the conventional slide show function or the display image switching function, when switching and displaying a plurality of images photographed at different heights (that is, height from the ground or floor) at the photographing position, the images before and after the switching The difference in height caused the viewer to feel uncomfortable.

  The present invention has been made in view of such a problem, and the purpose thereof is to select the height of the photographing position of the images before and after switching when switching and displaying a plurality of images having different heights of the photographing position. It is to reduce the discomfort given to the viewer by the difference.

  An image processing apparatus according to an embodiment of the present invention for switching and displaying a plurality of wide angle of view images includes: an image acquisition unit that acquires an image of a switching source and an image of a switching destination; Height information acquisition means for acquiring height information of the photographing position when photographing each image of the switching destination, height information of the photographing position of the image of the switching source, and height of the photographing position of the image of the switching destination A switching effect selecting means for selecting a switching effect of an image based on a difference between the depth information and a generating means for generating a display image for switching based on the selected switching effect.

  According to the present invention, when switching and displaying a plurality of images having different shooting positions, it is possible to reduce a sense of discomfort given to the viewer due to the difference in the shooting positions of the images before and after switching.

It is a figure showing an example of composition of an image processing device. It is a block diagram showing an example of functional composition of an image processing device. It is a flowchart which shows the whole flow of image processing. It is a flow chart which shows the details of display image generation processing. It is a flow chart which shows the details of change effect selection processing. It is a figure which shows the example of a stereo omnidirectional image. It is a figure which shows the example of a display image. FIG. 6 is a diagram showing image coordinates for representing pixel positions on a display. It is a figure which shows the example of distortion on an image coordinate. It is a figure which shows the example of the relationship between the point on an image coordinate, and a three-dimensional position. It is an image figure of switching effect application processing when slide down or slide up is chosen. It is an image figure of the change effect when fade down or fade up is chosen. It is a flow chart which shows the details of change effect selection processing. It is an image figure of the switching effect when cross fade (dissolve) or fade out fade-in is selected.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following embodiments do not necessarily limit the present invention. Moreover, not all combinations of features described in the following embodiments are essential to the present invention. The same components will be described with the same reference numerals.

First Embodiment
In the present embodiment, a method will be described in which a plurality of omnidirectional images captured at different shooting positions (that is, height from the ground or floor) are switched and displayed by an image display device such as an HMD. When the partial image sampled from the omnidirectional image is displayed according to the gaze direction of the viewer wearing the HMD, the HMD can display an image with a wider angle of view than the viewer's field of view. Images can be viewed as if they were in the position where the omnidirectional image was taken. However, when switching the display image, if the heights of the shooting positions of the images before and after the switching are different, the viewer may feel a sense of strangeness as if the position of oneself or the height of the line of sight suddenly changes. Therefore, in the present embodiment, a process for reducing a sense of incongruity given to a viewer due to the difference between the shooting positions of images before and after switching as described above will be described. Specifically, at the time of switching the images in the slide show, an appropriate switching effect is selected based on the difference in height information indicating the heights of the shooting positions of the switching source image and the switching destination image. By applying the selected switching effect and switching the image, the sense of incongruity given to the viewer is reduced. The switching effect in the present embodiment refers to a special effect in which the display image is switched gradually from the switching source image to the switching destination image. In addition, the omnidirectional image in the present embodiment can be a stereo omnidirectional image that enables stereoscopic vision by an image display device such as an HMD. In addition, the omnidirectional image in the present embodiment can be generated by combining a plurality of images of each azimuth captured from a certain point in the three-dimensional space.

  FIG. 1 shows a configuration example of an image processing apparatus in the present embodiment.

  The CPU 101 uses the RAM 102 as a work memory, executes a program stored in the ROM 103 or the hard disk drive (HDD) 105, and controls each component described later via the system bus 100. Thereby, various processes to be described later can be performed to realize various functions.

  An HDD interface (I / F) 104 is, for example, an interface such as serial ATA (SATA), and connects the HDD 105 and a secondary storage device such as an optical disk drive (not shown) to the system bus 100. The CPU 101 can read data from the HDD 105 and write data to the HDD 105 via the HDD I / F 104. Further, the CPU 101 can expand the data stored in the HDD 105 in the RAM 102, and can store the data expanded in the RAM 102 in the HDD 105. The CPU 101 can execute the data expanded in the RAM 102 as a program.

  The input I / F 106 is, for example, a serial bus interface such as USB or IEEE 1394, and connects the input device 107 such as a keyboard, a mouse, a digital camera, a scanner, and the system bus 100. The CPU 101 can read data from the input device 107 via the input I / F 106.

  An output I / F 108 is an image output interface such as DVI or HDMI (registered trademark), for example, and connects an output device 109 such as an HMD to the system bus 100. The CPU 101 can send image data to the output device 109 via the output I / F 108 for display. Also, the output device 109 can be an HMD or the like that can be worn by a viewer of an image. That is, the image processing apparatus in the present embodiment can be implemented as an image display apparatus capable of controlling the display of the processed image.

  The attitude detection I / F 110 is a serial bus interface such as USB or IEEE 1394, for example, and connects the attitude detection device 111 such as an acceleration sensor or an angular velocity sensor to the system bus 100. The posture detection device 111 is attached to an image display surface (that is, a display) of the output device 109, and detects posture information of a viewer wearing the output device 109. The CPU 101 can read the posture information of the browser from the posture detection device 111 via the posture detection I / F 110. Posture information is data indicating the orientation of the output device 109 (that is, the orientation of the viewer's face) detected by the posture detection device 111, as described later. Posture information can also be input using a mouse, a keyboard or the like.

  The overall flow of image processing performed by the image processing apparatus according to this embodiment will be described below with reference to FIGS. 2 and 3.

  FIG. 2 is a block diagram showing an example of the functional configuration of the image processing apparatus according to this embodiment. The image processing apparatus according to the present embodiment includes an image acquisition unit 201, a height information acquisition unit 202, a switching effect selection unit 203, a switching effect application unit 204, an image data output unit 205, a posture information acquisition unit 206, and a display parameter acquisition unit 207. And a display image generation unit 208.

  The CPU 101 reads a program stored in the ROM 103 or the HDD 105 and executes the program by using the RAM 102 as a work area, thereby playing a role as each functional block shown in FIG. It is not necessary for the CPU 101 to play the role of all the functional blocks, and dedicated processing circuits (for example, ASICs) corresponding to the respective functional blocks may be provided to realize the respective functions.

  FIG. 3 is a flowchart showing an overall flow of image processing performed by the image processing apparatus of the present embodiment. Each step of the flowchart is performed by each functional block of the image processing apparatus shown in FIG.

  In step S301, the display parameter acquisition unit 207 acquires, from the input device 107, display parameters which are parameters necessary for displaying the omnidirectional stereo image on the output device 109 (that is, the HMD). The display parameters may be acquired once stored in the RAM 102 or the HDD 105. The display parameters include the focal length of the eyepiece forming the HMD, the distortion parameter, the center position of the eyepiece on the display, the display resolution, the display size and the like.

  In step S302, the posture information acquisition unit 206 acquires posture information from the posture detection device 111 or the input device 107. Posture information may be acquired once stored in the RAM 102 or the HDD 105. Posture information is data representing the orientation of the HMD, and can also be regarded as the orientation of the face of the viewer. In the present embodiment, posture information is represented using a 3 × 3 rotation matrix R and roll, pitch, and yaw expressions. The posture information may be expressed using quaternions (quaternions).

In step S303, the image acquisition unit 201 acquires omnidirectional image data (in particular, stereo omnidirectional image data in the present embodiment) corresponding to each of the slide show switching source image and the switching destination image from the HDD 105 or the input device 107. Do. The omnidirectional image data corresponding to the switching source image and the switching destination image are also referred to as a switching source omnidirectional image and a switching destination omnidirectional image, respectively. FIG. 6 shows an example of a stereo omnidirectional image. The stereo omnidirectional image data is image data to be displayed on the left eye side of the viewer (all-sky image for left eye) I _L and image data to be displayed on the right eye side of the viewer (all-sky image for right eye) I _R including. That is, the HMD in the present embodiment enables stereoscopic viewing of the viewer using two images for left eye and right eye having parallax. The stereo omnidirectional image is image data in which the color of light in all directions is stored. The horizontal axis (horizontal direction) of the image data is an azimuth angle, and the range is [0 to 2π]. Further, the vertical axis (vertical direction) of the image data is an elevation angle, and the range is [−π / 2 to π / 2].

In step S304, the display image generation unit 208 generates left and right display image data to be displayed on the output device 109 from the display parameter, the stereo omnidirectional image data, and the posture information. FIG. 7 shows an example of a display image. Display image includes a left eye display image J _L to be presented to the left eye of the viewer, the right eye display image J _R to be presented to the right eye of the viewer. Each image is corrected for eyepiece distortion so that the viewer sees without distortion when viewed through the HMD eyepiece. The display image generation processing is performed on each of the omnidirectional image data of the switching source and the omnidirectional image data of the switching destination, and the switching source display image data and the switching destination display image data are generated. Details of the display image generation process will be described later.

  In step S305, the image data output unit 205 outputs the switching source display image data as the first display image to the output device 109. The output device 109 displays the switching source display image data output by the image data output unit 205 on the display.

In step S306, the image data output unit 205 determines whether to switch the display image to the next image (that is, the switching destination display image). For example, when the display time of the switching source image exceeds the preset time _TINT , it may be determined that the image data output unit 205 switches the display image. If it is determined that the display image is to be switched, the process proceeds to step S307. If it is determined that the display image is not to be switched, the process ends.

In step S307, the height information acquisition unit 202 acquires height information from the stereo omnidirectional image data corresponding to each of the switching source image and the switching destination image acquired in step S303. The height information is information indicating the height of the imaging position when the stereo omnidirectional image corresponding to each of the switching source image and the switching destination image is captured. The height information is acquired, for example, from EXIF information incidental to stereo omnidirectional image data. In this case, the height information may be included using an EXIF expansion tag. Alternatively, when the user inputs height information using the input device 107, the height information acquisition unit 202 may acquire it. That is, the stereo omnidirectional image data in the present embodiment includes, as additional information, height information from the ground or floor of the imaging device that has captured the stereo omnidirectional image. Further, the imaging device for capturing a stereo omnidirectional image is configured to include a height sensor using a laser or the like, and the height sensor may be used to measure the height from the ground or the floor. Good. Here, switch the height at the time of photographing the original image and H _1, the height at the time of photographing the switching destination image and H _2.

In step S308, the height information acquisition unit 202 calculates the difference value H _DIFF of the acquired height information according to the following equation (1).
H _DIFF = H _2- H ₁ (1)

In step S309, the switching effect selection unit 203 selects a switching effect at the time of image switching according to the difference value _HDIFF of the height information calculated in the previous step. Details of the switching effect selection process will be described later.

In step S310, the switching effect application unit 204 applies the switching effect to the switching source display image data and the switching destination display image data according to the switching effect selected in the previous step, and generates display image data at the time of switching (Switching effect application processing). In the present embodiment, the switching effect is a special effect that is switched gradually from the switching source image to the switching destination image. Therefore, the switching display image data generated here consists of a plurality of frames displayed within a predetermined time. As described below, the switching effect application unit 204 displays the switching effect while adjusting the application degree of the switching effect so that the switching effect is smoothly applied temporally according to the preset switching time T _C. Generate image data.

FIG. 11 is an image diagram of switching effect application processing when slide down or slide up is selected as the switching effect. The slide down is a switching effect in which the switching destination display image is displayed so as to move from the top to the bottom of the screen. Therefore, in the switching effect application process, the switching display image which causes such a moving effect by continuously displaying is generated by combining the switching source display image data and the switching destination display image data. That is, the switching display image is image data composed of a plurality of frames representing a state in which switching is performed in stages. In a plurality of frames, switching lines for switching images transition from top to bottom. The switching destination display image data is synthesized above the switching line, and the switching source display image data is synthesized below the switching line. The frame displayed at the start of the switching time has the switching line at the upper side, the ratio indicated by the switching source display image data is large, and the frame displayed at the end has the switching line at the lower side. The proportion will increase. On the other hand, the slide up is a switching effect in which the switching destination display image is displayed so as to move from the bottom of the screen to the top. Therefore, in the switching effect application process, the switching display image which causes such a moving effect by continuously displaying is generated by combining the switching source display image data and the switching destination display image data. In this case, the switching line is positioned higher as the switching time approaches the end of the frame. The vertical axis in the figure indicates a time axis. If switching from the switching source display image starts at time 0, it means that switching to the switching destination display image is completed at the switching time T _c . For example, the switching effect application unit 204 calculates the movement amount of the switching destination display image so as to have a linear relationship with time, and combines the switching source display image data and the switching destination display image data based on this. Thus, display image data at the time of switching is generated.

  In step S311, the image data output unit 205 outputs the display image data at the time of switching to the output device 109 and the switching destination display image data. The output device 109 sequentially displays the switching display image data and the switching destination display image data output by the image data output unit 205. That is, according to the image processing apparatus of the present embodiment, it is possible to gradually change, switch and display the switching source image data and the switching destination image data in accordance with the selected switching effect.

<Display image generation process>
Hereinafter, the details of the display image generation process in step S304 described above will be described.

  The display image generation unit 208 generates the left and right display images to be displayed on the HMD (that is, the output device 109) by sampling from the stereo omnidirectional image according to the face direction of the viewer wearing the HMD. Do. In order to sample the pixel values of the display image from the stereo omnidirectional image, the display image generation unit 208 generates a ray of light in any direction at each pixel position on the HMD display based on the posture information acquired by the posture information acquisition unit 206. Calculate whether to display. The direction of a ray is represented by an azimuth angle θ that represents a horizontal angle and an elevation angle φ that represents a vertical direction. The direction (θ, φ) of the light beam is calculated with (x, y) as the pixel position in the coordinate system with the lens center of the eyepiece as the origin, and the pixel value of the display image is determined from that direction.

  Hereinafter, the details of the display image generation process performed by the display image generation unit 208 will be described using the flowchart of FIG. 4. The processing represented by the illustrated flowchart is performed on each of the left and right display images.

  In step S401, the display image generation unit 208 acquires a distortion factor due to lens aberration at the pixel position (x, y). FIG. 8 shows image coordinates for representing a pixel position on the display. As illustrated, the pixel position (x, y) on the display is represented by the xy coordinate system in each of the left and right display images 801 and 802. FIG. 9 shows an example of distortion on image coordinates. Since the distortion rate is determined by the distance r from the lens center and the optical characteristics of the lens, the display image generation unit 208 uses a look-up table of distortion rates with respect to the distance r from the lens center prepared in advance according to the lens used. Get the distortion rate. If the distortion factor differs for each wavelength depending on the characteristics of the lens, look-up tables are prepared for each of the R, G, and B wavelengths in the display. Then, distortion rates for each of R, G, and B are acquired, and processing is performed using different distortion rates for each of R, G, and B in subsequent steps. When the distance r from the lens center at the pixel position (x, y) is far and out of the viewable range of the eyepiece, the pixel value of the display image may be blackened to end the processing of the pixel.

In step S402, the display image generation unit 208 calculates a pixel position at which the pixel position (x, y) is actually viewed by lens distortion (that is, the pixel position after distortion). The pixel position (x _d , y _d ) after distortion can be calculated by integrating the distortion factor d to the x coordinate and y coordinate of the pixel position as shown in the following equations (2) and (3).
x _d = x × d (2)
y _d = y × d (3)

In step S403, the display image generation unit 208 converts the distorted pixel position (x _d , y _d ) into a three-dimensional position (X, Y, Z) of a coordinate system having the position of the eye as the origin. FIG. 10 shows an example of the relationship between the point (x, y) on the image coordinates and the three-dimensional position (X, Y, Z). Assuming that the orientation information of the viewer is a 3 × 3 rotation matrix R and the focal length represented in pixel units is f, the three-dimensional position (X, Y, Z) can be calculated by the following equation (4).
[X Y Z] ^t = R × [x _d y _d f] ^t (4)

In step S404, the display image generation unit 208 calculates the azimuth angle θ and the elevation angle φ from the three-dimensional position (X, Y, Z) of the pixel. Assuming that the distance from the origin (ie, the position of the eye) to the three-dimensional position (X, Y, Z) is L = (X ² + Y ² + Z ² ) ^1/2 , the azimuth angle θ and elevation angle φ are respectively It can be calculated by the equations (5) and (6).
φ = asin (Y / L) (5)
θ = asin (Z / L / cosφ) (6)

  In step S405, the display image generation unit 208 acquires pixel values in the direction (θ, φ) of the light beam from the corresponding omnidirectional image, and stores the pixel values as display pixel values. When the absolute value of the roll angle α of the posture information is 90 degrees or less (that is, | α | ≦ 90 °), when generating a display image for the left eye, sampling is performed from the omnidirectional image for the left eye. Moreover, when producing | generating the display image for right eyes, it samples from the omnidirectional image for right eyes. When the absolute value of the roll angle α in the posture information is larger than 90 degrees (ie, | α |> 90 °), the left and right display images are reversed because the positional relationship between the eyes is reversed as in the case of | α | Swap. That is, sampling of the display image for the left eye is performed from the omnidirectional image for the right eye, and sampling of the display image for the right eye is performed from the omnidirectional image for the left eye. The acquired pixel value is stored as a pixel value at a position moved by (x, y) from the origin on the display image. When sampling pixel values from an omnidirectional image, the nearest pixel values may be acquired, or interpolation may be performed using neighboring pixel values such as bilinear interpolation or bicubic interpolation.

<Switching effect selection processing>
Hereinafter, the details of the switching effect selection process in step S309 described above will be described using the flowchart of FIG. Each step of the flowchart is performed by the switching effect selection unit 203.

In step S501, the switching effect selection unit 203 determines whether the height of the switching destination image is higher than that of the switching source image. Specifically, if the difference value _HDIFF calculated in step S308 is 0 or more, it is determined that the height of the switching destination image is higher, and if the difference value _HDIFF is smaller than 0, the switching destination image is Judge that the height is lower. If it is determined that the height of the switching destination image is higher, the process proceeds to step S502. If not, the process proceeds to step S503.

  In step S502, the switching effect selection unit 203 slides down from among a plurality of switching effects stored in advance as a switching effect with a moving effect displayed so that the switching destination display image moves downward from the top of the screen. Choose Note that the switching effect to be selected is not limited to the slide down, as long as the switching destination display image is displayed so as to move from the top to the bottom of the screen. The HMD according to the present embodiment moves the switching destination display image from the top to the bottom of the screen when the height of the shooting position of the switching destination display image is higher than the height of the shooting position of the switching source display image. indicate. As a result, it is possible to reduce the sense of incongruity, such as a sudden standing position or line of sight given to the viewer at the time of switching the image.

  In step S503, the switching effect selection unit 203 slides up from among a plurality of switching effects stored in advance as a switching effect with a moving effect displayed so that the switching destination display image moves upward from the bottom of the screen. Choose Note that the switching effect to be selected is not limited to slide-up, as long as the switching destination display image is displayed so as to move from the bottom to the top of the screen. The HMD according to the present embodiment moves the switching destination display image upward from the bottom of the screen when the height of the shooting position of the switching destination display image is lower than the height of the shooting position of the switching source display image. indicate. As a result, it is possible to reduce the sense of incongruity that the standing position or the sight line is suddenly given to the viewer at the time of switching the image.

  In such a case, since it is sufficient to use a combination of switching effects with the display image moving from the top to the bottom of the screen and from the bottom to the top of the screen, for example, using a combination of fade down and fade up. The same effect can be obtained.

FIG. 12 is a diagram showing an image of the switching effect application process when fade down or fade up is selected as the switching effect. Fade down and fade up are examples of the switching effect that can be used as an alternative to slide down and slide up described above with reference to FIG. 11, and the display image to be switched is from top to bottom or from bottom to top of the screen. Appears to move. The vertical axis in the figure indicates a time axis. At time 0, the entire switching source display image is displayed. If switching is started from time 0, it means that switching to the switching destination display image is completed at time _Tc . As illustrated, fade-down is a switching effect such that the switching source display image gradually disappears from the top and the switching destination display image gradually appears from the top. That is, fade-down produces a moving effect such that the switching destination display image is displayed moving from the top to the bottom of the screen. On the other hand, fade-up is a switching effect such that the switching source display image gradually disappears from its lower portion and the switching destination display image gradually appears from its lower portion. That is, fade-up produces a moving effect such that the switching destination display image is displayed moving from the bottom to the top of the screen. As described above, even if fade-down and fade-up are used instead of slide-down and slide-up, it is possible to reduce the discomfort given to the viewer when switching the image.

  As described above, in the present embodiment, when displaying omnidirectional images captured at different heights, for example, as a slide show, the difference between the height information of each of the switching source image and the switching destination image Determine the switching effect based on Then, by applying the determined switching effect to switch the image, the sense of incongruity given to the viewer is reduced due to the difference in height of the image before and after the switching.

  In the present embodiment, an image obtained by sampling an omnidirectional image is displayed on the HMD, but the present invention is not limited to the omnidirectional image. For example, it may be a hemispherical image or a panoramic image. If an image with an angle of view wider than the field of view of a person is displayed, the effect of this embodiment can be obtained.

Second Embodiment
In the present embodiment, the switching effect is selected based on the posture information of the browser before the difference in height of the image before and after switching is determined to select the switching effect. In particular, when switching the image while the viewer is facing upward or downward with respect to the horizontal direction, by selecting and applying the switching effect without the moving effect, the discomfort caused to the viewer is alleviated. .

  The overall flow of the image processing in the present embodiment is the same as the flowchart shown in FIG. 3 of the first embodiment described above, and therefore the description thereof is omitted. In the present embodiment, since the details of the switching effect selection processing in step S309 described above with reference to FIG. 5 are different, the details of the switching effect selection processing in the present embodiment will be described below.

  FIG. 13 is a flowchart showing details of the switching effect selection process in the present embodiment. Each step of the flowchart is performed by the switching effect selection unit 203.

  In step S1301, the switching effect selection unit 203 determines whether the viewer is facing upward or downward with respect to the horizontal direction. Specifically, in the posture information acquired in step S302, whether the absolute value of the pitch angle β corresponding to the vertical direction of the viewer's face is greater than 45 degrees (ie, | β |> 45 °) Determine If the absolute value of the pitch angle β is larger than 45 degrees, the process proceeds to step S1302, and if the absolute value of the pitch angle β is 45 degrees or less, the process proceeds to step S1303. It is not necessary to determine the pitch angle β based on 45 degrees, as long as the vertical orientation (ie elevation angle) of the viewer's face can be determined to be substantially horizontal. Further, in consideration of the individual difference of the viewer using the HMD, a different value may be set as the threshold value for each viewer.

  In step S1302, the switching effect selection unit 203 selects a crossfade (also referred to as dissolve) from among a plurality of switching effects stored in advance as a switching effect without moving effect. The switching effect to be selected is not limited to the cross fade but may be fade out fade in or the like.

  FIG. 14 is an illustration of a switching effect application process when cross fade (dissolve) or fade out fade in is selected as the switching effect. As illustrated, the fade-out fade-in causes the switching source display image to gradually disappear (fade out) first, and after the switching source display image disappears, the switching destination display image gradually appears (fade out). In) switching effect. That is, fade out fade in does not have a moving effect when switching images. The cross fade is a switching effect such that the switching source display image gradually gradually disappears before the switching source display image gradually disappears while the switching source display image gradually disappears. In the cross fade, fade out and fade in cross (cross). Crossfading also has no moving effect when switching images. As described above, in the present embodiment, by selecting and applying the switching effect not accompanied by the moving effect, it is possible to reduce the sense of discomfort given to the viewer at the time of switching the image.

  In steps S1303 to S1305, the switching effect selection unit 203 selects the switching effect of slide down or slide up based on the difference between the height of the switching source image and the height of the switching destination image. The processes in steps S1303 to S1305 are the same as the processes in steps S501 to S503 of FIG. 5 described above, and thus detailed description will be omitted. Also in the present embodiment, the switching effect to be selected is not limited to slide-down and slide-up. For example, the above-described fade-down and fade-up may be performed.

  As described above, in this embodiment, particularly when switching the image while the viewer is facing upward or downward, the switching of the image is performed by selecting and applying the switching effect without the moving effect. Reduce the sense of discomfort to the viewer at times.

In addition, when the difference between the height of the switching source image and the height of the switching destination image is small, that is, when the difference value _HDIFF is close to 0, a switching effect without a moving effect such as cross fade or fade out fade in is applied. You may For example, the switching effect selection section 203 compares the differential value H _DIFF advance and stored predetermined threshold, if the difference value H _DIFF is less than a predetermined threshold value, so as to select a switching effect without transfer effects May be The same applies to the first embodiment.

Furthermore, when the slide show is started, since there is no corresponding switching source display image for the first image to be displayed first, the difference value _HDIFF of the height information can not be obtained. In this case, the HMD (or the image processing apparatus) may acquire the heights of the eyes of the viewer and replace them with the height of the switching source display image to perform each processing. By doing so, the same effect can be obtained. The height of the viewer's eyes may be obtained, for example, by user input via an input device, or height from the ground using a height sensor using a laser or the like provided in the HMD (or an image processing apparatus). May be detected and acquired.

Other Embodiments
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Processing is also feasible. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

201 image acquisition unit 202 height information acquisition unit 203 switching effect selection unit 204 switching effect application unit 205 image data output unit 206 posture information acquisition unit 207 display parameter acquisition unit 208 display image generation unit

Claims (14)

An image processing apparatus for switching and displaying a plurality of wide angle of view images,
An image acquisition unit that acquires a switching source image and a switching destination image;
Height information acquisition means for acquiring height information of the shooting position when each of the switching source image and the switching destination image is shot;
Switching effect selecting means for selecting an image switching effect based on a difference between height information of the photographing position of the switching source image and height information of the photographing position of the switching destination image;
An image processing apparatus comprising: generation means for generating a display image for switching based on the selected switching effect.   The switching effect selecting means selects a switching effect of switching so that the display image of the switching destination moves downward from the top of the screen when the height at which the image of the switching destination is photographed is higher, and the switching is performed 2. The switching effect according to claim 1, wherein a switching image is switched so that the display image of the switching destination is moved upward from the bottom of the screen when the height at which the previous image was captured is lower. Image processing device.   The image processing apparatus according to claim 2, wherein the switching effect of switching the display image of the switching destination to move from the top to the bottom of the screen is slide down or fade down.   The image processing apparatus according to claim 2, wherein the switching effect of switching the display image of the switching destination to move upward from the bottom of the screen is slide-up or fade-up. It further comprises posture information acquisition means for acquiring posture information of the viewer including the angle corresponding to the vertical direction of the viewer's face,
5. The switching effect selecting means according to any one of claims 1 to 4, wherein, when the angle is not substantially horizontal, the switching effect selecting means selects a switching effect without moving effect switching so as to move the display image of the switching destination. An image processing apparatus according to item 1.   6. The image processing apparatus according to claim 5, wherein the switching effect selection unit selects the switching effect without the movement effect when the absolute value of the angle is larger than 45 degrees.   The switching effect selecting means selects a switching effect not accompanied by a moving effect such that the display image of the switching destination is moved when the difference of the height information is smaller than a predetermined threshold. An image processing apparatus according to any one of items 1 to 6.   The switching effect without the moving effect is that the display image of the switching source gradually fades out, and after the display image of the switching source disappears, it is a fade-out fade-in that the display image of the switching destination gradually appears. The image processing apparatus according to any one of claims 5 to 7, characterized in that:   The switching effect without the movement effect is a cross fade or dissolve in which the display image of the switching destination gradually appears before the display image of the switching source disappears gradually and the display image of the switching source gradually disappears. The image processing apparatus according to any one of claims 5 to 7, characterized in that It further comprises detection means for detecting the eye height of the viewer,
10. The display control method according to claim 1, wherein the height information acquiring unit acquires the height of the detected eye and height information of the photographing position of the first image when displaying the first display image. The image processing apparatus according to any one of the above.   The image processing apparatus according to any one of claims 1 to 10, wherein each of the switching source image and the switching destination image is an image generated from an omnidirectional image. Image generation means for generating a display image from each of a plurality of wide angle of view images;
An image display apparatus comprising: display means for switching the generated display image and displaying it on the screen,
The display means switches the display image downward from above the screen when the height at which the switching destination image is captured is higher than the height at which the switching source image is captured, and the switching destination image An image display apparatus, wherein the display image is switched from the bottom to the top of the screen when the height at which the image is taken is lower than the height at which the image of the switching source is taken. A control method of an image processing apparatus for switching and displaying a plurality of wide angle of view images,
Acquiring a switching source image and a switching destination image;
Acquiring height information of shooting positions when shooting each of the switching source image and the switching destination image;
Selecting an image switching effect based on a difference between height information of the shooting position of the switching source image and height information of the shooting position of the switching destination image;
And generating a display image for switching based on the selected switching effect.   A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 11.

Images