JP2022059098A - Information processing device, information processing method, and program - Google Patents

Abstract

To make it possible not to lose sight of a point worthy of note and to reduce a body burden.SOLUTION: A video processing unit arranges an attractive region in a frame after scene switching of a wide-angle video of a size exceeding a user's field of vision at a presentation position where an amount of torsion of a user's body becomes smaller relative to a direction of a user's line of sight before the scene switching. The present disclosure can be applied to, for example, an information processing device with which a prescribed process is carried out on an omnidirectional video or a full-surrounding video.SELECTED DRAWING: Figure 6

Description

The present disclosure relates to an information processing device, an information processing method, and a program, and more particularly to an information processing device, an information processing method, and a program capable of reducing a physical load when viewing a video.

Conventionally, VR (Virtual Reality) content that presents spherical images taken in all directions such as horizontal and vertical directions and all-around images taken in 360 degrees in the horizontal direction is an image that surrounds the viewer. By presenting, a highly immersive viewing experience is realized.

In Patent Document 1, when the head is moved to view an image displayed in an area outside the viewer's field of view in the spherical image, the reproduction speed of the image is reduced, which is desired by the viewer. A technique for appropriately viewing a video is disclosed.

JP-A-2017-126899

By the way, when the scene is switched in the spherical image or the surrounding image, the presentation position of the highlight changes, so that the viewer loses the highlight or has to continue watching with the neck twisted. was there.

The present disclosure has been made in view of such a situation, and is intended to enable the user to not lose sight of the highlights when viewing a wide-angle image and to reduce the load on the body.

The information processing apparatus of the present disclosure includes an image processing unit that performs predetermined processing on a wide-angle image having a size exceeding the user's field of view, and the image processing unit sets an attractive region in a frame after the scene of the wide-angle image is switched. This is an information processing device arranged at a presentation position where the amount of twist of the user's body is smaller than the line-of-sight direction of the user before the scene is switched.

In the information processing method of the present disclosure, the information processing apparatus applies a predetermined process to a wide-angle image having a size exceeding the user's field of view, and the attraction region in the frame after the scene of the wide-angle image is switched is set to the attraction region before the scene is switched. This is an information processing method in which the information processing method is arranged at a presentation position where the amount of twist of the user's neck or waist is smaller than the user's line-of-sight direction.

In the program of the present disclosure, a computer is subjected to predetermined processing on a wide-angle image having a size exceeding the user's field of view, and an attraction region in a frame after the scene of the wide-angle image is switched is set to the line-of-sight direction of the user before the scene is switched. This is a program for executing a process of arranging the user at a presentation position where the twist amount of the user's neck or waist is smaller.

In the present disclosure, a predetermined process is applied to a wide-angle image having a size exceeding the user's field of view, and the attraction region in the frame after the scene change of the wide-angle image is set with respect to the line-of-sight direction of the user before the scene change. It is arranged in a presentation position where the amount of twist of the user's neck or waist is smaller.

It is a figure explaining the presentation of a wide-angle image. It is a figure explaining the image presentation at the time of a scene change. It is a figure explaining the image presentation at the time of a scene change. It is a block diagram which shows the configuration example of a video presentation system. It is a block diagram which shows the hardware configuration example of an information processing apparatus. It is a block diagram which shows the structural example of a video processing part. It is a flowchart explaining the flow of a video reproduction process. It is a flowchart explaining the flow of the image rotation processing. It is a figure which shows the example of the detection of a scene change. It is a figure which shows the example of the detection of the twist amount of a neck. It is a figure which shows another example of the detection of the twist amount of a neck. It is a figure explaining the rotation of a wide-angle image. It is a block diagram which shows the other configuration example of the image processing part. It is a flowchart explaining the flow of a video display process. It is a block diagram which shows the other configuration example of a video presentation system. It is a block diagram which shows the configuration example of a video editing system. It is a flowchart explaining the flow of a video editing process.

Hereinafter, embodiments for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The explanation will be given in the following order.

1. 1. Wide-angle video presentation and scene switching 2. Configuration and operation of video presentation system (example of video presentation to HMD)
3. 3. Other configurations of video presentation system (example of video presentation on screen)
4. Application to video editing system 5. others

<1. Wide-angle video presentation and scene switching>
The video presentation system to which the technology according to the present disclosure (the present technology) applies the video so that the highlights are arranged according to the posture of the user who is the viewer in the wide-angle video such as the spherical image and the omnidirectional image. Rotate and present.

FIG. 1 is a diagram illustrating the presentation of a wide-angle image.

FIG. 1 shows a user 10 viewing a wide-angle image presented on a cylindrical omnidirectional screen 20 surrounding 360 degrees in the horizontal direction.

The wide-angle image is an image having a size exceeding the user's field of view, and is, for example, an image taken with a viewing angle of 180 degrees or more in the horizontal direction. FIG. 1 shows an example in which an omnidirectional image taken 360 degrees in the horizontal direction is presented on the omnidirectional screen 20 as a wide-angle image, but an omnidirectional image taken in all directions such as the horizontal direction and the vertical direction is shown. The image may be presented on a dome-shaped spherical screen. Further, the wide-angle image may be presented on an HMD (Head-Mounted Display) capable of presenting VR content.

As shown on the left side of FIG. 1, at the start of video viewing, a person 21 who is the highlight of the video is presented in front of the user 10 on the omnidirectional screen 20. Since the person 21 which is the highlight is presented in a different arrangement for each scene in the video, depending on the scene, the user 10 greatly twists his / her neck to the right to watch the video, as shown on the right side of FIG. It will be.

FIG. 2 shows an example of video presentation when the scene is switched while the user 10 is viewing the video with his / her neck twisted to the right. In the example of FIG. 2, the highlight after the scene change is changed from the person 21 to the ball 22.

As shown in the upper right part of FIG. 2, when the ball 22 which is the highlight after the scene change is presented in front of the user 10 as in the case of starting the video viewing, the ball 22 is out of the user 10's view. There is a risk that the user 10 will lose sight of the highlights.

On the other hand, when the ball 22, which is the highlight after the scene change, is presented at the same position as the person 21 before the scene change, as shown in the lower right part of FIG. 2, the viewer is viewed with his neck twisted. Must be continued, which puts a load on the neck of the user 10.

Therefore, in the video presentation system to which the present technology is applied, as shown in FIG. 3, the ball 22, which is the highlight after the scene change, is near the middle between the front of the user 10 and the position of the person 21 before the scene change. To be presented to.

As a result, the user 10 can reduce the load on the body without losing the highlight when viewing the wide-angle image.

Hereinafter, a configuration for realizing the above-mentioned video presentation will be described.

<2. Configuration and operation of video presentation system>
(Configuration example of video presentation system)
FIG. 4 is a block diagram showing a configuration example of a video presentation system that realizes the video presentation described above.

The video presentation system of FIG. 4 is composed of an information processing device 100 and an HMD 200.

The information processing apparatus 100 is configured as, for example, a PC (Personal Computer) or the like. The information processing apparatus 100 performs predetermined processing on the wide-angle video data (video data) obtained by shooting, and supplies the data to the HMD 200.

The HMD 200 is configured as a display device for VR mounted on the head of the user 10, and includes a non-transparent display unit. A wide-angle image that realizes a virtual space is presented on the display unit.

The information processing device 100 includes a video processing unit 110 and a display control unit 120.

The image processing unit 110 performs predetermined processing on the wide-angle image (video data) and supplies it to the display control unit 120. The display control unit 120 controls the display of the wide-angle image processed by the image processing unit 110 on the HMD 200. The display control unit 120 may be provided on the HMD200 side.

(Configuration example of information processing device)
FIG. 5 is a block diagram showing a hardware configuration example of the information processing apparatus 100.

As shown in FIG. 5, the CPU (Central Processor Unit) 151, the ROM (Read Only Memory) 152, and the RAM (Random Access Memory) 153 are connected to each other via the bus 154.

The CPU 151 realizes various functions (specifically, the functions of the video processing unit 110 and the display control unit 120) included in the information processing apparatus 100 according to the program stored in the ROM 152 or the program loaded in the RAM 153. Perform the processing for. The RAM 153 also appropriately stores data and the like necessary for the CPU 151 to execute various processes.

An input / output interface 155 is also connected to the bus 154.

An input unit 156, an output unit 157, a storage unit 158, and a communication unit 159 are connected to the input / output interface 155.

The input unit 156 is composed of keys, buttons, a touch panel, a microphone, and the like, and the output unit 157 is composed of a display, a speaker, and the like.

The storage unit 158 is configured by a non-volatile memory or the like. The storage unit 158 stores various data such as a video presented in the HMD 200 in addition to the program executed by the CPU 151.

The communication unit 159 is configured by a network interface or the like. The communication unit 159 performs wireless or wired communication with the HMD 200 or an external device.

A drive 160 is also connected to the input / output interface 155 as needed, and a removable media 161 composed of a semiconductor memory or the like is appropriately mounted. The program read from the removable media 161 is installed in the storage unit 158 as needed.

(Configuration example of video processing unit)
Next, a configuration example of the video processing unit 110 of FIG. 4 will be described with reference to FIG.

The image processing unit 110 includes a switching detection unit 171, an attraction region detection unit 172, a twist amount detection unit 173, and a rotation processing unit 174.

The change detection unit 171 detects the change of the scene in the wide-angle image based on the image data of the wide-angle image. The information indicating the detected scene change is supplied to the rotation processing unit 174.

The attracting region detection unit 172 detects an attracting region, which is a region having a relatively high attracting property in the wide-angle video, based on the video data of the wide-angle video. The attraction area is a so-called highlight that the viewer wants to pay the most attention to, and is a person area detected in the wide-angle image, an area designated in advance by the photographer or the image creator of the wide-angle image, and the like. Information indicating the position of the attraction region in the wide-angle image is supplied to the rotation processing unit 174.

The twist amount detection unit 173 detects the twist amount of the body of the user 10, specifically, the twist amount of the neck, based on the rotation amount of the HMD 200. The detected twist amount of the neck is supplied to the rotation processing unit 174.

Based on the information indicating the scene change, the information indicating the position of the attraction area, and the amount of twisting of the neck, the rotation processing unit 174 sets the attraction area after the scene change with respect to the line-of-sight direction of the user 10 before the scene change. Place 10 in a presentation position where the amount of twist in the neck is smaller. Specifically, the rotation processing unit 174 rotates the wide-angle image in the yaw direction so that the attraction region (highlight) after the scene is switched is arranged at the presentation position where the twist amount of the user's neck is smaller. The rotated video data is supplied to the display control unit 120 (FIG. 4). The display control unit 120 causes the HMD 200 to display a wide-angle image after rotation.

(Flow of video playback processing)
FIG. 7 is a flowchart illustrating a flow of video reproduction processing in the above-mentioned video presentation system.

When the user wearing the HMD 200 instructs the information processing apparatus 100 to reproduce the video, for example, an application for reproducing the video, the information processing apparatus 100 starts reproducing the video in step S11.

In step S12, the video processing unit 110 performs video rotation processing on the wide-angle video to be reproduced.

In step S13, the display control unit 120 causes the HMD 200 to display a wide-angle image that has undergone image rotation processing.

In the process of FIG. 7, the video rotation process is executed in real time when the video is reproduced.

(Flow of video rotation processing)
Next, the details of the flow of the video rotation processing by the video processing unit 110 will be described with reference to the flowchart of FIG.

In step S31, the video processing unit 110 reads the video data of the wide-angle video frame by frame.

In step S32, the switching detection unit 171 determines whether or not the scene switching is detected in the next frame of the read current frame.

As for scene switching, a timing that does not give a sense of discomfort even when the wide-angle image is rotated in the yaw direction is detected.

For example, as shown in FIG. 9, it is assumed that the shooting is stopped during the shooting of the video and then the shooting of the video is resumed. If the image changes significantly before and after the shooting is stopped, many viewers will not notice it even if the presented image is rotated in the yaw direction at the timing when the shooting is stopped. Therefore, by saving the information indicating the timing at which shooting is stopped together with the video data, it can be used for detecting the scene change during video reproduction.

In addition to stopping shooting, if there is a scene in the video being shot where the contents of the screen, such as colors, change significantly and the positional relationship of the subject before and after the change is not particularly important, the change. Timing may be detected as a scene change.

When the scene change is detected in the next frame, in step S33, the attraction area detection unit 172 detects the highlight (attraction area) in the next frame.

As described above, the attraction area is an area that the viewer wants to pay the most attention to in the video, and specifically, it is a person who appears large in the video or a person who is moving his or her mouth.

In the detection of the attraction region, for example, face recognition is performed by image processing, and an region in which a person with a moving mouth is reflected is specified.

Further, the video creator may check the video in advance and add information representing an area to be designated as a highlight to the video as metadata.

Further, a plurality of viewers may be made to watch the video in advance, and the area where more viewers are paying attention may be used as the attraction area.

In step S34, the twist amount detecting unit 173 determines whether or not the twist amount of the neck of the user 10 detected at that time is a certain amount or more.

The amount of twisting of the neck is the amount of rotation in the line-of-sight direction (face direction) of the user 10 with respect to the start of viewing the wide-angle image. For example, as shown in FIG. 10, the line-of-sight direction of the user 10 at the start of viewing a wide-angle image is set to the front direction (initial position) # 11, and the line-of-sight direction # 11 of the front direction # 11 is changed to the line-of-sight direction # 12 of the current user 10. The amount of rotation Ta1 of the neck is detected as the amount of twist of the neck.

Specifically, the amount of rotation from the initial position acquired by the HMD200 mounted on the head of the user 10 is detected as the amount of twisting of the neck.

Further, the twist amount may be the amount of rotation in the line-of-sight direction of the user 10 relative to the direction of the body of the user 10. For example, as shown in FIG. 11, when the body of the user 10 is rotated, the current body direction of the user 10 is set to the front direction # 21, and the line-of-sight direction of the current user 10 is set from the front direction # 21. The amount of rotation Ta2 of the neck relative to # 22 may be detected as the amount of twisting of the neck.

In this case, the detection of the front direction # 21 (body orientation) of the user 10 can be realized by, for example, providing a sensor for detecting the amount of rotation in the chair on which the user 10 sits.

The amount of twisting of the neck may always be detected, but it may be detected at least when the scene change is detected in the next frame.

If it is determined that the twist amount of the neck of the user 10 is a certain amount or more, the process proceeds to step S35, and the rotation processing unit 174 displays a wide-angle image based on the twist amount of the neck and the presentation position of the highlight of the next frame. Rotate.

First, the rotation processing unit 174 determines the presentation position of the highlight of the next frame (after the scene is switched) based on the amount of twisting of the neck at the time of scene switching.

For example, as shown in FIG. 12, it is assumed that the user 10 is facing the line-of-sight direction # 32 by twisting his / her neck to the right from the front direction # 31 at the time of scene switching. In this case, the ball 22, which is the highlight after the scene is switched, is arranged at the presentation position (between the front direction # 31 and the line-of-sight direction # 32) where the twist amount of the neck of the user 10 is small with respect to the line-of-sight direction # 32. To do so.

When the presentation position of the highlight of the next frame is determined, the rotation processing unit 174 rotates the wide-angle image after the next frame in the yaw direction so that the highlight is presented at the presentation position. In the example of FIG. 12, the wide-angle image is rotated in the yaw direction by the amount of rotation Ra.

The amount of rotation Ra (that is, how close the presentation position of the ball 22 is to the front direction # 31) may be arbitrarily specified by the application to which the present technology is applied.

However, since it is desired to eliminate the twist of the neck as soon as the twist amount of the neck is large, it is preferable that the presentation position of the highlight is closer to the front direction # 31 as the twist amount of the neck is larger.

Then, in step S36, the rotation processing unit 174 outputs the rotated video data to the display control unit 120.

If it is determined in step S32 that the scene change is not detected in the next frame, steps S33 to S35 are skipped, and in step S36, the unrotated video data is output.

If it is determined in step S34 that the amount of twisting of the neck is not more than a certain amount, step S35 is skipped, and in step S36, the video data as it is not rotated is output.

According to the above processing, the user can reduce the load on the body without losing the highlight when viewing the wide-angle image.

(Structure and operation to estimate the amount of neck twist)
In the above, it is assumed that the twist amount of the user's neck is actually detected, but the twist amount of the user's neck is estimated based on the presentation position of the highlight (attracting area) before and after the scene change. May be good.

In the following, a configuration for estimating the amount of twisting of the user's neck and its operation will be described.

(Configuration example of video processing unit)
FIG. 13 is a block diagram showing a configuration example of the video processing unit 110 that estimates the twist amount of the user's neck.

The image processing unit 110 of FIG. 13 is composed of a switching detection unit 191, an attraction region detection unit 192, a twist amount estimation unit 193, and a rotation processing unit 194.

The switching detection unit 191 and the attraction area detection unit 192 have the same functions as the switching detection unit 171 and the attraction area detection unit 172 in FIG.

The twist amount estimation unit 193 estimates the twist amount of the user's body, specifically, the twist amount of the neck, based on the information indicating the scene change and the information indicating the position of the attraction region. The estimated amount of neck twist is supplied to the rotation processing unit 194.

Based on the information indicating the scene change, the information indicating the position of the attracting area, and the estimated amount of neck twist, the rotation processing unit 194 sets the estimated amount of neck twist in the attracted area after the scene change to be smaller. Place it in the presentation position.

(Flow of video rotation processing)
Next, the flow of the video rotation processing by the video processing unit 110 of FIG. 13 will be described with reference to the flowchart of FIG.

In step S51, the video processing unit 110 reads the video data of the wide-angle video frame by frame.

In step S52, the switching detection unit 191 determines whether or not the scene switching is detected in the next frame of the read current frame.

When the scene change is detected in the next frame, in step S53, the attraction area detection unit 192 detects the highlight in the current frame.

Next, in step S54, the attraction region detection unit 192 detects the highlight in the next frame.

In step S34, the twist amount estimation unit 193 determines whether or not the twist amount of the user's neck is estimated to be a certain amount or more based on the position of the highlight (attracting area) of the current frame before the scene is switched. ..

Here, assuming that the user is always looking at the highlight, the amount of neck twist is estimated by setting the position of the highlight before the scene change as the user's line-of-sight direction (face direction) with respect to the front direction at the start of video viewing. Will be done.

If it is estimated that the twist amount of the user's neck is more than a certain amount, the process proceeds to step S56, and the rotation processing unit 194 displays a wide-angle image based on the estimated twist amount of the neck and the presentation position of the highlight of the next frame. Rotate.

Then, in step S57, the rotation processing unit 194 outputs the rotated video data to the display control unit 120.

If it is determined in step S52 that the scene change is not detected in the next frame, steps S53 to S56 are skipped, and in step S57, the unrotated video data is output.

If it is not estimated in step S55 that the amount of twisting of the neck is equal to or more than a certain amount, step S56 is skipped, and in step S57, the video data as it is not rotated is output.

By the above processing, the user can reduce the load on the body without losing the highlight when viewing the wide-angle image.

<3. Other configurations of video presentation system>
In the above-mentioned image presentation system, the HMD 200 is used as the display device for presenting the wide-angle image, but as shown in FIG. 15, the screen 300 may be used.

The screen 300 may be configured as an omnidirectional screen in which an omnidirectional image is presented as a wide-angle image, or may be configured as an omnidirectional screen in which an omnidirectional image is presented as a wide-angle image.

Also in the configuration of FIG. 15, the user can reduce the load on the body without losing the highlight when viewing the wide-angle image.

<4. Application to video editing system>
In the above, an example of applying this technology to a video presentation system has been described, but this technology can also be applied to a video editing system.

FIG. 16 is a block diagram showing a functional configuration example of an information processing apparatus that realizes a video editing system to which the present technology is applied.

The information processing device 400 is configured as, for example, a PC having a video editing function, a server of a video publishing site, or the like, and edits a wide-angle video into a video that has less burden on the user's body (for example, the neck).

The information processing apparatus 400 includes a video processing unit 410, a recording control unit 420, and a recording unit 430.

The video processing unit 410 has the same function as the video processing unit 110 described above, performs predetermined processing on a wide-angle video (video data), and supplies the wide-angle video (video data) to the recording control unit 420. The recording control unit 420 records the wide-angle image processed by the image processing unit 410 in the recording unit 430, or supplies it to an external device, the Internet, or the like.

FIG. 17 is a flowchart illustrating a flow of video editing processing in the video editing system described above.

When the video editor instructs, for example, an application for video editing to edit the video, the information processing apparatus 400 starts editing the video in step S111.

In step S112, the video processing unit 410 performs video rotation processing on the wide-angle video to be edited. Here, the process described with reference to the flowchart of FIG. 14 is executed.

In step S113, the recording control unit 420 records the wide-angle image subjected to the image rotation processing in the file of the recording unit 430.

As described above, the wide-angle image is edited into an image that does not burden the user's body.

<5. Others>
In the above, as the amount of twist of the user's body, the highlight is arranged at the presentation position where the amount of twist of the neck (head with respect to the torso) is small, but the amount of twist of the user's body to be used is limited to this. I can't. For example, as the amount of twist of the user's body, the highlight can be arranged at the presentation position where the amount of twist of the waist (upper body with respect to the lower body) is small.

Further, for example, while the user is wearing an HMD and viewing a wide-angle image, the user's line-of-sight direction, including the rotation of the entire body, rotates more than 180 degrees from the direction with respect to the start of viewing. Can be considered. In this case, the video rotation process described above may cause the user's line-of-sight direction to rotate even larger (in a direction close to 360 degrees), and the cable connected to the HMD may be wrapped around the user's body. Therefore, in such a case, the cable can be prevented from being wrapped around the user's body by arranging the highlight at the presentation position where the rotation in the line-of-sight direction of the user is returned.

When the image presentation system is composed of the HMD and the sensor that detects the physical condition of the user independent of the HMD is not provided in the external environment, it is basically necessary to detect the twist amount with the sensor provided in the HMD. be. As such a sensor, an inertial measurement unit (IMU) is generally known. Such a sensor is not limited to one type, and a sensor system to which a so-called sensor fusion technique in which a plurality of types of sensors are combined may be used.

When the HMD is provided with a single sensor or a single sensor system, the user's body movement according to the output value needs to be distinguished by the rotation of the whole body and the twist of the neck (or the twist of the waist). There is. In general, the rotational acceleration of the entire body and the rotational acceleration of the twist of the neck caused by the user's unconscious movement due to the inertial mass of the entire body, the range of motion of the joints, the motion characteristics of the individual user, etc. The output value of the sensor can be different. Specifically, the rotational acceleration of the entire body can be smaller than the rotational acceleration of the twisting of the neck. Therefore, by comparing the preset threshold value with the output value of the sensor, it may be determined whether the user's body movement is rotation of the entire body or twisting of the neck. This threshold value may be set by acquiring the output value of the sensor according to the user's operation in advance and using machine learning. Also, at least two thresholds may be set to distinguish between neck twist, hip twist, and whole body rotation.

The series of processes described above can be executed by hardware or software. When the series of processes described above is executed by software, the programs constituting the software are installed from a network or a recording medium.

As shown in FIG. 5, for example, the recording medium is composed of a removable medium 161 distributed for distributing a program to a user of the information processing apparatus 100, separately from the apparatus main body. The removable media 161 is composed of a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM and a DVD), a magneto-optical disk, a semiconductor memory, or the like on which the program is recorded. In addition to these, the recording medium is composed of a ROM 152 in which a program to be distributed to the administrator is recorded in a state of being preliminarily incorporated in the main body of the apparatus, a hard disk included in the storage unit 158, and the like.

In this specification, the steps for describing a program to be recorded on a recording medium are not only processed in chronological order in the order described, but also in parallel or in parallel, even if they are not necessarily processed in chronological order. It also includes processes that are executed individually.

Further, the embodiment of the present disclosure is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present disclosure.

Further, the effects described in the present specification are merely exemplary and not limited, and other effects may be used.

Further, the present disclosure may have the following structure.
(1)
Equipped with a video processing unit that performs predetermined processing on wide-angle video that exceeds the user's field of view.
The image processing unit arranges the attraction region in the frame after the scene change of the wide-angle image at the presentation position where the twist amount of the user's body becomes smaller with respect to the line-of-sight direction of the user before the scene change. Processing equipment.
(2)
The information processing device according to (1), wherein the image processing unit rotates the wide-angle image in the yaw direction so that the attraction region after the scene is switched is arranged at the presentation position.
(3)
The information processing apparatus according to (1) or (2), wherein the twist amount includes the twist amount of the user's neck.
(4)
The information processing apparatus according to (1) or (2), wherein the twist amount includes the twist amount of the waist of the user.
(5)
Further provided with a detection unit for detecting the twist amount,
The image processing unit is described in any one of (1) to (4), wherein the attraction region after the scene switching is arranged at the presentation position where the twist amount detected before the scene switching is smaller. Information processing device.
(6)
The information processing device according to (7), wherein the video processing unit arranges the attraction region after the scene switching at the presentation position when the twist amount detected before the scene switching is a certain amount or more.
(7)
The information processing apparatus according to (5) or (6), wherein the twist amount is a rotation amount in the line-of-sight direction of the user with respect to the start of viewing of the wide-angle image.
(8)
The information processing apparatus according to (5) or (6), wherein the twist amount is a rotation amount in the line-of-sight direction of the user relative to the orientation of the user's body.
(9)
Further, an estimation unit for estimating the twist amount based on the position of the attraction region in the frame before the scene change is provided.
The information processing apparatus according to any one of (1) to (4), wherein the video processing unit arranges the attraction region after the scene switching at the presentation position where the estimated twist amount is smaller.
(10)
The information processing apparatus according to any one of (1) to (9), wherein the attraction area is a person area detected in the wide-angle image.
(11)
The information processing apparatus according to any one of (1) to (9), wherein the attraction region is a region designated in advance in the wide-angle image.
(12)
The information processing apparatus according to any one of (1) to (11), wherein the wide-angle image is an image taken at a viewing angle of 180 degrees or more in the horizontal direction.
(13)
The information processing apparatus according to (12), wherein the wide-angle image is an omnidirectional image.
(14)
The information processing apparatus according to (12), wherein the wide-angle image is a spherical image.
(15)
The information processing apparatus according to any one of (1) to (14), further comprising a display control unit that controls display of the wide-angle image processed by the image processing unit on the display device.
(16)
The information processing device according to (15), wherein the display device is an HMD.
(17)
The information processing device according to (15), wherein the display device is an omnidirectional screen.
(18)
The information processing device according to (15), wherein the display device is a spherical screen.
(19)
Information processing equipment
A wide-angle image that exceeds the user's field of view is processed in a predetermined manner.
An information processing method in which an attraction region in a frame after a scene change of a wide-angle image is arranged at a presentation position where the amount of twist of the user's neck or waist is smaller than the line-of-sight direction of the user before the scene change.
(20)
On the computer
A wide-angle image that exceeds the user's field of view is processed in a predetermined manner.
A program for arranging the attraction area in the frame after the scene change of the wide-angle image at the presentation position where the twist amount of the user's neck or waist is smaller than the line-of-sight direction of the user before the scene change. ..

100 Information processing device, 110 Video processing unit, 120 Display control unit, 171 switching detection unit, 172 invitation area detection unit, 173 twist amount detection unit, 174 rotation processing unit, 191 switching detection unit, 192 invitation area detection unit, 193 twist Quantity estimation unit, 194 rotation processing unit, 200 HMD, 300 screen

Claims (20)

Equipped with a video processing unit that performs predetermined processing on wide-angle video that exceeds the user's field of view.
The image processing unit arranges the attraction region in the frame after the scene change of the wide-angle image at the presentation position where the twist amount of the user's body becomes smaller with respect to the line-of-sight direction of the user before the scene change. Processing equipment. The information processing device according to claim 1, wherein the image processing unit rotates the wide-angle image in the yaw direction so that the attraction region after the scene is switched is arranged at the presentation position. The information processing device according to claim 1, wherein the twist amount includes the twist amount of the user's neck. The information processing apparatus according to claim 1, wherein the twist amount includes the twist amount of the waist of the user. Further provided with a detection unit for detecting the twist amount,
The information processing apparatus according to claim 1, wherein the video processing unit arranges the attraction region after the scene switching at the presentation position where the twist amount detected before the scene switching is smaller. The information processing device according to claim 5, wherein the video processing unit arranges the attraction region after the scene switching at the presentation position when the twist amount detected before the scene switching is a certain amount or more. The information processing device according to claim 6, wherein the twist amount is a rotation amount in the line-of-sight direction of the user with respect to the start of viewing of the wide-angle image. The information processing apparatus according to claim 6, wherein the twist amount is a rotation amount in the line-of-sight direction of the user relative to the orientation of the user's body. Further, an estimation unit for estimating the twist amount based on the position of the attraction region in the frame before the scene change is provided.
The information processing device according to claim 1, wherein the video processing unit arranges the attraction region after the scene switching at the presentation position where the estimated twist amount is smaller. The information processing apparatus according to claim 1, wherein the attraction area is a person area detected in the wide-angle image. The information processing apparatus according to claim 1, wherein the attraction region is a region designated in advance in the wide-angle image. The information processing apparatus according to claim 1, wherein the wide-angle image is an image taken at a viewing angle of 180 degrees or more in the horizontal direction. The information processing apparatus according to claim 12, wherein the wide-angle image is an omnidirectional image. The information processing device according to claim 12, wherein the wide-angle image is a spherical image. The information processing apparatus according to claim 1, further comprising a display control unit that controls display of the wide-angle image processed by the image processing unit on the display device. The information processing device according to claim 15, wherein the display device is an HMD. The information processing device according to claim 15, wherein the display device is an omnidirectional screen. The information processing device according to claim 15, wherein the display device is a spherical screen. Information processing equipment
A wide-angle image that exceeds the user's field of view is processed in a predetermined manner.
An information processing method in which an attraction region in a frame after a scene change of a wide-angle image is arranged at a presentation position where the amount of twist of the user's neck or waist is smaller than the line-of-sight direction of the user before the scene change. On the computer
A wide-angle image that exceeds the user's field of view is processed in a predetermined manner.
A program for arranging the attraction area in the frame after the scene change of the wide-angle image at the presentation position where the twist amount of the user's neck or waist is smaller than the line-of-sight direction of the user before the scene change. ..

Images