WO2021230001A1 - Information processing apparatus and information processing method - Google Patents

Abstract

Provided is an information processing apparatus (30) including an acquisition unit (308) for acquiring, in real time, video content data to be played back to a user according to information on the user's visual acuity.

Description

Information processing equipment and information processing method

This disclosure relates to an information processing device and an information processing method.

Currently, various video distribution services are used, and as the display devices used for viewing them become higher resolution to HD (High Definition), full HD, 4K, and 8K. , Content distributed by the above distribution service (for example, content of two-dimensional video) is also required to have higher resolution.

In recent years, all-sky images taken by all-sky cameras and the like that can look around in all directions have also been distributed. In addition, recently, viewers can look around in all directions (freely select the line-of-sight direction) and freely move in three-dimensional space (freely select the viewpoint position) 6DoF. (Degree of Freedom) Development of technology for distributing video (also referred to as 6DoF content) is underway. In the distribution of such 6DoF contents, the three-dimensional space is dynamically reproduced by one or a plurality of three-dimensional objects according to the viewpoint position, the line-of-sight direction, and the viewing angle of the viewer at each time.

As a method of distributing such 6DoF contents, for example, a method of constructing a three-dimensional space with a plurality of three-dimensional objects and distributing data related to these a plurality of objects has been proposed. The data distributed by this method includes data in which a scene in a three-dimensional space, for example, a scene description, is represented by a graph having a tree hierarchical structure called a scene graph, and the scene graph is represented in a binary format or a text format (). For example, Non-Patent Document 1).

Then, as the resolution of the delivered content increases, the amount of data to be delivered increases, the load on the communication network increases (that is, the bandwidth of the communication network is insufficient), and the viewer who enjoys the delivered content (that is, the viewer (that is, the bandwidth of the communication network is insufficient)). For the user), the amount of money to be paid as communication costs will also increase.

Therefore, in this disclosure, we propose an information processing device and an information processing method that can avoid an increase in the amount of distributed data and an increase in the load on a communication network while avoiding the viewer's feeling of deterioration in image quality.

According to the present disclosure, an information processing device provided with an acquisition unit for acquiring video content data to be played back for the user in real time according to the visual acuity information of the user is provided.

Further, according to the present disclosure, an information processing method including acquiring video content data to be played back for the user in real time according to the visual acuity information of the user is provided.

It is explanatory drawing (the 1) for demonstrating the outline of embodiment of this disclosure. It is explanatory drawing (the 2) for demonstrating the outline of embodiment of this disclosure. It is explanatory drawing for demonstrating the outline of the moving image content distribution system which concerns on 1st Embodiment of this disclosure. It is a block diagram which shows the functional composition example of the reproduction apparatus which concerns on 1st Embodiment of this disclosure. It is a figure which showed the flowchart of the information processing method which concerns on the same embodiment. It is explanatory drawing for demonstrating the outline of the 2nd Embodiment of this disclosure. It is a figure which showed the flowchart of the information processing method which concerns on the same embodiment. It is explanatory drawing (the 1) for demonstrating the outline of the 3rd Embodiment of this disclosure. It is explanatory drawing (the 2) for demonstrating the outline of the embodiment. It is explanatory drawing (the 3) for demonstrating the outline of the embodiment. It is explanatory drawing (the 4) for demonstrating the outline of the embodiment. It is explanatory drawing (the 5) for demonstrating the outline of the embodiment. It is a block diagram which shows the functional composition example of the reproduction apparatus which concerns on the same embodiment. It is a figure which showed the flowchart of the information processing method which concerns on the same embodiment. It is a figure which showed the flowchart of the information processing method which concerns on 4th Embodiment of this disclosure. It is a hardware block diagram which shows an example of the computer 1000 which realizes the function of the reproduction apparatus 30 and the like. It is a figure which shows an example of the schematic structure of an endoscopic surgery system. It is a block diagram which shows an example of the functional structure of the camera head and CCU shown in FIG.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, and duplicate description will be omitted. Further, in the present specification and the drawings, a plurality of components having substantially the same or similar functional configurations may be distinguished by adding different alphabets after the same reference numerals. However, if it is not necessary to particularly distinguish each of the plurality of components having substantially the same or similar functional configurations, only the same reference numerals are given.

The explanations will be given in the following order.
1. 1. Background 2. Overview of the embodiments of the present disclosure 3. First Embodiment 3.1 System configuration 3.2 Configuration of playback device 3.3 Information processing method 4. 2nd Embodiment 4.1 Outline of 2nd Embodiment 4.2 System configuration 4.3 Configuration of playback device 4.4 Information processing method 5. Third embodiment 5.1 Outline of the third embodiment 5.2 System configuration 5.3 Configuration of playback device 5.4 Information processing method 6. Fourth Embodiment 6.1 Outline of the Fourth Embodiment 6.2 System Configuration and Playback Device Configuration 6.3 Information Processing Method 7. Summary 8. Hardware configuration example 9. Application example 10. supplement

<< 1. Background >>
First, the background of the present disclosure will be described with reference to FIG. Currently, various video distribution services are used, and as the display devices used for viewing them become higher resolution to HD (High Definition), full HD, 4K, and 8K. , Content distributed by video distribution services is also required to have higher resolution. As the resolution of such contents increases, the amount of data to be delivered increases, the load on the communication network increases (that is, the bandwidth of the communication network is insufficient), and the viewers who enjoy the delivered contents (that is, the viewers who enjoy the delivered contents). For the user), the amount of money to be paid as communication costs will also increase.

Therefore, in 2D video distribution, a distribution technology using DASH (Dynamic Adaptive Streaming over HTTP, ISO / IEC 23009-1) is known. In the distribution technology, a plurality of stream variations of the same content but having different bit rates are prepared in advance in the distribution server, and the streams are switched according to the fluctuation of the communication network bandwidth. Specifically, in the distribution technology, for example, when the bandwidth of the communication network is insufficient, the distribution is switched to the distribution of a stream variation having a low bit rate. By doing so, it is possible to reduce the load on the communication network due to the increase in the amount of data to be delivered, but the image quality is deteriorated.

Furthermore, in recent years, all-sky images taken by all-sky cameras and the like, which can look around in all directions, have come to be distributed. In addition, recently, the development of a technology for distributing 6DoF content that allows viewers to look around in all directions and freely move in a three-dimensional space is underway. As explained above, 6DoF content dynamically reproduces 3D space with one or more 3D objects according to the viewer's viewpoint position, line-of-sight direction, and viewing angle at each time. be. Then, in the distribution of 6DoF contents, it is considered to combine the above-mentioned DASH with a method using scene description data used in 3D CG (computer graphics).

Specifically, in the above 6DoF content distribution, object data relating to each 3D object is displayed in order to display all objects (3D objects) existing in the 3D space even if the position of the viewer moves. To deliver. The object data is a combination of mesh data, which is the shape of a polyhedron of a three-dimensional object, and texture data, which is data to be attached to the surface, or Point Cloud (point cloud), which is a set of multiple points. Consists of data.

Furthermore, in the distribution, along with the above-mentioned object data, data called a scene description for displaying a three-dimensional object according to the viewpoint position of the viewer is also distributed. The scene description not only defines the position and posture of the three-dimensional object in the three-dimensional space, but also provides a plurality of different LODs (Level Of Detail) according to the viewpoint position of the viewer for one three-dimensional object. Define. The LOD defines, for example, the number of points (details) representing the same three-dimensional object in the case of a three-dimensional object represented by Point Cloud. Specifically, as a representation of the three-dimensional space, a three-dimensional object far from the viewer's viewpoint position makes the display small, and a three-dimensional object close to the viewer's viewpoint position makes the display large. Further, in the expression, a three-dimensional object having a large display has a high LOD of its display, and a three-dimensional object having a small display has a low LOD of its display. The higher the LOD, the larger the amount of data for the display. By expressing a 3D object in a 3D space using the LOD defined in this way, a 3D object with a large display is reproduced with high resolution, and a 3D object with a small display is reproduced with low resolution. Therefore, for the viewer, the three-dimensional object located nearby can be clearly seen as in the real space. Therefore, by expressing the three-dimensional space in this way, it is possible for the viewer to view the data without discomfort while suppressing an increase in the amount of data.

However, even in the 6DoF content distribution technology, it is difficult to avoid an increase in the amount of data to be distributed and an increase in the load on the communication network due to the increase in the resolution of the content accompanying the increase in the resolution of the display device.

Currently, there are various standards for scene description. Basically, a scene is represented by a graph with a tree hierarchical structure called a scene graph, and the scene graph is represented in binary format or text format. Here, the scene graph is spatial display control information based on the viewpoint position of the viewer, and the information regarding the display of the three-dimensional object at the viewpoint position is defined by defining the node as a constituent unit, and a plurality of nodes are hierarchically combined. It consists of. There are nodes for position information and size information of 3D objects, nodes for access information to mesh data and texture data, and nodes for information to be displayed appropriately according to the distance from the viewpoint position. These nodes will be used for each 3D object. For example, the data of the scene description can conform to MPEG-4 Scene Description (ISO / IEC 14496-11). The MPEG-4 Scene Description data is obtained by binarizing the scene graph in the format of BIFS (Binary Form at for Scenes). It is possible to convert this scene graph to BIFS with a fixed algorithm. In addition, by storing in ISO base media file format, it is possible to specify a scene for each time, and it is possible to express a moving object or the like.

<< 2. Summary of Embodiments of the present disclosure >>
In such a situation, the present inventors are diligently studying whether it is possible to avoid an increase in the amount of distribution data and an increase in the load on the communication network due to the high resolution in the distribution of video content. rice field. Then, while proceeding with such a study, the present inventors, in the above-mentioned distribution technology, the data of the video content to be distributed is the amount of data according to the resolution of the display device, the band that can be used in the communication network, and the like. I noticed that it has the resolution of, but the resolution does not correspond to the resolution of the eyes of the viewer who watches the video content. That is, the present inventors have noticed that there are cases where the data of the moving image content having an unnecessarily high resolution with respect to the resolution of the viewer's eyes is delivered. Then, based on the above viewpoint, the present inventors can avoid an increase in the amount of distributed data and an increase in the load on the communication network while avoiding the viewer from feeling the deterioration of the image quality. It came to create the embodiment independently.

The outline of the embodiment of the present disclosure created by the present inventors will be described below with reference to FIGS. 1 and 2. 1 and 2 are explanatory views for explaining an outline of the embodiment of the present disclosure, and FIG. 1 is a diagram showing a positional relationship between a viewer and a display device as viewed from above, and FIG. 2 is a diagram. Is a diagram showing the positional relationship between the viewer and the display device as viewed from the side.

In the embodiment of the present disclosure, for the viewer, based on the visual acuity information regarding the visual acuity of the viewer, the display unit information regarding the size of the display device, and the position information indicating the position of the viewer with respect to the display device. Calculate the necessary and sufficient resolution (user resolution) for the viewer. Then, according to the embodiment of the present disclosure, by distributing the data of the video content according to the calculated resolution, the amount of the distributed data can be increased and the communication network can be increased while avoiding the viewer from feeling the deterioration of the image quality. It is possible to avoid an increase in load.

Therefore, in the following, a method of calculating a resolution necessary and sufficient for the resolution of the viewer's eyes will be described based on the viewer's visual acuity and the distance between the viewer and the display device. Specifically, when trying to express the resolution of the human eye with the resolution used for the display device, a unit called PPD (Pixels Per Degree) is used, and generally, the human eye with a visual acuity of 1.0 is used. The resolution is said to be 60 PPD. Since there are various theories about the specific value of the resolution of the human eye with a visual acuity of 1.0, in the following description, the resolution of the human eye with a visual acuity of 1.0 is expressed by a constant P. ..

Therefore, the necessary and sufficient resolution for the human eye with a visual acuity of 1.0 can be satisfied by the presence of P (pixel) in the range of length X on the display device corresponding to one visual field range. Therefore, the width direction resolution (Width) and the height direction resolution (Height) of the display device for the viewers (viewers with "visual acuity") in the situations shown in FIGS. 1 and 2 are determined. , Can be expressed by the following mathematical formula (1).

And, as is clear from FIGS. 1 and 2, the above x can be expressed by the following mathematical formula (2).

Therefore, when the mathematical formula (1) is rewritten using the mathematical formula (2), the width direction of the display device for the viewers (viewers with "visual acuity") in the situations shown in FIGS. 1 and 2. The resolution (Width) and the resolution (Height) in the height direction can be expressed by the following mathematical formula (3).

Here, as an example, a viewer with a visual acuity of 1.0 who watches from a position L = 1.44 m away from a display device having a size of W (width) = 1.6 m and H (height) = 0.9 m. Using the mathematical formula (3), the necessary and sufficient resolutions for A and the viewer B with a visual acuity of 0.5 are as follows (here, P = 60PPD).

In the case of viewer A (visual acuity 1.0), the resolution in the width direction (Width) is 3819.62, and the resolution in the height direction (Height) is 2148.54.

In the case of viewer B (visual acuity 0.5), the resolution in the width direction (Width) is 1909.81, and the resolution in the height direction (Height) is 1074.27.

Then, the playback device has a resolution higher than the resolution calculated by using the mathematical formula (3) from a plurality of content data of the same content but different resolutions from the distribution server, and is the most among the content data in the distribution server. Acquire content data with low resolution. For example, the distribution server has SD (Standard definition) (720 x 480), HD (1280 x 720), full HD (1920 x 1080), 4K (3840 x 2160), and 8K (7680 x 4320) resolutions. It is assumed that the content data to be stored is stored.

In such a case, 4K (3840 × 2160) content data may be selected for viewer A, and full HD (1920 × 1080) content data may be selected for viewer B. Will be.

Even if the resolution of the display device is 8K (7680 × 4320), instead of acquiring 8K content data according to the display device, in the embodiment of the present disclosure, it is for the viewer A. 4K content data and full HD content data for viewer B will be acquired. By doing so, in the embodiment of the present disclosure, it is possible to avoid acquiring content data having a resolution unnecessary for the resolution of the viewer's eyes, and the content necessary and sufficient for the resolution of the viewer's eyes. You can get the data. Therefore, according to the embodiment of the present disclosure, it is possible to avoid an increase in the amount of distribution data and an increase in the load on the communication network while avoiding the viewer from feeling the deterioration of the image quality. Hereinafter, details of each embodiment of the present disclosure will be sequentially described.

<< 3. First Embodiment >>
<3.1 System configuration>
First, the video content distribution system 1 according to the first embodiment of the present disclosure will be described with reference to FIG. FIG. 3 is an explanatory diagram for explaining an outline of the video content distribution system according to the first embodiment of the present disclosure. In the present embodiment, the distribution system 1 distributes moving image content composed of two-dimensional video.

Specifically, as shown in FIG. 3, the video content distribution system 1 according to the present embodiment includes a content distribution server 10 that distributes content data of video content in response to a request from the playback device 30. The distribution system 1 further includes a reproduction device 30 for reproducing the distributed moving image content, and a display device 20 for displaying the moving image content according to the control from the reproduction device 30. Further, the content distribution server 10 and the playback device 30 are connected to each other by a communication network 40. In the present embodiment, the reproduction device 30 and the display device 20 may be separate devices as shown in FIG. 3, or may be an integrated device, and are not particularly limited. Further, as the communication method used in the communication network 40, any method can be applied regardless of whether it is wired or wireless, but it is desirable to use a communication method capable of maintaining stable operation. The outline of each device included in the distribution system 1 will be described below.

(Content distribution server 10)
The content distribution server 10 distributes the content data of the moving image content having a predetermined resolution to the reproduction device 30 in response to the request from the reproduction device 30. Further, the content distribution server 10 stores a plurality of content data of the same video content having different resolutions (for example, SD (720 × 480), HD (1280 × 720), full HD (1920 × 1080), 4K). (3840 x 2160), 8K (7680 x 4320)).

(Display device 20)
The display device 20 is a two-dimensional display such as a television, a tablet, and a smartphone. In the present embodiment, the display device 20 incorporates a distance measuring device (for example, a ToF (Time of Flight) sensor) and a sensor (distance measuring unit) such as a camera, and is based on the sensing data of these sensors. It shall be possible to detect the relative position (position and distance) with the viewer. Here, the distance between the viewer and the display device 20 is assumed to be the distance from the viewer to the closest point on the display surface of the display device 20 as seen from the viewer. Further, the display device 20 has a built-in communication unit that wirelessly communicates with a communication device carried by the viewer, and is a relative position (position and distance) with the viewer based on the sensing data by the communication unit. May be detected. Alternatively, in the present embodiment, the display device 20 is relative to the viewer based on the position information from the distance measuring device carried by the viewer (for example, a GNSS (Global Navigation Satellite System) signal receiver). The position (position and distance) may be detected.

(Reproduction device 30)
The reproduction device 30 acquires the content data of the moving image content in real time from the content distribution server 10 according to the visual acuity information of the viewer, reproduces the acquired content data, and outputs the acquired content data to the display device 20. The details of the reproduction device 30 will be described later.

Note that the above configuration described with reference to FIG. 3 is merely an example, and the configuration of the distribution system 1 according to the present embodiment is not limited to such an example. That is, the configuration of the distribution system 1 according to the present embodiment can be flexibly modified according to the specifications and operation.

<3.2 Configuration of playback device>
Next, a configuration example of the reproduction device 30 according to the present embodiment will be described with reference to FIG. FIG. 4 is a block diagram showing a functional configuration example of the reproduction device 30 according to the present embodiment. Specifically, as shown in FIG. 4, the reproduction device 30 mainly includes a display control unit 300, a main control unit 330, a storage unit 340, and a transmission / reception unit 350. Hereinafter, details of each functional unit of the reproduction device 30 will be sequentially described.

(Display control unit 300)
The display control unit 300 is composed of, for example, a CPU (Central Processing Unit) (not shown), an MPU (Micro Processing Unit) (not shown), or the like, and uses programs and data stored in the storage unit 340 described later. , The process according to this embodiment is executed. Specifically, the display control unit 300 includes a position information acquisition unit 302, a calculation unit 304, and a comparison unit 306 for acquiring content data of video content from the content distribution server 10 according to the visual acuity information of the viewer. , Acquiring unit 308. Further, the display control unit 300 includes a processing unit 322, a decoding unit 324, and a display information generation unit 326 (in the present specification, these functional units are also referred to as a decoding block 320) that decode the acquired content data. Have.

The position information acquisition unit 302 acquires distance information (position information) for the viewer's display device 20 from the display device 20, for example, and outputs the distance information (position information) to the calculation unit 304, which will be described later.

The calculation unit 304 is output from the viewer's visual information (visual information), the size (width, height) information (display unit information) of the display device 20, and the position information acquisition unit 302, which are acquired in advance. Based on the distance information (distance to the display device 20 of the viewer), the resolution (user resolution) in the height direction and the width direction of the display device 20 for the viewer is calculated. Further, the calculation unit 304 outputs the calculated resolution to the comparison unit 306 described later.

The comparison unit 306 compares the resolution calculated by the calculation unit 304 with the resolution of the display device 20 acquired in advance, and outputs the comparison result to the acquisition unit 308 described later.

The acquisition unit 308 acquires content data having a resolution necessary and sufficient for the resolution of the viewer's eyes in real time based on the viewer's visual acuity (visual acuity information). Specifically, the acquisition unit 308 acquires content data having a resolution based on the comparison result with respect to the calculated resolution. When there are a plurality of viewers, in the present embodiment, the acquisition unit 308 acquires content data having a resolution based on the comparison result of the resolution comparison for each viewer by the comparison unit 306. Will be done. Then, the acquisition unit 308 outputs the acquired content data to the processing unit 322 of the decoding block 320.

The processing unit 322 is a functional unit that performs processing related to playback of video content, performs processing for switching acquisition content data (DASH), analyzes acquired content data, and decodes processing in the decoding unit 324, which will be described later. Can be converted to a file format that can be used. Further, the processing unit 322 outputs the processed content data to the decoding unit 324.

The decoding unit 324 performs decoding processing (decoding) on the content data output from the processing unit 322, and outputs the decoded content data to the display information generation unit 326 described later.

The display information generation unit 326 processes the decoded content data output from the decoding unit 324, generates a display screen for display on the display device 20, and outputs the display screen to the display device 20. The display information generation unit 326 may cut out (render) the image according to the viewpoint of the viewer.

(Main control unit 330)
The main control unit 330 has a functional configuration that comprehensively controls all the processing performed by the playback device 30, and is composed of, for example, a CPU (not shown), an MPU (not shown), and the like, and is stored in a storage unit 340 described later. Execute the process using the program or data.

(Memory unit 340)
The storage unit 340 is a functional unit that stores various types of information. For example, the storage unit 340 stores programs, content data, parameters, and the like used by each functional unit of the reproduction device 30. Further, the storage unit 360 is realized by, for example, a magnetic recording medium such as a hard disk (Hard Disk: HD), a non-volatile memory, or the like.

(Transmission / reception unit 350)
The transmission / reception unit 350 performs various communications with the content distribution server 10. Specifically, the transmission / reception unit 350 is a communication interface having a function of transmitting / receiving data, and is realized by, for example, a communication device (not shown) such as a communication antenna, a transmission / reception circuit, and a port.

Further, in the present embodiment, the reproduction device 30 has an input unit (not shown), and acquires the visual acuity information of the viewer by receiving an input operation of the visual acuity information from the viewer to the input unit. Can be done. The above configuration described with reference to FIG. 4 is merely an example, and the configuration of the reproduction device 30 according to the present embodiment is not limited to such an example. That is, the configuration of the reproduction device 30 according to the present embodiment can be flexibly modified according to the specifications and operation.

<3.3 Information processing method>
The detailed configuration of the reproduction device 30 according to the present embodiment has been described above. Next, the information processing method according to the present embodiment will be described with reference to FIG. FIG. 5 is a diagram showing a flowchart of the information processing method according to the present embodiment. In detail, as shown in FIG. 5, the information processing method according to the present embodiment includes steps S101 to S110. Hereinafter, each step of the information processing method according to the present embodiment will be described.

First, the reproduction device 30 acquires information (display unit information) of the size (width, height) of the display device 20 (step S101). Next, the reproduction device 30 acquires the visual acuity information (visual acuity information) of the viewer by the input from the viewer (step S102). Further, the reproduction device 30 acquires information (distance information) of the distance to the display device 20 of the viewer from, for example, the display device 20 (step S103). In this embodiment, the distance information is acquired every time the position of the viewer moves, and the subsequent processing is performed.

Next, the reproduction device 30 obtains the viewer's sight information, the size (width, height) information of the display device 20, and the distance information of the viewer with respect to the display device 20 acquired in steps S101 to S103. Based on (distance information), the resolution (user resolution) of the viewer in the height direction and the width direction of the display device 20 is calculated (step S105). Since the calculation method has already been described, the description thereof will be omitted here.

Then, the playback device 30 determines whether or not the resolutions of all the viewers have been calculated (step S105). The playback device 30 proceeds to step S106 when the resolution has been calculated for all the viewers (step S105: Yes), while the resolution has not been calculated for all the viewers. In that case (step S105: No), the process returns to step S102 described above. That is, in the present embodiment, the processes from step S102 to step S105 are repeated until the resolution is calculated for all the viewers.

Next, the playback device 30 compares the viewer resolutions calculated so far for each of the height direction and the width direction of the display device 20, and specifies the highest resolution. In the present embodiment, when there are a plurality of viewers, it is possible to avoid causing all the viewers to feel the deterioration of the image quality by acquiring the content data based on the highest resolution. Further, the reproduction device 30 compares the specified resolution with the resolution of the display device 20 acquired in advance, and selects a lower resolution (step S106).

Then, the playback device 30 acquires the content data corresponding to the resolution selected in step S106 described above (step S107). Next, the playback device 30 performs decoding processing on the content data acquired in step S107 described above (step S108). Further, the reproduction device 30 displays the moving image content by outputting the content data decoded in the above-mentioned step S108 to the display device 20 (step S109).

Further, the playback device 30 determines whether or not processing has been performed up to the end of the stream (plurality of frames) included in the content data (step S109). When the reproduction device 30 is processing up to the end of the stream (step S109: Yes), the reproduction device 30 ends the information processing according to the present embodiment. On the other hand, if the reproduction device 30 has not processed to the end of the stream (step S109: No), the reproduction device 30 returns to the above-mentioned step S102. That is, in the present embodiment, the processing from step S102 to step S110 is repeatedly performed until the processing is completed up to the end of the stream.

As described above, in the present embodiment, it is possible to avoid acquiring the content data having a resolution unnecessary for the resolution of the viewer's eyes, and to acquire the content data necessary and sufficient for the resolution of the viewer's eyes. be able to. Therefore, according to the present embodiment, it is possible to avoid an increase in the amount of distributed data and an increase in the load on the communication network while avoiding the viewer from feeling the deterioration of the image quality.

<< 4. Second embodiment >>
<4.1 Outline of the second embodiment>
First, the outline of the second embodiment will be described with reference to FIG. FIG. 6 is an explanatory diagram for explaining the outline of the present embodiment. In the first embodiment described above, the resolution for the viewer is calculated based on the visual acuity (visual acuity information) of the viewer and the distance (distance information) to the display device 20 of the viewer, which will be described below. In the second embodiment, in addition to the visual acuity information and the distance information, the angle information indicating the angle of the viewer with respect to the display device 20 is used to more accurately calculate the resolution for the viewer. can do. That is, in the present embodiment, since the calculation is performed in consideration of the above angle information, it is possible to calculate the resolution suitable for the viewer with high accuracy, so that the viewer feels the image quality deterioration more effectively. While avoiding this, it is possible to avoid an increase in the amount of distributed data and an increase in the load on the communication network.

First, the details of the resolution calculation method in the present embodiment will be described with reference to FIG. FIG. 6 is an explanatory diagram for explaining the outline of the present embodiment, and the viewer displays the end of the display device 20 (the end portion of the display device 20 closest to the viewer) to the display device 20. It is a figure when the appearance of viewing from an oblique position with respect to the display surface of the viewer is seen from above the viewer.

Also in this embodiment, the resolution of the human eye with a visual acuity of 1.0 is expressed by a constant P, and the necessary and sufficient resolution for the human eye with a visual acuity of 1.0 is P (1 degree in the visual field range). resolution) It shall be possible to satisfy by being present. Therefore, assuming a viewer (viewer with "visual acuity") in the situation shown in FIG. 6, P / 2 is set to a length X (m) corresponding to a viewing range of 0.5 degrees. (Pixel) It suffices if it exists. Therefore, the width direction resolution (Width) of the display device 20 which is necessary and sufficient for the viewer in FIG. 6 can be expressed by the following mathematical formula (4).

Here, X is the distance L of the viewer to the end of the display device 20 (the end portion of the display device 20 closest to the viewer) and the angle θ with respect to the end of the display device 20 of the viewer. Can be expressed by the following mathematical formula (5).

Therefore, when the mathematical formula (4) is rewritten using the mathematical formula (5), the width direction resolution (Width) of the display device 20 necessary and sufficient for the viewer in FIG. 6 is expressed by the following mathematical formula (6). be able to.

Similarly, the height resolution (Height) of the display device 20 in the height direction, which is necessary and sufficient for the viewer in FIG. 6, can be expressed by the following mathematical formula (7). Here, the angle of the viewer with respect to the end of the display device 20 (the portion of the end of the display device 20 closest to the viewer) in the height direction is defined as β (degree).

Here, as an example, L = 1.44 m away from the display device 20 having a size of W (width) = 1.6 m and H (height) = 0.9 m, and viewing angles θ and β = 60 degrees. Using the mathematical formulas (6) and (7), the necessary and sufficient resolutions of the viewer A with a visual acuity of 1.0 to be viewed from the position of are as follows (here, P = 60PPD). ).

The resolution in the width direction (Width) is 1880.94, and the resolution in the height direction (Height) is 1058.03.

Here, for example, the content distribution server 10 has SD (720 × 480), HD (1280 × 720), full HD (1920 × 1080), 4K (3840 × 2160), and 8K (7680 × 4320) resolutions. It is assumed that the content data with is stored. In such a case, in the present embodiment, the full HD content data having the lowest resolution satisfying the resolution of the viewer A calculated as described above is selected and distributed.

In the present embodiment, even if the resolution of the display device 20 is 8K (7680 × 4320), it is necessary and sufficient for the viewer A to have a resolution of full HD (1920 × 1080). , It is possible to avoid acquiring content data having a resolution unnecessary for the resolution of the viewer's eyes. Therefore, according to the present embodiment, it is possible to avoid an increase in the amount of distributed data and an increase in the load on the communication network while avoiding the viewer from feeling the deterioration of the image quality. Further, in the present embodiment, since the resolution (full HD (1920 × 1080) in the above example) is calculated in consideration of the viewing angle information of the viewer, the resolution is calculated without considering the viewing angle. (In the above example, it is possible to obtain a resolution more suitable for the viewing state of the viewer as compared with 4K (3840 × 2160)), and to acquire content data having a resolution unnecessary for the resolution of the viewer's eyes. Can be avoided more. Hereinafter, the details of the present embodiment will be sequentially described.

<4.2 system configuration>
First, the video content distribution system 1 according to the present embodiment will be described. However, since the distribution system 1 according to the present embodiment is common to the first embodiment except for the following points, the common points are described here. The explanation is omitted, and only the differences are explained.

In the present embodiment, as in the first embodiment, the display device 20 incorporates a sensor (angle measuring unit) such as a distance measuring device or a camera, and displays the viewer based on these sensing data. It is assumed that the relative position and viewing angle with respect to the device 20 can also be detected. Here, the viewing angle of the viewer is assumed to be the angle of the viewer with respect to the closest end of the display surface of the display device 20 as seen from the viewer.

<4.3 Configuration of playback device>
Next, the configuration of the reproduction device 30 according to the present embodiment will be described. However, since the reproduction device 30 according to the present embodiment is common to the first embodiment except for the following points, the common points are described here. The explanation is omitted, and only the differences are explained. In the present embodiment, the position information acquisition unit 302 and the calculation unit 304 are different from the first embodiment in the following points.

The position information acquisition unit 302 acquires not only the distance information of the viewer with respect to the display device 20 but also the relative position information and the viewing angle information from the display device 20, and outputs the information to the calculation unit 304 described later.

The calculation unit 304 is output from the viewer's visual information (visual information) acquired in advance, the size (width, height) information (display unit information) of the display device 20, and the position information acquisition unit 302. In addition to the distance information, the resolution (user resolution) in the height direction and the width direction of the display device 20 for the viewer is calculated based on the viewing angle with respect to the display device 20 of the viewer.

<4.4 Information processing method>
The detailed configuration of the reproduction device 30 according to the present embodiment has been described above. Next, the information processing method according to the present embodiment will be described with reference to FIG. 7. FIG. 7 is a diagram showing a flowchart of the information processing method according to the embodiment. In detail, as shown in FIG. 7, the information processing method according to the present embodiment includes steps S201 to S213. The details of each of these steps according to the present embodiment will be described below. In the following description, only the points different from the above-mentioned first embodiment will be described, and the points common to the first embodiment will be omitted.

Since steps S201 and S202 are the same as steps S101 and S102 of the first embodiment shown in FIG. 5, the description thereof will be omitted here.

The playback device 30 acquires information on the relative position of the viewer with respect to the display device 20 as well as information on the distance to the viewer's display device 20 from the display device 20 (step S203). Then, the reproduction device 30 determines whether or not the viewer exists so as to face the frame of the display surface of the display device 20 based on the relative position acquired in the above-mentioned step S203 (step S204). When the viewer is present so as to face the frame of the display surface of the display device 20 (step S204: Yes), the playback device 30 proceeds to step S205, while the viewer is on the display surface of the display device 20. If they do not exist so as to face each other in the frame (step S204: No), the process proceeds to step S206.

When the viewers are present so as to face each other within the frame of the display surface of the display device 20, the viewing angle between the viewer and the display surface can be considered to be 0 degrees. There is no need to get angle information. Therefore, the playback device 30 sets the viewing angle to 0 degrees (step S205). Further, the playback device 30 acquires information on the viewing angle with respect to the viewer's display device 20 from the display device 20 (step S206).

The reproduction device 30 has acquired the viewer's sight information, the size (width, height) information of the display device 20, and the distance information (distance information) with respect to the viewer's display device 20 acquired from the above-mentioned steps S201 to S203. ), And based on the viewing angle set or acquired in steps S205 and S206, the resolution (user resolution) of the viewer in the height direction and the width direction of the display device 20 is calculated (step S207). Since the calculation method has already been described, the description thereof will be omitted here.

Since steps S209 to S213 are the same as steps S106 to S110 of the first embodiment shown in FIG. 5, the description thereof will be omitted here.

As described above, in the present embodiment, since the resolution is calculated in consideration of the information of the viewing angle of the viewer, the viewing state of the viewer is more as compared with the case where the resolution is calculated without considering the viewing angle. It is possible to obtain a resolution that matches the above. As a result, according to the present embodiment, it is possible to further avoid acquiring content data having a resolution unnecessary for the resolution of the viewer's eyes, that is, distribution while avoiding the viewer's feeling of image quality deterioration. It is possible to avoid an increase in the amount of data and an increase in the load on the communication network.

<< 5. Third Embodiment >>
<5.1 Outline of the third embodiment>
First, the outline of the third embodiment will be described with reference to FIGS. 8 to 12. 8 to 12 are explanatory views for explaining the outline of the present embodiment. The moving image content delivered in the present embodiment is assumed to be 360-degree virtual space video data (three-dimensional space data) including all-sky video data and 6DoF content (three-dimensional space data). And, as explained above, the content data of these contents includes the scene description (three-dimensional space description data) that defines the configuration of the three-dimensional space and the object data that defines the three-dimensional object in the three-dimensional space. It shall be muted.

By the way, as described above, in the representation of a three-dimensional space, a three-dimensional object that is far from the viewer's viewpoint position makes its display smaller, and a three-dimensional object that is close to the viewer's viewpoint position is its display. Enlarge the display. Further, in the expression, a three-dimensional object having a large display has a high LOD of its display, and a three-dimensional object having a small display has a low LOD of its display. Then, by expressing the 3D object in the 3D space using the LOD defined in this way, the 3D object with a large display is reproduced with high resolution, and the 3D object with a small display is reproduced with low resolution. Therefore, for the viewer, the three-dimensional object located nearby can be clearly seen as in the real space. Therefore, by expressing the three-dimensional space in this way, it is possible for the viewer to view the data without discomfort while suppressing an increase in the amount of data. Therefore, in the present embodiment, a plurality of object data of the same 3D object having different LODs are prepared in advance in the content distribution server 10, and the viewpoint of the viewer and the 3D object in the virtual 3D space are prepared in advance. The object data is switched so as to correspond to the LOD defined in the scene description according to the distance between the object and the object.

Therefore, in the present embodiment, the resolution of the eyes of the viewer is determined based on the visual acuity information of the viewer and the distance information between the viewpoint of the viewer and the three-dimensional object in the virtual three-dimensional space. Calculate the necessary and sufficient LOD selection reference value. Then, the object data for displaying the three-dimensional object is selected based on the calculated LOD selection reference value. By doing so, it is possible to avoid acquiring the data of the three-dimensional object of LOD which is unnecessary for the resolution of the viewer's eyes. Therefore, according to the present embodiment, it is possible to avoid an increase in the amount of distributed data and an increase in the load on the communication network while avoiding the viewer from feeling the deterioration of the image quality. First, the details of the method of calculating the selection reference value of LOD in the present embodiment will be described with reference to FIGS. 8 to 12. 8 to 12 are explanatory views for explaining the outline of the present embodiment.

In detail, FIG. 8 shows changes in the size of the displayed 3D object and the required LOD in accordance with the change in the viewpoint position of the viewer in the 3D virtual space in the present embodiment. It is a figure. As shown in FIG. 8, when the viewer's viewpoint is close to the object 1 in the three-dimensional virtual space, the size of the displayed object 1 becomes large. Therefore, the display LOD of the displayed object 1 is high (for example, when the object 1 is represented by the Point Cloud, the number of points is large), and the object 1 is reproduced with high resolution. Further, when the viewpoint of the viewer is at a medium distance from the object 1, the size of the displayed object 1 is medium. Therefore, the display LOD of the displayed object 1 is medium, and it is reproduced at a medium resolution. Further, when the viewer's viewpoint is far from the object 1, the size of the displayed object 1 becomes small. Therefore, the display LOD of the displayed object 1 becomes low, and it is reproduced at a low resolution.

Therefore, in the present embodiment, a plurality of object data having different LODs are prepared in the content distribution server 10 for the same three-dimensional object. Then, as shown in FIG. 9, the object to be referred to is linked to the LOD selection reference value, which is the distance (LOD distance) between the viewer's viewpoint and the three-dimensional object in the virtual three-dimensional space. The playback device 30a (see FIG. 13) acquires object data of a three-dimensional object according to a table showing data reference information (for example, a reference URL (Uniform Resource Locator) or the like).

The table shown in FIG. 9 is set in consideration of the resolution of the display device 20, and if the resolution of the target display device 20 itself is different, the table will change accordingly. The details required for displaying a three-dimensional object are limited by the resolution (number of pixels) of the display device 20 to be displayed before the displayed size. Therefore, in the present embodiment, the number of points (LOD) associated with the distance (LOD distance) between the viewer's viewpoint and the three-dimensional object in the virtual three-dimensional space is displayed on the display device 20. It is required to satisfy the condition that it is sufficient for the number of pixels in.

Therefore, even if the display size of the three-dimensional object is the same, if the resolution of the display device 20 is 4K (3840 × 2160), the density of points required to display the three-dimensional object (in two dimensions). The density) is 1/4 of that when the resolution of the display device 20 is 8K (7680 × 4320). Therefore, when the resolution of the display device 20 is 4K (3840 × 2160), the density in one dimension along the 20 width direction or the height direction of the display device is 8K (7680 × 4320) for the display device 20. It is halved compared to the case of. For example, as shown in FIG. 10, if the resolution of the display device 20 is different, the number of points required for display on the 4K display device 20 is 8K even if the three-dimensional objects are displayed in the same size. It is 1/2 of the number of points required for display on the display device 20.

Further, in FIG. 11, it is assumed that the entire three-dimensional object having a height h is displayed on the 8K display device 20. At this time, it is assumed that the viewer is located at a distance I in the virtual space so that the viewer's viewing range can include the entire 3D object, and the number of points in the height direction of the 3D object is determined. It is assumed that the number of pixels in the height direction of the display device 20 is 4320, which is the same as the number of pixels. That is, since the number of points and the number of pixels are the same, it can be said that the display of the three-dimensional object has a sufficient point density for display on the 8K display device 20.

In such a situation, the viewer shall move to a position at a distance of I / 2 in the virtual space with respect to the 3D object. At this time, half of the range of the three-dimensional object is displayed so as to occupy the height direction of the display surface of the display device 20. In this case, as shown in FIG. 11, the 3D object displayed with the same object data as the initial situation will have 2160 points in the height direction. However, in such a case, the number of points is insufficient compared to the number of pixels (4320) in the height direction of the display device. Therefore, under the condition of the distance of I / 2 on the virtual space, the three dimensions are met. The object cannot be properly displayed on the 8K display device 20. On the other hand, if the display device 20 has a resolution of 4K (3840 × 2160), the number of points and the number of pixels are the same. Therefore, under the condition of the distance of I / 2 on the virtual space, the three-dimensional object is 4K. It can be appropriately displayed on the display device 20.

That is, if the resolution in the one-dimensional direction of the target display device 20 is halved, the distance (LOD distance) between the viewer's viewpoint and the three-dimensional object in the virtual three-dimensional space is also halved. That is, when the necessary and sufficient resolution of the display device calculated from the viewer's eyesight and distance when viewing on the 8K display device is 4K, the viewer in the virtual three-dimensional space, which is the selection reference value of LOD. It can be seen that the distance (LOD distance) between the viewpoint and the 3D object is half of the value defined in the original scene description.

Further, in the first embodiment described above, the necessary and sufficient resolution for the viewer is calculated based on the viewer's visual acuity (visual acuity information) and the viewer's distance to the display device 20 (distance information). However, the resolution can be considered to be the resolution of the display device 20 that is necessary and sufficient under a predetermined viewing situation.

Therefore, based on the viewer's eyesight (vision information) and the viewer's distance to the display device 20 (distance information), the necessary and sufficient resolution (necessary and sufficient resolution) for the viewer and the resolution of the display device 20 (display). The relationship between the LOD selection reference value defined in the initial scene description and the LOD selection reference value necessary and sufficient for the actual viewer can be shown by the following formula (8).

Although the formula (8) uses the resolution in the width direction of the display device 20, in the present embodiment, the resolution in the height direction may be used instead of the resolution in the width direction.

Here, as an example, the visual acuity of viewing from a position L = 1.44 m away from the 8K display device 20 having a size of W (width) = 1.6 m and H (height) = 0.9 m is 1.0. The ratio of the new LOD selection reference value to the LOD selection reference value defined in the initial scene description between the viewer A and the viewer B with a visual acuity of 0.5 is as follows using the formula (8). Is calculated as follows. In the case of viewer A (visual acuity 1.0), 3819.62 / 7680 ≈ 1/2, and in the case of viewer B (visual acuity 0.5), 1909.81 / 7680 ≈ 1/4.

Therefore, the new LOD selection reference values for viewer A and viewer B calculated based on such an idea are shown in a table as shown in FIG. Here, the new LOD selection reference value for the viewer A is shown as the case (1) replacement distance, and the new LOD selection reference value for the viewer B is shown as the case (2) replacement distance.

Specifically, when the distance between the viewer's viewpoint and the three-dimensional object in the virtual three-dimensional space is 3 m, the H-LOD object, which is initially high LOD object data, is initially used. -When selecting data, according to the present embodiment, for the viewer A, the object data of the M-LOD, which is the object data of the medium LOD, may be selected, and for the viewer B, the object data of the M-LOD may be selected. L-LOD object data, which is low LOD object data, may be selected. By doing so, in the present embodiment, it is possible to avoid acquiring the object data of the three-dimensional object of LOD which is unnecessary for the resolution of the eyes of the viewer. Therefore, according to the present embodiment, it is possible to avoid an increase in the amount of distributed data and an increase in the load on the communication network while avoiding the viewer from feeling the deterioration of the image quality. Hereinafter, the details of the present embodiment will be sequentially described.

<5.2 System configuration>
First, the video content distribution system 1 according to the present embodiment will be described. However, since the distribution system 1 according to the present embodiment is common to the first embodiment except for the following points, the common points are described here. The explanation is omitted, and only the differences are explained.

(Display device 20)
In the present embodiment, the display device 20 is not limited to a two-dimensional display such as a television, a tablet, and a smartphone, and is, for example, an AR (Augmented Reality) glass or HMD worn on the head of a viewer. It may be a wearable device such as (Head Mounted Display). These HMDs and the like may include a positioning sensor (not shown) and a motion sensor (not shown), and in this case, the position of the viewer, the direction and inclination of the body, the movement, the moving speed, and the like can be detected.

<5.3 Configuration of playback device>
Next, the configuration of the reproduction device 30a according to the present embodiment will be described with reference to FIG. 13. FIG. 13 is a block diagram showing a functional configuration example of the reproduction device 30a according to the present embodiment. Since the reproduction device 30a according to the present embodiment is common to the first embodiment except for the following points, the description of the common points will be omitted here, and only the differences will be described.

The present embodiment is different from the first embodiment in that the display control unit 300 of the reproduction device 30a includes the LOD calculation unit (selection reference value calculation unit) 310. The LOD calculation unit 310 is based on the viewer's visual acuity (visual acuity information) and the viewer's distance to the display device 20 (distance information), and based on the necessary and sufficient resolution (user resolution) for the viewer. Calculate a new LOD selection reference value for. Since the calculation method has been described above, the description thereof will be omitted here. Further, in the present embodiment, the acquisition unit 308 has object data (three-dimensional object in the moving image content) having a LOD necessary and sufficient for the resolution of the viewer's eyes based on the newly calculated LOD selection reference value. Content data for displaying)).

<5.4 Information processing method>
The detailed configuration of the reproduction device 30a according to the present embodiment has been described above. Next, the information processing method according to the present embodiment will be described with reference to FIG. FIG. 14 is a diagram showing a flowchart of the information processing method according to the embodiment. As shown in FIG. 14, the distribution processing method according to the present embodiment includes steps S301 to S311. The details of each of these steps according to the present embodiment will be described below. In the following description, only the points different from the above-mentioned first embodiment will be described, and the points common to the first embodiment will be omitted.

Since steps S301 to S306 are the same as steps S101 to S106 of the first embodiment shown in FIG. 5, the description thereof will be omitted here.

The playback device 30a calculates a new LOD selection reference value for the viewer based on the resolution specified in step S306 described above (step S307). Since the calculation method has been described above, the description thereof will be omitted here. Then, the reproduction device 30a acquires the object data (content data) of the three-dimensional object having the LOD corresponding to the LOD selection reference value selected in step S307 described above (step S308).

Since steps S309 to S311 are the same as steps S108 to S110 of the first embodiment shown in FIG. 5, the description thereof will be omitted here.

As described above, in the present embodiment, it is possible to avoid acquiring the object data of the three-dimensional object of LOD which is unnecessary for the resolution of the eyes of the viewer. Therefore, according to the present embodiment, it is possible to avoid an increase in the amount of distributed data and an increase in the load on the communication network while avoiding the viewer from feeling the deterioration of the image quality.

<< 6. Fourth Embodiment >>
<6.1 Outline of the fourth embodiment>
Also in the third embodiment described above, as in the second embodiment described above, in addition to the visual acuity information and the distance information of the viewer, information on the viewing angle indicating the angle with respect to the display device 20 of the viewer is further provided. By using it, the LOD selection reference value can be calculated more accurately. In the present embodiment, since the calculation is performed in consideration of the viewing angle, the LOD selection reference value suitable for the viewer can be calculated with high accuracy, so that the viewer can more effectively deteriorate the image quality. While avoiding feeling, it is possible to avoid an increase in the amount of distributed data and an increase in the load on the communication network.

Specifically, in the present embodiment, as in the second embodiment, for the viewer, based on the viewer's visual acuity (visual acuity information), the viewer's distance to the display device 20 (distance information), and the viewing angle. Calculate the necessary and sufficient resolution (necessary and sufficient resolution). However, in the third embodiment, either the resolution in the width direction or the resolution in the height direction may be used, but in the present embodiment, the viewing angle θ and the viewing angle are the necessary and sufficient resolutions. If β is different from each other, the calculated values will also change significantly, so the larger value of the resolution in the height direction and the resolution in the width direction will be used. Therefore, in the present embodiment, the relationship between the LOD selection reference value defined in the initial scene description and the LOD selection reference value necessary and sufficient for the actual viewer can be shown by the following mathematical formula (9).

<6.2 System configuration and playback device configuration>
First, the video content distribution system 1 and the reproduction device 30a according to the present embodiment will be described, but the distribution system and the reproduction device 30a according to the present embodiment are common to the second and third embodiments described so far. Therefore, the description is omitted here.

<6.3 Information processing method>
Next, the information processing method according to the present embodiment will be described with reference to FIG. FIG. 15 is a diagram showing a flowchart of the information processing method according to the embodiment. As shown in FIG. 15, the information processing method according to the present embodiment includes steps S401 to S414. The details of each of these steps according to the present embodiment will be described below. In the following description, only the points different from the above-mentioned second and third embodiments will be described, and the points common to the second and third embodiments will be omitted.

Specifically, since steps S401 to S409 are the same as steps S201 to S209 of the second embodiment shown in FIG. 7, the description thereof will be omitted here.

Further, since steps S410 to S414 are the same as steps S307 to S311 of the third embodiment shown in FIG. 14, the description thereof will be omitted here.

As described above, in the present embodiment, by using the viewing angle information indicating the angle of the viewer with respect to the display device 20, in addition to the visual acuity information and the distance information of the viewer, the LOD can be performed more accurately. The selection reference value can be calculated. Therefore, according to the present embodiment, since the calculation is performed in consideration of the viewing angle, the LOD selection reference value suitable for the viewer can be calculated with high accuracy, so that the viewer can more effectively calculate. While avoiding the deterioration of image quality, it is possible to avoid an increase in the amount of distributed data and an increase in the load on the communication network.

<< 7. Summary >>
As described above, in each embodiment of the present disclosure, it is possible to avoid acquiring content data having a resolution unnecessary for the resolution of the viewer's eyes, and the content necessary and sufficient for the resolution of the viewer's eyes. You can get the data. Therefore, according to each embodiment of the present disclosure, it is possible to avoid an increase in the amount of distributed data and an increase in the load on the communication network while avoiding the viewer from feeling the deterioration of the image quality. It should be noted that each embodiment of the present disclosure is not limited to application to the provision of entertainment services accompanied by video distribution, but can also be applied to education, medical support, and the like.

<< 8. Hardware configuration example >>
The information processing device such as the reproduction device 30 according to each of the above-described embodiments is realized by, for example, a computer 1000 having a configuration as shown in FIG. Hereinafter, the reproduction device 30 according to the embodiment of the present disclosure will be described as an example. FIG. 16 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the playback device 30 and the like. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each part of the computer 1000 is connected by a bus 1050.

The CPU 1100 operates based on the program stored in the ROM 1300 or the HDD 1400, and controls each part. For example, the CPU 1100 expands the program stored in the ROM 1300 or the HDD 1400 into the RAM 1200, and executes processing corresponding to various programs.

The ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started, a program depending on the hardware of the computer 1000, and the like.

The HDD 1400 is a computer-readable recording medium that non-temporarily records a program executed by the CPU 1100 and data used by such a program. Specifically, the HDD 1400 is a recording medium for recording an information processing program according to the present disclosure, which is an example of program data 1450.

The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (for example, the Internet). For example, the CPU 1100 receives data from another device or transmits data generated by the CPU 1100 to another device via the communication interface 1500.

The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input / output device 1650 such as a keyboard, a mouse, and a microphone (microphone) via the input / output interface 1600. Further, the CPU 1100 transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. Further, the input / output interface 1600 may function as a media interface for reading a program or the like recorded on a predetermined recording medium (media). The media includes, for example, an optical recording medium such as a DVD (Digital Versaille Disc), a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magnet-Optical disc), a tape medium, a magnetic recording medium, a semiconductor memory, or the like. Is.

For example, when the computer 1000 functions as the reproduction device 30 according to the embodiment of the present disclosure, the CPU 1100 of the computer 1000 realizes the functions of the display control unit 310 and the like by executing the program stored in the RAM 1200. Further, the processing program and the like according to the present disclosure are stored in the HDD 1400. The CPU 1100 reads the program data 1450 from the HDD 1400 and executes the program, but as another example, these programs may be acquired from another device via the external network 1550.

Further, the information processing device according to the present embodiment may be applied to a system including a plurality of devices, which is premised on connection to a network (or communication between each device), such as cloud computing. .. That is, the information processing device according to the present embodiment described above can be realized as an information processing system according to the present embodiment by, for example, a plurality of devices.

The above is an example of the hardware configuration of the playback device 30 and the like. Each of the above-mentioned components may be configured by using general-purpose members, or may be configured by hardware specialized for the function of each component. Such a configuration may be appropriately modified depending on the technical level at the time of implementation.

<< 9. Application example >>
The technology according to the present disclosure can be applied to various products. For example, the techniques according to the present disclosure may be applied to an endoscopic surgery system.

FIG. 17 is a diagram showing an example of a schematic configuration of an endoscopic surgery system 5000 to which the technique according to the present disclosure can be applied. FIG. 17 shows a surgeon (doctor) 5067 performing surgery on patient 5071 on patient bed 5069 using the endoscopic surgery system 5000. As shown in the figure, the endoscopic surgery system 5000 includes an endoscope 5001, other surgical tools 5017, a support arm device 5027 for supporting the endoscope 5001, and various devices for endoscopic surgery. It is composed of a cart 5037 and a cart 5037.

In endoscopic surgery, instead of cutting and opening the abdominal wall, multiple tubular laparotomy instruments called trocca 5025a to 5025d are punctured into the abdominal wall. Then, from the trocca 5025a to 5025d, the lens barrel 5003 of the endoscope 5001 and other surgical tools 5017 are inserted into the body cavity of the patient 5071. In the illustrated example, as other surgical tools 5017, a pneumoperitoneum tube 5019, an energy treatment tool 5021 and forceps 5023 are inserted into the body cavity of patient 5071. Further, the energy treatment tool 5021 is a treatment tool for incising and peeling a tissue, sealing a blood vessel, or the like by using a high frequency current or ultrasonic vibration. However, the surgical tool 5017 shown in the illustration is merely an example, and as the surgical tool 5017, various surgical tools generally used in endoscopic surgery such as a sword and a retractor may be used.

The image of the surgical site in the body cavity of the patient 5071 taken by the endoscope 5001 is displayed on the display device 5041. The surgeon 5067 performs a procedure such as excising the affected area by using the energy treatment tool 5021 or the forceps 5023 while viewing the image of the surgical site displayed on the display device 5041 in real time. Although not shown, the pneumoperitoneum tube 5019, the energy treatment tool 5021, and the forceps 5023 are supported by the operator 5067, an assistant, or the like during the operation.

(Support arm device)
The support arm device 5027 includes an arm portion 5031 extending from the base portion 5029. In the illustrated example, the arm portion 5031 is composed of joint portions 5033a, 5033b, 5033c, and links 5035a, 5035b, and is driven by control from the arm control device 5045. The endoscope 5001 is supported by the arm portion 5031, and its position and posture are controlled. Thereby, the stable position fixing of the endoscope 5001 can be realized.

(Endoscope)
The endoscope 5001 is composed of a lens barrel 5003 in which a region having a predetermined length from the tip is inserted into the body cavity of the patient 5071, and a camera head 5005 connected to the base end of the lens barrel 5003. In the illustrated example, the endoscope 5001 configured as a so-called rigid mirror having a rigid barrel 5003 is illustrated, but the endoscope 5001 is configured as a so-called flexible mirror having a flexible barrel 5003. May be good.

An opening in which an objective lens is fitted is provided at the tip of the lens barrel 5003. A light source device 5043 is connected to the endoscope 5001, and the light generated by the light source device 5043 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 5003, and is an objective. It is irradiated toward the observation target in the body cavity of the patient 5071 through the lens. The endoscope 5001 may be a direct endoscope, a perspective mirror, or a side endoscope.

An optical system and an image pickup element are provided inside the camera head 5005, and the reflected light (observation light) from the observation target is focused on the image pickup element by the optical system. The observation light is photoelectrically converted by the image pickup device, and an electric signal corresponding to the observation light, that is, an image signal corresponding to the observation image is generated. The image signal is transmitted as RAW data to the camera control unit (CCU: Camera Control Unit) 5039. The camera head 5005 is equipped with a function of adjusting the magnification and the focal length by appropriately driving the optical system thereof.

Note that, for example, in order to support stereoscopic viewing (3D display) and the like, the camera head 5005 may be provided with a plurality of image pickup elements. In this case, a plurality of relay optical systems are provided inside the lens barrel 5003 in order to guide the observation light to each of the plurality of image pickup elements.

(Various devices mounted on the cart)
The CCU 5039 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and the like, and comprehensively controls the operations of the endoscope 5001 and the display device 5041. Specifically, the CCU 5039 performs various image processing for displaying an image based on the image signal, such as a development process (demosaic process), on the image signal received from the camera head 5005. The CCU 5039 provides the image signal subjected to the image processing to the display device 5041. Further, the CCU 5039 transmits a control signal to the camera head 5005 and controls the driving thereof. The control signal may include information about imaging conditions such as magnification and focal length.

The display device 5041 displays an image based on the image signal processed by the CCU 5039 under the control of the CCU 5039. When the endoscope 5001 is compatible with high-resolution shooting such as 4K (horizontal pixel number 3840 x vertical pixel number 2160) or 8K (horizontal pixel number 7680 x vertical pixel number 4320), and / or 3D display. As the display device 5041, a display device capable of displaying a high resolution and / or a device capable of displaying in 3D can be used. When a display device 5041 having a size of 55 inches or more is used for high-resolution shooting such as 4K or 8K, a further immersive feeling can be obtained. Further, a plurality of display devices 5041 having different resolutions and sizes may be provided depending on the application.

The light source device 5043 is composed of, for example, a light source such as an LED (Light Emitting Diode), and supplies irradiation light for photographing the surgical site to the endoscope 5001.

The arm control device 5045 is configured by a processor such as a CPU, and operates according to a predetermined program to control the drive of the arm portion 5031 of the support arm device 5027 according to a predetermined control method.

The input device 5047 is an input interface for the endoscopic surgery system 5000. The user can input various information and input instructions to the endoscopic surgery system 5000 via the input device 5047. For example, the user inputs various information related to the surgery, such as physical information of the patient and information about the surgical procedure, via the input device 5047. Further, for example, the user is instructed to drive the arm portion 5031 via the input device 5047, or is instructed to change the imaging conditions (type of irradiation light, magnification, focal length, etc.) by the endoscope 5001. , Instructions to drive the energy treatment tool 5021, etc. are input.

The type of the input device 5047 is not limited, and the input device 5047 may be various known input devices. As the input device 5047, for example, a mouse, a keyboard, a touch panel, a switch, a foot switch 5057 and / or a lever and the like can be applied. When a touch panel is used as the input device 5047, the touch panel may be provided on the display surface of the display device 5041.

Alternatively, the input device 5047 is a device worn by the user, such as a glasses-type wearable device or an HMD (Head Mounted Display), and various inputs are made according to the user's gesture and line of sight detected by these devices. Is done. Further, the input device 5047 includes a camera capable of detecting the movement of the user, and various inputs are performed according to the gesture and the line of sight of the user detected from the image captured by the camera. Further, the input device 5047 includes a microphone capable of picking up the voice of the user, and various inputs are performed by voice via the microphone. In this way, the input device 5047 is configured to be able to input various information in a non-contact manner, so that a user who belongs to a clean area (for example, an operator 5067) can operate a device belonging to the unclean area in a non-contact manner. Is possible. In addition, the user can operate the device without taking his / her hand off the surgical tool that he / she has, which improves the convenience of the user.

The treatment tool control device 5049 controls the drive of the energy treatment tool 5021 for cauterizing tissue, incising, sealing a blood vessel, or the like. The pneumoperitoneum device 5051 gas in the body cavity of the patient 5071 via the pneumoperitoneum tube 5019 in order to inflate the body cavity of the patient 5071 for the purpose of securing the field of view by the endoscope 5001 and securing the work space of the operator. Is sent. The recorder 5053 is a device capable of recording various information related to surgery. The printer 5055 is a device capable of printing various information related to surgery in various formats such as text, images, and graphs.

Hereinafter, a particularly characteristic configuration of the endoscopic surgery system 5000 will be described in more detail.

(Support arm device)
The support arm device 5027 includes a base portion 5029 which is a base, and an arm portion 5031 extending from the base portion 5029. In the illustrated example, the arm portion 5031 is composed of a plurality of joint portions 5033a, 5033b, 5033c and a plurality of links 5035a, 5035b connected by the joint portions 5033b, but in FIG. 17, for the sake of simplicity. , The configuration of the arm portion 5031 is simplified and illustrated. Actually, the shapes, numbers and arrangements of the joint portions 5033a to 5033c and the links 5035a and 5035b, the direction of the rotation axis of the joint portions 5033a to 5033c, and the like are appropriately set so that the arm portion 5031 has a desired degree of freedom. obtain. For example, the arm portion 5031 may be preferably configured to have more than 6 degrees of freedom. As a result, the endoscope 5001 can be freely moved within the movable range of the arm portion 5031, so that the lens barrel 5003 of the endoscope 5001 can be inserted into the body cavity of the patient 5071 from a desired direction. It will be possible.

Actuators are provided in the joint portions 5033a to 5033c, and the joint portions 5033a to 5033c are configured to be rotatable around a predetermined rotation axis by driving the actuator. By controlling the drive of the actuator by the arm control device 5045, the rotation angles of the joint portions 5033a to 5033c are controlled, and the drive of the arm portion 5031 is controlled. Thereby, control of the position and posture of the endoscope 5001 can be realized. At this time, the arm control device 5045 can control the drive of the arm unit 5031 by various known control methods such as force control or position control.

For example, when the operator 5067 appropriately inputs an operation via the input device 5047 (including the foot switch 5057), the drive of the arm unit 5031 is appropriately controlled by the arm control device 5045 according to the operation input. The position and orientation of the endoscope 5001 may be controlled. By this control, the endoscope 5001 at the tip of the arm portion 5031 can be moved from an arbitrary position to an arbitrary position, and then fixedly supported at the position after the movement. The arm portion 5031 may be operated by a so-called master slave method. In this case, the arm portion 5031 can be remotely controlled by the user via an input device 5047 installed at a location away from the operating room.

When force control is applied, the arm control device 5045 receives an external force from the user, and the actuators of the joint portions 5033a to 5033c are arranged so that the arm portion 5031 moves smoothly according to the external force. So-called power assist control for driving may be performed. As a result, when the user moves the arm portion 5031 while directly touching the arm portion 5031, the arm portion 5031 can be moved with a relatively light force. Therefore, the endoscope 5001 can be moved more intuitively and with a simpler operation, and the convenience of the user can be improved.

Here, in general, in endoscopic surgery, the endoscope 5001 was supported by a doctor called a scopist. On the other hand, by using the support arm device 5027, the position of the endoscope 5001 can be more reliably fixed without human intervention, so that an image of the surgical site can be stably obtained. , It becomes possible to perform surgery smoothly.

The arm control device 5045 does not necessarily have to be provided on the cart 5037. Further, the arm control device 5045 does not necessarily have to be one device. For example, the arm control device 5045 may be provided in each of the joint portions 5033a to 5033c of the arm portion 5031 of the support arm device 5027, and the plurality of arm control devices 5045 cooperate with each other to drive the arm portion 5031. Control may be realized.

(Light source device)
The light source device 5043 supplies the endoscope 5001 with irradiation light for photographing the surgical site. The light source device 5043 is composed of, for example, an LED, a laser light source, or a white light source composed of a combination thereof. At this time, when the white light source is configured by the combination of the RGB laser light sources, the output intensity and the output timing of each color (each wavelength) can be controlled with high accuracy, so that the white balance of the captured image in the light source device 5043 can be controlled. Can be adjusted. Further, in this case, the laser light from each of the RGB laser light sources is irradiated to the observation target in a time-division manner, and the drive of the image sensor of the camera head 5005 is controlled in synchronization with the irradiation timing to correspond to each of RGB. It is also possible to capture the image in a time-division manner. According to this method, a color image can be obtained without providing a color filter in the image pickup device.

Further, the drive of the light source device 5043 may be controlled so as to change the intensity of the output light at predetermined time intervals. By controlling the drive of the image sensor of the camera head 5005 in synchronization with the timing of the change of the light intensity to acquire an image in time division and synthesizing the image, so-called high dynamic without blackout and overexposure. Range images can be generated.

Further, the light source device 5043 may be configured to be able to supply light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, by utilizing the wavelength dependence of light absorption in body tissue, the surface layer of the mucous membrane is irradiated with light in a narrower band than the irradiation light (that is, white light) during normal observation. A so-called narrow band imaging (Narrow Band Imaging) is performed in which a predetermined tissue such as a blood vessel is photographed with high contrast. Alternatively, in special light observation, fluorescence observation may be performed in which an image is obtained by fluorescence generated by irradiating with excitation light. In fluorescence observation, the body tissue is irradiated with excitation light to observe the fluorescence from the body tissue (autofluorescence observation), or a reagent such as indocyanine green (ICG) is locally injected into the body tissue and the body tissue is injected. An excitation light corresponding to the fluorescence wavelength of the reagent may be irradiated to obtain a fluorescence image. The light source device 5043 may be configured to be capable of supplying narrowband light and / or excitation light corresponding to such special light observation.

(Camera head and CCU)
The functions of the camera head 5005 and the CCU 5039 of the endoscope 5001 will be described in more detail with reference to FIG. FIG. 18 is a block diagram showing an example of the functional configuration of the camera head 5005 and CCU5039 shown in FIG.

Referring to FIG. 18, the camera head 5005 has a lens unit 5007, an image pickup unit 5009, a drive unit 5011, a communication unit 5013, and a camera head control unit 5015 as its functions. Further, the CCU 5039 has a communication unit 5059, an image processing unit 5061, and a control unit 5063 as its functions. The camera head 5005 and the CCU 5039 are bidirectionally connected by a transmission cable 5065 so as to be communicable.

First, the functional configuration of the camera head 5005 will be described. The lens unit 5007 is an optical system provided at a connection portion with the lens barrel 5003. The observation light taken in from the tip of the lens barrel 5003 is guided to the camera head 5005 and incident on the lens unit 5007. The lens unit 5007 is configured by combining a plurality of lenses including a zoom lens and a focus lens. The optical characteristics of the lens unit 5007 are adjusted so as to collect the observation light on the light receiving surface of the image pickup element of the image pickup unit 5009. Further, the zoom lens and the focus lens are configured so that their positions on the optical axis can be moved in order to adjust the magnification and the focus of the captured image.

The image pickup unit 5009 is composed of an image pickup element and is arranged after the lens unit 5007. The observation light that has passed through the lens unit 5007 is focused on the light receiving surface of the image pickup device, and an image signal corresponding to the observation image is generated by photoelectric conversion. The image signal generated by the image pickup unit 5009 is provided to the communication unit 5013.

As the image pickup element constituting the image pickup unit 5009, for example, a CMOS (Complementary Metal Oxide Semiconductor) type image sensor having a Bayer array and capable of color photographing is used. As the image pickup device, for example, an image pickup device capable of capturing a high-resolution image of 4K or higher may be used. By obtaining the image of the surgical site with high resolution, the surgeon 5067 can grasp the state of the surgical site in more detail, and the operation can proceed more smoothly.

Further, the image pickup elements constituting the image pickup unit 5009 are configured to have a pair of image pickup elements for acquiring image signals for the right eye and the left eye corresponding to 3D display, respectively. The 3D display enables the surgeon 5067 to more accurately grasp the depth of the living tissue in the surgical site. When the image pickup unit 5009 is composed of a multi-plate type, a plurality of lens units 5007 are also provided corresponding to each image pickup element.

Further, the image pickup unit 5009 does not necessarily have to be provided on the camera head 5005. For example, the image pickup unit 5009 may be provided inside the lens barrel 5003 immediately after the objective lens.

The drive unit 5011 is composed of an actuator, and the zoom lens and the focus lens of the lens unit 5007 are moved by a predetermined distance along the optical axis under the control of the camera head control unit 5015. As a result, the magnification and focus of the image captured by the image pickup unit 5009 can be adjusted as appropriate.

The communication unit 5013 is composed of a communication device for transmitting and receiving various information to and from the CCU 5039. The communication unit 5013 transmits the image signal obtained from the image pickup unit 5009 as RAW data to the CCU 5039 via the transmission cable 5065. At this time, in order to display the captured image of the surgical site with low latency, it is preferable that the image signal is transmitted by optical communication. At the time of surgery, the surgeon 5067 performs the surgery while observing the condition of the affected area with the captured image, so for safer and more reliable surgery, the moving image of the surgical site is displayed in real time as much as possible. This is because it is required. When optical communication is performed, the communication unit 5013 is provided with a photoelectric conversion module that converts an electric signal into an optical signal. The image signal is converted into an optical signal by the photoelectric conversion module, and then transmitted to the CCU 5039 via the transmission cable 5065.

Further, the communication unit 5013 receives a control signal for controlling the drive of the camera head 5005 from the CCU 5039. The control signal includes, for example, information to specify the frame rate of the captured image, information to specify the exposure value at the time of imaging, and / or information to specify the magnification and focus of the captured image. Contains information about the condition. The communication unit 5013 provides the received control signal to the camera head control unit 5015. The control signal from the CCU 5039 may also be transmitted by optical communication. In this case, the communication unit 5013 is provided with a photoelectric conversion module that converts an optical signal into an electric signal, and the control signal is converted into an electric signal by the photoelectric conversion module and then provided to the camera head control unit 5015.

The image pickup conditions such as the frame rate, the exposure value, the magnification, and the focal point are automatically set by the control unit 5063 of the CCU 5039 based on the acquired image signal. That is, the so-called AE (Auto Exposure) function, AF (Auto Focus) function, and AWB (Auto White Balance) function are mounted on the endoscope 5001.

The camera head control unit 5015 controls the drive of the camera head 5005 based on the control signal from the CCU 5039 received via the communication unit 5013. For example, the camera head control unit 5015 controls the drive of the image pickup element of the image pickup unit 5009 based on the information to specify the frame rate of the image pickup image and / or the information to specify the exposure at the time of image pickup. Further, for example, the camera head control unit 5015 appropriately moves the zoom lens and the focus lens of the lens unit 5007 via the drive unit 5011 based on the information that the magnification and the focus of the captured image are specified. The camera head control unit 5015 may further have a function of storing information for identifying the lens barrel 5003 and the camera head 5005.

By arranging the configuration of the lens unit 5007, the image pickup unit 5009, and the like in a sealed structure having high airtightness and waterproofness, the camera head 5005 can be made resistant to autoclave sterilization.

Next, the functional configuration of CCU5039 will be described. The communication unit 5059 is configured by a communication device for transmitting and receiving various information to and from the camera head 5005. The communication unit 5059 receives an image signal transmitted from the camera head 5005 via the transmission cable 5065. At this time, as described above, the image signal can be suitably transmitted by optical communication. In this case, corresponding to optical communication, the communication unit 5059 is provided with a photoelectric conversion module that converts an optical signal into an electric signal. The communication unit 5059 provides the image processing unit 5061 with an image signal converted into an electric signal.

Further, the communication unit 5059 transmits a control signal for controlling the drive of the camera head 5005 to the camera head 5005. The control signal may also be transmitted by optical communication.

The image processing unit 5061 performs various image processing on the image signal which is the RAW data transmitted from the camera head 5005. The image processing includes, for example, development processing, high image quality processing (band enhancement processing, super-resolution processing, NR (Noise Reduction) processing and / or camera shake correction processing, etc.), and / or enlargement processing (electronic zoom processing). Etc., various known signal processing is included. Further, the image processing unit 5061 performs detection processing on the image signal for performing AE, AF and AWB.

The image processing unit 5061 is composed of a processor such as a CPU or GPU, and the processor operates according to a predetermined program, so that the above-mentioned image processing and detection processing can be performed. When the image processing unit 5061 is composed of a plurality of GPUs, the image processing unit 5061 appropriately divides the information related to the image signal and performs image processing in parallel by the plurality of GPUs.

The control unit 5063 performs various controls regarding the imaging of the surgical site by the endoscope 5001 and the display of the captured image. For example, the control unit 5063 generates a control signal for controlling the drive of the camera head 5005. At this time, when the imaging condition is input by the user, the control unit 5063 generates a control signal based on the input by the user. Alternatively, when the endoscope 5001 is equipped with an AE function, an AF function, and an AWB function, the control unit 5063 has an optimum exposure value, a focal length, and an optimum exposure value according to the result of detection processing by the image processing unit 5061. The white balance is calculated appropriately and a control signal is generated.

Further, the control unit 5063 causes the display device 5041 to display the image of the surgical unit based on the image signal processed by the image processing unit 5061. At this time, the control unit 5063 recognizes various objects in the surgical unit image by using various image recognition techniques. For example, the control unit 5063 detects a surgical tool such as forceps, a specific biological part, bleeding, a mist when using the energy treatment tool 5021, etc. by detecting the shape, color, etc. of the edge of the object included in the surgical site image. Can be recognized. When displaying the image of the surgical site on the display device 5041, the control unit 5063 uses the recognition result to superimpose and display various surgical support information on the image of the surgical site. By superimposing the surgical support information and presenting it to the surgeon 5067, it becomes possible to proceed with the surgery more safely and surely.

The transmission cable 5065 connecting the camera head 5005 and the CCU 5039 is an electric signal cable compatible with electric signal communication, an optical fiber compatible with optical communication, or a composite cable thereof.

Here, in the illustrated example, the communication is performed by wire using the transmission cable 5065, but the communication between the camera head 5005 and the CCU 5039 may be performed wirelessly. When the communication between the two is performed wirelessly, it is not necessary to lay the transmission cable 5065 in the operating room, so that the situation where the movement of the medical staff in the operating room is hindered by the transmission cable 5065 can be solved.

The above is an example of the endoscopic surgery system 5000 to which the technique according to the present disclosure can be applied. Although the endoscopic surgery system 5000 has been described here as an example, the system to which the technique according to the present disclosure can be applied is not limited to such an example. For example, the technique according to the present disclosure may be applied to a flexible endoscopic system for examination or a microsurgery system.

<< 10. Supplement >>
It should be noted that the embodiments of the present disclosure described above are, for example, a processing method executed by the reproduction device 30 as described above, a program for operating the device, and a non-temporary tangible program in which the program is recorded. Can include media. Further, the program may be distributed via a communication line (including wireless communication) such as the Internet.

Further, each step in the information processing method of the embodiment of the present disclosure described above does not necessarily have to be processed in the order described. For example, each step may be processed in an appropriately reordered manner. Further, each step may be partially processed in parallel or individually instead of being processed in chronological order. Further, the processing of each step does not necessarily have to be processed according to the described method, and may be processed by another method, for example, by another functional unit.

Although the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that anyone with ordinary knowledge in the technical field of the present disclosure may come up with various modifications or modifications within the scope of the technical ideas set forth in the claims. Is, of course, understood to belong to the technical scope of the present disclosure.

Further, the effects described in the present specification are merely explanatory or exemplary and are not limited. That is, the technique according to the present disclosure may have other effects apparent to those skilled in the art from the description herein, in addition to or in place of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1)
An information processing device including an acquisition unit that acquires video content data to be played back to the user in real time according to the user's visual acuity information.
(2)
Further provided is a calculation unit that calculates the user resolution for the user in real time based on the user's visual acuity information, the display unit information regarding the display unit that displays the moving image content, and the position information of the user with respect to the display unit. ,
The acquisition unit acquires the moving image content data based on the user resolution.
The information processing device according to (1) above.
(3)
The display unit information includes information on the height and width of the display unit.
The calculation unit calculates the user resolution in the height direction and the user resolution in the width direction.
The information processing device according to (2) above.
(4)
Further, a comparison unit for comparing the user resolution and the resolution of the display unit is provided.
The acquisition unit acquires the video content data based on the comparison result.
The information processing device according to (2) or (3) above.
(5)
The information processing device according to any one of (2) to (4) above, wherein the calculation unit calculates the user resolution based on the distance of the user to the display unit.
(6)
The information processing device according to (5) above, wherein the calculation unit calculates the user resolution based on the angle of the user with respect to the display unit.
(7)
The acquisition unit acquires the video content data for reproducing an object in the virtual space in the video content based on the user resolution.
The information processing apparatus according to any one of (2) to (6) above.
(8)
Further, a selection reference value calculation unit for newly calculating a selection reference value for selecting the moving image content data based on the user resolution is provided.
The information processing device according to (7) above.
(9)
The information processing apparatus according to any one of (2) to (8) above, further comprising a position information acquisition unit for acquiring the position information.
(10)
The acquisition unit acquires the video content data to be reproduced for the plurality of users according to the visual acuity information of the plurality of users.
The information processing apparatus according to any one of (1) to (9) above.
(11)
Further provided with a decoding unit for decoding the moving image content data.
The information processing apparatus according to any one of (1) to (9) above.
(12)
The information processing apparatus according to any one of (2) to (9) above, further comprising the display unit.
(13)
The information processing device according to any one of (1) to (12) above, wherein the moving image content data is all-sky video data or virtual space video data.
(14)
Including acquiring video content data to be played back to the user in real time according to the user's visual acuity information.
Information processing method.

1 Distribution system 10 Content distribution server 20 Display device 30, 30a Playback device 40 Communication network 300 Display control unit 302 Location information acquisition unit 304 Calculation unit 306 Comparison unit 308 Acquisition unit 310 LOD calculation unit 320 Decoding block 322 Processing unit 324 Decoding unit 326 Display information generation unit 330 Main control unit 340 Storage unit 350 Transmission / reception unit

Claims (14)

1. An information processing device equipped with an acquisition unit that acquires video content data to be played back to the user in real time according to the user's visual acuity information.
2. Further provided is a calculation unit that calculates the user resolution for the user in real time based on the user's visual acuity information, the display unit information regarding the display unit that displays the moving image content, and the position information of the user with respect to the display unit. ,
   The acquisition unit acquires the moving image content data based on the user resolution.
   The information processing apparatus according to claim 1.
3. The display unit information includes information on the height and width of the display unit.
   The calculation unit calculates the user resolution in the height direction and the user resolution in the width direction.
   The information processing apparatus according to claim 2.
4. Further, a comparison unit for comparing the user resolution and the resolution of the display unit is provided.
   The acquisition unit acquires the video content data based on the comparison result.
   The information processing apparatus according to claim 2.
5. The information processing device according to claim 2, wherein the calculation unit calculates the user resolution based on the distance of the user to the display unit.
6. The information processing device according to claim 5, wherein the calculation unit calculates the user resolution based on the angle of the user with respect to the display unit.
7. The acquisition unit acquires the video content data for reproducing an object in the virtual space in the video content based on the user resolution.
   The information processing apparatus according to claim 2.
8. Further, a selection reference value calculation unit for newly calculating a selection reference value for selecting the moving image content data based on the user resolution is provided.
   The information processing apparatus according to claim 7.
9. The information processing device according to claim 2, further comprising a position information acquisition unit for acquiring the position information.
10. The acquisition unit acquires the video content data to be reproduced for the plurality of users according to the visual acuity information of the plurality of users.
    The information processing apparatus according to claim 1.
11. Further provided with a decoding unit for decoding the moving image content data.
    The information processing apparatus according to claim 1.
12. The information processing device according to claim 2, further comprising the display unit.
13. The information processing device according to claim 1, wherein the moving image content data is all-sky video data or virtual space video data.
14. Including acquiring video content data to be played back to the user in real time according to the user's visual acuity information.
    Information processing method.

Images