ES2642263T3 - Virtual reality content control - Google Patents

Description

5

10

fifteen

twenty

25

30

35

40

Four. Five

fifty

55

60

65

DESCRIPTION

Virtual reality content control Technology field

The embodiments of the present invention relate to controlling the progress of virtual reality content. Such virtual reality content can be visualized, for example, using a head mounted display device.

Background

Virtual reality provides a computer simulated environment that simulates the presence of an observer in that environment. An observer can experience, for example, a virtual reality environment using a head-mounted display device comprising a stereoscopic screen.

WO 2013/036237 discloses a method comprising detecting what a user is looking at on a display screen and responding by modifying the reproduction of audio or visual media.

US 2006/0256133 discloses a system that determines whether a user's gaze falls within a predetermined spatial limit of an advertising display area and reproduces a video-based advertisement in response to such a determination.

US 2010/0079508 discloses an electronic device that has look detection capabilities that allow the device to detect when a user is looking at the device.

WO 2013/095416 discloses a processor that determines the characteristics of the interaction of a user with a scene of a streaming video and responds.

Brief Summary

According to various embodiments of the invention, but not necessarily all, a method is provided, comprising: determining at least a first region of interest at a first instant in a virtual virtual reality content that is configured to extend beyond a field of view of the observer when the visual virtual reality content is displayed; monitor, while an observer is visualizing the visual virtual reality content, if a defined proportion of the observer's field of vision coincides with the first region of interest in the visual virtual reality content; monitor the progress of the visual virtual reality content based on whether the defined proportion of the observer's field of vision coincides with the first region of interest determined in the visual virtual reality content; determine a new second region of interest, located outside the observer's field of vision while the observer is visualizing the content of visual virtual reality, in a second instant in the content of visual virtual reality; determine, while the observer is visualizing the visual virtual reality content, that the defined proportion of the observer's field of vision does not match the new second region of interest in the visual virtual reality content; and stop the advance of at least a portion of the visual virtual reality content, which comprises the new second region of interest, based on the determination that the defined proportion of the observer's field of vision does not match the new second region of interest in the content of visual virtual reality.

In accordance with various embodiments of the invention, but not necessarily all, computer program code is provided which, when performed by at least one processor, causes at least the following to be done: determining at least a first region of interest in a first instant in a visual virtual reality content that is configured to extend beyond an observer's field of vision when visual virtual reality content is displayed; monitor, while an observer is visualizing the visual virtual reality content, if a defined proportion of the observer's field of vision coincides with the first region of interest in the visual virtual reality content; monitor the progress of the visual virtual reality content based on whether the defined proportion of the observer's field of vision coincides with the first region of interest determined in the visual virtual reality content; determine a new second region of interest, located outside the observer's field of vision while the observer is visualizing the content of visual virtual reality, in a second instant in the content of visual virtual reality; determine, while the observer is visualizing the visual virtual reality content, that the defined proportion of the observer's field of vision does not match the new second region of interest in the visual virtual reality content; and stop the advance of at least a portion of the visual virtual reality content, which comprises the new second region of interest, based on the determination that the defined proportion of the observer's field of vision does not match the new second region of interest in the content of visual virtual reality.

The computer program code can be provided by a computer program. The computer program can be stored in a non-transient computer readable medium.

5

10

fifteen

twenty

25

30

35

40

Four. Five

fifty

55

60

65

In accordance with various embodiments of the invention, but not necessarily all, an apparatus is provided, comprising: means for determining at least a first region of interest in visual virtual reality content that is configured to extend beyond a field of vision of the observer when the visual virtual reality content is displayed; means for monitoring, while an observer is visualizing the visual virtual reality content, if a defined proportion of an observer's field of vision coincides with the first region of interest in the visual virtual reality content; and means for controlling the progress of the visual virtual reality content based on whether the at least a defined portion of the observer's field of vision coincides with the at least one region of interest determined in the visual virtual reality content; means for determining a new second region of interest, located outside the observer's field of vision while the observer is visualizing the content of visual virtual reality, in a second instant in the content of visual virtual reality; means for determining, while the observer is visualizing the visual virtual reality content, that the defined proportion of the observer's field of vision does not match the new second region of interest in the visual virtual reality content; and means to stop the advance of at least a portion of the visual virtual reality content, which comprises the new second region of interest, based on the determination that the defined proportion of the observer's field of vision does not match the new second region of Interest in the proportion of visual virtual reality content of an observer's field of vision coincides with the at least one region of interest identified by the at least one identifier.

According to various embodiments of the invention, but not necessarily all, examples are provided as claimed in the appended claims.

Short description

For a better understanding of various examples described in the detailed description, reference will now be made by way of example only to the accompanying drawings in which:

Figure 1 illustrates a first apparatus in the form of a chip or chipset;

Figure 2 illustrates a second apparatus in the form of an electronic device comprising the first apparatus illustrated in Figure 1;

Figure 3 illustrates a third apparatus in the form of a chip / chipset;

Figure 4 illustrates a fourth apparatus in the form of an electronic device comprising the apparatus illustrated in Figure 3;

Figure 5 illustrates a flow chart of a first method performed by the first and / or second apparatus;

Figure 6 illustrates a flow chart of a second method performed by the third and / or fourth apparatus;

Figures 7 and 8 illustrate schematic details in which virtual reality content is being created;

Figure 9 illustrates a region of interest in a first instant in a virtual virtual reality content;

Figure 10 illustrates multiple regions of interest in a second instant in the visual virtual reality content;

Figure 11 illustrates a region of interest in a third instant in the visual virtual reality content;

Figures 12A and 12B illustrate two different examples of a defined proportion of an observer's field of vision;

Figure 13 illustrates a schematic diagram of an observer who visualizes the visual virtual reality content in the first instant in the visual virtual reality content, as schematically illustrated in Figure 9;

Figure 14 illustrates an observer who visualizes the visual virtual reality content in the third instant in the visual virtual reality content, as schematically illustrated in Figure 11; Y

Figure 15 schematically illustrates the observer who visualizes the visual virtual reality content in the third instant in the visual virtual reality content, as schematically illustrated in Figure 11, where the user has moved his field of vision in relation to the Figure 14.

Detailed description

The embodiments of the invention relate to controlling the progress of visual virtual reality content based on whether at least a defined proportion of a user's field of vision coincides with at least one predefined region of interest in the visual virtual reality content.

Figure 1 illustrates an apparatus 10 that can be a chip or a chipset. The apparatus 10 can be part of an information device such as that illustrated in Figure 2.

The apparatus 10 comprises at least one processor 12 and at least one memory 14. A single processor 12 and a single memory 14 are shown in Figure 1 and are analyzed below simply for purposes of illustration.

The processor 12 is configured to read from and write to the memory 14. The processor 12 may comprise an output interface through which data and / or commands are emitted by the processor 12 and an input interface through which data is entered and / o commands to the processor 12.

5

10

fifteen

twenty

25

30

35

40

Four. Five

fifty

55

60

65

The memory 14 is illustrated by storing a computer program 17 comprising instructions / computer program code 18 that control the operation of the apparatus 10 when loaded into the processor 12. The processor 12, reading the memory 14, can load and execute the software code. computer program 18. The computer program code 18 provides the logic and routines that enable the apparatus 10 to perform the methods illustrated in Figure 5 and described below. In this sense, processor 12, memory 14 and computer program code 18 provide means for performing the methods illustrated in Figure 5 and described below.

Although memory 14 is illustrated as a single component in Figure 1, it can be implemented as one or more separate components, some or all of which may be integrated and / or removable and / or can provide dynamic / permanent / semi-cache storage -permanent.

The computer program code 18 can reach the apparatus 10 by any suitable delivery mechanism 30. The delivery mechanism 30 may be, for example, a non-transient computer readable storage medium such as an optical disk or a memory card. The delivery mechanism 30 may be a signal configured to reliably transfer the computer program code 18. The apparatus 10 can cause the propagation or transmission of the computer program code 18 as a computer data signal.

Figure 2 illustrates a second apparatus 20 in the form of an informatic device comprising the first apparatus 10. The second apparatus 20 may be, for example, a personal computer.

In the example illustrated in Figure 2, processor 12 and memory 14 are co-located in a housing / body 27. The second apparatus 20 may additionally comprise a screen and one or more user input devices (such as a mouse informatic), for example.

Elements 12 and 14 are operationally coupled and there may be any number or combination of intermediate elements between them (including non-intermediate elements).

In Figure 2, memory 14 is illustrated by storing virtual reality content data 21. Virtual reality content data 21 may comprise, for example, audiovisual content in the form of a video (such as a movie) or a video game. The virtual reality content data 21 provides virtual reality content that can be experienced, for example, by an observer using a head-mounted visualization device. The virtual reality content data 21 comprises visual virtual reality content data 22, a plurality of identifiers 24 and multiple audio tracks 26.

The visual virtual reality content data 22 provides visual virtual reality content. The visual virtual reality content may be stereoscopic and / or panoramic content that extends beyond an observer's field of vision when viewed. In some examples, the visual virtual reality content is 360 ° visual virtual reality content in which an observer can experience a computer-simulated virtual environment in 360 °. In some other examples, the visual virtual reality content may cover less than the full 360 ° around the observer, such as 270 ° or some other amount. In some implementations, the visual virtual reality content is visible using a head-mounted display device and an observer can experience the computer-simulated virtual environment by moving, for example, his head while carrying the head-mounted display device. . In other implementations, the visual virtual reality content is visible without using a head-mounted display device.

In some implementations, the visual virtual reality content data 22 may include different parts that form the visual virtual reality content data 22. Different parts of the data 22 may have been encoded separately and reference different types of content. Different types of content can be represented separately. For example, one or more parts of the data 22 may have been encoded to provide background / visual scenario content, while one or more other parts of the data 22 may have been encoded to provide foreground / moving object content. The background / stage content can be represented separately from the foreground / moving object content.

The identifiers 24 identify predefined regions of interest in the visual virtual reality content data. The identifiers 24 are for use in controlling the progress of the visual virtual reality content. This will be described in greater detail below.

The multiple audio tracks 26 provide audio that accompanies the visual virtual reality content. Different audio tracks can provide audio for different virtual regions of the environment (360 °) simulated in the visual virtual reality content. Additionally, some audio tracks may provide, for example, background / ambient audio while other audio tracks may provide foreground audio that may include, for example, dialogue.

Figure 3 illustrates a third apparatus 110 that can be a chip or a chipset. The third apparatus 110 may be part of an informatic device such as that illustrated in Figure 4.

5

10

fifteen

twenty

25

30

35

40

Four. Five

fifty

55

60

65

The apparatus 110 comprises at least one processor 112 and at least one memory 114. A single processor 112 and a single memory 114 are shown in Figure 3 and are analyzed below simply for purposes of illustration.

Processor 112 is configured to read from and write to memory 114. Processor 112 may comprise an output interface through which data and / or commands are issued by processor 112 and an input interface through which data is entered and / o commands to processor 112.

The memory 114 is illustrated by storing a computer program 117 comprising the instructions / program code 118 that control the operation of the apparatus 110 when loaded into the processor 112. The processor 112, reading the memory 114, can load and execute the code of the computer program 118. The computer program code 118 provides the logic and routines that enable the apparatus 110 to perform the methods illustrated in Figure 6 and described below. In this sense, the processor 112, the memory 114 and the computer program code 118 provide means for carrying out the methods illustrated in Figure 6 and described below.

Although memory 114 is illustrated as a single component in Figure 3, it may be implemented as one or more separate components, some or all of which may be integrated / removable and / or may provide dynamic / permanent / semi-permanent cache storage .

The computer program code 118 can arrive in the apparatus 110 by any suitable delivery mechanism 130. The delivery mechanism 130 may be, for example, a non-transient computer readable storage medium such as an optical disk or a memory card. The delivery mechanism 130 may be a signal configured to reliably transfer the computer program code 118. The apparatus 110 may cause the propagation or transmission of the computer program code 118 as a computer data signal.

Figure 4 illustrates a fourth apparatus 120 in the form of an informatic device comprising the third apparatus 110. In some implementations, the apparatus 120 is configured to connect to a visualization device mounted on the head. In these implementations, the apparatus 120 may be, for example, a game console or a personal computer. In other implementations, the apparatus 120 may be a head mounted display device or a combination of a game console / personal computer and a head mounted display device.

In the example illustrated in Figure 4, memory 114 is illustrated by storing the virtual reality content data 20 explained above. The processor 112 and memory 114 are co-located in a housing / body 127.

Elements 12 and 14 are operationally coupled and there may be any number or combination of intermediate elements between them (not including intermediate elements).

A head mounted display device may comprise a stereoscopic screen and one or more motion sensors. The stereoscopic screen comprises optics and one or more display panels, and is configured to enable the observer to visualize the stored virtual virtual reality content as the visual virtual reality content data 22. The one or more motion sensors can be configured to detect the orientation of the display device mounted on the head in three dimensions and to detect the movement of the display device mounted on the head in three dimensions. The one or more motion sensors may comprise, for example, one or more accelerometers, one or more magnetometers and / or one or more gyroscopes.

A head-mounted display device may also comprise headphones or earphones for transporting audio in audio tracks 26 to a user / observer. Alternatively, the audio in the audio tracks 26 can be transported to a user / observer by separate speakers.

A first example of a first method according to the embodiments of the invention will now be described in relation to Figure 5.

In the first example of the first method, in block 501 in Figure 5, virtual reality content is created and stored as virtual reality content data 21. The visual aspect is stored as visual virtual reality content data 22 and the Audible aspect is stored as multiple audio tracks 26.

In block 502 in Figure 5, a user provides user input in the second apparatus 20 that identifies various regions of interest in the visual virtual reality content data 22 in the memory 14 of the second apparatus 20. The user input causes that the processor 12 of the second apparatus 20 stores the identifiers 24 in the virtual reality content data 21 that identifies those regions of interest in the visual virtual reality content data 22.

5

10

fifteen

twenty

25

30

35

40

Four. Five

fifty

55

60

65

The stored identifiers 24 identify different regions of interest in the visual virtual reality content at different times in the visual virtual reality content. The identification of regions of interest in the visual virtual reality content makes it possible to advance the visual virtual reality content to be controlled when consumed. This is described below in relation to Figure 6.

A first example of a second method according to the embodiments of the invention will now be described in relation to Figure 6.

In this first example of the second method, the virtual reality content data 21 stored in the memory 14 of the second apparatus 20 is provided to the fourth apparatus 120 illustrated in Figure 4. In some implementations of the invention, the reality content data virtual 21 can be stored on a server and downloaded or sent by continuous flow to the fourth apparatus 120. In other implementations, the virtual reality content data 21 can be provided to the fourth apparatus 120 by an optical disk or a memory card, for example.

In block 601 in Figure 6, an observer is consuming the virtual reality content provided by the virtual reality content data 21. For example, he may be consuming the virtual reality content through a head-mounted display device . The processor 112 of the fourth apparatus 120 causes and enables the observer to consume the virtual reality content. As explained above, the fourth apparatus 120 may be the head mounted display device or a computer device that is connected to the head mounted display device, such as a game console or a personal computer, or it may be A combination of the two.

In block 601 in Figure 6, the processor 112 of the apparatus 120 analyzes the visual virtual reality content data 22 and determines that there is at least one region of interest in the visual virtual reality content (provided by the content data of virtual reality visuals 22) that is part of the virtual reality content that is consumed by the user. The at least one region of interest has been predefined (in the manner described above in relation to Figure 5) and is identified by at least one of the identifiers 24 that is part of the virtual reality content data 21.

In block 602 in Figure 6, the processor 112 of the fourth apparatus 120 monitors whether at least a defined proportion of an observer's field of vision coincides with the at least one region of interest determined in the visual virtual reality content. In some implementations, the processor 112 begins to monitor the position and orientation of the defined proportion of the observer's field of vision before a region of interest has been determined. In other implementations, this monitoring only begins after a region of interest has been identified by the processor 112.

The total field of view of the observer when viewing visual virtual reality content may depend on the visualization device mounted on the head that is being used to view the visual virtual reality content and, in particular, on the optics of virtual reality content. visual.

In some implementations, the "defined proportion" of the observer's field of vision may be the total field of view of the observer when visual virtual reality content is displayed.

For example, if the total field of view of the observer when the visual virtual reality content is displayed is 100 ° in a transverse plane, then in such implementations the defined proportion of the field of view of the observer covers the entire 100 °.

In other implementations, the defined proportion of the observer's field of vision may be less than the total field of view of the observer when the visual virtual reality content is displayed. For example, the defined ratio could be much smaller than the total field of view of the observer, such as the central 20 ° of the field of view of the observer in a transverse plane, when the total field of view of the observer is 100 ° in the plane cross. Alternatively, the defined ratio could now be a single line of observation within the observer's field of vision, such as a line located in the center of the observer's field of vision.

In block 603 in Figure 6, the processor 112 of the fourth apparatus 120 controls the progress of the visual virtual reality content based on whether the at least a defined proportion of the observer's field of vision coincides with the at least one region of particular interest. in the visual virtual reality content. In some implementations, if the processor 112 determines that the defined proportion of the observer's field of vision coincides with the at least one region of interest in the visual virtual reality content, it enables the advancement of the visual virtual reality content. For example, if the visual virtual reality content is a video (such as a movie), it allows you to continue playing the video. If the visual virtual reality content is a videogame, it allows the videogame to advance.

5

10

fifteen

twenty

25

30

35

40

Four. Five

fifty

55

60

65

Alternatively, if the defined proportion of the observer's field of vision does not match the at least one region of interest determined in the visual virtual reality content, the processor 112 may stop the advance of at least a portion of the visual virtual reality content. . For example, if the visual virtual reality content is a video (such as a movie), the processor 112 may pause all or a region of the visual virtual reality content (such as a region containing the at least one region of interest determined) until the defined proportion of the observer's field of vision and the at least one region of particular interest begin to coincide. Similarly, if the visual virtual reality content is a video game, the processor 112 may stop the advance of at least a portion of the video game (such as a region containing the at least one region of particular interest) until a defined proportion The observer's field of vision begins to coincide with the at least one region of particular interest.

In implementations where different parts (such as foreground / moving objects and background / stage) are represented separately, the portion that stops may be foreground / moving objects and the background / stage may continue moving / cheering. For example, leaves can move and water can flow. The movement / animation can be done in a loop.

In addition to the above, the processor 112 can control the audio advance based on whether the defined proportion of the observer's field of vision coincides with at least one region of particular interest. For example, you can pause one or more audio tracks if the defined proportion of the observer's field of vision does not match the at least one region of particular interest. The reproduction of that / those audio track or tracks can begin if and when the defined proportion of the observer's field of vision begins to coincide with the at least one region of particular interest. The paused / restarted audio track or tracks may include audio that is intended to be synchronized with events in a portion of the visual virtual reality content where progress has been stopped. The paused / restarted audio track or tracks could be directional audio from the at least one region of particular interest. The paused / restarted audio track or tracks may include, for example, dialogue of one or more actors in the region of particular interest or music or sound effects.

In some implementations, the processor 112 may enable the playback of one or more audio tracks to continue or begin while the playback of one or more other audio tracks has been paused. The other track or audio tracks could include ambient music or ambient sounds, for example. The other track or audio tracks may be looped in some implementations.

Controlling the progress of visual virtual reality content based on whether a defined proportion of an observer's field of vision matches one or more predefined regions of interest is advantageous because it provides a mechanism to prevent an observer from missing important parts of reality content. virtual visual For example, you can prevent an observer from missing prominent representation points in visual virtual reality content by pausing at least part of the visual virtual reality content until the observer's field of vision is in an appropriate position. The observer, therefore, can explore a virtual environment completely without worrying about missing important points.

A second example of the first method according to the embodiments of the invention will now be described in relation to Figures 5, 7 and 8.

Figure 7 illustrates a schematic diagram in which virtual reality content (visual and audible) is recorded by one or more virtual reality content recording devices 703. Virtual reality content can be professionally recorded content, or content recorded by an amateur. The virtual reality content recording device or devices 703 are recordings of both visual and audible content. In the scene being recorded, a first actor 701 and a second actor 702 are located on a beach 704. The first actor 701 throws a ball 705 at the second actor 702.

Figure 7 illustrates the scene at a first instant where the first actor 701 has not yet thrown the ball 705 to the second actor 702. Figure 8 illustrates the scene at a point at the instant where the ball 705 has been caught by the second actor 702. The dashed line designated by reference number 706 represents the movement of ball 705.

Figure 9 illustrates the visual virtual reality content provided by the visual virtual reality content data 22 that is created by the virtual reality content creation device or devices 703 when the scene is captured in Figures 7 and 8. A The film director chooses to identify the first actor 701 and the ball 705 as a region of interest 901. He provides appropriate user input to the second apparatus 120 and, in response, the processor 12 of the second apparatus 20 creates and stores an identifier that identifies the region of interest 901 in the virtual reality content data 21, in the first instant in the reproduction of the virtual virtual reality content that is represented in Figure 9.

Figure 10 illustrates a second instant in the visual virtual reality content, which is later than the first instant. In the second instant, the first actor 701 has thrown the ball 705 to the second actor 702, but the ball 705 has not reached the second actor 702. The director of the film decides to identify the first actor 701 as a first

5

10

fifteen

twenty

25

30

35

40

Four. Five

fifty

55

60

65

region of interest 1001, ball 705 as a second region of interest 1002 and the second actor 702 as a third region of interest 1003 for the second instant in the visual virtual reality content. The director provides appropriate user input to the second apparatus 120 and, in response, processor 112 creates and stores identifiers 24 in memory 14 that identify regions of interest 1001, 1002 and 1003 in visual virtual reality content data 22 in the second instant in the reproduction of visual virtual reality content.

Figure 11 illustrates the visual virtual reality content in a third instant, after the first and second instants. In the third instant, the second actor 702 has taken the ball 705. The director of the film decides that he wishes to identify the second actor 702 and the ball 705 as a region of interest 1101 and provides appropriate user input to the second apparatus 120. In response, the processor 112 of the second apparatus 120 creates and stores an identifier that identifies the region of interest 1101 in the visual virtual reality content at the third instant in the reproduction of the visual virtual reality content.

In some implementations, regions of interest may not be identified by a user manually by providing user input. Alternatively, regions of interest can be automatically identified by image analysis performed by processor 12, and the corresponding identifiers can also be created and stored automatically.

A second example of the second method according to the embodiments of the invention will now be described in relation to Figures 6 and 12A to 15.

Figure 12A illustrates a first example of a defined proportion of a field of view of an observer in a transverse plane. In the example illustrated in Figure 12A, the position of visualization of the observer is designated by the point labeled with the reference number 1203. A first dashed line 1201 illustrates a first end of a field of view of the observer in the transverse plane and a second broken line 1202 designates a second end of a field of view of the observer in the transverse plane. The curved arrow labeled with reference number 1204 designates a third end of the viewer's field of view in the transverse plane. The area delimited by lines 1201, 1202 and 1204 represents the area that is visible by the observer in the transverse plane, when visual virtual reality content is visualized (for example, by a display device mounted on the head).

In the first example, the lattice in Figure 12A indicates that the entire field of view of the observer, when visual virtual reality content is visualized (for example, by means of a head-mounted display device), is considered to be the "definite proportion" of that field of vision.

A field of vision of the real life of the human being can be almost 180 ° in a transversal plane. The field of vision of a total observer while viewing the content of virtual visual reality may be the same or less than a field of vision of the real life of the human being.

Figure 12B illustrates a second example of a defined proportion of an observer's field of vision. In this example, the area between the first broken line labeled with the reference number 1201, the broken line labeled with the reference number 1202 and the curved arrow labeled with the reference number 1204 represent the area that is visible by an observer in a transversal plane, when visual virtual reality content is visualized, as described above. However, the example illustrated in Figure 12B differs from the example illustrated in Figure 12A in that the "defined proportion" of the observer's field of vision is less than the total field of view of the observer when visual virtual reality content is displayed . In this example, the defined proportion of the observer's field of vision is the angular region between the third and fourth dashed lines labeled with reference numbers 1205 and 1206 and is indicated by grid. It covers a central portion of the observer's field of vision. In this example, the defined proportion of the field of vision could be any amount that is smaller than the total field of view of the observer when visual virtual reality content is displayed.

It will be appreciated that Figures 12A and 12B provide examples of a defined proportion of a field of view of an observer in a transverse plane for purposes of illustration. In practice, the processor 112 can also monitor whether a defined proportion of the observer's field of vision in a vertical plane coincides with a region of interest. The processor 112 can determine, for example, a solid angle to determine if a defined proportion of an observer's field of vision coincides with a region of interest in visual virtual reality content.

By monitoring whether a defined proportion of an observer's field of vision matches a region of interest in visual virtual reality content, the processor 112 can be configured to determine an observer's point of vision (for example, by tracking an observer's head) and be configured to determine if a line (theory), area or volume emanating from the point of view of the observer intersects the region of interest.

5

10

fifteen

twenty

25

30

35

40

Four. Five

fifty

55

60

65

In some implementations, the processor 112 may be configured to monitor whether a defined proportion of an observer's field of vision coincides with a region of interest in visual virtual reality content using one or more cameras to track an observer's gaze. This may be the case in implementations where virtual reality content is visualized using a head-mounted visualization device and in implementations where it is not. If a head-mounted display device is not used, the visual virtual reality content could be displayed, for example, on one or more screens surrounding the observer.

In implementations where the observer's gaze is traced, if the visual virtual reality content is stereoscopic, the processor 112 can be configured to determine the depth of the region or regions of interest and determine the depth of an observer's gaze. The processor 112 may be further configured to monitor whether at least a defined proportion of an observer's field of view coincides with the region or regions of interest determined by comparing the depth of the observer's gaze with the depth of the region or regions of interest.

Figures 13 to 15 illustrate a schematic diagram of an observer who visualizes the visual virtual reality content that was shown recorded in Figures 7 and 8 and that includes the identifiers that were created and stored as described above and illustrated in Figures. 9 to 11.

In this second example of the second method, the processor 112 can cycle continuously through the method illustrated in Figure 6. This is described in detail below.

The reproduction of the visual virtual reality content begins and reaches the first moment in the reproduction of the visual virtual reality content, as illustrated in Figure 9. Figure 13 schematically illustrates an observer 1301 visualizing the visual virtual reality content in the First instant in the reproduction of the virtual virtual reality content. When the observer reaches the first instant in the reproduction of the visual virtual reality content, the processor 112, which cycles continuously through the method illustrated in Figure 6, determines that there is an identifier stored in the virtual reality content data 21 that identify a region of interest 901 in the first instant in the reproduction of the visual virtual reality content (as shown in Figure 9). In block 602 in Figure 6, the processor 112 determines that the defined proportion of the observer's field of vision coincides with the region of interest 901 at the first instant in the reproduction of the visual virtual reality content. Consequently, in block 603 in Figure 6, the processor 112 controls the progress of the visual virtual reality content by enabling the reproduction of the visual virtual reality content and that any audio synchronized with the events in the region of interest 901 continues.

The advance / reproduction of the visual virtual reality content continues and, in this example, the observer's field of vision is still directed towards the first actor 701. Figure 14 illustrates an observer who visualizes the visual virtual reality content in the third instant in the reproduction of the visual virtual reality content (as shown in Figure 11) when the ball 705 has been thrown from the first actor 701 to the second actor 702.

While the ball 705 is in the air, the first actor 701 is indicated to be a region of interest, as represented by the dashed line frame 1001 in Figure 10. Therefore, the processor 112 of the fourth apparatus 120 continues to allow visual virtual reality content to advance while cycling through the method of Figure 6. However, at the third instant (as illustrated in Figure 11), the first actor 701 is no longer a region of interest , and the only region of interest in the visual virtual reality content is the second actor 702 that holds the ball 705 as indicated by the dashed line box 1101 in Figure 11. The processor 112 determines that the (new) region of interest 1101 is the only region of interest in the third instant in the reproduction of visual virtual reality content, and monitors whether at least a defined proportion of the observer's field of vision coincides with the (new) region of interest 110 1 determined. However, initially, as depicted in Figure 14, the defined proportion of the observer's field of vision no longer coincides with any regions of interest in the third instant in the reproduction of visual virtual reality content. Consequently, in this example, in block 603 in Figure 6, the processor 112 controls the advance of the visual virtual reality content causing the advance / reproduction of at least a portion of the visual virtual reality content to stop. For example, the entire visual virtual reality content can be paused, or simply a portion that comprises the at least one region of interest.

In some implementations, the processor 112 may not stop the advance of the virtual virtual reality content instantaneously when it determines that the defined proportion of the observer's field of vision does not match a region of interest. There may be, for example, a short delay before it does and, if the defined proportion of the observer's field of vision begins to coincide with a region of interest during the delay period, the progress of virtual reality content does not stop visual. This can help avoid frequent pause and restart of the content.

Processor 112 may not stop the advance / reproduction of visual virtual reality content in some implementations. Instead, it can slow down the progress / reproduction of the entire visual virtual reality content or a portion thereof (such as a portion comprising the at least one region of interest).

5

10

fifteen

twenty

25

30

35

40

Four. Five

fifty

55

60

65

The observer then moves his field of vision so that the defined proportion of his field of vision coincides with the region of interest 1101 in the third instant in the reproduction of the visual virtual reality content, as depicted in Figure 15. The Processor 112 determines that the defined proportion of the observer's field of vision now coincides with the region of interest 1101 in the third instant in the reproduction of the visual virtual reality content and controls the progress of the visual virtual reality content again enabling the advance / Reproduction of visual virtual reality content.

In some implementations, it may not be necessary for the defined proportion of the observer's field of vision to match a region of particular interest so that the processor 112 again enables the advancement of the visual virtual reality content. For example, processor 112 may be configured to monitor a motion path of the defined proportion of the observer's field of vision and to control the progress of the visual virtual reality content based on whether the defined proportion of the observer's field of vision is expected to match a region of particular interest. This can be done, for example, by interpolating the movement path of the defined proportion of the observer's field of vision and determining whether the interpolated trajectory coincides with the region of interest determined.

References to 'computer readable storage medium', 'computer', 'processor' etc., should be understood to cover not only computers that have different architectures such as single / multiple processor architectures and sequential architectures (Von Neumann) / parallel but also specialized circuits such as programmable door matrix (FPGA) fields, application specific circuits (ASIC), signal processing devices and other processing circuitry. References to computer software, instructions, code etc., should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device, whether instructions for a processor, or configuration settings for a processor. fixed function device, door array or programmable logic device etc.

As used in this application, the term 'circuiterla' refers to all of the following:

(a) only hardware circuit implementations (such as analog and / or digital circuit only implementations) and

(b) to combinations of circuits and software (and / or firmware), such as (where applicable): (i) to a combination of processor or processors or (ii) to portions of processor or processors / software (including processor or digital signal processors), software, and memory or memories that work together to cause a device, such as a mobile phone or server, to perform various functions) and

(c) to circuits, such as a microprocessor or microprocessors or a portion of a microprocessor or microprocessors, that require software or firmware for operation, even if the software or firmware is not physically present.

This definition of 'circuit board' applies to all uses of this term in this application, including in any claim. As an additional example, as used in this application, the term "circuit board" would also be covered by an implementation of simply one processor (or multiple processors) or portion of a processor and its (or its) software and / or firmware attached. The term "circuit" also covers, for example and if applicable to the particular claim element, an integrated baseband circuit or application processor integrated circuit for a mobile phone or a similar integrated circuit on a server, a network device cell phone or other network device.

The blocks illustrated in Figures 5 and 6 may represent steps in a method and / or code sections in computer programs 17 and 18. The illustration of a particular order of the blocks does not necessarily imply that it is a required or preferred order of the blocks and the order and disposition of the block can be varied. Additionally, it may be possible to skip some blocks.

Although the embodiments of the present invention have been described in the preceding paragraphs with reference to various examples, it should be appreciated that modifications to the examples can be made without departing from the scope of the invention as claimed. For example, in some implementations, processor 112 may be configured to cause visualization of one or more indicators (such as one or more arrows) that indicate to an observer where the region or regions of interest are / are at a particular moment in content of virtual visual reality. The processor 112 may cause the indicator or indicators to appear, for example, if the progress of the visual virtual reality content has stopped / paused, or if it has stopped / paused for more than a threshold time period.

The form of visual virtual reality content may be different from the one described above. For example, you can dynamically update information content such as one or more sources of dynamic update web page.

The features described in the description above can be used in combinations other than the combinations explicitly described.

Although the functions have been described with reference to certain characteristics, these functions can be performed by other characteristics whether they have been described or not.

Although the characteristics have been described with reference to certain embodiments, these characteristics may also be present in other embodiments, whether they have been described or not.

Although it has been tried in the previous specification to call attention to those characteristics of the invention considered of particular importance, it should be understood that the applicant claims protection with respect to any patentable characteristic or combination of characteristics mentioned above and / or 10 shown in the drawings already whether or not particular emphasis has been placed on them.

Claims (15)

5 10 fifteen twenty 25 30 35 40 Four. Five fifty 55 60 65 1. A method, comprising: determine at least a first region of interest (901) in a first instant in a virtual virtual reality content that is configured to extend beyond an observer's field of vision when visual virtual reality content is visualized; monitor, while an observer (1301; 1302) is visualizing the visual virtual reality content, if a defined proportion of the observer's field of vision coincides with the first region of interest in the visual virtual reality content; monitor the progress of the visual virtual reality content based on whether the defined proportion of the observer's field of vision coincides with the first region of interest determined in the visual virtual reality content; determine a new second region of interest (1101), located outside the observer's field of vision while the observer is visualizing the visual virtual reality content, in a second instant in the visual virtual reality content; determine, while the observer is visualizing the visual virtual reality content, that the defined proportion of the observer's field of vision does not match the new second region of interest in the visual virtual reality content; Y stop the advance of at least a portion of the visual virtual reality content, which comprises the new second region of interest, based on the determination that the defined proportion of the observer's field of vision does not match the new second region of interest in the Visual virtual reality content. 2. The method of claim 1, wherein controlling the progress of the visual virtual reality content comprises enabling the advance of the visual virtual reality content in response to the determination that the defined proportion of the observer's field of vision coincides with the First or second regions of interest in visual virtual reality content. 3. The method of claims 1 and 2, wherein controlling the advance of the visual virtual reality content comprises stopping the advance of at least a portion of the visual virtual reality content in response to the determination that the defined proportion of the field The viewer's vision does not match the first or second regions of interest in the virtual virtual reality content. 4. The method of claim 3, wherein the advance of a portion of the visual virtual reality content comprising the first or second regions of interest is stopped. 5. The method of any of the preceding claims, further comprising: cause the reproduction of at least one audio track to be paused in response to the determination that the defined proportion of the observer's field of vision does not match the first region of interest in the visual virtual reality content. 6. The method of claim 5, further comprising: enable the reproduction of at least one additional audio track, different from the at least one audio track, when the playback of the at least one audio track has been paused. 7. The method of any of the preceding claims, wherein the visual virtual reality content is virtual reality video content and controlling the progress of the visual virtual reality content comprises controlling the reproduction of the virtual reality video content. 8. The method of any of the preceding claims, further comprising: determine that the first region of interest is no longer a region of interest. 9. The method of any one of the preceding claims, wherein the virtual virtual reality content is provided by virtual reality content data (21) stored in a memory and the virtual reality content data comprises a plurality of identifiers (24 ) that identify regions of interest in the visual virtual reality content. 10. The method of any of the preceding claims, wherein monitoring whether a defined proportion of an observer's field of vision coincides with the first or second regions of interest in the visual virtual reality content comprises tracking head movements. of an observer and / or an observer look. 11. The method of any of the preceding claims, wherein the defined proportion of the observer's field of vision is the total field of view of the observer when the visual virtual reality content is displayed. 5 10 fifteen twenty 25 30 or is less than the total field of view of the observer. 12. The method of claim 11, wherein the visual virtual reality content is 360 ° visual virtual reality content. 13. Computer program code which, when performed by at least one processor, causes the method of any of the preceding claims to be performed. 14. An apparatus (110; 120), comprising: means (112, 114, 117, 118) for determining at least a first region of interest in a visual virtual reality content that is configured to extend beyond an observer's field of vision when visual virtual reality content is displayed; means (112, 114, 117, 118) for monitoring, while an observer is viewing the visual virtual reality content, if a defined proportion of an observer's field of vision coincides with the first region of interest in the visual virtual reality content ; Y means (112, 114, 117, 118) to control the progress of the visual virtual reality content based on whether the at least a defined portion of the observer's field of vision coincides with the at least one region of interest determined in the reality content virtual visual; means (112, 114, 117, 118) to determine a new second region of interest, located outside the observer's field of vision while the observer is visualizing the visual virtual reality content, in a second instant in the visual virtual reality content ; means (112, 114, 117, 118) to determine, while the observer is visualizing the visual virtual reality content, that the defined proportion of the observer's field of vision does not match the new second region of interest in the virtual reality content visual; Y means (112, 114, 117, 118) to stop the advance of at least a portion of the visual virtual reality content, which comprises the new second region of interest, based on the determination that the defined proportion of the observer's field of vision does not coincide with the new second region of interest in the content of visual virtual reality. 15. The apparatus of claim 14, further comprising means (112, 114, 117, 118) for performing the method of one or more of claims 2 to 12.