WO2020206647A1

WO2020206647A1 - Method and apparatus for controlling, by means of following motion of user, playing of video content

Info

Publication number: WO2020206647A1
Application number: PCT/CN2019/082177
Authority: WO
Inventors: 陈泽天
Original assignee: 华为技术有限公司
Priority date: 2019-04-11
Filing date: 2019-04-11
Publication date: 2020-10-15
Also published as: CN118400584A; CN113170231A

Abstract

Provided are a method and apparatus for controlling, by means of following the motion of a user, the playing of video content. The method comprises: acquiring the position of a user relative to a display device, wherein the relative position is acquired by an image sensor when the display device plays a video; according to the relative position, determining, in a video source, target video content, wherein the target video content is video content in the video source that matches the range of line of sight of the user when at the relative position; and playing the target video content on a display screen of the display device, such that the played video content follows the motion of the user, thereby increasing the sense of presence of the user when watching the video, and improving the user experience.

Description

Method and device for controlling playing video content following user motion

Technical field

This application relates to the field of video technology, and in particular to a method and device for playing video content following user motion control.

Background technique

In recent years, televisions and various display technologies have developed rapidly, and manufacturers have continued to develop new technologies to improve the visual experience of viewers (or users). For video technologies, they are pursuing immersive experience.

In order to create immersive effects, the continuous development of video technology makes the video more high-definition and more three-dimensional. Although the video is more high-definition and more three-dimensional, the video screens that are played are edited by the on-site director. The audience cannot watch the interesting parts according to their own choices like in the video recording site, resulting in poor user experience.

Summary of the invention

The embodiments of the present application provide a method and device for playing video content according to the user's motion control, which controls the played video content according to the user's position, which increases the user's sense of presence and perspective of watching the video, and improves the user experience.

In the first aspect, a method for playing video content following user motion control is provided. The method includes:

Obtain the relative position of the user and the display device. The relative position is obtained by the sensor when the display device is playing the video; the target video content is determined in the video source according to the relative position, and the target video content is the visual range of the video source when the user is at the relative position Matching video content; the target video content is played on the display screen of the display device, so that the video content follows the user's movement and improves the user experience.

The method for controlling playback of video content provided by the embodiments of the application obtains the relative position of the user and the display device, and determines the target video content in the video source according to the relative position. The target video content is the line of sight of the video source when the user is at the relative position The video content that matches the scope is played, and the target video content is played on the display screen of the display device. This realizes that the played video follows the user's movement, so that the user can choose the angle and field of view to watch the video according to the user’s personal interest. The relative position of the display device presents the video content of different viewing angles to the audience, increases the presence and perspective of the video program, simulates the feeling of being on the scene, and improves the user experience.

With reference to the first aspect, in the first possible implementation manner of the first aspect, determining the target video content in the video source according to the relative position includes: determining the line of sight angle of the user in the relative position according to the relative position, and the line of sight angle is used for Reflects the line of sight range, the line of sight angle is the spatial angle of the user’s line of sight, the line of sight of the user is the connection between the user’s eyes and the center point of the display screen when the user is in a relative position; the target video content is determined from the video source according to the line of sight angle, and the target video content is Video content that matches the user’s eye angle.

In combination with the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the line of sight angle includes a first angle and a second angle; the first angle is that the user's line of sight is mapped onto the xy plane When, the included angle with the y-axis, the xy plane is the plane formed by the x-axis and the y-axis; the second angle is the included angle with the z-axis when the user’s line of sight is mapped to the yz plane, the yz plane is y The plane formed by the axis and the z axis; where the y axis is perpendicular to the plane where the display screen is located, the z axis is perpendicular to the ground and parallel to the plane where the display screen is located, and the x and z axes form an xz plane, and the xz plane is where the display screen is located The plane, the x-axis, y-axis and z-axis intersect at the origin of the coordinate axis, which is the center point of the display screen.

In combination with the second possible implementation of the first aspect, in the third possible implementation of the first aspect, determining the target video content from the video source according to the line-of-sight angle includes: Determine the target video content in the video source, the projection of the connection between the center point of the target video content and the center point of the display screen on the xy plane and the y axis is the first angle, and the center point of the target video content and the display screen The video content of the second angle between the projection of the line of the center point on the yz plane and the z axis is the target video content.

In combination with the second possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the relative position is the distance L1 between the user and the center point of the display screen, and the video source includes the video shooting distance L2 and the video content Viewing angle range, L2 is the distance from the video content of the video source to the center point of the display screen; determine the target video content in the video source according to the relative position, including: determining the target according to the line of sight angle, L1, L2, viewing angle range and the size of the display screen Video content.

In combination with the fourth possible implementation of the first aspect, in the fifth possible implementation of the first aspect, the viewing angle range includes the horizontal display viewing angle and the vertical display viewing angle. The horizontal display viewing angle is the video content of the video source mapped to xy The display range angle on the plane, the vertical display angle of view is the display range angle of the video content of the video source mapped to the yz plane; the size of the display includes the width D of the display and the height H of the display; according to the line of sight angle, L1, L2 , The viewing angle range and the size of the display screen determine the target video content, including: determining the display range of the target video content on the xy plane according to the first angle, L1, L2, horizontal display viewing angle, and the width D of the display screen; according to the second angle, L1, L2, the vertical display viewing angle and the height H of the display screen determine the display range of the target video content on the yz plane.

In combination with the first aspect, in the sixth possible implementation of the first aspect, determining the target video content in the video source according to the relative position includes: determining the connection between the user's eyes and the center point of the display screen when the user is in the relative position The first intersection with the video source; the video content in the video source with the first intersection as the center point and the same aspect ratio as the display screen is taken as the target video content.

With reference to the first aspect, or any of the foregoing possible implementations of the first aspect, in the seventh possible implementation of the first aspect, determining the target video content in the video source according to the relative position includes: according to the relative position and user movement The speed determines the target video content.

With reference to the first aspect, or any of the foregoing possible implementations of the first aspect, in the eighth possible implementation of the first aspect, the method further includes: obtaining user action information; and determining the target video according to the relative position and the action information content.

In combination with the eighth possible implementation manner of the first aspect, in the ninth possible implementation manner of the first aspect, determining the target video content according to the relative position and motion information includes: determining the first in the video source according to the relative position Video content; the presentation rule of the pre-stored video content is determined according to the action information, and the presentation rule corresponds to the action information; the target video content is determined according to the first video content and the presentation rule.

In combination with the eighth or ninth possible implementation of the first aspect, in the tenth possible implementation of the first aspect, the motion information includes information about telescope gestures, information about gestures for pushing both hands forward, and hands backward. Move at least one of the gestures.

In combination with the ninth or tenth possible implementation manner of the first aspect, in the eleventh possible implementation manner of the first aspect, determining the target video content according to the first video content and the presentation rule includes: The center point of the video content is the center to enlarge or reduce the first video content to determine the target video content.

With reference to the first aspect, or any of the foregoing possible implementation manners of the first aspect, in the twelfth possible implementation manner of the first aspect, determining the target video content in the video source according to the relative position includes: according to the relative position Determine the cropping or scaling strategy of the video source to obtain the target video content.

With reference to the first aspect, or any of the foregoing possible implementation manners of the first aspect, in the thirteenth possible implementation manner of the first aspect, the video source includes at least one of the following: a video with a resolution greater than or equal to 4K, a field of view Video with an angle greater than or equal to 140 degrees, or long strip video.

In combination with the first aspect, or any of the foregoing possible implementation manners of the first aspect, in the fourteenth possible implementation manner of the first aspect, in the initial state, the preset second video content is displayed on the display screen, The second video content is part of the video content of the video source.

In a second aspect, a device for playing video content following user motion control is provided. The device includes a processor and a transmission interface; the transmission interface is used to receive the position of the user obtained by the sensor when the user watches the video; and the processor is used to Determine the target video content in the video source according to the user's location. The target video content is the video content in the video source that matches the user's line of sight at the location; the transmission interface is also used to transmit the target video content to the display screen , In order to make the display screen play the target video content, realize the video content follows the user's movement, and improve the user experience.

It should be understood that when the device is a chip, both the processor and the transmission interface belong to the chip, or it can be said that the transmission interface is an interface used by the processor to send and receive data.

With reference to the second aspect, in the first possible implementation manner of the second aspect, the processor is specifically configured to determine the line-of-sight angle of the user at the location according to the position of the user, the line-of-sight angle is used to reflect the line-of-sight range, and the line-of-sight angle It is the spatial angle of the user's line of sight, which is the connection between the user's eyes and the center point of the display screen when the user is at the position; the target video content is determined from the video source according to the line of sight angle, and the target video content is the line of sight angle matching the user Video content.

In combination with the first possible implementation of the second aspect, in the second possible implementation of the second aspect, the line of sight angle includes a first angle and a second angle; the first angle is that the user's line of sight is mapped onto the xy plane When, the included angle with the y-axis, the xy plane is the plane formed by the x-axis and the y-axis; the second angle is the included angle with the z-axis when the user’s line of sight is mapped to the yz plane, the yz plane is y The plane formed by the axis and the z axis; where the y axis is perpendicular to the plane where the display screen is located, the z axis is perpendicular to the ground and parallel to the plane where the display screen is located, and the x and z axes form an xz plane, and the xz plane is where the display screen is located The plane, the x-axis, y-axis and z-axis intersect at the origin of the coordinate axis, which is the center point of the display screen.

In combination with the second possible implementation manner of the second aspect, in the third possible implementation manner of the second aspect, the processor is specifically configured to: determine the target video content from the video source according to the first angle and the second angle, The connection between the center point of the target video content and the center point of the display screen is the first angle between the projection on the xy plane and the y axis, and the line between the center point of the target video content and the center point of the display screen is at yz The video content with the second angle between the projection on the plane and the z-axis is the target video content.

In combination with the second possible implementation of the second aspect, in the fourth possible implementation of the second aspect, the position of the user and the display device is the distance L1 between the user and the center point of the display screen, and the video source includes the video shooting distance L2 and the viewing angle range of the video content, L2 is the distance from the video content of the video source to the center point of the display screen; the processor is specifically used to determine the target video content according to the line of sight angle, L1, L2, viewing angle range and the size of the display screen.

In combination with the fourth possible implementation of the second aspect, in the fifth possible implementation of the second aspect, the viewing angle range includes the horizontal display viewing angle and the vertical display viewing angle, and the horizontal display viewing angle is the video content of the video source mapped to xy The display range angle on the plane, the vertical display angle of view is the display range angle of the video content of the video source mapped to the yz plane; the size of the display screen includes the width D of the display screen and the height H of the display screen; the processor is specifically used for: The first angle, L1, L2, horizontal display angle of view and the width D of the display screen determine the display range of the target video content on the xy plane; the target video is determined according to the second angle, L1, L2, vertical display angle of view and the height H of the display screen The display range of the content on the yz plane.

With reference to the second aspect, in a sixth possible implementation manner of the second aspect, the processor is specifically configured to: determine that the connection between the user's eyes and the center point of the display screen when the user is at the position and the first line of the video source Intersection point: The video content in the video source with the first intersection point as the center point and the same aspect ratio as the display screen is taken as the target video content.

With reference to the second aspect, or any of the foregoing possible implementation manners of the second aspect, in the seventh possible implementation manner of the second aspect, the processor is specifically configured to: determine the target video according to the position of the user and the speed of the user's movement content.

With reference to the second aspect, or any of the foregoing possible implementation manners of the second aspect, in the eighth possible implementation manner of the second aspect, the transmission interface is also used to receive the user's motion information obtained by the sensor; the processor; It is also used to determine the target video content according to the user's location and action information.

In combination with the eighth possible implementation manner of the second aspect, in the ninth possible implementation manner of the second aspect, the processor is specifically configured to: determine the first video content in the video source according to the location of the user; and according to the action information Determine the presentation rule of the pre-stored video content, and the presentation rule corresponds to the action information; determine the target video content according to the first video content and the presentation rule.

In combination with the eighth or ninth possible implementation of the second aspect, in the tenth possible implementation of the second aspect, the action information includes information about the telescope gesture, information about the gesture of pushing both hands forward, and the information about the gesture of pushing both hands backward. Move at least one of the gestures.

With reference to the ninth or tenth possible implementation manner of the second aspect, in the eleventh possible implementation manner of the second aspect, the processor is specifically configured to: zoom in or out with the center point of the first video content as the center The first video content is reduced to determine the target video content.

With reference to the second aspect, or any of the foregoing possible implementation manners of the second aspect, in the twelfth possible implementation manner of the second aspect, the processor is specifically configured to: determine the crop or zoom of the video source according to the user's position Strategies to obtain target video content.

With reference to the second aspect, or any of the foregoing possible implementation manners of the second aspect, in the thirteenth possible implementation manner of the second aspect, the video source includes at least one of the following: a video with a resolution greater than or equal to 4K, a field of view A video with a degree of 140 degrees or more, or a long strip video.

In combination with the second aspect, or any of the foregoing possible implementation manners of the second aspect, in the fourteenth possible implementation manner of the second aspect, the transmission interface is also used to transmit the preset second video content to the display screen , So that the display screen plays the second video content.

With reference to the second aspect, or any of the foregoing possible implementation manners of the second aspect, in the fifteenth possible implementation manner of the second aspect, the device includes a display screen.

With reference to the second aspect, or any of the foregoing possible implementation manners of the second aspect, in the sixteenth possible implementation manner of the second aspect, the device includes a camera, and the camera includes an image sensor.

In a third aspect, a device for playing video content following user motion control is provided, and the device includes:

The receiving unit is used to receive the location of the user acquired by the sensor when the user is watching the video; the determining unit is used to determine the target video content in the video source according to the user's location, and the target video content is the time when the user is in the video source and the user is at the location. The sending unit is used to transmit the target video content to the display screen, so that the display screen can play the target video content, so that the video content follows the user's movement and improves the user experience.

With reference to the third aspect, in the first possible implementation manner of the third aspect, the determining unit is specifically configured to:

Determine the line-of-sight angle of the user at the location according to the user's location. The line-of-sight angle is used to reflect the line of sight range. The line-of-sight angle is the spatial angle of the user's line of sight. The line of sight of the user is the center point of the user's eyes and the display screen when the user is at the position The connection; the target video content is determined from the video source according to the line of sight angle, and the target video is also the video content that matches the user’s line of sight angle.

In combination with the first possible implementation of the third aspect, in the second possible implementation of the third aspect, the line of sight angle includes a first angle and a second angle; the first angle is that the user's line of sight is mapped onto the xy plane When, the included angle with the y-axis, the xy plane is the plane formed by the x-axis and the y-axis; the second angle is the included angle with the z-axis when the user’s line of sight is mapped to the yz plane, the yz plane is y The plane formed by the axis and the z axis; where the y axis is perpendicular to the plane where the display screen is located, the z axis is perpendicular to the ground and parallel to the plane where the display screen is located, and the x and z axes form an xz plane, and the xz plane is where the display screen is located The plane, the x-axis, y-axis and z-axis intersect at the origin of the coordinate axis, which is the center point of the display screen.

In combination with the second possible implementation manner of the third aspect, in the third possible implementation manner of the third aspect, the determining unit determines the target video content from the video source according to the first angle and the second angle. The projection of the connection between the central point and the central point of the display screen on the xy plane and the y-axis is the first angle, and the projection of the connection between the central point of the target video content and the central point of the display screen on the yz plane The video content whose angle with the z axis is the second angle is the target video content.

In combination with the second possible implementation of the third aspect, in the fourth possible implementation of the third aspect, the position of the user and the display device is the distance L1 between the user and the center point of the display, and the video source includes the video shooting distance L2 and the viewing angle range of the video content, L2 is the distance from the video content of the video source to the center point of the display screen; the determining unit is specifically used to determine the target video content according to the line of sight angle, L1, L2, the viewing angle range and the size of the display screen.

Combined with the fourth possible implementation of the third aspect, in the fifth possible implementation of the third aspect, the viewing angle range includes the horizontal display viewing angle and the vertical display viewing angle, and the horizontal display viewing angle is the video content of the video source mapped to xy The display range angle on the plane, the vertical display angle is the display range angle of the video content of the video source mapped to the yz plane; the size of the display screen includes the width D of the display screen and the height H of the display screen; the determining unit is specifically used for: The first angle, L1, L2, horizontal display angle of view and the width D of the display screen determine the display range of the target video content on the xy plane; the target video is determined according to the second angle, L1, L2, vertical display angle of view and the height H of the display screen The display range of the content on the yz plane.

With reference to the third aspect, in the sixth possible implementation manner of the third aspect, the determining unit is specifically configured to: determine that the connection between the user's eyes and the center point of the display screen when the user is at the position and the first line of the video source Intersection point: The video content in the video source with the first intersection point as the center point and the same aspect ratio as the display screen is taken as the target video content.

With reference to the third aspect, or any of the foregoing possible implementation manners of the third aspect, in the seventh possible implementation manner of the third aspect, the determining unit is specifically configured to: determine the target video according to the position of the user and the speed of the user's movement content.

With reference to the third aspect, or any of the foregoing possible implementation manners of the third aspect, in the eighth possible implementation manner of the third aspect, the receiving unit is further configured to receive the user's motion information obtained by the sensor; the determining unit , Also used to determine the target video content based on the user’s location and action information.

In combination with the eighth possible implementation manner of the third aspect, in the ninth possible implementation manner of the third aspect, the determining unit is specifically configured to: determine the first video content in the video source according to the location of the user; and according to the action information Determine the presentation rule of the pre-stored video content, and the presentation rule corresponds to the action information; determine the target video content according to the first video content and the presentation rule.

In combination with the eighth or ninth possible implementation of the third aspect, in the tenth possible implementation of the third aspect, the motion information includes information on the telescope gesture, information on the gesture of pushing both hands forward, and the hands backward Move at least one of the gestures.

In combination with the ninth or tenth possible implementation manner of the third aspect, in the eleventh possible implementation manner of the third aspect, the determining unit is specifically configured to: zoom in or out with the center point of the first video content as the center The first video content is reduced to determine the target video content.

With reference to the third aspect, or any of the foregoing possible implementation manners of the third aspect, in the twelfth possible implementation manner of the third aspect, the determining unit is specifically configured to: determine the cropping or scaling of the video source according to the user's position Strategies to obtain target video content.

With reference to the third aspect, or any of the foregoing possible implementation manners of the third aspect, in the thirteenth possible implementation manner of the third aspect, the video source includes at least one of the following: a video with a resolution greater than or equal to 4K, a field of view A video with a degree of 140 degrees or more, or a long strip video.

With reference to the third aspect, or any of the foregoing possible implementation manners of the third aspect, in the fourteenth possible implementation manner of the third aspect, the sending unit is further configured to transmit the preset second video content to the display screen , So that the display screen plays the second video content.

In a fourth aspect, a computer-readable storage medium is provided, and instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer or a processor, the computer or the processor executes the first aspect or any of the first aspects. One of the possible implementation methods.

In a fifth aspect, a computer program product containing instructions is provided, which when running on a computer or processor, causes the computer or processor to execute the method in the first aspect or any possible implementation of the first aspect.

Based on the provided method and device for following user motion control to play video content, obtain the relative position of the user and the display device, and determine the target video content in the video source according to the relative position. The target video content is when the user is in the relative position of the video source The target video content is played on the display screen of the display device, which realizes that the played video follows the user's movement, so that the user can choose the angle and field of view of the video according to the part of the user’s personal interest. The relative position of the user and the display device presents the video content of different viewing angles to the audience, increasing the presence and perspective of the video program, simulating the feeling of being in person, and improving the user experience.

Description of the drawings

Fig. 1a is a schematic diagram of video content in a video source provided by an embodiment of the present application;

FIG. 1b is a schematic diagram of video content in another video source provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a sight range provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a user's sight range after the user moves a position according to an embodiment of the present application;

4 is a schematic diagram of a viewing angle provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of determining target video content in a horizontal direction according to an embodiment of the present application;

FIG. 6 is a schematic diagram of determining target video content in a vertical direction according to an embodiment of the present application;

FIG. 7 is another schematic diagram of determining target video content in a horizontal direction according to an embodiment of the present application;

FIG. 8 is another schematic diagram of determining target video content in a vertical direction according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a display device provided by an embodiment of the present application;

FIG. 10 is a schematic structural diagram of another display device provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of a flow of controlling playing video content according to an embodiment of the present application;

FIG. 12 is a schematic flowchart of a method for playing video following user motion control provided by an embodiment of the present application;

FIG. 13 is a schematic diagram of a sight range provided by an embodiment of the present application;

FIG. 14 is a schematic diagram of another sight range provided by an embodiment of the present application;

15 is a schematic diagram of a line of sight range provided by an embodiment of the present application;

FIG. 16 is a schematic diagram of a sight range provided by an embodiment of the present application;

FIG. 17 is a schematic diagram of a sight range provided by an embodiment of the present application;

18 is a schematic flowchart of a method for playing video content following user motion control provided by an embodiment of the present application;

FIG. 19a is a schematic diagram of a sight range provided by an embodiment of the present application;

FIG. 19b is a schematic diagram of another sight range provided by an embodiment of the present application;

20 is a schematic structural diagram of an apparatus for playing video content following user motion control provided by an embodiment of the present application;

FIG. 21 is a schematic structural diagram of a device for playing video content following user motion control provided by an embodiment of the present application;

FIG. 22 is a schematic structural diagram of an apparatus for playing video content following user motion control provided by an embodiment of the present application.

detailed description

In the development of display technology, in order to enable viewers (also known as users) to achieve an immersive experience, the development of video technology mainly has the following development trends:

1. Clearer. From high-definition (HD) to full-high-definition (full-high-definition, FHD), to 4K resolution (resolution), and then to 8K resolution, the increase in display screen (or display) resolution brings more Clear video picture. However, as a display screen that always needs to be viewed at a certain distance, the further improvement of resolution will limit the improvement of video effects, and the market penetration rate is relatively low.

2. More three-dimensional. In order to improve the video viewing experience, there are currently a variety of stereoscopic video solutions, including 3D display with glasses, naked-eye 3D, 360-degree video, and virtual reality (VR) glasses. The implementation of these solutions requires wearing special equipment, and the long-term user experience is poor, and it is difficult for multiple people to experience it together. In addition, these solutions have been around for many years and are still niche applications, which are difficult to popularize.

3. Customization. For live video programs, some operators provide functions such as video-on-demand, review, time shift, etc., but in essence, they are all programs-on-demand and are not creative.

On the contrary, many new technologies have been born in the field of video games. For example, the "somatosensory" devices represented by Microsoft XboxKinect have enriched the gameplay. "Somatosensory" devices recognize the user's position and actions to control the game, getting rid of the traditional gamepad. However, the application scenarios of many "somatosensory" devices are limited to the game field, and there is no application combined with video, and there is no technology to use related devices to enhance the video experience.

Later, the technology of controlling the TV through user actions was gradually developed. Use the camera to capture user motion information, such as gestures: swing left and right, control the TV to switch channels or increase or decrease the volume. This technology replaces remote control operation. However, the realization of this technology is actually no different from using the remote control. It just associates the user's actions with several keys of the remote control, and does not improve the user experience.

There is also a technology to watch 360-degree videos through a mobile phone. This technology is to install a gyroscope in the mobile phone, and use the built-in gyroscope to sense the direction change of the mobile phone. When the gyroscope senses that the mobile phone changes its angle, the video area displayed on the mobile phone screen varies The movement to achieve 360-degree multi-angle viewing. However, this technology must rely on the built-in gyroscope of the mobile phone. If you change the scene, such as a display device such as a TV that cannot change the direction, the technology cannot be used.

No matter how current technology develops, TVs and other display devices present users with only a flat picture, similar to a picture. No matter where the user stands, the picture that the user sees is always the same, even if 3D video is derived. For special video programs, the user stands at different positions in front of the TV or other display devices, and the program content of the same program that the user sees is still the same. Because the program content or video screen that you see is the program content or video screen edited and produced by the director, the user cannot watch the program content or video screen of interest according to his own choice. For example, in the stadium box, the user can use the window You can watch almost the entire stadium, and users can freely choose the part of interest, and control the viewing angle and field of view by themselves. Current display devices such as TVs cannot provide users with an experience similar to live viewing.

Therefore, in order to be able to realize the immersive user experience and freely choose the user experience of watching video content, the embodiments of the present application provide a method and device for playing video content according to user motion control, which can be applied to various scenarios, such as Program rebroadcast (such as sports match rebroadcast), advertising machine and other scenes.

In the embodiment of the present application, an ultra-wide-angle or panoramic camera is first used to record an ultra-high-definition video with a resolution greater than or equal to 4K, for example, a 4K-resolution ultra-high-definition video or an 8K-resolution ultra-high-definition video. The ultra-high-definition video recorded in this solution can watch the video content in the entire scene as shown in Figure 1a, while the traditional recorded video can only watch part of the video content in the entire scene as shown in Figure 1b.

When the ultra-high-definition video starts to be played on the display, the device only presents (or displays) the video content within the conventional angle of view in the video content to the user. For example, the original video is a video in the 8K resolution range, and only the middle is displayed by default. For videos in the 4K resolution range, as shown in Figure 2, the wide-angle part of the edge is not displayed.

Among them, the field of view, also known as the field of view, is the range of space that the eyes can see when the eyes are fixed at one point. The visual field is divided into static visual field and dynamic visual field; static visual field refers to the spatial range that can be seen by the eyes when looking at the object directly in front of the person with the head and eyeballs fixed; dynamic visual field refers to what the eyes can see when turning Visible spatial extent. Static vision and dynamic vision are often expressed by angles. In this embodiment, the field of view is called the line of sight range, and the line of sight is the connection line between the user's eyes and the center point of the display screen, as shown in FIG. 2.

When the device detects the user's movement, that is, when the device detects that the user watching the video changes the viewing position, the device determines the user's view in the video source according to the relative position of the user equipment, and the user's line of sight and the line of sight of the display device And play the video content on the screen of the display device. Among them, the line of sight angle is used to reflect the line of sight range, and the line of sight angle is the spatial angle of the user's line of sight.

It should be noted that before the user changes the viewing position and after the viewing position is changed, the video content in the user's field of view may not be the same video content of the video source, as shown in Figure 2 and Figure 3, where Figure 2 can be considered as the user’s The video content watched before moving the location, Figure 3 is the user's line of sight after moving the location.

Using the method of following the user’s motion to control the playback of video content produces the effect of the line of sight moving following the user’s movement, so that the played video content moves with the user’s motion, simulating the effect of on-site viewing, and allowing the user to experience an immersive scene. Improve the user experience.

For traditional videos, because they are edited by the director, when the video is played on the display device, only part of the video content of the entire scene can be seen. Even if the user changes the viewing position, only the same video content can be seen.

In one embodiment, the device includes a sensor, for example, the device includes a camera, and the sensor is located in the camera of the device. The sensor may be an image sensor, an ultrasonic sensor, or an infrared sensor in the camera, and is used to obtain the relative position of the user and the display device. . In one embodiment, the camera can acquire the location of a specific user among multiple users; in one embodiment, the camera can acquire the location of multiple users at the same time. After the device obtains the relative position of the user through the camera, it calculates the line of sight of each user at its relative position and the line of sight angle of the display device; where the line of sight of the user is the line connecting the user's eyes and the center point of the display screen, as shown in Figure 4. Shown.

In Figure 4, the center point of the display screen is called the origin, the axis passing through the origin and perpendicular to the plane where the display screen is located is called the y axis, and the axis passing through the origin perpendicular to the ground and parallel to the plane where the display screen is located is called the z axis. The axis passing through the origin parallel to the plane of the display screen and perpendicular to the y-axis and z-axis is called the x-axis; the plane formed by the x-axis and the y-axis is called the xy plane (also called the horizontal plane, the direction pointed by the x-axis) The plane formed by the y-axis and the z-axis is called the yz plane (also called the vertical plane, and the direction pointed by the z-axis is called the vertical direction), and the plane formed by the x-axis and the z-axis is called the xz plane.

The device calculates the line of sight angle of the user in the relative position according to the relative position of the user. The line of sight angle includes angle a (also called the first angle) and angle b (also called the second angle), as shown in FIG. 4. The device calculates that the user’s line of sight is mapped to the xy plane and the included angle a with the y-axis, and the user’s line of sight is mapped to the yz plane, and the included angle b with the z-axis. Among them, the included angle a is also called the horizontal angle. Angle, the included angle b is also called the vertical included angle.

In one embodiment, when the user moves relative to the display device, the device obtains the relative position of the user and the display device through the sensor, and the device calculates the line of sight angle of the user at the relative position, and determines the target video content in the video source according to the line of sight angle. The video is the video content that will be played on the display screen after the user moves, and the target video content is played on the display screen of the display device, as shown in Figure 3, so that the user can select the angle of viewing the video according to the part of the user’s personal interest According to the relative position of the user and the display device, the video content of different perspectives is presented to the audience, which increases the sense of presence and perspective of the video program, and plays the video content following the user's movement, so that the user can experience the scene when watching the video. Feeling, which simulates the feeling of on-site viewing, improves user experience.

The device determines the target video content in the video source according to the line of sight angle, including: the device determines the target video content from the video source according to the first angle and the second angle, and the connection between the center point of the target video content and the center point of the display screen is at xy The angle between the projection on the plane and the y axis is the first angle, as shown in Figure 5, and the angle between the projection on the yz plane and the z axis of the connection between the center point of the target video content and the center point of the display screen is The video content of the second angle (as shown in FIG. 6) is the target video content.

Fig. 5 is a schematic diagram of determining the target video content in the horizontal direction. Fig. 6 is a schematic diagram of determining the target video content in the vertical direction. In Figures 5 and 6, the axis passing through the center point of the display screen of the display device and perpendicular to the plane where the display screen is located is called the y axis; the axis perpendicular to the y axis and parallel to the plane where the display screen is located is called the z axis ; The axis that is perpendicular to the y-axis and z-axis and parallel to the plane of the display is called the x-axis, and the x-axis, y-axis and z-axis intersect at the center of the display; the direction of the x-axis is called the horizontal direction , The direction pointed by the z axis is called the vertical direction; the plane formed by the x axis and the y axis is called the xy plane; the plane formed by the y axis and the z axis is called the yz plane.

The angle formed by the projection of the user’s line of sight on the yz plane and the z-axis is the second angle, angle b, as shown in Figure 5, the connection between the center point of the display screen and the center point of the target video content and the z-axis The included angle is also the second angle, namely angle b.

The angle between the projection of the user’s line of sight on the xy plane and the y-axis is the first angle, namely angle a, as shown in Figure 6, the connection between the center point of the display screen and the center point of the target video content and the y-axis The included angle is also the first angle, namely angle a.

In other words, it is determined that the line connecting the user's line of sight to the center point of the video content is projected onto the xy plane, and the angle between the y-axis is the first angle of the video content, and the line of sight of the user is determined to be connected to the center point of the video content. The video content whose line projected to the yz plane and the z-axis is the second angle is the target video content.

In an embodiment, as shown in FIGS. 7 and 8, the relative position of the user and the display device also includes the distance L1 from the user to the center point of the display screen, and the line-of-sight angle of the user watching the target video content, including angle a and angle b , The intersection a and angle b are the same as the angle a and angle b in FIG. 5 and FIG. 6. The video content in the video source includes the shooting distance of the video content when the video content is shot, that is, the distance L2 from the video content in the video source to the center point of the display screen, and the viewing angle range of the video content, also known as the viewing angle. The viewing angle range includes the display range angle of the video content of the video source mapped to the xy plane, that is, the horizontal angle range A (as shown in Figure 7), also known as the horizontal display angle of view, and the display range angle mapped to the yz plane, namely The vertical angle range B (as shown in Figure 8) is also called the vertical display viewing angle. Among them, the size of the display screen includes the width of the display screen being D and the height of the display screen being H.

As shown in Figures 7 and 8, the device determines the target video content in the video source according to the relative position of the user and the display device, including: the device determines the target video content according to the line of sight, distance L1, distance L2, viewing angle range, and the size of the display screen . The specific process is:

As shown in Figure 7, the device determines the user's line of sight range in the xy plane according to the angle b, the distance L1, and the width D of the display device, that is, the horizontal line of sight range, and then according to the horizontal line of sight range, the distance L2 and the horizontal angle range A Determine the target video content in the xy plane.

As shown in Figure 8, the device determines the user's line of sight range in the yz plane according to the angle a, the distance L1 and the width of the display device, that is, the vertical line of sight range, and then the vertical line of sight range, as well as the distance L2 and the vertical angle range B. The target video content in the yz plane.

That is, the device determines the display range of the target video content on the xy plane according to the angle b, the distance L1, the distance L2, the horizontal display viewing angle, and the width D of the display screen; the device determines the display range of the target video content on the xy plane according to the second angle, L1, L2, and the vertical display viewing angle and the display The height H determines the display range of the target video content on the yz plane.

The device determines that the three-dimensional video content composed of the target video content in the x-y plane and the target video content in the y-z plane is the target video content.

In an embodiment, as shown in Figures 5 and 6, the apparatus determines the target video content in the video source according to the relative position of the user and the display device, including:

Determine the first intersection of the line between the user’s eyes and the center point of the display screen when the user is in the relative position and the first intersection of the video source; set the first intersection as the center point of the video content in the video source and have the same aspect ratio as the display screen As the target video content.

Among them, the aspect ratio refers to the ratio of the width to the height of the video content. In one embodiment, there may be a certain distance between the display screen and the video source, and the aspect ratio of the target video content displayed on the display screen is the same as the aspect ratio of the target video content determined in the video source, that is, the display screen displays The target video content of is proportional to the target video content determined in the video source.

The embodiment of the application also provides a method for playing video content following the user's motion control. The device obtains the user's motion information through the camera, such as gesture information, such as telescope gestures, open hands outward, close hands inward, and make camera gestures with both hands , One-hand forward and other gestures; according to the specific motion information, the video field of view is changed accordingly.

The following first describes the device provided in this application example. In one embodiment, as shown in FIG. 9, the device may be a display device 100, including a camera 110, a display screen 120, a decoder 130, and a processor 140. The camera 110 may be set on a display device, for example, the display device may be a mobile phone, and the camera may be set on the mobile phone. The camera 110 includes an image sensor.

In an embodiment, the camera 110 may also be used as a supporting device of the display device 100. In this case, as shown in FIG. 10, the camera 110 and the display device may be connected via a USB or other high-speed bus.

The display device 100 may be a device that supports a video with a resolution greater than or equal to 4K, such as a TV with 4K resolution and 8K resolution video decoding and playback, a smart terminal (such as a mobile phone, a tablet computer), and other devices with a display screen and a camera.

During video playback, the camera 110 is used to obtain the user's position or movement information. The movement information can be gesture information, such as a telescope gesture, a gesture of opening both hands outward, a gesture of folding both hands inward, a camera gesture with both hands, single Hand forward gesture, etc.

The display screen 120 is used to display the video content to be displayed.

In an embodiment, as shown in FIG. 10, the display device 100 may further include a video signal source interface 150 for receiving a video stream from a video signal source. The display device 100 may also include a wired network interface or a wireless network module 160 for connecting to a network via a wired or wireless connection, so that the display device 100 can obtain a video stream from the network in a wired or wireless manner. The display device 100 may also include a peripheral device interface 170, which is used to connect peripheral devices, such as a USB flash drive, for the display device 100 to obtain a video stream from the peripheral device.

The decoder 130 is used to decode the video stream received by the display device 100 from a video signal source, network or storage device. The processor 140 is configured to process the video content to be displayed according to the user's position or motion information acquired by the camera 110. For example, when the processor 140 determines that the user is moving, the processor 140 determines to control the field of view and controls the display screen 120 Display the video content within the field of view; for another example, the processor 140 controls the field of view according to the user's motion information obtained by the camera 110. For example, the motion information is a telescope gesture, and the processor 140 enlarges the video content, narrows the field of view, and controls The display screen 120 displays the enlarged video content.

The flow of the method for the display device 100 to control the playback of video content is shown in FIG. 11. In Figure 11, the solid line is the data flow, and the dashed line is the control flow.

The display device can obtain the video stream from the network, or the video signal source (devices that can receive the video signal such as radar), or the storage device, and send it to the decoder, that is, step 1: the decoder obtains the video stream; step 2: The decoder decodes the video stream to obtain the video source; Step 3: The decoder sends the video source to the memory and stores it in the memory, or can also be stored in the video buffer. When the camera of the display device obtains the user's position or movement information, step 4: the camera sends the obtained user position or movement information to the processor; step 5: the processor determines the video clipping or zooming scheme according to the user's position or movement information Step 6: Obtain the video source from the memory; Step 7: The processor trims or zooms the video source according to the position information or the action information, so that the trimmed or zoomed video content is the video content to be displayed; Step 8: Processing The device sends the trimmed or zoomed video content to the display screen; Step 9: Displays the trimmed or zoomed video content on the display screen.

In one embodiment, as shown in FIG. 9, the display device includes a memory 180, which is used to store a video stream.

The following describes a specific solution for playing video content following user motion control provided by an embodiment of the present application.

Fig. 12 is a schematic flowchart of a method for playing video content following user motion control provided by an embodiment of the present application. As shown in FIG. 12, the method is executed by the device, and the method may include the following steps:

S201: Obtain the relative position of the user and the display device, where the relative position is obtained by the sensor when the display device is playing a video.

In one embodiment, when playing a video, the device obtains the relative position of the user and the display device through a sensor in the camera.

In an embodiment, the sensor may be an image sensor, an ultrasonic sensor or an infrared sensor in the camera.

S202: Determine the target video content in the video source according to the relative position, where the target video content is the video content in the video source that matches the realizing range of the user in the relative position.

In one embodiment, as shown in Figures 5 and 6, the device determines the user's line of sight angle when the user is in a relative position. The line of sight angle is used to reflect the line of sight range. The line of sight angle is the spatial angle of the user's line of sight. The connection between the eye and the center point of the display screen in the relative position; the device determines the target video content from the video source according to the line of sight angle, and the target video content is the video content that matches the line of sight angle of the user.

As shown in Figs. 5 and 6, the line of sight angle includes angle a and angle b. The device determines the target video content from the video source according to the line of sight angle, including: the device determines the target video content from the video source according to angle a and angle b, specifically: the device determines the connection between the center point of the video content and the center point of the display screen The angle between the projection on the xy plane and the y axis is angle a, and the connection between the center of the video content and the center point of the display screen is the angle b between the projection on the yz plane and the z axis as the target. Video content. For specific descriptions, please refer to the descriptions of FIG. 5 and FIG.

In one embodiment, the device determines the target video content in the video source according to the relative position of the user and the display device, including: the device determines the target video content according to the line-of-sight angle, the distance L1, the distance L2, the viewing angle range, and the size of the display screen. The specific process is: the device determines the display range of the target video content on the xy plane according to the angle b, the distance L1, the distance L2, the horizontal display viewing angle, and the width D of the display screen; the device determines the display range of the target video content on the xy plane according to the second angle, L1, L2, and the vertical display viewing angle And the height H of the display screen determine the display range of the target video content on the yz plane. For specific description, please refer to the descriptions of FIG. 7 and FIG. 8. For the sake of brevity, the details are not repeated here.

In one embodiment, as shown in FIGS. 5 and 6, the device determines the target video content in the video source according to the relative position, including: the device determines the connection between the user's eyes and the center point of the display screen when the user is in the relative position. The first intersection point of the video source, the video content in the video source centered at the first focus and with the same aspect ratio as the display screen is used as the target video content, that is, the device determines the center point of the target video content, the target video content and the display device Have the same aspect ratio.

In one embodiment, the device determines the target video content from the video source according to the relative position of the user and the display device, including: the device determines the cropping strategy of the video source according to the relative position of the user and the display device, and obtains the video source from the video source according to the cropping strategy. Crop the target video content to obtain the target video content.

The video source may be at least one of a video with a resolution greater than or equal to 4K, a video with a viewing angle greater than or equal to 140 degrees, or a long strip video. It should be understood that the viewing angle greater than or equal to 140 degrees means that the horizontal angle range of the video source in FIG. 5 and/or the vertical angle range of the video source in FIG. 6 are both greater than or equal to 140 degrees.

S203: Play the target video content on the display screen of the display device.

In one embodiment, after determining the target video content, the device sends the target video content to the display screen of the display device, and plays the target video content on the display screen.

In an embodiment, the video source may be at least one of a video with a resolution of 4K or more, a video with a viewing angle of 140 degrees or more, or a long strip video. Among them, the long-strip video is mainly suitable for scenes that move horizontally and do not move vertically, such as advertising videos on the wall.

In one embodiment, such as the broadcast of a sports game, the video source is a video with a resolution greater than or equal to 4K and a viewing angle greater than or equal to 140 degrees. As shown in Figure 13, when the display screen of the display device plays video content, the video output range of the display screen displays part of the video content in the entire video source.

The device acquires the relative position of the user and the display device in real time or periodically, and correspondingly determines the target video content in the video source in real time or periodically. When the device obtains the relative position of the user and the display, when the relative position of the user and the display device changes as shown in FIG. 13, as shown in FIG. 14, when the user moves to the left relative to the display device, the device changes according to the user and the display device after the exercise. The relative position of the display device determines the target video content from the video source. The method for determining the target video content by the device is shown in Fig. 5, Fig. 6 or Fig. 7 and Fig. 8, which is a concise description and will not be repeated here. The target video content determined by the device is shown in Figure 14. In Figure 14, the user moves to the left relative to the display device, and the user's line of sight moves to the right relative to the user's line of sight in Figure 13, to determine the target video content, and finally display the determined target video content on the display screen. The effect of video content following the user's movement is realized.

In one embodiment, the video source is a long strip video, such as a video played on an advertising machine, as shown in Figure 15, Figure 16, and Figure 17. In Figure 15, Figure 16, and Figure 17, only the horizontal part of the video content is displayed .

In Figure 15, the dashed box is part of the video content displayed by default. The device obtains the relative position of the user and the display device in real time or periodically, and determines the target video content from the video source according to the relative position.

When the video source is a strip video, in one embodiment, the device determines the target video content according to the relative position, including: the device determines the user's line of sight angle when the user is in the relative position, including angle a and angle b, as shown in FIG. 4, The device determines the target video content from the video source according to the intersection a and angle b, specifically: the device determines that the center point of the video content and the center point of the display screen are projected on the xy plane and the angle between the y axis is angle a , And the projection of the line connecting the center point of the video content and the center point of the display screen on the yz plane and the z-axis is the angle b. The video content is the target video content, as shown in Figure 5 and Figure 6, the specific description It is the same as the description in FIG. 5 and FIG. 6, and will not be repeated here for concise description.

In one embodiment, the device determines the target video content according to the relative position, including: the device determines the first intersection between the line of the user's eyes and the center point of the display screen and the video source when the user is in the relative position, and the first intersection of the video source A video content with an intersection as the center point and the same aspect ratio as the display screen is used as the target video content.

In one embodiment, the device determines the target video content in the video source according to the relative position, including: the device determines the target video content according to the relative position of the user and the display device, and the speed of the user's movement.

When the device determines that the user moves relative to the display device, such as moving from FIG. 15 to the position shown in FIG. 16, the device is based on the relative position of the user and the display device, or the device is based on the relative position of the user and the display device, and the user moves from FIG. 15 The moving speed to the position in FIG. 16 determines the target video content from the video source, so that the played video content is always at the closest distance to the user according to the user's movement, which improves the user experience.

In an embodiment, FIG. 16 may also be used as a schematic diagram of video content playback before the user moves in FIG. 17. When the device determines the user's moving position, that is, from the position shown in FIG. 16 to the position shown in FIG. 17, the device is based on the relative position of the user and the display device, or the device is based on the relative position of the user and the display device, and the user moves from the position shown in FIG. The moving speed to the position in FIG. 17 determines the target video content from the video source, so that the played video content is always at the closest distance to the user according to the user's movement, which improves the user experience.

When the video source is a long strip video, it can be used for display applications, such as an advertising machine, etc. The video screen moves with the user, which can attract the user's attention, extend the time for the video content (such as advertisement) to be close to the user, and increase the commercial value .

Optionally, in one embodiment, the method further includes: in the initial state, that is, when the display screen just plays the video content, the display screen displays the preset video content (also called the second video content), as shown in FIG. 2 As shown, the display screen displays the 4K resolution video content in the middle; or, as shown in Figure 15, the display screen displays part of the video content in the horizontal direction.

Optionally, in an embodiment, as shown in FIG. 18, the method may further include:

S204: Obtain user action information.

In one embodiment, the sensor of the device acquires the user's action information. The motion information may include gesture information, such as telescope gestures, open hands outwards, close hands inward, make camera gestures with both hands, and move forward with one hand.

S205: The apparatus determines the target video content according to the relative position and action information of the user and the display device.

In one embodiment, the device determining the target video content according to the relative position of the user and the display device and the action information includes: the device determines the first video content in the video source according to the relative position; the device determines and stores the presentation of the video content according to the action information Rules, where the presentation rule corresponds to the action information; the device determines the target video content according to the first video content and the presentation rule.

In one embodiment, the device determining the target video content according to the first video content and the presentation rule includes: the device zooms in or out of the first video content with a center point of the first video content as a center to determine the target video content.

In one embodiment, when the motion information is a telescope gesture, the device zooms in or out of the currently viewed video content according to the telescope gesture according to a preset zoom-in or zoom-out ratio factor, and obtains the target video content through a zoom strategy.

For example, the video content currently being watched by the user is shown in Fig. 19a. When the camera of the display device acquires the user’s telescope gesture, the device enlarges the video content currently being watched, and the line of sight is narrowed, as shown in Fig. 19b.

In an embodiment, whether the device zooms in or out of the currently viewed video content according to the telescope gesture can be set in advance according to user requirements, which is not limited in this embodiment.

For another example, when the action information is a gesture of opening both hands outward, the device enlarges the video content, narrows the line of sight, and displays the enlarged video content on the display screen.

When the action information is a gesture of folding your hands inward, the device reduces the video content, widens the line of sight, and displays the reduced video on the display screen, as well as other video content within the line of sight after the widened line of sight.

When the action information is the gesture of the camera gesture with both hands, the device intercepts the current video content and displays the process of intercepting the current video content on the display screen.

When the action information is a single-hand forward gesture, the device keeps the currently playing video content still, records a picture of the current video content, and displays the process of recording the current video content on the display screen, and so on.

It should be noted that the display device shown in FIG. 18 obtains the user's action information and controls the video content to be played according to the action information. It can be used alone or combined with the display device in FIG. 12 to obtain the relative position of the user and the display device. Control the video content played.

The embodiment of the present application provides a device for playing video content according to user motion control. As shown in FIG. 20, the device 300 includes a receiving unit 310, a determining unit 320, and a sending unit 330.

The receiving unit 310 is configured to receive the position of the user acquired by the sensor when the user watches the video.

The determining unit 320 is configured to determine target video content in the video source according to the location of the user, and the target video content is the video content in the video source that matches the line of sight of the user at the location.

The sending unit 330 is configured to transmit the target video content to the display screen, so that the display screen plays the target video content, which realizes that the video content follows the user's movement and improves the user experience.

In one embodiment, the determining unit 320 is specifically configured to determine the line-of-sight angle of the user at the location according to the position of the user, the line-of-sight angle is used to reflect the line-of-sight range, the line-of-sight angle is the spatial angle of the line of sight of the user, and the line of sight of the user is the user The connection between the eye and the center point of the display screen at the position; the target video content is determined from the video source according to the line of sight angle, and the target video is also the video content matching the user's line of sight angle.

Wherein, the line of sight angle includes a first angle and a second angle. In one embodiment, the first angle is the angle between the user's line of sight and the y-axis when the user's line of sight is mapped onto the xy plane. The xy plane is the plane formed by the x-axis and the y-axis; When on the yz plane, the included angle with the z axis, the yz plane is the plane formed by the y axis and the z axis; where the y axis is perpendicular to the plane where the display is located, and the z axis is perpendicular to the ground and parallel to where the display is located The plane, the x-axis and the z-axis form the xz plane, the xz plane is the plane where the display screen is located, the x-axis, the y-axis, and the z-axis intersect at the origin of the coordinate axis, and the origin of the coordinate axis is the center point of the display screen.

In one embodiment, the determining unit 320 determines the target video content from the video source according to the first angle and the second angle. The projection of the line between the center point of the target video content and the center point of the display screen on the xy plane and the y axis The included angle of is the first angle, and the video content whose projection on the yz plane and the z-axis of the line connecting the center point of the target video content and the center point of the display screen is the second angle is the target video content.

In one embodiment, the position of the user and the display device is the distance L1 between the user and the center point of the display screen, the video source includes the video shooting distance L2 and the viewing angle range of the video content, and L2 is the distance from the video content of the video source to the center point of the display screen. The distance; the determining unit 320 is specifically configured to determine the target video content according to the line of sight angle, L1, L2, the viewing angle range, and the size of the display screen.

Among them, the viewing angle range includes the horizontal display viewing angle and the vertical display viewing angle. The horizontal display viewing angle is the display range angle where the video content of the video source is mapped to the xy plane, and the vertical display viewing angle is the display range angle where the video content of the video source is mapped to the yz plane. The size of the display screen includes the width D of the display screen and the height H of the display screen; the determining unit 320 is specifically configured to: determine the target video content in the xy plane according to the first angle, L1, L2, the horizontal display angle of view, and the width D of the display screen According to the second angle, L1, L2, the vertical display angle of view, and the height H of the display screen, determine the display range of the target video content on the yz plane.

In another embodiment, the determining unit 320 is specifically configured to: determine the first intersection point between the line of the user's eyes and the center point of the display screen and the video source when the user is at the position; take the first intersection point as the video source The video content with the center point and the same aspect ratio as the display screen is taken as the target video content.

In another embodiment, the determining unit 320 is specifically configured to determine the target video content according to the position of the user and the speed of the user's movement.

In one embodiment, the receiving unit 310 is further configured to receive the acquired action information of the user; the determining unit 320 is further configured to determine the target video content according to the user's position and action information.

In one embodiment, the determining unit 320 is specifically configured to: determine the first video content in the video source according to the location of the user; determine the presentation rule of the pre-stored video content according to the action information, and the presentation rule corresponds to the action information; Video content and presentation rules determine the target video content.

In one embodiment, the motion information includes at least one of telescope gesture information, information about a gesture of pushing both hands forward, and a gesture of moving both hands backward.

In one embodiment, when the camera acquires telescope gesture information, the determining unit 320 enlarges or reduces the fourth video content according to the telescope gesture information.

When the camera acquires the gesture information of opening the hands outward, the determining unit 320 enlarges the video content, narrows the field of view, and displays the enlarged video content on the display screen.

When the camera acquires the gesture information that the hands are folded inward, the determining unit 320 reduces the video content, widens the field of view, and displays the reduced video content on the display screen.

When the camera acquires the gesture information of both hands making the camera posture, the determining unit 320 captures the current video content and displays the screenshot process on the display screen.

When the camera acquires the gesture information of one palm forward, the determining unit 320 keeps the video content still, records the current video content screen, and displays the current video content recording screen on the display screen, and so on.

In an embodiment, the determining unit 320 is specifically configured to: zoom in or zoom out the first video content with the center point of the first video content as the center to determine the target video content.

In an embodiment, the determining unit 320 is specifically configured to determine a cropping or zooming strategy of the video source according to the location of the user to obtain the target video content.

In one embodiment, the video source includes at least one of the following: a video with a resolution greater than or equal to 4K, a video with a field of view greater than or equal to 140 degrees, or a long strip video.

In an embodiment, the sending unit 330 is further configured to transmit the preset second video content to the display screen, so that the display screen plays the second video content.

The function of each functional unit in the device can be realized by the steps performed by the device in the embodiment shown in FIG. 12 and FIG. 18, and achieve the corresponding technical effect. Therefore, the device provided by the embodiment of the present invention The specific working process of the company will not be repeated here.

The embodiment of the present application provides a device for playing video content according to user motion control. As shown in FIG. 21, the device 400 includes a transmission interface 410 and a processor 420. It should be understood that the transmission interface 410 is an interface for receiving or sending data in the device. The transmission interface may also be a part of the processor 420, and the transmission interface and the processor belong to the same chip.

The transmission interface 410 is used to receive the position of the user acquired by the sensor when the user watches the video.

The processor 420 is configured to determine target video content in the video source according to the location of the user, where the target video content is the video content in the video source that matches the line of sight of the user at the location.

The transmission interface 410 is also used to transmit the target video content to the display screen, so that the display screen plays the target video content, which realizes that the video content follows the user's movement and improves the user experience.

In an embodiment, the processor 420 is specifically configured to: determine the line of sight angle of the user at the position according to the user's position, the line of sight angle is used to reflect the line of sight range, the line of sight angle is the spatial angle of the line of sight of the user, and the line of sight of the user is The connection between the user's eyes and the center point of the display screen when the user is at the position; the target video content is determined from the video source according to the line of sight angle, and the target video content is the video content matching the line of sight angle of the user.

In one embodiment, the processor 420 is specifically configured to determine the target video content from the video source according to the first angle and the second angle, and the connection between the center point of the target video content and the center point of the display screen is on the xy plane. The angle between the projection and the y-axis is the first angle, and the connection between the center point of the target video content and the center point of the display screen on the yz plane is the second angle between the projection and the z-axis is the target video content.

In one embodiment, the position of the user and the display device is the distance L1 between the user and the center point of the display screen, the video source includes the video shooting distance L2 and the viewing angle range of the video content, and L2 is the distance from the video content of the video source to the center point of the display screen. Distance: The processor 420 is specifically configured to determine the target video content according to the line-of-sight angle, L1, L2, viewing angle range, and size of the display screen.

Among them, the viewing angle range includes the horizontal display viewing angle and the vertical display viewing angle. The horizontal display viewing angle is the display range angle where the video content of the video source is mapped to the xy plane, and the vertical display viewing angle is the display range angle where the video content of the video source is mapped to the yz plane. The size of the display screen includes the width D of the display screen and the height H of the display screen; the processor 420 is specifically configured to: determine the target video content in the xy plane according to the first angle, L1, L2, the horizontal display angle of view, and the width D of the display screen According to the second angle, L1, L2, the vertical display angle of view, and the height H of the display screen, determine the display range of the target video content on the yz plane.

In another embodiment, the processor 420 is specifically configured to: determine the first intersection point between the line of the user's eyes and the center point of the display screen and the video source when the user is at the position; and take the first intersection point in the video source as The video content with the center point and the same aspect ratio as the display screen is taken as the target video content.

In another embodiment, the processor 420 is specifically configured to determine the target video content according to the position of the user and the speed of the user's movement.

In an embodiment, the transmission interface 410 is also used to receive the acquired user's action information; the processor 420 is also used to determine the target video content according to the user's location and action information.

In one embodiment, the processor 420 is specifically configured to: determine the first video content in the video source according to the location of the user; determine the presentation rule of the pre-stored video content according to the action information, and the presentation rule corresponds to the action information; Video content and presentation rules determine the target video content.

In one embodiment, when the camera acquires telescope gesture information, the processor 420 zooms in or out of the fourth video content according to the telescope gesture information.

When the camera acquires the gesture information of the open hands, the processor 420 enlarges the video content, narrows the field of view, and displays the enlarged video content on the display screen.

When the camera acquires the gesture information that the hands are folded inward, the processor 420 reduces the video content, widens the field of view, and displays the reduced video content on the display screen.

When the camera acquires the gesture information of both hands making the camera posture, the processor 420 captures the current video content and displays the screenshot process on the display screen.

When the camera acquires the gesture information of one palm forward, the processor 420 keeps the video content still, records a picture of the current video content, and displays the picture of the current video content on the display screen, and so on.

In an embodiment, the processor 420 is specifically configured to: zoom in or zoom out the first video content with the center point of the first video content as the center, so as to determine the target video content.

In one embodiment, the processor 420 is specifically configured to determine a cropping or scaling strategy of the video source according to the user's position, so as to obtain the target video content.

In one embodiment, the transmission interface 410 is also used to transmit the preset second video content to the display screen, so that the display screen can play the second video content.

Optionally, in an embodiment, the apparatus 400 further includes a memory 430 for storing information such as the relative position of the user and the display device.

In an embodiment, as shown in FIG. 22, the device 400 may further include a display screen 440.

In an embodiment, as shown in FIG. 22, the device 400 may further include a camera 450, and the camera includes a sensor.

The function of each functional device in the device can be realized by the steps performed by the device in the embodiment shown in FIG. 12 and FIG. 18, and achieve the corresponding technical effect. Therefore, the device provided in the embodiment of the present application The specific working process of the company will not be repeated here.

The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores instructions. When the instructions run on a computer or a processor, the computer or the processor executes the steps in FIG. 12 and FIG. 18 Methods.

The embodiment of the present application also provides a computer program product containing instructions, which when it runs on a computer or a processor, causes the computer or the processor to execute the method of each step in FIG. 12 and FIG. 18.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Should be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for playing video content according to user motion control, characterized in that the method includes:

Acquiring a relative position between the user and the display device, where the relative position is acquired by a sensor when the display device is playing a video;

Determining target video content in a video source according to the relative position, where the target video content is video content in the video source that matches the user's line of sight range when the user is at the relative position;

The target video content is played on the display screen of the display device.
The method according to claim 1, wherein the determining the target video content in the video source according to the relative position comprises:

Determine the line of sight angle of the user at the relative position according to the relative position, the line of sight angle is used to reflect the line of sight range, the line of sight angle is the spatial angle of the line of sight of the user, and the line of sight of the user Is the connection line between the user's eyes and the center point of the display screen when the user is in the relative position;

The target video content is determined from the video source according to the line of sight angle, and the target video content is video content matching the line of sight angle of the user.
The method according to claim 2, wherein the line of sight angle includes a first angle and a second angle; the first angle is when the line of sight of the user is mapped onto the xy plane, the line of sight corresponds to the y axis. The included angle, the xy plane is the plane formed by the x-axis and the y-axis; the second angle is the included angle with the z-axis when the line of sight of the user is mapped onto the yz plane, the yz plane Is the plane formed by the y-axis and the z-axis; wherein, the y-axis is perpendicular to the plane where the display screen is located, the z-axis is perpendicular to the ground and parallel to the plane where the display screen is located, the The x axis and the z axis form an xz plane, the xz plane is the plane where the display screen is located, the x axis, the y axis, and the z axis intersect at the origin of the coordinate axis, and the origin of the coordinate axis is The center point of the display screen.
The method according to claim 3, wherein:

The determining the target video content from the video source according to the line of sight angle includes:

The target video content is determined from the video source according to the first angle and the second angle, and the connection line between the center point of the target video content and the center point of the display screen is on the xy plane The angle between the projection of and the y-axis is the first angle, and the projection of the line between the center point of the target video content and the center point of the display screen on the yz plane and the z-axis The video content whose included angle is the second angle is the target video content.
The method according to claim 3, wherein the relative position is the distance L1 between the user and the center point of the display screen, the video source includes a video shooting distance L2 and a video content viewing angle range, and the L2 is The distance from the video content of the video source to the center point of the display screen;

The determining the target video content in the video source according to the relative position includes:

The target video content is determined according to the line of sight angle, the L1, the L2, the viewing angle range, and the size of the display screen.
The method according to claim 5, wherein the viewing angle range includes a horizontal display viewing angle and a vertical display viewing angle, and the horizontal display viewing angle is a display range angle of the video content of the video source mapped onto the xy plane. The vertical display angle of view is the display range angle of the video content of the video source mapped onto the yz plane; the size of the display screen includes the width D of the display screen and the height H of the display screen;

Determining the target video content according to the line of sight angle, the L1, the L2, the viewing angle range, and the size of the display screen includes:

Determining the display range of the target video content on the x-y plane according to the first angle, the L1, the L2, the horizontal display angle of view, and the width D of the display screen;

The display range of the target video content on the y-z plane is determined according to the second angle, the L1, the L2, the vertical display angle of view, and the height H of the display screen.
The method according to claim 1, wherein the determining the target video content in the video source according to the relative position comprises:

Determining the first intersection point between the line connecting the user's eyes and the center point of the display screen and the video source when the user is in the relative position;

Taking the video content of the video source that has the first intersection as the center point and has the same aspect ratio as the display screen as the target video content.
The method according to any one of claims 1 to 7, wherein the determining the target video content in the video source according to the relative position comprises:

The target video content is determined according to the relative position and the speed of the user's movement.
The method according to any one of claims 1 to 8, wherein the method further comprises:

Acquiring the user's action information;

The target video content is determined according to the relative position and the action information.
The method according to claim 9, wherein the determining the target video content according to the relative position and the action information comprises:

Determining the first video content in the video source according to the relative position;

Determining a pre-stored presentation rule of the video content according to the action information, the presentation rule corresponding to the action information;

The target video content is determined according to the first video content and the presentation rule.
The method according to claim 9 or 10, wherein the motion information includes at least one of telescope gesture information, information about a gesture of pushing both hands forward, and a gesture of moving both hands backward.
The method according to claim 10 or 11, wherein the determining the target video content according to the first video content and the presentation rule comprises:

Enlarging or reducing the first video content with a center point of the first video content as a center to determine the target video content.
The method according to any one of claims 1 to 12, wherein the determining the target video content in the video source according to the relative position comprises:

The cropping or scaling strategy of the video source is determined according to the relative position to obtain the target video content.
The method according to any one of claims 1 to 13, wherein the video source includes at least one of the following: a video with a resolution greater than or equal to 4K, a video with a viewing angle greater than or equal to 140 degrees, or a long strip video .
A device for playing video content following user motion control, characterized in that the device includes a processor and a transmission interface,

The transmission interface is configured to receive the user's position obtained by the sensor when the user is watching the video;

The processor is configured to determine target video content in a video source according to the location, where the target video content is the video content in the video source that matches the line of sight of the user at the location;

The transmission interface is also used to transmit the target video content to the display screen, so that the display screen plays the target video content.
The device according to claim 15, wherein the processor is specifically configured to:

Determine the line-of-sight angle of the user at the location according to the position, the line-of-sight angle is used to reflect the line-of-sight range, the line-of-sight angle is the spatial angle of the line of sight of the user, and the line of sight of the user The connection between the user's eyes and the center point of the display screen when the user is at the position;

The target video content is determined from the video source according to the line of sight angle, and the target video content is video content matching the line of sight angle of the user.
The device according to claim 16, wherein the line of sight angle includes a first angle and a second angle; the first angle is the line of sight of the user when mapped onto the xy plane, and the line of sight The included angle, the xy plane is the plane formed by the x-axis and the y-axis; the second angle is the included angle with the z-axis when the line of sight of the user is mapped onto the yz plane, the yz plane Is the plane formed by the y-axis and the z-axis; wherein, the y-axis is perpendicular to the plane where the display screen is located, the z-axis is perpendicular to the ground and parallel to the plane where the display screen is located, the The x axis and the z axis form an xz plane, the xz plane is the plane where the display screen is located, the x axis, the y axis, and the z axis intersect at the origin of the coordinate axis, and the origin of the coordinate axis is The center point of the display screen.
The device according to claim 17, wherein the processor is specifically configured to:

The target video content is determined from the video source according to the first angle and the second angle, and the connection line between the center point of the target video content and the center point of the display screen is on the xy plane The angle between the projection of and the y-axis is the first angle, and the projection of the line between the center point of the target video content and the center point of the display screen on the yz plane and the z-axis The video content whose included angle is the second angle is the target video content.
The device according to claim 17, wherein the position is the distance L1 between the user and the center point of the display screen, the video source includes a video shooting distance L2 and a video content viewing angle range, and the L2 is the The distance from the video content of the video source to the center point of the display screen; the processor is specifically configured to:

The target video content is determined according to the line of sight angle, the L1, the L2, the viewing angle range, and the size of the display screen.
The device according to claim 19, wherein the viewing angle range comprises a horizontal display viewing angle and a vertical display viewing angle, and the horizontal display viewing angle is a display range angle of the video content of the video source mapped onto the xy plane. The vertical display angle of view is the angle of the display range where the video content of the video source is mapped onto the yz plane; the size of the display screen includes the width D of the display screen and the height H of the display screen; The processor is specifically used for:

Determining the display range of the target video content on the x-y plane according to the first angle, the L1, the L2, the horizontal display angle of view, and the width D of the display screen;

The display range of the target video content on the y-z plane is determined according to the second angle, the L1, the L2, the vertical display angle of view, and the height H of the display screen.
The device according to claim 15, wherein the processor is specifically configured to:

Determining the first intersection point between the line connecting the user's eyes and the center point of the display screen and the video source when the user is at the position;

Taking the video content of the video source that has the first intersection as the center point and has the same aspect ratio as the display screen as the target video content.
The device according to any one of claims 15 to 21, wherein the processor is specifically configured to:

The target video content is determined according to the position and the speed of the user's movement.
The device according to any one of claims 15 to 22, characterized in that:

The transmission interface is also used to receive the acquired action information of the user;

The processor is further configured to determine the target video content according to the position and the action information.
The device according to claim 23, wherein the processor is specifically configured to:

Determining the first video content in the video source according to the location;

Determining a pre-stored presentation rule of the video content according to the action information, the presentation rule corresponding to the action information;

The target video content is determined according to the first video content and the presentation rule.
The device according to claim 23 or 24, wherein the motion information includes at least one of telescope gesture information, information about a gesture of pushing both hands forward, and a gesture of moving both hands backward.
The device according to claim 24 or 25, wherein the processor is specifically configured to:

Enlarging or reducing the first video content with a center point of the first video content as a center to determine the target video content.
A computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium, and when the instructions run on a computer or a processor, the computer or the processor executes the 14. The method of any one.
A computer program product containing instructions, which is characterized in that when it runs on a computer or a processor, the computer or the processor executes the method according to any one of claims 1 to 14.