CN114205669A - Free visual angle video playing method and device and electronic equipment - Google Patents
Free visual angle video playing method and device and electronic equipment Download PDFInfo
- Publication number
- CN114205669A CN114205669A CN202111611869.6A CN202111611869A CN114205669A CN 114205669 A CN114205669 A CN 114205669A CN 202111611869 A CN202111611869 A CN 202111611869A CN 114205669 A CN114205669 A CN 114205669A
- Authority
- CN
- China
- Prior art keywords
- playing
- visual angle
- determining
- video
- free
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/65—Transmission of management data between client and server
- H04N21/658—Transmission by the client directed to the server
- H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection
Abstract
The embodiment of the invention relates to the technical field of data processing, and discloses a free visual angle video playing method and device and electronic equipment. The method comprises the following steps: when a user watches a free visual angle video, tracking the eyeball action of the user so as to determine the attention position of the user to the free visual angle video on a playing screen; and switching the playing visual angle of the free visual angle video according to the attention position. Through the mode, the embodiment of the invention improves the visual angle switching efficiency and improves the watching experience of users.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a free visual angle video playing method and device and electronic equipment.
Background
With the continuous development of network technology, the watching amount of videos is higher and higher. Compared with the common single-view video, the free-view video enables a user to watch the video based on a plurality of different views, and the watching experience of the user is improved.
In the related art, when a user watches a free view video, if the user needs to change the watching view, the user must manually operate the playing device. The playback device determines a new video playback angle of view from a manual operation signal of a user, and plays the free-angle-of-view video based on the new video playback angle of view. However, the inventors found in the course of implementing embodiments of the present invention that: the process of switching the playing view angle of the free view angle video in the related art is complicated, so that the attention of a user is dispersed, and the watching experience of the user is influenced.
Disclosure of Invention
In view of the foregoing problems, embodiments of the present invention provide a method and an apparatus for playing a free-view video, and an electronic device, so as to solve the problem that a view switching process is complicated in the prior art.
According to an aspect of an embodiment of the present invention, there is provided a method for playing a free-view video, the method including:
when a user watches a free visual angle video, tracking the eyeball action of the user so as to determine the attention position of the user to the free visual angle video on a playing screen;
and switching the playing visual angle of the free visual angle video according to the attention position.
In an optional manner, the focus position includes a gaze point of a user on a playing screen, and the switching the playing perspective of the free perspective video according to the focus position includes:
determining a trigger point for triggering the switching of the visual angle according to the fixation point, and determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen;
determining the sum of the covering angle differences of adjacent machine positions between the target boundary machine position and the playing main machine position according to the offset direction;
and determining a visual angle switching angle according to the sum of the adjacent machine position covering angle differences and the offset proportion, and switching the playing visual angle of the free visual angle video according to the visual angle switching angle.
In an alternative mode, the determining the offset direction and the offset ratio of the trigger point relative to the center point of the playing screen includes:
determining the screen position of the trigger point, and determining the offset direction of the trigger point relative to the central point of the playing screen according to the position relationship between the screen position and the screen position of the central point of the playing screen;
determining a boundary point of a playing screen corresponding to the offset direction;
and determining the coordinate ratio of the trigger point to the boundary point of the playing screen on a preset screen coordinate axis as the offset proportion of the trigger point relative to the central point of the playing screen.
In an alternative mode, before the determining the offset direction and the offset proportion of the trigger point relative to the center point of the playing screen, the method includes:
determining whether the trigger point is located in a preset trigger area;
and when the trigger point is positioned in the preset trigger area, the step of determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen is executed.
In an optional manner, the determining the view switching angle according to the offset ratio and the sum of the adjacent airplane coverage angle differences includes:
calculating the product of the sum of the adjacent machine position covering angle differences and the offset proportion;
determining the product as a view switching angle.
In an optional manner, before tracking the eyeball motion of the user to determine the attention position of the user to the free-view video on the playing screen when the user watches the free-view video, the method further comprises:
acquiring a free visual angle video to be played and machine position information corresponding to the free visual angle video to be played from a video cloud;
determining a manufacturing host position of the free visual angle video to be played according to the machine position information;
and determining the manufacturing host position as a playing host position for playing the free visual angle video to be played, and playing the free visual angle video to be played according to the visual angle of the playing host position.
In an optional manner, the determining, according to the offset direction, a sum of adjacent position coverage angle differences between the target boundary position and the play main position includes:
determining a total machine position number and all adjacent machine position covering angle differences according to the machine position information;
determining a target boundary machine position according to the offset direction and the total machine position;
and determining the sum of the adjacent machine position coverage angle differences between the target boundary machine position and the play main machine position according to the target boundary machine position and the all adjacent machine position coverage angle differences.
According to another aspect of the embodiments of the present invention, there is provided a free-view video playing apparatus, including:
the determining module is used for tracking the eyeball action of the user when the user watches the free visual angle video so as to determine the attention position of the user to the free visual angle video on a playing screen;
and the switching module is used for switching the playing visual angle of the free visual angle video according to the attention position.
According to another aspect of the embodiments of the present invention, there is provided an electronic device including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation of the free-view video playing method.
According to another aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, in which at least one executable instruction is stored, and when the executable instruction is executed on an electronic device, the electronic device executes the operations of the above-mentioned playing method of the freeview video.
In the embodiment of the invention, when a user watches the free visual angle video, along with the change of the main focus area of the user on the free visual angle video, the projection point of the eyeball of the user on the playing screen also changes; based on the tracking of the eyeball action of the user, the attention position of the user to the free view video can be determined on the playing screen, so that the playing view angle of the free view video is switched according to the attention position. Therefore, the method and the device can intelligently identify the main attention area of the user on the free visual angle video, further switch the playing visual angle of the free visual angle video according to the identification result, realize the switching of the playing visual angle of the free visual angle video without manual operation of the user, greatly simplify the switching process of the playing visual angle of the free visual angle video and improve the watching experience of the user.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and the embodiments of the present invention can be implemented according to the content of the description in order to make the technical means of the embodiments of the present invention more clearly understood, and the detailed description of the present invention is provided below in order to make the foregoing and other objects, features, and advantages of the embodiments of the present invention more clearly understandable.
Drawings
The drawings are only for purposes of illustrating embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a schematic flow chart illustrating a playing method of a free-view video according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a camera position arrangement of a camera provided by an embodiment of the invention;
FIG. 3 is a schematic diagram illustrating a gaze point route of a user provided by an embodiment of the invention;
FIG. 4 is a diagram illustrating a display screen provided by an embodiment of the present invention;
fig. 5 is a schematic structural diagram illustrating a playing apparatus for freeview video according to an embodiment of the present invention;
fig. 6 shows a schematic structural diagram of an electronic device provided in an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein.
Fig. 1 is a flowchart illustrating a playing method of a freeview video according to an embodiment of the present invention, where the method is executed by an electronic device. The memory of the electronic device is used for storing at least one executable instruction, and the executable instruction causes the processor of the electronic device to execute the operation of the free-view video playing method.
As shown in fig. 1, the method comprises the steps of:
step 110: when a user watches the free view video, the eyeball action of the user is tracked to determine the attention position of the user to the free view video on a playing screen.
The free view video is a video that can be viewed using a plurality of views. When a free view video is manufactured, a certain number of cameras need to be deployed on a shooting site for video acquisition, and then the videos acquired by each camera are synchronously aligned, spliced and transcoded. When video acquisition is carried out, the total number of the camera is firstly determined according to the shooting site and the arrangement of the camera positions, and then the shooting focus is determined according to the video content. For example, when the video content is a basketball game, the video content may beTo determine the shooting focus as basketball. Furthermore, each machine position of the camera can be numbered according to a certain sequence, and the current manufacturing main machine position, the total covering angle of all the machine positions and the covering angle difference of each adjacent machine position are calculated in real time according to the arrangement of each machine position and the distance from each machine position to the shooting focus. For example, after numbering, the positions of the cameras are C1、C2.....CNThe real-time calculated position of the manufacturing host is Cm(make host site C)mIs C1To CNOne of the above) of all the stands, the total coverage angle of all stands is PVT, and the difference of the coverage angles of adjacent stands is PV(i,j)And i and j represent two adjacent machine positions.
Fig. 2 shows a schematic view of a camera position arrangement of a camera provided by an embodiment of the present invention. As shown in fig. 2, the total number of machine positions N is 17, and the coverage angle difference PV between adjacent machine positions is shown(i,j)Where i is 8, j is 9, the host bit is made to be C9. Further, all adjacent flight coverage angle differences can be expressed as the following array.
PV1,N-1＝[PV(1,2) PV(2,3) ...... PV(N-1,N)]
The total angle of coverage PVT for all stands can be calculated by the following formula.
After video acquisition is completed by each camera in a shooting site (acquisition end), video streams acquired by each machine position and machine position information can be transmitted to a video cloud in a low-delay manner. The bit information CIT of the current time T can comprise a total bit number N and a current production host bit CmThe total coverage angle PVT of all current machine positions and the coverage angle difference of each adjacent machine position are as follows:
CIT＝CI(N,Cm,PVT,[PV(1,2) PV(2,3) ...... PV(N-1,N)])
after the video cloud receives the video stream and the machine position information returned by the shooting site, the N-based video stream and the machine position information can be obtainedAnd finishing synchronous alignment, splicing and transcoding of each channel of machine position video stream by using a video frame timestamp of a TP (Network Time Protocol) clock, and sending the transcoded video stream and the machine position information to a terminal for playing. After the terminal receives the video stream and the machine position information sent by the video cloud, the video stream can be played according to the machine position information. When playing the video stream according to the position information, the playing host position can be set as the making host position CmI.e. default to make host bit CmThe video playing is performed at the visual angle.
When a user watches the free-view video, along with the change of the main attention area of the user to the free-view video, the projection point of the eyeball of the user on the playing screen also changes. Therefore, based on the tracking of the eyeball motion of the user, the attention position of the user to the free-view video can be determined on the playing screen. The user's focus location on the freeview video may include, for example, the user's gaze point on the play screen. Furthermore, the playing screen of the playing device can be gridded, so that each point on the playing screen corresponds to a unique screen coordinate.
Step 120: and switching the playing visual angle of the free visual angle video according to the attention position.
The current optimal playing visual angle can be determined in real time according to the attention position of the user to the free visual angle video, and the playing visual angle of the free visual angle video is switched based on the current optimal playing visual angle. Further, when the playing visual angle of the free visual angle video is switched according to the attention position, a trigger point for triggering visual angle switching can be determined according to the fixation point of the user on the playing screen. For example, the gaze point of the user on the display screen may be extracted by an eye tracking technique, the gaze point constituting a basic unit of measurement, and one gaze point being equal to one sampling point captured by the eye tracker. Further, a fixation point with a duration exceeding a first preset duration may be determined as a fixation point, which is a drop point of the eye locked on the screen; and determining the fixation point with the duration exceeding a second preset time as the trigger point.
Fig. 3 shows a schematic diagram of a gaze point route of a user provided by an embodiment of the present invention. As shown in fig. 3, the gaze point route schematic diagram includes 5 points including (i), (ii), (iii), (iv) and (iv), wherein (i), (ii), (iii), (iv) and (iv) are all gaze points, and the duration of (i) and (iv) exceeds a second preset duration as a trigger point.
After the trigger point for triggering the view switching is determined, the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen can be determined, the sum of the covering angle differences of adjacent machine positions between the target boundary machine position and the playing main machine position is determined according to the offset direction, the view switching angle is determined according to the sum of the covering angle differences of the adjacent machine positions and the offset proportion, and the playing view of the free view video is switched according to the view switching angle. Before determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen, whether the trigger point is located in a preset trigger area can be further determined, and when the trigger point is determined to be located in the preset trigger area, the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen are further determined. Furthermore, a hot map area G can be set on the playing screen according to the free visual angle video watching habit of the user, and an area outside the hot map area G on the playing screen is set as a preset trigger area. The hot map area G is generally a certain area near the center of the playing screen, and a trigger point in this area does not trigger the view switching of the free view video.
When determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen, firstly determining the screen position of the trigger point, and determining the offset direction of the trigger point relative to the central point of the playing screen according to the position relationship between the screen position of the trigger point and the screen position of the central point of the playing screen; further, a boundary point of the playing screen corresponding to the offset direction is determined, and a coordinate ratio of the trigger point to the boundary point of the playing screen on a preset screen coordinate axis is determined as an offset ratio of the trigger point to a center point of the playing screen, where the preset screen coordinate axis may be a horizontal coordinate axis or a vertical coordinate axis.
For example, the preset screen coordinate axis is a coordinate axis in the horizontal direction, the coordinate of the trigger point is C (n, m), n is a coordinate of the trigger point in the horizontal direction, m is a coordinate of the trigger point in the vertical direction, the coordinate of the center point of the playback screen is (0,0), and if n < 0, the offset direction d (direction) of the trigger point with respect to the center point of the playback screen is l (left); if n > 0, the offset direction d (direction) of the trigger point with respect to the center point of the display screen is r (right). L represents the left offset of the trigger point to the center point of the playing screen, and R represents the right offset of the trigger point to the center point of the playing screen. When determining the coordinates of the boundary points of the playing screen on the coordinate axes of the preset screen, if the trigger point deviates to the left side of the central point of the playing screen, determining the horizontal coordinates of the boundary points on the left side of the playing screen as the coordinates of the boundary points of the playing screen on the coordinate axes of the preset screen; and if the trigger point deviates to the right side of the central point of the playing screen, determining the horizontal coordinate of the boundary point on the right side of the playing screen as the coordinate of the boundary point of the playing screen on the coordinate axis of the preset screen.
Fig. 4 is a schematic diagram of a display screen provided by an embodiment of the present invention. As shown in fig. 4, the X axis is a horizontal coordinate axis, the Y axis is a vertical coordinate axis, X1 is a boundary point on the right side of the playback screen, and X2 is a boundary point on the left side of the playback screen. When the trigger point is shifted to the left side of the center point of the playing screen, the shift proportion S is n/x 2; when the trigger point is shifted to the right of the center point of the playback screen, the shift ratio S is n/x 1.
When a user watches the free visual angle video, the eyeball action of the user is tracked so as to determine the attention position of the user to the free visual angle video on a playing screen, the free visual angle video to be played and the machine position information corresponding to the free visual angle video to be played can be further acquired from the video cloud, the manufacturing host position of the free visual angle video to be played is determined according to the machine position information, the manufacturing host position is determined as the playing host position for playing the free visual angle video to be played, and the free visual angle video to be played is played at the visual angle of the playing host position. When the sum of the covering angle differences of adjacent machine positions between the target boundary machine position and the playing main machine position is determined according to the offset direction, the total machine position and the covering angle differences of all adjacent machine positions can be determined according to the machine position information, and the target boundary machine position is determined according to the offset direction and the total machine position; and determining the sum of the adjacent machine position coverage angle differences between the target boundary machine position and the playing main machine position according to the target boundary machine position and the coverage angle differences of all the adjacent machine positions.
For example, when determining the sum of the coverage angle differences of adjacent positions between the target boundary position and the playback main position according to the offset direction, if the trigger point is offset to the left of the center point of the playback screen, the target boundary position may be determined as C1The sum PVL of the angle differences between the target boundary machine position and the adjacent machine position coverage angle between the target boundary machine position and the play host machine position can be expressed as
If the trigger point deviates to the right of the central point of the playing screen, the target boundary position can be determined as CNThe sum PVL of the angle differences between the target boundary machine position and the adjacent machine position coverage angle between the target boundary machine position and the play host machine position can be expressed as
When the viewing angle switching angle is determined according to the sum of the adjacent airplane coverage angle differences and the offset proportion, the product of the sum of the adjacent airplane coverage angle differences and the offset proportion can be calculated, and the product is determined as the viewing angle switching angle. For example, if the trigger point is shifted to the left of the center point of the playback screen, the view switching angle SA is PVL × S; if the trigger point shifts to the left side of the center point of the playing screen, the view switching angle SA is PVR. Further, the terminal playing device can be smoothly switched to a new playing host position based on the angle of view switching.
In the embodiment of the invention, when a user watches the free visual angle video, along with the change of the main focus area of the user on the free visual angle video, the projection point of the eyeball of the user on the playing screen also changes; based on the tracking of the eyeball action of the user, the attention position of the user to the free view video can be determined on the playing screen, so that the playing view angle of the free view video is switched according to the attention position. Therefore, the method and the device can intelligently identify the main attention area of the user on the free visual angle video, further switch the playing visual angle of the free visual angle video according to the identification result, realize the switching of the playing visual angle of the free visual angle video without manual operation of the user, greatly simplify the switching process of the playing visual angle of the free visual angle video and improve the watching experience of the user.
Fig. 5 is a schematic structural diagram illustrating a playing apparatus of a freeview video according to an embodiment of the present invention. As shown in fig. 5, the apparatus 300 includes: a determination module 310 and a switching module 320.
The determining module 310 is configured to track an eye movement of a user when the user watches a free view video, so as to determine a position of the user focusing on the free view video on a playing screen; the switching module 320 is configured to switch a playing view of the free view video according to the focus position.
In an alternative manner, the focus position includes a user's gaze point on the display screen, and the switching module 320 is configured to:
determining a trigger point for triggering the switching of the visual angle according to the fixation point, and determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen;
determining the sum of the covering angle differences of adjacent machine positions between the target boundary machine position and the playing main machine position according to the offset direction;
and determining a visual angle switching angle according to the sum of the adjacent machine position covering angle differences and the offset proportion, and switching the playing visual angle of the free visual angle video according to the visual angle switching angle.
In an alternative manner, the switching module 320 is configured to:
determining the screen position of the trigger point, and determining the offset direction of the trigger point relative to the central point of the playing screen according to the position relationship between the screen position and the screen position of the central point of the playing screen;
determining a boundary point of a playing screen corresponding to the offset direction;
and determining the coordinate ratio of the trigger point to the boundary point of the playing screen on a preset screen coordinate axis as the offset proportion of the trigger point relative to the central point of the playing screen.
In an alternative manner, the switching module 320 is configured to:
before the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen are determined, determining whether the trigger point is located in a preset trigger area;
and when the trigger point is positioned in the preset trigger area, determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen.
In an alternative manner, the switching module 320 is configured to:
calculating the product of the sum of the adjacent machine position covering angle differences and the offset proportion;
determining the product as a view switching angle.
In an alternative manner, the switching module 320 is configured to:
when the user watches the free visual angle video, tracking the eyeball action of the user so as to obtain the free visual angle video to be played and the machine position information corresponding to the free visual angle video to be played from the video cloud before determining the attention position of the user to the free visual angle video on a playing screen;
determining a manufacturing host position of the free visual angle video to be played according to the machine position information;
and determining the manufacturing host position as a playing host position for playing the free visual angle video to be played, and playing the free visual angle video to be played according to the visual angle of the playing host position.
In an alternative manner, the switching module 320 is configured to:
determining a total machine position number and all adjacent machine position covering angle differences according to the machine position information;
determining a target boundary machine position according to the offset direction and the total machine position;
and determining the sum of the adjacent machine position coverage angle differences between the target boundary machine position and the play main machine position according to the target boundary machine position and the all adjacent machine position coverage angle differences.
In the embodiment of the invention, when a user watches the free visual angle video, along with the change of the main focus area of the user on the free visual angle video, the projection point of the eyeball of the user on the playing screen also changes; based on the tracking of the eyeball action of the user, the attention position of the user to the free view video can be determined on the playing screen, so that the playing view angle of the free view video is switched according to the attention position. Therefore, the method and the device can intelligently identify the main attention area of the user on the free visual angle video, further switch the playing visual angle of the free visual angle video according to the identification result, realize the switching of the playing visual angle of the free visual angle video without manual operation of the user, greatly simplify the switching process of the playing visual angle of the free visual angle video and improve the watching experience of the user.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and a specific implementation of the electronic device is not limited in the specific embodiment of the present invention.
As shown in fig. 6, the electronic device may include: a processor (processor)402, a Communications Interface 404, a memory 406, and a Communications bus 408.
Wherein: the processor 402, communication interface 404, and memory 406 communicate with each other via a communication bus 408. A communication interface 404 for communicating with network elements of other devices, such as clients or other servers. The processor 402 is configured to execute the program 410, and may specifically execute the relevant steps in the above-described embodiment of the playing method for the freeview video.
In particular, program 410 may include program code comprising computer-executable instructions.
The processor 402 may be a central processing unit CPU or an application Specific Integrated circuit asic or one or more Integrated circuits configured to implement embodiments of the present invention. The electronic device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 406 for storing a program 410. Memory 406 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 410 may specifically be invoked by the processor 402 to cause the electronic device to perform the following operations:
when a user watches a free visual angle video, tracking the eyeball action of the user so as to determine the attention position of the user to the free visual angle video on a playing screen;
and switching the playing visual angle of the free visual angle video according to the attention position.
In an alternative approach, where the focus location comprises a user's gaze point on a playback screen, the program 410 is invoked by the processor 402 to cause the electronic device to:
determining a trigger point for triggering the switching of the visual angle according to the fixation point, and determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen;
determining the sum of the covering angle differences of adjacent machine positions between the target boundary machine position and the playing main machine position according to the offset direction;
and determining a visual angle switching angle according to the sum of the adjacent machine position covering angle differences and the offset proportion, and switching the playing visual angle of the free visual angle video according to the visual angle switching angle.
In an alternative, the program 410 is invoked by the processor 402 to cause the electronic device to perform the following operations:
determining the screen position of the trigger point, and determining the offset direction of the trigger point relative to the central point of the playing screen according to the position relationship between the screen position and the screen position of the central point of the playing screen;
determining a boundary point of a playing screen corresponding to the offset direction;
and determining the coordinate ratio of the trigger point to the boundary point of the playing screen on a preset screen coordinate axis as the offset proportion of the trigger point relative to the central point of the playing screen.
In an alternative approach, where the focus location comprises a user's gaze point on a playback screen, the program 410 is invoked by the processor 402 to cause the electronic device to:
before the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen are determined, determining whether the trigger point is located in a preset trigger area;
and when the trigger point is positioned in the preset trigger area, the step of determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen is executed.
In an alternative approach, where the focus location comprises a user's gaze point on a playback screen, the program 410 is invoked by the processor 402 to cause the electronic device to:
calculating the product of the sum of the adjacent machine position covering angle differences and the offset proportion;
determining the product as a view switching angle.
In an alternative approach, where the focus location comprises a user's gaze point on a playback screen, the program 410 is invoked by the processor 402 to cause the electronic device to:
when the user watches the free visual angle video, tracking the eyeball action of the user so as to obtain the free visual angle video to be played and the machine position information corresponding to the free visual angle video to be played from the video cloud before determining the attention position of the user to the free visual angle video on a playing screen;
determining a manufacturing host position of the free visual angle video to be played according to the machine position information;
and determining the manufacturing host position as a playing host position for playing the free visual angle video to be played, and playing the free visual angle video to be played according to the visual angle of the playing host position.
In an alternative approach, where the focus location comprises a user's gaze point on a playback screen, the program 410 is invoked by the processor 402 to cause the electronic device to:
determining a total machine position number and all adjacent machine position covering angle differences according to the machine position information;
determining a target boundary machine position according to the offset direction and the total machine position;
and determining the sum of the adjacent machine position coverage angle differences between the target boundary machine position and the play main machine position according to the target boundary machine position and the all adjacent machine position coverage angle differences.
In the embodiment of the invention, when a user watches the free visual angle video, along with the change of the main focus area of the user on the free visual angle video, the projection point of the eyeball of the user on the playing screen also changes; based on the tracking of the eyeball action of the user, the attention position of the user to the free view video can be determined on the playing screen, so that the playing view angle of the free view video is switched according to the attention position. Therefore, the method and the device can intelligently identify the main attention area of the user on the free visual angle video, further switch the playing visual angle of the free visual angle video according to the identification result, realize the switching of the playing visual angle of the free visual angle video without manual operation of the user, greatly simplify the switching process of the playing visual angle of the free visual angle video and improve the watching experience of the user.
An embodiment of the present invention provides a computer-readable storage medium, where the storage medium stores at least one executable instruction, and when the executable instruction is executed on an electronic device, the electronic device is enabled to execute a playing method of a free-view video in any method embodiment described above.
The embodiment of the invention provides a free visual angle video playing device, which is used for executing the free visual angle video playing method.
Embodiments of the present invention provide a computer program, where the computer program can be called by a processor to enable an electronic device to execute a playing method of a free-view video in any of the above method embodiments.
Embodiments of the present invention provide a computer program product, where the computer program product includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions, when the program instructions are run on a computer, cause the computer to execute the method for playing a free-view video in any of the above method embodiments.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specified otherwise.
Claims (10)
1. A method for playing free-view video, the method comprising:
when a user watches a free visual angle video, tracking the eyeball action of the user so as to determine the attention position of the user to the free visual angle video on a playing screen;
and switching the playing visual angle of the free visual angle video according to the attention position.
2. The method of claim 1, wherein the focus position comprises a gaze point of a user on a playback screen, and wherein switching the playback perspective of the freeview video according to the focus position comprises:
determining a trigger point for triggering the switching of the visual angle according to the fixation point, and determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen;
determining the sum of the covering angle differences of adjacent machine positions between the target boundary machine position and the playing main machine position according to the offset direction;
and determining a visual angle switching angle according to the sum of the adjacent machine position covering angle differences and the offset proportion, and switching the playing visual angle of the free visual angle video according to the visual angle switching angle.
3. The method of claim 2, wherein determining the offset direction and the offset ratio of the trigger point relative to the center point of the display screen comprises:
determining the screen position of the trigger point, and determining the offset direction of the trigger point relative to the central point of the playing screen according to the position relationship between the screen position and the screen position of the central point of the playing screen;
determining a boundary point of a playing screen corresponding to the offset direction;
and determining the coordinate ratio of the trigger point to the boundary point of the playing screen on a preset screen coordinate axis as the offset proportion of the trigger point relative to the central point of the playing screen.
4. The method according to claim 2 or 3, wherein before determining the offset direction and offset ratio of the trigger point relative to the center point of the display screen, the method comprises:
determining whether the trigger point is located in a preset trigger area;
and when the trigger point is positioned in the preset trigger area, the step of determining the offset direction and the offset proportion of the trigger point relative to the central point of the playing screen is executed.
5. The method of claim 2, wherein determining the view switching angle according to the offset ratio and the sum of the adjacent airplane coverage angle differences comprises:
calculating the product of the sum of the adjacent machine position covering angle differences and the offset proportion;
determining the product as a view switching angle.
6. The method according to claim 2, wherein before tracking the eyeball motion of the user to determine the focus position of the user on the free-view video on the playing screen while the user is watching the free-view video, the method further comprises:
acquiring a free visual angle video to be played and machine position information corresponding to the free visual angle video to be played from a video cloud;
determining a manufacturing host position of the free visual angle video to be played according to the machine position information;
and determining the manufacturing host position as a playing host position for playing the free visual angle video to be played, and playing the free visual angle video to be played according to the visual angle of the playing host position.
7. The method of claim 6, wherein determining the sum of the angular differences between the target boundary bit and the play main bit for the neighboring bit coverage according to the offset direction comprises:
determining a total machine position number and all adjacent machine position covering angle differences according to the machine position information;
determining a target boundary machine position according to the offset direction and the total machine position;
and determining the sum of the adjacent machine position coverage angle differences between the target boundary machine position and the play main machine position according to the target boundary machine position and the all adjacent machine position coverage angle differences.
8. An apparatus for playing a free-view video, the apparatus comprising:
the determining module is used for tracking the eyeball action of the user when the user watches the free visual angle video so as to determine the attention position of the user to the free visual angle video on a playing screen;
and the switching module is used for switching the playing visual angle of the free visual angle video according to the attention position.
9. An electronic device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation of the free-view video playing method according to any one of claims 1 to 7.
10. A computer-readable storage medium, wherein at least one executable instruction is stored in the storage medium, and when the executable instruction is executed on an electronic device, the electronic device executes the operation of the playing method of the free-view video according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111611869.6A CN114205669B (en) | 2021-12-27 | 2021-12-27 | Free view video playing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111611869.6A CN114205669B (en) | 2021-12-27 | 2021-12-27 | Free view video playing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114205669A true CN114205669A (en) | 2022-03-18 |
| CN114205669B CN114205669B (en) | 2023-10-17 |
Family
ID=80656616
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111611869.6A Active CN114205669B (en) | 2021-12-27 | 2021-12-27 | Free view video playing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114205669B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114979732A (en) * | 2022-05-12 | 2022-08-30 | 咪咕数字传媒有限公司 | Video stream pushing method and device, electronic equipment and medium |
| CN115103213A (en) * | 2022-06-10 | 2022-09-23 | 咪咕视讯科技有限公司 | Information processing method, device, equipment and computer readable storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102342100A (en) * | 2009-03-09 | 2012-02-01 | 思科技术公司 | System and method for providing three dimensional imaging in network environment |
| CN102347043A (en) * | 2010-07-30 | 2012-02-08 | 腾讯科技（北京）有限公司 | Method for playing multi-angle video and system |
| CN103108126A (en) * | 2013-01-21 | 2013-05-15 | Tcl集团股份有限公司 | Video interactive system, method, interactive glasses and terminals |
| CN104740874A (en) * | 2015-03-26 | 2015-07-01 | 广州博冠信息科技有限公司 | Method and system for playing videos in two-dimension game scene |
| CN106447788A (en) * | 2016-09-26 | 2017-02-22 | 北京疯景科技有限公司 | Watching angle indication method and device |
| CN106791794A (en) * | 2016-12-30 | 2017-05-31 | 重庆卓美华视光电有限公司 | A kind of display device, image processing method and device |
| US20170186234A1 (en) * | 2015-12-27 | 2017-06-29 | Le Holdings (Beijing) Co., Ltd. | Method and device for free viewing of three-dimensional video |
| KR20180079051A (en) * | 2016-12-30 | 2018-07-10 | 엘지전자 주식회사 | Mobile terninal and method for controlling the same |
| CN109799899A (en) * | 2017-11-17 | 2019-05-24 | 腾讯科技（深圳）有限公司 | Interaction control method, device, storage medium and computer equipment |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
| CN113170231A (en) * | 2019-04-11 | 2021-07-23 | 华为技术有限公司 | Method and device for controlling playing of video content following user motion |
-
2021
- 2021-12-27 CN CN202111611869.6A patent/CN114205669B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102342100A (en) * | 2009-03-09 | 2012-02-01 | 思科技术公司 | System and method for providing three dimensional imaging in network environment |
| CN102347043A (en) * | 2010-07-30 | 2012-02-08 | 腾讯科技（北京）有限公司 | Method for playing multi-angle video and system |
| CN103108126A (en) * | 2013-01-21 | 2013-05-15 | Tcl集团股份有限公司 | Video interactive system, method, interactive glasses and terminals |
| CN104740874A (en) * | 2015-03-26 | 2015-07-01 | 广州博冠信息科技有限公司 | Method and system for playing videos in two-dimension game scene |
| US20170186234A1 (en) * | 2015-12-27 | 2017-06-29 | Le Holdings (Beijing) Co., Ltd. | Method and device for free viewing of three-dimensional video |
| CN106447788A (en) * | 2016-09-26 | 2017-02-22 | 北京疯景科技有限公司 | Watching angle indication method and device |
| CN106791794A (en) * | 2016-12-30 | 2017-05-31 | 重庆卓美华视光电有限公司 | A kind of display device, image processing method and device |
| KR20180079051A (en) * | 2016-12-30 | 2018-07-10 | 엘지전자 주식회사 | Mobile terninal and method for controlling the same |
| CN109799899A (en) * | 2017-11-17 | 2019-05-24 | 腾讯科技（深圳）有限公司 | Interaction control method, device, storage medium and computer equipment |
| CN113170231A (en) * | 2019-04-11 | 2021-07-23 | 华为技术有限公司 | Method and device for controlling playing of video content following user motion |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 严德赛 等: "基于数字图像处理的眼球控制精度提高方法", 计算机应用, no. 10, pages 267 - 270 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114979732A (en) * | 2022-05-12 | 2022-08-30 | 咪咕数字传媒有限公司 | Video stream pushing method and device, electronic equipment and medium |
| CN114979732B (en) * | 2022-05-12 | 2023-10-20 | 咪咕数字传媒有限公司 | Video stream pushing method and device, electronic equipment and medium |
| CN115103213A (en) * | 2022-06-10 | 2022-09-23 | 咪咕视讯科技有限公司 | Information processing method, device, equipment and computer readable storage medium |
| CN115103213B (en) * | 2022-06-10 | 2023-10-17 | 咪咕视讯科技有限公司 | Information processing method, apparatus, device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114205669B (en) | 2023-10-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111698390B (en) | Virtual camera control method and device, and virtual studio implementation method and system | |
| JP2022119858A5 (en) | ||
| US20170125064A1 (en) | Method and Apparatus for Automatic Video Production | |
| JP2019512769A5 (en) | ||
| CN114205669B (en) | Free view video playing method and device and electronic equipment | |
| US20170244894A1 (en) | Method and Apparatus for Managing Latency of Remote Video Production | |
| RU2738220C1 (en) | Display control device, display control method and storage medium | |
| US20190132511A1 (en) | Control of viewing angles for 360-degree video playback | |
| US9589317B2 (en) | Image processing apparatus, image processing method, and program | |
| US11272153B2 (en) | Information processing apparatus, method for controlling the same, and recording medium | |
| CN107167077B (en) | Stereoscopic vision measuring system and stereoscopic vision measuring method | |
| JP7042571B2 (en) | Image processing device and its control method, program | |
| JP2020086983A (en) | Image processing device, image processing method, and program | |
| US11652978B2 (en) | Depth map generation device | |
| CN112492231B (en) | Remote interaction method, device, electronic equipment and computer readable storage medium | |
| CN106060523A (en) | Methods for collecting and displaying panoramic stereo images, and corresponding devices | |
| CN108737743A (en) | Video-splicing device based on image mosaic and video-splicing method | |
| CN114339302A (en) | Broadcasting guide method, device, equipment and computer storage medium | |
| KR102176805B1 (en) | System and method for providing virtual reality contents indicated view direction | |
| CN116523962A (en) | Visual tracking method, device, system, equipment and medium for target object | |
| CN109240344A (en) | Method for tracking target, device, pod equipment and aircraft | |
| US20230328355A1 (en) | Information processing apparatus, information processing method, and program | |
| CN108985275B (en) | Augmented reality equipment and display tracking method and device of electronic equipment | |
| CN114037146A (en) | Queuing waiting time length determining method and device | |
| US20240078687A1 (en) | Information processing apparatus, information processing method, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |