WO2023246302A1 - 字幕的显示方法、装置、设备及介质 - Google Patents

字幕的显示方法、装置、设备及介质 Download PDF

Info

Publication number
WO2023246302A1
WO2023246302A1 PCT/CN2023/091203 CN2023091203W WO2023246302A1 WO 2023246302 A1 WO2023246302 A1 WO 2023246302A1 CN 2023091203 W CN2023091203 W CN 2023091203W WO 2023246302 A1 WO2023246302 A1 WO 2023246302A1
Authority
WO
WIPO (PCT)
Prior art keywords
subtitle
layer
display
preset
virtual reality
Prior art date
Application number
PCT/CN2023/091203
Other languages
English (en)
French (fr)
Inventor
庞娜
杨毅平
方迟
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Publication of WO2023246302A1 publication Critical patent/WO2023246302A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43074—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H04N21/4884—Data services, e.g. news ticker for displaying subtitles

Definitions

  • the present disclosure relates to the field of virtual reality technology, and in particular, to a method, device, equipment and medium for displaying subtitles.
  • Virtual Reality (VR) technology includes computers, electronic information, and simulation technologies. Its basic implementation method is to simulate a virtual environment with a computer to give people a sense of immersion in the environment. With the popularity of VR applications, it has become a common scenario to display subtitles in virtual reality space in scenarios such as video display.
  • the present disclosure provides a subtitle display method, device, equipment and medium, which follows the user's line of sight direction to determine the position of the subtitle content rendered in the virtual reality space, Improves the user's viewing experience and renders subtitle content at the corresponding location independent of VR video frames
  • the subtitle layer serves as a carrier for displaying subtitle content, which avoids display distortion of subtitle content and improves the display effect of subtitle content.
  • Embodiments of the present disclosure provide a method for displaying subtitles.
  • the method includes: obtaining subtitle content corresponding to the currently played virtual reality video frame; determining the target space position in the virtual reality panoramic space according to the user's current line of sight direction; A subtitle layer is rendered according to the subtitle content at the target space position, and the subtitle content is synchronously rendered in the subtitle layer.
  • Embodiments of the present disclosure also provide a device for displaying subtitles.
  • the device includes: an acquisition module for acquiring subtitle content corresponding to the currently played virtual reality video frame; and a position determination module for determining the subtitle content according to the user's current line of sight direction. , determine the target space position in the virtual reality panoramic space; the rendering display module is used to render the subtitle layer according to the subtitle content at the target space position, and synchronously render the subtitle content in the subtitle layer.
  • An embodiment of the present disclosure also provides an electronic device.
  • the electronic device includes: a processor; a memory used to store instructions executable by the processor; and the processor is used to read the instruction from the memory.
  • the instructions can be executed and executed to implement the subtitle display method provided by the embodiments of the present disclosure.
  • Embodiments of the present disclosure also provide a computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute the subtitle display method provided by the embodiments of the present disclosure.
  • the subtitle display scheme obtained by the embodiment of the present disclosure obtains the subtitle content corresponding to the currently played virtual reality video frame, determines the target space position in the virtual reality panoramic space according to the user's current sight direction, and determines the target space position according to the subtitle Content renders the subtitle layer, and renders the subtitle content synchronously in the subtitle layer. After determining the subtitle content, follow the user's line of sight direction to determine the position of the subtitle content rendered in the virtual reality space. On the basis of realizing the display of the subtitle content, the user's viewing experience is improved, and the subtitle content is rendered at the corresponding position to Display subtitles as a subtitle layer independent of VR video frames
  • the content carrier avoids distortion of the display of subtitle content and improves the display effect of subtitle content.
  • Figure 1 is a schematic flowchart of a subtitle display method provided by an embodiment of the present disclosure
  • Figure 2 is a schematic diagram of a subtitle display scene provided by an embodiment of the present disclosure
  • Figure 3 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 4 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 5 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 6 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • FIG. 7 is a schematic flowchart of another subtitle display method provided by an embodiment of the present disclosure.
  • Figure 8 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 9 is a schematic flowchart of another subtitle display method provided by an embodiment of the present disclosure.
  • Figure 10 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 11 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 12 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 13 is a schematic flowchart of another subtitle display method provided by an embodiment of the present disclosure.
  • Figure 14A is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 14B is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 15 is a schematic diagram of another subtitle display scene provided by an embodiment of the present disclosure.
  • Figure 16 is a schematic structural diagram of a subtitle display device provided by an embodiment of the present disclosure.
  • Figure 17 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the term “include” and its variations are open-ended, ie, “including but not limited to.”
  • the term “based on” means “based at least in part on.”
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; and the term “some embodiments” means “at least some embodiments”. Relevant definitions of other terms will be given in the description below.
  • embodiments of the present disclosure provide a method for displaying subtitles.
  • this method no longer relies on a fixed mode of displaying subtitle content, but releases the fixed position relationship between the subtitle content and the screen content, and follows the user's movements.
  • the viewing direction adaptively adjusts the display position of the subtitle content to avoid visual distortion of the subtitle content and improve the user's visual viewing experience.
  • FIG. 1 is a schematic flowchart of a subtitle display method provided by an embodiment of the present disclosure.
  • the method can be executed by a subtitle display device, where the device can be implemented using software and/or hardware, and can generally be integrated into an electronic device. As shown in Figure 1, the method includes:
  • Step 101 Obtain subtitle content corresponding to the currently played virtual reality video frame.
  • the corresponding subtitle content can be obtained based on different methods.
  • the audio stream corresponding to the currently played virtual reality video frame is determined, and the audio stream is subjected to speech recognition processing to obtain the subtitle content. For example, the timestamp information corresponding to each audio stream is obtained, based on the timestamp. The time stamp information of the information and the virtual video frame matches the subtitle content corresponding to the virtual reality video frame, etc.
  • the correspondence between each virtual reality video frame and the corresponding subtitle content can be identified and constructed in advance, and the correspondence can be stored in the corresponding preset database, so that the preset can be retrieved by querying Database to obtain the subtitle content corresponding to the currently played virtual reality video frame.
  • Step 102 Determine the target space position in the virtual reality panoramic space according to the user's current line of sight direction.
  • the virtual video frame is no longer used as the basis for determining the display position of the subtitle content.
  • the target space position is determined in the virtual reality panoramic space according to the user's current line of sight direction.
  • the target space position is consistent with the user's current line of sight.
  • the target space position is located in the user's current line of sight direction. This ensures that the target space position in the line of sight is consistent with the line of sight direction. , further providing technical support for the display effect of subtitle content.
  • Step 103 Render the subtitle layer according to the subtitle content at the target space position, and render the subtitle content simultaneously in the subtitle layer.
  • the target space after determining the target space position, in the target space Render the subtitle layer at a certain position, and render the subtitle content in the subtitle layer simultaneously.
  • the subtitle layer will follow the change in the line of sight direction, and the spatial position in the direction of line of sight will be Display the subtitle layer, in which the subtitle content is rendered simultaneously on the subtitle layer. Since the subtitle layer is in the direction of the user's line of sight, it is ensured that the subtitle content can be presented to the user without distortion, ensuring the visual display effect.
  • the subtitle content is rendered separately with a subtitle layer that is independent of the virtual reality video frame, which avoids display distortion of the subtitle content, and the subtitle layer is rendered according to the subtitle content, giving a stronger visual sense of technology and improving the user experience. Visual viewing experience.
  • background display processing can also be performed on the subtitle layer, and background special effects can be added to the subtitle layer to enhance the intelligence of the display.
  • the subtitle layer can also be processed into a layer with higher transparency.
  • the subtitle layer is a backlight sub-layer and a quantum dot matrix sub-image.
  • the quantum color of the quantum dot sub-layer can be any color such as blue
  • the quantum dot sub-layer can be displayed in a gradient form or with uniform brightness, as shown in Figure 3, the backlight sub-layer and the quantum dot sub-layer
  • the subtitle layer formed by superposition can give users a stronger sense of technology visually.
  • light animations can be rendered in the subtitle layer, such as aurora animations, white light flow animations, etc., which will not be described here.
  • the addition of light animations can enhance the technological sense of the display.
  • the subtitle layer can also be dynamically rendered according to the subtitle content.
  • the display quantity of subtitle content is obtained, and a subtitle layer matching the display quantity is rendered, thereby achieving the effect of dynamically displaying the subtitle layer as the subtitle display quantity changes.
  • the display quantity of subtitle content is obtained, and the real-time subtitle width and subtitle real-time height are determined based on the display quantity, the preset unit subtitle width and the preset unit subtitle height.
  • the number of displayed subtitle content refers to the number of all subtitle content displayed in the current virtual reality video frame. For example, if speech recognition obtains the number of subtitle content in the content, one or more words can be recognized or read and a display event of the subtitle content can be triggered, thereby changing the number of displayed subtitle content.
  • the preset unit subtitle width refers to the predetermined width of each subtitle content
  • the preset unit subtitle height refers to the prespecified height of each subtitle content.
  • the subtitle implementation height and the subtitle real-time width are determined so as to dynamically render the subtitle layer based on the subtitle implementation height and the subtitle real-time width.
  • the preset database can be queried according to the display quantity, the preset unit subtitle width and the preset unit subtitle height to obtain the subtitle real-time width and subtitle real-time height.
  • the real-time subtitle width is determined based on the preset subtitle quantity threshold and the unit subtitle width. For example, the product of the preset subtitle quantity threshold and the unit subtitle width is calculated to determine the subtitle real-time width.
  • the number of subtitle lines can be determined based on the preset subtitle quantity threshold and the display quantity. For example, the ratio of the display quantity and the preset subtitle quantity threshold is calculated, and the smallest integer greater than or equal to the ratio is determined as the number of subtitle lines. Then, according to the subtitle line The number and unit subtitle height determine the real-time height of the subtitle, such as calculating the product of the number of subtitle lines and the unit subtitle height to determine the real-time height of the subtitle.
  • the real-time width of the subtitles is determined based on the unit subtitle width and the displayed number, that is, the product of the unit subtitle width and the displayed number is calculated to determine the subtitles Real-time width, and determine the unit subtitle height as the subtitle real-time height.
  • the rendering matches the subtitle content according to the layer width of the unit subtitle and the real-time width of the subtitles.
  • the real-time subtitle layer width where the layer width of the unit subtitle is the width direction of each unit subtitle The unit width by which the subtitle layer increases when increased.
  • the layer height of the unit subtitle is the unit height of the subtitle layer that increases when the height of each unit subtitle increases upward. That is, the subtitle layer can expand and change horizontally and vertically along with the subtitle content.
  • the width of the subtitle layer is determined based on the real-time width of the subtitles displayed in real time (the subtitle content in the figure is " What is displayed here is the subtitle content of the *th line", and the subtitle layer has a gradient brightness). For example, each time one or more subtitle content is newly displayed in real time, the corresponding subtitle content will be displayed according to the width of the unit subtitle, and at the same time, the corresponding subtitle content will be displayed according to the width of the unit subtitle.
  • the layer width extends the subtitle layer.
  • the unit subtitle width is smaller than the unit subtitle layer width to ensure that the subtitle content can be visually wrapped by the subtitle layer. Therefore, it is visually realized that the subtitle background is displayed following the display content of the subtitle in the width direction.
  • the subtitle content displayed in real time is greater than the preset subtitle number threshold, the subtitle content is displayed in a line wrap, and the rendering matches the subtitle content according to the height change of the real-time height of the subtitle.
  • the height of the real-time subtitle layer For example, when the width of the subtitles input in real time is greater than the preset maximum line width of the subtitles, that is, when it needs to be displayed in a new line, each time one or more subtitles are newly displayed in real time, they will be displayed according to the unit subtitle height.
  • the display height of the subtitle layer is expanded according to the layer height of the unit subtitle.
  • the height of the unit subtitle is smaller than the layer height of the unit subtitle to ensure that the subtitle content can be visually wrapped by the subtitle layer.
  • the subtitle layer is displayed according to the display content of the subtitles, and a scrolling display effect is visually formed.
  • the layer heights of unit subtitles corresponding to different rows can be the same or different.
  • the first row of unit subtitles can be stipulated to have the same layer height.
  • the layer height of the bit subtitles is 72, the layer height of the second row of unit subtitles is 100, the layer height of the third row of unit subtitles is 120, the layer height of the fourth row of unit subtitles is 132, and the layer height of the fifth row of unit subtitles is 132.
  • the layer height is 156, the layer height of the sixth row of unit subtitles is 180, etc.
  • the preset unit subtitle width can also be reduced, and/or the preset unit subtitle height can be reduced. to display subtitle content.
  • control subtitle layer when the above-mentioned control subtitle layer is dynamically rendered according to the subtitle content, when the content of the last line of subtitles rendered does not reach the preset subtitle quantity threshold, an entire line of images can be rendered according to the preset subtitle quantity threshold as shown in Figure 5 above.
  • layer, as shown in Figure 6, the subtitle layer width of the last row can also be determined based on the number of subtitle content in the last row and the preset unit subtitle width to further improve the intelligence of the subtitle layer following rendering.
  • the subtitle display method of the embodiment of the present disclosure obtains the subtitle content corresponding to the currently played virtual reality video frame, and determines the target space position in the virtual reality panoramic space according to the user's current line of sight direction. At the target space position Render the subtitle layer according to the subtitle content, and render the subtitle content synchronously in the subtitle layer. After determining the subtitle content, follow the user's line of sight direction to determine the position of the subtitle content rendered in the virtual reality space. On the basis of realizing the display of the subtitle content, the user's viewing experience is improved, and the subtitle content is rendered at the corresponding position to
  • the subtitle layer that is independent of the virtual reality video frame serves as a carrier for displaying subtitle content, which avoids display distortion of the subtitle content and improves the display effect of the subtitle content.
  • the target space position is determined in different ways in the virtual reality panoramic space according to the user's current line of sight direction. Examples are as follows:
  • determining the target space position in the virtual reality panoramic space according to the user's current line of sight direction includes:
  • Step 701 Determine the center point position of the virtual reality panoramic space and obtain a preset radius distance.
  • the center point position of the virtual reality panoramic space is determined.
  • the center point position is located at the center of the virtual reality panoramic space.
  • the center point position is related to the shape of the virtual reality panoramic space.
  • a preset radius distance is obtained, where the preset radius distance can be set in advance according to the size of the virtual reality panoramic space, usually the length from the preset radius distance to the center point position, It will not exceed the length from the center point in the virtual reality panoramic space to the surface of the virtual reality panoramic space, thus ensuring that the displayed subtitle layer is located in front of the virtual reality video frame, forming a barrier to enhance the technological sense of subtitle layer display.
  • Step 702 Starting from the center point position, the position extending from the user's current line of sight direction to a preset radius distance is used as the target space position.
  • the target space position since the length from the usually preset radius distance to the center point position will not exceed the length from the center point position in the virtual reality panoramic space to the surface of the virtual reality panoramic space, therefore, starting from the center point position, according to The position where the user's current line of sight extends to the preset radius distance is used as the target space position. On the one hand, it can ensure that the target space position is within the virtual space and ensure the display effect. On the other hand, it can ensure that the target space position is consistent with the user's line of sight direction. The consistency ensures that the subtitle content displayed in the target space position is consistent with the user's line of sight, ensuring that the user can clearly view the subtitle content and improving the viewing experience.
  • the virtual reality panoramic space is a "box-shaped" cubic space
  • the preset radius distance is R1
  • the center point position of the virtual reality panoramic space is O1
  • the user's current line of sight direction is determined.
  • the position extending from the user's current sight direction to the preset radius distance is used as the target space position.
  • determining the target space position in the virtual reality panoramic space according to the user's current line of sight direction includes:
  • Step 901 Obtain the historical spatial position corresponding to the subtitle content of the previous frame displayed in the virtual reality panoramic space.
  • the historical spatial position corresponding to the subtitle content of the previous frame displayed in the virtual reality panoramic space is obtained.
  • the historical spatial position is determined based on the user's line of sight direction under the previous frame of subtitles.
  • Step 902 Obtain the line of sight change information between the user's current line of sight direction and the line of sight direction of the previous frame.
  • the sight change information is used to represent the change in angle between the user's current sight direction and the sight direction of the previous frame.
  • the sight change information may include the user's current sight direction relative to the sight direction of the previous frame. The changing direction and changing angle of sight between directions, etc.
  • the horizontal axis rotation angle of the camera in the virtual reality device worn by the user relative to the previous frame in the horizontal direction is obtained, and the horizontal axis rotation angle is determined to be the horizontal line of sight of the user from viewing the previous frame. direction to the change information of the horizontal line of sight direction of viewing the current frame.
  • the line of sight change information includes the change information of the horizontal line of sight direction, it can be ensured that when the user rotates left and right to watch, the target space position can follow the user's line of sight direction. This further ensures that the subtitle content displayed at the target space position is consistent with the user's line of sight direction.
  • Step 903 Determine the target spatial location based on the line of sight change information and historical spatial location.
  • the target spatial position can be determined based on the line of sight change information and the historical spatial location.
  • the historical spatial position can be moved based on the line of sight change information to determine the target spatial location.
  • the target space position is determined only based on the line of sight change information between the user's current line of sight direction and the line of sight direction of the previous frame, and does not take into account the current user's position in the virtual reality space.
  • the user was in the virtual space in the previous frame Changes in position, regardless of the current user's position in the virtual reality space, whether the user's position in the virtual space is closer to the surface of the virtual space or further away from the surface of the virtual space relative to the previous frame, will not affect the target space position.
  • the target space position is only related to the line of sight change information between the user's current line of sight direction and the line of sight direction of the previous frame. Therefore, visually, the user will not feel the difference between the displayed subtitle content and the user's human eyes. It has a fixed display distance to further ensure the user's visual effect.
  • the virtual reality panoramic space is a "box-shaped" cubic space.
  • the historical space position corresponding to the subtitle content of the previous frame displayed in the virtual reality panoramic space is W1, and the user's current sight direction S2 is obtained.
  • the movement of the historical spatial position is controlled based on the line of sight change information to determine the moved position W2 as the target spatial position.
  • a preset circle center position in the virtual reality panoramic space is obtained, where the circle center position is usually set as the center of the circle at the center of the virtual reality space to ensure that when the circle center is used to surround and follow the subtitles, Subtitle content can be displayed in the virtual reality space.
  • the initial position of the virtual reality device in the virtual reality panoramic space can be obtained, where the initial position is the center point position in the virtual reality panoramic space, and the initial position is used as the midpoint position in the virtual reality panoramic space. Construct a virtual reality panoramic space, and then set the center point position in the virtual reality panoramic space as the center position of the circle.
  • the initial position of the virtual reality device in the virtual reality panoramic space is used as the center position of the circle, in the actual execution process, when the virtual reality device moves in the virtual reality panoramic space, the initial position will continue to be
  • the center point position is set as the center position of the circle, instead of continuing to use the user's VR head-mounted display device as the center position of the circle in the virtual reality panoramic space.
  • Parameters such as the moving distance of the VR head-mounted display device that have nothing to do with the line of sight direction are visually related to the subtitles.
  • the display position of the content is peeled off to improve the visual display effect. As a result, it is avoided that when the subtitle content is always associated with the moving distance of the VR head-mounted display device, the subtitle content is always linked to the VR head-mounted display device, affecting the user's visual experience.
  • the horizontal axis rotation angle is used as the circle center rotation angle from the previous frame to the current frame, and the historical spatial position of the subtitle content of the previous frame relative to the circle center is determined, and then, according to the circle center position, The historical spatial position and the circle center rotation angle are used to determine the target spatial position.
  • the neural network model can be trained in advance based on the sample data, and the center position of the circle, the historical spatial position, and the horizontal axis rotation angle are input into the neural network model, and the current line of sight direction is used in the virtual reality panoramic space.
  • the corresponding target space position can be trained in advance based on the sample data, and the center position of the circle, the historical spatial position, and the horizontal axis rotation angle are input into the neural network model, and the current line of sight direction is used in the virtual reality panoramic space.
  • the corresponding target space position can be trained in advance based on the sample data, and the center position of the circle, the historical spatial position, and the horizontal axis rotation angle are input into the neural network model, and the current line of sight direction is used in the virtual reality panoramic space. The corresponding target space position.
  • a preset algorithm can be used to calculate the target space position corresponding to the current line of sight direction in the virtual reality panoramic space.
  • the preset algorithm can be as shown in the following formula (1), where, In formula (1), x, y, and z are the coordinates of the historical spatial position on the three coordinate axes respectively, x1, y1, and z1 are the coordinates of the target spatial position on the three coordinate axes respectively, and ⁇ is the rotation angle of the center of the circle. Tx, Ty, and Tz are the coordinates of the circle center position on the three coordinate axes respectively.
  • the preset radius distance can be obtained, and the initial spatial position of the subtitle content of the initial frame relative to the center of the circle is determined based on the center position of the circle, the radius distance, and the user's initial line of sight direction. This initial spatial position is used as the rendering position of the first frame of subtitle content.
  • the subtitle display method of the embodiment of the present disclosure can flexibly determine the target space position of the subtitle content to be rendered in the virtual reality panoramic space according to the needs of the scene, ensuring that the target space position follows the user's current line of sight direction, and realizing the subtitle content follow the movement of the user's current line of sight to ensure the visual display effect of the subtitle content.
  • the subtitle layer since the subtitle layer is displayed as a layer independently of the virtual reality video frame, it may cause occlusion to other displayed layers. For example, as shown in Figure 12, if the user is watching the video, After the display operation of the playback panel layer is performed, the above-mentioned determination principle of the target space position will be used in the user's line of sight direction to determine the display position of the playback panel layer and display it. As a result, the playback panel layer may be blocked, making it difficult for the user to perform playback control operations on the playback panel.
  • the display status information of the other layers is identified, and the subtitles are adjusted according to the display status information of the other layers. Display status information of layers to ensure that users can clearly view subtitle content and other layers such as playback panel layers.
  • the display status information includes the displayed spatial position, including:
  • the display status information of the subtitle layer is adjusted according to the display status information of other layers, including :
  • Step 1301 Determine whether the reference space position and the target space position of other layers meet the preset occlusion conditions.
  • the other layers may include any layer that follows the user's direction, such as the playback panel layer, etc.
  • the other layers are relative to The subtitle layer is closer to virtual reality video Frame, determine whether other layers and subtitle layers meet the preset occlusion conditions, that is, determine whether the subtitle layer will block other layers.
  • the first center point position of other layers on the plane perpendicular to the line of sight is identified based on the reference spatial position
  • the second center point of the subtitle layer on the plane perpendicular to the line of sight is identified based on the target spatial position.
  • Position, where the plane perpendicular to the line of sight can be understood as the plane facing the user's line of sight. For example, if a coordinate system is established with the user's line of sight as the Z-axis, the plane perpendicular to the line of sight can be understood as the XY plane.
  • the position difference between the first center point position and the second center point position is less than a preset distance threshold. If it is less than the preset distance threshold, the coincidence degree of other layers and the subtitle layer is determined. If the subtitle layer is too high, the subtitle layer may overlap with other layers and block other layers. Therefore, the preset occlusion conditions must be met.
  • the area of the overlapping area between the subtitle layer and other layers is determined on a plane perpendicular to the line of sight according to the reference space position and the target space position.
  • the overlapping area can be determined based on the size of the overlapping area.
  • Area for another example, the area of the overlapping area can be determined based on the number of pixels in the overlapping area, etc.
  • the area ratio of the area of the overlapping area and the area of other layers can be calculated to determine whether the area ratio is greater than the preset area ratio threshold, where, if is greater than the preset area ratio threshold, it is determined that the preset occlusion conditions are met.
  • the layer level of other layers can also be determined, and the layer level can be determined to be higher than the preset level threshold, wherein the layer level can be determined by identifying the current layer type of other layers, Query the preset corresponding relationship and get etc.
  • Step 1302 if the occlusion condition is met, determine the target movement position and/or the target layer display size of the subtitle layer, where the target movement position and/or the target layer display size correspond to the subtitle layer and other layers do not satisfy occlusion conditions.
  • the target movement position of the subtitle layer is determined, wherein the subtitle layer and other layers at the target movement position do not meet the occlusion condition.
  • the target movement position can be any position as close as possible to the user's line of sight without causing obstruction to other layers.
  • the first height information of the upper edge line of other layers is determined.
  • the first height information can be understood as a Y-axis coordinate value, etc.
  • the lower edge line of the subtitle layer is determined based on the first height information.
  • the second height information wherein the second height information is greater than or equal to the first height information, that is, the second height information can be slightly higher than the first height information or just above the first height information, and further, determined according to the second height information Target movement position.
  • the other layers and the subtitle layer are both rectangular, then when the other layers and the subtitle layer meet the occlusion conditions, continue as shown in Figure 14A to determine the upper and lower edges of the other layers.
  • the first height information of the edge line translates the subtitle layer upward, so that the subtitle layer moves from in front of other layers to above other layers, avoiding occlusion of other layers.
  • first width information of the left edge line of other layers is determined.
  • the first width information can be understood as an X-axis coordinate value, etc.
  • the right edge of the subtitle layer is determined based on the first width information.
  • the second width information of the line, where the second width information is less than or equal to the first width information, that is, the second width information can be slightly smaller than the first width information or it can be on the first width information, and further, determined according to the second width information Target movement position.
  • controlling the subtitle layer to move upward or to the left in the above embodiment is only a possible example.
  • the subtitle layer can also be controlled to move below or to the right relative to other layers.
  • the principle of movement is Similar, not listed here.
  • the display size of the subtitle layer is determined.
  • the reference display size of other layers can be determined based on the reference spatial position, and the target layer display size of the subtitle layer can be determined based on the reference display size.
  • the product of the reference display size and the preset reduction ratio is calculated, where , the preset reduction ratio is less than 1, and the display size of the target layer is determined based on the product value; for another example, the size difference between the reference display size and the initial display size of the subtitle layer is calculated, and the preset correspondence is determined based on the size difference.
  • the scaling ratio The larger the size difference, the smaller the scaling ratio.
  • the initial display size is scaled according to the scaling ratio to obtain the target layer display size.
  • the other layers and the subtitle layer are both rectangular, and the other layers and the subtitle layer meet the occlusion conditions, they can be determined based on the reference display size of the other layers.
  • the target layer display size of the subtitle layer has a smaller blocking area for other layers, thus not affecting the viewing of other layers.
  • the target movement position and the target layer display size of the subtitle layer can also be determined simultaneously, that is, the adjustment of the position and size is combined to avoid occlusion of other layers.
  • Step 1303 Display the subtitle layer according to the target movement position and/or the target layer display size.
  • the subtitle layer is controlled to move to the target moving position and/or the target layer display size.
  • the subtitle content rendered on the subtitle layer also moves synchronously. The user can not only watch the subtitle content but also the subtitle content. to the contents of other layers, etc.
  • the above display status information may also include a display time point, and the display time point of the subtitle layer is adjusted based on the display time point of other layers to ensure that the user can view the display content of other layers.
  • adjusting the display time point of the subtitle layer can control the subtitle layer to be displayed intermittently according to the preset display frequency at the display time point of other layers. For example, if the display time point of other layers is a-b, you can Control other layers to display for d duration every c time at a-b time points, thereby ensuring that users can view the display content of other layers.
  • the method for displaying subtitles in the embodiment of the present disclosure can also change the display state of the subtitle layer after the subtitle layer may block other layers, so as to further ensure that the user can view the content of other layers. Improved display flexibility.
  • the present disclosure also provides a subtitle display device.
  • Figure 16 is a schematic structural diagram of a subtitle display device provided by an embodiment of the present disclosure.
  • the device can be implemented by software and/or hardware, and can generally be integrated in an electronic device to display subtitles.
  • the device includes: an acquisition module 1610, a position determination module 1620, and a rendering and display module 1630, wherein,
  • the acquisition module 1610 is used to acquire the subtitle content corresponding to the currently played virtual reality video frame
  • the position determination module 1620 is used to determine the target space position in the virtual reality panoramic space according to the user's current line of sight direction;
  • the rendering and display module 1630 is used to render the subtitle layer according to the subtitle content at the target spatial position, and to synchronously render the subtitle content in the subtitle layer.
  • the subtitle display device provided by the embodiments of the present disclosure can execute the subtitle display method provided by any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
  • the present disclosure also proposes a computer program product, which includes a computer program/instructions.
  • a computer program product which includes a computer program/instructions.
  • the computer program/instructions are executed by a processor, the subtitle display method in the above embodiments is implemented.
  • Figure 17 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the electronic device 1700 in the embodiment of the present disclosure may include, but is not limited to, mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals ( Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG. 17 is only an example and should not bring any limitations to the functions and scope of use of the embodiments of the present disclosure.
  • electronic device 1700 may include a processor (eg, central processing unit, graphics processor, etc.) 1701 that may be loaded into random access memory according to a program stored in read-only memory (ROM) 1702 or from memory 1708 (RAM) 1703 to perform various appropriate actions and processes.
  • ROM read-only memory
  • RAM memory 1708
  • various programs and data required for the operation of the electronic device 1700 are also stored.
  • the processor 1701, ROM 1702 and RAM 1703 are connected to each other through a bus 1704.
  • An input/output (I/O) interface 1705 is also connected to bus 1704.
  • the following devices may be connected to the I/O interface 1705: input devices 1706 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 1707 such as a computer; a memory 1708 including a magnetic tape, a hard disk, etc.; and a communication device 1709. Communication device 1709 may allow electronic device 1700 to communicate wirelessly or wiredly with other devices to exchange data.
  • FIG. 17 illustrates electronic device 1700 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via communication device 1709, or from memory 1708, or from ROM 1702. In this calculation program When the program is executed by the processor 1701, the above functions defined in the subtitle display method of the embodiment of the present disclosure are performed.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmd read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and server can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium.
  • Communications e.g., communications network
  • Examples of communication networks include local area networks (“LAN”), wide area networks (“WAN”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or developed in the future network of.
  • LAN local area networks
  • WAN wide area networks
  • the Internet e.g., the Internet
  • end-to-end networks e.g., ad hoc end-to-end networks
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device obtains the subtitle content corresponding to the currently played virtual reality video frame, and obtains the subtitle content corresponding to the currently played virtual reality video frame.
  • the target space position is determined in the virtual reality panoramic space
  • the subtitle layer is rendered based on the subtitle content at the target space position
  • the subtitle content is simultaneously rendered in the subtitle layer.
  • follow the user's line of sight direction to determine the position of the subtitle content rendered in the virtual reality space.
  • the subtitle layer that is independent of the virtual reality video frame serves as a carrier for displaying subtitle content, which avoids display distortion of the subtitle content and improves the display effect of the subtitle content.
  • the electronic device may have computer program code for performing operations of the present disclosure written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, or a combination thereof. , also includes conventional procedural programming languages—such as "C" or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).
  • LAN local area network
  • WAN wide area network
  • Internet service provider such as an Internet service provider through Internet connection
  • each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.
  • the units involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of a unit does not constitute a limitation on the unit itself under certain circumstances.
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • SOCs Systems on Chips
  • CPLD Complex Programmable Logical device
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
  • the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM portable compact disk read-only memory
  • magnetic storage device or any suitable combination of the above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Studio Circuits (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本公开实施例涉及一种字幕的显示方法、装置、设备及介质，其中该方法包括：获取与当前播放的虚拟现实视频帧对应的字幕内容；根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。本公开的实施例中，确定字幕内容后，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，在实现字幕内容显示的基础上，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧的字幕图层作为显示字幕内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。

Description

字幕的显示方法、装置、设备及介质
本申请要求申请日为2022年06月20日、名称为“字幕的显示方法、装置、设备及介质”、申请号为202210698095.3的中国发明专利申请的优先权。
技术领域
本公开涉及虚拟现实技术领域，尤其涉及一种字幕的显示方法、装置、设备及介质。
背景技术
虚拟现实技术(Virtual Reality，VR)，囊括计算机、电子信息、仿真技术，其基本实现方式是计算机模拟虚拟环境从而给人以环境沉浸感。随着VR应用的普及，在视频显示等场景下实现字幕在虚拟现实空间中的显示成为一种常见的场景。
相关技术中，对于一些不自带字幕内容的视频源而言，基于网页播放器等工具执行对视频源对应的音频流的翻译功能，获取翻译得到的字幕内容，在对应的视频帧上的固定位置显示翻译得到的字幕内容，然而，在VR显示场景下，为了实现立体显示效果，视频帧的显示方位会出现变化，从而，导致与视频帧相对固定设置的字幕内容可能会在视觉上出现扭曲，影响观看效果。
发明内容
为了解决上述技术问题或者至少部分地解决上述技术问题，本公开提供了一种字幕的显示方法、装置、设备及介质，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧 的字幕图层作为显示字幕内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。
本公开实施例提供了一种字幕的显示方法，所述方法包括：获取与当前播放的虚拟现实视频帧对应的字幕内容；根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；在所述目标空间位置上根据所述字幕内容渲染字幕图层，并在所述字幕图层中同步渲染所述字幕内容。
本公开实施例还提供了一种字幕的显示装置，所述装置包括：获取模块，用于获取与当前播放的虚拟现实视频帧对应的字幕内容；位置确定模块，用于根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；渲染显示模块，用于在所述目标空间位置上根据所述字幕内容渲染字幕图层，并在所述字幕图层中同步渲染所述字幕内容。
本公开实施例还提供了一种电子设备，所述电子设备包括：处理器；用于存储所述处理器可执行指令的存储器；所述处理器，用于从所述存储器中读取所述可执行指令，并执行所述指令以实现如本公开实施例提供的字幕的显示方法。
本公开实施例还提供了一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行如本公开实施例提供的字幕的显示方法。
本公开实施例提供的技术方案与现有技术相比具有如下优点：
本公开实施例提供的字幕的显示方案，获取与当前播放的虚拟现实视频帧对应的字幕内容，根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。确定字幕内容后，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，在实现字幕内容显示的基础上，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧的字幕图层作为显示字幕 内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。
附图说明
结合附图并参考以下具体实施方式，本公开各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素不一定按照比例绘制。
图1为本公开实施例提供的一种字幕的显示方法的流程示意图；
图2为本公开实施例提供的一种字幕的显示场景示意图；
图3为本公开实施例提供的另一种字幕的显示场景示意图；
图4为本公开实施例提供的另一种字幕的显示场景示意图；
图5为本公开实施例提供的另一种字幕的显示场景示意图；
图6为本公开实施例提供的另一种字幕的显示场景示意图；
图7为本公开实施例提供的另一种字幕的显示方法的流程示意图；
图8为本公开实施例提供的另一种字幕的显示场景示意图；
图9为本公开实施例提供的另一种字幕的显示方法的流程示意图；
图10为本公开实施例提供的另一种字幕的显示场景示意图；
图11为本公开实施例提供的另一种字幕的显示场景示意图；
图12为本公开实施例提供的另一种字幕的显示场景示意图；
图13为本公开实施例提供的另一种字幕的显示方法的流程示意图；
图14A为本公开实施例提供的另一种字幕的显示场景示意图；
图14B为本公开实施例提供的另一种字幕的显示场景示意图；
图15为本公开实施例提供的另一种字幕的显示场景示意图；
图16为本公开实施例提供的一种字幕的显示装置的结构示意图；
图17为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
下面将参照附图更详细地描述本公开的实施例。虽然附图中显示 了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
应当理解，本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
为了解决上述问题，本公开实施例提供了一种字幕的显示方法，在该方法中，不再依赖于字幕内容显示的固定模式，而是解除字幕内容和画面内容的位置固定关系，跟随用户的视线方向适应性的调整字幕内容的显示位置，避免字幕内容在视觉上出现扭曲，提升了用户的视觉观看体验。
下面结合具体的实施例对该方法进行介绍。
图1为本公开实施例提供的一种字幕的显示方法的流程示意图，该方法可以由字幕的显示装置执行，其中该装置可以采用软件和/或硬件实现，一般可集成在电子设备中。如图1所示，该方法包括：
步骤101，获取与当前播放的虚拟现实视频帧对应的字幕内容。
在本实施例中，对于一些不自带字幕的虚拟现实视频帧而言，可以基于不同的方式来获取对应的字幕内容。
在一些可能的实施例中，确定与当前播放的虚拟现实视频帧对应的音频流，对音频流进行语音识别处理以获取字幕内容，比如，获取每段音频流对应的时间戳信息，基于时间戳信息和虚拟视频帧的时间戳信息匹配虚拟现实视频帧对应的字幕内容等。
在另一些可能的实施例中，可以预先识别并构建每个虚拟现实视频帧和对应的字幕内容的对应关系，并将该对应关系存储在对应的预设数据库中，从而，可以通过查询预设数据库以获取当前播放的虚拟现实视频帧对应的字幕内容。
步骤102，根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置。
在本实施例中，为了避免字幕内容显示的扭曲，不再以虚拟视频帧作为字幕内容显示位置的确定依据，而是根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置。
即实现目标空间位置与用户的当前视线方向一致，比如，如图2所示，目标空间位置位于用户当前的视线方向上，由此，可以保证在视线方向上的目标空间位置和视线方向的一致，进一步为字幕内容的显示效果提供了技术支撑。
其中，根据用户当前的视线方向在虚拟现实全景空间中确定目标空间位置的方式，可以参照后续实施例，在此不再赘述。
步骤103，在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。
在本公开的一个实施例中，在确定了目标空间位置后，在目标空 间位置渲染字幕图层，并在字幕图层中同步渲染字幕内容，比如，继续参照图2，若是用户的视线方向水平变化，则字幕图层跟随视线方向的变化，在视线方向上的空间位置显示字幕图层，其中，字幕内容同步在字幕图层渲染。由于字幕图层在用户的视线方向上，因此，保证了字幕内容可以无畸变扭曲的呈现给用户，保证了视觉显示效果。
在本实施例中，以独立于虚拟现实视频帧的字幕图层单独渲染字幕内容，避免了字幕内容的显示扭曲，且字幕图层根据字幕内容渲染，视觉上科技感更强，提升了用户的视觉观看体验。
在公开的一个本实施例中，为了进一步提升字幕图层的智能感，还可以对字幕图层进行背景添加显示处理，对字幕图层添加背景特效提升显示的智能感，在一些可能的实施例中，为了避免对虚拟现实视频帧的遮挡，还可以将字幕图层处理为透明度较高的图层，比如，在一些可能的实施例中，字幕图层为背光子图层和量子点阵子图层，其中，量子点阵子图层的量子颜色可以为蓝色等任意颜色，量子点阵子可以渐变形式显示，也可以均匀亮度显示，如图3所示，背光子图层和量子点阵子图层的叠加形成的字幕图层，在视觉上可以给用户更强的科技感。在另一些可能的实施例中，可以在字幕图层中渲染灯光动画，比如，极光动画，比如白光流转动画等，在此不一一赘述，通过灯光动画的添加，提升显示的科技感。
在实际执行过程中，为了进一步提升在视觉上的显示效果，还可以根据字幕内容动态渲染字幕图层。
在本公开的一个实施例中，获取字幕内容的显示数量，渲染与显示数量匹配的字幕图层，由此，实现了伴随字幕显示数量变化动态的显示字幕图层的效果。
在一些可能的实施例中，获取字幕内容的显示数量，根据显示数量、预设的单位字幕宽度和预设的单位字幕高度，确定字幕实时宽度和字幕实时高度。其中，字幕内容的显示数量指的是当前虚拟现实视频帧中显示的所有字幕内容的数量，比如，若是语音识别得到字幕内 容，则可以每识别或者是读取得到一个或多个文字及触发对字幕内容的显示事件，从而，字幕内容的显示数量发生变化。预设的单位字幕宽度指的是预先规定的每个字幕内容的宽度，预设的单位字幕高度指的是预先规定的每个字幕内容的高度。
在本实施例中，确定字幕实施高度和字幕实时宽度，以便于基于字幕实施高度和字幕实时宽度动态渲染字幕图层。
在一些可能的实施例中，可以根据显示数量、预设的单位字幕宽度和预设的单位字幕高度查询预设数据库以获取字幕实时宽度和字幕实时高度。
在另一些可能的实施例中，可以确定显示数量是否大于宽度方向上的预设字幕数量阈值，其中，预设字幕数量阈值可以理解为预先规定的每一行显示的最多的字幕内容数量，若是大于预设字幕数量阈值，则表明字幕内容需要换行处理，因此，根据预设字幕数量阈值和单位字幕宽度确定字幕实时宽度，比如计算预设字幕数量阈值和单位字幕宽度的乘积以确定字幕实时宽度。
同样的，可以根据预设字幕数量阈值和显示数量确定字幕行数，比如，计算显示数量和预设字幕数量阈值的比值，确定大于等于该比值的最小整数作为字幕行数，进而，根据字幕行数和单位字幕高度确定字幕实时高度，比如计算字幕行数和单位字幕高度的乘积以确定字幕实时高度。
在本实施例中，若是显示数量不大于预设字幕数量阈值，即字幕显示数量不足一行，则根据单位字幕宽度和显示数量确定字幕实时宽度，即计算单位字幕宽度和显示数量的乘积以确定字幕实时宽度，并确定单位字幕高度为字幕实时高度。
进一步地，在确定字幕实时宽度和字幕实时高度后，若是字幕实时宽度发生了宽度变化，则响应于字幕实时宽度的宽度变化，根据单位字幕的图层宽度和字幕实时宽度，渲染与字幕内容匹配的实时字幕图层宽度，其中，单位字幕的图层宽度为每个单位字幕在宽度方向上 增加时，字幕图层增加的单位宽度。
同样的，在确定字幕实时宽度和字幕实时高度后，若是字幕实时高度发生了高度变化，则响应于字幕实时高度的高度变化，根据预设的单位字幕的图层高度和字幕实时高度，渲染与字幕内容匹配的实时字幕图层高度。其中，单位字幕的图层高度为每个单位字幕在高度向上增加时，字幕图层增加的单位高度。即字幕图层可以伴随字幕内容进行横向和纵向的扩展变化。
即在本实施例中，如图4所示，当实时显示的字幕内容的宽度小于或者等于预设字幕数量阈值时，根据实时显示的字幕实时宽度确定字幕图层宽度(图中字幕内容为“这里显示的是第*行字幕内容”，字幕图层为渐变亮度)，比如，每次实时新增显示一个或多个字幕内容，则根据单位字幕宽度显示对应的字幕内容，同时根据单位字幕的图层宽度扩展字幕图层。通常单位字幕宽度小于单位字幕的图层宽度，以保证字幕内容在视觉上可以被字幕图层包裹。从而，在视觉上实现了在宽度方向上，根据字幕的显示内容跟随显示字幕背景。
在本公开的一个实施例中，如图5所示，若是实时显示的字幕内容的宽度大于预设字幕数量阈值，对字幕内容进行换行显示，根据字幕实时高度的高度变化，渲染与字幕内容匹配的实时字幕图层高度，比如，当实时输入的字幕宽度大于预先设置的字幕最大行宽时，即需要换行显示时，每次实时新增显示一个或多个字幕内容，则根据单位字幕高度显示对应的字幕内容，同时根据单位字幕的图层高度扩展字幕图层的显示高度，通常单位字幕高度小于单位字幕的图层高度，以保证字幕内容在视觉上可以被字幕图层包裹。
从而，在视觉上实现了在高度方向上，根据字幕的显示内容跟随显示字幕图层，在视觉上形成了滚动显示的效果，继续参照图5所示，伴随着字幕内容从第一行增加到了第六行，字幕图层高度逐渐向下扩展，形成了滚动显示的效果。其中，在实际执行过程中，不同行对应的单位字幕的图层高度可以相同也可不同，比如，可以规定第一行单 位字幕的图层高度为72，第二行单位字幕的图层高度为100，第三行单位字幕的图层高度为120，第四行单位字幕的图层高度为132，第五行单位字幕的图层高度为156，第六行单位字幕的图层高度为180等。
在实际执行过程中，若是显示数量较多，比如大于预设值，则为了避免对虚拟现实视频帧的遮挡，还可以通过缩小预设的单位字幕宽度，和/或，预设的单位字幕高度来显示字幕内容。
又或者，可以判断当前计算得到的字幕行数是否大于预设行数阈值，其中，若是大于该预设行数阈值，则首先显示预设行数阈值的字幕内容，将其他的剩余的字幕内容滚动显示，比如，若是预设行数阈值为6行，但是计算得到的字幕行数为7，则可以先显示前6行字幕内容，在显示预设时长后，不再显示第1行的字幕内容，并将第2-6行的字幕内容向上滚动并在第6行显示原来的第7行的字幕内容，或者也可以在显示预设时长后，将第1-6行的字幕内容不再显示，仅仅显示原来的第7行的字幕内容等。
另外，在上述控制字幕图层根据字幕内容动态渲染时，当渲染的最后一行字幕内容没有达到预设字幕数量阈值时，可以如上述图5所示，根据预设字幕数量阈值渲染一整行图层，也可以如图6所示，根据最后一行的字幕内容数量以及预设的单位字幕宽度确定最后一行的字幕图层宽度，以进一步提升字幕图层跟随渲染的智能化程度。
综上，本公开实施例的字幕的显示方法，获取与当前播放的虚拟现实视频帧对应的字幕内容，根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。确定字幕内容后，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，在实现字幕内容显示的基础上，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧的字幕图层作为显示字幕内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。
在不同的应用场景中，根据用户当前的视线方向在虚拟现实全景空间中确定目标空间位置的方式不同，示例如下：
在本公开的一个实施例中，如图7所示，根据用户当前的视线方向在虚拟现实全景空间中确定目标空间位置，包括：
步骤701，确定虚拟现实全景空间的中心点位置，以及获取预设的半径距离。
在本实施例中，确定虚拟现实全景空间的中心点位置，该中心点位置位于虚拟现实全景空间的中心处，中心点位置和虚拟现实全景空间的形状有关。在确定虚拟现实全景空间的中心点位置后，获取预设的半径距离，其中，预设半径距离可以是预先根据虚拟现实全景空间的尺寸设置，通常预设的半径距离到中心点位置的长度，不会超过虚拟现实全景空间中中心点位置到虚拟现实全景空间的表面的长度，从而，保证显示的字幕图层位于虚拟现实视频帧前面，形成一道屏障，提升字幕图层显示的科技感。
步骤702，从中心点位置开始，按照用户当前的视线方向延伸到预设的半径距离的位置作为目标空间位置。
在本实施例中，由于通常预设的半径距离到中心点位置的长度，不会超过虚拟现实全景空间中中心点位置到虚拟现实全景空间的表面的长度，因此，从中心点位置开始，按照用户当前的视线方向延伸到预设的半径距离的位置作为目标空间位置，一方面，可以保证目标空间位置在虚拟空间内部，保证了显示效果，另一方面，可以保证目标空间位置与用户视线方向的一致性，使得显示在目标空间位置的字幕内容与用户的视线方向一致，保证了用户可以清晰的观看到字幕内容，提升了观看体验。
举例而言，如图8所示，虚拟现实全景空间为“盒状”立方体空间，预设的半径距离为R1，虚拟现实全景空间的中心点位置为O1，则在确定了用户当前的视线方向后，直接按照用户当前的视线方向延伸到预设的半径距离的位置作为目标空间位置。
在本公开的一个实施例中，如图9所示，根据用户当前的视线方向在虚拟现实全景空间中确定目标空间位置，包括：
步骤901，获取在虚拟现实全景空间中显示上一帧的字幕内容对应的历史空间位置。
在本实施例中，获取在虚拟现实全景空间中显示上一帧的字幕内容对应的历史空间位置，该历史空间位置是根据上一帧字幕下用户的视线方向确定的。
步骤902，获取用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息。
其中，视线变化信息用于表示用户当前的视线方向相对于观看上一帧视线方向之间的视线的角度的变化等，比如，视线变化信息可以包括用户当前的视线方向相对于观看上一帧视线方向之间的视线的变化方向以及变化角度等。
在本公开的一个实施例中，获取用户穿戴的虚拟现实设备中的相机在水平方向上相对于上一帧的水平轴旋转角度，确定该水平轴旋转角度为用户从观看上一帧的水平视线方向到观看当前帧的水平视线方向的变化信息，在本实施例中，由于视线变化信息包括水平视线方向的变化信息，可以保证用户在左右旋转观看时，目标空间位置可以跟随用户的视线方向，进一步保证了显示在目标空间位置的字幕内容与用户的视线方向一致性。
步骤903，根据视线变化信息以及历史空间位置确定目标空间位置。
在本实施例中，在确定了视线变化信息后，可以根据视线变化信息以及历史空间位置确定目标空间位置，比如，可以根据视线变化信息移动历史空间位置以确定目标空间位置。
需要强调的是，在本实施例中，仅仅根据用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息确定目标空间位置，并不考虑当前用户在虚拟现实空间中的位置以及上一帧用户在虚拟空间 中位置的变化，无论当前用户在虚拟现实空间中的位置，相对于上一帧用户在虚拟空间中位置是进一步接近虚拟空间的表面，还是进一步远离了虚拟空间的表面，都不影响目标空间位置的确定，目标空间位置仅仅与用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息有关，因此，在视觉上使得用户不会感觉到显示的字幕内容与用户人眼之间具有固定显示距离，进一步保证了用户的视觉效果。
举例而言，如图10所示，虚拟现实全景空间为“盒状”立方体空间，在虚拟现实全景空间中显示上一帧的字幕内容对应的历史空间位置为W1，获取用户当前的视线方向S2相对于观看上一帧视线方向S1之间的视线变化信息如图中所示，则根据视线变化信息控制历史空间位置的移动，以确定移动后的位置W2为目标空间位置。
需要说明的是，在不同的应用场景中，根据视线变化信息以及历史空间位置确定目标空间位置的方式不同，示例如下：
在本公开的一个实施例中，获取在虚拟现实全景空间中预先设置的圆心位置，其中，该圆心位置作为圆心通常设置在虚拟现实空间的中心位置，以保证在作为圆心使字幕环绕跟随时，字幕内容可以显示在虚拟现实空间中。
在本实施例中，可获取虚拟现实设备在虚拟现实全景空间中的初始位置，其中，初始位置为虚拟现实全景空间中的中心点位置，以初始位置作为在虚拟现实全景空间中的中点位置进行虚拟现实全景空间的搭建，进而，将虚拟现实全景空间中的中心点位置设置为作为圆心位置。
其中，在本公开的一个实施例中，虽然将虚拟现实设备在虚拟现实全景空间中的初始位置作为圆心位置，但是在实际执行过程中，当虚拟现实设备在虚拟现实全景空间移动时，继续将该中心点位置设置为圆心位置，而不继续以用户的VR头戴显示设备作为虚拟现实全景空间中的圆心位置，视觉上将VR头戴显示设备的移动距离等和视线方向无关的参数与字幕内容的显示的位置剥离开，提升了视觉显示效 果，避免字幕内容始终与VR头戴显示设备的移动距离等关联时，形成字幕内容始终与VR头戴显示设备联动，影响用户的视觉体验。
进一步地，在获取了圆心位置后，将水平轴旋转角度作为从上一帧到当前帧的圆心旋转角度，并且确定上一帧的字幕内容相对于圆心的历史空间位置，进而，根据圆心位置、历史空间位置、以及圆心旋转角度，确定目标空间位置。
举例而言，如图11所示，获取用户穿戴的虚拟现实设备中的相机在水平方向上相对于上一帧的水平轴旋转角度从当前播放的视频帧相对于上一帧的水平轴旋转角度后，获取在虚拟现实全景空间中预先设置的圆心位置，将水平轴旋转角度设置为旋转根节点的圆心旋转角度，进而，确定上一帧的字幕内容相对于圆心的历史空间位置，根据该圆心旋转角度旋转，带动字幕内容以圆心位置为旋转中心，将字幕内容从历史空间位置旋转到与用户视线方向一致的目标空间位置，保证了后续显示在目标空间位置的字幕内容的显示效果。
其中，在一些可能的实施例中，可以预先根据样本数据训练神经网络模型，将圆心位置、历史空间位置、以水平轴旋转角度输入该神经网络模型，以当前的视线方向在虚拟现实全景空间中对应的目标空间位置。
在另一些可能的实施例中，可以采用预设算法计算得到当前的视线方向在虚拟现实全景空间中对应的目标空间位置，其中，预设算法可以如下述公式(1)所示，其中，在公式(1)中，x，y，z分别为历史空间位置在三个坐标轴上的坐标，x1，y1，z1分别为目标空间位置在三个坐标轴上的坐标，θ为圆心旋转角度，Tx，Ty，Tz分别为圆心位置在三个坐标轴上的坐标。
其中，若是当前为首次显示的第一帧字幕内容，可以获取预设的半径距离，根据圆心位置、半径距离，以及用户的初始视线方向确定初始帧的字幕内容相对于圆心的初始空间位置，将该初始空间位置作为第一帧字幕内容的渲染位置。
综上，本公开实施例的字幕的显示方法，可根据场景需要灵活的确定字幕内容在虚拟现实全景空间中渲染的目标空间位置，保证了目标空间位置跟随用户当前的视线方向，实现了字幕内容跟随用户当前的视线方向的移动，保证了字幕内容的视觉显示效果。
基于上述实施例，由于字幕图层独立于虚拟现实视频帧作为一个图层单独显示，因此，有可能对其他显示的图层造成遮挡，比如，如图12所示，若是在观看视频时，用户执行了播放面板图层的显示操作，则会在用户的视线方向上采用上述目标空间位置的确定原理，确定出播放面板图层的显示位置并进行显示。从而，可能对播放面板图层造成了遮挡，用户难以对播放面板执行播控操作。
因此，为了避免上述问题，在本公开的一个实施例中，若是监测到在虚拟现实全景空间中显示其他图层，则识别其他图层的显示状态信息，根据其他图层的显示状态信息调整字幕图层的显示状态信息，以保证用户对字幕内容以及播放面板图层等其他图层可以清晰的观看到。
在本公开的一个实施例中，显示状态信息包括显示的空间位置，包括：在本实施例中，如图13所示，根据其他图层的显示状态信息调整字幕图层的显示状态信息，包括：
步骤1301，确定其他图层所在的参考空间位置与目标空间位置是否满足预设的遮挡条件。
在本实施例中，若是监测到在虚拟现实全景空间中显示其他图层，其他图层可以包括任意跟随用户实现方向显示的图层，比如，播控面板图层等，通常其他图层相对于字幕图层来说更为贴近虚拟现实视频 帧，确定其他图层与字幕图层是否满足预设的遮挡条件，即判断字幕图层是否会遮挡其他图层。
需要说明的是，在不同的应用场景下，确定其他图层所在的参考空间位置与目标空间位置是否满足预设的遮挡条件的方式不同，示例如下：
在一些可能的实施例中，根据参考空间位置识别其他图层在垂直视线方向的平面上的第一中心点位置，根据目标空间位置识别字幕图层在垂直视线方向的平面上的第二中心点位置，其中，垂直视线方向的平面可以理解为用户视线上正对观看的平面，比如，若是以用户视线方向为Z轴建立坐标系，则垂直视线方向的平面可以理解为XY平面。
在本实施例中，确定第一中心点位置与第二中心点位置的位置差值是否小于预设距离阈值，其中，若是小于预设距离阈值，则确定其他图层和字幕图层的重合度较高，字幕图层可能会和其他图层重合，对其他图层造成遮挡，因此，确定满足预设的遮挡条件。
在另一些可能的实施例中，根据参考空间位置和目标空间位置，在垂直视线方向的平面上，确定字幕图层与其他图层的重叠区域面积，比如，可以根据重叠区域的尺寸确定重叠区域面积，又比如，可以根据重叠区域的像素的个数确定重叠区域面积等，计算重叠区域面积与其他图层的图层面积的面积比值，确定面积比值是否大于预设面积比值阈值，其中，若是大于预设面积比值阈值，则确定满足预设的遮挡条件。
当然，在本公开的一个实施例中，并非是对所有的其他图层都判断是否遮挡，对于一些可遮挡的图层比如广告图层等，并不需要判断字幕图层是否对其造成遮挡，因此，在本实施例中，还可以确定其他图层的图层等级，并确定图层等级高于预设等级阈值，其中，图层等级可以通过识别当前的其他图层的图层类型后，查询预设对应关系得到等。
步骤1302，若是满足遮挡条件，则确定字幕图层的目标移动位置和/或目标图层显示尺寸，其中，目标移动位置和/或目标图层显示尺寸对应的字幕图层与其他图层不满足遮挡条件。
在本公开的一个实施例中，若是满足预设遮挡条件，则确定字幕图层的目标移动位置，其中，目标移动位置的字幕图层与其他图层不满足遮挡条件。
在实际执行过程中，目标移动位置可以为尽量靠近用户视线方向但是不会对其他图层造成遮挡的任意位置。
在本公开的一个实施例中，确定其他图层的上边缘线的第一高度信息，该第一高度信息可以理解为Y轴坐标值等，根据第一高度信息确定字幕图层的下边缘线的第二高度信息，其中，第二高度信息大于等于第一高度信息，即第二高度信息可以稍微高于第一高度信息也可以就在第一高度信息上，进而，根据第二高度信息确定目标移动位置。
举例而言，如图14A所示，若是其他图层和字幕图层都为矩形，则其他图层与字幕图层满足遮挡条件时，则继续如图14A所示，可以确定其他图层的上边缘线的第一高度信息，将字幕图层向上平移，使得字幕图层从位于其他图层的前方移动到其他图层的上方，避免了对其他图层的遮挡。
在本公开的另一个实施例中，确定其他图层的左边缘线的第一宽度信息，该第一宽度信息可以理解为X轴坐标值等，根据第一宽度信息确定字幕图层的右边缘线的第二宽度信息，其中，第二宽度信息小于等于第一宽度信息，即第二宽度信息可以稍微小于第一宽度信息也可以就在第一宽度信息上，进而，根据第二宽度信息确定目标移动位置。
举例而言，如图14B所示，若是其他图层和字幕图层都为矩形，则其他图层与字幕图层满足遮挡条件时，则继续如图14B所示，可以确定其他图层的左边缘线的第一宽度信息，将字幕图层向左平移，使得字幕图层从位于其他图层的前方移动到其他图层的左方，避免了对 其他图层的遮挡。
当然，上述实施例中控制字幕图层向上移动或者向左移动仅仅是一种可能的示例，在实际执行时，还可以控制字幕图层相对于其他图层的下方或者右方移动，其移动原理类似，在此不一一列举。
在本公开的另一个实施例中，若是满足遮挡条件，则确定字幕图层的显示尺寸。
在本实施例中，可根据参考空间位置确定其他图层的参考显示尺寸，根据参考显示尺寸确定字幕图层的目标图层显示尺寸，比如，计算参考显示尺寸和预设缩小比例的乘积，其中，预设缩小比例小于1，根据乘积值确定目标图层显示尺寸；又比如，发计算参考显示尺寸和字幕图层的初始显示尺寸的尺寸差值，根据该尺寸差值查询预设对应关系确定缩放比例，其中，尺寸差值越大，则缩放比例越小，根据该缩放比例缩放初始显示尺寸以得到目标图层显示尺寸。
在本公开的实施例中，如图15所示，若是其他图层和字幕图层都为矩形，则其他图层与字幕图层满足遮挡条件时，则可以根据其他图层的参考显示尺寸确定字幕图层的目标图层显示尺寸，目标图层显示尺寸的字幕图层对其他图层的遮挡区域较小，从而，不影响对其他图层的观看。
在本公开的另一个实施例中，也可以同时确定字幕图层的目标移动位置和目标图层显示尺寸，即结合位置和尺寸的调整来避免对其他图层的遮挡。
步骤1303，根据目标移动位置和/或目标图层显示尺寸显示字幕图层。
在本实施例中，控制字幕图层移动到目标移动位置和/或目标图层显示尺寸显示，其中，渲染在字幕图层上的字幕内容也同步移动，用户不但可以观看到字幕内容也可以观看到其他图层的内容等。
考虑到其他图层可能都是暂时显示的图层，因此，为了进一步提升视觉显示效果，在本公开的一个实施例中，若是监测到其他图层的 关闭显示指令，则控制字幕图层移动到目标空间位置上显示。
在本公开的一个实施例中，上述显示状态信息也可以包括显示时间点，通过其他图层的显示时间点，调整字幕图层的显示时间点，保证用户可以观看到其他图层的显示内容。其中，调整字幕图层的显示时间点可以控制字幕图层在其他图层的显示时间点下，根据预设显示频次断续显示等，比如，若是其他图层的显示时间点为a-b，则可以控制其他图层在a-b时间点下每隔c时间显示d时长，从而，保证用户可以观看到其他图层的显示内容。
综上，本公开实施例的字幕的显示方法，在字幕图层可能会对其他图层造成遮挡后，还可以改变字幕图层的显示在状态，以进一步保证用户对其他图层的内容观看，提升了显示的灵活性。
为了实现上述实施例，本公开还提出了一种字幕的显示装置。
图16为本公开实施例提供的一种字幕的显示装置的结构示意图，该装置可由软件和/或硬件实现，一般可集成在电子设备中进行字幕的显示。如图16所示，该装置包括：获取模块1610、位置确定模块1620、渲染显示模块1630，其中，
获取模块1610，用于获取与当前播放的虚拟现实视频帧对应的字幕内容；
位置确定模块1620，用于根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；
渲染显示模块1630，用于在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。
本公开实施例所提供的字幕的显示装置可执行本公开任意实施例所提供的字幕的显示方法，具备执行方法相应的功能模块和有益效果。
为了实现上述实施例，本公开还提出一种计算机程序产品，包括计算机程序/指令，该计算机程序/指令被处理器执行时实现上述实施例中的字幕的显示方法。
图17为本公开实施例提供的一种电子设备的结构示意图。
下面具体参考图17，其示出了适于用来实现本公开实施例中的电子设备1700的结构示意图。本公开实施例中的电子设备1700可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图17示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图17所示，电子设备1700可以包括处理器(例如中央处理器、图形处理器等)1701，其可以根据存储在只读存储器(ROM)1702中的程序或者从存储器1708加载到随机访问存储器(RAM)1703中的程序而执行各种适当的动作和处理。在RAM 1703中，还存储有电子设备1700操作所需的各种程序和数据。处理器1701、ROM 1702以及RAM 1703通过总线1704彼此相连。输入/输出(I/O)接口1705也连接至总线1704。
通常，以下装置可以连接至I/O接口1705：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置1706；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置1707；包括例如磁带、硬盘等的存储器1708；以及通信装置1709。通信装置1709可以允许电子设备1700与其他设备进行无线或有线通信以交换数据。虽然图17示出了具有各种装置的电子设备1700，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置1709从网络上被下载和安装，或者从存储器1708被安装，或者从ROM 1702被安装。在该计算机程 序被处理器1701执行时，执行本公开实施例的字幕的显示方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(HyperText Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：获取与当前播放的虚拟现实视频帧对应的字幕内容，根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，在目标空间位置上根据字幕内容渲染字幕图层，并在字幕图层中同步渲染字幕内容。确定字幕内容后，跟随用户的视线方向，确定在虚拟现实空间中渲染的字幕内容的位置，在实现字幕内容显示的基础上，提升了用户的观看体验，并在对应的位置渲染字幕内容，以独立于虚拟现实视频帧的字幕图层作为显示字幕内容的载体，避免了字幕内容显示扭曲，提升了字幕内容的显示效果。
电子设备可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。 例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于) 具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (21)

  1. 一种字幕的显示方法，其特征在于，包括以下步骤：
    获取与当前播放的虚拟现实视频帧对应的字幕内容；
    根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；
    在所述目标空间位置上根据所述字幕内容渲染字幕图层，并在所述字幕图层中同步渲染所述字幕内容。
  2. 如权利要求1所述的方法，其特征在于，所述获取与当前播放的虚拟现实视频帧对应的字幕内容，包括：
    对所述当前播放的虚拟现实视频帧对应的音频流进行语音识别处理以获取所述字幕内容；或者，
    查询预设数据库以获取与所述当前播放的虚拟现实视频帧对应的字幕内容。
  3. 如权利要求1所述的方法，其特征在于，所述根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，包括：
    确定所述虚拟现实全景空间的中心点位置，以及获取预设的半径距离；
    从所述中心点位置开始，按照所述用户当前的视线方向延伸到所述预设的半径距离的位置作为所述目标空间位置。
  4. 如权利要求1所述的方法，其特征在于，所述根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置，包括：
    获取在所述虚拟现实全景空间中显示上一帧的字幕内容对应的历史空间位置；
    获取所述用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息；
    根据所述视线变化信息以及所述历史空间位置确定所述目标空间位置。
  5. 如权利要求4所述的方法，其特征在于，所述获取所述用户当前的视线方向相对于观看上一帧视线方向之间的视线变化信息，包括：
    获取用户穿戴的虚拟现实设备中的相机在水平方向上相对于上一帧的水平轴旋转角度，其中，
    所述水平轴旋转角度为所述用户从观看上一帧的水平视线方向到观看当前帧的水平视线方向的变化信息。
  6. 如权利要求1所述的方法，其特征在于，所述在所述目标空间位置上根据所述字幕内容渲染字幕图层，包括：
    获取字幕内容的显示数量；
    渲染与所述显示数量匹配的字幕图层。
  7. 如权利要求6所述的方法，其特征在于，所述渲染与所述显示数量匹配的字幕图层，包括：根据所述显示数量、预设的单位字幕宽度和预设的单位字幕高度，确定字幕实时宽度和字幕实时高度；
    响应于所述字幕实时宽度的宽度变化，根据单位字幕的图层宽度和所述字幕实时宽度，渲染与所述字幕内容匹配的实时字幕图层宽度；和/或，
    响应于所述字幕实时高度的高度变化，根据预设的单位字幕的图层高度和所述字幕实时高度，渲染与所述字幕内容匹配的实时字幕图层高度。
  8. 如权利要求7所述的方法，其特征在于，所述根据所述显示数量、预设的单位字幕宽度和预设的单位字幕高度，确定字幕实时宽度和字幕实时高度，包括：
    确定所述显示数量是否大于宽度方向上的预设字幕数量阈值；
    若是大于所述预设字幕数量阈值，则根据所述预设字幕数量阈值和所述单位字幕宽度确定所述字幕实时宽度；
    根据所述预设字幕数量阈值和所述显示数量确定字幕行数，并根据所述字幕行数和所述单位字幕高度确定所述字幕实时高度。
  9. 如权利要求8所述的方法，其特征在于，还包括：
    若是不大于所述预设字幕数量阈值，则根据所述单位字幕宽度和所述显示数量确定所述字幕实时宽度；
    确定所述单位字幕高度为所述字幕实时高度。
  10. 如权利要求1-9任一所述的方法，其特征在于，还包括：
    若是监测到在所述虚拟现实全景空间中显示其他图层，则识别所述其他图层的显示状态信息；
    根据所述其他图层的显示状态信息调整所述字幕图层的显示状态信息。
  11. 如权利要求10所述的方法，其特征在于，所述显示状态信息包括显示的空间位置，所述根据所述其他图层的显示状态信息调整所述字幕图层的显示状态信息，包括：
    确定所述其他图层所在的参考空间位置与所述目标空间位置是否满足预设的遮挡条件；
    若是满足所述遮挡条件，则确定所述字幕图层的目标移动位置和/或目标图层显示尺寸，其中，所述目标移动位置和/或目标图层显示尺寸对应的所述字幕图层与所述其他图层不满足所述遮挡条件；
    根据所述目标移动位置和/或目标图层显示尺寸显示所述字幕图层。
  12. 如权利要求10所述的方法，其特征在于，在所述根据所述其他图层的显示状态信息调整所述字幕图层的显示状态信息之前，还包括：
    确定所述其他图层的图层等级，并确定所述图层等级高于预设等级阈值。
  13. 如权利要求11所述的方法，其特征在于，所述确定所述其他图层所在的参考空间位置与所述目标空间位置是否满足预设的遮挡条件，包括：
    根据所述参考空间位置，识别所述其他图层在垂直所述视线方向的平面上的第一中心点位置；
    根据所述目标空间位置，识别所述字幕图层在垂直所述视线方向的平面上的第二中心点位置；
    确定所述第一中心点位置与所述第二中心点位置的位置差值是否小于预设距离阈值，其中，若是小于所述预设距离阈值，则确定满足所述预设的遮挡条件。
  14. 如权利要求11所述的方法，其特征在于，所述确定所述其他图层所在的参考空间位置与所述目标空间位置是否满足预设的遮挡条件，包括：
    根据所述参考空间位置和所述目标空间位置，确定所述字幕图层与所述其他图层在垂直所述视线方向的平面上的重叠区域面积；
    计算所述重叠区域面积与所述其他图层的图层面积的面积比值；
    确定所述面积比值是否大于预设面积比值阈值，其中，若是大于所述预设面积比值阈值，则确定满足所述预设的遮挡条件。
  15. 如权利要求11所述的方法，其特征在于，所述确定所述字幕图层的目标移动位置和/或目标图层显示尺寸，包括：
    根据所述参考空间位置，确定所述其他图层的上边缘线的第一高度信息，
    根据所述第一高度信息确定所述字幕图层的下边缘线的第二高度信息，其中，所述第二高度信息大于等于所述第一高度信息，
    根据所述第二高度信息确定所述目标移动位置；和/或，
    根据所述参考空间位置，确定所述其他图层的参考显示尺寸，
    根据所述参考显示尺寸确定所述字幕图层的目标图层显示尺寸。
  16. 如权利要求11所述的方法，其特征在于，还包括：
    若是监测到所述其他图层的关闭显示指令，则控制所述字幕图层移动到所述目标空间位置上显示。
  17. 如权利要求1-9任一所述的方法，其特征在于，还包括：
    对所述字幕图层进行背景添加显示处理。
  18. 如权利要求17所述的方法，其特征在于，所述对所述字幕图 层进行背景添加显示处理，包括：
    在所述字幕图层下添加背光子图层和量子点阵子图层；和/或，
    在所述字幕图层上渲染灯光动画。
  19. 一种字幕的显示装置，其特征在于，包括：
    获取模块，用于获取与当前播放的虚拟现实视频帧对应的字幕内容；
    位置确定模块，用于根据用户当前的视线方向，在虚拟现实全景空间中确定目标空间位置；
    渲染显示模块，用于在所述目标空间位置上根据所述字幕内容渲染字幕图层，并在所述字幕图层中同步渲染所述字幕内容。
  20. 一种电子设备，其特征在于，所述电子设备包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    所述处理器，用于从所述存储器中读取所述可执行指令，并执行所述可执行指令以实现上述权利要求1-18中任一所述的字幕的显示方法。
  21. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序用于执行上述权利要求1-18中任一所述的字幕的显示方法。
PCT/CN2023/091203 2022-06-20 2023-04-27 字幕的显示方法、装置、设备及介质 WO2023246302A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210698095.3A CN117319725A (zh) 2022-06-20 2022-06-20 字幕的显示方法、装置、设备及介质
CN202210698095.3 2022-06-20

Publications (1)

Publication Number Publication Date
WO2023246302A1 true WO2023246302A1 (zh) 2023-12-28

Family

ID=89259027

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/091203 WO2023246302A1 (zh) 2022-06-20 2023-04-27 字幕的显示方法、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN117319725A (zh)
WO (1) WO2023246302A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117676053A (zh) * 2024-01-31 2024-03-08 成都华栖云科技有限公司 一种动态字幕渲染方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104918097A (zh) * 2015-06-01 2015-09-16 无锡天脉聚源传媒科技有限公司 一种字幕生成方法及装置
CN106412718A (zh) * 2016-09-18 2017-02-15 乐视控股（北京）有限公司 一种字幕在3d空间中的渲染方法及其装置
CN107613405A (zh) * 2017-09-30 2018-01-19 福州瑞芯微电子股份有限公司 一种vr视频字幕显示方法和装置
EP3493028A1 (en) * 2017-12-04 2019-06-05 Thomson Licensing Devices, systems and methods for enhancing content viewer experience using augmented reality
CN112019772A (zh) * 2020-09-07 2020-12-01 中图云创智能科技(北京)有限公司 一种全景视频中纠正字幕桶状变形的方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104918097A (zh) * 2015-06-01 2015-09-16 无锡天脉聚源传媒科技有限公司 一种字幕生成方法及装置
CN106412718A (zh) * 2016-09-18 2017-02-15 乐视控股（北京）有限公司 一种字幕在3d空间中的渲染方法及其装置
CN107613405A (zh) * 2017-09-30 2018-01-19 福州瑞芯微电子股份有限公司 一种vr视频字幕显示方法和装置
EP3493028A1 (en) * 2017-12-04 2019-06-05 Thomson Licensing Devices, systems and methods for enhancing content viewer experience using augmented reality
CN112019772A (zh) * 2020-09-07 2020-12-01 中图云创智能科技(北京)有限公司 一种全景视频中纠正字幕桶状变形的方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117676053A (zh) * 2024-01-31 2024-03-08 成都华栖云科技有限公司 一种动态字幕渲染方法及系统
CN117676053B (zh) * 2024-01-31 2024-04-16 成都华栖云科技有限公司 一种动态字幕渲染方法及系统

Also Published As

Publication number Publication date
CN117319725A (zh) 2023-12-29

Similar Documents

Publication Publication Date Title
US11838518B2 (en) Reprojecting holographic video to enhance streaming bandwidth/quality
EP3368965B1 (en) Remote rendering for virtual images
WO2021139408A1 (zh) 显示特效的方法、装置、存储介质及电子设备
JP2024505995A (ja) 特殊効果展示方法、装置、機器および媒体
WO2021204296A1 (zh) 三维模型远程展示方法、第一终端、电子设备及存储介质
WO2023221409A1 (zh) 虚拟现实空间的字幕渲染方法、装置、设备及介质
WO2020140758A1 (zh) 图像显示方法、图像处理方法和相关设备
WO2021139382A1 (zh) 人脸图像的处理方法、装置、可读介质和电子设备
WO2021254502A1 (zh) 目标对象显示方法、装置及电子设备
US20230209031A1 (en) Methods, systems, and media for generating and rendering immersive video content
WO2023246302A1 (zh) 字幕的显示方法、装置、设备及介质
WO2023138559A1 (zh) 虚拟现实交互方法、装置、设备和存储介质
WO2023151524A1 (zh) 图像显示方法、装置、电子设备及存储介质
WO2023169305A1 (zh) 特效视频生成方法、装置、电子设备及存储介质
CN113589926B (zh) 虚拟界面操作方法、头戴式显示设备和计算机可读介质
WO2021244651A1 (zh) 信息显示方法、装置、终端及存储介质
CN113703704A (zh) 界面显示方法、头戴式显示设备和计算机可读介质
JP2023537721A (ja) 顔画像表示方法、装置、電子機器及び記憶媒体
CN109727315B (zh) 一对多集群渲染方法、装置、设备及存储介质
US20230405475A1 (en) Shooting method, apparatus, device and medium based on virtual reality space
WO2023231662A1 (zh) 信息的显示方法、装置、终端和存储介质
WO2023025181A1 (zh) 图像识别方法、装置和电子设备
WO2023134537A1 (zh) 分屏特效道具生成方法、装置、设备和介质
WO2023231666A1 (zh) 信息交互方法、装置、电子设备和存储介质
US20230262406A1 (en) Visual content presentation with viewer position-based audio

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23825955

Country of ref document: EP

Kind code of ref document: A1