CN106454547B - A kind of real-time subtitle broadcasting method and system - Google Patents
A kind of real-time subtitle broadcasting method and system Download PDFInfo
- Publication number
- CN106454547B CN106454547B CN201510491214.8A CN201510491214A CN106454547B CN 106454547 B CN106454547 B CN 106454547B CN 201510491214 A CN201510491214 A CN 201510491214A CN 106454547 B CN106454547 B CN 106454547B
- Authority
- CN
- China
- Prior art keywords
- subtitle
- media stream
- segment
- stream
- media
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012545 processing Methods 0.000 claims description 12
- 239000012634 fragment Substances 0.000 claims 3
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 208000032041 Hearing impaired Diseases 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/488—Data services, e.g. news ticker
- H04N21/4884—Data services, e.g. news ticker for displaying subtitles
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Description
技术领域technical field
本发明涉及多媒体通信领域,尤其涉及一种实时字幕播出方法及系统。The invention relates to the field of multimedia communication, in particular to a real-time subtitle broadcasting method and system.
背景技术Background technique
近年来,随着我国宽带网络建设和媒体技术的飞速发展,流媒体业务已逐渐成为互联网中最具代表性的应用。基于分段思想的媒体分发方法具有快速启动、自适应码率切换、用户体验好等优点,得到广泛应用。In recent years, with the rapid development of my country's broadband network construction and media technology, streaming media services have gradually become the most representative applications in the Internet. The media distribution method based on segmentation idea has the advantages of fast startup, adaptive bit rate switching, and good user experience, and is widely used.
目前,流媒体技术已经是比较成熟的媒体技术,广泛应用于电子商务、新闻发布、视频直播、视频点播、视频会议和即时通信等互联网信息服务上。为了给用户带来更丰富的体验,除了向用户提供优质的视频服务外,一般还提供字幕服务。对于点播媒体,工作人员有足够的时间对视频做后期处理,加上字幕;对于字幕显示位置固定或变化不大的直播,比如体彩号、比赛比分等,可以基于字幕模板实时更新固定位置的字幕信息;但对于更为广泛的直播,字幕无法实时生成。对于直播现场比较混乱的情况或者有听力障碍的观众,没有字幕的直播,将导致观众无法获取到准确信息,用户体验满意度将大大降低。At present, streaming media technology is a relatively mature media technology, which is widely used in Internet information services such as e-commerce, news release, live video, video on demand, video conferencing and instant messaging. In order to bring a richer experience to users, in addition to providing users with high-quality video services, subtitle services are also generally provided. For on-demand media, the staff has enough time to post-process the video and add subtitles; for live broadcasts where the subtitle display position is fixed or does not change much, such as sports lottery numbers, game scores, etc., the subtitles at the fixed position can be updated in real time based on the subtitle template information; however, for more extensive live broadcasts, subtitles cannot be generated in real-time. For audiences with chaotic live broadcasts or hearing-impaired audiences, live broadcasts without subtitles will result in the inability of audiences to obtain accurate information, and user experience satisfaction will be greatly reduced.
发明内容SUMMARY OF THE INVENTION
本发明的目的是为了解决现有技术存在的上述不足,提供一种通过对直播过程中的流媒体片段进行音频信息提取,从而生成字幕文件,使终端同步播放字幕内容和流媒体片段的方法。The purpose of the present invention is to solve the above-mentioned deficiencies in the prior art, and provide a method for generating subtitle files by extracting audio information from the streaming media segments in the live broadcast process, so that the terminal can synchronously play the subtitle content and the streaming media segments.
为实现上述目的,第一方面,本发明提供了一种实时字幕播出方法,该方法包括以下步骤:In order to achieve the above object, in the first aspect, the present invention provides a real-time subtitle broadcasting method, the method includes the following steps:
将媒体流分段生成多个媒体流片段;segment the media stream to generate multiple media stream segments;
提取多个媒体流片段中的每个媒体流片段的音频信息;extracting audio information for each of the multiple media stream segments;
根据音频信息生成字幕流,其中,字幕流包括,字幕文件和字幕文件对应的索引信息;Generate a subtitle stream according to the audio information, wherein the subtitle stream includes a subtitle file and index information corresponding to the subtitle file;
存储多个字幕流;Store multiple subtitle streams;
接收终端发送的播放请求,播放请求用于查找第一流媒体片段对应的第一字幕流;receiving a playback request sent by the terminal, where the playback request is used to find the first subtitle stream corresponding to the first streaming media segment;
当确定第一字幕流已存储,向终端发送第一字幕流和第一流媒体片段,以便终端解析第一字幕流,得到第一字幕内容,使第一字幕内容和第一媒体流片段同步播放。When it is determined that the first subtitle stream has been stored, the first subtitle stream and the first streaming media segment are sent to the terminal, so that the terminal can parse the first subtitle stream to obtain the first subtitle content and play the first subtitle content and the first media stream segment synchronously.
优选地,该方法还包括:当第一字幕流发送完毕,将存储的第一字幕流进行删除。Preferably, the method further includes: when the first subtitle stream is sent, deleting the stored first subtitle stream.
优选地,将媒体流分段生成多个媒体流片段,还包括:Preferably, segmenting the media stream to generate multiple media stream segments also includes:
确定多个媒体流片段中的一个媒体流片段为第一段媒体流片段;Determining that one media stream segment among the multiple media stream segments is the first segment of the media stream segment;
当多个媒体流片段中的一个媒体流片段为第一段媒体流片段时,生成第一段媒体流片段对应的索引信息。When one media stream segment among the multiple media stream segments is the first segment of the media stream segment, index information corresponding to the first segment of the media stream segment is generated.
优选地,将媒体流分段生成多个媒体流片段,还包括:Preferably, segmenting the media stream to generate multiple media stream segments also includes:
当多个媒体流片段中的一个媒体流片段为第N段媒体流片段时,确定第N-1段媒体流片段对应的字幕流已存储,其中,N为大于1的整数;When one of the multiple media stream segments is the Nth media stream segment, it is determined that the subtitle stream corresponding to the N-1th media stream segment has been stored, where N is an integer greater than 1;
当第N-1段媒体流片段对应的字幕流已存储时,生成第N段媒体流片段对应的索引信息。When the subtitle stream corresponding to the media stream segment of the N-1th segment has been stored, the index information corresponding to the media stream segment of the Nth segment is generated.
第二方面,本发明提供了一种实时字幕播出方法,该方法包括以下步骤:In a second aspect, the present invention provides a method for broadcasting real-time subtitles, the method comprising the following steps:
向服务器发送播放请求,播放请求用于查找第一流媒体片段对应的第一字幕流;Sending a playback request to the server, where the playback request is used to find the first subtitle stream corresponding to the first streaming media segment;
接收服务器发送的第一流媒体片段和第一媒体流片段对应的第一字幕流;receiving the first streaming media segment sent by the server and the first subtitle stream corresponding to the first media streaming segment;
解析第一字幕流,得到第一字幕内容,使第一字幕内容和第一媒体流片段同步播放。The first subtitle stream is parsed to obtain the first subtitle content, so that the first subtitle content and the first media stream segment are played synchronously.
第三方面,本发明提供了一种服务器,该服务器包括:编码模块、字幕生成模块、处理模块;In a third aspect, the present invention provides a server, the server comprising: an encoding module, a subtitle generating module, and a processing module;
编码模块用于将媒体流分段生成多个媒体流片段;The encoding module is used for segmenting the media stream to generate multiple media stream segments;
字幕生成模块用于提取多个媒体流片段中的每个媒体流片段的音频信息;并且,根据音频信息生成字幕流,其中,字幕流包括,字幕文件和字幕文件对应的索引信息;The subtitle generation module is used to extract the audio information of each media stream segment in the plurality of media stream segments; and, generate a subtitle stream according to the audio information, wherein the subtitle stream includes the subtitle file and the index information corresponding to the subtitle file;
处理模块用于存储多个字幕流;The processing module is used to store multiple subtitle streams;
处理模块还用于接收终端发送的播放请求,并根据播放请求,将第一媒体流片段和第一媒体流片段对应的第一字幕流发送至终端。The processing module is further configured to receive a play request sent by the terminal, and according to the play request, send the first media stream segment and the first subtitle stream corresponding to the first media stream segment to the terminal.
优选地,处理模块还用于,当第一流字幕流发送完毕,将存储的第一字幕流进行删除。Preferably, the processing module is further configured to delete the stored first subtitle stream when the first subtitle stream is sent.
优选地,编码模块还用于:Preferably, the encoding module is also used for:
确定多个媒体流片段中的一个媒体流片段为第一段媒体流片段;Determining that one media stream segment among the multiple media stream segments is the first segment of the media stream segment;
当多个媒体流片段中的一个媒体流片段为第一段媒体流片段时,生成第一段媒体流片段对应的索引信息。When one media stream segment among the multiple media stream segments is the first segment of the media stream segment, index information corresponding to the first segment of the media stream segment is generated.
优选地,编码模块还用于:Preferably, the encoding module is also used for:
当多个媒体流片段中的一个媒体流片段为第N段媒体流片段时,确定第N-1段媒体流片段对应的字幕流已存储,其中,N为大于1的整数;When one of the multiple media stream segments is the Nth media stream segment, it is determined that the subtitle stream corresponding to the N-1th media stream segment has been stored, where N is an integer greater than 1;
当第N-1段媒体流片段对应的字幕流已存储时,生成第N段媒体流片段对应的索引信息。When the subtitle stream corresponding to the media stream segment of the N-1th segment has been stored, the index information corresponding to the media stream segment of the Nth segment is generated.
第四方面,本发明提供了一种系统,该系统包括:终端和服务器;In a fourth aspect, the present invention provides a system, the system comprising: a terminal and a server;
该终端包括:发送模块、接收模块、解析模块和显示模块;The terminal includes: a sending module, a receiving module, a parsing module and a display module;
发送模块用于向服务器发送播放请求,播放请求用于查找第一流媒体片段对应的第一字幕流;The sending module is used to send a play request to the server, and the play request is used to find the first subtitle stream corresponding to the first streaming media segment;
接收模块用于接收服务器发送的第一流媒体片段和第一媒体流片段对应的第一字幕流;The receiving module is configured to receive the first streaming media segment sent by the server and the first subtitle stream corresponding to the first media streaming segment;
解析模块用于解析第一字幕流,得到第一字幕内容;The parsing module is used to parse the first subtitle stream to obtain the first subtitle content;
显示模块用于同步播放第一字幕内容和第一媒体流片段。The display module is used for synchronously playing the first subtitle content and the first media stream segment.
本发明通过对直播过程中的流媒体片段进行音频信息提取,实时生成每个流媒体片段对应的字幕流,从而使终端同步播放字幕内容和流媒体片段。同时,本发明将字幕流与流媒体片段分开传输,使字幕流只需以简单文本格式即可进行传输,简单高效;并且字幕的结构不受限制,可以轻易扩展,极大地提升了用户体验度。The present invention generates the subtitle stream corresponding to each streaming media segment in real time by extracting the audio information of the streaming media segment in the live broadcast process, so that the terminal can play the subtitle content and the streaming media segment synchronously. At the same time, the present invention transmits the subtitle stream and the streaming media segment separately, so that the subtitle stream can be transmitted only in a simple text format, which is simple and efficient; and the structure of the subtitle is not limited and can be easily expanded, which greatly improves the user experience. .
附图说明Description of drawings
图1为本发明实施例提供的一种实时字幕播出方法的示意过程图;1 is a schematic process diagram of a method for broadcasting real-time subtitles according to an embodiment of the present invention;
图2为本发明实施例提供的一种生成多个媒体流片段的方法流程图;2 is a flowchart of a method for generating multiple media stream segments provided by an embodiment of the present invention;
图3为本发明实施例提供的一种安装应用程序的方法流程图;3 is a flowchart of a method for installing an application program provided by an embodiment of the present invention;
图4为本发明实施例提供的一种实时字幕播出系统的结构性框图。FIG. 4 is a structural block diagram of a real-time subtitle broadcasting system according to an embodiment of the present invention.
具体实施方式Detailed ways
为使本发明实施例的技术方案以及优点表达的更清楚,下面通过附图和实施例,对本发明的技术方案做进一步的详细描述。In order to express the technical solutions and advantages of the embodiments of the present invention more clearly, the technical solutions of the present invention are further described in detail below through the accompanying drawings and embodiments.
图1为本发明实施例提供的一种实时字幕播出方法流程图,如图1所示,该方法包括:FIG. 1 is a flowchart of a method for broadcasting real-time subtitles provided by an embodiment of the present invention. As shown in FIG. 1 , the method includes:
步骤110,将媒体流分段生成多个媒体流片段。Step 110, segment the media stream to generate multiple media stream segments.
具体地,如图2所示,步骤110,将媒体流分段生成多个媒体流片段,还包括:Specifically, as shown in FIG. 2, step 110, segmenting the media stream to generate multiple media stream segments, further comprising:
步骤111,确定多个媒体流片段中的一个媒体流片段为第一段媒体流片段。Step 111: Determine one media stream segment among the multiple media stream segments as the first media stream segment.
当多个媒体流片段中的一个媒体流片段为第一段媒体流片段时,执行步骤113;When one of the multiple media stream segments is the first media stream segment, perform step 113;
当所述多个媒体流片段中的一个媒体流片段为第N段媒体流片段时,其中,N为大于1的整数,执行步骤112。When one of the multiple media stream segments is the Nth media stream segment, where N is an integer greater than 1, step 112 is performed.
步骤112,确定第N-1段媒体流片段对应的字幕流已存储,其中,N为大于1的整数。Step 112: Determine that the subtitle stream corresponding to the N-1th media stream segment has been stored, where N is an integer greater than 1.
当第N-1段媒体流片段对应的字幕流已存储完毕时,执行步骤113;When the subtitle stream corresponding to the N-1 media stream segment has been stored, step 113 is performed;
若第N-1段媒体流片段对应的字幕流未存储完毕,则挂起等待,直至第N-1段媒体流片段对应的字幕流存储完毕,再执行步骤113。If the subtitle stream corresponding to the N-1 segment of the media stream segment has not been stored, the process suspends and waits until the subtitle stream corresponding to the N-1 segment of the media stream segment is stored, and then executes step 113.
步骤113,生成第N段媒体流片段对应的索引信息。Step 113: Generate index information corresponding to the Nth media stream segment.
步骤120,提取多个媒体流片段中的每个媒体流片段的音频信息。Step 120: Extract audio information of each media stream segment in the multiple media stream segments.
需要说明的是,若无媒体流片段生成时,则挂起等待。It should be noted that, if no media stream segment is generated, it will suspend and wait.
步骤130,根据音频信息生成字幕流,其中,字幕流包括,字幕文件和字幕文件对应的索引信息。Step 130: Generate a subtitle stream according to the audio information, wherein the subtitle stream includes subtitle files and index information corresponding to the subtitle files.
具体地,生成字幕时需记录音频相对时间与当前字幕内容的对应关系以及当前音频片段在整个直播媒体片段中的序号,并以该媒体片段序号为区分特征为该字幕文件命名。索引信息指向字幕文件的存储位置,使终端能正确地请求到该字幕文件。Specifically, when generating subtitles, it is necessary to record the corresponding relationship between the audio relative time and the current subtitle content and the serial number of the current audio clip in the entire live media clip, and use the media clip serial number as a distinguishing feature to name the subtitle file. The index information points to the storage location of the subtitle file, so that the terminal can correctly request the subtitle file.
步骤140,存储多个字幕流。Step 140, store multiple subtitle streams.
具体地,当存储第一段字幕流时,新建一个索引文件,并添加索引信息。当存储第N段字幕流时,将新生成的索引信息添加到原索引文件末尾,同时删除索引文件中最前端、已被请求过的片段索引信息。同时,将字幕文件存到索引指定的位置,并删除索引文件中最前端索引对应的字幕文件。Specifically, when the first subtitle stream is stored, a new index file is created, and index information is added. When storing the subtitle stream of the Nth segment, the newly generated index information is added to the end of the original index file, and at the same time, the segment index information that has been requested at the front end of the index file is deleted. At the same time, the subtitle file is saved to the position specified by the index, and the subtitle file corresponding to the foremost index in the index file is deleted.
步骤145,终端向服务器发送播放请求,播放请求用于查找第一流媒体片段对应的第一字幕流。Step 145: The terminal sends a play request to the server, where the play request is used to search for the first subtitle stream corresponding to the first streaming media segment.
可选地,播放请求可以包括:请求第一索引文件,根据第一索引文件信息请求第一字幕文件。Optionally, the play request may include: requesting a first index file, and requesting a first subtitle file according to the information of the first index file.
步骤150,接收终端发送的播放请求,播放请求用于查找第一流媒体片段对应的第一字幕流。Step 150: Receive a play request sent by the terminal, where the play request is used to search for the first subtitle stream corresponding to the first streaming media segment.
步骤160,当确定第一字幕流已存储。Step 160, when it is determined that the first subtitle stream has been stored.
需要说明的是,当第一字幕流未存储,则拒绝该请求。终端收到请求失败信息后,将增大向服务器请求字幕索引文件的频率,继续请求该索引文件,直至请求成功或超时后进入下一段的请求过程。It should be noted that when the first subtitle stream is not stored, the request is rejected. After receiving the request failure information, the terminal will increase the frequency of requesting the subtitle index file from the server, and continue to request the index file until the request succeeds or the next stage of the request process is entered after timeout.
步骤165,向终端发送第一字幕流和第一流媒体片段,以便终端解析第一字幕流,得到第一字幕内容,使第一字幕内容和第一媒体流片段同步播放。Step 165: Send the first subtitle stream and the first streaming media segment to the terminal, so that the terminal parses the first subtitle stream, obtains the first subtitle content, and causes the first subtitle content and the first media stream segment to be played synchronously.
步骤170,接收服务器发送的第一流媒体片段和第一媒体流片段对应的第一字幕流。Step 170: Receive the first streaming media segment sent by the server and the first subtitle stream corresponding to the first media streaming segment.
步骤180,解析第一字幕流,得到第一字幕内容。Step 180: Parse the first subtitle stream to obtain first subtitle content.
具体地,如图3所示,解析第一字幕流具体步骤如下:Specifically, as shown in Figure 3, the specific steps of parsing the first subtitle stream are as follows:
步骤181,解析字幕格式。Step 181, parse the subtitle format.
若字幕格式正确,则执行步骤182;若字幕格式不正确,停止操作。If the subtitle format is correct, go to step 182; if the subtitle format is incorrect, stop the operation.
步骤182,解析字幕文件。Step 182, parse the subtitle file.
具体地,解析字幕文件名,提取段序号信息n;解析字幕文件序号,该序号为字幕文件中当前字幕的排序;解析字幕的相对显示时间,设某个序号字幕的相对显示时间为t1和t2。Specifically, parse the subtitle file name to extract the segment serial number information n; parse the subtitle file serial number, which is the order of the current subtitles in the subtitle file; parse the relative display time of the subtitle, and set the relative display time of a certain serial number subtitle to be t1 and t2 .
步骤183,生成绝对显示时间戳。Step 183, generate an absolute display time stamp.
绝对显示时间T1=(n-1)·d+t1,同理,T2=(n-1)·d+t2,其中,d为媒体流片段大小,单位秒。The absolute display time T 1 =(n-1)·d+t 1 , and similarly, T 2 =(n-1)·d+t 2 , where d is the size of the media stream segment, in seconds.
若步骤182中解析的信息不足以生成绝对时间戳,停止操作。If the information parsed in step 182 is insufficient to generate an absolute timestamp, the operation is stopped.
步骤184,更新字幕显示时间。Step 184, update the subtitle display time.
具体地,将相对显示时间替换为绝对显示时间。Specifically, the relative display time is replaced with the absolute display time.
步骤185,生成字幕内容。Step 185, generating subtitle content.
步骤190,同步播放第一字幕内容和第一媒体流片段。Step 190: Play the first subtitle content and the first media stream segment synchronously.
首先获取第一媒体流片段的时间位置信息,将此位置信息转为以秒为单位的播放时间值,然后用当前播放时间值与字幕文件中绝对显示时间值作比较,当该播放时间值落入某个序号绝对显示时间内时,显示当前序号下字幕内容。First, obtain the time position information of the first media stream segment, convert this position information into the playback time value in seconds, and then compare the current playback time value with the absolute display time value in the subtitle file. When the playback time value falls When the absolute display time of a certain serial number is entered, the subtitle content under the current serial number is displayed.
步骤195,当第一流字幕流发送完毕,将存储的第一字幕流进行删除。Step 195, when the first subtitle stream is sent, delete the stored first subtitle stream.
本发明实施例通过对直播过程中的流媒体片段进行音频信息提取,实时生成每个流媒体片段对应的字幕流,从而使终端同步播放字幕内容和流媒体片段。同时,本发明实施例将字幕流与流媒体片段分开传输,使字幕流只需以简单文本格式即可进行传输,简单高效;并且字幕的结构不受限制,可以轻易扩展,极大地提升了用户体验度The embodiment of the present invention generates a subtitle stream corresponding to each streaming media segment in real time by extracting audio information from the streaming media segment in the live broadcast process, so that the terminal can play the subtitle content and the streaming media segment synchronously. At the same time, the embodiment of the present invention transmits the subtitle stream and the streaming media segment separately, so that the subtitle stream can be transmitted only in a simple text format, which is simple and efficient; and the structure of the subtitle is not limited and can be easily expanded, which greatly improves the user experience. Experience
第二方面,图4为本发明实施例提供的一种实时字幕播出系统的结构性框图,如图4所示,该系统包括:服务器40和终端50。In the second aspect, FIG. 4 is a structural block diagram of a real-time subtitle broadcasting system provided by an embodiment of the present invention. As shown in FIG. 4 , the system includes: a
该服务器40包括:编码模块41、字幕生成模块42、处理模块43。The
编码模块41用于将媒体流分段生成多个媒体流片段。The
字幕生成模块42用于提取多个媒体流片段中的每个媒体流片段的音频信息;并且,根据音频信息生成字幕流,其中,字幕流包括,字幕文件和字幕文件对应的索引信息。The subtitle generation module 42 is configured to extract audio information of each media stream segment in the multiple media stream segments; and generate a subtitle stream according to the audio information, wherein the subtitle stream includes subtitle files and index information corresponding to the subtitle files.
处理模块43用于存储多个字幕流。The
处理模块43还用于接收终端50发送的播放请求,并根据播放请求,将第一媒体流片段和第一媒体流片段对应的第一字幕流发送至终端50。The
可选地,处理模块43还用于,当第一流字幕流发送完毕,将存储的第一字幕流进行删除。Optionally, the
具体地,编码模块41还用于:Specifically, the
确定多个媒体流片段中的一个媒体流片段为第一段媒体流片段;Determining that one media stream segment among the multiple media stream segments is the first segment of the media stream segment;
当多个媒体流片段中的一个媒体流片段为第一段媒体流片段时,生成第一段媒体流片段对应的索引信息。When one media stream segment among the multiple media stream segments is the first segment of the media stream segment, index information corresponding to the first segment of the media stream segment is generated.
具体地,编码模块41还用于:Specifically, the
当多个媒体流片段中的一个媒体流片段为第N段媒体流片段时,确定第N-1段媒体流片段对应的字幕流已存储,其中,N为大于1的整数;When one of the multiple media stream segments is the Nth media stream segment, it is determined that the subtitle stream corresponding to the N-1th media stream segment has been stored, where N is an integer greater than 1;
当第N-1段媒体流片段对应的字幕流已存储时,生成第N段媒体流片段对应的索引信息。When the subtitle stream corresponding to the media stream segment of the N-1th segment has been stored, the index information corresponding to the media stream segment of the Nth segment is generated.
该终端50包括:发送模块51、接收模块52、解析模块53和显示模块54。The terminal 50 includes: a sending module 51 , a receiving
发送模块51用于向服务器40发送播放请求,播放请求用于查找第一流媒体片段对应的第一字幕流。The sending module 51 is configured to send a play request to the
接收模块52用于接收服务器40发送的第一流媒体片段和第一媒体流片段对应的第一字幕流。The receiving
解析模块53用于解析第一字幕流,得到第一字幕内容。The
显示模块54用于同步播放第一字幕内容和第一媒体流片段。The
本发明实施例通过对直播过程中的流媒体片段进行音频信息提取,实时生成每个流媒体片段对应的字幕流,从而使终端同步播放字幕内容和流媒体片段。同时,本发明实施例将字幕流与流媒体片段分开传输,使字幕流只需以简单文本格式即可进行传输,简单高效;并且字幕的结构不受限制,可以轻易扩展,极大地提升了用户体验度。The embodiment of the present invention generates a subtitle stream corresponding to each streaming media segment in real time by extracting audio information from the streaming media segment in the live broadcast process, so that the terminal can play the subtitle content and the streaming media segment synchronously. At the same time, the embodiment of the present invention transmits the subtitle stream and the streaming media segment separately, so that the subtitle stream can be transmitted only in a simple text format, which is simple and efficient; and the structure of the subtitle is not limited and can be easily expanded, which greatly improves the user experience. experience.
专业人员应该还可以进一步意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本发明的范围。Professionals should be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. Interchangeability, the above description has generally described the components and steps of each example in terms of function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.
结合本文中所公开的实施例描述的方法或算法的步骤可以用硬件、处理器执行的软件模块,或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。The steps of a method or algorithm described in connection with the embodiments disclosed herein may be implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.
以上所述的具体实施方式,对本发明的目的、技术方案和有益效果进行了进一步详细说明,所应理解的是,以上所述仅为本发明的具体实施方式而已,并不用于限定本发明的保护范围,凡在本发明的精神和原则之内,所做的任何修改、等同替换、改进等,均应包含在本发明的保护范围之内。The specific embodiments described above further describe the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims (8)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510491214.8A CN106454547B (en) | 2015-08-11 | 2015-08-11 | A kind of real-time subtitle broadcasting method and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510491214.8A CN106454547B (en) | 2015-08-11 | 2015-08-11 | A kind of real-time subtitle broadcasting method and system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106454547A CN106454547A (en) | 2017-02-22 |
CN106454547B true CN106454547B (en) | 2020-01-31 |
Family
ID=58093718
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510491214.8A Active CN106454547B (en) | 2015-08-11 | 2015-08-11 | A kind of real-time subtitle broadcasting method and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106454547B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019194742A1 (en) * | 2018-04-04 | 2019-10-10 | Nooggi Pte Ltd | A method and system for promoting interaction during live streaming events |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101382937A (en) * | 2008-07-01 | 2009-03-11 | 深圳先进技术研究院 | Speech recognition-based multimedia resource processing method and its online teaching system |
CN101540847A (en) * | 2008-03-21 | 2009-09-23 | 株式会社康巴思 | Caption creation system and caption creation method |
CN102802044A (en) * | 2012-06-29 | 2012-11-28 | 华为终端有限公司 | Video processing method, terminal and subtitle server |
CN103297709A (en) * | 2013-06-19 | 2013-09-11 | 江苏华音信息科技有限公司 | Device for adding Chinese subtitles to Chinese audio video data |
CN103544978A (en) * | 2013-11-07 | 2014-01-29 | 上海斐讯数据通信技术有限公司 | Multimedia file manufacturing and playing method and intelligent terminal |
CN103561217A (en) * | 2013-10-14 | 2014-02-05 | 深圳创维数字技术股份有限公司 | Method and terminal for generating captions |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8914534B2 (en) * | 2011-01-05 | 2014-12-16 | Sonic Ip, Inc. | Systems and methods for adaptive bitrate streaming of media stored in matroska container files using hypertext transfer protocol |
-
2015
- 2015-08-11 CN CN201510491214.8A patent/CN106454547B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101540847A (en) * | 2008-03-21 | 2009-09-23 | 株式会社康巴思 | Caption creation system and caption creation method |
CN101382937A (en) * | 2008-07-01 | 2009-03-11 | 深圳先进技术研究院 | Speech recognition-based multimedia resource processing method and its online teaching system |
CN102802044A (en) * | 2012-06-29 | 2012-11-28 | 华为终端有限公司 | Video processing method, terminal and subtitle server |
CN103297709A (en) * | 2013-06-19 | 2013-09-11 | 江苏华音信息科技有限公司 | Device for adding Chinese subtitles to Chinese audio video data |
CN103561217A (en) * | 2013-10-14 | 2014-02-05 | 深圳创维数字技术股份有限公司 | Method and terminal for generating captions |
CN103544978A (en) * | 2013-11-07 | 2014-01-29 | 上海斐讯数据通信技术有限公司 | Multimedia file manufacturing and playing method and intelligent terminal |
Also Published As
Publication number | Publication date |
---|---|
CN106454547A (en) | 2017-02-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR102091414B1 (en) | Enriching broadcast media related electronic messaging | |
US9961398B2 (en) | Method and device for switching video streams | |
US20160316233A1 (en) | System and method for inserting, delivering and tracking advertisements in a media program | |
CN104023250B (en) | Based on the real-time interactive method and system of Streaming Media | |
WO2015090095A1 (en) | Information pushing method, device, and system | |
CN104902343B (en) | A kind of method, server and the terminal of transmission and playing audio-video and message | |
JP2002297630A (en) | Method and device for index generation, index addition system, program, and storage medium | |
US12206961B2 (en) | Systems and methods for summarizing missed portions of storylines | |
KR101991188B1 (en) | Promotion information processing method, device, and apparatus, and non-volatile computer storage medium | |
CN112954434B (en) | Subtitle processing method, system, electronic device and storage medium | |
US11540028B2 (en) | Information presenting method, terminal device, server and system | |
TWI802797B (en) | Method for detecting and responding to a fingerprint mismatch detected after a previously detected fingerprint match, non-transitory computer-readable storage medium, and computing system | |
CN109168020A (en) | Method for processing video frequency, device, calculating equipment and storage medium based on live streaming | |
CN103179450A (en) | Method, device and system for playing video, and audio track server | |
US20220321970A1 (en) | Dynamic Real-Time Audio-Visual Search Result Assembly | |
US20170180445A1 (en) | Advertisement data acquisition method and electronic equipment | |
CN111512635A (en) | Method and system for selectively skipping media content | |
CN116346794A (en) | Method, device and non-volatile computer-readable storage medium for receiving media data | |
CN109842804A (en) | Processing method and server, the computer storage medium of audio, video data | |
CN115623264A (en) | Live stream subtitle processing method and device and live stream playing method and device | |
WO2012071931A1 (en) | Method and system for dynamically inserting cut-in program into segmented program | |
US20120150990A1 (en) | System and method for synchronizing with multimedia broadcast program and computer program product thereof | |
CN106454547B (en) | A kind of real-time subtitle broadcasting method and system | |
CN104602046A (en) | HLS protocol based information publishing method, device and system | |
CN109587517A (en) | Multimedia file playing method and device, server and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right |
Effective date of registration: 20210804 Address after: Room 1601, 16th floor, East Tower, Ximei building, No. 6, Changchun Road, high tech Industrial Development Zone, Zhengzhou, Henan 450001 Patentee after: Zhengzhou xinrand Network Technology Co.,Ltd. Address before: 100190, No. 21 West Fourth Ring Road, Beijing, Haidian District Patentee before: INSTITUTE OF ACOUSTICS, CHINESE ACADEMY OF SCIENCES |
|
TR01 | Transfer of patent right |