TWI665663B - Video and audio reverse playback device and system and method thereof - Google Patents

Abstract

An audio and video reverse playback device includes a transceiver module that receives a sound signal, an identification module connected to the transceiver module, a processing module connected to the identification module, and a playback module connected to the processing module. The identification module Recognize the sound signal and generate a pair of text information corresponding to the sound signal, and the text information includes plural texts, the processing module rearranges the texts according to an inverted playback condition, and generates an inverted playback text file, the playback module will The reverse playback text file is played synchronously with the reversed sound signal.

Description

Video and audio reverse playback device and system and method thereof

The present invention relates to a video and audio reverse playback device and system, in particular to a video and audio reverse playback device and system capable of synchronously playing sounds and subtitle tracks when performing reverse playback.

In traditional video playback operations, functions such as forward, seek, reverse, and playback can be used to display multimedia content in a media player. As technology advances, many consumer electronics devices often support additional enhanced playback modes, such as variable speed playback and reverse playback.

However, in the normal reverse playback process, the processing method for the sound track is often played in a silent manner. This way not only loses the fun that it brings to the viewer when playing backwards, but it also fails to reflect the way the sound track should appear when playing backwards.

Therefore, an object of the present invention is to provide a video and audio reverse playback device that can simultaneously play sounds and subtitle tracks when performing reverse playback.

Therefore, the audio and video reverse playback device of the present invention includes a transceiver module that receives a sound signal, an identification module connected to the transceiver module, a processing module connected to the identification module, and a playback module connected to the processing module. The recognition module recognizes the sound signal and generates a pair of text information corresponding to the sound signal, and the text information includes plural texts. The processing module rearranges the texts according to a reverse playback condition, and generates a reverse playback text file. The module will play the reversed text file synchronously with the reversed audio signal.

In an embodiment, the transceiver module also receives an image signal related to the sound signal, and the playback module plays the reverse playback text file synchronously with the reversed audio signal and image signal.

In one embodiment, the audio and video reverse playback device further includes a storage module connected to the identification module and the processing module, for storing text information and playing back the text file in reverse.

In one embodiment, the processing module reversely arranges the words in units of words, words, or sentences according to the reverse playback conditions. The reverse playback conditions include the representation settings of the words / words / sentences and the individual words / words / sentences. Number setting.

Further, the text information also includes a plurality of durations corresponding to the texts, and the reverse playback condition further includes a setting based on the duration or a basic unit of words / words / sentences.

In one embodiment, the file format of the reverse playback text file is a plain text file, a file in the form of a subtitle track, or metadata with the concept of hierarchy and time data fields.

In addition, another object of the present invention is to provide an audio and video reverse playback system that can simultaneously play sounds and subtitle tracks when performing reverse playback.

The video reverse playback system of the present invention includes a video reverse playback device, a recognition server and a voice analysis server. The video reverse playback device communicates with the recognition server and the voice analysis server, and includes a transceiver module for receiving a voice signal. And a playback module connected to the transceiver module, the recognition server recognizes the sound signal and generates a pair of text information corresponding to the sound signal, and the text information includes plural words, and the voice analysis server The text is rearranged, and a reverse playback text file is generated, so that the playback module plays the reverse playback text file synchronously with the reversed sound signal.

In one embodiment, the recognition server and the speech analysis server are integrated into a same server.

In an embodiment, the transceiver module also receives an image signal related to the sound signal, and the playback module plays the inverted play text file in synchronization with the inverted sound signal and the image signal.

In an embodiment, the speech analysis server sorts the words back in units of words, words, or sentences according to the reverse playback conditions. The reverse playback conditions include the representation settings of the words / words / sentences and the Number setting.

Further, the text information also includes a plurality of durations corresponding to the texts, and the reverse playback condition further includes a setting based on the duration or a basic unit of words / words / sentences.

In one embodiment, the file format of the reverse playback text file may be a plain text file, a file in the form of a subtitle track, or metadata with the concept of hierarchy and time data fields.

In addition, another object of the present invention is to provide a video and audio reverse playback method that can simultaneously play sounds and subtitle tracks when performing reverse playback.

The method for reversing playback of video and audio in the present invention is to identify a sound signal and generate a pair of text information corresponding to the sound signal, and to rearrange the plural texts in the text information according to a reverse playback condition, and the rearranged Wait for the text to play in sync with the reversed sound signal.

In one embodiment, the present method for reversing playback of an audio and video is implemented in a device for reversing playback of an audio and video.

In one embodiment, the present video reverse playback method is applied to a video reverse playback system. The video reverse playback system includes a video reverse playback device, a recognition server and a voice analysis server, a video reverse playback device and a recognition servo. And the voice analysis server communicate with each other, and includes a transceiver module for receiving a sound signal and a playback module connected to the transceiver module. The recognition server recognizes the sound signal and generates text information. The voice analysis server will The texts in the text information are rearranged, so that the playback module plays the rearranged texts in sync with the inverted sound signal.

In one embodiment, the present method for reversing playback of audio and video is to rearrange the plural texts in the text information according to the playback conditions of the reverse to generate a reverse playback text file, and synchronously play the reverse playback text file with the reversed sound signal, the reverse The file format of the play text file is a plain text file, a file in the form of a subtitle track, or metadata with the concept of hierarchy and time data fields.

In one embodiment, the method for reversing playback of the video and audio is to arrange the words back in units of words, words, or sentences according to the reversing playback conditions. The reversing playback conditions include the representation settings of the words / words / sentences and the words / words / sentences The number of settings.

Further, the text information includes the duration of the text and the plural corresponding to the text, and the reverse playback condition also includes a setting based on the duration or a basic unit of words / words / sentences.

The effect of the present invention is that when performing reverse playback, the sound track and subtitle track are played back synchronously with the image to increase the fun.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are represented by the same numbers.

Referring to FIG. 1, a first embodiment of a video reverse playback device 100 according to the present invention. The video reverse playback device 100 enables multimedia information (such as movie files or online videos) to be synchronized and its sound and subtitle tracks can be synchronized during reverse playback. Play for fun.

In this embodiment, the audio and video reverse playback device 100 may be a smart phone that implements a voice recognition function and playback software, and includes a transceiver module 10, a recognition module 20 connected to the transceiver module 10, and a connection recognition module. The processing module 30 of the group 20 and a playback module 40 connected to the processing module 30.

The transceiver module 10 receives a multimedia information, the multimedia information includes a sound signal and an image signal related to the sound signal, such as video and audio in a video file, but in different application requirements The multimedia information may also include only audio signals (for example, music files), which is not limited to this embodiment.

The recognition module 20 recognizes the sound signal in the multimedia information and generates a pair of text information corresponding to the sound signal, and the text information includes a plurality of texts, and these texts can be expressed in three levels of words, words, and sentences. For example, in Figure 2, there are 33 basic units (words) of the first layer, 19 basic units (words) of the second layer, and 3 basic units (sentences) of the third layer, since words are composed of words Sentences are composed of words, so the words in the second hierarchy are composed of words in the first hierarchy, and the sentences in the third hierarchy are composed of words in the second hierarchy.

In addition, in some embodiments, the text information may include a plurality of texts and a plurality of durations corresponding to the texts. Taking FIG. 3 as an example, the text information includes “I”, “will”, “be "8", "back", "soon", "you", "are", "fired", each of which corresponds to a duration of T1 to T8. If these characters are represented by the first level (word), The corresponding list with time is shown in Table 1 below. Table I First class Basic grammar Duration (ms) 1 I 300 (T1) 2 will 400 (T2) 3 be 200 (T3) 4 back 500 (T4) 5 soon 300 (T5) 6 you 200 (T6) 7 are 200 (T7) 8 fired 500 (T8)

If the characters are represented by the second level (words), their correspondence with time is shown in Table 2 below. Table II Second tier Basic grammar Duration (ms) 1 I will be back soon 1700 (T1 ～ T5) 2 You are fired 900 (T6 ～ T8)

In particular, the above is only an example, and the representation and duration of each text in the text information can be adjusted according to different settings and applications, not limited to the above.

Referring to FIG. 1, the processing module 30 rearranges the characters according to a reverse playback condition, and generates a reverse playback text file. In this embodiment, the reverse playback conditions are the above-mentioned different levels of representation, the number of words / words / sentences, etc. The user can set it through an application (APP) or software. Taking the screen in FIG. 4 as an example, Reverse playback conditions include playback units, restricted units, and the number of restrictions. The playback unit is to select the level of text representation, that is, the subtitles are displayed in words, words, or sentences as the basic unit during reverse playback. The restricted unit is to select time or Reverse playback of the unit. If time is selected (for example: 2 seconds), the subtitle will be played back based on the time (for example, every 2 seconds as the time zone). If the unit is selected, the subtitle will be displayed in words, words, or The basic unit of a sentence is the standard reverse playback; the limit number is the number of time units or units of words / words / sentences that are set for reverse playback.

Taking the example of the second sentence in Figure 2 as an example, if the playback unit is set to the second level (words, represented by solid lines), the limit unit is set to unit playback, and the limit number is set to 1, the text will be as shown in the figure. The flashback arrangement shown in Figure 5, where each "word" belongs to a reverse playback unit, in the reverse playback unit is played in the forward order. If the playback unit is set to the second level (words), the limit unit is set to unit playback, and the limit number is set to 2 (indicated by the dot chain line), then these words will be performed with 2 words as a reverse playback unit body Flashback arrangement, the results of rearrangement will be shown in Figure 6.

Taking the example in FIG. 3 as an example, if the playback unit is set to the first level (word), the limit unit is set to time playback, and the limit number is set to 900ms, the result after the text rearrangement will be shown in FIG. 7, where The limit unit is set to time, so every 900ms belongs to a reverse playback unit, and the reverse playback unit is played in the forward order, and some characters (such as be) will be repeatedly played because they fall into two time intervals at the same time.

Therefore, the processing module 30 rearranges the text and generates a reverse playback text file according to a preset or user-set reverse playback condition. The file format of the reverse playback text file can be a plain text file or a subtitle track. (For example: webvtt), or any metadata with the concept of hierarchy and time data fields, and output the inverted playback text file to the playback module 40.

It is added that the video reverse playback device 100 of this embodiment further includes a storage module 50 connected to the identification module 20 and the processing module 30 to store text information generated by the identification module 20 after identifying the audio signal. And the reverse playback text file generated by the processing module 30, both can be stored in the form of plain text files or subtitle track files, or any metadata with the concept of hierarchy and time data fields. In addition, the reverse playback text file can be combined with the text information generated by the recognition module 20 into the same file, or saved in a new file, or it can be stored as a data track of the original multimedia file. limit.

The playback module 40 may be a display screen, a speaker, or a circuit or a combination of executable playback software. The playback module 40 plays the inverted playback text file synchronously with the inverted sound signal and video signal. In this way, when the user When performing reverse playback, in addition to the image will be reversed, the sound and corresponding text will also be reversed accordingly, which will increase the fun of making videos or using it.

Referring to FIG. 8, a second embodiment of the video reverse playback device 100 according to the present invention. In this embodiment, the video reverse playback device 100 can be applied to a video reverse playback system. The video reverse playback system includes the video reverse playback device 100. And an identification server 200 and a speech analysis server 300 in communication with the video reverse playback device 100, wherein the function and operation of the identification server 200 are the same as the identification module 20 (see FIG. 1) of the first embodiment, or include a Like the circuit of the identification module 20 of the first embodiment, the speech analysis server 300 includes a processing module 30 and a storage module 50 connected to the processing module 30. The processing module 30 and the storage module 50 The functions and operations are the same as those of the first embodiment, so they will not be described in detail.

The video reverse playback device 100 in this embodiment may be a smart phone, a smart TV, or any electronic device with a multimedia player. The functions and operations of the transceiver module 10 and the playback module 40 are the same as those of the first embodiment. When the transceiver module 10 receives a multimedia message, the multimedia message can be transmitted to the identification server 200 through the network, so that the identification server 200 can identify the audio signal in the multimedia information and generate a pair of audio signals corresponding to the audio signal. Text information, and return the text information to the video reverse playback device 100, when the user wants to perform reverse playback, the video reverse playback device 100 will transmit the text information to the voice analysis server 300, so that The playback conditions rearrange the text in the text information, and generate a reverse playback text file, and return the reverse playback text file to the audio and video reverse playback device 100 for the playback module 40 to convert the reverse playback text file and the reversed The audio signal is played synchronously. In this way, through the services of the cloud server (identification server 200 and voice analysis server 300), the audio track and subtitle track can also be played back synchronously with the image to increase the fun.

In particular, the identification server 200 and the speech analysis server 300 can be integrated into a same server (for example, an audio and video server), which can provide users to connect to upload / download multimedia videos and perform speech analysis processing. In addition, the reverse playback text file generated by the voice analysis server 300 can also be stored in the storage module 50, or returned to the video reverse playback device 100, or both, which can be used for different purposes. Demand.

Referring to FIG. 9, a method for reverse playback of video and audio according to the present invention is applicable to the video reverse playback device 100 and the video reverse playback system of the above two embodiments, or any electronic device that can execute the method. The following will explain in detail the process of this video reverse playback method.

In step S10, the audio / video reverse playback device 100 can obtain multimedia information through a network or the like. The multimedia information may include a sound signal and an image signal related to the sound signal, or only a sound signal.

In step S20, the audio and video reverse playback device 100 (or through the recognition server 200) recognizes the sound signal and generates a pair of text information corresponding to the sound signal. The text information may include only plural texts (plain text files) or plural texts. And plural corresponds to the duration of such words.

In step S30, the audio and video reverse playback device 100 (or through the voice analysis server 300) rearranges the text in the text information according to a reverse playback condition and generates a reverse playback text file. The reverse playback conditions may include a playback unit, a restricted unit, and The number of files is limited, and the file format of the reverse playback text file can be a plain text file, a subtitle track format file, or metadata with the concept of hierarchy and time data fields.

In step S40, the audio / video reverse playback device 100 synchronously plays the reverse playback text file and the reversed audio signal, so that when the reverse playback is performed, the audio track and the subtitle track are played back synchronously with the image to increase the fun.

In addition, in some specific applications, the video reverse playback method of the present invention can be presented by a computer program product, so that the computer or any electronic device can load the program and execute the above-mentioned video reverse playback method.

In summary, the audio and video reverse playback device 100 and the audio and video reverse playback system of the present invention, through speech recognition and text analysis, enable the audio track and subtitle track of multimedia information to be played synchronously during reverse playback to increase interest , So it can indeed achieve the purpose of cost invention.

However, the above are only examples of the present invention. When the scope of implementation of the present invention cannot be limited in this way, any simple equivalent changes and modifications made in accordance with the scope of the patent application and the content of the patent specification of the present invention are still Within the scope of the invention patent.

100‧‧‧ Reverse playback device

200‧‧‧Identification server

300‧‧‧Voice Analysis Server

10‧‧‧ transceiver module

20‧‧‧Identification Module

30‧‧‧Processing Module

40‧‧‧playback module

50‧‧‧Storage Module

S10 ～ S40‧‧‧‧Steps

Other features and effects of the present invention will be clearly presented in the embodiment with reference to the drawings, in which: FIG. 1 is a schematic circuit block diagram of a first embodiment of the video reverse playback device of the present invention; FIG. 2 is a first embodiment Schematic diagram of Chinese character information; Figure 3 is a schematic diagram of another aspect of the text information in the first embodiment; Figure 4 is a schematic diagram of the reverse playback condition setting of the first embodiment; Figure 5 is a second sentence of Figure 2 The playback unit is the second layer, the playback unit is the playback unit, and the limit is 1. The reverse playback result is set as shown in Figure 1. Figure 6 shows the second sentence in Figure 2 as the playback unit and the playback unit as the second unit. , The reverse playback result under the setting of 2 is limited; FIG. 7 is the reverse playback result under the setting of FIG. 3 with the playback unit as the first level, the limit unit is time playback, and the limit number is 900ms; FIG. 8 is A schematic circuit block diagram of a second embodiment of the video reverse playback device of the present invention; and FIG. 9 is a flowchart of the video reverse playback method of the present invention.

Claims (19)

An audio and video reverse playback device includes: a transceiver module that receives a sound signal; an identification module connected to the transceiver module, the identification module recognizes the sound signal and generates a pair of text information corresponding to the sound signal, and the The text information includes plural texts; a processing module connected to the recognition module, the processing module rearranges the texts according to a reverse playback condition, and generates a reverse playback text file; and a playback module connected to the processing Module, the playback module plays the reverse playback text file synchronously with the reversed sound signal. The audio and video reverse playback device according to claim 1, wherein the transceiver module further receives an image signal related to the sound signal, and the playback module is to reverse the playback text file and the sound signal and the image reversed The signal plays synchronously. The audio and video reverse playback device according to claim 1, further comprising a storage module connected to the identification module and the processing module, for storing the text information and the reverse playback text file. The video and audio reverse playback device according to claim 1, wherein the processing module reversely arranges the words in units of words, words, or sentences according to the reverse playback conditions, and the reverse playback conditions include representations of words / words / sentences Settings and the number of words / words / sentences. The audio and video reverse playback device according to claim 4, wherein the text information further includes a plurality of durations corresponding to the texts, and the reverse playback conditions further include a basic unit based on the duration or in words / words / sentences Set the benchmark. The audio and video reverse playback device according to claim 1, wherein the file format of the reverse playback text file is a plain text file, a subtitle track format file, or metadata with the concept of hierarchy and time data fields. An audio and video reverse playback system includes: an audio and video reverse playback device including a transceiver module that receives a sound signal and a playback module connected to the transceiver module; an identification server that communicates with the audio and video reverse playback device. The recognition server recognizes the sound signal and generates a pair of text information corresponding to the sound signal, and the text information includes plural words; and a voice analysis server, which communicates with the video reverse playback device, and the voice analysis server plays according to a reverse playback Conditionally rearrange the words, and generate a reverse playback text file, so that the playback module plays the reverse playback text file in synchronization with the reversed sound signal. The audio-visual reverse playback system according to claim 7, wherein the recognition server and the voice analysis server are integrated into a same server. The audio and video reverse playback system according to claim 7, wherein the transceiver module further receives an image signal related to the sound signal, and the playback module converts the reverse play text file and the reversed sound signal and the image signal Play synchronously. The audio and video reverse playback system according to claim 7, wherein the voice analysis server reversely arranges the characters in words, words or sentences according to the reverse playback conditions, and the reverse playback conditions include words / words / sentences And the number of words / words / sentences. The audio and video reverse playback system according to claim 10, wherein the text information further includes a plurality of durations corresponding to the texts, and the reverse playback conditions further include a basic unit based on the duration or in words / words / sentences Set the benchmark. The audio and video reverse playback system according to claim 7, wherein the file format of the reverse playback text file is a plain text file, a subtitle track format file, or metadata with the concept of hierarchy and time data fields. An audio and video reverse playback method is based on identifying a sound signal and generating a pair of text information corresponding to the sound signal, and rearranging the plural text in the text information according to a reverse playback condition. The text plays in sync with the reversed sound signal. The video reverse playback method according to claim 13, wherein the video reverse playback method is implemented in a video reverse playback device. The video reverse playback method according to claim 13, wherein the video reverse playback method is applied to a video reverse playback system, and the video reverse playback system includes a video reverse playback device, a recognition server, and a voice analysis server The audio and video reverse playback device communicates with the recognition server and the voice analysis server, and includes a transceiver module receiving the sound signal and a playback module connected to the transceiver module, and the recognition server recognizes the sound signal The text information is generated, and the speech analysis server rearranges the text in the text information according to the reverse playback condition, so that the playback module synchronizes the rearranged text with the reversed sound signal. The video reverse playback method according to claim 13, wherein the video reverse playback method rearranges the plural characters in the text information according to the reverse playback conditions to generate a reverse playback text file, and the reverse playback text file Synchronized playback with the reversed audio signal. The file format of the reverse playback text file is a plain text file, a subtitle track format file, or metadata with the concept of hierarchy and time data fields. The video reverse playback method according to claim 13, wherein the video reverse playback method arranges the words in words, words or sentences according to the reverse playback conditions, and the reverse playback conditions include words / words / sentences. And the number of words / words / sentences. The video and audio reverse playback method according to claim 17, wherein the text information includes the duration of the text and the plural corresponding to the text, and the reverse playback condition also includes the duration as a reference or a word / word / sentence The basic unit is set as the benchmark. A computer program product. After the program is loaded and executed on a computer, the method described in any one of claims 13 to 18 can be completed.