CN113630650B - Digital television playing method and device based on audio and video switching and computer equipment - Google Patents
Digital television playing method and device based on audio and video switching and computer equipment Download PDFInfo
- Publication number
- CN113630650B CN113630650B CN202111184902.1A CN202111184902A CN113630650B CN 113630650 B CN113630650 B CN 113630650B CN 202111184902 A CN202111184902 A CN 202111184902A CN 113630650 B CN113630650 B CN 113630650B
- Authority
- CN
- China
- Prior art keywords
- data
- digital
- playing
- audio data
- display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000004590 computer program Methods 0.000 claims description 7
- 238000013075 data extraction Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 2
- 208000003464 asthenopia Diseases 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application discloses a digital television playing method based on audio and video switching, which is used for acquiring digital data to be played; extracting first digital video data, and playing a video according to the first digital video data; extracting first digital audio data, and playing audio according to the first digital audio data; carrying out image acquisition processing on a viewer in front of the display to obtain an image of the viewer, and judging whether the sight of the viewer looks directly at the display or not; if the display is directly viewed, judging whether the display is in a switching node; if the node is at the switching node, stopping video playing on the display and simultaneously stopping audio playing on the sound player; playing audio according to the second digital audio data; the method and the device continuously perform the image acquisition processing, the sight judgment processing and the data switching processing of the viewer, thereby solving the problem that the video content played by the display cannot be received by the viewer, and still keeping a smooth playing state in the process.
Description
Technical Field
The present application relates to the field of computers, and in particular, to a digital television playing method and apparatus based on audio/video switching, a computer device, and a storage medium.
Background
The playing of the television program needs to pass through the display, and therefore, the viewer needs to look directly at the display and keep a direct-view state during the playing of the television program. When the viewer does not look directly at the display, the video content played by the display cannot be received by the viewer. The prior art lacks a solution to this problem.
Disclosure of Invention
The application provides a digital television playing method based on audio and video switching, which comprises the following steps:
s1, the digital television playing terminal acquires digital data to be played; the digital data to be played comprises first digital video data, first digital audio data and second digital audio data, wherein the first digital video data, the first digital audio data and the second digital audio data have the same playing time sequence, and a plurality of switching nodes are marked in the playing time sequence;
s2, extracting first digital video data from the digital data to be played, and playing a video on a preset display according to the first digital video data;
s3, extracting first digital audio data from the digital data to be played, and playing audio by using a preset sound player according to the first digital audio data;
s4, carrying out image acquisition processing on a viewer in front of the display by adopting a preset camera to obtain a viewer image, and judging whether the sight of the viewer looks directly at the display or not according to the viewer image;
s5, if the line of sight of the viewer is directly looking at the display, judging whether the first digital video data or the first digital audio data played at the current time is at a switching node;
s6, if the first digital video data or the first digital audio data played at the current time is at the switching node, stopping playing the video on the display and stopping playing the audio on the audio player; then according to the second digital audio data, playing audio by using a sound player;
s7, continuously performing viewer image acquisition processing, viewer sight line judgment processing and data switching processing, so that when the viewer sight line looks straight at the display, a video corresponding to the first digital video data is played on the display, and an audio corresponding to the first digital audio data is played by using a sound player; playing audio corresponding to the second digital audio data using a sound player when a viewer does not look directly at the display; wherein, the playing time sequence is still kept before and after the data switching processing.
Further, the first digital video data is obtained by performing video acquisition on a digital television program scene;
the first digital audio data corresponds to audio data of a speech of the first digital video data;
the second digital audio data is audio data for describing the plot of the digital television program by adopting voice;
in the same time window, the first digital video data, the first digital audio data and the second digital audio data all correspond to the same program episode.
Further, the step S4 of having a plurality of viewers in front of the display, and acquiring images of the viewers in front of the display by using a preset camera to obtain images of the viewers, and determining whether the line of sight of the viewers is directly viewed on the display according to the images of the viewers includes:
s401, carrying out image acquisition processing on a viewer in front of the display by adopting a preset camera to obtain a viewer image;
s402, judging whether the visual lines of all the viewers are directly watching the display according to the images of the viewers;
s403, if the line of sight of all the viewers is not directly looking at the display, acquiring a first position of the viewer directly looking at the display, and acquiring a second position of the viewer not directly looking at the display;
s404, judging whether the first digital video data or the first digital audio data played at the current time is at a switching node;
s405, if the first digital video data or the first digital audio data played at the current time is at a switching node, maintaining the video playing on the display and simultaneously stopping the audio playing on the sound player;
s406, according to the first digital audio data, a preset directional sound generator is adopted to only send a first directional sound to the first position;
and S407, sending a second directional sound only to the second position by adopting a preset directional sound generator according to the second digital audio data.
Further, the second digital audio data is composed of a plurality of sub-data segments, the plurality of sub-data segments use the plurality of switching nodes as boundaries, and at least one of the plurality of sub-data segments is a null data segment.
Further, before the step S1 of acquiring the digital data to be played by the digital television playing terminal, the method includes: s01, the digital signal sending end cuts the original second digital audio data to obtain a plurality of data segments; wherein the plurality of data segments use the plurality of switching nodes as boundaries;
s02, clustering the data segments to obtain an important cluster and an unimportant cluster; the insignificant cluster is not an empty set;
s03, replacing all data segments in the unimportant cluster by using empty data segments, so as to convert the original second digital audio data into final second digital audio data;
s04, taking the first digital video data, the first digital audio data and the final second digital audio data as digital data to be played;
and S05, sending the digital data to be played to the digital television playing terminal.
Further, after the step S3 of extracting first digital audio data from the digital data to be played and playing audio by using a preset sound player according to the first digital audio data, the method includes:
s31, judging whether the first digital video data or the first digital audio data played at the current time is at a switching node;
s32, if the first digital video data or the first digital audio data played at the current time is at a switching node, judging whether a data segment in second digital audio data after the switching node is an empty data segment;
s33, if the data segment in the second digital audio data behind the switching node is not a null data segment, acquiring an image of a viewer in front of the display by using a preset camera to obtain a viewer image, and judging whether the line of sight of the viewer looks directly at the display according to the viewer image;
s34, if the line of sight of the viewer directly looks at the display, stopping video playing on the display and stopping audio playing on the sound player;
and S35, playing the audio by using the sound player according to the second digital audio data.
Further, after the step S32 of determining whether the data segment in the second digital audio data after the switching node is an empty data segment if the first digital video data or the first digital audio data played at the current time is at the switching node, the method includes:
s321, if the data segment in the second digital audio data after the node is switched is an empty data segment, maintaining the current video playing and audio playing, and not turning on the camera.
Further, after the step S5 of determining whether the first digital video data or the first digital audio data played at the current time is located at the node switching point if the viewer looks directly at the display, the method includes:
s501, if the first digital video data or the first digital audio data played at the current time are not located at a switching node, maintaining the current video playing and audio playing until the first digital video data or the first digital audio data are located at the switching node;
s502, stopping video playing on the display and stopping audio playing on the sound player; then according to the second digital audio data, playing audio by using a sound player;
s503, continuously carrying out viewer image acquisition processing, viewer sight line judgment processing and data switching processing so that when the viewer sight line is directly viewed on the display, a video corresponding to the first digital video data is played on the display, and an audio corresponding to the first digital audio data is played by using a sound player; playing audio corresponding to the second digital audio data using a sound player when the viewer does not look directly at the display; wherein, the playing time sequence is still kept before and after the data switching processing.
The application discloses digital television play device based on audio frequency and video switches includes:
a to-be-played digital data acquisition unit, configured to instruct a digital television playing terminal to acquire to-be-played digital data; the digital data to be played comprises first digital video data, first digital audio data and second digital audio data, wherein the first digital video data, the first digital audio data and the second digital audio data have the same playing time sequence, and a plurality of switching nodes are marked in the playing time sequence;
the first digital video data extraction unit is used for indicating that first digital video data are extracted from the digital data to be played and playing a video on a preset display according to the first digital video data;
the first digital audio data extraction unit is used for indicating that first digital audio data are extracted from the digital data to be played and playing audio by using a preset sound player according to the first digital audio data;
the viewer image acquisition unit is used for indicating a preset camera to acquire and process an image of a viewer in front of the display to obtain an image of the viewer, and judging whether the sight of the viewer looks directly at the display or not according to the image of the viewer;
a switching node judging unit, configured to instruct, if a line of sight of a viewer looks directly at the display, to judge whether the first digital video data or the first digital audio data played at a current time is at a switching node;
the second digital audio data playing unit is used for indicating that if the first digital video data or the first digital audio data played at the current time is positioned at a switching node, the video playing on the display is stopped, and the audio playing on the sound player is stopped at the same time; then according to the second digital audio data, playing audio by using a sound player;
a continuous image acquisition unit, configured to instruct continuous viewer image acquisition processing, viewer gaze determination processing, and data switching processing to be performed, so that when a viewer gaze is directed directly to the display, a video corresponding to the first digital video data is played on the display, and an audio corresponding to the first digital audio data is played using a sound player; playing audio corresponding to the second digital audio data using a sound player when a viewer is not looking directly at the display; wherein, the playing time sequence is still kept before and after the data switching processing.
The present application provides a computer device comprising a memory storing a computer program and a processor implementing the steps of any of the above methods when the processor executes the computer program.
According to the digital television playing method and device based on audio and video switching and the computer equipment, digital data to be played are obtained; extracting first digital video data, and playing a video according to the first digital video data; extracting first digital audio data, and playing audio according to the first digital audio data; carrying out image acquisition processing on a viewer in front of the display to obtain a viewer image, and judging whether the sight of the viewer looks directly at the display or not; if the line of sight of the viewer is directly looking at the display, judging whether the display is positioned at a switching node; if the node is switched, stopping video playing on the display and simultaneously stopping audio playing on the sound player; then according to the second digital audio data, playing audio by using a sound player; the method and the device continuously perform the image acquisition processing, the sight line judgment processing and the data switching processing of the viewer, solve the problem that the video content played by the display cannot be received by the viewer when the sight line of the viewer does not look directly at the display, and still keep a smooth playing state in the process.
Drawings
Fig. 1 is a schematic flowchart of a digital television playing method based on audio/video switching according to an embodiment of the present application;
fig. 2 is a schematic block diagram of a structure of a digital television playing device based on audio/video switching according to an embodiment of the present application;
fig. 3 is a block diagram illustrating a structure of a computer device according to an embodiment of the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Referring to fig. 1, an embodiment of the present application provides a digital television playing method based on audio/video switching, including the following steps:
s1, the digital television playing terminal acquires digital data to be played; the digital data to be played comprises first digital video data, first digital audio data and second digital audio data, wherein the first digital video data, the first digital audio data and the second digital audio data have the same playing time sequence, and a plurality of switching nodes are marked in the playing time sequence;
s2, extracting first digital video data from the digital data to be played, and playing a video on a preset display according to the first digital video data;
s3, extracting first digital audio data from the digital data to be played, and playing audio by using a preset sound player according to the first digital audio data;
s4, carrying out image acquisition processing on a viewer in front of the display by adopting a preset camera to obtain a viewer image, and judging whether the sight of the viewer looks directly at the display or not according to the viewer image;
s5, if the line of sight of the viewer is directly looking at the display, judging whether the first digital video data or the first digital audio data played at the current time is at a switching node;
s6, if the first digital video data or the first digital audio data played at the current time is located at a switch node, stopping video playing on the display and stopping audio playing on the sound player; then according to the second digital audio data, playing audio by using a sound player;
s7, continuously performing viewer image acquisition processing, viewer sight line judgment processing and data switching processing, so that when the viewer sight line looks straight at the display, a video corresponding to the first digital video data is played on the display, and an audio corresponding to the first digital audio data is played by using a sound player; playing audio corresponding to the second digital audio data using a sound player when a viewer does not look directly at the display; wherein, the playing time sequence is still kept before and after the data switching processing.
The digital television playing terminal acquires digital data to be played; the digital data to be played comprises first digital video data, first digital audio data and second digital audio data, the first digital video data, the first digital audio data and the second digital audio data all have the same playing time sequence, and a plurality of switching nodes are marked in the playing time sequence.
The digital data to be played in the present application is greatly different from the common digital television signal data in that the common digital television signal data only includes the first digital video data and the first digital audio data, but the digital data to be played in the present application also includes the second digital audio data. According to the first digital video data and the first digital audio data, common digital television program playing can be carried out as the common digital television playing scheme. And the second digital audio data is used to overcome the problem when the viewer's line of sight leaves the display, the specific process being explained in connection with the subsequent steps.
The application adds the second digital audio data, but the second digital audio data is not added mechanically, and the second digital audio data has corresponding relations with the first digital audio data and the first digital video data, specifically: the first digital video data, the first digital audio data and the second digital audio data all have the same playing time sequence, and a plurality of switching nodes are marked in the playing time sequence.
The playing time sequence refers to that when the first digital video data, the first digital audio data and the second digital audio data are played at a certain time point or a certain time window at the same playing speed, the first digital video data, the first digital audio data and the second digital audio data all correspond to the same program plot.
The switching node may be calibrated by a computer or by manual calibration, but this does not belong to the digital television playing content of the present application, and the present application can be implemented as long as the switching node exists, and therefore, details are not described herein.
Further, the first digital video data is obtained by performing video acquisition on a digital television program scene; the first digital audio data corresponds to audio data of a speech of the first digital video data; the second digital audio data is audio data for describing the plot of the digital television program by adopting voice; and in the same time window, the first digital video data, the first digital audio data and the second digital audio data all correspond to the same program plot.
Thereby, the first digital audio data, the first digital video data and the second digital audio data are in correspondence, and the playing on the program plot is consistent.
In the implementation process of the scheme of the application, data switching processing needs to be performed, and the playing time sequence is still maintained before and after the data switching processing, which means that continuity of the digital television program scenario is not interrupted before and after the data switching processing, that is, when the first digital video data and the first digital audio data describe the first program scenario and after the data switching processing (of course, switching is performed at a switching node, the first program scenario is required before the switching node and the second program scenario is required after the switching node), the second digital audio data describe the second program scenario following the first program scenario.
At the beginning of playing digital television programs, executing a common digital television playing step, namely extracting first digital video data from the digital data to be played, and playing a video on a preset display according to the first digital video data; and extracting first digital audio data from the digital data to be played, and playing audio by using a preset sound player according to the first digital audio data.
In order to determine whether a viewer receives a video signal displayed by a display and determine whether data switching should be performed, a preset camera is adopted, image acquisition processing is performed on the viewer in front of the display to obtain a viewer image, and whether the line of sight of the viewer directly views the display is judged according to the viewer image; and if the line of sight of the viewer is directly looking at the display, judging whether the first digital video data or the first digital audio data played at the current time is in a switching node.
The method for determining the sight line of the viewer through the image of the viewer relates to an image recognition technology, and can be realized by adopting an intelligent recognition technology based on machine learning and a posture recognition technology based on computer vision. The specific implementation process is as follows: enlarging an image of a viewer to extract a head image and a reference image; relative position analysis is performed to determine the relative position of the viewer's head towards a reference object (e.g., a sofa, a tile slot, etc.) corresponding to the reference image, and from this relative position and the pre-positioned position of the reference object and the display, it is determined whether the viewer's line of sight is looking directly at the display.
Whether the current playing progress is the switching node or not needs to be considered, data switching processing can be carried out only when the node is switched, mechanical interruption is avoided, the program is scenic (the program can be known in the program shooting process, for example, a movie is composed of a plurality of shots and a plurality of scenes), and the switching node can be marked at a position where the plot is continuous and not tight, so that the scheme is favorably and smoothly carried out.
It should be noted that the switching node is not an absolute time point, and is described by taking a progress bar for playing a video as an example, that is, a point on the progress bar.
Since the first digital video data and the first digital audio data correspond to each other and have the same play timing, it is possible to determine whether the first digital video data and the first digital audio data are at a switching node by using any one of the first digital video data and the first digital audio data.
If the first digital video data or the first digital audio data played at the current time is at a switching node, stopping video playing on the display and simultaneously stopping audio playing on a sound player; then according to the second digital audio data, playing audio by using a sound player; continuously performing viewer image acquisition processing, viewer sight line judgment processing and data switching processing so that when the viewer looks straight at the display, a video corresponding to the first digital video data is played on the display, and an audio corresponding to the first digital audio data is played by using a sound player; playing audio corresponding to the second digital audio data using a sound player when a viewer does not look directly at the display; wherein, the playing time sequence is still kept before and after the data switching processing.
When the node is switched, it is only necessary to stop the playing of the first type digital television program, that is, stop the playing of the first digital video data and the first digital audio data, and start the playing of the second type digital television program (that is, the playing of the second digital audio data). At this time, the second digital audio data adopts a simple voice description mode to introduce the next scenario, and the scenario of the next scenario at least continues to the next switching node. And when the next switching node is used, whether the first type of digital television program should be switched back to play is continuously judged.
In the present embodiment, it is necessary to continue the viewer image capture processing, the viewer sight line determination processing, and the data switching processing, and therefore, it is a feature of the present embodiment that the camera is normally open. In another embodiment, the camera does not need to be in a normally open state, but only needs to be opened at a required time, which will be described in the following embodiments.
As mentioned above, the playing timing sequence is still maintained before and after the data switching process, which means that the continuity of the digital tv program scenario is not interrupted before and after the data switching process, that is, when the first digital video data and the first digital audio data describe the first program scenario, and after the data switching process (of course, switching is performed at a switching node, the first program scenario is before the switching node, and the second program scenario is after the switching node), the second digital audio data describe the second program scenario following the first program scenario.
Therefore, when the viewer is in visual fatigue, the viewer can have a rest by closing eyes, and meanwhile, playing interruption cannot be caused, and the viewer can still receive sufficient program plot information.
In one embodiment, the step S4 of using a preset camera to capture an image of a viewer in front of a display to obtain a viewer image, and determining whether the viewer looks directly at the display according to the viewer image includes:
s401, carrying out image acquisition processing on a viewer in front of the display by adopting a preset camera to obtain a viewer image;
s402, judging whether the visual lines of all the viewers are directly watching the display according to the images of the viewers;
s403, if the line of sight of all the viewers is not directly looking at the display, acquiring a first position of the viewer directly looking at the display, and acquiring a second position of the viewer not directly looking at the display;
s404, judging whether the first digital video data or the first digital audio data played at the current time is at a switching node;
s405, if the first digital video data or the first digital audio data played at the current time is at a switching node, maintaining the video playing on the display and simultaneously stopping the audio playing on the sound player;
s406, according to the first digital audio data, a preset directional sound generator is adopted to only send a first directional sound to the first position;
and S407, sending a second directional sound only to the second position by adopting a preset directional sound generator according to the second digital audio data.
Therefore, the directional sound transmission technology is utilized, and the data switching of multiple viewers is realized. When there are multiple viewers, since not all the viewers are necessarily away from the display, if the data switching manner is directly adopted, i.e., the first digital video data and the first digital audio data are forcibly switched to the second digital audio data, it is obviously not suitable for the viewers still watching the display. Therefore, according to the present invention, the first digital video data and the first digital audio data are played by the viewer who looks at the display, but the second digital audio data is played by the viewer who does not look at the display. Moreover, since both types of program broadcasting involve digital audio data, if the broadcasting is performed in a sound full-coverage manner, the two types of audio interfere with each other. Thus, the present application employs a directional sound generator for directional sound transmission to transmit two sounds separately. The position is located through an image, which is the prior art and is not described herein again. The principle of the directional sound generator is various, for example, the sound to be transmitted is used as a carrier wave through ultrasonic waves, and the ultrasonic waves have good directivity, so that the sound can be transmitted directionally. Specifically, ultrasonic waves of two frequencies are transmitted in a predetermined direction, and when both of the ultrasonic waves reach a predetermined position, a new sound wave having a frequency that is the difference (difference frequency) between the frequencies of the original ultrasonic waves is generated, and the new sound wave is the sound wave intended to be transmitted in a directional manner.
Further, the second digital audio data is composed of a plurality of sub-data segments, the plurality of sub-data segments use the plurality of switching nodes as boundaries, and at least one of the plurality of sub-data segments is a null data segment.
This makes a part of the second digital audio data null data, so that a part of the data transfer amount can be saved during the data transfer. This can be done because it is not necessary to have a speech to describe the episode for a part of the program episode, which may be that the content importance of this part is not high, and a rough episode can be inferred already by the lines corresponding to the first digital audio data (meaning that even though the introduction of the episode by means of speech description is used, this episode can be inferred already by the lines and therefore is not necessary); of course, other situations suitable for generating null data segments are not excluded. So that there can be empty data segments. When data switching is performed, it should be determined whether there is a null data segment, and if so, switching from the first digital audio data and the first digital video data to the second audio data should not be performed, so that the output sound of the second audio data at this time is null.
In one embodiment, before the step S1 of acquiring digital data to be played by the digital television playing terminal, the method includes: s01, the digital signal sending end cuts the original second digital audio data to obtain a plurality of data segments; wherein the plurality of data segments use the plurality of switching nodes as boundaries;
s02, clustering the data segments to obtain an important cluster and an unimportant cluster; the insignificant cluster is not an empty set;
s03, replacing all data segments in the unimportant cluster by using empty data segments, so as to convert the original second digital audio data into final second digital audio data;
s04, taking the first digital video data, the first digital audio data and the final second digital audio data as digital data to be played;
and S05, sending the digital data to be played to the digital television playing terminal.
Thereby reducing the amount of data transmission. The clustering process is performed to classify the programs with low influence into one category and to mark the classified programs as unimportant clustering, and to classify the programs with high influence into another category and to mark the classified programs as important clustering. The clustering can be carried out in any feasible mode, for example, in a manual clustering mode, and because the clustering is carried out at the digital signal transmitting end, operators corresponding to the digital signal transmitting end have enough time to carry out manual clustering, so that the manual clustering mode is feasible. Further, a computer may also be used for clustering, for example, a prediction model trained based on a deep neural network model is used to determine whether a certain data segment has a higher influence on the program scenario, and if so, the data segment is classified as an important cluster, and if so, the data segment is classified as an unimportant cluster.
And then, carrying out replacement processing on the empty data segment, namely reducing the data volume of the transmission data, and sending the reduced data to a digital television playing terminal as digital data to be played.
Further, after the step S3 of extracting first digital audio data from the digital data to be played and playing audio by using a preset sound player according to the first digital audio data, the method includes:
s31, judging whether the first digital video data or the first digital audio data played at the current time is at a switching node;
s32, if the first digital video data or the first digital audio data played at the current time is at a switching node, judging whether a data segment in second digital audio data after the switching node is an empty data segment;
s33, if the data segment in the second digital audio data behind the switching node is not a null data segment, acquiring an image of a viewer in front of the display by using a preset camera to obtain a viewer image, and judging whether the line of sight of the viewer looks directly at the display according to the viewer image;
s34, if the line of sight of the viewer directly looks at the display, stopping video playing on the display and stopping audio playing on the sound player;
and S35, playing the audio by using the sound player according to the second digital audio data.
In the foregoing, it is mentioned that it is required to keep the camera in the normally open state in the implementation process, but in the present embodiment, the camera may not be kept in the normally open state. At this time, since the second digital audio data includes a null data segment, even if a certain switching node is reached, the data switching process may not be performed because the second digital audio data at this time is null. Therefore, in the present embodiment, it is determined whether the switching node is located first, and then it is determined whether the data segment is empty, and based on this, image acquisition is performed, and it is determined whether the viewer looks directly at the display. Therefore, the camera does not need to be kept in a normally open state.
Further, after the step S32 of determining whether the data segment in the second digital audio data after the switching node is an empty data segment if the first digital video data or the first digital audio data played at the current time is at the switching node, the method includes:
s321, if the data segment in the second digital audio data after the node is switched is an empty data segment, maintaining the current video playing and audio playing, and not turning on the camera.
The working time of the camera is reduced. Because of the existence of the empty data segment, even if the second digital audio data is switched to, the sound cannot be output, so if the camera is still started according to the original process and images are collected, then the analysis of the sight line of the viewer becomes meaningless, and accordingly the camera is not started.
In one embodiment, after the step S5 of determining whether the first digital video data or the first digital audio data played at the current time is at the node switching point if the viewer looks directly at the display, the method includes:
s501, if the first digital video data or the first digital audio data played at the current time are not located at a switching node, maintaining the current video playing and audio playing until the first digital video data or the first digital audio data are located at the switching node;
s502, stopping video playing on the display and stopping audio playing on the sound player; then according to the second digital audio data, playing audio by using a sound player;
s503, continuously carrying out viewer image acquisition processing, viewer sight line judgment processing and data switching processing so that when the viewer sight line is directly viewed on the display, a video corresponding to the first digital video data is played on the display, and an audio corresponding to the first digital audio data is played by using a sound player; playing audio corresponding to the second digital audio data using a sound player when a viewer does not look directly at the display; wherein, the playing time sequence is still kept before and after the data switching processing.
Therefore, data switching is only carried out at the switching node, and the continuity of digital television program playing is improved. In fact, the above operation is that when the viewer's line of sight is determined to be away from the display, the data switching is performed at the switching node, and only at the switching node. Of course, if it is determined through image acquisition that the line of sight of the viewer returns to the display before data switching, the data switching operation is not performed any more.
According to the digital television playing method based on audio and video switching, digital data to be played are obtained; extracting first digital video data, and playing a video according to the first digital video data; extracting first digital audio data, and playing audio according to the first digital audio data; carrying out image acquisition processing on a viewer in front of the display to obtain a viewer image, and judging whether the sight of the viewer looks directly at the display or not; if the line of sight of the viewer is directly looking at the display, judging whether the display is positioned at a switching node; if the node is switched, stopping video playing on the display and simultaneously stopping audio playing on the sound player; then according to the second digital audio data, playing audio by using a sound player; the method and the device continuously perform the image acquisition processing, the sight line judgment processing and the data switching processing of the viewer, solve the problem that the video content played by the display cannot be received by the viewer when the sight line of the viewer does not look directly at the display, and still keep a smooth playing state in the process.
Referring to fig. 2, an embodiment of the present application provides a digital television playing device based on audio/video switching, including:
a to-be-played digital data obtaining unit 10, configured to instruct a digital television playing terminal to obtain to-be-played digital data; the digital data to be played comprises first digital video data, first digital audio data and second digital audio data, wherein the first digital video data, the first digital audio data and the second digital audio data have the same playing time sequence, and a plurality of switching nodes are marked in the playing time sequence;
a first digital video data extraction unit 20, configured to instruct to extract first digital video data from the digital data to be played, and play a video on a preset display according to the first digital video data;
a first digital audio data extraction unit 30, configured to instruct to extract first digital audio data from the digital data to be played, and play audio using a preset sound player according to the first digital audio data;
the viewer image acquisition unit 40 is configured to instruct a preset camera to perform image acquisition processing on a viewer in front of the display to obtain a viewer image, and determine whether the line of sight of the viewer looks directly at the display according to the viewer image;
a switching node determination unit 50, configured to instruct, if a line of sight of a viewer looks directly at the display, to determine whether the first digital video data or the first digital audio data played at the current time is at a switching node;
a second digital audio data playing unit 60, configured to instruct that, if the first digital video data or the first digital audio data played at the current time is at a switching node, video playing on the display is stopped, and audio playing on a sound player is stopped at the same time; then according to the second digital audio data, playing audio by using a sound player;
a continuous image capturing unit 70 configured to instruct continuous viewer image capturing processing, viewer gaze determination processing, and data switching processing to be performed so that, when the viewer gazes directly at the display, a video corresponding to the first digital video data is played on the display, and an audio corresponding to the first digital audio data is played using a sound player; playing audio corresponding to the second digital audio data using a sound player when a viewer does not look directly at the display; wherein, the playing time sequence is still kept before and after the data switching processing.
The operations respectively executed by the above units correspond to the steps of the digital television playing method based on audio/video switching in the foregoing embodiment one to one, and are not described herein again.
The digital television playing device based on audio and video switching acquires digital data to be played; extracting first digital video data, and playing a video according to the first digital video data; extracting first digital audio data, and playing audio according to the first digital audio data; carrying out image acquisition processing on a viewer in front of the display to obtain a viewer image, and judging whether the sight of the viewer looks directly at the display or not; if the line of sight of the viewer is directly looking at the display, judging whether the display is positioned at a switching node; if the node is switched, stopping video playing on the display and simultaneously stopping audio playing on the sound player; then according to the second digital audio data, playing audio by using a sound player; the method and the device continuously perform the image acquisition processing, the sight line judgment processing and the data switching processing of the viewer, solve the problem that the video content played by the display cannot be received by the viewer when the sight line of the viewer does not look directly at the display, and still keep a smooth playing state in the process.
Referring to fig. 3, an embodiment of the present invention further provides a computer device, where the computer device may be a server, and an internal structure of the computer device may be as shown in the figure. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the computer designed processor is used to provide computational and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer equipment is used for storing data used by the digital television playing method based on audio and video switching. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to realize a digital television playing method based on audio/video switching. The computer device further comprises a display screen and an input device for displaying the human interactive interface and for receiving input data, respectively.
The processor executes the digital television playing method based on audio and video switching, wherein the steps of the method are respectively in one-to-one correspondence with the steps of executing the digital television playing method based on audio and video switching of the foregoing embodiment, and are not described herein again.
It will be understood by those skilled in the art that the structures shown in the drawings are only block diagrams of some of the structures associated with the embodiments of the present application and do not constitute a limitation on the computer apparatus to which the embodiments of the present application may be applied.
The computer equipment acquires digital data to be played; extracting first digital video data, and playing a video according to the first digital video data; extracting first digital audio data, and playing audio according to the first digital audio data; carrying out image acquisition processing on a viewer in front of the display to obtain a viewer image, and judging whether the sight of the viewer looks directly at the display or not; if the line of sight of the viewer is directly looking at the display, judging whether the display is positioned at a switching node; if the node is switched, stopping video playing on the display and simultaneously stopping audio playing on the sound player; then according to the second digital audio data, playing audio by using a sound player; the method and the device continuously perform the image acquisition processing, the sight line judgment processing and the data switching processing of the viewer, solve the problem that the video content played by the display cannot be received by the viewer when the sight line of the viewer does not look directly at the display, and still keep a smooth playing state in the process.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.
Claims (8)
1. A digital television playing method based on audio and video switching is characterized by comprising the following steps:
the digital television playing terminal acquires digital data to be played; the digital data to be played comprises first digital video data, first digital audio data and second digital audio data, wherein the first digital video data, the first digital audio data and the second digital audio data have the same playing time sequence, and a plurality of switching nodes are marked in the playing time sequence;
extracting first digital video data from the digital data to be played, and playing a video on a preset display according to the first digital video data; a plurality of viewers are in front of the display;
extracting first digital audio data from the digital data to be played, and playing audio by using a preset sound player according to the first digital audio data;
adopting a preset camera to acquire and process an image of a viewer in front of the display to obtain an image of the viewer;
judging whether the sight lines of all the viewers are directly looking at the display or not according to the viewer images;
if the sight lines of all the viewers are not in direct view of the display, acquiring a first position of the viewer who is in direct view of the display, and acquiring a second position of the viewer who is not in direct view of the display;
judging whether the first digital video data or the first digital audio data played at the current time is at a switching node;
if the first digital video data or the first digital audio data played at the current time is at a switching node, maintaining the video playing on the display and simultaneously stopping the audio playing on the sound player;
sending a first directional sound to the first position only by adopting a preset directional sound generator according to the first digital audio data;
and sending a second directional sound only to the second position by adopting a preset directional sound generator according to the second digital audio data.
2. The digital TV playing method based on audio-video switching as claimed in claim 1,
the first digital video data is obtained by carrying out video acquisition on a digital television program scene;
the first digital audio data corresponds to audio data of a speech of the first digital video data;
the second digital audio data is audio data for describing the plot of the digital television program by adopting voice;
and in the same time window, the first digital video data, the first digital audio data and the second digital audio data all correspond to the same program plot.
3. The method of claim 1, wherein the second digital audio data is composed of a plurality of sub-data segments, the sub-data segments are demarcated by the plurality of switching nodes, and at least one of the sub-data segments is a null data segment.
4. The digital television playing method based on audio/video switching according to claim 1, wherein before the digital television playing terminal acquires the digital data to be played, the method comprises the following steps: the digital signal sending end cuts the original second digital audio data to obtain a plurality of data segments; wherein the plurality of data segments use the plurality of switching nodes as boundaries;
clustering the plurality of data segments to obtain an important cluster and an unimportant cluster; the insignificant cluster is not an empty set;
replacing all data segments in the unimportant cluster by using empty data segments, so as to convert the original second digital audio data into final second digital audio data;
the first digital video data, the first digital audio data and the final second digital audio data are jointly used as digital data to be played;
and sending the digital data to be played to the digital television playing terminal.
5. The digital television playing method based on audio/video switching according to claim 4, wherein the image acquisition processing is performed on the viewer in front of the display by using a preset camera to obtain the viewer image, and the method comprises the following steps:
judging whether the first digital video data or the first digital audio data played at the current time is at a switching node;
if the first digital video data or the first digital audio data played at the current time is at a switching node, judging whether a data segment in the final second digital audio data after the switching node is an empty data segment;
and if the data segment in the final second digital audio data after the switching node is not the empty data segment, adopting a preset camera to acquire an image of a viewer in front of the display so as to obtain an image of the viewer.
6. The method for playing the digital television based on audio/video switching according to claim 5, wherein if the first digital video data or the first digital audio data played at the current time is at the switching node, determining whether a data segment in the final second digital audio data after the switching node is an empty data segment, includes:
and if the data segment in the second digital audio data behind the switching node is an empty data segment, maintaining the current video playing and audio playing and not starting the camera.
7. A digital television playing device based on audio and video switching is characterized by comprising:
a to-be-played digital data acquisition unit, configured to instruct a digital television playing terminal to acquire to-be-played digital data; the digital data to be played comprises first digital video data, first digital audio data and second digital audio data, wherein the first digital video data, the first digital audio data and the second digital audio data have the same playing time sequence, and a plurality of switching nodes are marked in the playing time sequence;
the first digital video data extraction unit is used for indicating that first digital video data are extracted from the digital data to be played and playing a video on a preset display according to the first digital video data; a plurality of viewers are in front of the display;
the first digital audio data extraction unit is used for indicating that first digital audio data are extracted from the digital data to be played and playing audio by using a preset sound player according to the first digital audio data;
the extracting of the first digital video data from the digital data to be played, and after playing a video on a preset display according to the first digital video data, and extracting the first digital audio data from the digital data to be played, and after playing an audio using a preset sound player according to the first digital audio data, includes:
adopting a preset camera to acquire and process an image of a viewer in front of the display to obtain an image of the viewer;
judging whether the sight lines of all the viewers are directly looking at the display or not according to the viewer images;
if the sight lines of all the viewers are not in direct view of the display, acquiring a first position of the viewer who is in direct view of the display, and acquiring a second position of the viewer who is not in direct view of the display;
judging whether the first digital video data or the first digital audio data played at the current time is at a switching node;
if the first digital video data or the first digital audio data played at the current time is at a switching node, maintaining the video playing on the display and simultaneously stopping the audio playing on the sound player;
sending a first directional sound to the first position only by adopting a preset directional sound generator according to the first digital audio data;
and sending a second directional sound only to the second position by adopting a preset directional sound generator according to the second digital audio data.
8. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111184902.1A CN113630650B (en) | 2021-10-12 | 2021-10-12 | Digital television playing method and device based on audio and video switching and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111184902.1A CN113630650B (en) | 2021-10-12 | 2021-10-12 | Digital television playing method and device based on audio and video switching and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113630650A CN113630650A (en) | 2021-11-09 |
| CN113630650B true CN113630650B (en) | 2022-08-09 |
Family
ID=78390992
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111184902.1A Active CN113630650B (en) | 2021-10-12 | 2021-10-12 | Digital television playing method and device based on audio and video switching and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113630650B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114531557B (en) * | 2022-01-25 | 2024-03-29 | 深圳佳力拓科技有限公司 | Digital television signal acquisition method and device based on mixed data packet |
| CN117560538B (en) * | 2024-01-12 | 2024-03-22 | 江西微博科技有限公司 | Service method of interactive voice video based on cloud platform |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101984648A (en) * | 2010-11-02 | 2011-03-09 | 中兴通讯股份有限公司 | Audio video file playing method and terminal thereof |
| CN102186038A (en) * | 2011-05-17 | 2011-09-14 | 浪潮(山东)电子信息有限公司 | Method for synchronously playing multi-viewing-angle pictures on digital television screen |
| CN102263990A (en) * | 2011-07-29 | 2011-11-30 | 宇龙计算机通信科技(深圳)有限公司 | Method, device and system for playing digital TV programs |
| CN104808946A (en) * | 2015-04-29 | 2015-07-29 | 天脉聚源(北京)传媒科技有限公司 | Image playing and controlling method and device |
| CN107223337A (en) * | 2017-04-01 | 2017-09-29 | 深圳市智晟达科技有限公司 | The method and DTV of a kind of automatic pause video playback |
| CN107371058A (en) * | 2017-08-04 | 2017-11-21 | 深圳市创维软件有限公司 | A kind of player method, smart machine and the storage medium of multimedia file sound intermediate frequency data |
| CN107423020A (en) * | 2017-08-22 | 2017-12-01 | 京东方科技集团股份有限公司 | Player method and play system |
| CN108282687A (en) * | 2017-12-15 | 2018-07-13 | 北京歌华有线电视网络股份有限公司 | Digital television playing method and set top box |
| CN109168086A (en) * | 2018-09-04 | 2019-01-08 | 黔东南民族职业技术学院 | A kind of control method of video playback apparatus, system and readable storage medium storing program for executing |
| CN109788350A (en) * | 2019-01-18 | 2019-05-21 | 北京睿峰文化发展有限公司 | It is a kind of that the seamless method and apparatus continuously played are selected based on video display plot |
| CN111580678A (en) * | 2020-05-26 | 2020-08-25 | 京东方科技集团股份有限公司 | Audio and video playing system, playing method and playing device |
| CN113438548A (en) * | 2021-08-30 | 2021-09-24 | 深圳佳力拓科技有限公司 | Digital television display method and device based on video data packet and audio data packet |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4359720B2 (en) * | 2006-05-12 | 2009-11-04 | 株式会社カシオ日立モバイルコミュニケーションズ | Video / audio playback device |
-
2021
- 2021-10-12 CN CN202111184902.1A patent/CN113630650B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101984648A (en) * | 2010-11-02 | 2011-03-09 | 中兴通讯股份有限公司 | Audio video file playing method and terminal thereof |
| CN102186038A (en) * | 2011-05-17 | 2011-09-14 | 浪潮(山东)电子信息有限公司 | Method for synchronously playing multi-viewing-angle pictures on digital television screen |
| CN102263990A (en) * | 2011-07-29 | 2011-11-30 | 宇龙计算机通信科技(深圳)有限公司 | Method, device and system for playing digital TV programs |
| CN104808946A (en) * | 2015-04-29 | 2015-07-29 | 天脉聚源(北京)传媒科技有限公司 | Image playing and controlling method and device |
| CN107223337A (en) * | 2017-04-01 | 2017-09-29 | 深圳市智晟达科技有限公司 | The method and DTV of a kind of automatic pause video playback |
| CN107371058A (en) * | 2017-08-04 | 2017-11-21 | 深圳市创维软件有限公司 | A kind of player method, smart machine and the storage medium of multimedia file sound intermediate frequency data |
| CN107423020A (en) * | 2017-08-22 | 2017-12-01 | 京东方科技集团股份有限公司 | Player method and play system |
| CN108282687A (en) * | 2017-12-15 | 2018-07-13 | 北京歌华有线电视网络股份有限公司 | Digital television playing method and set top box |
| CN109168086A (en) * | 2018-09-04 | 2019-01-08 | 黔东南民族职业技术学院 | A kind of control method of video playback apparatus, system and readable storage medium storing program for executing |
| CN109788350A (en) * | 2019-01-18 | 2019-05-21 | 北京睿峰文化发展有限公司 | It is a kind of that the seamless method and apparatus continuously played are selected based on video display plot |
| CN111580678A (en) * | 2020-05-26 | 2020-08-25 | 京东方科技集团股份有限公司 | Audio and video playing system, playing method and playing device |
| CN113438548A (en) * | 2021-08-30 | 2021-09-24 | 深圳佳力拓科技有限公司 | Digital television display method and device based on video data packet and audio data packet |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113630650A (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113630650B (en) | Digital television playing method and device based on audio and video switching and computer equipment | |
| CN107316520B (en) | Video teaching interaction method, device, equipment and storage medium | |
| US20080235724A1 (en) | Face Annotation In Streaming Video | |
| CN111083397B (en) | Recorded broadcast picture switching method, system, readable storage medium and equipment | |
| EP2894852A1 (en) | Process for increasing the quality of experience for users that watch on their terminals a high definition video stream | |
| CN109698949B (en) | Video processing method, device and system based on virtual reality scene | |
| JP5316286B2 (en) | Video conference system, server device, and video conference program | |
| CN111405339B (en) | Split screen display method, electronic equipment and storage medium | |
| EP2665290A1 (en) | Simultaneous display of a reference video and the corresponding video capturing the viewer/sportsperson in front of said video display | |
| CN111757137A (en) | Multi-channel close-up playing method and device based on single-shot live video | |
| WO2023279793A1 (en) | Video playing method and apparatus | |
| CN107948737A (en) | The recommendation method and device of TV programme | |
| CN106162357A (en) | Obtain the method and device of video content | |
| KR101900471B1 (en) | Broadcasting system inserted user reaction effect | |
| CN114339302B (en) | Method, device, equipment and computer storage medium for guiding broadcast | |
| CN111757138A (en) | Close-up display method and device based on single-shot live video | |
| CN110933350A (en) | Electronic cloud mirror recording and broadcasting system, method and device | |
| CN108320331B (en) | Method and equipment for generating augmented reality video information of user scene | |
| CN109391769A (en) | Control equipment, control method and storage medium | |
| US20040054721A1 (en) | Visual media viewing system and method | |
| JP3759216B2 (en) | Television camera communication device and multipoint connection device | |
| CN113573151B (en) | Digital television playing method and device based on focusing degree value | |
| CN108024121B (en) | Voice barrage synchronization method and system | |
| US11908340B2 (en) | Magnification enhancement of video for visually impaired viewers | |
| KR102301076B1 (en) | Apparatus for broadcast contents process and control method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |