CN111354384A - GIF picture generation method, playing method and device and electronic equipment - Google Patents

GIF picture generation method, playing method and device and electronic equipment Download PDF

Info

Publication number
CN111354384A
CN111354384A CN202010123020.3A CN202010123020A CN111354384A CN 111354384 A CN111354384 A CN 111354384A CN 202010123020 A CN202010123020 A CN 202010123020A CN 111354384 A CN111354384 A CN 111354384A
Authority
CN
China
Prior art keywords
gif
audio
picture
data
playing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010123020.3A
Other languages
Chinese (zh)
Inventor
张海兵
吴乐宝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing QIYI Century Science and Technology Co Ltd
Original Assignee
Beijing QIYI Century Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing QIYI Century Science and Technology Co Ltd filed Critical Beijing QIYI Century Science and Technology Co Ltd
Priority to CN202010123020.3A priority Critical patent/CN111354384A/en
Publication of CN111354384A publication Critical patent/CN111354384A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/036—Insert-editing
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10—Digital recording or reproducing
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10—Digital recording or reproducing
    • G11B2020/10935—Digital recording or reproducing wherein a time constraint must be met
    • G11B2020/10953—Concurrent recording or playback of different streams or files

Abstract

The embodiment of the invention provides a GIF picture generation method, a GIF picture playing device and electronic equipment. And writing audio data at the tail of the GIF data in the target GIF picture, adding an audio identifier to generate the GIF picture with the audio, identifying the GIF picture with the audio according to the audio identifier in the playing process, separating the GIF data and the audio data carried by the GIF picture, and synchronously playing after respectively analyzing. And the carried audio data is written in the GIF picture, so that the GIF picture can play sound while displaying the picture sequence, the playing mode of the dynamic GIF picture is enriched, and the playing effect of the dynamic GIF picture is improved.

Description

GIF picture generation method, playing method and device and electronic equipment
Technical Field
The invention relates to the technical field of images, in particular to a GIF image generation method, a GIF image playing method, a GIF image generating device and an electronic device.
Background
The GIF is generally called Graphics Interchange Format, which can be translated into a graphic Interchange Format for displaying index color images in an HTML (Hypertext Markup Language) manner, and can be specifically classified into static GIF and dynamic GIF.
The standard GIF picture is in a picture format and can only show a picture animation effect.
Therefore, the problems of single playing mode and poor playing effect of the GIF picture exist in the prior art.
Disclosure of Invention
The embodiment of the invention aims to provide a GIF picture generation method, a GIF picture playing device and electronic equipment, and aims to solve the problems that in the prior art, the GIF picture is single in playing mode and poor in playing effect. The specific technical scheme is as follows:
in a first aspect of the present invention, a GIF picture generating method is provided, including:
acquiring audio data in an audio file corresponding to the target GIF picture;
writing the audio data at the tail of the GIF data in the target GIF picture, and adding an audio identifier at the tail of the audio data or between the GIF data and the audio data to generate a GIF picture with audio;
wherein the audio identifier is used to identify the GIF picture with audio.
In a second aspect of the present invention, a GIF picture playing method is further provided, where the GIF picture is generated according to the method provided in the first aspect of the present invention, and includes:
searching an audio identifier, and identifying the GIF picture as the GIF picture with audio;
separating GIF data and audio data carried by the GIF picture;
analyzing the GIF data to obtain a GIF frame corresponding to the GIF picture, and analyzing the audio data to obtain an audio frame corresponding to the GIF picture;
and synchronously playing the GIF frame and the audio frame.
In a third aspect of the present invention, there is further provided a GIF picture generating apparatus, including:
the acquisition module is used for acquiring audio data in the audio file corresponding to the target GIF picture;
a generating module, configured to write the audio data at the end of the GIF data in the target GIF picture, and add an audio identifier at the end of the audio data, or between the GIF data and the audio data, to generate a GIF picture with audio;
wherein the audio identifier is used to identify the GIF picture with audio.
In a fourth aspect of the present invention, there is further provided a GIF picture playing apparatus, where the GIF picture is generated according to the method provided in the first aspect of the present invention, and the method includes:
an identification module for searching the audio identifier and identifying the GIF picture as the GIF picture with audio
The separation module is used for separating the GIF data and the audio data carried by the GIF picture according to the audio identifier;
the analysis module is used for analyzing the GIF data to obtain a GIF frame corresponding to the GIF picture and analyzing the audio data to obtain an audio frame corresponding to the GIF picture;
and the playing module is used for synchronously playing the GIF frame and the audio frame.
In a fifth aspect of the present invention, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
a processor, configured to implement the method steps of the first aspect of the embodiment of the present invention or the second aspect of the embodiment of the present invention when executing the program stored in the memory.
In yet another aspect of the present invention, there is also provided a computer-readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform any one of the above-mentioned GIF picture generating method or GIF picture playing method.
In yet another aspect of the present invention, there is also provided a computer program product including instructions, which when run on a computer, causes the computer to execute any one of the above-mentioned GIF picture generating methods or GIF picture playing methods.
In the GIF picture generation method, the GIF picture playing device and the electronic equipment, the audio data are written in the tail end of the GIF data in the target GIF picture, the audio identifier is added to generate the GIF picture with the audio, the GIF picture with the audio is identified according to the audio identifier in the playing process of the GIF picture, the GIF data and the audio data carried by the GIF picture are separated, and the GIF picture and the audio data are synchronously played after being respectively analyzed. And the carried audio data is written in the GIF picture, so that the GIF picture can play sound while displaying the picture sequence, the playing mode of the dynamic GIF picture is enriched, and the playing effect of the dynamic GIF picture is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a flowchart of a GIF picture generation method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a GIF picture playing method according to an embodiment of the present invention;
FIG. 3 is a second flowchart illustrating a GIF picture playing method according to an embodiment of the present invention;
FIG. 4 is a third flowchart illustrating a GIF picture playing method according to an embodiment of the present invention;
FIG. 5 is a block diagram of a GIF picture generation apparatus according to an embodiment of the present invention;
FIG. 6 is a block diagram of a GIF picture playing apparatus according to an embodiment of the present invention;
fig. 7 is a block diagram of an electronic device in an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.
Example one
Referring to fig. 1, fig. 1 is a flowchart of a GIF picture generating method according to an embodiment of the present invention, where the method is applied to a GIF picture generating device, and the GIF picture generating device may be software specifically used for generating a GIF picture with audio.
Step 101, acquiring audio data in an audio file corresponding to a target GIF picture;
step 102, writing the audio data at the tail of the GIF data in the target GIF picture, and adding an audio identifier at the tail of the audio data or between the GIF data and the audio data to generate a GIF picture with audio;
wherein the audio identifier is used to identify the GIF picture with audio.
The GIF picture is divided into two types, a static GIF picture and a dynamic GIF picture. The dynamic GIF picture is composed of continuous picture sequences, only contains picture data and does not contain audio data, so that the dynamic GIF picture only can show the dynamic effect formed by the picture sequences but cannot play sound, and the playing mode is single.
In the embodiment of the invention, in order to enable the GIF picture to play sound, the audio data of the corresponding audio file is written in the GIF picture, and the audio identifier is added to identify that the current GIF picture is the GIF picture with audio, the GIF picture with audio can be played by using a GIF picture playing device, and the GIF picture playing device can determine the corresponding playing method according to the generating method of the GIF picture with audio.
The audio identifier may be composed of one or more character strings, including an identifier for identifying the presence of audio data, which may be represented as "audiosz" or other forms, and is not limited herein.
In step 101, the audio file corresponding to the GIF picture may be an original audio file matched with the GIF picture, for example, the current GIF picture is a dynamic picture captured from a certain video, and the captured video clip itself has audio data; the audio file corresponding to the GIF picture may also be an audio file selected by another user and matched with the GIF picture, for example, audio data recorded by the user according to the GIF picture.
The user can import the GIF picture to which the audio data need to be added and the audio file corresponding to the GIF picture into the GIF picture generating device, and the GIF picture generating device reads the audio data from the audio file.
In step 102, audio data is written at the end of the GIF data in the GIF picture.
The GIF data in the GIF picture comprises a GIF file header, a picture block and an end block. The GIF file header is a data block with data stream of the identified GIF format, and the general-purpose picture viewing software can identify and read the GIF format according to the GIF file header. The GIF picture block includes GIF picture data and a picture control block, wherein the GIF picture data is composed of a binary character stream having a bit depth of 1-8 bits. The end block is an end flag region of the GIF picture, which marks the end of the entire GIF data stream in the GIF picture.
The audio data is written at the end of the GIF data in the GIF picture, that is, after the end block. On one hand, the GIF data in the audio GIF picture can be kept at the starting position of the file, and even if the audio GIF picture is imported into the general view software, the general view software can still identify the GIF data according to the GIF file header at the beginning of the file and read to the ending block to show the picture sequence in the ending block. On the other hand, writing the audio data after the end block preserves the integrity of the GIF data, facilitating the subsequent complete separation of the GIF data and the audio data.
In step 102, the audio identifier is used to identify the GIF picture with audio, and the audio identifier may be added at the end of the audio data, i.e. after the audio data, or between the GIF data and the audio data, i.e. before the audio data.
Adding an audio identifier at the end of the audio data can be understood as: after the GIF picture generation device acquires the target GIF picture, the GIF data in the target GIF picture are read first, when the reading pointer deviates to the tail of the file, namely the tail of the ending block, the data streams of the audio data are written in sequence, and after the audio data are written in, the audio identifier is added to the tail of the audio data.
Adding an audio identifier between GIF data and audio data can be understood as: after acquiring the target GIF picture, the GIF picture generating device firstly reads the GIF data in the target GIF picture, when the reading pointer deviates to the tail of the file, namely the tail of the ending block, firstly adds the audio identifier, and sequentially writes the data stream of the audio data at the tail of the audio identifier.
It should be noted that, the above-mentioned processes of writing audio data and adding an audio identifier may refer to the specific operations of reading and writing data in the prior art, and are not described herein again.
In the embodiment of the invention, the audio data of the corresponding audio file is written into the tail of the GIF data in the GIF picture, and the audio identifier is added to generate the GIF picture with the audio. On the premise of not damaging the integrity of GIF data in the GIF picture and not influencing the analysis and display of the GIF data by the general picture viewing software, the audio data is written into the GIF picture, so that the generated GIF picture can display the corresponding picture sequence and play the corresponding audio according to the written audio data, the playing form of the GIF picture is enriched, and the playing effect of the GIF picture is improved.
Optionally, the audio identifier includes a first identifier for identifying the size of the audio data, and the first identifier is used to separate audio data carried in the GIF picture with audio when the GIF picture with audio is played.
The audio identifier may include one or more identifiers for identifying other properties in addition to the identifier for identifying the presence of audio data. In this embodiment, the audio identifier includes a first identifier for identifying the size of the audio data, and the audio identifier including the first identifier may be added to the end of the audio data or may be added between the GIF data and the audio data.
Therein, the first identifier may consist of one or more character strings, which may be represented as "12345678" or other forms. The first identifier may be located after an identifier identifying the presence of audio data, such as "audiosz 123456789," or may be located before an identifier identifying the presence of audio data. It is to be understood that the implementation form of the first identifier is not limited thereto, and is not limited thereto.
When the subsequent GIF picture with the audio is analyzed and played, if the audio identifier comprises a first identifier for identifying the size of the audio data, the GIF picture playing device can acquire the size of the audio data while finding the audio identifier, and read the data with the length corresponding to the size of the audio data according to the position of the audio identifier so as to separate the audio data carried in the GIF picture with the audio.
Further, in some embodiments, the audio identifier comprises a first identifier and the audio identifier is added at the end of the audio data.
In the above embodiment, the audio identifier including the first identifier is added to the end of the audio data, and it is possible to set the GIF picture with audio generated in this way to read data from the end of the file when the GIF picture playing apparatus plays it. Therefore, the GIF picture playing device can quickly find the audio identifier to identify the current GIF picture with the audio, acquire the size of the audio data carried in the GIF picture, and improve the analysis efficiency of the GIF picture with the audio during playing.
In other embodiments, the audio identifier comprises the first identifier and the audio identifier is added between the GIF data and the audio data. In the above embodiment, it may be set that the data is read from the initial position of the file when the GIF picture playing device plays the file, or it may be set that the data is read from the initial position of the file when the GIF picture playing device plays the file.
It can be understood that the playing method of the GIF picture with audio and the generating method of the GIF picture with audio correspond to each other, and when the GIF picture with audio is generated, different playing methods can be correspondingly set for the GIF picture playing device according to different situations of the audio identifier.
In some embodiments, the audio identifier may also include a second identifier that identifies the location of the audio identifier, which may be located at the end of the audio identifier and may likewise consist of one or more character strings. For example, the second identifier of the audio identifier may be "01" when the audio identifier is added at the end of the audio data, and may be "02" when the audio identifier is added between the GIF data and the audio data. It is to be understood that the implementation form of the second identifier is not limited thereto, and is not limited thereto.
In summary, in the GIF picture generation method provided by the embodiment of the present invention, the audio data is written at the end of the GIF data in the target GIF picture, and the audio identifier is added to generate the GIF picture with the audio, during the playing process of the GIF picture, the GIF picture with the audio is identified according to the audio identifier, and the GIF data and the audio data carried by the GIF picture are separated and are synchronously played after being respectively analyzed. And the carried audio data is written in the GIF picture, so that the GIF picture can play sound while displaying the picture sequence, the playing mode of the dynamic GIF picture is enriched, and the playing effect of the dynamic GIF picture is improved.
Example two
Referring to fig. 2, fig. 2 is a flowchart of a GIF picture playing method according to an embodiment of the present invention, which can be applied to a GIF picture playing device, where the GIF picture playing device may be software dedicated to playing GIF pictures with audio.
The GIF picture is a GIF picture with audio, and is generated according to the GIF picture generation method provided by the embodiment of the invention, and the GIF picture with audio is written with audio data at the end of the GIF data and is added with an audio identifier.
As shown in fig. 2, the method comprises the steps of:
step 201, searching an audio identifier, and identifying the GIF picture as the GIF picture with audio
Step 202, separating GIF data and audio data carried by the GIF picture;
step 203, analyzing the GIF data to obtain a GIF frame corresponding to the GIF picture, and analyzing the audio data to obtain an audio frame corresponding to the GIF picture;
and step 204, synchronously playing the GIF frame and the audio frame.
In the embodiment of the invention, the GIF picture is a GIF picture with audio, and carries audio data and an audio identifier, and the audio identifier is used for identifying the GIF picture with the audio.
In step 201, the GIF picture playing apparatus may search for an audio identifier included in the GIF picture file to determine whether the current GIF picture file carries audio data, and start to perform a subsequent parsing and playing step.
In step 202, when playing the GIF picture with audio, the GIF data and the audio data carried by the GIF picture with audio need to be separated.
Since the audio GIF picture is generated by writing the audio data to the end of the GIF data, the GIF data remains at the start position of the audio GIF picture. The GIF data includes three parts, a GIF file header, a picture block, and an end block. The end block is an end flag region of the GIF picture, and marks the end of the entire GIF data stream in the GIF picture.
The GIF picture playing device starts to read data from the starting position of the GIF picture file with the audio, the reading pointer sequentially reads three parts of a GIF file header, a picture block and an ending block of the GIF data in the GIF picture with the audio, when the reading pointer deviates to the tail of the ending block of the GIF data, the fact that the reading of the GIF data is finished is marked, the read data are the GIF data in the GIF picture with the audio, and the GIF picture playing device can send the GIF data to a corresponding analysis module for analysis.
And then, the GIF picture playing device continuously reads the residual data in the file backwards, the residual data in the file comprises the audio data and the audio identifier carried in the GIF picture with the audio, and the GIF picture playing device can distinguish the audio data and the audio identifier in the residual data because the audio identifier is different from the binary data stream of the audio data, and can send the audio data and the audio identifier to a corresponding analysis module for analysis after only identifying the audio data therein:
when the audio identifier is added to the audio data, when the GIF picture playing device reads the residual data in the file, and when the audio identifier is read, the read data stream is the data stream corresponding to the audio data;
when the audio identifier is added between the GIF data and the audio data, namely before the audio data, when the GIF picture playing device reads the residual data in the file, the data stream corresponding to the audio identifier is read first, and then the data stream corresponding to the audio data can be read.
The GIF picture playing device can also sequentially read data from the tail of the GIF picture with the audio in a reverse order, when an ending block in the GIF data is read, the read data comprises the audio data and the audio identifier, and the GIF picture playing device can distinguish the audio data and the audio identifier in the read data because the audio identifier is different from a binary data stream of the audio data, and can send the audio data and the audio identifier to a corresponding analysis module for analysis after only identifying the audio data. And then, the GIF picture playing device continues to read the residual data in the file backwards, wherein the residual data are the GIF data in the GIF picture with the audio.
It can be understood that, according to the three data streams with identification properties, i.e., the GIF file header and the end block of the GIF data in the GIF picture with audio and the audio identifier, the GIF picture playing apparatus can respectively read the GIF data and the audio data carried in the GIF picture with audio and send the GIF data and the audio data to the corresponding parsing module for subsequent parsing.
In step 203, the separated GIF data and audio data are analyzed, respectively.
And analyzing the GIF data to obtain corresponding GIF frames, wherein the frame data of each GIF frame comprises frame separators, frame data description, dot matrix data stored with color list index values, time stamps, frame control extension blocks of each GIF frame and other related original GIF information. The frame control extension block of each GIF frame is used for determining the performance behavior of the current GIF frame, such as delay time, rendering mode, etc.
And analyzing the audio data to obtain corresponding audio frames, and analyzing to obtain the total frame number, the sampling frequency, the bit rate, the time stamp and other related audio sampling data corresponding to the audio data according to the specific audio coding mode.
It should be noted that, the analysis of the GIF data may be performed by referring to an analysis method in the prior art, and the analysis of the audio data may also be performed by referring to an analysis method in the prior art, which is not described herein again.
In step 204, the analyzed GIF data and audio data are respectively sent to corresponding playing modules for synchronous playing. The analyzed GIF frame data can be sent to a system display card to be converted, then the converted GIF frame data is input to display equipment such as a display to be displayed, and the analyzed audio frame data can be sent to a sound card to be converted, then the converted audio frame data is input to sound equipment such as an earphone and a loudspeaker to be played.
Synchronous playing of the GIF frame and the audio frame is a dynamic adjusting process, and in the playing process, the playing needs to be adjusted according to the same reference clock and timestamps corresponding to the GIF frame and the audio frame. Because the sensitivity of the human body to sound is stronger than the sensitivity to vision, in the embodiment of the invention, the playing of the GIF frame can be correspondingly adjusted according to the playing time of the audio frame as a reference clock. The current playing speed of the GIF frame is judged by comparing the time stamps between the audio frame and the GIF frame, and corresponding adjustment is carried out, so that synchronous playing of the GIF frame and the audio frame is realized, and the effect of playing the GIF picture is further improved.
In the embodiment of the invention, the audio GIF picture is taken as the audio GIF picture through the audio identifier, and then the carried GIF data and the audio data are separated, analyzed and synchronously played. On the premise of not damaging the integrity of GIF data in the GIF picture and not influencing the analysis and display of the GIF data by the general picture viewing software, the picture sequence in the GIF picture can be displayed, and the audio in the GIF picture can be played, so that the playing form of the GIF picture is enriched, and the playing effect of the GIF picture is improved.
EXAMPLE III
Referring to fig. 3, fig. 3 is a flowchart of another GIF picture playing method according to an embodiment of the present invention, which can be applied to a GIF picture playing device, where the GIF picture playing device can be software dedicated to playing GIF pictures with audio.
The GIF picture is a GIF picture with audio, and is generated according to the GIF picture generation method provided by the embodiment of the present invention, wherein the GIF picture with audio has audio data written at the end of the GIF data, and the audio identifier includes a first identifier for identifying the size of the audio data.
As shown in fig. 3, the method comprises the steps of:
step 301, searching an audio identifier, and identifying that the GIF picture is a GIF picture with audio;
step 302, determining the size of the audio data carried by the GIF picture according to the first identifier;
step 303, separating the audio data according to the position of the audio identifier and the size of the audio data;
step 304, analyzing the GIF data to obtain a GIF frame corresponding to the GIF picture, and analyzing the audio data to obtain an audio frame corresponding to the GIF picture;
step 305, synchronously playing the GIF frame and the audio frame.
In this embodiment, the GIF picture is a GIF picture with audio, and carries audio data and an audio identifier, where the audio identifier includes a first identifier for identifying the size of the audio data, and the first identifier is used to separate the GIF picture with audio from the GIF picture with audio when the GIF picture with audio is played.
The specific implementation form of step 301 may refer to the description of step 201 in the embodiment shown in fig. 2, and is not described herein again to avoid repetition.
In this embodiment, the audio identifier including the first identifier may be added at the end of the audio data, or may be added between the GIF data and the audio data, and the GIF picture playing apparatus may read the data stream in the forward order from the start position of the audio GIF picture, or may read the data stream in the reverse order from the end position of the audio GIF picture. When the GIF picture playing device reads the audio identifier, the size of the audio data in the GIF picture with audio, namely the data stream length of the audio data, can be determined according to the first identifier contained in the audio identifier, the data stream with the length corresponding to the size of the audio data is read in combination with the position of the audio identifier, and the audio data in the data stream can be separated.
Further, in some embodiments, an audio identifier is added at the end of the audio data.
In the above embodiment, when reading the GIF picture with audio, the GIF picture playing apparatus may shift the reading pointer to the end of the GIF picture with audio, and sequentially read the data in reverse order, so that the audio identifier may be quickly read to obtain the size of the audio data. After the GIF picture playing apparatus finishes reading the data stream length (here, set as a) corresponding to the audio identifier in the reverse order, the size of the audio data, that is, the data stream length (here, set as B) of the audio data, may be obtained according to the read first identifier. And then, continuously reading the data stream with the length corresponding to the B in a forward and backward sequence, wherein the data stream is the data stream of the audio data, and the data stream which is not read in the audio GIF picture is the GIF data.
In other embodiments, an audio identifier is added between the GIF data and the audio data.
In the above embodiment, the GIF picture generating device may sequentially read data from the start position of the audio GIF picture in the forward order to the audio identifier, or sequentially read data from the end of the audio GIF picture in the reverse order to the audio identifier, which is not limited herein.
After the GIF data and the audio data carried by the GIF picture with audio are successfully separated, the GIF data and the audio data may be respectively analyzed in step 304, and the specific implementation form of step 304 may refer to the description of step 203 in the embodiment shown in fig. 2, and is not described herein again in order to avoid repetition.
Optionally, step 305 includes:
calculating a difference value between the timestamp of the currently played GIF frame and the timestamp of the currently played audio frame;
if the difference value is larger than zero, calculating the remaining playing time length of the currently played audio frame;
continuously displaying the current GIF frame in the residual playing time length under the condition that the difference value is greater than the residual playing time length, and playing the next GIF frame while playing the next audio frame;
and if the difference is less than or equal to zero, directly broadcasting the next GIF frame.
The synchronous playing of the GIF frame and the audio frame is a dynamic adjustment process, and whether the sound and the picture are synchronous is judged frame by frame according to whether the content of the GIF frame currently played is matched with the content of the audio frame currently played, and the playing duration of the GIF frame and the audio frame is adjusted when the sound and the picture are not synchronous.
The playing duration of each GIF frame is controlled by a frame control extension block, the frame control extension block comprises delay time, the delay time refers to the display time of the current frame, the unit is 0.01 second, the display time is paused, the next GIF frame is continuously processed, for example, when the delay time is set to be 2, the next GIF frame is continuously processed after the display time is paused for 0.02 second. The presentation duration of each GIF frame can be controlled by assigning different values to the delay time in the frame control extension block.
The playing duration of each audio frame may be determined according to the following equation:
playing time per frame (millisecond) ═ number of samples per frame/sampling frequency 10000
In this embodiment, based on the fact that the human auditory sensitivity is stronger than the visual sensitivity, the playing time of the audio frame is used as the reference clock, and the playing of the GIF frame is correspondingly adjusted. Since the delay time of each GIF frame is different from the playing duration of each audio frame, it is necessary to determine whether the currently played GIF frame matches the currently played audio frame by comprehensively considering the offsets on the time stamps of the GIF frame and the audio frame and the remaining playing duration of the audio frame.
For simplicity of presentation, in describing this embodiment, G is usednRepresenting a currently played GIF frame; gn-t represents GnThe timestamp of (2) can also be understood as the current played GIF duration. Where n denotes the number of frames of the currently played GIF frame. A. themIndicating current playThe audio frame of (2); a. them-t represents AmThe timestamp of (2) can also be understood as the current played audio duration. Where m represents the number of frames of the currently played audio frame.
First, it is able to calculate the currently played GIF frame GnTime stamp G ofnT and the currently playing audio frame AmTime stamp A ofm-t, which can be understood as the offset of the currently played GIF duration from the played audio duration, and can be expressed in particular as:
ΔT＝Gn-t-Am-t
if Δ T > 0, i.e. Gn-t＞Am-t, represents GnHas a time stamp faster than AmTime stamp of (G), andncontent played with AmWhether the played contents are correspondingly synchronized or not needs to be further calculated AmThe remaining playing time of AmThe remaining play-out time of (a) can be expressed as:
Ama remaining playing time ofmIs played for a time length-amHas long broadcast time
G in case of Δ T > remaining playing time periodnThe played content exceeds AmOf the remaining play-out duration range, GnContent played faster than AmThe played contents are not correspondingly synchronized, and G needs to be adjustednThe playing of (3). After G has been playednIn the case of (1), G is required to benContinuously display AmThe remaining playing time of the next audio frame Am+1While playing the next GIF frame Gn+1. In particular by reaction of GnIs adjusted to the above AmThe remaining play-out time length. For example, AmIs 0.02 second, G is setnIs assigned a delay time of 2, GnAfter the pause is displayed for 0.02 second, the next audio frame A is playedm+1While playing the next GIF frame Gn+1。
In the case where Δ T ≦ remaining play-out duration, even GnAnd AmThe current playing time point does not completely correspond to the current playing time point, but GnThe played content is still at AmWithin the remaining play-out time range of GnContent played with AmThe played contents are correspondingly synchronous, and the played contents displayed in the sound are judged to be matched from the perspective of user perception. In this case, the pair G is not requirednAnd adjusting and normally playing.
If Δ T < 0, i.e. Gn-t＜Am-t, represents GnIs slower than AmThe timestamp of (1) can be directly relayed to next GIF frame Gn+1To catch up with the progress of the audio frame playing, if the next GIF frame G is skippedn+1Then, the situation that the delta T is larger than 0 occurs, and the specific adjustment is carried out according to the adjustment scheme when the delta T is larger than 0.
If the difference Δ T is equal to 0, i.e. Gn-t＝AmT, indicating that the current GIF frame is synchronized with the audio frame play, and the next GIF frame can be directly relayed. If skip to next GIF frame Gn+1Then, the situation that the delta T is larger than 0 occurs, and the specific adjustment is carried out according to the adjustment scheme when the delta T is larger than 0.
In this embodiment, the playing time of the audio frame is used as a reference clock, whether the sound and the picture are synchronous is judged according to the time difference between the timestamp of the currently played GIF frame and the timestamp of the currently played audio frame and the remaining playing duration of the audio frame, and the playing of the GIF frame is adjusted to realize the synchronous playing of the GIF frame and the audio frame.
It should be noted that the technical solutions in the above embodiments are also applicable to the embodiment shown in fig. 2, and have the same beneficial effects, and are not described herein again to avoid repetition.
A complete embodiment of the invention is described below, as shown in fig. 4:
after receiving the GIF picture to be played, the GIF picture playing device firstly reads the file, searches whether an audio identifier exists in the current GIF picture, and identifies the current GIF picture as the GIF picture with the audio according to the audio identifier.
In this embodiment, the audio identifier includes a first identifier that identifies the size of the audio data. And after the GIF picture playing device reads the audio identifier, the size of the audio data can be obtained, and then the GIF data and the audio data carried in the audio GIF picture are separated.
Respectively sending the GIF data and the audio data to corresponding analysis modules for respective analysis, wherein the GIF data are analyzed to obtain corresponding GIF frames and related control information; and audio data is subjected to audio decoding to obtain corresponding audio frames.
And sending the decoded GIF frame to a display card of the GIF picture playing device for playing, sending the decoded audio frame to a sound card of the GIF picture playing device for playing, and controlling and adjusting the playing of the GIF frame by taking the playing time of the audio frame as a reference clock so as to realize synchronous playing of the GIF frame and the audio frame.
In this embodiment, on the basis of the second embodiment, multiple implementation forms are added, and the playing effect of the dynamic GIF picture is further improved.
Referring to fig. 5, fig. 5 is a structural diagram of a GIF picture generating device according to an embodiment of the present invention.
As shown in fig. 5, the GIF picture generating apparatus 500 includes:
an obtaining module 501, configured to obtain audio data in an audio file corresponding to the target GIF picture;
a generating module 502, configured to write the audio data at the end of the GIF data in the target GIF picture, and add an audio identifier at the end of the audio data, or between the GIF data and the audio data, to generate a GIF picture with audio;
wherein the audio identifier is used to identify the GIF picture with audio.
Optionally, the audio identifier includes a first identifier for identifying the size of the audio data, and the first identifier is used to separate audio data carried in the GIF picture with audio when the GIF picture with audio is played.
The GIF picture generating apparatus 500 can implement the processes implemented in the method embodiment of fig. 1, and is not described herein again to avoid repetition.
Referring to fig. 6, fig. 6 is a structural diagram of a GIF picture playing device according to an embodiment of the present invention.
As shown in fig. 6, the GIF picture playing apparatus 600 includes:
an identifying module 601, configured to search for an audio identifier, and identify the GIF picture as a GIF picture with audio
A separation module 602, configured to separate GIF data and audio data carried by the GIF picture according to an audio identifier;
the analyzing module 603 is configured to analyze the GIF data to obtain a GIF frame corresponding to the GIF picture, and analyze the audio data to obtain an audio frame corresponding to the GIF picture;
a playing module 604, configured to play the GIF frame and the audio frame synchronously.
Optionally, the audio identifier comprises a first identifier identifying a size of the audio data;
the separation module 602 includes:
a determining unit, configured to determine, according to the first identifier, a size of audio data carried by the GIF picture;
a separating unit for separating the GIF data and the audio data according to the size of the GIF data and the size of the audio data.
Optionally, the playing module 604 includes:
a first calculating unit for calculating a difference between a time stamp of the currently played GIF frame and a time stamp of the currently played audio frame;
the second calculating unit is used for calculating the remaining playing time of the currently played audio frame if the difference value is greater than zero;
the first playing unit is used for continuously displaying the currently played GIF frame in the residual playing time length under the condition that the difference value is larger than the residual playing time length, and playing the next GIF frame while playing the next audio frame;
and a second playing unit, configured to play the next GIF frame if the difference is less than or equal to zero.
The GIF picture generating device 600 can implement each process implemented in the method embodiments shown in fig. 2 to fig. 3, and is not described herein again to avoid repetition.
Referring to fig. 7, fig. 7 is a structural diagram of an electronic device according to an embodiment of the invention.
As shown in fig. 7, the electronic device 700 includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702 and the memory 703 complete communication with each other through the communication bus 704.
The memory 703 is used for storing a computer program.
When the processor 701 is configured to execute the program stored in the memory 703, the following steps are implemented:
acquiring audio data in an audio file corresponding to the target GIF picture;
writing the audio data at the tail of the GIF data in the target GIF picture, and adding an audio identifier at the tail of the audio data or between the GIF data and the audio data to generate a GIF picture with audio;
wherein the audio identifier is used to identify the GIF picture with audio.
Optionally, the audio identifier includes a first identifier for identifying the size of the audio data, and the first identifier is used to separate audio data carried in the GIF picture with audio when the GIF picture with audio is played.
Alternatively, the processor 701 further implements the steps of:
searching an audio identifier, and identifying the GIF picture as the GIF picture with audio;
separating GIF data and audio data carried by the GIF picture;
analyzing the GIF data to obtain a GIF frame corresponding to the GIF picture, and analyzing the audio data to obtain an audio frame corresponding to the GIF picture;
and synchronously playing the GIF frame and the audio frame.
Optionally, the audio identifier comprises a first identifier identifying a size of the audio data;
the separating the GIF data and the audio data carried by the GIF picture comprises the following steps:
determining the size of audio data carried by the GIF picture according to the first identifier;
separating the audio data according to the position of the audio identifier and the size of the audio data.
Optionally, the synchronously playing the GIF frame and the audio frame includes:
calculating a difference value between the timestamp of the currently played GIF frame and the timestamp of the currently played audio frame;
if the difference value is larger than zero, calculating the remaining playing time length of the currently played audio frame;
continuously displaying the current GIF frame in the residual playing time length under the condition that the difference value is greater than the residual playing time length, and playing the next GIF frame while playing the next audio frame;
and if the difference value is less than or equal to zero, playing the next GIF frame.
The communication bus 704 mentioned in the electronic device 700 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus 704 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface 702 is used for communication between the electronic apparatus 700 and other apparatuses.
The Memory 703 may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor 701 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In another embodiment of the present invention, there is further provided a computer-readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to execute the GIF picture generating method or the GIF picture playing method described in any one of the above embodiments.
In another embodiment of the present invention, there is also provided a computer program product including instructions, which when run on a computer, causes the computer to execute the GIF picture generating method or the GIF picture playing method described in any one of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (12)

1. A GIF picture generation method, comprising:
acquiring audio data in an audio file corresponding to the target GIF picture;
writing the audio data at the tail of the GIF data in the target GIF picture, and adding an audio identifier at the tail of the audio data or between the GIF data and the audio data to generate a GIF picture with audio;
wherein the audio identifier is used to identify the GIF picture with audio.
2. The generation method according to claim 1, wherein the audio identifier includes a first identifier for identifying the size of the audio data, and the first identifier is used for separating audio data carried in audio GIF pictures when playing the audio GIF pictures.
3. A GIF picture playing method, wherein the GIF picture is generated according to the method of claim 1, and the playing method comprises:
searching an audio identifier, and identifying the GIF picture as the GIF picture with audio;
separating GIF data and audio data carried by the GIF picture;
analyzing the GIF data to obtain a GIF frame corresponding to the GIF picture, and analyzing the audio data to obtain an audio frame corresponding to the GIF picture;
and synchronously playing the GIF frame and the audio frame.
4. The playback method according to claim 3, wherein the audio identifier includes a first identifier that identifies a size of the audio data;
the separating the GIF data and the audio data carried by the GIF picture comprises the following steps:
determining the size of audio data carried by the GIF picture according to the first identifier;
separating the audio data according to the position of the audio identifier and the size of the audio data.
5. The playback method of claim 3, wherein the synchronously playing back the GIF frames and the audio frames comprises:
calculating a difference value between the timestamp of the currently played GIF frame and the timestamp of the currently played audio frame;
if the difference value is larger than zero, calculating the remaining playing time length of the currently played audio frame;
continuously displaying the current GIF frame in the residual playing time length under the condition that the difference value is greater than the residual playing time length, and playing the next GIF frame while playing the next audio frame;
and if the difference is less than or equal to zero, directly broadcasting the next GIF frame.
6. A GIF picture generation apparatus, comprising:
the acquisition module is used for acquiring audio data in the audio file corresponding to the target GIF picture;
a generating module, configured to write the audio data at the end of the GIF data in the target GIF picture, and add an audio identifier at the end of the audio data, or between the GIF data and the audio data, to generate a GIF picture with audio;
wherein the audio identifier is used to identify the GIF picture with audio.
7. The apparatus of claim 6, wherein the audio identifier comprises a first identifier identifying a size of the audio data, and wherein the first identifier is configured to separate audio-bearing GIF pictures from audio-bearing GIF pictures when the audio-bearing GIF pictures are played.
8. A GIF picture playback apparatus, wherein the GIF picture is generated according to the method of claim 1, the playback apparatus comprising:
an identification module for searching the audio identifier and identifying the GIF picture as the GIF picture with audio
The separation module is used for separating the GIF data and the audio data carried by the GIF picture;
the analysis module is used for analyzing the GIF data to obtain a GIF frame corresponding to the GIF picture and analyzing the audio data to obtain an audio frame corresponding to the GIF picture;
and the playing module is used for synchronously playing the GIF frame and the audio frame.
9. The apparatus of claim 8, wherein the audio identifier comprises a first identifier identifying a size of the audio data;
the separation module includes:
a determining unit, configured to determine, according to the first identifier, a size of audio data carried by the GIF picture;
a separation unit for separating the audio data according to the position of the audio identifier and the size of the audio data.
10. The apparatus of claim 8, wherein the playback module comprises:
a first calculating unit for calculating a difference between a time stamp of the currently played GIF frame and a time stamp of the currently played audio frame;
the second calculating unit is used for calculating the remaining playing time of the currently played audio frame if the difference value is greater than zero;
the first playing unit is used for continuously displaying the currently played GIF frame in the residual playing time length under the condition that the difference value is larger than the residual playing time length, and playing the next GIF frame while playing the next audio frame;
and the second playing unit is used for directly playing the next GIF frame if the difference value is less than or equal to zero.
11. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 5 when executing a program stored in the memory.
12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 5.
CN202010123020.3A 2020-02-27 2020-02-27 GIF picture generation method, playing method and device and electronic equipment Pending CN111354384A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010123020.3A CN111354384A (en) 2020-02-27 2020-02-27 GIF picture generation method, playing method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010123020.3A CN111354384A (en) 2020-02-27 2020-02-27 GIF picture generation method, playing method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN111354384A true CN111354384A (en) 2020-06-30

Family

ID=71197179

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010123020.3A Pending CN111354384A (en) 2020-02-27 2020-02-27 GIF picture generation method, playing method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN111354384A (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1777265A (en) * 2005-08-19 2006-05-24 上海晨兴电子科技有限公司 Image-sound synchronous recording and playing method
CN1960509A (en) * 2006-08-29 2007-05-09 中兴通讯股份有限公司 Method for implementing fault isolation in time of transferring media data of mobile multimedia broadcast
CN103458269A (en) * 2012-05-28 2013-12-18 中国移动通信集团公司 Mobile multimedia service access method, hotspot server and mobile multimedia service system
CN103646048A (en) * 2013-11-25 2014-03-19 宇龙计算机通信科技(深圳)有限公司 Method and device for achieving multimedia pictures
CN104916298A (en) * 2015-05-28 2015-09-16 努比亚技术有限公司 Coding and decoding methods, coding and decoding devices, electronic equipment and audio picture generating method
CN105261054A (en) * 2015-08-27 2016-01-20 努比亚技术有限公司 Device and method for compositing audio GIF image
CN105472289A (en) * 2015-11-11 2016-04-06 努比亚技术有限公司 Device and method for editing audio photo
CN108846054A (en) * 2018-05-31 2018-11-20 出门问问信息科技有限公司 A kind of audio data continuous playing method and device
CN109166165A (en) * 2018-06-25 2019-01-08 网宿科技股份有限公司 A kind of playback method of dynamic picture, terminal and can storage medium
CN110418189A (en) * 2019-08-02 2019-11-05 钟国波 A kind of low latency can be used for transmitting game, high frame per second audio/video transmission method

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1777265A (en) * 2005-08-19 2006-05-24 上海晨兴电子科技有限公司 Image-sound synchronous recording and playing method
CN1960509A (en) * 2006-08-29 2007-05-09 中兴通讯股份有限公司 Method for implementing fault isolation in time of transferring media data of mobile multimedia broadcast
CN103458269A (en) * 2012-05-28 2013-12-18 中国移动通信集团公司 Mobile multimedia service access method, hotspot server and mobile multimedia service system
CN103646048A (en) * 2013-11-25 2014-03-19 宇龙计算机通信科技(深圳)有限公司 Method and device for achieving multimedia pictures
CN104916298A (en) * 2015-05-28 2015-09-16 努比亚技术有限公司 Coding and decoding methods, coding and decoding devices, electronic equipment and audio picture generating method
CN105261054A (en) * 2015-08-27 2016-01-20 努比亚技术有限公司 Device and method for compositing audio GIF image
CN105472289A (en) * 2015-11-11 2016-04-06 努比亚技术有限公司 Device and method for editing audio photo
CN108846054A (en) * 2018-05-31 2018-11-20 出门问问信息科技有限公司 A kind of audio data continuous playing method and device
CN109166165A (en) * 2018-06-25 2019-01-08 网宿科技股份有限公司 A kind of playback method of dynamic picture, terminal and can storage medium
CN110418189A (en) * 2019-08-02 2019-11-05 钟国波 A kind of low latency can be used for transmitting game, high frame per second audio/video transmission method

Similar Documents

Publication Publication Date Title
CN110418186B (en) Audio and video playing method and device, computer equipment and storage medium
US8307403B2 (en) Triggerless interactive television
CN1581951A (en) Information processing apparatus and method
CN110996182A (en) Timestamp processing method and device, electronic equipment and computer storage medium
CN112437337A (en) Method, system and equipment for realizing live broadcast real-time subtitles
US20190342428A1 (en) Content evaluator
CN111246288A (en) Method and terminal for demonstrating PPTX document in video mode
CN113992926B (en) Interface display method, device, electronic equipment and storage medium
CN114117120A (en) Video file intelligent index generation system and method based on content analysis
CN113391745A (en) Method, device, equipment and storage medium for processing key contents of network courses
CN110322525B (en) Method and terminal for processing dynamic diagram
CN116708892A (en) Sound and picture synchronous detection method, device, equipment and storage medium
CN111354384A (en) GIF picture generation method, playing method and device and electronic equipment
US20050155071A1 (en) Demultiplexing device
CN116437068A (en) Lip synchronization test method and device, electronic equipment and storage medium
JP2002057990A (en) Video reproduction system and data synchronization system used therefor
CN113891108A (en) Subtitle optimization method and device, electronic equipment and storage medium
CN116962741A (en) Sound and picture synchronization detection method and device, computer equipment and storage medium
CN113573088B (en) Method and equipment for synchronously drawing identification object for live video stream
US20070047910A1 (en) Image replay apparatus and method for moving-picture streams
CN112437289A (en) Switching time delay obtaining method
JP2008219608A (en) Method and apparatus for displaying information with content, program therefor and recording medium
KR102659489B1 (en) Information processing devices, information processing devices and programs
JP6252083B2 (en) CONFERENCE SYSTEM, SERVER DEVICE, CLIENT TERMINAL, AND PROGRAM
CN109275007B (en) Processing method and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200630

RJ01 Rejection of invention patent application after publication