CN102156734B - Video content management method based on semantic hidden indexing - Google Patents

Video content management method based on semantic hidden indexing Download PDF

Info

Publication number
CN102156734B
CN102156734B CN 201110090360 CN201110090360A CN102156734B CN 102156734 B CN102156734 B CN 102156734B CN 201110090360 CN201110090360 CN 201110090360 CN 201110090360 A CN201110090360 A CN 201110090360A CN 102156734 B CN102156734 B CN 102156734B
Authority
CN
China
Prior art keywords
semantic
video
data
space table
semantic space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN 201110090360
Other languages
Chinese (zh)
Other versions
CN102156734A (en
Inventor
邢玲
黎茂锋
陈淼
马建国
马强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southwest University of Science and Technology
Original Assignee
Southwest University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southwest University of Science and Technology filed Critical Southwest University of Science and Technology
Priority to CN 201110090360 priority Critical patent/CN102156734B/en
Publication of CN102156734A publication Critical patent/CN102156734A/en
Application granted granted Critical
Publication of CN102156734B publication Critical patent/CN102156734B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The invention discloses a video content management method based on semantic hidden indexing, which ensures that a semantic space table (SST) and video data coexist all the time in the network migration and transmission process by defining the SST and performing integrated semantic indexing on the SST and the video data by an information hiding method. For a video application system (such as an intelligent play and download agent system, a video classification management system, a network management and control system and the like), the SST of the video data is extracted to be subjected to comparative calculation with a semantic request table (SRT) at a destination end, so that the calculation result can help the system to decide a specific processing mode of the video data. The semantic information of the hidden indexing is hard to erase or falsify, so that the semantic information loss of the video data in the secondary transmission process is prevented; and the content video data can be effectively unified and correlated, so that the redundant transmission of data is reduced, and the utilization efficiency of the network is greatly improved. Meanwhile, the video search engine and the other video application systems can be helped to better perform the selection, rejection, abandonment and other operations on the video data, so that the video data in the network space can be transmitted orderly, and the video can be found more efficiently and quickly.

Description

A kind of video content management method of semantic-based hidden indexing
Technical field
The invention belongs to the video content management technical field, more specifically say, relate to a kind of video content management method of semantic-based hidden indexing.
Background technology
Except traditional CATV (cable television), terrestrial television open-circuit television and satellite video, the internet video Data Growth is rapid at present, Internet video has become the important media that people obtain the digital contents such as film, TV, video, has occupied most flows on the internet.27 China Internet development reports of CNNIC (CNNIC) issue claim, by in Dec, 2010, the domestic network video user reaches 2.84 hundred million, permeability in the netizen reaches 62.1%, compare with 12 ends in 2009, increase by 4,354 ten thousand people Internet video user number year, annual growth 18.1%.Watch online the ratio fast lifting of video, accounting reaches 95.4%, finds and downloads user's accounting of watching and reach 47.6%.The user shares more active, and in time communication and social network service (SNS) becomes the important channel of carrying out video sharing on the internet, and 85.4% user carries out video sharing and forwarding by communication chat tool, social network sites.Simultaneously, utilizing smart mobile phone or portable terminal to carry out video data by the mobile network obtains with to share ascendant trend also very obvious, Cisco's prediction from 2009 to 2014, mobile video quantity will increase to original 66 times, the Mobile data total flow will increase progressively with annual 108% rate of growth, to reach per month 3.6 peta-bytes to Mobile data total flow in 2014, video content accounts for 66% of Mobile data.Corresponding video market scale development is swift and violent with it, easily sees the world and expects Chinese Internet video market scale in 2013 and will reach 69.9 hundred million yuan, and city original singer's animations such as several big cities such as Hangzhou, Wuxi, Chongqing, Beijing etc. all reach several very much clock output.
So huge video market exists, and has also brought simultaneously a series of problem in internet.Because the needs of competition, the duplicate construction of video website randomness causes various finished product video products to repeat issue, social participatory video sharing website and p2p share web have further improved the repeatability issue of video, relative download and reception more make redundancy repeat transmission sharp increase therewith, idle flow has occupied the overwhelming majority, greatly limit the performance performance in other respects of internet, hindered the further fast development of internet miscellaneous service.
Redundancy is spread unchecked when bringing wasted storage and bandwidth waste, the power series rising characteristic of hot video repetition transfer rate has been sheltered the propagation of the valuable video of low focus and has been obtained, the search engine search results collection is full of by hot video, has greatly improved real goal and has extruded rate.
Simultaneously and since video terminal technology, video processing technique, transmission of video and the progress of obtaining technology with popularize, the video of various contents is mobility and migration, netizen's also growth on a large scale of autonomous video in the world, and copyright management is caused great difficulty.
Cause the reason of above problem to be, the internet design is the information sharing system that grows up from paying attention to communicating by letter with building.Communication as Shannon information theory is exactly effective transmission of paying attention to data, pay attention to information grammatical form (Code And Decode), and to the semanteme of information namely content too much do not pay close attention to, information is moved the content that can not pay attention to itself in the transmission course at network like this.Further, the contained content of video data itself and implication are very abundant, and it is very difficult to carry out the contents semantic analysis with machine, video data in network, move with transmission course in to its carry out that semanteme is surveyed and according to semanteme determine whether allow to propagate, confirm to play and receive download just very difficult.The semantic net that proposes at present has ideal disposal route to information resources, but because too desirable, and the system requirements Global Internet integrates integraty and the basic unambiguity finished information resources and describes, to such an extent as to the too large feasibility of engineering is too poor, makes progress without obvious in real work for many years.
In the process that video content is managed, usually come describing video contents itself with the mode of text meta-data, organize together sequence of pictures content to be expressed to avoid machine to go to understand with time basis.Video content management comprises the management of video file management and video data stream, namely static data and the data of video file under spread state that are present in the mediums such as disk, CD.
1, video file management
The main emphasis of video file management described the text mode of video file content, and the relation between the text can structuring and deposited ripe database in, such as relevant database.Usually the mode with text meta-data is described video content itself, text meta-data is described the content and the attribute relevant with content that video file represents on the one hand, on the one hand describe between the video file in addition or video file and other resources such as storage medium between relation.Pass through so fast database search, just can find the video file clauses and subclauses that the user searches, and offer user-dependent video content by file system.
2, video data flow management
Using at present of video accounts for the overwhelming majority with user's broadcast as final purpose, so the migration of video data desalinated gradually all downloads and play afterwards and become the mode of broadcasting with Streaming Media, and the user can play while downloading, and has greatly saved download time.The management of video data stream just becomes extremely important like this.Video data stream relates generally to the information source tissue traffic model, transmission and the stay of two nights are obtained and are play three parts, (it is represented as (the Internet Protocol based on IP to correspond respectively to streaming video broadcast front end tissue (also being Play Server) and real-time Transmission channel, Internet protocol) RTP of network (Real-time TransportProtocol, RTP) transmission mode and based on digital broadcast channel DVB (Digital VideoBroadcasting, abbreviation DVB) TS (Transport Stream, transport stream) transmission mode, and playback terminal.
Fig. 1 is digital video broadcast system provides video broadcast service in the mode of transport stream schematic diagram.It is multiplexing and put into channel and transmit that this digital video broadcast system is packaged into the regular length pack arrangement to the video file in the video file storehouse.Select in order to allow client (stay of two nights) can carry out program simultaneously, service end (information source) also is combined together to form EPG (Electronic Program Guide according to video file associated text metadata in conjunction with transport stream (TS) stream code of reproduction time arrangement and video frequency program, electric program menu) programme, and programme also is transferred to client with the TS type of flow allows the user browse, thereby from multiplexing TS stream, identify interested video frequency program.In this process, the audio-video document of information source end and this audio-video document content description information are to leave in respectively in audio-video document and the text meta-data storehouse; The text meta-data of denoting contents also is separately to transmit with different TS stream with audio/video flow in the channel; In the also separately independent existence of audio-video document of the EPG of stay of two nights end data and user selection, logically relevant, physically isolation.Like this, if when client is deposited as video file again, will lose valuable content description information if link to store with similar service end text meta-data storehouse ambiguously.Certainly broadcast in the service at DVB; legal so again storage demand is not ubiquitous; but the secondary storage is a large amount of the existence with moving in today that network is widely applied again; so not only cause service media not have beyond the content description text message, the copyright protection that breaks away from the free video media of former host also becomes a great problem.
Fig. 2 is text meta-data storage, transmission and the record principle figure of audio-video document and content description in the rtp streaming formula media services process.
Do not carrying out on the internet that interactive service can be provided of streaming media service with regular length TS bag, as shown in Figure 2, the text meta-data of audio-video document and content description is relevant and also separate three ingredients that are present in the interactive communication model in the rtp streaming formula media services process, namely in information source, channel and the stay of two nights.Service end, be that the information source part arranges according to broadcasting, audio-video document provides the mode of stream packet switch that service is provided by streaming media server to client, certainly the stream of program request is just to begin to send after the client files a request, simultaneously the content description information of video file is formed the audio frequency and video service catalogue inventory of WEB mode, each directoryof item corresponds to relevant program stream service.Client just can be spread by program Xiang Laixiang TS on the click WEB server and be put the request that server proposes to send program stream like this, and RTP packet sequence corresponding to simultaneously client control media reception stored after treatment, play or again service.
In this process, can see obviously that the audio-video document of information source end (service end) and the text meta-data of content description separately deposit; Carry out data transmission with different internet agreements especially in the channel; And separately process at stay of two nights end, cause the difficulty of serving again if the final video data that receives moves with regard to being very easy to because corresponding text meta-data in the last time receiving course does not have appropriate organize to lose again.
Certainly, mention the problem that separated transmission is brought above can solving by the text meta-data that embeds content description at the video file head.But the robustness in the mode of audio-video document head embedded text metadata is not too enough.By simple data structure analysis, just can remove substitutions and modifications.And in video file compressed format transfer process, also very easily lose such information, because text meta-data except describing video contents, also may be described the relevant sensitization information such as copyright, be unfavorable for the protection of copyright after the deletion.
The demanding Semantic Indexing of robustness is necessary in the situation that the internet video content is greatly enriched like this.At first extremely abundant video file is because of on the facilities such as the free on the internet different server of numerous netizens' propagation, PC, cloud service platform, and the Semantic Indexing of the content description of free video file itself, be that text meta-data lacks very much, collection to the video file data is very difficult, if video file is not lost because of the change of variation, form and the form of network site with video file by the Semantic Indexing information of the content description of itself in the process of numerous user's downloads, transmission, conversion, organized again and retrieved with regard to easier.
The a large amount of existence on the internet of video file and flow data cause that also the another one problem is exactly that some popular or hot video can be because netizen's massive duplication, forwarding, serve and be the power series amplification effect again.And because there is not unified content to describe the difficult size of distinguishing redundancy and amount of redundancy, some sparse services useful to specific crowd of stream are squeezed in the discovery that redundancy services information overload effect has been buried in oblivion other useful video datas.
Summary of the invention
The object of the invention is to overcome the deficiencies in the prior art, a kind of video content management method of semantic-based hidden indexing is provided, not strong to solve the Semantic Indexing robustness, and the problem that the Semantic Indexing information of bringing is organized easily again, the video data redundancy is spread unchecked on retrieval and the internet.
For achieving the above object, the present invention is based on the video content management method of semantic hidden indexing, it is characterized in that, may further comprise the steps:
(1), at the information source end, go out the semantic space table of the video content that video data comprises according to video content and correlation attribute definition, the semantic space table comprises that content description, ownership describe and resource identification three parts semantic information;
(2), at the information source end, with the method for Information hiding semantic space table data are embedded in the video data;
(3), the stay of two nights end after video data moves, during use, utilize the inverse approach of Information hiding to extract semantic space table data in the video data, obtain the semantic space table of video data;
(4), at stay of two nights end, set up the application semantics demand schedule, and compare and computing with the semantic space table that extracts, independently determine the processing mode of the video data that this semantic space table is corresponding.
Goal of the invention of the present invention is achieved in that
The present invention is by definition semantic space table, video data is carried out the one Semantic Indexing, then its method with Information hiding is embedded in the video data, thereby allow video data in network migration and communication process, exist always, Video Applications is in the stay of two nights systems such as broadcast, smart download agency, video spatial scalable management, network management and control, by extracting the semantic space table of video data, and get the semantic requirement table with stay of two nights end and compare, decide the processing mode of video data.Because information concealing method embeds semantic space table data in the video data, semantic information extremely difficulty is erased and is distorted, high steadiness is arranged, being present in like this in the cyberspace the same video data of arbitrary position can significantly unite, can prevent video data in the secondary communication process semantic information lose redundant request with video data.Simultaneously, according to the semantic space table that extracts, can further help video search engine and other Video Applications systems the operation such as to select, refuse, abandon, video data is propagated joint in order.
Description of drawings
Fig. 1 is digital video broadcast system provides video broadcast service in the mode of transport stream schematic diagram;
Fig. 2 is text meta-data storage, transmission and the record principle figure of audio-video document and content description in the rtp streaming formula media services process;
Fig. 3 is a kind of embodiment schematic diagram of video content management method that the present invention is based on semantic hidden indexing;
Fig. 4 is semantic space table packet structure figure shown in Figure 1;
Fig. 5 is the telescopiny schematic diagram of looking audio file;
Fig. 6 is MPEG 2 video code flow decode procedure synoptic diagram;
Fig. 7 is the synoptic diagram that embeds semantic space table SST packet in MPEG 2 video code flows;
Fig. 8 is based on the telescopiny synoptic diagram of the semantic space table SST packet of DCT coefficient modifying;
Fig. 9 looks the procedure chart that extracts semantic empty table SST ' in the audio file;
Figure 10 is the procedure chart that detects reduction semantic space table SST ' at MPEG 2 files;
Figure 11 is that intelligent video is acted on behalf of receiving system application synoptic diagram;
Figure 12 is that video content classification receiving system is used synoptic diagram;
Figure 13 is that the community intelligent video user agent of colony serves the application synoptic diagram.
Embodiment
Below in conjunction with accompanying drawing the specific embodiment of the present invention is described, so that those skilled in the art understands the present invention better.What need to point out especially is that in the following description, when perhaps the detailed description of known function and design can desalinate main contents of the present invention, these were described in here and will be left in the basket.
Embodiment
Fig. 3 is a kind of embodiment schematic diagram of video content management method that the present invention is based on semantic hidden indexing.
As shown in Figure 3, in the present embodiment, the video content management method that the present invention is based on semantic hidden indexing may further comprise the steps:
One, semantic space table of definition
At the information source end, go out semantic space table (the Semantic Space Table of the video content that video data comprises according to video content and correlation attribute definition, be called for short SST), the semantic space table comprises content description, ownership description and resource identification three parts semantic information.
1, the structure of semantic space table
Each data item in the semantic space table, be semantic item representative be the composition of semantic space table, each semantic item can further be added qualifier and be expanded semantic concrete meaning, and searchable engine is found or application system is used, and semantic item has embodied certain aspect implication of video data or the attribute of video.The semantic space table can be organized semantic item by certain format, organizes namely the semanteme table is carried out the plaintext coding, the separative sign between the synchronous first symbol of the main consideration of coding semantic item and end mark, the semantic item.
Semantic item has comprised two parts, and a part is the semantic item title, and a part is the corresponding semantic values of semantic item.A semantic item can contain many-valued, can need to expressly distinguish semantic key name and semantic item value with concrete character or separating character in the coding thus, and the differentiation of semantic item between many-valued, the qualifier of semantic item also comprises to be modified key name and modifies a value, a semantic item can have zero to a plurality of modifications items, each is modified item and also can by many-valued, can distinguish according to special separating character in the plaintext coding of semantic space table.If in semantic values, include separating character before this character, add ESC as " ".Need after the plaintext end-of-encode of semantic space table that the semantic space table is carried out entropy and encode to reduce semantic table embedded quantity, make the embedding more can be successful, simultaneously, but encode according to the application system selective entropy.
The semantic space table is used for representing the index body of certain video data, with Dublin Core Metadata proposal (DublinCore Metadata Initiative, be called for short DCMI) similar for the Dublin Core Metadata collection of publication definition, in the present embodiment, semantic space table SST forms with the associated metadata elements of one dimension, specifically is shown in Table 1:
Table 1
In the present embodiment, as shown in table 1, the semantic space table comprises content description, ownership description and resource identification three parts semantic information, and wherein each part is comprised of a plurality of semantic item.The semantic item title chooses with explanation, data encoding and standard, indispensability, whether be many-valued, whether qualifier, data type and data occupy-place length are arranged, needing could be as unified requirement in the situation that national standard or industry standard formulate, before not having national standard and industry standard, can oneself define the correlative senses of a dictionary entry and use.In table 1, last " other " part is for no other reason than that the semantic item of relevant video semanteme spatial table is too many and the not a kind of embodiment under the stable condition of national standard or industry standard, and expression also has a lot of semantic item can select to add in the semantic space table and define.The method with Information hiding of considering embeds, and affected by concrete video and hide the embedding data amount, need to simplify as far as possible semantic space table data volume.
Whether the coding of each semantic item or standard refer to this semantic item value from some standards in the table 1, the such value of semantic item standard more, " middle submeter " such as theme Subject semantic item coding criterion, " middle submeter " i.e. be " Chinese descriptor sorted table ", in the sorted table numerous descriptor is processed disaggregatedly by subject, and carried out corresponding one by one with the code means of character and digit.Inserting S028133 such as us in the theme subject of certain video value, is " optical computer " this descriptor by what look into that " Chinese descriptor sorted table " just know expression.Obviously, complicated word occupies the word of more data bit and can process to save data volume with such code.Whether indispensability refers to must have value corresponding to this semantic item to exist in the final embedding video data process, because some semantic item obviously is very important such as copyright notice/information, be indispensable therefore.Ambiguity is similar to the repeatability of CDMI definition metadata, can be a plurality of and be not only one such as the producer of certain video.Qualifier is further describe of carrying out in order to further describe certain aspect correlation properties of video data.What data type was used is field data types commonly used in the database.Data length was mentioned in front.
More deep for the content description of video data is got, make things convenient for inquiry and the retrieval of video, some semantic item of semantic space table SST in the table 1 can also conduct further description by modifier, and certain semantic item is segmented.The qualifier of each certain semantic is not illustrated one by one at this, only set forth for typical example.Generality such as " summary description " this semantic item itself is too strong, just is used for describing the content of whole video, not segmentation.Such as qualifier " summary (abstract) " in addition, so just in the space of summary, add specially the summary info about this video, just can further as dividing chapters and sections, DVD continue to describe to every section video such as adding " chapters and sections (chaper) ", convenient meticulousr inquiry, simultaneously, for video data, the a large amount of popular label that on the network video is added at present also can be introduced qualifier " label (Tags) ", allows limited semantic item describing video contents as much as possible.
2, the semantic item of semantic space table coding organizational form
Understand that in a upper joint acceptance of the bid semantic item of the semantic space table of video data forms, wherein certain is semantic according to the description degree of depth to content, can also further utilize qualifier (Modifier) identifier to be described, but concerning qualifier, the modification symbol of lower one deck no longer is set in principle, namely double-deck element building form.
Be the hierarchical structure of semantic item such as following table 2, it has explained the grammer of a semantic item and qualifier and word, wherein<and Element Identifier〉be the semantic item identifier, concrete value corresponding to " [] " expression behind the semantic item identifier is if value allows many-valuedly can separate them with the western language comma.If then there is qualifier in this semantic item, then qualifier comes the second row with the mode of indentation, and with arabic numeral qualifier is sorted, the present invention does not limit and uses specific separator, the separator that this section is given an example is such as "<", "〉", " [", "] " can be replaced by the separator of other form.
Figure BDA0000054826390000091
Table 2
Like this, a plurality of semantic item and value form to the semantic description of video data just such as table 3 mode
Figure BDA0000054826390000092
Table 3
Represent that with an example code semantic item summary description Description is as shown in table 4
Figure BDA0000054826390000093
Table 4
Above form is being carried out data storage or xml, and RDF is easy to process with the syntax format of XML in the time of conversion, in the present embodiment, and all need to be with such concept type syntax format and other infosystems exchange metadata.In the present embodiment, the NameSpace literary style of semantic space table is a kind of grammer signal, can also carry out simple in specific implementation semantic space table data telescopiny.
Semantic space table payload (SST Payload) can be expressed as the data that order is deposited like this, shown in grammer code in the table 5, and the separator "<" of the division between semantic item and the semantic item, " ", " [", " "] " processes.If the character with the separator conflict is arranged in the semantic space table payload, before filling just conflict add before the character ESC " ", shape is such as "<", " ", if itself there is ESC “ among the payload payload ", then also add escape symbol “ in its front ".
Figure BDA0000054826390000101
Table 5
In the present embodiment, in order to distinguish whether semantic space table information of the information that from video data, detects, in order to further analyze and process, when definition semantic space table packet assembling structure, can before semantic space table payload SST payload, add SST packet header, use in order to differentiate.
Fig. 4 is semantic space table packet structure figure shown in Figure 1.
In the present embodiment, as shown in Figure 4, to identify this bag with 12 bits be semantic space table data stuffing bag to SST Identifier in the semantic space table packet, be decided to be complete 1 bit string, " 11111111111111 ", version version indicates the version of current semantic space table data, and namely the subsequent data structure should meet the standard of current version definition, represents with binary number " 0001 "; Whether compress cmpr and account for 1 binary number, explanation be payload payload, namely whether semantic space table data have used compress mode, be to be 1, be not 0; Representing to wrap big or small Packet Size with 15 bits, determine according to the occupy-place length of semantic space table payload, is 1 if whether compress cmpr value, calculating be length after compressing.Because video data may be very long, whenever might repeat after a while to inject this semantic space table data, represent sequence number Sequence No with 16 bits, what sequence number represented is the sequence number of current semantic space table data, if semantic space table packet is unduplicated, the situation of can find so whether video data is truncated, part such as distorting at the aspect.
At the information source end, according to semantic space list structure and the content of video data and the semantic space table that association attributes obtains video data of definition, after then encoding, form semantic space table data.As shown in Figure 1, for guaranteeing the safety of semantic space table, in the present embodiment, semantic space table data also will be encrypted with public keys K, after the semantic space table data packing after the encryption, with video data V 0Hide embedding.
In this enforcement, video data comprises video file, looks audio file, real-time video and dynamic GOP length sequence.
Two, the embedding of semantic space table data
As shown in Figure 1, at the information source end, with the method F of Information hiding MThe packet of semantic space table SST data is embedded into video data V oIn, obtain comprising the video data V of semantic space table SST data SST
Information concealing method F MSemantic space table data are embedded, embed such as the mode that can adopt digital watermarking, concrete information concealing method F MDecide as the case may be with video data format.In the present embodiment, semantic space table data SST makes packet, puts into the compression section of video data, as revising DCT coefficient and motion vector data part, is difficult for causing like this losing of data in the file migration process.Therefore concerning video file, because video file often comprises audio track and track of video simultaneously, therefore at first need to isolate track of video, by in track of video, finding out picture group (Group ofPictures after the entropy decoding, be called for short GOP), then locate layer by layer the DCT coefficient of all types of frames, macro block and macro block and the motion vector of macro block, reach concrete 0 and 1 data bit of hiding semantic space table SST packet by DCT coefficient and the motion vector of revising macro block.For the data that camera or other modes are caught in real time, in the process of compression, can revise DCT coefficient and motion vector, reach the purpose that embeds semantic space table data b it position.
After having embedded a semantic space table SST packet, continue as required in ensuing video data, to embed second, the 3rd semantic space table SST packet, and each semantic space table SST packet is carried out auto-increment number, until the end of video data, repeated and redundant embeds like this, still can recover semantic space table SST packet by cyclic redundancy check (CRC) in the situation that makes things convenient for the Internet video data to make mistakes, reach simultaneously from the video data any point and begin completely to recover backward semantic space table SST data, in order to help the better managing video data of Video Applications system.
Fig. 5 is the telescopiny schematic diagram of looking audio file.
For present a large amount of video data, carry out the storage of file mainly with the audio frequency and video hybrid mode greatly, in this enforcement, the embedded mode of looking on the audio file in static state take the semantic space table describes as example.
All comprised multi-orbit data depending on audio file, generally comprised video code flow, one or multi-orbit audio code stream.Since for make audio frequency and video synchronously, in audio code stream and video code flow, have relevant synchronically controlling information to control decoded synchronous broadcast.
As shown in Figure 5, the embedding of semantic space table data is divided into two main lines, and a main line is that semantic space table SST semantic item is abstract, the also packing of constructing semantic spatial table; An other main line is to read to look audio file F and carry out, and then semantic space table SST data is embedded into and looks among the audio file F.
For looking audio file, at first be to determine that the semantic item of sdi video table SST is abstract, can carry out in artificial mode, certainly also can take out the text-type semantic (TextualSematic) that semantic space table SST needs by the vision semantic (Visual Semantic) that audio file is looked at present some advanced method analyses, present vision semanteme is converted into also difficulty relatively of text mode descriptive semantics certainly.Next be the structure of semantic space table SST, i.e. encapsulation and the coding of semantic space table SST before embedding.The purpose of encapsulation is convenient the extraction later on and reduction in application process; The purpose one of coding is the needs that are secret and special applications, the 2nd, can dwindle code length in some cases, and improve the validity that watermark embeds.
Fig. 5 has illustrated telescopiny in audio-video document simultaneously.At first order reads that to look audio file F be data stream V All, in reading process, differentiate simultaneously and tell target video code stream V FWith other code stream V Other, then target video code stream V FThe semantic space table SST packet utilization that has generated is fit to target video code stream V FInformation concealing method F M(such as video watermark) hides embedding operation, and output embeds the new video stream V that semantic space table SST data are arranged SST, can be represented by the formula:
V SST=F M(V F,SST)
F wherein MBe selected information concealing method, the selection of carrying out with concrete video compression coding is relevant.New video code flow V SSTAgain with other code stream V OtherTogether by the new file F ' of synchronizing information Sync production control, so new file F ' is just the same with original F to have the file that is equal to fully and packs form, do not destroy original storage organization, just because the hiding embedding of semantic space table SST packet has affected total file size, be difficult for detecting like this particular location at concrete semantic primitive place by fixing structured data mode, increased the hidden attribute of SST.
The below is to information concealing method F M, adopt in the present embodiment DCT (Discrete CosineTransform, discrete cosine transform) watermarking algorithm to further specify.
In the present embodiment, for looking audio file, wherein Video coding is regarded compressed video data substantially as, but compressed video data has more algorithm, for different packed datas, has different watermarking algorithms to select.Choose that suitable watermarking algorithm can make the information capacity of embed watermark larger, robustness is better or make to embed with the computation complexity that detects watermark and reduce, improve to embed and the speed that detects semantic space table SST data.
Fig. 6 is MPEG 2 video code flow decode procedure synoptic diagram.
The watermark embedding that the below does further semantic space table SST packet with common MPEG2 compressed video data describes.
As shown in Figure 6, because for the audio file of looking generally be through the data of overcompression, as for the MPEG2 formatted data, carrying out the watermark embedding at dct transform coefficient is one of common mode, and the DCT coefficient in the packed data can not directly extract from the demonstration data structure of video code flow.Video data V among Fig. 5 FNeed just can see the DCT coefficient by decoding length changeable code first, next the data behind inverse quantization, motion compensation, the anti-DCT have just formed frame data, then just deliver to display device and show.
Fig. 7 is the synoptic diagram that embeds semantic space table SST packet in MPEG 2 video code flows.
Fig. 7 has showed first video code flow has only been done variable length decoding, then decoded data is done the embedding of semantic space table SST data, does simultaneously compensation deals and writes video data, and output embeds the new video stream V of semantic space table SST data SST.It is little because the watermark information embedding of carrying out on the DCT coefficient is the operation to key frame I frame substantially, and key frame is the reference frame of P frame and B frame, the variation cumulative errors of I frame will have influence on the reduction of P frame, can bring the decline of video quality, therefore need to do compensation deals.
Fig. 8 is based on the telescopiny synoptic diagram of the semantic space table SST packet of DCT coefficient modifying.
In the present embodiment, as shown in Figure 8, to information concealing method F MEmbed semantic space table SST and done further instruction, left side block diagram is for reading video codeword data stream and embedding, and the right embeds for preparing semantic space table SST packet and offering the left side.
In the on the left side block diagram, at first read video codeword data stream to frame buffer zone, then identify video group (GOP), ensuing DCT coefficient modifying is to make amendment at key frame, be key frame so get first frame, namely then the I frame continues picture bar and macro block among the identification MPEG 2, extract the DCT coefficient of each macro block, finished decoding length changeable code.
Getting access to can be according to information concealing method F behind the macro block DCT coefficient MJudge the possibility of macro block embed watermark, and this information is fed back in the flow process of the right block diagram, one one of semantic space table SST packet is extracted the DCT coefficient is made amendment.After modification is finished, because having quoted the P frame of this key frame can be affected, so need in the current video group, navigate to the P frame that is right after, do not make any modification to using the macro block of intraframe coding in the P frame, but the DCT difference value coefficient to the macro block of inter prediction is made amendment, and is the same when being modified to unmodified I frame.Then amended data are carried out variable length coding, and be written as new video code flow V SST, next judge whether arrive the end of file of input video stream, if do not have, just continue to read video codeword data stream, continue to embed semantic space table SST packet.
Can find out from the block diagram on the right side, semantic space table SST packet repeats to be embedded in the video code flow, if each afterbody to semantic space table SST packet, will increase by 1 to the sequence number (Sequence_No) of the semantic space table SST packet that next will embed, be exactly that repeated and redundant ground embeds semantic space table SST data like this, can improve outside the semantic space table SST correcting data error, in the situation that video is blocked by editor, still can effectively detect.Also have on the right a transcoding process in the block diagram, namely ' 1 ', position in the original semantic space table SST data sequence is transformed into integer 1, ' 0 ', the position programming integer-1 in the sequence, and modulate watermark in the block diagram of the left side with this.
After the DCT coefficient of I frame is made amendment, be bound to have influence on P two field picture among this GOP, therefore need to revise the prediction chromatic value of this I frame back the one P frame.Namely this P frame is predicted again that the macro block of intraframe coding in the P frame is remained unchanged, and the macro block of interframe encode is revised the colour difference score value, made the chromatic value of this piece after the decoding and embed front not identical.
In the present embodiment, DCT telescopiny and the method for encoding with MPEG 2 describe, and are not necessarily to require the embedding of video code flow semantic space table necessarily to select DCT, for as H.263/AVC waiting, can select suitable watermark embedding method to embed.
Three, the extraction of semantic space table data
As shown in Figure 1, at video data V SSTStay of two nights end after moving during use, utilizes the inverse approach F of Information hiding M -1Extract video data V SSTIn semantic space table data, obtain the semantic space table SST ' of video data.
As shown in Figure 1, in the present embodiment, video data V SSTIn the transition process, add video noise n, therefore, become the video data V that comprises semantic space table SST data SSTn
Video data V SSTn, look the inverse approach F that utilizes Information hiding in audio file or real-time streaming data or the downloading streaming data such as static state M -1Extract and recover semantic space table SST data.To static file, at first parse the video data track, then then entropy decoding locates GOP, macro block, judges each value of semantic space table SST packet from packed data such as DCT coefficient and motion vector data, and a data bit of obtaining is spliced.If find that in spliced data initial code and data length field value are less than or equal to the splicing data length, then the data before the head and this partial data are merged, and under public key cryptosyst, decode as required, then carry out the entropy decoding of semantic space table, thereby recovering the plaintext semantic space table SST ' data of semantic table.
The below describes with the extraction of video file semantic space table data.
Need from being stored in the location semantic item content that audio file detects semantic space table SST ' data of looking on the medium in a lot of situations, just need to organize relevant metadata such as looking the audio file content with the relational database that speed is high, handling capacity is large such as video frequency searching storehouse of tissue, and it is oversize to look the spended time of audio file own by scanning, discomfort is fit to do data search, therefore looks audio file and many times only carries out for the first time later on reading of text-type data as file migration.When arranging Streaming Media to broadcast, if certain video-audio program does not also have the metadata of related content, at this moment can directly from the video file that has semantic space table SST, read semanteme, form the relevant demonstration information of traditional E PG or Broadcast Control platform.
Fig. 9 looks the procedure chart that extracts semantic empty table SST ' in the audio file.
As shown in Figure 9, order reads to be had SST and looks audio file F ', and from video flowing V AllIn read wherein video code flow V F, next use method of detecting watermarks, i.e. the inverse approach F of Information hiding M -1Detect semantic space table SST ' data, become semantic space table SST ' expressly by key recovery again, semantic item wherein can directly be extracted among the payload Payload from semantic space table SST ' packet.Key K ey wherein can formulate corresponding key management strategy according to the difference of using.Simultaneously when detecting watermark information, if having SST looks audio file F ' and has and edited or introduced may existing of noise, the process need that then detects watermark is determined by semantic space table SST ' reducing degree, if reducing degree does not reach requirement, will continue to read and look audio file data and reduce, have SST and look audio file until meet the demands or travel through complete.
Figure 10 is the procedure chart that detects reduction semantic space table SST ' at MPEG 2 files.
Figure 10 is the process of MPEG 2 files being carried out watermark detection, by reading the stream of video data of looking in the audio file, and identify GOP, identify key frame, i.e. I frame among the GOP, identification is as bar and located macroblock, obtain picture piece DCT coefficient, read the DCT coefficient and do conversion, obtain watermark, read watermark, the watermark bit of reading is filled semantic space table SST ' data through connecting conversion.Until be reduced into the data of requirement or run into the end of file.
Four, the management of video content
As shown in Figure 1, at stay of two nights end, set up the application semantics demand schedule, and compare and computing with the semantic space table that extracts, independently determine the processing mode of the video data that this semantic space table is corresponding.In the present embodiment, this step is looked closely frequently application management system by the stay of two nights and is finished.
In the present embodiment, having proposed two typical cases uses, " migration network " wherein is that the video data of a broad sense shifts environment, video data also may be copy, movement, the limited editor between the local file system, it also can be the migration between from the file system to Database Systems, also can be the transmission of video data in digital video broadcasting network such as DVB or AVS system, also can be based on the data dissemination on internet or the mobile network, the transmission between heterogeneous network also is classified as migration and the conversion of data certainly.Can regard the communication system communication process as at the video data transition process, the video content of communication information source end or be to receive the data that have semantic space table SST of coming, or be to carry out the data transmitted after semantic space table SST data embed; The application system of stay of two nights end just need to be carried out the detection of semantic space table SST and according to intended application relevant treatment be carried out in the mapping of semantic space table, thereby make the Video Applications system more convenient to video content management, make system no longer only pay attention to data transmission, and also can better pay close attention to data content intension in the migration simultaneously, make the easier realization of system intelligence.In data migration process, to stay of two nights end, total process of extracting semantic space table SST that exists, this process always needs association key K ' to authorize as required just can carry out the contrary method F of Information hiding M -1Extraction and reduction process, this key K ' and key K be a pair of in the public key cryptosyst, also different to the key management distribution policy that dissimilar application is adopted.Attention is in following two examples, in transition process, shown that all video data carries out the embedding of semantic space table SST, be not that any application system all needs the semantic space table SST that takes into account information source end and stay of two nights end to process, said as the front, the semantic space table SST of information source end may have, and that the information source end only is responsible for the data transfer service is just passable.And this moment stay of two nights end K ' when needing to carry out at first the embedding of semantic space table SST with this video data used K match mutually.
1, " intelligent video is acted on behalf of receiving system " applicating example
Figure 11 is that intelligent video is acted on behalf of receiving system application synoptic diagram.As shown in figure 11, in resource like this great today, allow computer help it is found that to meet the resource of user's request to become more and more important, Figure 11 has expressed such scene, the video data that exists at network exactly embeds semantic space table SST, in each video resource access, also can regard as in the transition process, just can reduce easily, compare in the situation of computing with subscriber's local semantic requirement table SRT again, just can find easily the whether video data of user's request of the video data that moves.
Figure 11 has showed that intelligent adaptation using implicit expression semantic embedding method acts on behalf of the application principle figure of receiving system, semantic space table SST has comprised and necessary has carried out the semantic item of content description about video data among this figure, can move in network after implicit expression embeds carrying out.User's intelligent proxy system is according to the user interest agent model, be that semantic requirement table SRT, watermark detection key calculate whether user's object of interest of target video data, and carry out corresponding control decision (Control Decision) CD, reach robotization, intelligentized purpose.
2, " video content classification receiving system " applicating example
Figure 12 video content classification receiving system is used synoptic diagram.As shown in figure 12, more and more universal at network, resource transmission more and more is easy to today, and it is to be developed that video content classification broadcast and transmission have the great market space to have.Children grow up healthy and sound for protection, and it is the measure an of necessity that video is carried out the classification broadcast.Figure 12 has showed a video content classification reception application system schematic diagram that uses implicit expression semantic embedding system, user's semantic requirement among the figure, and the semantic table of interest SRT comes from the computer user Feature Semantics space that is gathered by computer system.
The SST semantic item is concentrated the relevant semanteme of having introduced about safety among the figure, such as the video content rating information, such as 18 years old among the figure, expression was fit to that the people watches more than 18 years old, corresponding to the right user semantic demand, the semantic item during namely Feature Semantics table SRT shows:<yrs〉[13].In like manner, in condition receiving system, carry out semantic test according to the watermark detection key, the semantic space table SST ' that detects carried out relevant operation with the conditional information of user side semantic requirement table SRT, as play, store, abandon, the action such as information.
3, " the community intelligent video user agent of colony service " applicating example
Figure 13 is one typical " the community intelligent video user agent of colony service ", can regard individual video storage intelligent agent similar application as, an intelligent proxy server of expression Local Area Network exit position in the dotted line frame, the application of this agency service can reduce the outlet volume of traffic of Local Area Network greatly, and it is congested and improve regional access speed that the mode of namely using edge storage reduces core network.Excavation and the analysis to the Visitor Logs of Community Group passed through in this buffer memory service, make up colony's Access Interest collection of illustrative plates, represent with GDNA, GDNA in fact is equal to general user side semantic requirement table SRT, but the semantic contents semantic spatial table SST ' that detects the outer extensive video resource of module analyzed area network passes through the computing of SST ' and GDNA and predicts the resource V that shoots straight o, use control decision C DThe control effect device is target V SSTnBe buffered among the regional memory NAS.A large amount of access of so regional user will directly have access in the storage nearby, only have a small amount of non-general character resource just need to access outside regional agency service, greatly reduce the data traffic of backbone network, improve regional user's service quality.
Although the above is described the illustrative embodiment of the present invention; so that those skilled in the art understand the present invention; but should be clear; the invention is not restricted to the scope of embodiment; to those skilled in the art; as long as various variations appended claim limit and the spirit and scope of the present invention determined in, these variations are apparent, all utilize innovation and creation that the present invention conceives all at the row of protection.

Claims (6)

1. the video content management method of a semantic-based hidden indexing is characterized in that, may further comprise the steps:
(1), at the information source end, go out the semantic space table of the video content that video data comprises according to video content and correlation attribute definition, the semantic space table comprises that content description, ownership describe and resource identification three parts semantic information;
(2), at the information source end, with the method for Information hiding semantic space table data are embedded in the video data;
(3), the stay of two nights end after video data moves, during use, utilize the inverse approach of Information hiding to extract semantic space table data in the video data, obtain the semantic space table of video data;
(4), at stay of two nights end, set up the application semantics demand schedule, and compare and computing with the semantic space table that extracts, independently determine the processing mode of the video data that this semantic space table is corresponding;
The described semantic space table of step (1) is by each data item, and namely semantic item forms;
Semantic item includes the corresponding semantic values of semantic item title and semantic item, and it is many-valued that semantic item can contain, and expressly distinguishing semantic key name and semantic item value with concrete character or separating character in the coding, and semantic item is many-valued;
Semantic item also includes the semantic item qualifier, the semantic item qualifier comprises to be modified key name and modifies a value, a semantic item can have zero to a plurality of modifications items, and each modification item also can have many-valued, distinguishes according to separating character in the plaintext coding of semantic space table.
2. the video content management method of semantic-based hidden indexing according to claim 1 is characterized in that, the described semantic space table of step (2) data are packet;
In the semantic space table packet, identifying this bag with 12 bits is semantic space table data stuffing bag; Indicate the version of current semantic space table data with 4 bits; Indicating whether compress cmpr with 1 bit, whether used compress mode with the semantic space table data among the explanation payload payload, is to be 1, is not 0; Representing to wrap big or small Packet Size with 15 bits, determine the bag size according to the occupy-place length computation of semantic space table payload, is 1 if whether compress cmpr value, calculating be length after compressing; Represent sequence number Sequence No with 16 bits, what sequence number represented is the sequence number of current semantic space table data;
The payload of semantic space table packet is the data of the expression semantic space predicative senses of a dictionary entry.
3. the video content management method of semantic-based hidden indexing according to claim 1, it is characterized in that, described semantic space table data also will be encrypted with public keys, after the semantic space table data packing after the encryption, hide embedding with video data.
4. the video content management method of semantic-based hidden indexing according to claim 1 is characterized in that, described video data is for looking audio file, described being embedded as of step (2):
At first order reads that to look audio file F be data stream V All, in reading process, differentiate simultaneously and tell target video code stream V FWith other code stream V Other, then target video code stream V FThe semantic space table SST packet utilization that has generated is fit to target video code stream V FInformation concealing method F MHide embedding operation, and output embeds the new video stream V that semantic space table SST data are arranged SST:
V SST=F M(V F,SST)
F wherein MBe selected information concealing method, the selection of carrying out with concrete video compression coding is relevant; New video code flow V SSTAgain with other code stream V OtherGo out to contain together the new file F ' of semantic space table SST data by synchronizing information Sync production control.
5. the video content management method of semantic-based hidden indexing according to claim 1 is characterized in that, described video data is the MPEG2 video code flow, described being embedded as of step (2):
At first read video codeword data stream to frame buffer zone, then identify the video group, next carry out the discrete cosine transform coefficient modifying at key frame, then continue picture bar and macro block among the identification MPEG2, extract the discrete cosine transform coefficient of each macro block, finish decoding length changeable code;
After getting access to macro block discrete cosine transform coefficient, according to information concealing method F MJudge the possibility of macro block embed watermark, and according to this information, one one of semantic space table SST packet is extracted the discrete cosine transform coefficient is made amendment; After modification is finished, in the P frame, the discrete cosine transform difference value coefficient of the macro block of inter prediction is made amendment, the same when being modified to unmodified I frame; Then amended data are carried out variable length coding, and be written as new video code flow V SST, next judge whether arrive the end of file of input video stream, if do not have, just continue to read video codeword data stream, continue to embed semantic space table SST packet.
6. the video content management method of semantic-based hidden indexing according to claim 1, it is characterized in that, in the described embedding of step (2), after having embedded a semantic space table SST packet, continuation embeds second, the 3rd semantic space table SST packet in ensuing video data, and each semantic space table SST packet is carried out auto-increment number, until the end of video data.
CN 201110090360 2011-04-12 2011-04-12 Video content management method based on semantic hidden indexing Expired - Fee Related CN102156734B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110090360 CN102156734B (en) 2011-04-12 2011-04-12 Video content management method based on semantic hidden indexing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110090360 CN102156734B (en) 2011-04-12 2011-04-12 Video content management method based on semantic hidden indexing

Publications (2)

Publication Number Publication Date
CN102156734A CN102156734A (en) 2011-08-17
CN102156734B true CN102156734B (en) 2013-05-29

Family

ID=44438233

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110090360 Expired - Fee Related CN102156734B (en) 2011-04-12 2011-04-12 Video content management method based on semantic hidden indexing

Country Status (1)

Country Link
CN (1) CN102156734B (en)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103179430A (en) * 2011-12-20 2013-06-26 中国电信股份有限公司 Method, device and server for audio and video content transcoding on basis of cloud computing
CN102547407B (en) * 2011-12-30 2014-07-02 成都捷成优联信息技术有限公司 Method for achieving correlation between text and video by embedding text content in video
CN102724554B (en) * 2012-07-02 2014-06-25 西南科技大学 Scene-segmentation-based semantic watermark embedding method for video resource
CN103678308A (en) * 2012-09-03 2014-03-26 许丰 Intelligent navigation player
CN103678309A (en) * 2012-09-03 2014-03-26 许丰 Intelligent indexing navigation system
CN102946541A (en) * 2012-12-06 2013-02-27 四川长虹电器股份有限公司 Video content supervision method based on digital watermarking
US9514501B2 (en) 2013-06-14 2016-12-06 Tencent Technology (Shenzhen) Company Limited Systems and methods for multimedia-processing
CN104244112B (en) * 2013-06-14 2018-12-07 腾讯科技(深圳)有限公司 A kind of multi-media processing method, device and server
CN107026887B (en) * 2016-02-02 2019-12-06 上海交通大学 rapid information interaction method and network transmission method in multimedia system
CN106231348B (en) * 2016-09-19 2019-06-11 浙江宇视科技有限公司 A kind of back method of GOP data, device and system
CN107038221B (en) * 2017-03-22 2020-11-17 杭州电子科技大学 Video content description method based on semantic information guidance
CN110489450A (en) * 2019-08-08 2019-11-22 武汉东湖大数据交易中心股份有限公司 A kind of method of high amount of traffic inquiry
CN111294340B (en) * 2020-01-17 2022-05-17 河南芯盾网安科技发展有限公司 Encryption information steganography method based on zero-width characters

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1301460A (en) * 1998-04-21 2001-06-27 索拉纳技术开发公司 Multimedia adaptive scrambling system (MASS)
WO2002101588A1 (en) * 2001-06-11 2002-12-19 Matsushita Electric Industrial Co., Ltd. Content management system
CN1488223A (en) * 2000-12-22 2004-04-07 vpr����˾ Buried data stream in a wireless home network
CN101577619A (en) * 2008-05-08 2009-11-11 吴志军 Real-time speech secret communication system based on information hiding

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1301460A (en) * 1998-04-21 2001-06-27 索拉纳技术开发公司 Multimedia adaptive scrambling system (MASS)
CN1488223A (en) * 2000-12-22 2004-04-07 vpr����˾ Buried data stream in a wireless home network
WO2002101588A1 (en) * 2001-06-11 2002-12-19 Matsushita Electric Industrial Co., Ltd. Content management system
CN101577619A (en) * 2008-05-08 2009-11-11 吴志军 Real-time speech secret communication system based on information hiding

Also Published As

Publication number Publication date
CN102156734A (en) 2011-08-17

Similar Documents

Publication Publication Date Title
CN102156734B (en) Video content management method based on semantic hidden indexing
US20230319229A1 (en) System and method for modifying media streams using metadata
Nack et al. Everything you wanted to know about MPEG-7. 1
CN100419748C (en) Supporting advanced coding formats in media files
CN101675435B (en) Media stream recording into a reception hint track of a multimedia container file
CN1650627B (en) Method and apparatus for supporting AVC in MP4
CN103069769B (en) For the special-effect mode transmitted through the network crossfire of decoded video data
CN1764974B (en) Storage medium storing multimedia data, and method and apparatus for reproducing multimedia data
CN104380754A (en) Method and apparatus for transmitting adaptive media structure in multimedia system
CN101802823A (en) Segmented metadata and indexes for streamed multimedia data
CN102132562A (en) Method and apparatus for track and track subset grouping
JP2008136204A (en) System and method for streaming, receiving and processing flex mux stream
CN101842786A (en) Fast and editing-friendly sample association method for multimedia file formats
EP2614653A1 (en) A method and apparatus for adaptive streaming
CN100369480C (en) Multiplexing device and demultiplexing device
AU2007243966B2 (en) Method and apparatus for re-constructing media from a media representation
CN109889542B (en) Method and apparatus for receiving media data in a broadcast network
US8723703B2 (en) Method and apparatus for encoding and decoding structured data
CN105049904A (en) Play method and play device of multi-media files
CN103747065A (en) Client HTTP retrieval full-index container format media resource time slice method
KR20130107266A (en) Method and apparatus for encapsulating coded multi-component video
KR20090017027A (en) Method for creating and reproducing media metadata based on media object and apparatus thereof
CN104023245A (en) Video live stream quick-editing and accepting system based on B/S framework and realization method
KR20050006565A (en) System And Method For Managing And Editing Multimedia Data
JP2006519422A (en) How to encode structured documents

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C53 Correction of patent of invention or patent application
CB03 Change of inventor or designer information

Inventor after: Xing Ling

Inventor after: Li Maofeng

Inventor after: Chen Miao

Inventor after: Ma Jianguo

Inventor after: Ma Qiang

Inventor before: Ma Jianguo

Inventor before: Li Maofeng

Inventor before: Xing Ling

COR Change of bibliographic data

Free format text: CORRECT: INVENTOR; FROM: MA JIANGUO LI MAOFENG XING LING TO: XING LING LI MAOFENG CHEN MIAO MA JIANGUO MA QIANG

C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20130529