CN104572952B - The recognition methods of live multimedia file and device - Google Patents

The recognition methods of live multimedia file and device Download PDF

Info

Publication number
CN104572952B
CN104572952B CN201410849032.9A CN201410849032A CN104572952B CN 104572952 B CN104572952 B CN 104572952B CN 201410849032 A CN201410849032 A CN 201410849032A CN 104572952 B CN104572952 B CN 104572952B
Authority
CN
China
Prior art keywords
multimedia file
live
information
finger print
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201410849032.9A
Other languages
Chinese (zh)
Other versions
CN104572952A (en
Inventor
谭傅伦
许泽军
王晓萌
王英杰
袁斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Zhangmen Science and Technology Co Ltd
Original Assignee
LeTV Information Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by LeTV Information Technology Beijing Co Ltd filed Critical LeTV Information Technology Beijing Co Ltd
Priority to CN201410849032.9A priority Critical patent/CN104572952B/en
Publication of CN104572952A publication Critical patent/CN104572952A/en
Application granted granted Critical
Publication of CN104572952B publication Critical patent/CN104572952B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43—Querying
    • G06F16/432—Query formulation
    • G06F16/433—Query formulation using audio data

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Recognition methods and device the invention discloses a kind of live multimedia file.This method includes：The characteristic information of live multimedia file present period is obtained according to the real-time stream of the live multimedia file of input；Multimedia recording to be updated is positioned in property data base according to the identification information of live multimedia file；The feature samples in multimedia recording to be updated are updated according to the characteristic information of live multimedia file present period；The identification request of identification destination multimedia file is received, match cognization asks the feature samples in the characteristic information and property data base of the destination multimedia file included, to position the corresponding multimedia recording of destination multimedia file；Obtain the identification information of the corresponding multimedia file of destination multimedia file.By means of the invention it is possible to identify live video.

Description

The recognition methods of live multimedia file and device
Technical field
The present invention relates to video identification technology field, specifically, more particularly to a kind of identification of live multimedia file Method and device.
Background technology
Current video search mode, usually used is that video " keyword " scans for.This not only requires user to know Know the relevant information of the video, while also require search service to provide to safeguard in time with video correspondingly " keyword " Database.And in fact, we can usually suffer from such embarrassment：Met by chance before streets and lanes or television set one section it is interesting Video, but we are not familiar with may not even be aware that the information of this section of video, let alone search this section by " keyword " and regard Frequency.
Thus, just come into being based on voice recognition video under the promotion of this actual demand.Know based on sound In the technology of other video, when user needs to identify certain video, first by mobile terminal (such as：Smart mobile phone) sound pick-up outfit obtain The acoustic information in video is taken, by the property data base progress in the characteristic and cloud server that reflect the acoustic information Match somebody with somebody, and matching result (video flowing or the relevant information of video) is returned into mobile terminal.
But video file has the characteristics that quickly renewal, Quick thread, or even many video files using network Live form, so the video that user needs to identify is often just in live video.And in the above method of the prior art In, cloud server can just be built special after the complete video of video source generation is got according to the corresponding audio-frequency information of video Database is levied, therefore, the method for the prior art does not identify live video.
The problem of cannot identifying live video for the prior art, not yet propose effective solution method at present.
The content of the invention
It is existing to solve it is a primary object of the present invention to provide recognition methods and the device of a kind of live multimedia file Technology cannot identify the problem of live video.
According to one aspect of the present invention, there is provided a kind of recognition methods of live multimedia file.
The recognition methods of live multimedia file according to the present invention includes：According to the reality of the live multimedia file of input When data flow obtain live multimedia file present period characteristic information；According to the identification information of live multimedia file in spy Multimedia recording to be updated is positioned in sign database, wherein, property data base is used to store at least one multimedia recording, more Media recording includes feature samples, the identification information corresponding with feature samples of multimedia file, the time span of feature samples For first scheduled time；Updated according to the characteristic information of live multimedia file present period in multimedia recording to be updated Feature samples；Receive the identification request of identification destination multimedia file, the destination multimedia file that match cognization request includes Characteristic information and property data base in feature samples, to position the corresponding multimedia recording of destination multimedia file；Obtain The identification information of the corresponding multimedia file of destination multimedia file.
Further, characteristic information is the finger print information of the voice data of multimedia file, according to live more matchmakers of input The real-time stream of body file obtains the characteristic information of live multimedia file present period, including：Obtained according to real-time stream The voice data of the present period of cut-off playing multimedia file；The voice data of present period is divided into sequentially in time Multiple audio fragments of two scheduled times, wherein, second scheduled time was less than for first scheduled time；And each audio piece of extraction The finger print information of section, to obtain the characteristic information of the present period of live multimedia.
Further, feature samples are the finger print information of n audio fragment, the spy of the present period of live multimedia file Reference is ceased for the finger print information of m audio fragment, m<The time span of n, n audio fragments was first scheduled time, according to straight The feature samples that the characteristic information of playing multimedia file is updated in multimedia recording to be updated include：Delete more matchmakers to be updated The m earliest finger print information of feature samples in body record；M finger print information of the present period of live multimedia file is pressed Time sequencing is placed in the feature samples of multimedia recording to be updated.
Further, updated according to the characteristic information of live multimedia file present period in multimedia recording to be updated Feature samples, specifically include：Step S1：Feature pointer is directed toward the in the characteristic information of live multimedia file present period One finger print information, and timer is reset and starts feature extraction timing；Step S2：Obtain the fingerprint letter that feature pointer is directed toward Breath；Step S3：Extraction and the feature samples of the corresponding multimedia recording of identification information of live multimedia, it is special to obtain first Levy sample；Step S4：The finger print information that feature pointer is directed toward is spliced to the end of fisrt feature sample, to obtain second feature Sample；Step S5：A finger print information is deleted from the starting of second feature sample；Step S6：Judging the time in timer is No to reach for the 3rd scheduled time, if not up to the 3rd scheduled time, feature pointer is directed toward next finger print information, and repeats Step S2 to S6；If reaching for the 3rd scheduled time, multi-media tag in multimedia recording is replaced with obtained second feature sample Corresponding feature samples, wherein, the 3rd scheduled time was the reproduction time of the corresponding multimedia file of m finger print information.
Further, extracting the finger print information of audio fragment includes：Merge the left channel data and R channel of audio fragment Data, to obtain the stereo data of audio fragment；And the time-frequency characteristics data of the stereo data of extraction audio fragment are made For the finger print information of audio fragment.
Further, the characteristic information for the destination multimedia file that identification request includes working as live multimedia file N number of finger print information of preceding period, a finger print information in N number of finger print information is in N number of stereo data of destination multimedia A stereo data time-frequency characteristics data, wherein, i-th of stereo data in N number of stereo data for si '= Ai ' * l '+bi ' * r ', ai '+bi '=1, l ' are the left channel data of the present period of live multimedia file, and r ' is live more The right data of the present period of media file, ai ' and bi ' are default parameter, i=1,2,3 ... N.In the method, Feature samples in the characteristic information and property data base of the destination multimedia file included with identification request, to position target The corresponding multimedia recording of multimedia file includes：By each finger print information of destination multimedia file respectively with property data base In feature samples matching, obtain the matching rate of each finger print information；Will be more where the corresponding feature samples of maximum matching rate Media recording is as the corresponding multimedia recording of destination multimedia file.
According to one aspect of the present invention, there is provided a kind of identification device of live multimedia file.
The identification device of live multimedia file according to the present invention includes：Acquisition module, for according to the live of input The real-time stream of multimedia file obtains the characteristic information of live multimedia file present period；Locating module, for basis The identification information of live multimedia file positions multimedia recording to be updated in property data base, wherein, property data base For storing at least one multimedia recording, multimedia recording includes the feature samples, corresponding with feature samples of multimedia file Identification information, the time span of feature samples was first scheduled time；Update module, for being worked as according to live multimedia file The characteristic information of preceding period updates the feature samples in multimedia recording to be updated；Matching module, target is identified for receiving The identification request of multimedia file, the characteristic information and property data base of the destination multimedia file that match cognization request includes In feature samples, to position the corresponding multimedia recording of destination multimedia file；Identification module, for obtaining destination multimedia The identification information of the corresponding multimedia file of file.
Further, characteristic information is the finger print information of the voice data of multimedia file, and acquisition module includes：Audio number According to acquisition module, the voice data of the present period for obtaining live multimedia file according to real-time stream；Audio fragment Split module, for the voice data of present period to be divided into multiple audio pieces of second scheduled time sequentially in time Section, wherein, second scheduled time was less than for first scheduled time；And finger print information extraction module, for extracting each audio piece The finger print information of section, to obtain the characteristic information of the present period of live multimedia.
Further, feature samples are the finger print information of n audio fragment, the spy of the present period of live multimedia file Reference is ceased for the finger print information of m audio fragment, m<The time span of n, n audio fragments was first scheduled time, updated mould Block includes：Removing module, for deleting the m earliest finger print information of feature samples in multimedia recording to be updated；Addition Module, for m finger print information of the present period of live multimedia file to be placed in multimedia to be updated in chronological order In the feature samples of record.
Further, update module specifically performs following steps：
Step S1：Feature pointer is directed toward first fingerprint letter in the characteristic information of live multimedia file present period Breath, and timer is reset and starts feature extraction timing；Step S2：Obtain the finger print information that feature pointer is directed toward；Step S3：Carry The feature samples with the corresponding multimedia recording of the identification information of live multimedia are taken, to obtain fisrt feature sample；Step S4：The finger print information that feature pointer is directed toward is spliced to the end of fisrt feature sample, to obtain second feature sample；Step S5：A finger print information is deleted from the starting of second feature sample；Step S6：Judge whether the time in timer reaches the 3rd The scheduled time, if not up to the 3rd scheduled time, feature pointer is directed toward next finger print information, and repeats step S2 extremely S6；If reaching for the 3rd scheduled time, the corresponding spy of multi-media tag in multimedia recording is replaced with obtained second feature sample Sample is levied, wherein, the 3rd scheduled time was the reproduction time of the corresponding multimedia file of m finger print information.
Further, finger print information extraction module includes：Stereo data synthesis module, for merging a left side for audio fragment Channel data and right data, to obtain the stereo data of audio fragment；And time-frequency characteristics extraction module, for extracting Finger print information of the time-frequency characteristics data of the stereo data of audio fragment as audio fragment.
Further, the characteristic information for the destination multimedia file that identification request includes working as live multimedia file N number of finger print information of preceding period, a finger print information in N number of finger print information is in N number of stereo data of destination multimedia A stereo data time-frequency characteristics data, wherein, i-th of stereo data in N number of stereo data for si '= Ai ' * l '+bi ' * r ', ai '+bi '=1, l ' are the left channel data of the present period of live multimedia file, and r ' is live more The right data of the present period of media file, ai ' and bi ' are default parameter, i=1,2,3 ... N, in the apparatus, Include with module：Matching rate determining module, for by each finger print information of destination multimedia file respectively with property data base In feature samples matching, obtain the matching rate of each finger print information；Multimedia recording determining module, for by maximum matching rate Multimedia recording where corresponding feature samples is as the corresponding multimedia recording of destination multimedia file.
By the present invention, the characteristic information of a default characteristic library storage live multimedia, specifically, in this feature At least one multimedia recording is stored in database, multimedia recording includes the feature samples and feature samples of multimedia file Corresponding identification information, and the time span of feature samples was first scheduled time, was there is the real-time of live multimedia file Data flow input when, first according to the real-time stream of the live multimedia file of input obtain live multimedia file it is current when The characteristic information of section, then positions multimedia to be updated according to the identification information of live multimedia file in property data base Record, the feature samples in multimedia recording to be updated are updated according to the characteristic information of live multimedia file present period, So as to ensure that live multimedia file currently newest characteristic information is stored in property data base.Destination multimedia is identified receiving During the identification request of file, in the characteristic information and property data base of the destination multimedia file that match cognization request includes Feature samples, to position the corresponding multimedia recording of destination multimedia file, it is corresponding more then to obtain destination multimedia file The identification information of media file, to achieve the purpose that to identify destination multimedia file, solving cannot identify directly in the prior art The problem of broadcasting video.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention, And can be practiced according to the content of specification, and in order to allow above and other objects of the present invention, feature and advantage can Become apparent, below especially exemplified by the embodiment of the present invention.
Brief description of the drawings
By reading the detailed description of hereafter preferred embodiment, it is various other the advantages of and benefit it is common for this area Technical staff will be clear understanding.Attached drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention Limitation.And in whole attached drawing, identical component is denoted by the same reference numerals.In the accompanying drawings：
Fig. 1 is according to embodiments of the present invention one method flow diagram；
Fig. 2 is according to embodiments of the present invention two method flow diagram；
Fig. 3 is according to embodiments of the present invention three method flow diagram；
Fig. 4 is according to embodiments of the present invention four system schematic；
Fig. 5 is according to embodiments of the present invention four terminal block diagram；
Fig. 6 is according to embodiments of the present invention four video frequency searching server block diagram；
Fig. 7 is according to embodiments of the present invention four fingerprint management server block diagram；
Fig. 8 is according to embodiments of the present invention four video management server block diagram；And
Fig. 9 is according to embodiments of the present invention five device block diagram.
Embodiment
The present invention will be further described with reference to the accompanying drawings and detailed description.It is pointed out that do not conflicting In the case of, the feature in embodiment and embodiment in the application can be mutually combined.
An embodiment of the present invention provides the recognition methods of live multimedia file, in the method, preset feature data storehouse, During live multimedia file is live, property data base is updated according to the real-time stream of live multimedia file, with Ensure to be stored with the newest information of current live multimedia file in property data base.When user needs to identify that live target is more During media file, the characteristic information of destination multimedia file is matched with the feature samples in preset feature data storehouse, if The characteristic information of destination multimedia file and a certain feature samples successful match, then pass through the corresponding identification information of this feature sample It can reach the purpose of identification destination multimedia file.
It should be noted that the live time of live multimedia file and the generation time of real-time stream usually have one A time difference, the live time (time for being sent to user) of live multimedia file are later than the real-time of live multimedia file The generation time of data flow, the embodiment of the present invention are based on this time difference so that feature samples are more in property data base The new time earlier than or be synchronized with live time of live multimedia file so that property data base can keep storage current live The characteristic information of multimedia file, and then live destination multimedia file can be matched in property data base, reach knowledge The purpose of other destination multimedia file.
Specifically, one or more multimedia recording is stored with this feature database, every multimedia recording corresponds to one A live multimedia file, for example, the multimedia file is video, then every multimedia recording corresponds to a live video；Should Multimedia file is audio, then every multimedia recording corresponds to a live audio.Every multimedia recording is live including one The identification information of the feature samples of multimedia file and the live multimedia file, wherein, feature samples are by the preset set time The characteristic information composition of the multimedia file of length, such as feature samples are the characteristic information group of the video of set time length Into；Identification information is the information that can recognize different live multimedias.
When the real-time stream input for having live multimedia file, it is live that this is further got according to real-time stream The characteristic information of multimedia file present period, it is straight to find this by the identification information of live multimedia file in property data base The corresponding multimedia recording of playing multimedia file, then the feature samples in the multimedia recording are updated by characteristic information, so as to protect Live multimedia file currently newest characteristic information is stored in characteristics of syndrome database.
When receiving the identification request of identification destination multimedia file, further according to the identification more matchmakers of acquisition request target The characteristic information of body file, the then characteristic information by destination multimedia file and the feature samples phase in property data base Match somebody with somebody, a multimedia recording is navigated to by the feature samples matched, and then obtain the identification information in the multimedia recording, most The recognition result of destination multimedia file can be obtained by the identification information eventually.
Any recognition methods of the embodiment of the present invention is used equally in the searching method of live multimedia file.Live more In the searching method of media file, the more matchmakers of target are recognized by the recognition methods of the live multimedia file of the embodiment of the present invention Body file, namely after obtaining the identification information of destination multimedia file, the chain of destination multimedia file is found by identification information Connect, and then search destination multimedia file.
For example, mobile phone user is seen just on outdoor advertising screen in certain live video, it would be desirable to by mobile phone searching simultaneously The video is played to, at this time, user's operation mobile phone terminal, mobile phone terminal records live video, and live further according to record regards The voice data generation target video identification request of frequency is sent to cloud server, and cloud server is using the embodiment of the present invention After recognition methods recognizes target video, the identification information of target video can be back to mobile phone end by a kind of situation cloud server End, mobile phone terminal find the link of target video further according to the identification information of target video, so by searching for link Play video；Another situation cloud server returns after can finding the link of target video by the identification information of target video Mobile phone terminal is back to, and then mobile phone terminal plays video by the link.
Various embodiments provided by the present invention will be described in detail below.
Embodiment one
The embodiment one provides a kind of embodiment of the recognition methods of live multimedia file, the side which provides The executive agent of method is cloud server, and send the request of destination multimedia file identification is user terminal.Wherein, take beyond the clouds It is engaged in preset feature data storehouse in device, the newest information of current live multimedia file being stored with property data base.When user is whole When end needs to identify live destination multimedia file, cloud server receives identification request, by destination multimedia file Characteristic information is matched with the feature samples in preset feature data storehouse, passes through the corresponding mark of the feature samples of successful match Information identifies destination multimedia file.
Fig. 1 is according to embodiments of the present invention one method flow diagram, as shown in Figure 1, this method specifically includes following steps S102 to step S116, wherein, step S102 realizes the renewal of property data base, step S110 to step S116 to step S108 Realize and destination multimedia file is identified according to property data base.
Step S102：Obtain the real-time stream and identification information of live multimedia file.
Cloud server and the background data base server of live source are in communication with each other, and get live multimedia file in real time Real-time stream, while real-time stream is obtained, can also get the identification information of live multimedia file.At this Multimedia file can be video or audio, then the real-time stream got is video flowing or audio stream accordingly.It is live The identification information of multimedia file for can multiple multimedia files (including live multimedia file and non-live multimedia text Part) in unique identification and determine the live multimedia file information.
For live multimedia file, a live source the same time can only a live multimedia file, and And the identity information of live source has the characteristics that simple, unique and identification is high, thus, the identification information of live multimedia file The preferably identity information of live source, for example, when live multimedia file is video, its identification information is broadcasting live video The channel data of video source, specific such as live multimedia file is news hookup, its identification information is " CCTV1 "；And for example, it is live When multimedia file is audio, its identification information is specific such as live more matchmakers to play the channel data of the audio-source of live audio Body file broadcasts chain broadcast for live storytelling, its identification information is " Central People's Broadcasting Station ".
Step S104：The characteristic information of live multimedia file present period is obtained according to real-time stream.
After cloud server gets the real-time stream of live multimedia file every time, using default feature extraction mould Block, carries out data processing to real-time stream, to extract the characteristic of real-time stream, it is current to obtain live multimedia file The characteristic information of period.
For example, getting the video stream data of live video, video stream data is handled using characteristic extracting module, Audio-frequency fingerprint is corresponded to extract live video, obtains the characteristic information of live video present period.
Step S106：Multimedia to be updated is positioned in property data base according to the identification information of live multimedia file Record.
Wherein, property data base is stored with a plurality of multimedia recording, and each multimedia recording includes multimedia file Identification information two parts of feature samples and multimedia file corresponding with feature samples, wherein, the time span of feature samples It is fixed, after being updated every time, characteristic information respective change in feature samples is the characteristic information after renewal, but feature The time span of sample does not change, straight with ensure the feature samples of characteristic library storage always newest a period of time The characteristic information of playing multimedia file.
After the identification information that step S102 gets live multimedia file, in step S106, characteristic is searched According to storehouse, can be navigated in property data base with the corresponding multimedia recording of the identification information of live multimedia file, this is more Media recording is multimedia recording to be updated.
Step S108：The feature sample in multimedia recording to be updated is updated according to the characteristic information of live multimedia file This.
After navigating to multimedia recording to be updated, multimedia recording is updated using the characteristic information of live multimedia file In feature samples.During renewal, when the time span of the characteristic information of live multimedia file present period is greater than or equal to spy When levying the time span of sample, the characteristic information in feature samples can all be updated；When live multimedia file present period Characteristic information time span be less than feature samples time span when, part renewal can be carried out.
No matter use which type of update mode, to avoid data collision, current property data base can be backed up, The property data base of backup is updated, then with the former property data base of property data base covering of the backup after renewal.
Beyond the clouds in server, the data flow of real-time reception difference live multimedia file, passes through step S102 to step S108 timely updates property data base, so as to be directed to arbitrary live multimedia file, is always stored with property data base Characteristic information in current newest a period of time.
Step S110：Receive the identification request of identification destination multimedia file.
Identification request can be sent by user terminal, and user needs to identify certain (this Shen just in live multimedia file This to be identified is just please defined as destination multimedia file in live multimedia file), obtain the destination multimedia file A period of time in data flow, when the data flow data amount in this time is larger, can user terminal extract target it is more The characteristic information of media file, is encapsulated as identification request by the characteristic information extracted and sends to cloud server, cloud service Device receives the identification request of the characteristic information comprising destination multimedia file, to achieve the purpose that to reduce volume of transmitted data；When this When data flow data amount is smaller, directly data stream can be sent to cloud server, cloud server for identification request and connect The identification request of the data flow of packet receiving file containing destination multimedia, to reduce the requirement to user terminal data disposal ability.
Step S112：According to the characteristic information of identification acquisition request destination multimedia file.
Cloud server, according to identification acquisition request characteristic information, specifically, works as identification after identification request is received Request includes the characteristic information of destination multimedia file, and cloud server can obtain destination multimedia by parsing identification request The characteristic information of file；When identification request includes the data flow of destination multimedia file, cloud server calls default feature Extraction module, carries out data processing, to extract the characteristic information of destination multimedia file to the data flow of destination multimedia file.
Step S114：The feature samples in the characteristic information and property data base of destination multimedia file are matched, with positioning The corresponding multimedia recording of destination multimedia file.
Cloud server is after the characteristic information of destination multimedia file is obtained, with each feature samples in property data base Matched, if the characteristic information of destination multimedia file and a certain feature samples successful match, matching terminate, this feature sample Multimedia recording where this is the corresponding multimedia recording of destination multimedia file.
Step S116：The identification information in the corresponding multimedia recording of destination multimedia file is obtained, to identify that target is more Media file.
After navigating to the corresponding multimedia recording of destination multimedia file, it is by the identification information in the multimedia recording It can reach the purpose of identification destination multimedia file.For example, the identification information in the multimedia recording is " CCTV5 ", then target Multimedia file is CCTV5 just in live video.
In the recognition methods for the live multimedia file that the embodiment provides, a preset property data base is straight to store The feature samples and identification information of playing multimedia file, and by obtaining the real-time stream real-time update and dimension on live source backstage This feature database is protected, when there is destination multimedia file to need identified, is existed according to the characteristic information of destination multimedia file The identification information of corresponding live multimedia file is found in property data base, achievees the purpose that to identify live multimedia file. To sum up, the recognition methods of the live multimedia file provided using the embodiment, can identify live multimedia file in real time, So as to can also identify live video in real time.
Embodiment two
The embodiment two provides a kind of preferred embodiment of the recognition methods of live multimedia file, the embodiment be Further preferred embodiment, specifically thes improvement is that on the basis of embodiment one：
First, feature of the finger print information of the voice data of multimedia file as multimedia file is used in the embodiment Information, that is, the feature samples stored in property data base are the finger print information of voice data, the destination multimedia got The characteristic information of file also mutually should be the finger print information of the voice data of destination multimedia file, from regardless of whether in renewal characteristic During according to storehouse, or when obtaining the characteristic information of destination multimedia file, it is both needed to obtain the voice data of multimedia file, extraction The finger print information of voice data, significantly reduces the volume of transmitted data of system, reduces the consumption of flow, increases recognition methods Availability.Specifically, the voice data of the present period of live multimedia file is divided into multiple audio fragments, extraction is each The finger print information of audio fragment, so that fingerprint of the characteristic information of live multimedia file present period by the plurality of audio fragment Information structure.
Further, the time span (namely time span of n audio fragment) of feature samples is more than in the embodiment The time span (namely time span of m audio fragment) of the characteristic information of the live multimedia file got, is updating Property data base (when the feature samples stored in property data base are finger print information, is also known as fingerprint number by property data base According to storehouse, feature samples are known as sample fingerprint), the corresponding finger print information of live multimedia file is added in sample fingerprint, And delete the finger print information of the time span earliest in sample fingerprint, time span is addition characteristic information, so that a side Face ensure that the length of sample fingerprint, when matching sample fingerprint, ensure that any time period live destination multimedia file Effective identification, the real-time update of sample fingerprint is on the other hand ensure that, to ensure the real-time of identification.
Further, when fingerprint database updates, using the update mode of backup table combination timer, on the one hand can Data collision is avoided, on the other hand can control the update cycle according to being actually needed.
Specifically, Fig. 2 is according to embodiments of the present invention two method flow diagram, as shown in Fig. 2, this method specifically include with Lower step S202 to step S216.
Step S202：Obtain the real-time stream and identification information of live multimedia file.
Step S204：The voice data of live multimedia file present period is obtained according to real-time stream.
Cloud server can further obtain live more matchmakers when getting the real-time stream of live multimedia file The voice data of body file present period you, for example, the time span of the present period was the 3rd scheduled time, call audio to carry Modulus block, extracts the voice data in data flow, and the voice data obtained from is the data of the 3rd scheduled time.
Preferably, when getting voice data, format conversion, the voice data that will be got can be carried out to voice data The data of unified form are converted to, to facilitate subsequent treatment；Can also denoising be carried out to voice data, for example with sliding window The technology of denoising, removes " spine " in voice data；Can also down-sampling be carried out to voice data, before data precision is ensured Put, the amount of storage and operand of data can be reduced.
Step S206：The voice data of present period is divided into m audio fragment sequentially in time.
After the voice data for getting for the 3rd scheduled time, audio segmentation module is called in chronological order by finite state Automat For the audio fragment that m time span is the second scheduled time t.
It should be noted that when performing step S204 and step S206, also can be in the following ways：First by present period Interior real-time stream case time sequencing is divided into m data fragment, then obtains the voice data of each data slot and obtain sound Frequency fragment.Which is mutually equal with above-mentioned steps S204 and step S206, within the protection domain of the application.
Step S208：The finger print information of each audio fragment is extracted, to obtain the spy of live multimedia file present period Reference ceases.
After obtaining the audio fragment that m time span is t, the finger print information of each audio fragment is extracted, wherein, Suo Youyin The finger print information of frequency fragment forms the characteristic information of live multimedia file present period, so that when live multimedia file is current The characteristic information of section includes m finger print information, and the time span of this feature information mutually should be for the 3rd scheduled time.
The characteristic information of live multimedia file present period is by chronological sequence tactic m finger print information group Into first finger print information is an earliest finger print information, and m-th of finger print information is a newest finger print information.
Wherein, which is preferably stereo data, meanwhile, the characteristic information of destination multimedia file is also solid The finger print information of sound data, the unification of the two data source can improve matched accuracy.
When extracting the finger print information of audio fragment, the temporal signatures of audio, such as the width of extraction audio fragment can extract Value is used as finger print information, also can extract the time-frequency characteristics of audio, former data processing speed is fast, and the latter's anti-noise ability is stronger.
Step S210：Multimedia to be updated is positioned according to the identification information of live multimedia file in fingerprint database to remember Record.
Wherein, which is stored with a plurality of multimedia recording, and each multimedia recording includes multimedia file Feature samples and multimedia file corresponding with feature samples identification information two parts, wherein, feature samples are by temporally N finger print information of sequencing arrangement forms, and first finger print information in feature samples is an earliest finger print information, Last finger print information is a newest finger print information.The time span of each finger print information is t, the n finger print information Time span be the first scheduled time T, and m<N, or the 3rd scheduled time were less than for first scheduled time, namely live more The time span of the characteristic information of media file present period is less than the time span of sample fingerprint.
After identification information is obtained, the multimedia recording including the identification information can be navigated in fingerprint database.
Step S212：Delete the m earliest finger print information of sample fingerprint in multimedia recording to be updated.
Multimedia recording to be updated is being navigated to, the sample fingerprint in the multimedia recording is being updated, is being updated When, the preceding m finger print information in sample fingerprint is deleted, that is, current m earliest finger print information in sample fingerprint is deleted Remove.
Step S214：M finger print information of live multimedia file present period is placed in chronological order to be updated In multimedia recording in sample fingerprint.
In renewal, m finger print information of live multimedia file present period is added to the end of sample fingerprint, from And the finger print information added is newest finger print information in sample fingerprint.
It should be noted that in this embodiment, step S212 can be first carried out, and it is rear to perform step S214, it can also first carry out Step S214, it is rear to perform step S212.Wherein, when realizing step S214 and step S212, following specific method can be used Step is realized：
Step S1：Feature pointer is directed toward first fingerprint letter in the characteristic information of live multimedia file present period Breath, and timer is reset and starts feature extraction timing；
Step S2：Obtain the finger print information that feature pointer is directed toward；
Step S3：Extraction and the feature samples of the corresponding multimedia recording of identification information of live multimedia, to obtain Fisrt feature sample；
Step S4：The finger print information that feature pointer is directed toward is spliced to the end of fisrt feature sample, it is special to obtain second Levy sample；
Step S5：A finger print information is deleted from the starting of second feature sample；
Step S6：Judge whether the time in timer reached for the 3rd scheduled time, it is special if not up to the 3rd scheduled time Levy pointer and be directed toward next finger print information, and repeat step S2 to S6；If reaching for the 3rd scheduled time, with second obtained Feature samples replace the corresponding feature samples of multi-media tag in multimedia recording, wherein, the 3rd scheduled time believed for m fingerprint Cease the reproduction time of corresponding multimedia file.
Beyond the clouds in server, the data flow and identification information of real-time reception difference live multimedia file, pass through step S202 to step S214 timely updates fingerprint database, so that arbitrary live multimedia file is directed to, in fingerprint database In be always stored with the finger print information of live multimedia file in current newest a period of time.
Step S216：The identification request of identification destination multimedia file is received, and identifies destination multimedia file.
Specifically, step S216 is including the step S110 in above-described embodiment to step S116, and details are not described herein again.
Embodiment three
The embodiment three provides a kind of embodiment of the recognition methods of live multimedia file, which is to implement Further preferred embodiment, specifically thes improvement is that on the basis of example two：
First, the audio fragment of live multimedia file is to be merged to form by left channel data and right data, accordingly Ground, the characteristic information of destination multimedia file are also the finger print information of stereo data, and are merging left and right sound channels data During stereo data, weight parameter is set, so as to adjusting left and right acoustic channels data institute in stereo data according to being actually needed The proportion accounted for.
Further, when building the characteristic information of destination multimedia file, by setting multigroup weighted data, by target The left and right acoustic channels data of multimedia file are converted into multigroup stereo data, extract the corresponding fingerprint letter of every group of stereo data Breath, so that the characteristic information of destination multimedia file includes multigroup finger print information., will when carrying out destination multimedia file identification Every group of finger print information matches respectively with the sample fingerprint in fingerprint database, by where the corresponding sample fingerprint of maximum matching rate Multimedia recording as the corresponding multimedia recording of destination multimedia file, increase the accuracy of identification.
Further, when extracting the finger print information of stereo data of audio fragment, or in extraction destination multimedia During the time-frequency characteristics data of every group of stereo data of file, the time-frequency characteristics of stereo data are extracted, and according to energy level At the time of residing for big value point, residing frequency and energy structure fingerprint so that fingerprint can keep good stability.And by structure The fingerprint built uses Hash representation, facilitates data storage and processing.
Specifically, Fig. 3 is according to embodiments of the present invention three method flow diagram, as shown in figure 3, this method specifically include with Lower step S302 to step S318.
Step S302：The real-time stream and identification information of live multimedia file are obtained, and is obtained according to real-time stream To multiple audio fragments of live multimedia file present period.
Specifically, step S216 is including the step S202 in above-described embodiment to step S206, and details are not described herein again.
Step S304：For each audio fragment, merge the left channel data and right data of audio fragment, to obtain The stereo data of audio fragment.
Specifically, stereo data can be obtained using formula below：
S=a*l+b*r, wherein, a+b=1, s are the stereo data of audio fragment, and l is the L channel number of audio fragment According to r is the right data of audio fragment, and a and b are default parameter.
Step S306：Extract finger of the time-frequency characteristics data of the stereo data of each audio fragment as the audio fragment Line information, so as to obtain the characteristic information of live multimedia file present period.
When extracting the time-frequency characteristics data of stereo data of audio fragment, following step is specifically included：
Short Time Fourier Transform is carried out to the stereo data of audio fragment first, to obtain the stereo number of audio fragment According to time frequency distribution map, then obtain time frequency distribution map in energy maximum point, according to two maximum point A at different moments It is fp [ta, fa, fb, tb-ta] that [ta, fa, Va], B [tb, fb, Vb], which build a fingerprint, and is converted to Hash codes fp [hashData, ta], wherein, at the time of ta is residing for a little bigger A of extreme value, fa is the frequency residing for a little bigger A of extreme value, and Va is big for extreme value The energy of point A, at the time of tb is residing for a little bigger B of extreme value, fb is the frequency residing for a little bigger B of extreme value, and Vb is the energy of a little bigger B of extreme value Amount, ta<Tb, maximum point A and a little bigger B of extreme value are the energy maximum point that any two is adjacent in time frequency distribution map, finally will All fingerprints of structure combine to obtain the finger print information of audio fragment sequentially in time.
Step S308：Fingerprint database is updated according to the characteristic information of live multimedia file present period.
Specifically, step S216 is including the step S210 in above-described embodiment to step S214, and details are not described herein again.
Step S310：Receive the identification request of identification destination multimedia file.
Step S312：According to the characteristic information of identification acquisition request destination multimedia file.
Wherein, the characteristic information of destination multimedia file is the time-frequency characteristics number of the stereo data of destination multimedia file According to the time-frequency characteristics number of stereo data of the specific method for obtaining video frequency feature data with extracting audio fragment in step S306 Identical according to method, details are not described herein again,
Wherein, the stereo data of destination multimedia file is by the L channel number in the voice data of destination multimedia file Formed according to merging with right data, multiple stereo datas specifically can obtain using multigroup parameter, correspondingly, it is more to obtain target The characteristic information of media file is multiple finger print informations.
When building the finger print information of destination multimedia file, the characteristic information of destination multimedia file is believed for N number of fingerprint Cease, a finger print information in N number of finger print information is a stereo number in N number of stereo data of destination multimedia file According to time-frequency characteristics data, wherein, i-th of stereo data in N number of stereo data is si '=ai ' * l '+bi ' * r ', Ai '+bi '=1, i=1,2,3 ... N.
Step S314：By each finger print information of destination multimedia file respectively with the sample fingerprint in fingerprint database Match somebody with somebody, obtain the matching rate of each finger print information.
The corresponding finger print information of every group of stereo data is matched with the sample fingerprint in fingerprint database, for appointing Anticipate the finger print information of one group of stereo data, can obtain matching sample fingerprint, and the fingerprint sample each matched This can correspond to a matching rate, and the multimedia where the corresponding sample fingerprint of matching rate maximum (namely maximum matching rate) is remembered Record is as the corresponding multimedia recording of the destination multimedia file.
Step S316：Using the multimedia recording where the corresponding sample fingerprint of maximum matching rate as destination multimedia file Corresponding multimedia recording.
Step S318：The identification information in the corresponding multimedia recording of destination multimedia file is obtained, to identify that target is more Media file.
Example IV
The example IV provides a kind of recognition methods of live multimedia file, in the method, live multimedia text Part is live video, and time-frequency characteristics data when building fingerprint database using the voice data of video are used as sample fingerprint structure Build.
In the method, the voice data of video uses stereo data without exception, ensures the finger print information sound of target video Frequency source is consistent with the data format of sample fingerprint audio-source in fingerprint database.Meanwhile to the voice data of target video into During row pretreatment, influence of the ambient noise to voice data quality when obtaining the voice data of target video for record type, Auto-adaptive parameter is set so that the finger print information of the target video extracted more robust.
To sum up, in terms of user terminal and fingerprint database two, the fast, accurately identification to video is realized. In this method, by the real-time update to fingerprint database, the real-time online identification to network direct broadcasting video is realized.
Next, by from the angle for the system for realizing the embodiment method, the live more of embodiment offer are described in detail The recognition methods of media file.
As shown in figure 4, the system is made of 4 parts：User terminal, video search server, fingerprint management server, regard Frequency management server, wherein, video search server, fingerprint management server, video management server can collectively form high in the clouds Server.
Specifically, user terminal is responsible for obtaining the voice data of target video, and the result of video search is presented.Video Search server is responsible for the video identification request of different user terminals, and sends these requests to fingerprint management server； It is additionally operable to receive the video recognition result that fingerprint management server transmits, and returns result to and propose that the user of identification request is whole End.On the one hand fingerprint management server is responsible for searching for the corresponding sample fingerprint of target video in fingerprint database；On the other hand, It is responsible for creating, updating, safeguarding fingerprint database.On the one hand video management server is responsible for storage and manages what video source was sent Video data, video data is stored to video database；Meanwhile the corresponding voice data of video and video information are uploaded to Fingerprint management server.Video search server and fingerprint management server coordinate the search for realizing target video, and fingerprint pipe Reason server and video management server coordinate the establishment and renewal for realizing fingerprint database.
As shown in figure 5, user terminal is included with lower module：
Recording module：Voice data when being played for recorded video；Audio pretreatment module：Recording module is obtained The operations such as voice data is mixed, down-sampling, noise reduction, reduce influence of the noise for matching result of playback environ-ment；Fingerprint Extraction module：The finger print information of voice data after extraction pretreatment；Result display module：Using user terminal audio player, The hardware resources such as video player, display screen, show the result of video search (such as：Play recognition result, be in mobile phone screen Existing similar video)；Network transmission module：Realize the data transfer demands between user terminal and video search server, to Video search server sends content please for the identification of " scene information, target fingerprint (namely finger print information of target video) " Ask, receive the video recognition result that video search server is sent.
As shown in fig. 6, video search server is included with lower module：
Network transmission module 1：For the information exchange with user terminal.Receive the video identification request of user terminal.Will The result of video search returns to user terminal；Video search management module：Handle the video identification request of mass users.It will use Fingerprint management server is submitted in the video identification request that family terminal is sent；Recognition result management module：Handle video search As a result；Network transmission module 2：For with fingerprint management server information exchange.Receive the video that fingerprint management server is sent Recognition result.Future, the video identification request of user terminal was sent to fingerprint management server.
As shown in fig. 7, fingerprint management server is included with lower module：
Network transmission module 1：For the information exchange with video search server.Receive video identification request.By video The result of search returns to video search server；Fingerprint search module：Target fingerprint is searched in fingerprint database, returns and knows Other result；Fingerprint extraction module：The finger print information of the voice data from video management server is extracted, the fingerprint of generation is believed Breath is transmitted to fingerprint management module together with video information (namely identification information of video)；Fingerprint management module：According to video information The data needed for fingerprint database generated with finger print information, and the data of generation are stored into fingerprint database；Network passes Defeated module 2：For fingerprint management server and the information exchange of video management server.
As shown in figure 8, video management server is included with lower module：
Network transmission module 1：Realize video source and the data transfer of video management server；Video management module：According to The information of video source is (such as：Channel, ur l etc.), video flowing is stored into video database corresponding position, meanwhile, by video Stream is passed to audio extraction module together with video source information；Audio extraction module：Obtain video management server transmit data, carry The audio stream in video flowing is taken, audio stream is passed to audio segmentation module together with video source information；Audio pretreatment module：Will be double The voice data of different-format is changed into unified form, and voice data is carried out into stereo by channel audio data mixing Down-sampling, audio segmentation module is transmitted to by the voice data after processing together with video source information；Audio segmentation module：It is temporally suitable Voice data is divided into or is spliced into the audio fragment that time span is T by sequence, and audio fragment is uploaded together with video source information To fingerprint management server；Network transmission module 2：Realize the data transfer of video management server and fingerprint management server.
The method of the embodiment using above-mentioned system when realizing video identification, it is necessary to by following steps：Step 1, Obtain the voice data of target video to be identified；The voice data got is pre-processed；Step 3, obtains target and regards The finger print information of the voice data of frequency；Step 4, by the finger print information and the fingerprint in advance building (or real-time update) Sample fingerprint in database is matched, and obtains matching result；Step 5, user terminal, user are returned to by matching result Terminal can be according to acquisition as a result, being presented and playing relevant video content.
The video data of system is stored in multiple video databases using distributed storage mode, and by multiple video tubes Reason server is managed.Between different video databases, the shared of resource is carried out by unified list of videos, it is all Video management server shares a list of videos.When the list of videos of one of video database updates, then the video counts Message (list of videos after renewal is carried in message) is updated to the whole network broadcast lists according to the corresponding video management server in storehouse, Other video databases update the list of videos of oneself according to message.
System possesses unique fingerprint database, which is managed by fingerprint management server, in fingerprint In management server, fingerprint extraction module and fingerprint search module are configured with.Fingerprint extraction module is used for handling video management clothes The information that business device transmits, forms sample fingerprint；Fingerprint search module is used for handling the video identification that video search server is sent Request, searches for target fingerprint, and recognition result is returned to video search server in fingerprint database.
Interacting between video source and video management server：Video source produces new video data, to video management service Device submits the request message (video source information, video content information are included in message) and video flowing of uploaded videos data；Video Video source information, video content information and video flowing in management server extraction message；Renewal video management server regards Frequency list；On the one hand, above- mentioned information and video flowing are stored into video database, establishes list information and storage video data Between association, which is added in local list of videos, while to the whole network INVENTIONBroadcast video list update information, more The list of new the whole network list of videos.On the other hand, the voice data in video data is extracted, by above-mentioned video source and video content Information, newly-increased list of videos information are packaged into video library renewal message together with the voice data of extraction, by network to fingerprint pipe Reason server submits the message.
Interacting between fingerprint management server and video management server：(1) fingerprint management server receives video library Message is updated, according to video source information and video content information in message, generation should be with the unique corresponding Track ID of the video. Track ID are added into the fingerprint list of fingerprint management server.(2) voice data in message is obtained, extracts the audio The finger print information of data.For different types of video source (live/non-live), using different finger print information extraction schemes. (3) Track ID, finger print information, video source information and video content information are encapsulated, preserved into fingerprint database.Fingerprint number It is used for storing the finger print information of video data and the related information of video, the relevant information of video according to storehouse.
Fingerprint database is logically divided into two subdata base (1) live video fingerprint subdata bases；(2) video finger print Subdata base, the two databases are managed collectively by fingerprint management server jointly, realize the behaviour for creating, updating, safeguarding Make.
Live video fingerprint subdata base stores the relevant information of current live video：Track ID, finger print information, video Information, video related information.Each channel corresponds to unique Track ID.
Track ID：Unique mark of the finger print information of video in fingerprint database.
Finger print information：The finger print information of the live video of a length of T when only retaining newest.Finger print information is with live video The renewal of database is updated accordingly.Concrete implementation scheme is specifically described in next part.
Video information：Information relevant with live video content and video storage information.Including：It is video channel, live Title, live content, host, direct broadcast band ur l, storage location etc..
Video related information：Other video links similar to live video, the commodity occurred in video or place Information etc..
The renewal of live video fingerprint subdata base：
(1) video management server receives video information and the video flowing that direct broadcasting room (namely video source) is sent.Extract it Middle video information part.Audio extraction module is called, extracts the voice data of video.Audio pretreatment module is called to acquisition Voice data carries out pretreatment operation；It is t (t to call Video segmentation module that video flowing is divided into time span in chronological order Far smaller than T) audio fragment；All audio fragments are added into the transmit queue of network transmission module 1；By video information Video library is packaged into successively together with the audio fragment that length in transmit queue is t and updates message, is uploaded to fingerprint management server.
(2) fingerprint management server receives video library renewal message, and it is straight to parse this for video information part from message The channel of video is broadcast, generates corresponding Track ID；Fingerprint extraction module is called, extraction time length is the finger of t voice datas Line information；The update cycle of fingerprint database is P (T>>P=kt, k are integer).New fingerprint length is added to fingerprint number for kt According in the fingerprint list in storehouse, while the finger print information that original length is kt is removed, ensure the finger print information length in fingerprint list Degree remains T.
Within the system, for user terminal, by combining algorithm for recognizing fingerprint, fingerprint database real-time update technology, Enable the system to quickly identify live video.Meanwhile when creating fingerprint database, take the fingerprint letter from stereo data Breath, make in the finger print information bigger probability match for the target video that user terminal obtains in fingerprint database it is corresponding true Finger print information, improves the anti-noise ability of identification process.
Embodiment five
The embodiment five provides a kind of embodiment of the identification device of live multimedia file, which may be disposed at cloud Server is held, as shown in figure 9, the device includes acquisition module 610, locating module 620, update module 630, matching module 640 With identification module 650.
Wherein, cloud server and the background data base server of live source are in communication with each other, and get live more matchmakers in real time The real-time stream of body file, while real-time stream is obtained, can also get the identification information of live multimedia file. Acquisition module 610 is used to obtain live multimedia file present period according to the real-time stream of the live multimedia file of input Characteristic information.For example, get the video stream data of live video, using characteristic extracting module to video stream data at Reason, corresponds to audio-frequency fingerprint to extract live video, obtains the characteristic information of live video present period.
Property data base is provided with server beyond the clouds, this feature database is used to store at least one multimedia note Record, multimedia recording include feature samples, the identification information corresponding with feature samples of multimedia file, the time of feature samples Length was first scheduled time.Locating module 620 is used for the identification information according to live multimedia file in property data base Position multimedia recording to be updated.
Update module 630 is used to update multimedia to be updated according to the characteristic information of live multimedia file present period Feature samples in record, so as to be directed to arbitrary live multimedia file, are always stored with current newest in property data base Characteristic information in a period of time.
Matching module 640 is used for the identification request for receiving identification destination multimedia file, and match cognization request includes Feature samples in the characteristic information and property data base of destination multimedia file are corresponding more to position destination multimedia file Media recording.Identification module 650 is used for the identification information for obtaining the corresponding multimedia file of destination multimedia file.
The identification device of the live multimedia file provided using the embodiment, a preset property data base are straight to store The feature samples and identification information of playing multimedia file, and by obtaining the real-time stream real-time update and dimension on live source backstage This feature database is protected, when there is destination multimedia file to need identified, is existed according to the characteristic information of destination multimedia file The identification information of corresponding live multimedia file is found in property data base, achievees the purpose that to identify live multimedia file. To sum up, the identification device of the live multimedia file provided using the embodiment, can identify live multimedia file in real time, So as to can also identify live video in real time.
Preferably, characteristic information is the finger print information of the voice data of multimedia file, and acquisition module 610 includes audio number According to acquisition module, audio fragment segmentation module and finger print information extraction module.Wherein, voice data acquisition module is used for according to reality When data flow obtain live multimedia file present period voice data；Audio fragment segmentation module is used for present period Voice data be divided into multiple audio fragments of second scheduled time sequentially in time；Finger print information extraction module is used to carry The finger print information of each audio fragment is taken, to obtain the characteristic information of the present period of live multimedia, wherein, the second pre- timing Between be less than for first scheduled time.
Preferably, feature samples are the finger print information of n audio fragment, the feature of the present period of live multimedia file Information is the finger print information of m audio fragment, m<The time span of n, n audio fragments was first scheduled time, update module 630 include removing module and add module.Wherein, removing module is used to delete feature samples in multimedia recording to be updated M earliest finger print information；Add module is used for m finger print information of the present period of live multimedia file is temporally suitable Sequence is placed in the feature samples of multimedia recording to be updated.
It is further preferred that update module 630 specifically performs following steps：
Step S1：Feature pointer is directed toward first fingerprint letter in the characteristic information of live multimedia file present period Breath, and timer is reset and starts feature extraction timing；
Step S2：Obtain the finger print information that feature pointer is directed toward；
Step S3：Extraction and the feature samples of the corresponding multimedia recording of identification information of live multimedia, to obtain Fisrt feature sample；
Step S4：The finger print information that feature pointer is directed toward is spliced to the end of fisrt feature sample, it is special to obtain second Levy sample；
Step S5：A finger print information is deleted from the starting of second feature sample；
Step S6：Judge whether the time in timer reached for the 3rd scheduled time, it is special if not up to the 3rd scheduled time Levy pointer and be directed toward next finger print information, and repeat step S2 to S6；If reaching for the 3rd scheduled time, with second obtained Feature samples replace the corresponding feature samples of multi-media tag in multimedia recording, wherein, the 3rd scheduled time believed for m fingerprint Cease the reproduction time of corresponding multimedia file.
It is further preferred that finger print information extraction module includes stereo data synthesis module and time-frequency characteristics extraction mould Block.Wherein, stereo data synthesis module is used for the left channel data and right data for merging audio fragment, to obtain audio The stereo data of fragment；The time-frequency characteristics data that time-frequency characteristics extraction module is used to extract the stereo data of audio fragment are made For the finger print information of audio fragment.
Preferably, the characteristic information for the destination multimedia file that identification request includes is the current of live multimedia file N number of finger print information of period, a finger print information in N number of finger print information is in N number of stereo data of destination multimedia The time-frequency characteristics data of one stereo data, wherein, i-th of stereo data in N number of stereo data is si '=ai ' * L '+bi ' * r ', ai '+bi '=1, l ' are the left channel data of the present period of live multimedia file, and r ' is live multimedia The right data of the present period of file, ai ' and bi ' are default parameter, i=1,2,3 ... N, and matching module 640 includes With rate determining module and multimedia recording determining module.Wherein, matching rate determining module is used for the every of destination multimedia file A finger print information is matched with the feature samples in property data base respectively, to obtain the matching rate of each finger print information；Multimedia Determining module is recorded to be used for using the multimedia recording where the corresponding feature samples of maximum matching rate as destination multimedia file Corresponding multimedia recording.
The foregoing is only a preferred embodiment of the present invention, but protection scope of the present invention be not limited thereto, Any people for being familiar with the technology disclosed herein technical scope in, the change or replacement that can readily occur in should all be covered Within protection scope of the present invention.Therefore, protection scope of the present invention should be subject to scope of the claims.

Claims (10)

  1. A kind of 1. recognition methods of live multimedia file, it is characterised in that including：
    The feature of the live multimedia file present period is obtained according to the real-time stream of the live multimedia file of input Information；
    Multimedia recording to be updated is positioned in property data base according to the identification information of the live multimedia file, its In, the property data base is used to store at least one multimedia recording, and the multimedia recording includes multimedia file Feature samples, identification information corresponding with the feature samples, the time span of the feature samples was first scheduled time；
    Spy in the multimedia recording to be updated is updated according to the characteristic information of the live multimedia file present period Levy sample；
    The identification request of identification destination multimedia file is received, matches the destination multimedia text that the identification request includes The characteristic information of part and the feature samples in the property data base, to position the corresponding multimedia of the destination multimedia file Record；
    Obtain the identification information of the corresponding multimedia file of the destination multimedia file.
  2. 2. the recognition methods of the live multimedia file according to claim 1, it is characterised in that the characteristic information is The finger print information of the voice data of multimedia file, the real-time stream of the live multimedia file according to input obtain institute The characteristic information of live multimedia file present period is stated, including：
    The voice data of the present period of the live multimedia file is obtained according to the real-time stream；
    The voice data of the present period is divided into multiple audio fragments of second scheduled time sequentially in time, its In, second scheduled time is less than first scheduled time；And
    The finger print information of each audio fragment of extraction, to obtain the characteristic information of the present period of the live multimedia.
  3. 3. the recognition methods of the live multimedia file according to claim 2, it is characterised in that the feature samples are n The finger print information of a audio fragment, the characteristic information of the present period of the live multimedia file are the finger of m audio fragment Line information, m<N, the total time length of the n audio fragment is first scheduled time, according to the live multimedia The feature samples that the characteristic information of file is updated in the multimedia recording to be updated include：
    Delete the m earliest finger print information of feature samples in the multimedia recording to be updated；
    M finger print information of the present period of the live multimedia file is placed in chronological order described to be updated more In the feature samples of media recording.
  4. 4. the recognition methods of the live multimedia file according to claim 3, it is characterised in that described according to described straight The characteristic information of playing multimedia file present period updates the feature samples in the multimedia recording to be updated, specific bag Include：
    Step S1：Feature pointer is directed toward first fingerprint letter in the characteristic information of the live multimedia file present period Breath, and timer is reset and starts feature extraction timing；
    Step S2：Obtain the finger print information that the feature pointer is directed toward；
    Step S3：Extraction and the feature samples of the corresponding multimedia recording of identification information of the live multimedia, to obtain Fisrt feature sample；
    Step S4：The finger print information that the feature pointer is directed toward is spliced to the end of the fisrt feature sample, to obtain Two feature samples；
    Step S5：A finger print information is deleted from the starting of the second feature sample；
    Step S6：Judge whether the time in timer reached for the 3rd scheduled time, if not up to described 3rd scheduled time, The feature pointer is directed toward next finger print information, and repeats step S2 to S6；If reaching the 3rd scheduled time, The corresponding feature samples of multi-media tag described in the multimedia recording are replaced with the obtained second feature sample, its In, the 3rd scheduled time is the total duration of the reproduction time of the corresponding multimedia file of the m finger print information.
  5. 5. the recognition methods of the live multimedia file according to claim 2, it is characterised in that extract the audio piece The finger print information of section includes：
    Merge the left channel data and right data of the audio fragment, to obtain the stereo data of the audio fragment； And
    Extract finger print information of the time-frequency characteristics data of the stereo data of the audio fragment as the audio fragment.
  6. 6. the recognition methods of the live multimedia file according to claim 2, it is characterised in that in the identification request Including the destination multimedia file characteristic information for the live multimedia file present period N fingerprint believe Cease, a finger print information in the N finger print information is one in N stereo data of the destination multimedia The time-frequency characteristics data of stereo data, wherein, i-th stereo data in the N stereo data for si '= Ai ' * l '+bi ' * r ', wherein, ai '+bi '=1, l ' they are the L channel number of the present period of the live multimedia file According to the right data for the present period that, r ' is the live multimedia file, ai ' and bi ' are default parameter, i=1, 2,3… N,
    In the method, the characteristic information for the destination multimedia file that the matching identification request includes and institute The feature samples in property data base are stated, are included with positioning the corresponding multimedia recording of the destination multimedia file：
    Each finger print information of the destination multimedia file is matched with the feature samples in the property data base respectively, is obtained To the matching rate of each finger print information；
    Multimedia recording where the corresponding feature samples of maximum matching rate is corresponding more as the destination multimedia file Media recording.
  7. A kind of 7. identification device of live multimedia file, it is characterised in that including：
    Acquisition module, the real-time stream for the live multimedia file according to input obtain the live multimedia file and work as The characteristic information of preceding period；
    Locating module is to be updated more for being positioned according to the identification information of the live multimedia file in property data base Media recording, wherein, the property data base is used to store at least one multimedia recording, and the multimedia recording includes The feature samples of multimedia file, identification information corresponding with the feature samples, the time spans of the feature samples are the One scheduled time；
    Update module, for updating more matchmakers to be updated according to the characteristic information of the live multimedia file present period Feature samples in body record；
    Matching module, for receiving the identification request of identification destination multimedia file, matches the institute that the identification request includes The characteristic information of destination multimedia file and the feature samples in the property data base are stated, to position the destination multimedia text The corresponding multimedia recording of part；
    Identification module, for obtaining the identification information of the corresponding multimedia file of the destination multimedia file.
  8. 8. the identification device of the live multimedia file according to claim 7, it is characterised in that the characteristic information is The finger print information of the voice data of multimedia file, the acquisition module include：
    Voice data acquisition module, for the present period according to the real-time stream acquisition live multimedia file Voice data；
    Audio fragment splits module, for the voice data of the present period to be divided into the second pre- timing sequentially in time Between multiple audio fragments, wherein, second scheduled time is less than first scheduled time；And
    Finger print information extraction module, for extracting the finger print information of each audio fragment, to obtain the live multimedia Present period characteristic information.
  9. 9. the identification device of the live multimedia file according to claim 8, it is characterised in that the feature samples are n The finger print information of a audio fragment, the characteristic information of the present period of the live multimedia file are the finger of m audio fragment Line information, m<N, the total time length of the n audio fragment is first scheduled time, and the update module includes：
    Removing module, for deleting the m earliest finger print information of feature samples in the multimedia recording to be updated；
    Add module, for m finger print information of the present period of the live multimedia file to be placed in institute in chronological order In the feature samples for stating multimedia recording to be updated.
  10. 10. the identification device of the live multimedia file according to claim 9, it is characterised in that the update module tool Body performs following steps：
    Step S1：Feature pointer is directed toward first fingerprint letter in the characteristic information of the live multimedia file present period Breath, and timer is reset and starts feature extraction timing；
    Step S2：Obtain the finger print information that the feature pointer is directed toward；
    Step S3：Extraction and the feature samples of the corresponding multimedia recording of identification information of the live multimedia, to obtain Fisrt feature sample；
    Step S4：The finger print information that the feature pointer is directed toward is spliced to the end of the fisrt feature sample, to obtain Two feature samples；
    Step S5：A finger print information is deleted from the starting of the second feature sample；
    Step S6：Judge whether the time in timer reached for the 3rd scheduled time, if not up to described 3rd scheduled time, The feature pointer is directed toward next finger print information, and repeats step S2 to S6；If reaching the 3rd scheduled time, The corresponding feature samples of multi-media tag described in the multimedia recording are replaced with the obtained second feature sample, its In, the 3rd scheduled time is the total duration of the reproduction time of the corresponding multimedia file of the m finger print information.
CN201410849032.9A 2014-12-29 2014-12-29 The recognition methods of live multimedia file and device Active CN104572952B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410849032.9A CN104572952B (en) 2014-12-29 2014-12-29 The recognition methods of live multimedia file and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410849032.9A CN104572952B (en) 2014-12-29 2014-12-29 The recognition methods of live multimedia file and device

Publications (2)

Publication Number Publication Date
CN104572952A CN104572952A (en) 2015-04-29
CN104572952B true CN104572952B (en) 2018-04-17

Family

ID=53089014

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410849032.9A Active CN104572952B (en) 2014-12-29 2014-12-29 The recognition methods of live multimedia file and device

Country Status (1)

Country Link
CN (1) CN104572952B (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105828179A (en) * 2015-06-24 2016-08-03 维沃移动通信有限公司 Video positioning method and device
CN105550257B (en) * 2015-12-10 2019-05-03 杭州当虹科技股份有限公司 A kind of audio/video fingerprint recognition methods and a kind of tamper resistant systems based on audio/video fingerprint Streaming Media
CN105554590B (en) * 2015-12-10 2018-12-04 杭州当虹科技有限公司 A kind of live broadcast stream media identifying system based on audio-frequency fingerprint
CN107027061A (en) * 2016-02-01 2017-08-08 晨星半导体股份有限公司 Television system and multi-medium play method
CN105786967A (en) * 2016-02-01 2016-07-20 杭州当虹科技有限公司 Mobile phone photographing based live broadcast stream media identification system
CN105872586A (en) * 2016-04-01 2016-08-17 成都掌中全景信息技术有限公司 Real time video identification method based on real time video streaming collection
CN108271073A (en) * 2016-12-28 2018-07-10 上海昕丝文化传播有限公司 SDK flow chart of data processing
CN108600778B (en) * 2018-05-07 2020-11-03 广州酷狗计算机科技有限公司 Media stream transmitting method, device, system, server, terminal and storage medium
CN108881191A (en) * 2018-05-25 2018-11-23 广州酷狗计算机科技有限公司 Collection of media files acquisition methods, device, server and storage medium
CN109218743B (en) * 2018-09-17 2021-04-20 广州珠江数码集团股份有限公司 Information calibration method and system based on live program content
CN109857902A (en) * 2019-03-01 2019-06-07 腾讯音乐娱乐科技（深圳）有限公司 A kind of update method of audio query, system and storage medium and server
CN111182347B (en) * 2020-01-07 2021-03-23 腾讯科技（深圳）有限公司 Video clip cutting method, device, computer equipment and storage medium
CN115412777A (en) * 2021-05-28 2022-11-29 北京金山云网络技术有限公司 Streaming media data transmission method, device and system
CN114339442B (en) * 2021-12-31 2023-11-07 北京达佳互联信息技术有限公司 Method and device for configuring multimedia channels, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102411578A (en) * 2010-09-25 2012-04-11 盛乐信息技术（上海）有限公司 Multimedia playing system and method
CN102799605A (en) * 2012-05-02 2012-11-28 天脉聚源(北京)传媒科技有限公司 Method and system for monitoring advertisement broadcast
CN102984553A (en) * 2012-10-29 2013-03-20 北京海逸华清科技发展有限公司 Audio and video detection recognition method and audio and video detection recognition system
CN103618953A (en) * 2013-08-15 2014-03-05 北京中视广信科技有限公司 Audio frequency feature based method and system for marking and identifying broadcast television program

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050267750A1 (en) * 2004-05-27 2005-12-01 Anonymous Media, Llc Media usage monitoring and measurement system and method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102411578A (en) * 2010-09-25 2012-04-11 盛乐信息技术（上海）有限公司 Multimedia playing system and method
CN102799605A (en) * 2012-05-02 2012-11-28 天脉聚源(北京)传媒科技有限公司 Method and system for monitoring advertisement broadcast
CN102984553A (en) * 2012-10-29 2013-03-20 北京海逸华清科技发展有限公司 Audio and video detection recognition method and audio and video detection recognition system
CN103618953A (en) * 2013-08-15 2014-03-05 北京中视广信科技有限公司 Audio frequency feature based method and system for marking and identifying broadcast television program

Also Published As

Publication number Publication date
CN104572952A (en) 2015-04-29

Similar Documents

Publication Publication Date Title
CN104572952B (en) The recognition methods of live multimedia file and device
CN110198432B (en) Video data processing method and device, computer readable medium and electronic equipment
CN104584571B (en) Audio-frequency fingerprint sequence is produced at set top box
CN111447505B (en) Video clipping method, network device, and computer-readable storage medium
US11070851B2 (en) System and method for providing image-based video service
CN105450778B (en) Information transmission system
KR20170027648A (en) Method and apparatus for synchronous putting of real-time mobile advertisement based on audio fingerprint
CN105120304A (en) Information display method, device and system
CN110287346B (en) Data storage method, device, server and storage medium
CN104065979A (en) Method for dynamically displaying information related with video content and system thereof
WO2017080173A1 (en) Nature information recognition-based push system and method and client
EP2973034B1 (en) Methods and systems for arranging and searching a database of media content recordings
KR101396413B1 (en) Information providing system and method using digital fingerprinting
CN102882703A (en) Hyper text transfer protocol (HTTP)-analysis-based uniform resource locator (URL) automatically classifying and grading system and method
US11310326B2 (en) Methods and apparatus to facilitate meter to meter matching for media identification
CN109168020A (en) Method for processing video frequency, device, calculating equipment and storage medium based on live streaming
CN104023250A (en) Real-time interaction method and system based on streaming media
CN105163142A (en) User preference determination method, video recommendation method, user preference determination system and video recommendation system
US12058388B2 (en) Event progress detection in media items
CN103593356A (en) Method and system for information searching on basis of multimedia information fingerprint technology and application
KR20210091082A (en) Image processing apparatus, control method thereof and computer readable medium having computer program recorded therefor
CN113469745A (en) Method and system for sharing advertising content from a primary device to a secondary device
CN103365892A (en) Method and device for processing multiple contact objects
CN103023923A (en) Information transmission method and information transmission device
CN105791964B (en) cross-platform media file playing method and system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20200825

Address after: 7, No. 666, Zhang Heng Road, 201203, Shanghai, Pudong New Area, No. 1

Patentee after: SHANGHAI ZHANGMEN SCIENCE AND TECHNOLOGY Co.,Ltd.

Address before: 100089 room six, floor 19, building 68, 6184 South College Road, Beijing, Haidian District

Patentee before: LE HOLDINGS (BEIJING) Co.,Ltd.

TR01 Transfer of patent right