WO2019184523A1 - 一种媒体特征的比对方法及装置 - Google Patents

一种媒体特征的比对方法及装置 Download PDF

Info

Publication number
WO2019184523A1
WO2019184523A1 PCT/CN2018/125502 CN2018125502W WO2019184523A1 WO 2019184523 A1 WO2019184523 A1 WO 2019184523A1 CN 2018125502 W CN2018125502 W CN 2018125502W WO 2019184523 A1 WO2019184523 A1 WO 2019184523A1
Authority
WO
WIPO (PCT)
Prior art keywords
media
similarity
media feature
straight line
determining
Prior art date
Application number
PCT/CN2018/125502
Other languages
English (en)
French (fr)
Inventor
何轶
李磊
杨成
李�根
李亦锬
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Priority to JP2020520709A priority Critical patent/JP6987987B2/ja
Priority to SG11202000106WA priority patent/SG11202000106WA/en
Priority to US16/979,784 priority patent/US11593582B2/en
Publication of WO2019184523A1 publication Critical patent/WO2019184523A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/48—Matching video sequences
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Definitions

  • the present disclosure relates to the field of media processing technologies, and in particular, to a method and apparatus for comparing media features.
  • Media features such as video features, audio features, or media fingerprints, as well as media feature comparisons and media feature retrieval, are widely used in today's "multimedia information society.” By using media feature comparison, it is possible to avoid repeated uploading of video and audio, thereby preventing media theft and optimizing media storage, and also utilizing media feature comparison to perform media content monitoring, copyright detection and the like.
  • the existing media feature comparison method has the problems of poor accuracy and low efficiency, which will greatly consume both computing resources and storage resources.
  • the method for aligning media features includes the steps of: acquiring a first media feature sequence of a first media object and a second media feature sequence of a second media object, the first media feature sequence comprising sequentially arranged a plurality of first media feature units, the second media feature sequence comprising a plurality of second media feature cells arranged in sequence; determining between the first media feature unit and the second media feature unit a similarity degree between the first media feature sequence and the second media feature sequence according to the cell similarity; determining the first media object and the location according to the similarity matrix A similar situation of the second media object.
  • the object of the present disclosure can also be further achieved by the following technical measures.
  • the foregoing method for comparing media features wherein the first media feature unit and the second media feature unit are floating point numbers; the determining the first media feature unit and the second The monomer similarity between the media feature cells includes determining the cell similarity according to a cosine distance between the first media feature cell and the second media feature cell.
  • the foregoing method for aligning media features wherein the first media feature unit and the second media feature unit are binarized features and have the same feature cell length;
  • the unit similarity between a media feature unit and the second media feature unit includes: determining a location according to a Hamming distance between the first media feature unit and the second media feature unit Said monomer similarity.
  • the foregoing method for aligning media features wherein the acquiring the first media feature sequence of the first media object and the second media feature sequence of the second media object comprises: acquiring the plurality of types of the first media object a first sequence of media features, and acquiring a plurality of types of the second media feature sequences of the second media object; determining the single between the first media feature unit and the second media feature unit
  • the body similarity includes determining, respectively, a monomer similarity between the first media feature monomer and the second media feature monomer of the same type to obtain a plurality of the monomer similarities; Determining a similarity matrix between the first media feature sequence and the second media feature sequence according to the cell similarity includes determining an average value or a minimum value of the plurality of cell similarities, according to the The average or minimum of the plurality of monomer similarities determines the similarity matrix.
  • the foregoing method for aligning media features wherein the plurality of first media feature cells are arranged in chronological order in the first media feature sequence, and the plurality of second media feature cells are in the second The sequence of media features is arranged in chronological order.
  • a point in the similarity matrix corresponds to one of the single cell similarities; and a point of the similarity matrix is in a horizontal direction according to each of the first media feature elements Arranging sequentially in the sequence of the first media features, and arranging in the longitudinal direction according to the order of the respective second media feature elements in the second media feature sequence.
  • determining the similarity between the first media object and the second media object according to the similarity matrix comprises: determining, according to a straight line in the similarity matrix The degree of similarity and matching segment of the first media object with the second media object.
  • determining the similarity between the first media object and the second media object according to a straight line in the similarity matrix comprises: setting a slope with a preset slope
  • the plurality of straight lines of the fixed value are defined as alternative straight lines, and the straight line similarity of the candidate straight lines is determined according to the average value or the sum value of the individual similarities included in each of the candidate straight lines;
  • an alternative straight line that maximizes the similarity of the straight line is selected and defined as a first matching straight line;
  • the first media object is determined according to the straight line similarity of the first matching straight line a degree of similarity of the second media object; determining a start and end time of the matching segment of the first media object and the second media object according to a start point and an end point of the first matching line.
  • the slope setting value is a plurality
  • the candidate straight line is a straight line whose slope is any one of the plurality of slope setting values.
  • determining the similarity between the first media object and the second media object according to the similarity matrix comprises: selecting, in the similarity matrix, the a plurality of points having the largest single similarity as the similarity extreme point; based on the plurality of similarity extreme points, fitting a straight line as the second matching straight line in the similarity matrix; according to the second And determining, by the average value or the sum value of the monomer similarities included in the matching straight line, a degree of similarity between the first media object and the second media object; determining the first according to a start point and an end point of the second matching straight line The start and end time of a matching segment of a media object and the second media object.
  • the foregoing method for aligning media features wherein the fitting a straight line as the second matching line in the similarity matrix based on the plurality of similarity extreme points includes: using a random sampling consensus method, A straight line whose slope is a preset slope set value or whose slope is close to the preset slope set value is fitted in the similarity matrix as the second matching straight line.
  • determining the similarity between the first media object and the second media object according to the similarity matrix further comprises: determining the first matching line or the Whether the points at the beginning and the end of the second matching straight line reach the preset unit similarity setting value, and the portions of the beginning and the ending that do not reach the unit similarity setting value are removed, and the middle straight line is retained. And defining a third matching straight line; determining a similarity degree of the first media object and the second media object according to the straight line similarity of the third matching straight line, according to a starting point and an ending point of the third matching straight line Determine the start and end time of the matching segment.
  • the device for comparing media features includes: a media feature sequence obtaining module, configured to acquire a first media feature sequence of the first media object and a second media feature sequence of the second media object, the first media The feature sequence includes a plurality of first media feature cells arranged in sequence, the second media feature sequence includes a plurality of second media feature cells arranged in sequence; a single similarity determining module, configured to determine the first media a unit similarity between the feature unit and the second media feature unit; a similarity matrix determining module, configured to determine the first media feature sequence and the second media feature according to the single cell similarity a similarity matrix between the sequences; a similarity determining module, configured to determine a similarity between the first media object and the second media object according to the similarity matrix.
  • the object of the present disclosure can also be further achieved by the following technical measures.
  • the aforementioned media feature comparison device further includes means for performing the comparison method steps of any of the foregoing media features.
  • a media feature comparison hardware device comprising: a memory for storing non-transitory computer readable instructions; and a processor for executing the computer readable instructions such that when the processor executes An alignment method that implements any of the foregoing media features.
  • a terminal device includes any one of the foregoing media feature comparison devices.
  • a computer readable storage medium for storing non-transitory computer readable instructions, when the non-transitory computer readable instructions are executed by a computer, causing the computer to perform any of the foregoing media features Comparison method.
  • FIG. 1 is a block flow diagram of a method of comparing media features of an embodiment of the present disclosure.
  • FIG. 2 is a schematic diagram of gray levels corresponding to a similarity matrix provided by an embodiment of the present disclosure.
  • FIG. 3 is a flow chart of a comparison using a dynamic programming method according to an embodiment of the present disclosure.
  • FIG. 4 is a flow chart of a comparison using a uniform media method according to an embodiment of the present disclosure.
  • FIG. 5 is a flow chart of determining a similarity matrix based on multiple types of media feature sequences according to an embodiment of the present disclosure.
  • FIG. 6 is a structural block diagram of a comparison device of media features according to an embodiment of the present disclosure.
  • FIG. 7 is a structural block diagram of a similar situation determining module according to an embodiment of the present disclosure.
  • FIG. 8 is a structural block diagram of a similar situation determining module according to another embodiment of the present disclosure.
  • FIG. 9 is a structural block diagram of a media feature comparison apparatus for determining a similarity matrix based on a plurality of types of media feature sequences according to an embodiment of the present disclosure.
  • FIG. 10 is a hardware block diagram of a media feature comparison hardware device of an embodiment of the present disclosure.
  • FIG. 11 is a schematic diagram of a computer readable storage medium in accordance with an embodiment of the present disclosure.
  • FIG. 12 is a structural block diagram of a terminal device according to an embodiment of the present disclosure.
  • FIG. 1 is a schematic flow chart of an embodiment of a method for comparing media features of the present disclosure.
  • a method for comparing media features of the example of the present disclosure mainly includes the following steps:
  • Step S10 Obtain a media feature sequence of the first media object as a first media feature sequence, and obtain a media feature sequence of the second media object as a second media feature sequence.
  • the first media object and the second media object are two media to be compared, for example, various types of media such as audio, video, and multiple consecutive photos.
  • the sequence of media features therein may be audio features, video features or image features, etc., and in fact the video objects may be aligned by acquiring audio features of the video objects in accordance with the methods of the present disclosure.
  • the first media feature sequence includes a plurality of first media feature monomers arranged in sequence
  • the second media feature sequence includes a plurality of second media feature cells arranged in sequence, which may be assumed to be the first media feature sequence and the second media.
  • the lengths of the feature sequences are M 1 and M 2 , respectively, where M 1 and M 2 are positive integers, that is, the first media feature sequence includes M 1 first media feature monomers, and the second media feature sequence includes M 2 A second media feature unit.
  • the sequence described herein is arranged such that the plurality of first/second media feature elements are arranged in chronological order in the first/second media feature sequence: for example, In the process of extracting the media features in advance, the media objects are first framed, and then one media feature element is generated according to each frame, so that each media feature element corresponds to each frame of the media object, and then each media feature is selected.
  • the cells are arranged in a chronological order of the respective frames in the media object to obtain a sequence of media features. Therefore, the aforementioned media feature unit can also be referred to as a frame feature, and the aforementioned media feature sequence is referred to as a media feature.
  • the method for extracting the media feature sequence and the type of the media feature sequence are not limited, but the first media feature sequence and the second media feature sequence should be the same type of media feature obtained by the same feature extraction method.
  • the floating point feature sequence of the first media object and the second media object may be simultaneously acquired as the first media feature sequence and the second media feature sequence, each media feature list in the floating point feature sequence.
  • the body is a floating point number.
  • the binary feature sequence of the first media object and the second media object may be acquired simultaneously, or the obtained media features of the type may be binarized to obtain a binary feature sequence.
  • each feature monomer in the binary value feature sequence is a bit string composed of 0/1, and the media feature cells extracted by the same method have the same length (or called the number of bits).
  • Step S20 determining a monomer similarity between each of the first media feature monomers and each of the second media feature cells to obtain M 1 ⁇ M 2 cell similarities.
  • Each monomer similarity is used to indicate the degree of similarity between two media feature monomers. Specifically, the greater the monomer similarity, the more similar. Thereafter, the process proceeds to step S30.
  • a distance or metric capable of determining the degree of similarity of the two media features may be selected as the single cell similarity according to the type of the media feature.
  • the cosine distance between the first media feature unit and the second media feature unit may be used (or
  • the cosine similarity is determined to determine the similarity of the monomer; generally, the cosine distance can be directly determined as the monomer similarity.
  • the Hamming distance between the first media feature unit and the second media feature unit may be used ( Hamming distance) determines the monomer similarity. Specifically, first, calculate a Hamming distance between the first media feature unit and the second media feature unit, and then calculate a difference between the feature cell length (number of bits) and the Hamming distance, and The ratio of the difference to the length of the feature cell is determined as a single cell similarity to represent the proportion of the same bit in the two binarized features.
  • the Hamming distance is a commonly used metric in the field of information theory.
  • the Hamming distance between two equal-length strings is the number of different characters corresponding to the positions of the two strings. When actually calculating the Hamming distance, the two strings can be XORed and the result is a number of 1, and this number is the Hamming distance.
  • any distance or metric that can determine the similarity of the two media feature monomers can be utilized.
  • the monomer similarity may also be referred to as inter-frame similarity if each media feature unit corresponds to each frame of the media object.
  • Step S30 determining a similarity matrix between the first media feature sequence and the second media feature sequence according to each individual similarity.
  • each point in the similarity matrix corresponds to a single cell similarity, such that the similarity matrix records a cell similarity between each of the first media feature cells and each of the second media feature cells.
  • the respective points of the similarity matrix are arranged in the order of the first media feature elements in the first media feature sequence in the horizontal direction, and in the longitudinal direction, according to the second media feature in the second media feature. The order in the sequence.
  • the point located in the i-th row and the j-th column represents the monomer similarity between the i-th first media feature element of the first media object and the j-th second media feature element of the second media object, and further
  • the similarity matrix is an M 1 ⁇ M 2 matrix.
  • the similarity matrix can be converted into a gray scale diagram as shown in FIG. 2.
  • the gray level of each point is used to represent the magnitude of the monomer similarity of the corresponding position in the similarity matrix. Specifically, if the gradation of a point is closer to white, it means that the corresponding monomer similarity of the point is higher, such as the point at I indicated in FIG. 2; and if the gradation of a point is closer to black, it means The lower the monomer similarity corresponding to this point, such as the point at II indicated in Figure 2.
  • step S20 it is not necessary to first perform the calculation of the individual similarity of step S20, and then perform the determining similarity matrix of step S30, but directly determine the similarity matrix, and determine the similarity matrix.
  • the corresponding monomer similarity is calculated in the process of each point.
  • Step S40 Determine, according to the similarity matrix, a similar situation between the first media object and the second media object.
  • the determining the similarity situation includes: determining a degree of similarity between the first media object and the second media object according to the similarity matrix and may use the comparison score to express the similarity degree, and/or according to the similarity matrix A start time of the matching segment of the first media object and the second media object is determined.
  • the comparison score can be a score between 0 and 1. The larger the number, the more similar the two media objects are.
  • the method for comparing media features of the embodiments of the present disclosure determines the similarity between media objects based on the similarity matrix between the two media objects, thereby improving the efficiency and accuracy of the media comparison.
  • step S40 includes determining a similarity of the first media object to the second media object based on a straight line in the similarity matrix.
  • the media feature sequence generally contains a plurality of finite media feature elements
  • the similarity matrix is a finite matrix
  • the so-called "straight line” is a finite length composed of a plurality of points in the similarity matrix.
  • the line has a slope that is the slope of the line connecting the plurality of points included in the line.
  • the starting point and the ending point of the straight line may be any points in the similarity matrix, and are not necessarily points located at the edge.
  • the straight line referred to in the present disclosure includes a diagonal line in the similarity matrix and a line segment parallel to the diagonal line. These straight lines have a slope of 1 from the upper left to the lower right in the similarity matrix (as indicated in FIG. 2).
  • the straight line III) also includes a straight line with a slope of not 1. For example, it may be a straight line with a slope of approximately 1 to improve the robustness of the media alignment; it may be a straight line with a slope of 2, 3, ... or 1/2, 1/3, ..., etc.
  • the diagonal line is a line segment consisting of points at (1,1), (2,2), (3,3)... (actually a point starting from the point in the upper left corner and having a slope of 1) straight line).
  • each straight line in the similarity matrix is composed of a plurality of single similarities arranged in sequence, so that each straight line can represent a similar situation of a plurality of sequentially arranged media feature pairs, thereby being able to express a paragraph
  • Each of the media feature unit pairs includes a first media feature unit and a second media feature unit. That is, each line represents a degree of similarity between a plurality of sequentially arranged first media feature cells and a plurality of sequentially arranged second media feature cells.
  • the slope of the line and the end point of the starting point represent the length and position of the two segments of the media.
  • a straight line composed of (1,1), (2,3), (3,5), (4,7), because the first media feature unit with a ordinal number of 1 and a second media with a ordinal number of 1
  • the similarity between the characteristic monomers, the first media feature cell with the ordinal number of 2 and the ordinal number are the similarities between the 3 second media feature cells, ..., so that the straight line can react to the ordinal number of 1, 2
  • the segment of the first media object corresponding to the first media feature unit of 3, 4 is between the segment of the second media object corresponding to the segment of the second media feature corresponding to the second media feature of the ordinal number 1, 3, 5, and 7. Similar situation.
  • the similarity of the two media objects can be determined according to the straight line in the similarity matrix: it is possible to define the average case (or the overall situation) of the individual similarities included in one straight line as the straight line similarity of the straight line,
  • the straight line similarity can reflect the similarity between the corresponding plurality of first media feature monomers and the plurality of second media feature cells; determining a straight line with the highest linear similarity in the similarity matrix may be referred to as a matching straight line; Determining a straight line similarity of the matching straight line as a degree of similarity between the first media object and the second media object, and/or determining according to the plurality of first media feature monomers and the plurality of second media feature monomers corresponding to the matching straight line A matching segment of the first media object and the second media object.
  • the specific method for determining the matching segment according to the straight line in the similarity matrix may be: an ordinal number of the first media feature unit corresponding to the starting point of the straight line (or the abscissa in the similarity matrix) Determining a start time of the matching segment in the first media object, and determining a matching segment in the second media object according to an ordinal number of the second media feature unit corresponding to the starting point (or an ordinate in the similarity matrix) The start time; similarly, the end time of the matching segment in the first media object is determined according to the abscissa of the end point of the straight line, and the end time of the matching segment in the second media object is determined according to the ordinate of the end point.
  • a straight line with the highest linear similarity may be determined from a plurality of preset straight lines, for example, the preset multiple straight lines are all the slopes set to a preset slope.
  • a straight line with a fixed value such as a slope of 1
  • a straight line is fitted according to the points to generate A line that makes the straight line similarity the highest.
  • the method for comparing media features of the embodiments of the present disclosure determines the similarity degree and/or matching segments of the two media objects according to the straight line in the similarity matrix, which can greatly improve the efficiency and accuracy of the media comparison.
  • FIG. 3 is a schematic flow chart of an alignment using a dynamic programming method according to an embodiment of the present disclosure.
  • step S40 of the present disclosure includes the following specific steps:
  • Step S41a defining a plurality of straight lines whose slopes in the similarity matrix are preset slope set values as candidate straight lines, and determining a straight line similarity of the candidate straight lines according to each individual similarity included in each candidate straight line degree.
  • the straight line similarity of a straight line may be set as an average value of the individual similarities of the respective units included in the straight line, or may be set as the sum of the individual similarities of the respective units included in the straight line.
  • the slope setting value may be taken as 1, that is, the aforementioned alternative straight line is: a diagonal line in the similarity matrix and a straight line parallel to the diagonal line.
  • step S41a further includes: excluding, from the candidate line, those lines including the number of unit similarities less than a preset line length setting value, and then Proceeding to step S41b.
  • the candidate straight line must also satisfy that the number of included monomer similarities reaches a preset straight line length setting value.
  • Step S41b from the plurality of candidate straight lines, determine an alternative straight line that maximizes the similarity of the straight line, and defines it as the first matching straight line. Thereafter, the processing proceeds to step S41c.
  • Step S41c determining a straight line similarity of the first matching line as a comparison score for expressing a degree of similarity between the first media object and the second media object; determining two media objects according to the start point and the end point of the first matching line The start and end time of the matching segment in .
  • the preset slope setting value in step S41a may be multiple, that is, the candidate straight line is a straight line whose slope is equal to any one of the plurality of slope setting values,
  • the candidate straight line may be a straight line having a slope of 1, -1, 2, 1/2, etc.
  • step S41b one of the plurality of alternative straight lines having a slope of any one of the plurality of slope setting values is determined.
  • a matching line is determined.
  • the media feature comparison method proposed by the present disclosure can improve the accuracy of the comparison and the speed of the comparison by using the dynamic programming method to determine the alignment score and/or determine the matching media segments.
  • the uniform media method can also be utilized to determine the similarity of two media objects based on the similarity matrix.
  • FIG. 4 is a schematic flow chart of an alignment using a uniform media method according to an embodiment of the present disclosure. Referring to FIG. 4, in an embodiment, step S40 of the present disclosure includes the following specific steps:
  • step S42a a plurality of points having the largest single similarity are selected in the similarity matrix, and are defined as similarity extreme points.
  • the specific number of similarity extreme points taken may be preset. Thereafter, the processing proceeds to step S42b.
  • Step S42b based on the plurality of similarity extreme points, fitting a straight line as the second matching straight line in the similarity matrix.
  • a straight line having a preset slope set value or a preset slope set value is fitted as a second matching line based on the plurality of similarity extreme points, for example, fitting a line A line with a slope close to 1.
  • a random sample consistency method Random Sample Consensus method, RANSAC method for short
  • the RANSAC method is a commonly used method for calculating the mathematical model parameters of a data according to a set of sample data sets containing abnormal data to obtain valid sample data. Thereafter, the processing proceeds to step S42c.
  • Step S42c Determine a comparison score according to the plurality of single similarities included in the second matching straight line to represent the degree of similarity between the first media object and the second media object. Specifically, an average value of individual monomer similarities on the second matching straight line may be determined as the alignment score. In addition, the start and end time of the matching segments of the two media objects may be determined according to the start and end points of the second matching straight line.
  • the media feature comparison method proposed by the present disclosure can improve the accuracy of the comparison and the speed of the comparison by using the uniform media method to determine the alignment score and/or determine the matching media segments.
  • step S40 further includes: detecting the first part and the ending part of the obtained first matching line or the second matching line, and determining Whether the point (monomer similarity) of the beginning portion and the end portion of the first matching line/second matching line reaches a preset unit similarity setting value, and the beginning of the first matching line/second matching line is removed.
  • the portion of the ending that does not reach the monomer similarity setting value ie, the monomer similarity is not high
  • the degree of similarity of the media object to the second media object and/or determining the start and end time of the matching segment of the first media object and the second media object according to the start and end points of the third matching line.
  • the specific method for removing the portion of the matching straight line at the beginning/end of the matching line that does not reach the unit similarity setting value may be: checking from the start/end point of the matching straight line to the middle to determine whether the single similarity setting value is reached. After finding the first point that reaches the monomer similarity setpoint, remove the point to a number of points between the start/end point.
  • the monomer similarity setting value may be a specific value of a single unit similarity, and it is judged whether a point reaches the value during the inspection; or may be a proportional value, and a point is determined at the time of inspection. Whether the ratio value is reached compared to the average or maximum value of all points included in a matching straight line/second matching straight line.
  • the similarity matrix may be obtained by comprehensive consideration of various media similarities.
  • multiple types of first media feature sequences of the first media object and multiple types of second media feature sequences of the second media object obtained by using multiple extraction methods may be simultaneously acquired. And determining a similarity matrix according to the plurality of types of the first media feature sequence and the plurality of types of the second media feature sequences. The similarity matrix of the plurality of types of media feature sequences is then used to determine the similarity of the two media objects.
  • FIG. 5 is a schematic flow chart of determining a similarity matrix based on multiple types of first media feature sequences and second media feature sequences for media feature comparison according to an embodiment of the present disclosure.
  • the method for comparing media features of an embodiment of the present disclosure specifically includes:
  • Step S51 simultaneously acquiring multiple types of first media feature sequences of the first media object and multiple types of second media feature sequences of the second media object obtained by using multiple extraction methods, each of the first media feature sequences including A plurality of first media feature monomers, each of the second media feature sequences comprising a plurality of second media feature cells. For example, the aforementioned floating point feature sequence and the binarized feature sequence of the first media object and the second media object are simultaneously acquired. Thereafter, the processing proceeds to step S52.
  • Step S52 calculating, for each of the plurality of first media feature sequences and the plurality of second media feature sequences, a single similarity between the first media feature unit and the second media feature unit of the same type, which may be utilized.
  • the process shown in step S20 in the foregoing embodiment determines the individual monomer similarities. Thus, corresponding to a plurality of types of media feature sequences, a plurality of monomer similarities are obtained. Thereafter, the processing proceeds to step S53.
  • Step S53 determining an average value of the plurality of types of monomer similarities, determining an similarity matrix between the first media feature sequence and the second media feature sequence according to an average value of the plurality of cell similarities; or determining more
  • the minimum value of the similarity of the monomers is determined according to the minimum value of the similarity of the plurality of monomers.
  • the process shown in step S30 in the foregoing embodiment may be used to determine the similarity matrix.
  • step S40 the similarity matrix obtained based on the plurality of types of cell similarities is used in step S40 to determine the similarity of the first media object and the second media object.
  • the effect of determining the similarity matrix by using the average or minimum of a plurality of similarities is that the similarity (for example, the similarity matrix, the linear similarity, etc.) obtained by using a single media feature may have a mismatching of the media feature comparison.
  • the mismatching problem can be reduced or eliminated, thereby improving the accuracy of the media feature comparison.
  • the values of all types of monomer similarities can be determined in advance.
  • the range is set to 0 to 1, in fact, the foregoing example of the similarity of the monomers determined according to the cosine distance and the example of the similarity of the monomers determined according to the Hamming distance have taken the determined similarity of the monomers.
  • the value range is set from 0 to 1.
  • FIG. 6 is a schematic structural diagram of an embodiment of a comparison device 100 of a media feature of the present disclosure.
  • the comparing device 100 of the media feature of the example of the present disclosure mainly includes: a media feature sequence acquiring module 110 , a single similarity determining module 120 , a similarity matrix determining module 130 , and a similar situation determining module 140 .
  • the media feature sequence obtaining module 110 is configured to acquire a media feature sequence of the first media object as a first media feature sequence, and obtain a media feature sequence of the second media object as a second media feature sequence.
  • the first media object and the second media object are two media to be compared.
  • the first media feature sequence includes a plurality of first media feature cells arranged in sequence
  • the second media feature sequence includes a plurality of second media feature cells arranged in sequence.
  • the unit similarity determining module 120 is configured to determine a monomer similarity between each of the first media feature monomers and each of the second media feature cells. Each monomer similarity is used to indicate the degree of similarity between two media feature monomers. Specifically, the greater the monomer similarity, the more similar.
  • the single similarity determining module 120 includes a submodule for The unit similarity is determined according to a cosine distance (or, called cosine similarity) between the first media feature unit and the second media feature unit.
  • the single similarity determining module 120 when the first media feature sequence and the second media feature sequence acquired by the media feature sequence acquiring module 110 are simultaneously binarized features, the single similarity determining module 120 includes a submodule.
  • the unit similarity is determined according to a Hamming distance (Hamming distance) between the first media feature unit and the second media feature unit.
  • the similarity matrix determining module 130 is configured to determine a similarity matrix between the first media feature sequence and the second media feature sequence according to each individual similarity.
  • the single similarity determining module 120 and the similarity matrix determining module 130 are not necessarily independent, but the single similarity determining module 120 may be one of the similarity matrix determining modules 130.
  • the module, the similarity matrix determining module 130 is configured to determine a similarity matrix, and calculate a corresponding unit similarity in determining the respective points of the similarity matrix.
  • the similarity determining module 140 is configured to determine a similarity between the first media object and the second media object according to the similarity matrix. Specifically, the similarity determining module 140 is configured to determine, according to the similarity matrix, a similarity degree of the first media object and the second media object, and may use a comparison score to express the similarity degree, and/or according to the similarity matrix. A start time of the matching segment of the first media object and the second media object is determined.
  • the similarity determination module 140 includes a sub-module for determining a similarity of the first media object to the second media object based on a straight line in the similarity matrix.
  • the sub-module is configured to: determine a straight line with the highest linear similarity in the similarity matrix, which may be referred to as a matching straight line; determine the similarity of the straight line of the matching straight line as the similarity between the first media object and the second media object. Level, and/or determining matching segments of the first media object and the second media object according to the plurality of first media feature cells and the plurality of second media feature cells corresponding to the matching straight line.
  • the similarity determining module 140 may include a dynamic programming comparison sub-module (not shown) for determining the similarity of two media objects according to the similarity matrix by using a dynamic programming method.
  • FIG. 7 is a schematic structural diagram of a similarity determining module 140 including each unit of a dynamic programming comparison sub-module according to an embodiment of the present disclosure.
  • the similar situation determination module 140 of the example of the present disclosure includes:
  • the straight line similarity determining unit 141 is configured to determine a straight line similarity of the candidate straight line according to each individual similarity included in each candidate straight line.
  • the alternative straight line is a plurality of straight lines in which the slope in the similarity matrix is a preset slope set value.
  • the straight line similarity of a straight line may be set as an average value of the individual similarities of the respective units included in the straight line, or may be set as the sum of the individual similarities of the respective units included in the straight line.
  • the line similarity determining unit 141 further includes a subunit for excluding, from the candidate line, the number of the similarities of the included monomers is less than the preset straight line.
  • the candidate straight line utilized by the straight line similarity determining unit 141 must also satisfy that the number of included unit similarities reaches a preset straight line length setting value.
  • the first matching straight line determining unit 142 is configured to determine, from the plurality of candidate straight lines, an alternative straight line that maximizes the similarity of the straight line, and defines the first matching straight line.
  • a first comparison unit 143 configured to determine a straight line similarity of the first matching line as a comparison score for expressing a degree of similarity between the first media object and the second media object, and/or for using the first
  • the start and end points of the matching line determine the start and end time of the matching segments in the two media objects.
  • the similarity determining module 140 may include a uniform media comparison sub-module (not shown) for determining the similarity of two media objects according to the similarity matrix by using a uniform media method.
  • FIG. 8 is a schematic structural diagram of a similarity determining module 140 including each unit of a uniform media comparison sub-module according to an embodiment of the present disclosure.
  • the similar situation determination module 140 of the example of the present disclosure includes:
  • the extreme point determining unit 144 is configured to select a plurality of points with the largest single similarity in the similarity matrix, and define the similarity extreme points.
  • the second matching line determining unit 145 is configured to fit a straight line as the second matching line in the similarity matrix based on the plurality of similarity extreme points.
  • the second matching line determining unit 145 is specifically configured to fit a line having a preset slope setting value or a preset slope setting value based on the plurality of similarity extreme points.
  • Two matching lines. Specifically, the second matching straight line determining unit 145 may be configured to fit a straight line whose slope is close to the slope setting value in the similarity matrix by using a random sampling consensus method.
  • a second comparison unit 146 configured to determine a comparison score according to the plurality of unit similarities included in the second matching line (for example, an average value of each unit similarity on the second matching line may be determined
  • the degree of similarity between the first media object and the second media object is expressed, and/or used to determine the start and end time of the matching segments of the two media objects according to the start and end points of the second matching line .
  • the similarity determining module 140 further includes: a third matching straight line determining unit (not shown) for detecting the beginning and ending of the foregoing first matching straight line or the second matching straight line a part of determining whether a point (monomer similarity) of the first portion and the end portion of the first matching line/second matching line reaches a preset unit similarity setting value, and removing the first matching line/second matching line
  • the third comparison unit (not in the figure) Shown), for determining the alignment score according to the linear similarity of the third matching straight line, and determining the starting and ending time of the matching segment according to the starting point and the ending point of the third matching straight line.
  • FIG. 9 is a structural block diagram of a media feature comparison apparatus 100 for determining a similarity matrix based on a plurality of types of first media feature sequences and second media feature sequences according to an embodiment of the present disclosure.
  • the comparing device 100 of the media feature of an embodiment of the present disclosure specifically includes:
  • a multi-type media feature sequence sub-module 111 configured to simultaneously acquire multiple types of first media feature sequences of the first media object and multiple types of second media feature sequences of the second media object obtained by using multiple extraction methods,
  • Each of the first media feature sequences includes a plurality of first media feature cells
  • each of the second media feature sequences includes a plurality of second media feature cells.
  • the multi-type monomer similarity determination sub-module 121 is configured to calculate, for each of the plurality of first media feature sequences and the plurality of second media feature sequences, the same type of the first media feature unit and the second media feature unit.
  • the monomer similarity between the monomers gives a variety of monomer similarities.
  • the similarity matrix determining submodule 131 is configured to determine an average value or a minimum value of the plurality of single cell similarities, and determine the first media according to the average value or the minimum value of the plurality of single cell similarities.
  • the similarity determining module 140 is specifically configured to determine the similarity between the first media object and the second media object by using the similarity matrix obtained based on the plurality of types of monomer similarities.
  • FIG. 10 is a hardware block diagram illustrating a media feature comparison hardware device in accordance with an embodiment of the present disclosure.
  • the media feature comparison hardware device 200 according to an embodiment of the present disclosure includes a memory 201 and a processor 202.
  • the media features are interconnected by components of the hardware device 200 via a bus system and/or other form of connection mechanism (not shown).
  • the memory 201 is for storing non-transitory computer readable instructions.
  • memory 201 can include one or more computer program products, which can include various forms of computer readable storage media, such as volatile memory and/or nonvolatile memory.
  • the volatile memory may include, for example, random access memory (RAM) and/or cache or the like.
  • the nonvolatile memory may include, for example, a read only memory (ROM), a hard disk, a flash memory, or the like.
  • the processor 202 can be a central processing unit (CPU) or other form of processing unit with data processing capabilities and/or instruction execution capabilities, and can control media features to compare other components in the hardware device 200 to perform desired functions.
  • the processor 202 is configured to execute the computer readable instructions stored in the memory 201 such that the media feature compares the hardware device 200 to perform the media characteristics of the foregoing embodiments of the present disclosure. All or part of the steps of the method.
  • FIG. 11 is a schematic diagram illustrating a computer readable storage medium in accordance with an embodiment of the present disclosure.
  • a computer readable storage medium 300 according to an embodiment of the present disclosure has stored thereon non-transitory computer readable instructions 301.
  • the non-transitory computer readable instructions 301 are executed by a processor, all or part of the steps of the method of aligning the media features of the various embodiments of the present disclosure are performed.
  • FIG. 12 is a schematic diagram showing a hardware structure of a terminal device according to an embodiment of the present disclosure.
  • the terminal device may be implemented in various forms, and the terminal device in the present disclosure may include, but is not limited to, such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet), a PMP.
  • Mobile terminal devices portable multimedia players
  • navigation devices in-vehicle terminal devices, in-vehicle display terminals, in-vehicle electronic rearview mirrors, and the like, and fixed terminal devices such as digital TVs, desktop computers, and the like.
  • the terminal device 1100 may include a wireless communication unit 1110, an A/V (audio/video) input unit 1120, a user input unit 1130, a sensing unit 1140, an output unit 1150, a memory 1160, an interface unit 1170, and control.
  • Figure 12 illustrates a terminal device having various components, but it should be understood that not all illustrated components are required to be implemented. More or fewer components can be implemented instead.
  • the wireless communication unit 1110 allows radio communication between the terminal device 1100 and a wireless communication system or network.
  • the A/V input unit 1120 is for receiving an audio or video signal.
  • the user input unit 1130 can generate key input data according to a command input by the user to control various operations of the terminal device.
  • the sensing unit 1140 detects the current state of the terminal device 1100, the location of the terminal device 1100, the presence or absence of a user's touch input to the terminal device 1100, the orientation of the terminal device 1100, the acceleration or deceleration movement and direction of the terminal device 1100, and the like, and A command or signal for controlling the operation of the terminal device 1100 is generated.
  • the interface unit 1170 serves as an interface through which at least one external device can connect with the terminal device 1100.
  • Output unit 1150 is configured to provide an output signal in a visual, audio, and/or tactile manner.
  • the memory 1160 may store a software program or the like that performs processing and control operations performed by the controller 1180, or may temporarily store data that has been output or is to be output.
  • Memory 1160 can include at least one type of storage medium.
  • the terminal device 1100 can cooperate with a network storage device that performs a storage function of the memory 1160 through a network connection.
  • Controller 1180 typically controls the overall operation of the terminal device. Additionally, the controller 1180 can include a multimedia module for reproducing or playing back multimedia data.
  • the controller 1180 can perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image.
  • the power supply unit 1190 receives external power or internal power under the control of the controller 1180 and provides appropriate power required to operate the various components and components.
  • Various embodiments of the method of comparing media features proposed by the present disclosure may be implemented in a computer readable medium using, for example, computer software, hardware, or any combination thereof.
  • various embodiments of the media feature comparison method proposed by the present disclosure may be through the use of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device. (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, at least one of the electronic units designed to perform the functions described herein, in some cases
  • ASIC application specific integrated circuit
  • DSP digital signal processor
  • DSPD digital signal processing device
  • PLD programmable logic device.
  • FPGA field programmable gate array
  • processor controller, microcontroller, microprocessor, at least one of the electronic units designed to perform the functions described herein, in some cases
  • controller 1180 Various embodiments of the method of comparing media features proposed by the present disclosure may be implemented in the controller 1180.
  • various implementations of the method of comparing media features proposed by the present disclosure can be implemented with separate software modules that allow for the execution of at least one function or operation.
  • the software code can be implemented by a software application (or program) written in any suitable programming language, which can be stored in memory 1160 and executed by controller 1180.
  • the method, device, hardware device, computer readable storage medium, and terminal device of the media feature determine the similarity between the media objects based on the similarity matrix between the two media objects, Improve the efficiency and accuracy of media comparisons. Further, determining the similarity degree and/or matching segment of the two media objects according to the straight line in the similarity matrix can greatly improve the efficiency and accuracy of the media comparison; in addition, performing media features by using multiple types of media feature sequences. Comparison can greatly improve the accuracy of media comparison.
  • exemplary does not mean that the described examples are preferred or better than the other examples.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Image Analysis (AREA)

Abstract

本公开涉及一种媒体特征的比对方法及装置，该方法包括：获取第一媒体对象的第一媒体特征序列和第二媒体对象的第二媒体特征序列，所述第一媒体特征序列包含顺序排列的多个第一媒体特征单体，所述第二媒体特征序列包含顺序排列的多个第二媒体特征单体；确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度；根据所述单体相似度确定所述第一媒体特征序列与所述第二媒体特征序列之间的相似度矩阵；根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况。

Description

一种媒体特征的比对方法及装置
相关申请的交叉引用
本申请要求申请号为201810273673.2、申请日为2018年3月29日的中国专利申请的优先权，该文献的全部内容以引用方式并入本文。
技术领域
本公开涉及媒体处理技术领域，特别是涉及一种媒体特征的比对方法及装置。
背景技术
视频特征、音频特征等媒体特征(或者称为媒体指纹)以及媒体特征比对、媒体特征检索在如今的“多媒体信息社会”中具有广泛的应用。利用媒体特征比对，能够避免视频、音频的重复上传，进而防止媒体的盗用、优化媒体的存储，另外，利用媒体特征比对还能进行媒体内容监控，进行版权检测等等。
现有的媒体特征比对方法存在准确性差、效率慢的问题，这对运算资源和存储资源都会产生巨大消耗。
发明内容
本公开的目的在于，提供一种新的媒体特征的比对方法及装置。
本公开的目的是采用以下的技术方案来实现的。依据本公开提出的媒体特征的比对方法，包括以下步骤：获取第一媒体对象的第一媒体特征序列和第二媒体对象的第二媒体特征序列，所述第一媒体特征序列包含顺序排列的多个第一媒体特征单体，所述第二媒体特征序列包含顺序排列的多个第二媒体特征单体；确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度；根据所述单体相似度确定所述第一媒体特征序列与所述第二媒体特征序列之间的相似度矩阵；根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况。
本公开的目的还可以采用以下的技术措施来进一步实现。
前述的媒体特征的比对方法，其中，所述第一媒体特征单体和所述第二媒体特征单体为浮点数特征；所述的确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度包括：根据所述第一媒体特征单体与所述第二媒体特征单体之间的余弦距离，确定所述单体相似度。
前述的媒体特征的比对方法，其中，所述第一媒体特征单体和所述第二媒体特征单体为二值化特征，且具有相同的特征单体长度；所述的确定 所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度包括：根据所述第一媒体特征单体与所述第二媒体特征单体之间的汉明距离，确定所述单体相似度。
前述的媒体特征的比对方法，其中，所述的获取第一媒体对象的第一媒体特征序列和第二媒体对象的第二媒体特征序列包括，获取第一媒体对象的多种类型的所述第一媒体特征序列，并获取第二媒体对象的多种类型的所述第二媒体特征序列；所述的确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度包括，分别确定同种类型的所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度，以得到多种所述单体相似度；所述的根据所述单体相似度确定所述第一媒体特征序列与所述第二媒体特征序列之间的相似度矩阵包括，确定所述多种单体相似度的平均值或最小值，根据所述的多种单体相似度的平均值或最小值确定所述相似度矩阵。
前述的媒体特征的比对方法，其中，所述多个第一媒体特征单体在所述第一媒体特征序列中按时间顺序排列，所述多个第二媒体特征单体在所述第二媒体特征序列中按时间顺序排列。
前述的媒体特征的比对方法，其中所述相似度矩阵中的一个点对应一个所述单体相似度；所述相似度矩阵的点在横向上按照各个所述第一媒体特征单体在所述第一媒体特征序列中的先后顺序排列，且在纵向上按照各个所述第二媒体特征单体在所述第二媒体特征序列中的先后顺序排列。
前述的媒体特征的比对方法，其中，所述根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况包括：根据所述相似度矩阵中的直线确定所述第一媒体对象与所述第二媒体对象的相似程度和匹配片段。
前述的媒体特征的比对方法，其中，所述的根据所述相似度矩阵中的直线确定所述第一媒体对象与所述第二媒体对象的相似情况包括：将斜率为预设的斜率设定值的多条直线定义为备选直线，根据每条所述备选直线所包含的单体相似度的平均值或总和值，确定所述备选直线的直线相似度；在多条所述备选直线中，选取一条使得所述直线相似度最大的备选直线，并定义为第一匹配直线；根据所述第一匹配直线的所述直线相似度确定所述第一媒体对象与所述第二媒体对象的相似程度；根据所述第一匹配直线的起点和终点确定所述第一媒体对象与所述第二媒体对象的匹配片段的起止时间。
前述的媒体特征的比对方法，其中，所述斜率设定值为多个，所述备选直线为斜率为所述多个斜率设定值中任意一个的直线。
前述的媒体特征的比对方法，其中，所述的根据所述相似度矩阵确定 所述第一媒体对象与所述第二媒体对象的相似情况包括：在所述相似度矩阵中选取使得所述单体相似度最大的多个点作为相似度极值点；基于所述多个相似度极值点，在所述相似度矩阵中拟合出一条直线作为第二匹配直线；根据所述第二匹配直线所包含的所述单体相似度的平均值或总和值确定所述第一媒体对象与所述第二媒体对象的相似程度；根据所述第二匹配直线的起点和终点确定所述第一媒体对象与所述第二媒体对象的匹配片段的起止时间。
前述的媒体特征的比对方法，其中，所述的基于所述多个相似度极值点，在所述相似度矩阵中拟合出一条直线作为第二匹配直线包括：利用随机抽样一致法，在所述相似度矩阵中拟合出一条斜率为预设的斜率设定值或斜率接近预设的斜率设定值的直线作为第二匹配直线。
前述的媒体特征的比对方法，其中，所述的根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况还包括：判断所述第一匹配直线或所述第二匹配直线的开头和结尾的点是否达到预设的单体相似度设定值，去掉所述开头和所述结尾的未达到所述单体相似度设定值的部分，保留中间一段直线并定义为第三匹配直线；根据所述第三匹配直线的所述直线相似度确定所述第一媒体对象与所述第二媒体对象的相似程度，根据所述第三匹配直线的起点和终点确定匹配片段的起止时间。
本公开的目的还采用以下技术方案来实现。依据本公开提出的媒体特征的比对装置，包括：媒体特征序列获取模块，用于获取第一媒体对象的第一媒体特征序列和第二媒体对象的第二媒体特征序列，所述第一媒体特征序列包含顺序排列的多个第一媒体特征单体，所述第二媒体特征序列包含顺序排列的多个第二媒体特征单体；单体相似度确定模块，用于确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度；相似度矩阵确定模块，用于根据所述单体相似度确定所述第一媒体特征序列与所述第二媒体特征序列之间的相似度矩阵；相似情况确定模块，用于根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况。
本公开的目的还可以采用以下的技术措施来进一步实现。
前述的媒体特征的比对装置，其还包括执行前述任一媒体特征的比对方法步骤的模块。
本公开的目的还采用以下技术方案来实现。依据本公开提出的一种媒体特征比对硬件装置，包括：存储器，用于存储非暂时性计算机可读指令；以及处理器，用于运行所述计算机可读指令，使得所述处理器执行时实现前述任意一种媒体特征的比对方法。
本公开的目的还采用以下技术方案来实现。依据本公开提出的一种终端设备，包括前述任意一种媒体特征比对装置。
本公开的目的还采用以下技术方案来实现。依据本公开提出的一种计算机可读存储介质，用于存储非暂时性计算机可读指令，当所述非暂时性计算机可读指令由计算机执行时，使得所述计算机执行前述任意一种媒体特征的比对方法。
上述说明仅是本公开技术方案的概述，为了能更清楚了解本公开的技术手段，而可依照说明书的内容予以实施，并且为让本公开的上述和其他目的、特征和优点能够更明显易懂，以下特举较佳实施例,并配合附图，详细说明如下。
附图说明
图1是本公开一个实施例的媒体特征的比对方法的流程框图。
图2是本公开一个实施例提供的相似度矩阵所对应的灰度示意图。
图3是本公开一个实施例提供的利用动态规划法进行比对的流程框图。
图4是本公开一个实施例提供的利用匀速媒体法进行比对的流程框图。
图5是本公开一个实施例提供的基于多种类型的媒体特征序列确定相似度矩阵的流程框图。
图6是本公开一个实施例的媒体特征的比对装置的结构框图。
图7是本公开一个实施例提供的相似情况确定模块的结构框图。
图8是本公开另一实施例提供的相似情况确定模块的结构框图。
图9是本公开一个实施例的基于多种类型媒体特征序列确定相似度矩阵的媒体特征比对装置的结构框图。
图10是本公开一个实施例的媒体特征比对硬件装置的硬件框图。
图11是本公开一个实施例的计算机可读存储介质的示意图。
图12是本公开一个实施例的终端设备的结构框图。
具体实施方式
为更进一步阐述本公开为达成预定发明目的所采取的技术手段及功效，以下结合附图及较佳实施例，对依据本公开提出的媒体特征的比对方法及装置的具体实施方式、结构、特征及其功效，详细说明如后。
图1为本公开的媒体特征的比对方法一个实施例的示意性流程框图。请参阅图1，本公开示例的媒体特征的比对方法，主要包括以下步骤：
步骤S10，获取第一媒体对象的媒体特征序列作为第一媒体特征序列，获取第二媒体对象的媒体特征序列作为第二媒体特征序列。其中的第一媒体对象、第二媒体对象为待比对的两个媒体，例如可以是音频、视频、多张连拍的照片等各种类型的媒体。其中的媒体特征序列可以是音频特征、视频特征或图像特征等等，事实上可以按照本公开的方法通过获取视频对象的音频特征来比对视频对象。
具体地，第一媒体特征序列包含顺序排列的多个第一媒体特征单体，第二媒体特征序列包括顺序排列的多个第二媒体特征单体，不妨假设第一媒体特征序列、第二媒体特征序列的长度分别为M 1和M 2，其中的M 1和M 2为正整数，也就是说第一媒体特征序列包含M 1个第一媒体特征单体，第二媒体特征序列包含M 2个第二媒体特征单体。此后，处理进到步骤S20。
进一步地，在一些实施例中，这里所说的顺序排列为，在第一/第二媒体特征序列中该多个第一/第二媒体特征单体是按时间的先后顺序排列的：例如，在预先的提取媒体特征的过程中，先对媒体对象进行抽帧，再根据每一帧生成一个媒体特征单体，从而各个媒体特征单体与媒体对象的各个帧相对应，然后将各个媒体特征单体按照各个帧在媒体对象中的时间顺序进行排列得到媒体特征序列。因此也可以将前述的媒体特征单体称为帧特征，并将前述的媒体特征序列称为媒体特征。
值得注意的是，对媒体特征序列的提取方法以及媒体特征序列的类型不做限制，但第一媒体特征序列与第二媒体特征序列应该是通过同种特征提取方法得到的同种类型的媒体特征。在本公开的一种示例中，可以同时获取第一媒体对象和第二媒体对象的浮点数特征序列作为第一媒体特征序列和第二媒体特征序列，浮点数特征序列中的每个媒体特征单体是一个浮点数。而在另一种示例中，也可以同时获取第一媒体对象和第二媒体对象的二值数特征序列，或者将获得的它种类型的媒体特征进行二值化，以得到二值数特征序列。其中，二值数特征序列中的每个特征单体是一段由0/1组成的比特串，而通过同种方法提取得到的媒体特征单体具有相同的长度(或称为比特数)。
步骤S20，确定每个第一媒体特征单体与每个第二媒体特征单体之间的单体相似度，得到M 1×M 2个单体相似度。每个单体相似度用于表示两个媒体特征单体之间的相似程度，具体可以是，单体相似度越大表示越相似。此后，处理进到步骤S30。
具体地，可以根据媒体特征的类型，选择能够判断两个媒体特征的相似程度的距离或度量作为该单体相似度。
在本公开的实施例中，当第一媒体特征序列与第二媒体特征序列同时为浮点数特征时，可根据第一媒体特征单体与第二媒体特征单体之间的余弦距离(或者，称为余弦相似度)确定该单体相似度；一般可直接将该余弦距离确定为单体相似度。
在本公开的实施例中，当第一媒体特征序列与第二媒体特征序列同时为二值数特征时，可根据第一媒体特征单体与第二媒体特征单体之间的汉明距离(Hamming距离)确定该单体相似度。具体地，先计算第一媒体特征单体与第二媒体特征单体之间的汉明距离(Hamming距离)，再计算特征单 体长度(比特数)与该汉明距离的差值，并将该差值与该特征单体长度的比值确定为单体相似度，用以表示两个二值化特征中的相同比特所占的比例。其中的汉明距离是一种信息论领域中常用的度量，两个等长字符串之间的汉明距离是两个字符串对应位置的不同字符的个数。在实际计算汉明距离时，可以对两个字符串进行异或运算，并统计结果为1的个数，而这个数就是汉明距离。
值得注意的是，不限于利用余弦距离或汉明距离表示该单体相似度，而是可以利用任何可以判断两个媒体特征单体的相似程度的距离或度量。
需要说明的是，如果各个媒体特征单体与媒体对象的各个帧相对应则也可将单体相似度称为帧间相似度。
步骤S30，根据各个单体相似度，确定第一媒体特征序列与第二媒体特征序列之间的相似度矩阵(Similarity Matrix)。
具体地，该相似度矩阵中的每个点对应一个单体相似度，使得该相似度矩阵记录有各个第一媒体特征单体与各个第二媒体特征单体之间的单体相似度。并且，该相似度矩阵的各个点：在横向上按照各个第一媒体特征单体在第一媒体特征序列中的先后顺序排列，且在纵向上按照各个第二媒体特征单体在第二媒体特征序列中的先后顺序排列。从而位于第i行第j列的点表示第一媒体对象的第i个第一媒体特征单体与第二媒体对象的第j个第二媒体特征单体之间的单体相似度，进而该相似度矩阵为一个M 1×M 2矩阵。此后，处理进到步骤S40。
为了便于直观，可以将该相似度矩阵转化为如图2所示的灰度示意图，在图2中，利用每个点的灰度来表示相似度矩阵中对应位置的单体相似度的大小。具体地，如果一个点的灰度越接近白色，则表示该点对应的单体相似度越高，例如图2中标出的I处的点；而如果一个点的灰度越接近黑色，则表示该点对应的单体相似度越低，例如图2中标出的II处的点。
需要说明的是，在实际操作中，并非一定先进行步骤S20的计算各个单体相似度，再进行步骤S30的确定相似度矩阵，而是可以直接确定相似度矩阵，在确定该相似度矩阵的各个点的过程中计算对应的单体相似度。
步骤S40，根据该相似度矩阵，确定第一媒体对象与第二媒体对象的相似情况。具体地，所谓的确定相似情况包括：根据该相似度矩阵来确定第一媒体对象与第二媒体对象的相似程度并可以利用比对评分来表现该相似程度，和/或根据该相似度矩阵来确定第一媒体对象和第二媒体对象的匹配片段的起止时间。其中，比对评分可以是一个0到1之间的分数，数字越大表示两段媒体对象越相似。
本公开实施例的媒体特征的比对方法，基于两个媒体对象之间的相似度矩阵来确定媒体对象之间的相似情况，能够提高媒体比对的效率和准确 率。
在本公开的一些实施例中，步骤S40包括：根据相似度矩阵中的直线来确定第一媒体对象与第二媒体对象的相似情况。
需注意，由于媒体特征序列一般包含有穷的多个媒体特征单体，从而相似度矩阵为有穷矩阵，因此实际上所谓的“直线”是相似度矩阵中的多个点组成的有穷长的线段。该直线具有斜率，该斜率为直线所包括的多个点的连线的斜率。另外，该直线的起点和终点可以是相似度矩阵中的任意的点，不必是位于边缘的点。
本公开所说的直线包括相似度矩阵中的对角线、与该对角线相平行的各条线段这些在相似度矩阵中从左上到右下的斜率为1的直线(如图2中标出的直线III)，还包括斜率不为1的直线。例如，可以是的斜率近似于1的直线，以提高媒体比对的鲁棒性；可以是斜率为2、3、...或1/2、1/3、...等的直线，以应对经过调速的媒体对象的比对；甚至可以是斜率为负数的直线(在相似度矩阵中从左下到右上的直线)，以应对经过反向播放处理的媒体对象。其中的对角线为由位于(1,1)、(2,2)、(3,3)...的点组成的线段(事实上就是以左上角的点为起点且斜率为1的一条直线)。
事实上，相似度矩阵中的每条直线均由顺序排列的多个单体相似度构成，因此由于每条直线表现了多个顺序排列的媒体特征单体对的相似情况，从而能够表现一段第一媒体对象片段与一段第二媒体对象片段的相似程度。其中每个媒体特征单体对包括一个第一媒体特征单体和一个第二媒体特征单体。也就是说，每条直线表现了多个顺序排列的第一媒体特征单体与多个顺序排列的第二媒体特征单体之间的相似程度。而直线的斜率、起点终点表现了两段媒体片段的长度、位置。例如，由(1,1)、(2,3)、(3,5)、(4,7)构成的直线，由于表现了序数为1的第一媒体特征单体与序数为1第二媒体特征单体之间的相似情况、序数为2的第一媒体特征单体与序数为3第二媒体特征单体之间的相似情况、...，从而该直线能够反应序数为1、2、3、4的第一媒体特征单体所对应的一段第一媒体对象的片段与序数为1、3、5、7的第二媒体特征单体所对应的一段第二媒体对象的片段之间的相似情况。
因此，可以根据相似度矩阵中的直线来确定两个媒体对象的相似情况：不妨将一个直线所包含的各个单体相似度的平均情况(或总体情况)定义为该直线的直线相似度，该直线相似度能够体现对应的多个第一媒体特征单体与多个第二媒体特征单体之间的相似情况；在相似度矩阵中确定一条直线相似度最高的直线，不妨称为匹配直线；将匹配直线的直线相似度确定为第一媒体对象与第二媒体对象的相似程度，和/或根据匹配直线所对应的多个第一媒体特征单体和多个第二媒体特征单体来确定第一媒体对象与 第二媒体对象的匹配片段。
其中的根据相似度矩阵中的直线(例如匹配直线)来确定匹配片段的具体方法可以是：根据直线的起点所对应的第一媒体特征单体的序数(或者说，相似度矩阵中的横坐标)确定第一媒体对象中的匹配片段的开始时间，而根据该起点所对应的第二媒体特征单体的序数(或者说，相似度矩阵中的纵坐标)确定第二媒体对象中的匹配片段的开始时间；类似地，根据直线的终点的横坐标确定第一媒体对象中的匹配片段的结束时间，而根据该终点的纵坐标确定第二媒体对象中的匹配片段的结束时间。
需要注意的是，在确定匹配直线的过程中，可以是从预设的多条直线中确定一条直线相似度最高的直线，例如该预设的多条直线为所有的斜率为预设的斜率设定值(比如斜率为1)的直线，或者，也可以是先从相似度矩阵中选取使得单体相似度的大小排名靠前的多个点，再根据这些点拟合出一条直线，以生成一条使得直线相似度相对最高的直线。
本公开实施例的媒体特征的比对方法，根据相似度矩阵中的直线确定两个媒体对象的相似程度和/或匹配片段，能够大大提高媒体比对的效率和准确率。
在本公开的一个具体实施例中，可以利用动态规划法来根据相似度矩阵确定两个媒体对象的相似情况。图3为本公开一个实施例提供的利用动态规划法进行比对的示意性流程框图。请参阅图3，在一种实施例中，本公开的步骤S40包括以下具体步骤：
步骤S41a，将相似度矩阵中的斜率为预设的斜率设定值的多条直线定义为备选直线，根据每条备选直线所包含的各个单体相似度确定该备选直线的直线相似度。具体地，一条直线的直线相似度可以设置为该直线所包含的各个单体相似度的平均值，或者可以设置为该直线所包含的各个单体相似度的总和值。在一种具体示例中，可以将斜率设定值取为1，即前述的备选直线为：相似度矩阵中的对角线以及与该对角线平行的直线。此后，处理进到步骤S41b。
需要注意的是，在本公开的一种实施例中，步骤S41a还包括：先从备选直线中排除那些包含的单体相似度的数量少于预设的直线长度设定值的直线，然后再进到步骤S41b。或者说，在本实施例中，备选直线还须满足：包含的单体相似度的数量达到预设的直线长度设定值。通过排除单体相似度过少的直线，可以排除当直线包含的单体相似度过少而影响最终得到的比对结果的准确性的问题。
步骤S41b，从该多条备选直线中，确定一条使得该直线相似度最大的备选直线，并定义为第一匹配直线。此后，处理进到步骤S41c。
步骤S41c，将该第一匹配直线的直线相似度确定为比对评分，用以表 现第一媒体对象与第二媒体对象的相似程度；根据该第一匹配直线的起点和终点确定两个媒体对象中的匹配片段的起止时间。
需要注意的是，在本公开的一些实施例中，步骤S41a中的预设的斜率设定值可以为多个，即备选直线为斜率与多个斜率设定值中任意一个相等的直线，例如备选直线可以为斜率为1、-1、2、1/2等的直线，并且在步骤S41b中，从斜率为多个斜率设定值中任意一个的多条备选直线中确定一条第一匹配直线。
本公开提出的媒体特征比对方法，通过利用动态规划法来确定比对评分和/或确定相匹配的媒体片段，能够提高比对的准确性和比对的速度。
在本公开的另一个具体实施例中，也可以利用匀速媒体法来根据相似度矩阵确定两个媒体对象的相似情况。图4为本公开一个实施例提供的利用匀速媒体法进行比对的示意性流程框图。请参阅图4，在一种实施例中，本公开的步骤S40包括以下具体步骤：
步骤S42a，在相似度矩阵中选取单体相似度最大的多个点，并定义为相似度极值点。所取的相似度极值点的具体数量可以是预设的。此后，处理进到步骤S42b。
步骤S42b，基于该多个相似度极值点，在该相似度矩阵中拟合出一条直线作为第二匹配直线。在一些具体示例中，基于该多个相似度极值点拟合出一条具有预设的斜率设定值或接近预设的斜率设定值的直线作为第二匹配直线，例如，拟合出一条斜率接近1的直线。具体地，可以利用随机抽样一致法(Random Sample Consensus法，简称为RANSAC法)在该相似度矩阵中拟合出一条斜率接近斜率设定值的直线。其中的RANSAC法是一种常用的根据一组包含异常数据的样本数据集，计算出数据的数学模型参数，以得到有效样本数据的方法。此后，处理进到步骤S42c。
步骤S42c，根据该第二匹配直线所包含的多个单体相似度来确定比对评分，用以表现第一媒体对象与第二媒体对象的相似程度。具体地，可以将该第二匹配直线上的各个单体相似度的平均值确定为该比对评分。另外，可以根据该第二匹配直线的起点和终点确定两个媒体对象中的匹配片段的起止时间。
本公开提出的媒体特征比对方法，通过利用匀速媒体法来确定比对评分和/或确定相匹配的媒体片段，能够提高比对的准确性和比对的速度。
在本公开的一些实施例中(例如前述的图3和图4所示的实施例)，步骤S40还包括：检测所得到的第一匹配直线或第二匹配直线的开头部分和结尾部分，判断该第一匹配直线/第二匹配直线的开头部分和结尾部分的点(单体相似度)是否达到预设的单体相似度设定值，去掉第一匹配直线/第二匹配直线的开头和结尾的未达到该单体相似度设定值(即单体相似度不 高)的部分，保留中间一段直线并定义为第三匹配直线；根据该第三匹配直线的直线相似度来确定第一媒体对象与第二媒体对象的相似程度，和/或根据该第三匹配直线的起点和终点确定第一媒体对象与第二媒体对象的匹配片段的起止时间。通过去掉匹配直线开头结尾的相似度不高的部分、保留中间一段相似度较高的直线之后，再确定第一媒体对象与第二媒体对象的相似情况，能够提高比对的准确性，能够更准确地得到匹配片段的起止时间。
其中的去掉匹配直线开头/结尾的未达到该单体相似度设定值的部分的具体方法可以是：从匹配直线的起点/终点向中间依次检查，判断是否达到该单体相似度设定值，在找到第一个达到该单体相似度设定值的点后，去掉该点到起点/终点之间的多个点。
需要注意的是，该单体相似度设定值可以是一个单体相似度的具体数值，在检查时判断一个点是否达到该数值；也可以是一个比例值，在检查时判断一个点与第一匹配直线/第二匹配直线所包含的所有点的平均值或最大值相比，是否达到该比例值。
进一步地，其中相似度矩阵可以是由多种媒体相似度综合考量得到的。具体地，在本公开的实施例中，可以同时获取利用多种提取方法得到的第一媒体对象的多种类型的第一媒体特征序列和第二媒体对象的多种类型的第二媒体特征序列，根据多种类型的第一媒体特征序列和多种类型的第二媒体特征序列确定相似度矩阵。然后利用基于多种类型媒体特征序列的相似度矩阵来确定两个媒体对象的相似情况。
图5为本公开一个实施例的基于多种类型的第一媒体特征序列和第二媒体特征序列来确定相似度矩阵以进行媒体特征比对的示意性流程框图。请参阅图5，本公开的一个实施例的媒体特征的比对方法具体包括：
步骤S51，同时获取利用多种提取方法得到的第一媒体对象的多种类型的第一媒体特征序列和第二媒体对象的多种类型的第二媒体特征序列，每种第一媒体特征序列包含多个第一媒体特征单体，每种第二媒体特征序列包含多个第二媒体特征单体。例如，同时获取第一媒体对象和第二媒体对象的前述的浮点数特征序列和二值化特征序列。此后，处理进到步骤S52。
步骤S52，针对多种第一媒体特征序列和多种第二媒体特征序列，分别计算同种类型的第一媒体特征单体与第二媒体特征单体之间的单体相似度，具体可以利用前述实施例中的步骤S20所示过程来确定各个单体相似度。从而对应多种类型的媒体特征序列，得到多种单体相似度。此后，处理进到步骤S53。
步骤S53，确定该多种单体相似度的平均值，根据该多种单体相似度的平均值来确定第一媒体特征序列与第二媒体特征序列之间的相似度矩阵； 或者，确定多种单体相似度的最小值，根据该多种单体相似度的最小值来确定该相似度矩阵，具体可以利用前述实施例中的步骤S30所示过程来确定相似度矩阵。
此后，处理进到前述示例的步骤S40，并在步骤S40中利用该基于多种类型单体相似度得到的相似度矩阵来确定第一媒体对象与第二媒体对象的相似情况。
利用多种相似度的平均值或最小值确定相似度矩阵的效果在于：利用单种媒体特征得到相似度(例如前述的相似度矩阵、直线相似度等)进行媒体特征比对可能存在误匹配的情况，通过取多种媒体特征的相似度的平均值或取最小值，能够减少或排除该误匹配问题，进而提高媒体特征比对的准确性。
需要说明的是，在取多种单体相似度的平均值或最小值之前，需要确保各种单体相似度具有一致的取值范围，例如可以预先将所有类型的单体相似度的取值范围均设置为0到1，事实上，前述的根据余弦距离确定的单体相似度的示例以及根据汉明距离确定的单体相似度的示例，均已将所确定的单体相似度的取值范围设置为0到1。
图6为本公开的媒体特征的比对装置100一个实施例的示意性结构图。请参阅图6，本公开示例的媒体特征的比对装置100，主要包括：媒体特征序列获取模块110、单体相似度确定模块120、相似度矩阵确定模块130以及相似情况确定模块140。
该媒体特征序列获取模块110用于获取第一媒体对象的媒体特征序列作为第一媒体特征序列，获取第二媒体对象的媒体特征序列作为第二媒体特征序列。其中的第一媒体对象、第二媒体对象为待比对的两个媒体。具体地，第一媒体特征序列包含顺序排列的多个第一媒体特征单体，第二媒体特征序列包括顺序排列的多个第二媒体特征单体。
该单体相似度确定模块120用于确定各个第一媒体特征单体与各个第二媒体特征单体之间的单体相似度。每个单体相似度用于表示两个媒体特征单体之间的相似程度，具体可以是，单体相似度越大表示越相似。
在本公开的实施例中，当媒体特征序列获取模块110获取的第一媒体特征序列与第二媒体特征序列同时为浮点数特征时，该单体相似度确定模块120包括一个子模块，用于根据第一媒体特征单体与第二媒体特征单体之间的余弦距离(或者，称为余弦相似度)确定该单体相似度。
在本公开的实施例中，当媒体特征序列获取模块110获取的第一媒体特征序列与第二媒体特征序列同时为二值化特征时，该单体相似度确定模块120包括一个子模块，用于根据第一媒体特征单体与第二媒体特征单体之间的汉明距离(Hamming距离)确定该单体相似度。
该相似度矩阵确定模块130用于根据各个单体相似度，确定第一媒体特征序列与第二媒体特征序列之间的相似度矩阵。
需要说明的是，在实际操作中，单体相似度确定模块120与相似度矩阵确定模块130并非一定是独立的，而是单体相似度确定模块120可以是相似度矩阵确定模块130的一个子模块，该相似度矩阵确定模块130用于确定相似度矩阵，并在确定该相似度矩阵的各个点的过程中计算对应的单体相似度。
该相似情况确定模块140用于根据该相似度矩阵，确定第一媒体对象与第二媒体对象的相似情况。具体地，相似情况确定模块140用于根据该相似度矩阵来确定第一媒体对象与第二媒体对象的相似程度并可以利用比对评分来表现该相似程度，和/或根据该相似度矩阵来确定第一媒体对象和第二媒体对象的匹配片段的起止时间。
在本公开的一些实施例中，相似情况确定模块140包括一个子模块，用于基于相似度矩阵中的直线来确定第一媒体对象与第二媒体对象的相似情况。具体地，该子模块用于：在相似度矩阵中确定一条直线相似度最高的直线，不妨称为匹配直线；将匹配直线的直线相似度的确定为第一媒体对象与第二媒体对象的相似程度，和/或根据匹配直线所对应的多个第一媒体特征单体和多个第二媒体特征单体来确定第一媒体对象与第二媒体对象的匹配片段。
在本公开的一个具体实施例中，该相似情况确定模块140可以包括动态规划比对子模块(图中未示出)，用于利用动态规划法来根据相似度矩阵确定两个媒体对象的相似情况。图7为本公开一个实施例提供的包含动态规划比对子模块的各个单元的相似情况确定模块140的示意性结构图。请参阅图7，在一个实施例中，本公开示例的相似情况确定模块140包括：
直线相似度确定单元141，用于根据每条备选直线所包含的各个单体相似度确定该备选直线的直线相似度。其中备选直线为相似度矩阵中的斜率为预设的斜率设定值的多条直线。具体地，一条直线的直线相似度可以设置为该直线所包含的各个单体相似度的平均值，或者可以设置为该直线所包含的各个单体相似度的总和值。
需要注意的是，在本公开的一种实施例中，直线相似度确定单元141还包括一个子单元，用于从备选直线中排除那些包含的单体相似度的数量少于预设的直线长度设定值的直线。或者，直线相似度确定单元141所利用的备选直线还须满足：包含的单体相似度的数量达到预设的直线长度设定值。
第一匹配直线确定单元142，用于从该多条备选直线中，确定一条使得该直线相似度最大的备选直线，并定义为第一匹配直线。
第一比对单元143，用于将该第一匹配直线的直线相似度确定为比对评分，用以表现第一媒体对象与第二媒体对象的相似程度，和/或用于根据该第一匹配直线的起点和终点确定两个媒体对象中的匹配片段的起止时间。
在本公开的一个具体实施例中，该相似情况确定模块140可以包括匀速媒体比对子模块(图中未示出)，用于利用匀速媒体法来根据相似度矩阵确定两个媒体对象的相似情况。图8为本公开一个实施例提供的包含匀速媒体比对子模块的各个单元的相似情况确定模块140的示意性结构图。请参阅图8，在一个实施例中，本公开示例的相似情况确定模块140包括：
极值点确定单元144，用于在相似度矩阵中选取单体相似度最大的多个点，并定义为相似度极值点。
第二匹配直线确定单元145，用于基于该多个相似度极值点，在该相似度矩阵中拟合出一条直线作为第二匹配直线。在一些示例中，该第二匹配直线确定单元145具体用于基于该多个相似度极值点拟合出一条具有预设的斜率设定值或接近预设的斜率设定值的直线作为第二匹配直线。具体地，该第二匹配直线确定单元145可以用于利用随机抽样一致法在该相似度矩阵中拟合出一条斜率接近斜率设定值的直线。
第二比对单元146，用于根据该第二匹配直线所包含的多个单体相似度来确定比对评分(例如，可将该第二匹配直线上的各个单体相似度的平均值确定为该比对评分)，用以表现第一媒体对象与第二媒体对象的相似程度，和/或用于根据该第二匹配直线的起点和终点确定两个媒体对象中的匹配片段的起止时间。
在本公开的一些实施例中，该相似情况确定模块140还包括：第三匹配直线确定单元(图中未示出)用于检测前述的第一匹配直线或第二匹配直线的开头部分和结尾部分，判断该第一匹配直线/第二匹配直线的开头部分和结尾部分的点(单体相似度)是否达到预设的单体相似度设定值，去掉第一匹配直线/第二匹配直线的开头和结尾的未达到该单体相似度设定值(即单体相似度不高)的部分，保留中间一段直线并定义为第三匹配直线；以及，第三比对单元(图中未示出)，用于根据该第三匹配直线的直线相似度确定该比对评分，并且根据该第三匹配直线的起点和终点确定匹配片段的起止时间。
进一步地，其中相似度矩阵可以是由多种媒体相似度综合考量得到的。图9为本公开一个实施例的基于多种类型的第一媒体特征序列、第二媒体特征序列来确定相似度矩阵的媒体特征比对装置100的结构框图。请参阅图9，本公开的一个实施例的媒体特征的比对装置100具体包括：
多类型媒体特征序列子模块111，用于同时获取利用多种提取方法得到的第一媒体对象的多种类型的第一媒体特征序列和第二媒体对象的多种类 型的第二媒体特征序列，每种第一媒体特征序列包含多个第一媒体特征单体，每种第二媒体特征序列包含多个第二媒体特征单体。
多类型单体相似度确定子模块121，用于针对多种第一媒体特征序列和多种第二媒体特征序列，分别计算同种类型的第一媒体特征单体与第二媒体特征单体之间的单体相似度，得到多种单体相似度。
基于多类型媒体特征的相似度矩阵确定子模块131，用于确定该多种单体相似度的平均值或最小值，根据该多种单体相似度的平均值或最小值来确定第一媒体特征序列与第二媒体特征序列之间的相似度矩阵。
并且，相似情况确定模块140具体用于利用该基于多种类型单体相似度得到的相似度矩阵来确定第一媒体对象与第二媒体对象的相似情况。
图10是图示根据本公开的实施例的媒体特征比对硬件装置的硬件框图。如图10所示，根据本公开实施例的媒体特征比对硬件装置200包括存储器201和处理器202。媒体特征比对硬件装置200中的各组件通过总线系统和/或其它形式的连接机构(未示出)互连。
该存储器201用于存储非暂时性计算机可读指令。具体地，存储器201可以包括一个或多个计算机程序产品，该计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。该易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。该非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。
该处理器202可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元，并且可以控制媒体特征比对硬件装置200中的其它组件以执行期望的功能。在本公开的一个实施例中，该处理器202用于运行该存储器201中存储的该计算机可读指令，使得该媒体特征比对硬件装置200执行前述的本公开各实施例的媒体特征的比对方法的全部或部分步骤。
图11是图示根据本公开的实施例的计算机可读存储介质的示意图。如图11所示，根据本公开实施例的计算机可读存储介质300，其上存储有非暂时性计算机可读指令301。当该非暂时性计算机可读指令301由处理器运行时，执行前述的本公开各实施例的媒体特征的比对方法的全部或部分步骤。
图12是图示根据本公开实施例的终端设备的硬件结构示意图。终端设备可以以各种形式来实施，本公开中的终端设备可以包括但不限于诸如移动电话、智能电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、导航装置、车载终端设备、车载显示终端、车载电子后视镜等等的移动终端设备以及诸如数字TV、台 式计算机等等的固定终端设备。
如图12所示，终端设备1100可以包括无线通信单元1110、A/V(音频/视频)输入单元1120、用户输入单元1130、感测单元1140、输出单元1150、存储器1160、接口单元1170、控制器1180和电源单元1190等等。图12示出了具有各种组件的终端设备，但是应理解的是，并不要求实施所有示出的组件。可以替代地实施更多或更少的组件。
其中，无线通信单元1110允许终端设备1100与无线通信系统或网络之间的无线电通信。A/V输入单元1120用于接收音频或视频信号。用户输入单元1130可以根据用户输入的命令生成键输入数据以控制终端设备的各种操作。感测单元1140检测终端设备1100的当前状态、终端设备1100的位置、用户对于终端设备1100的触摸输入的有无、终端设备1100的取向、终端设备1100的加速或减速移动和方向等等，并且生成用于控制终端设备1100的操作的命令或信号。接口单元1170用作至少一个外部装置与终端设备1100连接可以通过的接口。输出单元1150被构造为以视觉、音频和/或触觉方式提供输出信号。存储器1160可以存储由控制器1180执行的处理和控制操作的软件程序等等，或者可以暂时地存储己经输出或将要输出的数据。存储器1160可以包括至少一种类型的存储介质。而且，终端设备1100可以与通过网络连接执行存储器1160的存储功能的网络存储装置协作。控制器1180通常控制终端设备的总体操作。另外，控制器1180可以包括用于再现或回放多媒体数据的多媒体模块。控制器1180可以执行模式识别处理，以将在触摸屏上执行的手写输入或者图片绘制输入识别为字符或图像。电源单元1190在控制器1180的控制下接收外部电力或内部电力并且提供操作各元件和组件所需的适当的电力。
本公开提出的媒体特征的比对方法的各种实施方式可以以使用例如计算机软件、硬件或其任何组合的计算机可读介质来实施。对于硬件实施，本公开提出的媒体特征的比对方法的各种实施方式可以通过使用特定用途集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理装置(DSPD)、可编程逻辑装置(PLD)、现场可编程门阵列(FPGA)、处理器、控制器、微控制器、微处理器、被设计为执行这里描述的功能的电子单元中的至少一种来实施，在一些情况下，本公开提出的媒体特征的比对方法的各种实施方式可以在控制器1180中实施。对于软件实施，本公开提出的媒体特征的比对方法的各种实施方式可以与允许执行至少一种功能或操作的单独的软件模块来实施。软件代码可以由以任何适当的编程语言编写的软件应用程序(或程序)来实施，软件代码可以存储在存储器1160中并且由控制器1180执行。
以上，根据本公开实施例的媒体特征的比对方法、装置、硬件装置、 计算机可读存储介质以及终端设备，基于两个媒体对象之间的相似度矩阵确定媒体对象之间的相似情况，能够提高媒体比对的效率和准确率。进一步地，根据相似度矩阵中的直线确定两个媒体对象的相似程度和/或匹配片段，能够大大提高媒体比对的效率和准确率；另外，通过基于多种类型的媒体特征序列进行媒体特征比对，能够大大提高媒体比对的准确率。
以上结合具体实施例描述了本公开的基本原理，但是，需要指出的是，在本公开中提及的优点、优势、效果等仅是示例而非限制，不能认为这些优点、优势、效果等是本公开的各个实施例必须具备的。另外，上述公开的具体细节仅是为了示例的作用和便于理解的作用，而非限制，上述细节并不限制本公开为必须采用上述具体的细节来实现。
本公开中涉及的器件、装置、设备、系统的方框图仅作为例示性的例子并且不意图要求或暗示必须按照方框图示出的方式进行连接、布置、配置。如本领域技术人员将认识到的，可以按任意方式连接、布置、配置这些器件、装置、设备、系统。诸如“包括”、“包含”、“具有”等等的词语是开放性词汇，指“包括但不限于”，且可与其互换使用。这里所使用的词汇“或”和“和”指词汇“和/或”，且可与其互换使用，除非上下文明确指示不是如此。这里所使用的词汇“诸如”指词组“诸如但不限于”，且可与其互换使用。
另外，如在此使用的，在以“至少一个”开始的项的列举中使用的“或”指示分离的列举，以便例如“A、B或C的至少一个”的列举意味着A或B或C，或AB或AC或BC，或ABC(即A和B和C)。此外，措辞“示例的”不意味着描述的例子是优选的或者比其他例子更好。
还需要指出的是，在本公开的系统和方法中，各部件或各步骤是可以分解和/或重新组合的。这些分解和/或重新组合应视为本公开的等效方案。
可以不脱离由所附权利要求定义的教导的技术而进行对在此所述的技术的各种改变、替换和更改。此外，本公开的权利要求的范围不限于以上所述的处理、机器、制造、事件的组成、手段、方法和动作的具体方面。可以利用与在此所述的相应方面进行基本相同的功能或者实现基本相同的结果的当前存在的或者稍后要开发的处理、机器、制造、事件的组成、手段、方法或动作。因而，所附权利要求包括在其范围内的这样的处理、机器、制造、事件的组成、手段、方法或动作。
提供所公开的方面的以上描述以使本领域的任何技术人员能够做出或者使用本公开。对这些方面的各种修改对于本领域技术人员而言是非常显而易见的，并且在此定义的一般原理可以应用于其他方面而不脱离本公开的范围。因此，本公开不意图被限制到在此示出的方面，而是按照与在此公开的原理和新颖的特征一致的最宽范围。
为了例示和描述的目的已经给出了以上描述。此外，此描述不意图将本公开的实施例限制到在此公开的形式。尽管以上已经讨论了多个示例方面和实施例，但是本领域技术人员将认识到其某些变型、修改、改变、添加和子组合。

Claims (17)

  1. 一种媒体特征的比对方法，所述方法包括：
    获取第一媒体对象的第一媒体特征序列和第二媒体对象的第二媒体特征序列，所述第一媒体特征序列包含顺序排列的多个第一媒体特征单体，所述第二媒体特征序列包含顺序排列的多个第二媒体特征单体；
    确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度；
    根据所述单体相似度确定所述第一媒体特征序列与所述第二媒体特征序列之间的相似度矩阵；
    根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况。
  2. 根据权利要求1所述的媒体特征的比对方法，其中，
    所述第一媒体特征单体和所述第二媒体特征单体为浮点数特征；
    所述的确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度包括：
    根据所述第一媒体特征单体与所述第二媒体特征单体之间的余弦距离，确定所述单体相似度。
  3. 根据权利要求1所述的媒体特征的比对方法，其中，
    所述第一媒体特征单体和所述第二媒体特征单体为二值化特征，且具有相同的特征单体长度；
    所述的确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度包括：
    根据所述第一媒体特征单体与所述第二媒体特征单体之间的汉明距离，确定所述单体相似度。
  4. 根据权利要求1所述的媒体特征的比对方法，其中，
    所述的获取第一媒体对象的第一媒体特征序列和第二媒体对象的第二媒体特征序列包括，获取第一媒体对象的多种类型的所述第一媒体特征序列，并获取第二媒体对象的多种类型的所述第二媒体特征序列；
    所述的确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度包括，分别确定同种类型的所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度，以得到多种所述单体相似度；
    所述的根据所述单体相似度确定所述第一媒体特征序列与所述第二媒体特征序列之间的相似度矩阵包括，确定所述多种单体相似度的平均值或最小值，根据所述的多种单体相似度的平均值或最小值确定所述相似度矩阵。
  5. 根据权利要求1所述的媒体特征的比对方法，其中，所述多个第一媒体特征单体在所述第一媒体特征序列中按时间顺序排列，所述多个第二媒体特征单体在所述第二媒体特征序列中按时间顺序排列。
  6. 根据权利要求5所述的媒体特征的比对方法，其中，所述相似度矩阵中的一个点对应一个所述单体相似度；所述相似度矩阵的点在横向上按照各个所述第一媒体特征单体在所述第一媒体特征序列中的先后顺序排列，且在纵向上按照各个所述第二媒体特征单体在所述第二媒体特征序列中的先后顺序排列。
  7. 根据权利要求6所述的媒体特征的比对方法，其中，所述根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况包括：根据所述相似度矩阵中的直线确定所述第一媒体对象与所述第二媒体对象的相似程度和匹配片段。
  8. 根据权利要求7所述的媒体特征的比对方法，其中，所述的根据所述相似度矩阵中的直线确定所述第一媒体对象与所述第二媒体对象的相似情况包括：
    将斜率为预设的斜率设定值的多条直线定义为备选直线，根据每条所述备选直线所包含的单体相似度的平均值或总和值，确定所述备选直线的直线相似度；
    在多条所述备选直线中，选取一条使得所述直线相似度最大的备选直线，并定义为第一匹配直线；
    根据所述第一匹配直线的所述直线相似度确定所述第一媒体对象与所述第二媒体对象的相似程度；根据所述第一匹配直线的起点和终点确定所述第一媒体对象与所述第二媒体对象的匹配片段的起止时间。
  9. 根据权利要求8所述的媒体特征的比对方法，其中，所述斜率设定值为多个，所述备选直线为斜率为所述多个斜率设定值中任意一个的直线。
  10. 根据权利要求6所述的媒体特征的比对方法，其中，所述的根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况包括：
    在所述相似度矩阵中选取使得所述单体相似度最大的多个点作为相似度极值点；
    基于所述多个相似度极值点，在所述相似度矩阵中拟合出一条直线作为第二匹配直线；
    根据所述第二匹配直线所包含的所述单体相似度的平均值或总和值确定所述第一媒体对象与所述第二媒体对象的相似程度；根据所述第二匹配直线的起点和终点确定所述第一媒体对象与所述第二媒体对象的匹配片段的起止时间。
  11. 根据权利要求10所述的媒体特征的比对方法，其中，所述的基于所述多个相似度极值点，在所述相似度矩阵中拟合出一条直线作为第二匹配直线包括：利用随机抽样一致法，在所述相似度矩阵中拟合出一条斜率为预设的斜率设定值或斜率接近预设的斜率设定值的直线作为第二匹配直线。
  12. 根据权利要求8或10所述的媒体特征的比对方法，其中，所述的根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况还包括：
    判断所述第一匹配直线或所述第二匹配直线的开头和结尾的点是否达到预设的单体相似度设定值，去掉所述开头和所述结尾的未达到所述单体相似度设定值的部分，保留中间一段直线并定义为第三匹配直线；
    根据所述第三匹配直线的所述直线相似度确定所述第一媒体对象与所述第二媒体对象的相似程度，根据所述第三匹配直线的起点和终点确定匹配片段的起止时间。
  13. 一种媒体特征的比对装置，所述装置包括：
    媒体特征序列获取模块，用于获取第一媒体对象的第一媒体特征序列和第二媒体对象的第二媒体特征序列，所述第一媒体特征序列包含顺序排列的多个第一媒体特征单体，所述第二媒体特征序列包含顺序排列的多个第二媒体特征单体；
    单体相似度确定模块，用于确定所述第一媒体特征单体与所述第二媒体特征单体之间的单体相似度；
    相似度矩阵确定模块，用于根据所述单体相似度确定所述第一媒体特征序列与所述第二媒体特征序列之间的相似度矩阵；
    相似情况确定模块，用于根据所述相似度矩阵确定所述第一媒体对象与所述第二媒体对象的相似情况。
  14. 根据权利要求13所述的媒体特征的比对装置，还包括执行权利要求2到12中任一权利要求所述步骤的模块。
  15. 一种媒体特征比对硬件装置，包括：
    存储器，用于存储非暂时性计算机可读指令；以及
    处理器，用于运行所述计算机可读指令，使得所述处理器执行时实现根据权利要求1到12中任意一项所述的媒体特征的比对方法。
  16. 一种计算机可读存储介质，用于存储非暂时性计算机可读指令，当所述非暂时性计算机可读指令由计算机执行时，使得所述计算机执行权利要求1到12中任意一项所述的媒体特征的比对方法。
  17. 一种终端设备，包括权利要求13或14所述的一种媒体特征的比对装置。
PCT/CN2018/125502 2018-03-29 2018-12-29 一种媒体特征的比对方法及装置 WO2019184523A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2020520709A JP6987987B2 (ja) 2018-03-29 2018-12-29 メディア特徴の比較方法及び装置
SG11202000106WA SG11202000106WA (en) 2018-03-29 2018-12-29 Method and device for comparing media features
US16/979,784 US11593582B2 (en) 2018-03-29 2018-12-29 Method and device for comparing media features

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810273673.2 2018-03-29
CN201810273673.2A CN110569373B (zh) 2018-03-29 2018-03-29 一种媒体特征的比对方法及装置

Publications (1)

Publication Number Publication Date
WO2019184523A1 true WO2019184523A1 (zh) 2019-10-03

Family

ID=68062187

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/125502 WO2019184523A1 (zh) 2018-03-29 2018-12-29 一种媒体特征的比对方法及装置

Country Status (5)

Country Link
US (1) US11593582B2 (zh)
JP (1) JP6987987B2 (zh)
CN (1) CN110569373B (zh)
SG (1) SG11202000106WA (zh)
WO (1) WO2019184523A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113190829A (zh) * 2021-05-18 2021-07-30 京东数科海益信息科技有限公司 一种权限数据的识别方法、装置、设备和介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110569373B (zh) * 2018-03-29 2022-05-13 北京字节跳动网络技术有限公司 一种媒体特征的比对方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060107216A1 (en) * 2004-11-12 2006-05-18 Fuji Xerox Co., Ltd. Video segmentation combining similarity analysis and classification
CN102117313A (zh) * 2010-12-29 2011-07-06 天脉聚源(北京)传媒科技有限公司 一种视频检索方法和系统
CN105426884A (zh) * 2015-11-10 2016-03-23 佛山科学技术学院 一种基于全幅特征提取的快速文档类型识别方法
CN106919662A (zh) * 2017-02-14 2017-07-04 复旦大学 一种音乐识别方法及系统

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7672369B2 (en) * 2002-02-13 2010-03-02 Reify Corporation Method and apparatus for acquisition, compression, and characterization of spatiotemporal signals
US7778469B2 (en) * 2003-10-03 2010-08-17 Fuji Xerox Co., Ltd. Methods and systems for discriminative keyframe selection
US7424150B2 (en) * 2003-12-08 2008-09-09 Fuji Xerox Co., Ltd. Systems and methods for media summarization
DE102004047069A1 (de) * 2004-09-28 2006-04-06 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Ändern einer Segmentierung eines Audiostücks
DE102004047032A1 (de) * 2004-09-28 2006-04-06 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Bezeichnen von verschiedenen Segmentklassen
JP5082257B2 (ja) * 2006-02-27 2012-11-28 大日本印刷株式会社 音響信号検索装置
US7925112B2 (en) * 2007-02-28 2011-04-12 Honeywell International Inc. Video data matching using clustering on covariance appearance
WO2009046438A1 (en) * 2007-10-05 2009-04-09 Dolby Laboratories Licensing Corp. Media fingerprints that reliably correspond to media content
JP5091806B2 (ja) * 2008-09-01 2012-12-05 株式会社東芝 映像処理装置及びその方法
JP5630541B2 (ja) * 2009-01-20 2014-11-26 ソニー株式会社 情報処理装置、情報処理方法およびプログラム
JP5326910B2 (ja) * 2009-01-20 2013-10-30 ソニー株式会社 情報処理装置、情報処理方法およびプログラム
GB0901263D0 (en) * 2009-01-26 2009-03-11 Mitsubishi Elec R&D Ct Europe Detection of similar video segments
US8861844B2 (en) * 2010-03-29 2014-10-14 Ebay Inc. Pre-computing digests for image similarity searching of image-based listings in a network-based publication system
US8407223B2 (en) * 2010-07-02 2013-03-26 Honeywell International Inc. System for information discovery in video-based data
EP2661710A2 (en) * 2011-01-07 2013-11-13 Alcatel-Lucent Method and apparatus for comparing videos
CN102956238B (zh) * 2011-08-19 2016-02-10 杜比实验室特许公司 用于在音频帧序列中检测重复模式的方法及设备
US8881213B2 (en) * 2013-03-13 2014-11-04 Verizon Patent And Licensing Inc. Alignment of video frames
US9436893B2 (en) * 2013-11-27 2016-09-06 Adobe Systems Incorporated Distributed similarity learning for high-dimensional image features
CN106034240A (zh) * 2015-03-13 2016-10-19 小米科技有限责任公司 视频检测方法及装置
JP6125577B2 (ja) * 2015-07-30 2017-05-10 日本電信電話株式会社 映像同定装置、方法、及びプログラム
US9959448B2 (en) 2016-08-19 2018-05-01 Google Llc Multi-step sequence alignment
CN110322897B (zh) * 2018-03-29 2021-09-03 北京字节跳动网络技术有限公司 一种音频检索识别方法及装置
CN110324660B (zh) * 2018-03-29 2021-01-19 北京字节跳动网络技术有限公司 一种重复视频的判断方法及装置
CN110569373B (zh) * 2018-03-29 2022-05-13 北京字节跳动网络技术有限公司 一种媒体特征的比对方法及装置
CN110555114A (zh) * 2018-03-29 2019-12-10 北京字节跳动网络技术有限公司 一种媒体检索方法及装置
JPWO2020100289A1 (ja) * 2018-11-16 2021-11-04 富士通株式会社 類似度計算装置、類似度計算方法および類似度計算プログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060107216A1 (en) * 2004-11-12 2006-05-18 Fuji Xerox Co., Ltd. Video segmentation combining similarity analysis and classification
CN102117313A (zh) * 2010-12-29 2011-07-06 天脉聚源(北京)传媒科技有限公司 一种视频检索方法和系统
CN105426884A (zh) * 2015-11-10 2016-03-23 佛山科学技术学院 一种基于全幅特征提取的快速文档类型识别方法
CN106919662A (zh) * 2017-02-14 2017-07-04 复旦大学 一种音乐识别方法及系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113190829A (zh) * 2021-05-18 2021-07-30 京东数科海益信息科技有限公司 一种权限数据的识别方法、装置、设备和介质
CN113190829B (zh) * 2021-05-18 2024-04-09 京东科技信息技术有限公司 一种权限数据的识别方法、装置、设备和介质

Also Published As

Publication number Publication date
US11593582B2 (en) 2023-02-28
CN110569373B (zh) 2022-05-13
CN110569373A (zh) 2019-12-13
JP6987987B2 (ja) 2022-01-05
JP2020525963A (ja) 2020-08-27
US20210042569A1 (en) 2021-02-11
SG11202000106WA (en) 2020-02-27

Similar Documents

Publication Publication Date Title
WO2019184522A1 (zh) 一种重复视频的判断方法及装置
US9100630B2 (en) Object detection metadata
US20180032784A1 (en) Fingerprint identification method and apparatus
WO2020042754A1 (zh) 一种全息防伪码校验方法及装置
WO2019184518A1 (zh) 一种音频检索识别方法及装置
WO2021189911A1 (zh) 基于视频流的目标物位置检测方法、装置、设备及介质
WO2016168982A1 (zh) 一种设置指纹识别器中断阈值的方法、装置和终端设备
JP6465215B2 (ja) 画像処理プログラムおよび画像処理装置
WO2017096972A1 (zh) 提升指纹识别性能的方法、装置及移动终端
WO2019184523A1 (zh) 一种媒体特征的比对方法及装置
WO2017156963A1 (zh) 一种指纹解锁的方法及终端
US20160110098A1 (en) Touch Zones on a Soft Keyboard
WO2022095359A1 (zh) 基于防摄屏的信息安全保护方法、装置、电子设备及介质
WO2019184520A1 (zh) 一种视频特征提取方法及装置
US11874869B2 (en) Media retrieval method and apparatus
WO2023010701A1 (en) Image generation method, apparatus, and electronic device
TW202129540A (zh) 字元識別方法、裝置及電腦可讀取存儲介質
WO2019184521A1 (zh) 一种视频特征提取方法及装置
CN112801130B (zh) 图像聚类簇质量评估方法、系统、介质及装置
CN112560813B (zh) 窄条形指纹的识别方法、存储介质及电子设备
US12032584B2 (en) Data management method and system for a security protection terminal, device and storage medium
TWI715252B (zh) 電子裝置及其利用觸控資料的物件資訊辨識方法
CN110717362B (zh) 数位影像的特征树结构的建立方法与影像物件辨识方法
US20230386241A1 (en) Person re-identification method, computer-readable storage medium, and terminal device
TW202123076A (zh) 影像擷取方法與影像擷取裝置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18911538

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020520709

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18911538

Country of ref document: EP

Kind code of ref document: A1