RU2606567C2 - Method of comparison of the first input audio track with the indexed audio track - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: invention describes a method of comparison of the first input audio track with the indexed audio track. Method is carried out on a server. Method involves: selection of indexed audio track as an audio-candidate from a plurality of indexed audio tracks; check of the audio-candidate for a match with the first audio track.

EFFECT: elimination of productive capacity costs that occur in the comparison of audio signatures larger parts.

23 cl, 8 dwg

Description

FIELD OF THE INVENTION

The present solution relates to a method and system for comparing audio tracks using chroma prints with a quick candidate selection procedure.

BACKGROUND

Recognizing audio files has two tasks: recognizing parts of audio recordings and comparing complete audio tracks.

Recognition technology is used to identify audio recordings using a relatively short, but potentially distorted and noisy fragment of an audio track. A typical example is Shazam, a smartphone-based commercial music recognition service. Shazam uses the smartphone’s built-in microphone to get a short snippet of the music being played. He creates an audio signature based on this fragment and compares it with the database. After recognition is made, there is no need for any further processing, for example, additional comparison with the files selected in the database. Therefore, when a user listens to a music mix from different tunes and tries to identify a melody, Shazam will determine the name of the melody that is currently being played.

The comparison task is to find duplicates. A duplicate of a music mix from the songs will be the same mix from the same songs. However, audio tracks can be considered as duplicates if the difference between them is a slight difference in duration, a slight shift in time or a difference in quality.

Since the purpose of the comparison is to find duplicates, you need to compare the larger parts of the audio signatures. Comparing larger parts of audio signatures is resource-intensive (expending productive power, etc.)

SUMMARY OF THE INVENTION

The objective of this solution is to eliminate at least some of the disadvantages inherent in the prior art.

SUBSTITUTE SHEET DESCRIPTION

The first object of the implementation of the decision is a method of comparing the first incoming audio track with an indexed audio track. The method is executed on the server. The method includes: (1) selecting an indexed audio track from a plurality of indexed audio tracks as a candidate audio track; the selection is carried out by performing the following steps: (a) determining the first short audio signature (audio fingerprint), the first short audio signature being the audio signature of the first part of the first incoming audio track and containing the first chroma word, the first part of the first audio track having a predetermined duration from the beginning of the first incoming audio track; (b) determining the candidate audio track from the set of indexed audio tracks, wherein the candidate audio track has a second short audio signature that contains a second chroma word, wherein the initial part of the second chroma word is identical to the initial part of the first chroma word, and the second short audio signature is the audio signature of the first part of the audio track a candidate, the first part of the candidate audio track having a first predetermined duration from the beginning of the candidate audio track; (2) checking that the candidate audio track matches the first audio track by performing the following steps: (a) determining the first long audio signature, which is the audio signature of the second part of the first incoming audio track; (b) extracting a second long audio signature, the second long audio signature being an audio signature of a second part of a candidate audio track; and the second part of the first audio track and the second part of the candidate audio track have a second predetermined duration from the beginning of the first audio track and the candidate audio track, respectively; and the first parts of the corresponding first audio track and the candidate audio track are fully included in the second parts of the corresponding first audio track and the candidate audio track; as well as making bitwise comparisons of the first long audio signature with the second long audio signature.

In some embodiments, the initial parts of the chroma word contain a combination of: (i) any of the first byte and the first multibyte sequence, the first multibyte sequence being the sequence of bytes at the beginning of the initial part of the corresponding chroma word, and the first multibyte sequence has a predetermined number of bytes; and (ii) any of the subsequent bytes and the second multibyte sequence, the second multibyte sequence being a sequence of bytes following either the first multibyte sequence or the first byte of each corresponding chroma word, and the second multibyte sequence has a predetermined number of bytes.

In some embodiments, the first predetermined duration is the shorter of the intervals: a predetermined duration in the range of 9 to 27 seconds or the duration of the corresponding audio track.

In some embodiments, the first predetermined duration is the shorter of the intervals: 21 seconds or the duration of the corresponding audio track.

In some embodiments, the second predetermined duration is the shorter of the intervals: a predetermined duration in the range of 96 to 141 seconds, or the duration of the corresponding audio track.

In some embodiments, the second predetermined duration is the shorter of the intervals: 120 seconds or the duration of the corresponding audio track.

In some embodiments, the implementation of both the first key and second key characterizes part of the corresponding audio track, and the duration of the part of the audio track is between 1/2 and 8 seconds.

In some embodiments, the method also comprises forming said first chroma word and said second chroma word.

In some embodiments, the implementation of both the first master and second master contains an indication of the identifier (ID) of the track associated with the corresponding audio track.

In some embodiments, each key word contains an indication of track duration information associated with a corresponding audio track.

In some embodiments, a track identifier is described by a third multibyte sequence following either (i) a subsequent byte or (ii) a second multibyte sequence.

In some embodiments, the duration of the track is characterized by one byte, which follows either (i) the next byte, or (ii) the second multibyte sequence.

In some embodiments, the definition of the candidate audio track comprises comparing the corresponding duration of the first incoming audio track and the candidate audio track.

In some embodiments, the method also comprises determining that the candidate audio track is not the same as the first incoming candidate track because the track duration differs by a value that exceeds a preset value.

In some embodiments, the candidate audio track comprises a plurality of candidate audio tracks, and the method comprises selecting a subgroup of a plurality of candidate audio tracks based on a predetermined threshold number of candidates.

In some embodiments, a bitwise comparison of the first long audio signature with the second long audio signature comprises offsetting the first long audio signature relative to the second long audio signature.

In some embodiments, the bias comprises a bias amplitude that is between plus or minus 20 seconds.

In some embodiments, the determination that the initial part of the second chroma word is identical to the initial part of the first chroma word is performed by determining that the entire sequence of bytes in the initial part of the second chroma word matches the entire sequence of bytes in the initial part of the first chroma word.

In some embodiments, at least one of the short audio signature and the long audio signature includes an indication of a track identifier (ID) associated with the corresponding audio track.

In some embodiments, prior to determining the first short audio signature, the method also comprises receiving at least a portion of the first incoming audio track by the server.

In some embodiments, obtaining a second short audio signature comprises extracting using an index.

In some embodiments, the implementation of the index is an inverted index of the audio track.

In some embodiments, the inverted audio track index is either: (i) a pruning index (from pruning), generated for a plurality of short audio signatures, or (ii) a verification (validation) index generated for a plurality of long audio signatures.

Another object of implementation is a server including a permanent medium of computer information (computer-readable medium). A permanent storage medium contains an inverted audio track index. The inverted index of the audio track contains: a plurality of sets of word-lists (posting lists), and each list of word-positions in the set of sets of word-lists is a list of word-positions of the chroma keys, and each chroma word is a component of the audio signature; each chroma word is divided into many bytes and contains the initial part of the chroma word, which is a subset of bytes having the first byte and the next byte, with the next byte located immediately after the first byte; within this set of word lists from many word lists: each word list contains chroma words that have the same first byte in the initial part of the chroma word and the first of the many word lists differs from the second of the many word lists in that the related chroma words have a different a subsequent byte that is unique to a given set of word position lists.

In some embodiments, each of these sets of wordlist sets is associated with an index key that is unique within a plurality of sets of wordlist.

In some embodiments, an index key is needed to quickly determine the position of a given set of word positions within an index.

In some embodiments, each word list from the given word list sets is associated with a sort key that is unique to the given word list set.

In some embodiments, a sort key is used to quickly locate a word list within a given set of word lists.

In some embodiments, the first byte is the first multibyte sequence, which is the sequence of bytes at the beginning of the initial part of each corresponding chroma word, and the number of bytes in each first multibyte sequence is the same.

In some embodiments, the index key comprises a first multibyte sequence.

In some embodiments, the subsequent byte is a second multibyte sequence, which is a sequence of bytes following either the first byte or the first multibyte sequence of each corresponding chroma word.

In some embodiments, the number of bytes in each of the second multibyte sequences is the same.

In some embodiments, the first multibyte sequence is a sequence of three bytes.

In some embodiments, the initial portion of each chroma word contains a predetermined number of bytes.

In some embodiments, the start of each chroma word contains 4 bytes.

In some embodiments, each of these wordlist sets is associated with an index key that is unique across a plurality of wordlist sets, and each word list in a given wordlist set is associated with a sort key that is unique within a given wordlist set .

In some embodiments, the audio signature is either (i) a short audio signature, which is the audio signature of the first part of the audio track, the first part of the audio track having a first predetermined length from the beginning of the audio track, or (ii) a long audio signature that is the audio signature of the second part of the audio track, the second part of the audio track has a second predefined duration from the beginning of the audio track.

In some embodiments, at least one of the short and long audio signatures comprises a track identifier (ID).

In some embodiments, either a short or a long audio signature is stored in an inverted audio track index, implemented either as: (i) a tuning index generated for a plurality of short audio signatures, or as (ii) a verification index generated for a plurality of long audio signatures.

Another object is a method for creating and maintaining an inverted index of an audio track, the method comprising: (i) obtaining an audio track for indexing; (ii) determining a first chroma word divided by many bytes, the first chroma word containing the initial part of the chroma word, which is a subset of the bytes with the first byte and the next byte, with the next byte immediately after the first byte; (iii) determining, on the basis of the first byte, a particular set of word lists within a plurality of sets of word lists, this particular set containing word lists that contain references to chroma words with the same first byte; (iv) saving the indication of the first chroma word in this list of word positions in response to the fact that the second byte of the first chroma word matches any second byte of any chroma words stored in this list of word positions within a specific set of word lists; (v) creating a new list of word positions within a specific set of word list lists storing the first chroma word in response to the fact that the second byte of the first chroma word does not match any second byte of any chroma words stored in the word lists.

In some embodiments, the first byte is the first multibyte sequence, which is the sequence of bytes at the beginning of the initial portion of a new chroma word, the number of bytes being predetermined.

In some embodiments, the subsequent byte is a second multibyte sequence, which is a sequence of bytes following either the first multibyte sequence or the first byte of a new chroma word, the number of bytes in each second multibyte sequence being predetermined.

In some embodiments, the method before determining the first chroma word associated with the new audio track for indexing comprises determining the audio signature of the new audio track for indexing, the chroma word being part of the audio signature.

In some embodiments, the audio signature is either (i) a short audio signature of the first part of the new audio track, the first part of the new audio track having a first predetermined duration from the start of the new audio track, or (ii) a long audio signature of the second part of the new audio track, the second part of the new audio track having a second predefined duration from the start of a new audio track.

In some embodiments, the first predetermined duration is a shorter interval of: a predetermined duration ranging from 9 to 27 seconds or the duration of a new audio track.

In some embodiments, the first predetermined duration is the shorter of the intervals: 21 seconds or the duration of a new audio track.

In some embodiments, the second predetermined duration is the shorter of the intervals: a predetermined duration in the range of 96 to 141 seconds or the duration of a new audio track.

In some embodiments, the second predetermined duration is the shorter of the intervals: 120 seconds or the duration of the corresponding audio track.

In some embodiments, the implementation of the chroma key characterizes part of the audio track, and the duration of the part of the audio track is between 1/2 and 8 seconds.

In some embodiments, the duration of the portion of the audio track is 3 seconds.

In some embodiments, each key word contains an indication of a track identifier (ID) associated with a corresponding audio track.

In some embodiments, each key word contains an indication of track duration information associated with a corresponding audio track.

In some embodiments, the track identifier (ID) is described by one byte, which follows either (i) the next byte or (ii) the second multibyte sequence.

In some embodiments, the duration of the track is described by one byte, which follows either (i) the next byte or (ii) the second multibyte sequence.

In some embodiments, the short audio signature is stored in the inverted index of the audio track, which index is executed as a tuning index generated for short audio signatures.

In the context of the present description, an “audio signature” is compressed digital information (signal) determinately created from an audio signal that can be used to identify an audio sample or to quickly determine the position of elements in an audio database.

In the context of the present description, “chroma” is a component of an audio signature, with each chroma word being a sequence of bytes describing a portion of the audio. As a non-limiting example, a chroma word can be a component of an audio signature created using chroma-print technology.

As used herein, an “audio track” is an audio file; a video file that contains an audio track; any other audio recording suitable for machine analysis; as well as an unrecorded audio signal, such as an audio signal coming from a transducer, such as a microphone.

In the context of the present description, “server” means a computer program running on appropriate equipment that is able to receive requests (for example, from client devices) over the network and execute these requests or initiate the execution of these requests. The equipment may be a single physical computer or a single physical computer system, but neither is required. In the context of the description, the use of the expression “server” does not mean that every task (for example, received commands or requests) or any specific task will be received, executed or initiated to be executed by the same server (that is, by the same software and / or hardware); this means that any number of software elements or hardware devices can be involved in receiving / transmitting, executing or initiating the execution of any request or the consequences of any request associated with the client device, and all this software and hardware can be one server or several servers , both options are included in the expression “at least one server”.

In the context of the present description, "client device" means a hardware device capable of working with software suitable for solving the corresponding problem. Thus, examples of client devices (among other things) include personal computers (desktop computers, laptops, netbooks, etc.) smartphones, tablets, as well as network equipment such as routers, switches, and gateways. It should be borne in mind that a device behaving as a client device in the present context may behave like a server in relation to other client devices. The use of the expression “client device” does not exclude the possibility of using multiple client devices to receive / send, execute, or initiate the execution of any task or request, or the consequences of any task or request, or the steps of the above method.

In the context of the present description, “database” means any structured data set that does not depend on a specific structure, database management software, hardware of the computer on which data is stored, used or otherwise made available for use. The database may reside on the same hardware that runs the process that stores or uses the information stored in the database, or it may reside on separate hardware, such as a dedicated server or multiple servers.

In the context of the present description, “information” includes any information of any type that can be stored in a database. Thus, information includes, among other things, audiovisual works (images, videos, sound recordings, presentations, etc.), data (location data, digital data, etc.), text (opinions, comments, questions , messages, etc.), documents, tables, etc.

In the context of the present description, a “component” means software (corresponding to a specific hardware context) that is necessary and sufficient to perform the specific specified function (s).

In the context of the present description, “computer-based storage medium of computer information” means a storage medium of absolutely any type and nature, including RAM, ROM, disks (CDs, DVDs, diskettes, hard disks, etc.), USB flash drives , solid state drives, tape drives, etc.

In the context of the present description, the words “first”, “second”, “third”, etc. used in the form of adjectives solely to distinguish the nouns to which they relate from each other, and not for the purpose of describing any specific relationship between these nouns. So, for example, it should be borne in mind that the use of the terms “first server” and “third server” does not imply any ordering, chronology, hierarchy or ranking (for example) of servers / between servers, as well as their use (in itself) does not imply that a certain "second server" must exist in a given situation. Hereinafter, as indicated here in other contexts, reference to the “first” element and the “second” element does not exclude the possibility that it is one and the same actual real element. So, for example, in some cases, the “first” server and the “second” server can be the same software and / or hardware, and in other cases they can be different software and / or hardware.

Each embodiment pursues at least one of the aforementioned objectives. It should be borne in mind that some objects of the solution obtained as a result of attempts to achieve the aforementioned goals may satisfy other goals not specifically indicated here.

Additional and / or alternative characteristics, aspects, and advantages of the embodiments will become apparent from the following description, the attached drawings, and the attached claims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the solution, as well as its other aspects and characteristics, a reference is made to the following description, which should be used in combination with the accompanying drawings, where:

In FIG. 1 is a schematic diagram illustrating an audio signature 1001, which is an embodiment of an audio signature in accordance with an embodiment.

In FIG. 2 is a schematic diagram illustrating an audio signature 1002, which is an embodiment of an audio signature in accordance with an embodiment.

In FIG. 3 is a schematic diagram showing a plurality of sets of word lists 200, wherein a plurality of sets of word lists 200 are formed in accordance with a combination of 202 bytes in accordance with embodiments.

In FIG. 4 is a schematic diagram illustrating a fragment 300 of a training index 724 for a particular set 2004 from word lists 304, wherein a fragment 300 of a training index 724 is implemented in accordance with non-limiting embodiments.

In FIG. 5 is a schematic diagram of a word list 304 implemented in accordance with embodiments.

In FIG. 6 is a flowchart of a method 500 for selecting an indexed audio track as a candidate audio track from a plurality of indexed audio tracks, the method 500 being implemented in accordance with an embodiment.

In FIG. 7 is a flowchart of a method 600 for reconciling a candidate audio track with a first audio track, the method 600 being implemented in accordance with embodiments of the present solution.

In FIG. 8 is a schematic diagram of a system 700 implemented in accordance with an embodiment of the solution.

DETAILED DESCRIPTION OF THE INVENTION

One object of this solution is a method for comparing an incoming audio track with audio tracks that are already indexed and / or stored in a database accessible by the server. In a broad sense, the method comprises the steps of (i) quickly selecting a candidate audio track and (ii) checking candidates.

Another object of this solution is a permanent storage medium of computer information (computer-readable storage medium), which stores the inverted index of the audio track. In some embodiments, the inverted audio track index may be implemented as two different indices: (i) a training index 7241 generated for a plurality of short audio signatures, or (ii) a verification index 7242 generated for a plurality of long audio signatures.

Another object of the present solution is a method for supporting an inverted index of an audio track, which can be performed as two separate indices: (i) a tuning index 7241 generated for a plurality of short audio signatures, or (ii) a verification index 7242 generated for a plurality of long audio signatures.

In FIG. 8 is a schematic diagram of a system 700 implemented in accordance with embodiments of the present solution.

A schematic diagram of a system 700 may be implemented as an embodiment of a solution without limiting the scope of the solution. It is important to keep in mind that the following description of system 700 is a description of exemplary embodiments. Thus, the entire following description is presented only as a description of a representative example. This description is not intended to determine the scope or scope of a decision. Some useful examples of modifications to the system 700 may also be covered by the following description. The purpose of this is also solely assistance in understanding, and not determining the scope and boundaries of the solution. These modifications are not an exhaustive list, and it will be understood by those skilled in the art that other modifications are possible. In addition, this should not be interpreted so that where it has not yet been done, i.e. where examples of modifications have not been set forth, no modifications are possible, and / or something that is described is the only embodiment of this element. As will be clear to a person skilled in the art, this is most likely not the case. In addition, it should be borne in mind that the system 700 is in some specific manifestations a fairly simple embodiment, and in such cases it is presented here in order to facilitate understanding. As will be understood by a person skilled in the art, many embodiments will have much greater complexity.

System 700 includes an electronic device 702. An electronic device 702 is typically associated with a user 740, and thus may sometimes be referred to as a “client device”. It should be noted that the fact that the electronic device 702 is associated with the user does not imply any particular mode of operation, nor does it require a login, registration, or the like.

The options for electronic device 702 are not particularly limited, but personal computers (desktop computers, laptops, netbooks, etc.) or wireless data transmission devices (smartphones, tablets, etc.) can be used as an example of electronic device 702. The electronic device 702 includes hardware and / or application software and / or system software (or a combination thereof), as is known in the art, to execute an application 704, which may be a web browser or any other application providing the ability to download audio tracks to server 720 or to media available to server 720.

In accordance with embodiments, the application 704 may support data transfer to the server 720 or the media available to the server 720. Indeed, the application 740 may also support downloading data from the server 720 or any other source.

Application 704 may be any known web browser or any other application that allows you to download content. By way of non-limiting example, application 704 may be implemented as a Yandex ™ browser application. It is important to keep in mind that any other commercially available or proprietary application can be used to implement embodiments.

An electronic device 702 is connected to a data network 710 via a data line (not individually numbered). In some embodiments, implementation, the data network 710 may be the Internet. In other embodiments, the data network 710 may be implemented differently — as a global data network, a local data network, a private data network, and the like.

The implementation of the data line is not limited and will depend on which electronic device 702 is used. By way of example, but not limitation, in these embodiments, in cases where the electronic device 702 is a wireless communication device (eg, a smartphone), the data line is a wireless data network (for example, inter alia, a 3G data line, 4G data line, Wireless Fidelity or shortly WiFi®, Bluetooth®, etc.). In cases where the electronic device 702 is a laptop computer, the data line can be either wireless (Wireless Internet Wireless Fidelity or shortly WiFi®, Bluetooth®, etc.) or wired (Ethernet-based connection).

It is important to keep in mind that embodiments of an electronic device 702, a data line and a data network 710 are provided for illustrative purposes only. Thus, those skilled in the art will be able to understand the details of other specific embodiments of the electronic device 702, the data line, and the data network 710. That is, the examples presented here do not limit the scope of the solution.

Server 720 may be a conventional computer server. In an example embodiment, the server 720 may be a Dell ™ PowerEdge ™ server that uses the Microsoft ™ Windows Server ™ operating system. Needless to say, server 720 may be any other suitable hardware and / or application software and / or system software, or a combination thereof. In general, the task of the server 720 is to provide the user 740 with the ability to download audio tracks; Compare received audio tracks and stored audio tracks.

Server 720 comprises a computer-used storage medium 722, also referred to as a storage medium 722 or computer-readable storage medium. The media 722 may contain any type of media, including but not limited to RAM, ROM, disks (CDs, DVDs, floppy disks, hard drives, etc.), solid state drives, tape drives, etc. In the General case, the task of the medium 722 is the storage of machine-readable commands (codes), as well as other data, for example, a testing index 7241 and a verification index 7242, etc. In alternative embodiments, the punening index 7241 and verification index 7242 may be performed as a single inverted audio track index 7241 (not shown). In alternative embodiments, the runinning index 7241 and check index 7242 may be performed on three or more indices.

The paging index 7241 comprises a plurality of sets of 200 word lists, wherein a plurality of sets 200 of word lists are generated in accordance with a combination of 202 bytes, in accordance with embodiments of the solution.

Puning index 7241 contains many sets of 200 lists of 304 word positions created for short audio signatures, as will be described in more detail below.

Verification index 7241 contains a key file and a signature file. The signature file contains a sequence of long signatures. In this implementation, the signatures are located in the signature file one after another, without delimiters. In this embodiment, each long signature in the signature file begins with the first two bytes describing the length of the corresponding signature. The key file contains track identifiers (not shown in Fig. 8) associated with the corresponding audio track and having links to the beginning of long signatures in the signature file, as will be described in more detail below.

To provide comparison, short and long audio signatures of all saved tracks are created. The length of all short audio signatures and all long audio signatures from the beginning of each corresponding audio file may be predetermined and may be constant for all corresponding short and long signatures.

For example, all short audio signatures can be created for no more than 20 first seconds of each stored audio track, and all long audio signatures can be created for no more than 120 first seconds of each saved audio track. However, if the length of a particular audio track is less than a predetermined value for the corresponding short or long audio signature, the corresponding short or long audio signature is created for the entire audio track.

Therefore, in another embodiment, the length of the corresponding short or long signature may be shorter than the corresponding predetermined value. In addition, if the length of the audio track is less than a predetermined value for the short audio signature, then the length of the short audio signature and the long audio signature may be identical.

Each audio signature contains a chroma word that characterizes small equal parts of a linked audio track. Khromaslov can characterize the overlapping parts of the audio track. Each key word can be represented as a sequence of bytes. The structure and characteristics of audio signatures and chromas will be described in more detail below, mainly when describing FIG. one.

When server 720 receives an incoming audio track, a short audio signature is created for no more than the first 20 seconds of the incoming audio track.

The comparison procedure contains two stages: the stage of quick selection of the candidate and the stage of verification of the candidate.

As mentioned previously, server 720 can support two indices: a pruning index 7241 containing short audio signatures, and a verification index 7241 containing long audio signatures. Byte sequences in the chromas can be used as keys to find matching audio tracks. How indexes are generated and maintained will be described below, mainly with reference to FIG. 3-5.

During the first stage of the comparison procedure, quick candidate selection, the server 720 finds the candidate audio track using the first short audio signature (short audio signature of the incoming audio track) in the tuning index 7241. The candidate audio track may also be an audio track that contains at least one key word, completely coinciding with the chroma key identified in the incoming audio track.

During the second step of the comparison procedure, the verification step, the server 720 creates a first long audio signature. The first long audio signature may describe a portion of the audio track up to 120 first seconds of the incoming audio track.

After that, the server 720 checks the candidate audio track by bitwise comparing the long audio signature of the incoming audio track with the long audio signature stored in the database and available to server 720 using the verification index 7241. At this point, server 720 can take into account the different quality of the compared audio tracks, as well as the time offset between the compared audio tracks.

At this point, the server 720 excludes from the list of candidates those indexed audio tracks whose length differs from the length of the incoming audio track by a predetermined time, for example, 10 seconds, 30 seconds, a minute, and so on. The coincidence of audio tracks is established when their audio signature matches at a predefined level. For example, in some embodiments, the match of the bits in the long audio signatures of the incoming audio track and the candidate audio track need not be one hundred percent. In such cases, the bytes of the long audio signature of the tracks match 90%, 80%, 70%, and so on.

It should be noted that specific embodiments of the audio signatures are not limited in any way, and in this case, various prior art approaches to creating audio signatures can be used. An example of such an approach, which can be used to create an audio signature, is proposed in US patent US 7013301 "Audio signature system and method."

Another approach to creating an audio signature is described in the article "How does chroma-print work?" authored by Lukáš Lalinský published on the web page https://oxvgene.sk/2011/01/how-does-chromaprint-work/ . Chromeprint is a client library that implements a custom algorithm for extracting audio signatures from an audio source. Source hyperlinks are published at https://acoustid.org/chromaprint . The source code is available for authorization under the LGPL2.1 + license (standard public limited license).

The Chromaprint technology allows you to imagine an audio track as a spectrogram, which shows how the intensity changes over time at specific frequencies. To create a spectrogram, the audio track is divided into many overlapping intervals, which are processed using the Fourier transform function.

In the case of a chroma-print, the incoming audio track can be converted with a frequency of 11025 Hz and an interval size of 4096 (0.371 s) with a 2/3 overlap. Chromaprint then processes the information, converting the frequencies into notes, and as a result, 12 cells are obtained, each of which contains a corresponding note. The result is an audio representation that is resilient to change.

Accordingly, for each of them a predefined set of 16 filters is applied, and these filters capture differences in intensity in notes over time. The result is a "chromo-word". A “chroma” is a component of an audio signature, with each chroma and a sequence of bytes describing a portion of the audio.

As will be understood by those skilled in the art, other methods for creating audio signatures may be used. In addition, the described methods with modifications may be used.

However, it should be understood that in some embodiments, after a particular method of creating audio signatures has been selected, it is used to create both short and long audio signatures. In other embodiments, various approaches can be applied to create short and long audio signatures.

In FIG. 1 illustrates an audio signature 1001. For example, an audio signature is an audio signature of an incoming audio track (the incoming audio track is an audio track received by server 720 from user 740 of electronic device 702 (as shown in FIG. 8).

At this point in time, it is not yet known whether this audio track is indexed and / or stored in a database (not shown) that is available for server 720. Server 720 creates an audio signature of the incoming audio track. The server 720 then compares the thus created audio signature with a plurality of second audio signatures, the second audio signatures being already indexed and stored in the databases and / or indexes (for example, in the verification index 7241) available to the server 720. If the audio signature matches one of the many second audio signatures, then a potentially matching track is determined.

More specifically, the second audio signature 1001 is the audio signature of an audio track that has already been indexed and / or stored in a database, which database is available to server 720. For the same audio track, there may be more than one second audio signature 1001, since a second audio signature can be created for an audio file with a first predetermined duration (second short audio signature) and with a second predefined duration (second long audio signature). For example, the second short audio signature 1001 and the second long audio signature 1001 can be created for the same audio track, but they will have different durations from the beginning of the audio track (for example, the first 21 seconds from the beginning of the audio track and the first 120 seconds from the beginning of the audio track). The second audio signature is stored in a database (not shown), which is available for server 720. The second audio signature 100 is a representation of one of the many second audio signatures stored in the database.

In some embodiments, each of the second audio signatures comprises an indication of a track identifier 402 associated with a corresponding audio track to which each of the second audio signatures belongs, and it is possible to determine that there is a matching stored audio track for the incoming audio track. This information can be used in many ways. For example, information about a matching stored audio track can be used to provide a better version of this music track to a user. Alternatively or additionally, this information can be used to determine the copyright holder and / or obtain authorization to use the audio track and / or to make a license payment (royalty) to the copyright holder. Alternatively or additionally, this information can be used to find and remove duplicates.

With reference to FIG. 1, it should be noted that the audio signature 1001 is associated with the first incoming audio track and that the length of the first incoming audio track in this example is 5 minutes 41 seconds.

As shown in FIG. 1, the audio signature 1001 contains chroma words 1021 created during the process of creating audio signatures from a first incoming audio track. The audio signature may contain (in addition to the chromas) other components, for example, an indication of the duration of the first incoming audio track and the like.

The size of each key word 1021 within the audio signature 1001 is the same: each key word 1021 consists of four bytes 104. Since one byte is eight bits and has 28 or 256 possible values, from 0 to 255, the bytes in the example shown in FIG. 1 are presented as values in a range from 5000 to 5255.

Bytes 104 of each chroma word 1021 of any audio signature can be grouped in at least two groups. The first two groups form the initial part 106 of the chroma word, as shown in FIG. 1. The number of bytes in each of the first two groups can be predefined. As an example, the number of bytes in the first two groups may be at least one.

As will be appreciated by those skilled in the art, in various embodiments, the number of bytes in each of the first two groups can be predefined in different ways. For example, in the initial part of the chroma word (the size of the initial part of the chroma word is four bytes), the following combinations are possible:

1 byte and 3 bytes;

2 bytes and 2 bytes;

3 bytes and 1 byte.

Therefore, in a broad sense, in any four-byte chroma word, the following combinations of groups are possible:

the first byte (not shown) and the second multibyte sequence (not shown);

a first multibyte sequence (which is a two-byte sequence) and a second multibyte sequence, which is also a two-byte sequence; and

a first multibyte sequence 108, which is a three byte sequence, and a subsequent byte 110.

Such a specific example is depicted in FIG. 1: bytes 104 in each chroma word 1021 of the audio signature 1001 are grouped into two groups: (i) a first multibyte sequence 108, which is a sequence of bytes 104, which, in turn, is located in the initial part of the corresponding chroma word, and (ii) the subsequent byte 110. In some embodiments, there may be several subsequent bytes: 110, 112, 114. In other words, the bytes in each chroma word 102 are divided into two groups, the size of the first group being three bytes and the size of the second group being one byte (as shown in FIG. one )

In the general case, the initial portion of the chroma word is the portion of the chroma word that contains a combination of the first multibyte sequence (or the first byte, as the case may be) and the second multibyte sequence (or the subsequent byte, as the case may be).

In alternative embodiments, bytes in the chroma words may be grouped into more than two groups, for example, as in FIG. 2.

However, the initial portion 106 of the chroma word may not necessarily coincide with the chroma word itself.

Server 720 may create an audio signature 1002 for the entire incoming track, as well as for part of the incoming audio track. In embodiments, server 720 may create more than one audio signature for an incoming audio track. For example, in some embodiments, the server 720 creates a first short audio signature and a first long audio signature.

The short audio signature is the audio signature of the first part of the audio track and lasts a first predetermined time from the start of the audio track. In some embodiments, the first predetermined duration is the shorter of the two intervals: a predetermined duration in the range of 9 to 27 seconds or the duration of the audio track.

In the example example of the embodiment, the first predetermined duration from the beginning of the audio track is the shorter of the two intervals: 21 seconds or the duration of the audio track. Given that the duration of the incoming audio track is 5 minutes 41 seconds, which is more than 21 seconds, the server 720 creates the first short audio signature for the first 21 seconds of the incoming audio track.

The long audio signature is the audio signature of the second part of the audio track and lasts a second predetermined time from the start of the audio track. In some embodiments, the second predetermined duration is the shorter of the two intervals: a predetermined duration in the range of 99 to 141 seconds or the duration of the audio track. In the example example of the embodiment, the second predetermined duration from the beginning of the audio track is the smaller of two intervals: 120 seconds or the duration of the audio track. Given that the duration of the incoming audio track is 5 minutes 41 seconds, which is more than 120 seconds, server 720 creates the first long audio signature for the first 120 seconds of the incoming audio track.

The audio signature 1002 shown in FIG. 2, there may be any of the following audio signatures: a first short audio signature, a first long audio signature, a second short audio signature, and a second long audio signature. The structure of these audio signatures of the incoming audio track may be the same, except that the first long audio signature contains more chroma. Both the first long and first short audio signatures of the same audio track are created using the same technology, taking into account the correspondingly long and short parts of the same audio track from its beginning, the first short audio signature being identical to the beginning of the first long audio signature.

As will be appreciated by those skilled in the art, in various embodiments, the number of bytes in each of the first two groups can be predefined in different ways. For example, in the initial part 106 of the chroma word (and the size of the initial part of the chroma word is four bytes), the following combinations are possible:

1 byte and 3 bytes;

2 bytes and 2 bytes;

3 bytes and 1 byte.

Therefore, in a broad sense, in any four-byte chroma word, the following combinations of groups are possible:

the first byte (not shown) and the second multibyte sequence (not shown);

a first multibyte sequence (which is a two-byte sequence) and a second multibyte sequence, which is also a two-byte sequence; and

a first multibyte sequence 108, which is a three byte sequence, and a subsequent byte 110.

In FIG. 3 is a schematic diagram of a plurality of sets 200 of positioning lists. Each set of vocabulary lists is depicted as a large quadrangle and separately numbered: 2001, 2002, 2003, ... 2004, 2005 ... etc.

A word list is an index data structure storing transformed content, such as an audio track, at its location in a database file, or in a document or in a set of documents. The task of the list of word positions is to provide the ability to quickly search for an audio track by enhancing the processing process when the audio track is added to the database.

The word list 304 is depicted in FIG. 4. The list of 304 word positions is one of the lists of word positions grouped into a set of 200 word lists. The word list 304 for each key word 1021 may contain a track identifier 402 associated with an audio track characterizing the corresponding key word 1021. Track identifier 402 may be a multibyte sequence. In this embodiment, track identifier 402 is a sequence of four bytes. A list of 304 word positions for each chroma word 1021 may contain an indication 404 of the track duration associated with the audio track from which the corresponding chroma word 1021 is created. In addition, a list of 304 word positions for each chroma word 1021 may contain an indication 406 of the position of the chroma word 1021 in the audio track that describes the corresponding word 1021.

There is only one specific list of 304 word positions for all chromas 1021, which have the same initial part 106. For example, if there are several chromas 1021 and even though the initial part of each of the three chromas102 looks like 5214-5255-5008-5047 , there will be only one list of word positions for all these three chromas 1021 with the initial part 106, which looks like 5214-5255-5008-5047.

In this set of 200 lists of 304 word positions of a set of lists of 304 word positions, each list of 304 word positions contains a chroma 1021 with the same first multibyte sequence 108 chroma 1021. It should be noted that the first multibyte sequence 108 chroma 1021 is smaller than the initial part 106 of chroma 1021. In addition to the first multibyte sequence 108, the initial part 106 of the chroma 1021 also contains the subsequent byte 110. In other words, each set 200 of wordform lists 304 contains only those wordform lists 304, which matured hromaslova link to 1021 with the same multi-byte sequence 108, e.g. 5214-5255-5008 (or the same first byte, depending on the circumstances in some embodiments). This means that each set 200 of word-lists 304 contains word-lists 304 that can be created for different chromas with the same first multibyte sequence 108, but different subsequent bytes 110; for example, one set of 200 lists of 304 word positions can be created for the following chromas: 5214-5255-5008-5008, 5214-5255-5008-5012, 5214-5255-5008-5047, 5214-5255-5008-5077, 5214-5255 -5008-5201.

FIG. 3 illustrates sets of 200 word lists 304 that are created for chroma 1021. Since each chroma 1021 has a first multibyte sequence 108, which is a sequence of three bytes, each word list is grouped into a set of 200 word lists 304, and the 200 begins with a sequence of three bytes.

Since in this example each key word has the first multibyte sequence of three bytes, each set of 200 lists of 304 word positions is also defined by any possible combination of three bytes, starting with the first set of 2001, which is determined by the combination of bytes 2021 (5000-5000-5000), and ending with the last set 2005, which is determined by a combination of bytes 2025 (5255-5255-5255).

All these combinations of bytes from 2021 to 2025 can be used as the corresponding index keys from the plurality of index keys 202, as will be described later. Since, as noted earlier, one byte is eight bits and has 28 or 256 possible values, from 0 to 255, the bytes in the example depicted in FIG. 2 are presented as values ranging from 5000 to 5255. Therefore, 16777216 combinations of three bytes (2563) are possible. All of these byte combinations will be used as index keys 202, as will be described below. Since the sets 200 of word lists are determined by the number of bytes equal to the number of bytes in the first multibyte sequence 108, there will be 16777216 sets of 200 lists of 304 word positions.

The number of sets 200 of lists of 304 word positions may be less or more than 16777216, depending on the length of the first multibyte sequence. For example, the first multibyte sequence 108 consists of a single byte (of the "first byte"), then there will be 256 sets of 200 lists of 304 word positions.

A situation is possible when there are no word lists for chromas, starting with a specific first byte or with a specific first multibyte sequence. Suppose that there are no audio signatures in the entire database of audio tracks that contain chroma words starting with the first multibyte sequence 5010-5127-5206. In this case, the set 200 associated with the multibyte sequence 5010-5127-5206 will be empty and will not contain any list of word positions. It is possible to add a list of word positions to the existing set 200, the set 200 being determined by a multibyte sequence 5010-5127-5206, if an audio signature of a new audio track is received and at least one new chroma 1021 begins with a multibyte sequence 5010-5127-5206.

Within this set of word lists, one of lists 304 differs from another word list 304 in that associated chromas 1021 have different subsequent bytes (or different second multibyte sequences, as the case may be in some embodiments). The various subsequent bytes (or second multibyte sequences (depending on the circumstances in some embodiments)) are unique to a given set of word lists.

Take, as an example, the kit 2004 shown in FIG. 3. Set 2004 contains all lists of 304 word positions for indexed chroma 1022 that have the same multibyte sequence 5214-5255-5008. All key words 1022 have the same first multibyte sequence 5214-5255-5008, differing from each other by the subsequent byte 110 (or the second multibyte sequence, as the case may be in some embodiments).

Since a specific list of word positions has references for all elements of a particular chromosword, such a list of word positions is unique in this set of word lists. Therefore, in the same set 200 of word lists 304, the subsequent byte 110 (or the second multibyte sequence, as the case may be in some embodiments) distinguishes one word list 304 from another.

For example, a set of 2004 word lists beginning with a sequence of bytes 5214-5255-5008 will contain many word lists starting with a combination of bytes 5214-5255-5008, where each word list will have a different next byte 302, corresponding to the next byte 110. Mentioned a combination of 2024 bytes 5214-5255-5008 can be used as an index key 2024 (see Fig. 4). The next byte 302 is identical to the subsequent byte 110 of the corresponding key word 1022 and is used as a sort key 302 for the inverted index of the audio track. In some embodiments, the next byte 302 is a multibyte sequence that is identical to the second multibyte sequence, and is also used as a sort key 302.

One of the objects of this solution is a permanent medium 722 of computer information, a schematic representation of which is shown in FIG. 8. The permanent computer information medium 722 comprises an inverted audio track index.

The inverted audio track index can be stored in two different indices, the tuning index 7241 and the verification index 7242. The same technology can be used to create and maintain the tuning index 7241 and verification index 7242. Both the tuning index 7241 and the verification index 7242 will also be collectively referred to as an inverted audio track index (not shown).

When a reference is made to the inverted audio track index, it applies to both the tuning index 7241 and the verification index 7242. As will be clear to a person skilled in the art, the presentation of the pruning index 7241 and verification index 7241 is simplified; both indices may contain other components that are not shown in FIG. 8, for example, index keys 202 and sort keys 302.

FIG. 4 represents a portion 300 of the routing index 7241 from a particular set 2004 of word lists. The entire inverted audio track index 7241 (not shown) contains a plurality of such parts 300 representing all possible sets of 200 word lists.

The inverted index of the audio track contains many 200 sets of lists of 304 word positions, and each such list contains chroma words 1022, each of which is a component of the audio signature 1012.

In some embodiments, each of the sets 200 of word lists 304 is associated with an index key 202. As shown in FIG. 3, all sets of 200 lists of 304 word positions are unique: set 2001, set 2002, set 2003, 2004, 2005 and so on, and each set of sets 200 is defined by a unique combination of 2021, 2022, 2023, 2024, 2025, etc. bytes. Since one particular index key 202 from the plurality of index keys is associated with one particular set of the plurality of sets 200, each index key 202 is unique, so that a particular index key 202 corresponds to a particular set of word lists 200. For example, a particular set of 2004 vocabulary lists, being unique, corresponds to a combination of 2024 bytes 5214-5255-5008, which is used as an index key 2024, which is also unique.

In some embodiments, an index key 202 is needed to quickly determine the position of a given set 200 of word lists 304 within an inverted audio track index 724. Using the index key 202 allows you to define a specialized set of 200 word lists, which contains a specific list of word positions for a specific chroma word. Therefore, narrowing the search allows you to quickly determine the position of this set of 200 lists of 304 word positions within the index.

In some embodiments, each word list 304 from a given word list set 200 is associated with a sort key 302, which is unique to a given word list set.

In some embodiments, the sort key 302 is needed to quickly determine the position of the list of slots within a given set of word lists, since using the sort key 302 allows you to distinguish one list 304 of word positions from all other lists of 304 word positions within this set of 200 word lists.

In some embodiments, the index key 202 comprises a first multibyte sequence 108. Therefore, the index key 202 will be identical to the first multibyte sequence 108, which also defines sets 200 of word lists 304.

Another object is a method for creating and maintaining the 7241 training index.

The method comprises receiving, by the server 720, an audio track for indexing. This receipt can be carried out, for example, from the copyright holder. The audio track can then be loaded into a database (not shown), stored on server 720, or accessible to server 720.

The method comprises determining a first chroma word 1021 associated with a new audio track for indexing, wherein the chroma 1021 is divided into many bytes 104, and the first chroma 1021 contains the initial part 106 of the chroma, which is a subset of bytes with the first byte and subsequent byte 110, with the subsequent byte 110 located immediately after the first byte. Therefore, in this embodiment, the initial portion 106 of the chroma 1021 contains two bytes.

In addition, the method comprises determining, on the basis of the first byte, a specific set of 200 lists of 304 word positions within a plurality of sets 200 of lists of 304 word positions, this particular set containing lists of word positions that contain links to the chroma words with the same first byte in the initial part 106. The number of sets 200 may be 256, because sets of wordlist lists can be created according to any possible first byte. Since, as mentioned above, one byte is equal to eight bits, 2 ⁸ or 256 values are possible.

In addition, the method comprises: storing an indication of the first chroma word in this list of 304 word locations in response to the fact that the subsequent byte 110 of the first chroma word 1021 matches any subsequent byte 110 of any chroma 1021 stored in this list of 304 word positions within a specific set of 200 lists 304 word positions. Alternatively, the method comprises: creating a new list of 304 word positions within a specific set of 200 lists of 304 word positions, storing an indication of a new key word 1021 in response to the fact that the subsequent byte 110 of the first key word 1021 does not match any subsequent byte 110 of any key words 1021 stored in this a list of 304 word positions.

In some embodiments, the method also prior to determining the first chroma 1021 associated with the new audio track for indexing, comprises determining the audio signature 1001 of the new audio track for indexing.

In other words, server 720 may create an audio signature 1001 for an incoming audio track; then it defines at least one chroma word 1021 for indexing, and this word is contained in the audio signature 1001. Chrome 1021 contains the initial part 106 of the chroma word 1021, and this initial part 106 contains the first byte and the subsequent byte 110. Using the first byte as the index key 202, server 720 determines the corresponding set of 200 word lists 304 as defined by the same byte, the first byte of the corresponding chroma word. Then, using the subsequent byte 110 as the sort key 302, the server 720 searches for such a list of word positions 302 so that all the words in it have the same combination of the first byte and the subsequent byte 110 in its initial part 106. A specific list of 304 word positions should be determined using the same two bytes contained in the initial part 106 of the chroma word 1021 for indexing.

It has already been pointed out earlier that a set of 200 lists of 304 word positions may be empty, provided that there are no indexed chroma 1021 with a specific first byte (or first multibyte sequence, as is the case in some embodiments). In this case, a search using the sort key 302 will show the absence of a relevant list of word positions. In this case, a new list of word positions will be created for this particular chromo word.

In some embodiments, the first byte is the first multibyte sequence 108, which is the sequence of bytes at the beginning of the initial portion 106 of each new chroma 1021, and the number of bytes 104 in each first multibyte sequence 108 is the same.

In some embodiments, the subsequent byte 110 is a second multibyte sequence, which is a sequence of bytes following either the first multibyte sequence 108 or the first byte of each new chroma 1021, the number of bytes 104 in each second multibyte sequence being the same.

In some embodiments, the audio signature 1001 is either a short or long audio signature.

In some embodiments, the implementation of the short audio signature is stored in the tuning index 7241, which is formed for many short audio signatures. The Puning Index 7241 is used to quickly select candidate audio tracks.

In some embodiments, the implementation of the long audio signature is stored in the index 7242 verification, which is formed for many long audio signatures. Verification index 7242 is used to check audio tracks to match the incoming audio track.

Verification index 7241 contains a key file and a signature file. The signature file contains a sequence of long signatures. In this implementation, the signatures are located in the signature file one after another, without delimiters. In this embodiment, each long signature in the signature file begins with the first two bytes describing the length of the corresponding signature. The key file contains the identifiers (ID) of the track (not shown in Fig. 8) associated with the corresponding audio track and having links to the beginning of long signatures in the signature file.

Another object of the solution is a method for comparing a first incoming audio track with an indexed audio track. The method runs on server 720.

In general, the method comprises two main steps: (1) selecting an indexed audio track from a plurality of indexed audio tracks as a candidate audio track; and (2) checking the candidate audio track for consistency with the first audio track:

In FIG. 6 is a flowchart of a method 500 for selecting an indexed audio track as a candidate audio track from a plurality of indexed audio tracks, the method 500 being implemented in accordance with an embodiment.

The method 500 begins at step 502.

At step 502, the server 720 determines a first short audio signature, which is the audio signature of the first part of the first incoming audio track and contains the first chroma 1021, the first part of the first audio track having a predetermined duration from the beginning of the first incoming audio track.

At step 504, the server 720 determines the key words 1021 of the first short audio signature. The definition of chromas 1021 is part of the process of creating audio signatures, as described previously.

At 506, server 720 searches for indexed chroma 1021 with the same initial portion 106 as word 1021 of the first short audio signature.

As a non-limiting example, a search can be performed using the tuning index 7241. The tuning index 7241 is used to quickly select candidate audio tracks. In some embodiments, the implementation of the short audio signature is stored in the tuning index 7241, which is formed for many short audio signatures.

The search can be performed as follows: first, server 720 retrieves the first multibyte sequence 108 of the corresponding chroma word of the first short audio signature and finds the key index 202 corresponding to the first multibyte sequence 108. If the chroma word 1021 of the first short audio signature looks like 5214-5255-5008-5047, the corresponding index 202 of the key is key index 2024 of key 5214-5255-5008 depicted in FIG. four.

As shown in FIG. 3, a set 2004 of word lists corresponds to an index key 2024. Server 720 retrieves this set of 2004 vocabulary lists. The server 720 then retrieves the subsequent byte 110 of the corresponding chroma 1021 of the first short audio signature and finds the sort key 302 corresponding to the subsequent byte 110 of the chroma 1021 of the first short audio signature.

For example, if the key word 1021 of the first short audio signature looks like 5214-5255-5008-5047, the corresponding sort key 302 will be the sort key 302 (5047) shown in FIG. 4. Using this sort key 302, server 720 retrieves a list of 304 word positions for key word 5214-5255-5008-5047.

Then, at step 508, with the list of word positions 304 already extracted, the server 720 can find and select audio tracks in which the loudspeaker 5214-5255-5008-5047 appears. As a non-limiting example, a candidate audio track can be found using the track identifier 402 contained in the word list of the corresponding indexed chroma 1021. Since the same chroma 1021 can be found on several different indexed audio tracks, several corresponding indexed audio tracks with several corresponding track identifiers 402 may be selected as candidate audio tracks. Thus, for one chroma 1021 of the first short audio signature, several candidate audio tracks can be selected.

At step 510, the server 720 repeats steps 504 through 508 for each key word 1021 of the first short audio signature. When an additional candidate audio track is determined in response to a search carried out taking into account each subsequent key word 1021 of the first short audio signature, these additional audio tracks must be added to the plurality of candidate audio tracks. Thus, a plurality of candidate audio tracks may contain all audio tracks, each audio track having at least one key 1021 that corresponds to any one of a plurality of key 1021 of the first short audio signature. It is possible that some candidate audio tracks contain a higher or lower number of chromos, corresponding to chroma 1021 of the first short audio signature.

Then, the method 500 ends.

In FIG. 7 is a flowchart of a method 600 for reconciling a candidate audio track with a first audio track, the method 600 being implemented in accordance with an embodiment.

The method 600 begins at block 602, in which the server 720 determines a first long audio signature, which is an audio signature of a second part of a first incoming audio track. The server 720 determines the second long audio signature using the identifier 402 of the audio track, which was determined at step 508.

As stated previously, if the same 1021 key was found in several different indexed audio tracks, several second long audio signatures could be detected.

The method then proceeds to step 604.

At 604, the server 720 determines a second long audio signature, which is the audio signature of the second part of the candidate audio track.

As shown in the description of method 500, server 720 determines the first short audio signature, consisting of chromas, and, using the tuning index 7241, finds the identifier of the audio track that contains the chromo word represented in the first short audio signature. The method 500 then provides the ability to determine the candidate audio track and its track ID 402. As those skilled in the art will understand, more than one candidate audio track and, accordingly, more than one track identifier 402 can be found.

The long audio signature of the candidate audio track is stored in the verification index 7242 with other long audio signatures of other tracks.

Verification index 7242 may be generated for a plurality of long audio signatures. The check index is used to check the audio tracks for consistency with the incoming audio track. Long audio signatures can be stored in the verification index 7242 as a single record, one long audio signature after another long audio signature. At the beginning of each long audio signature, a multi-byte sequence may be located indicating the length of the next long audio signature. In this embodiment, a multibyte sequence indicating the length of the subsequent long audio signature is a sequence of two bytes.

Server 720 retrieves a second long audio signature using audio track identifier 402.

The extraction can be done from a signature file. The extraction can be performed as follows.

Verification index 7242 contains a key file and a signature file.

The signature file may contain a sequence of second long signatures. In this embodiment, the second long signatures are located in the signature file one after another, without delimiters. In this implementation of the solution, every second long signature in the signature file begins with the first two bytes describing the length of the corresponding signature.

The key file may contain track identifiers (not shown in Fig. 7) associated with the corresponding audio track in the signature file and have links to the beginning of long audio signatures in the signature file, and these beginnings are, in this embodiment, the first two bytes describing the duration of the corresponding signatures.

As described above, when the steps of method 500 are performed, the server 720 determines the candidate audio track using the training index 7241 and finds the identifier of that track. To extract the second long audio signature from the signature file, server 720 can use the track identifier 402, which was determined in step 508. Server 720 can find the same track identifier 402 in the key file and then find the beginning of the second long signature in the signature file.

The server 720 then reads two bytes at the beginning of the corresponding second long audio signature and estimates the duration of the second long audio signature using these two bytes of the second long audio signature in the signature file, and extracts the second long audio signature.

The method then proceeds to step 606.

At 606, server 720 performs a bitwise comparison of the first long audio signature and the second long audio signature. A candidate audio track is considered verified when at least one of the plurality of second long audio signatures matches the first long audio signature at the stage of bitwise comparison.

In some embodiments, for the candidate audio track to be considered the same in the first audio track, the absolute match of the audio signature established by bitwise comparison is not necessary. Two audio signatures can be considered matching even though a certain number of bits do not match. For example, it may be preliminarily determined that the audio signatures match if in a plurality of bits one bit of a thirty-two bit sequence in each audio signature does not match.

Additionally or alternatively, the incoming audio track and the candidate audio track may have different durations. This can happen when an incoming audio track has no beginning or end, or there are a few seconds of silence at the beginning or at the end. Even if not all of the audio signature bits of both audio tracks are identical, these audio tracks may be considered matching.

In some embodiments, the determination of the fact that the candidate audio track is not the same as the first incoming audio track is carried out in response to the fact that the duration of the track differs by a value exceeding the preset value. For example, the preset value is 30 seconds, in another case, the corresponding candidate will be considered mismatched if its duration differs by more than 30 seconds compared to the incoming audio track.

Therefore, in some embodiments, the definition of the candidate audio track comprises comparing the corresponding duration of the first incoming audio track and the candidate audio track. As a non-limiting example, the duration of the candidate audio track can be retrieved from the word list 304 using the track identifier 402 associated with the audio track and an indication 404 of track duration information associated with the audio track. An indication of track duration information 404 may be a single byte or a multibyte sequence. In this embodiment, an indication 402 of track duration information is a sequence of two bytes.

Then, the method 600 ends.

In some embodiments, the candidate audio track comprises a plurality of candidate audio tracks, and the method further comprises selecting a subgroup of a plurality of candidate audio tracks based on a predetermined threshold number of candidates. The predetermined threshold for the number of candidates may be a predetermined minimum number of matching chroma 1021 in the first short audio signature and a second short audio signature in which a predetermined minimum number of matching chroma depends on the number of predetermined candidates and the number of matching chroma in the predetermined candidates. For example, if in a large number of predefined candidates a large number of pre-selected candidates has two or more matching chromas, a predetermined threshold number of candidates can be set as 2, or 3, or 4, and so on. Thus, if a predetermined threshold number of candidates is set to 3 chroma words, a subset of the plurality of candidate audio tracks will include second long audio signatures with at least three chroma 1021 that match the chroma 1021 from the first long audio signature.

In some embodiments, a bitwise comparison of the first long audio signature with the second long audio signature comprises offsetting the first long audio signature with respect to the second long audio signature. The offset of the first long audio signature relative to the second long audio signature is convenient when the incoming audio track is shortened at the beginning or its beginning contains a few seconds of silence, noise, interference, etc. In some embodiments, the bias comprises the bias amplitude. For example, the amplitude may be in the range of plus or minus 20 seconds or 15 seconds.

In some embodiments, the determination that the initial part of the second chroma word is identical to the initial part of the first chroma word is performed by determining that the entire sequence of bytes in the initial part of the second chroma word matches the entire sequence of bytes in the initial part of the first chroma word.

In some embodiments, at least one of the short audio signature and the long audio signature includes an indication of a track identifier 402 associated with the corresponding audio track.

In some embodiments, prior to determining the first short audio signature, the method comprises receiving at least 720 a portion of the first incoming audio track by the server 720. Obtaining can be carried out, for example, when a user of a computer device uploads a new audio track to a social network or to another place that is available to server 720.

In some embodiments, obtaining a second short audio signature comprises extracting using an index. By way of non-limiting example, an index may be an inverted index of an audio track. More specifically, the inverted index 7241 may be a verification index 7242.

Modifications and improvements to the above described embodiments will be apparent to those skilled in the art. The preceding description is provided as an example only and is not subject to any restrictions. Thus, the scope of the present solution is limited only by the scope of the attached claims.

Claims (35)

1. A method of comparing a first incoming audio track with an indexed audio track through a server, comprising: selecting an indexed audio track from a set of indexed audio tracks as a candidate audio track by: determining the first short audio signature, which is the audio signature of the first part of the first incoming audio track and containing the first chroma word, which is a component of the audio signature in the form of a sequence of bytes describing the audio part, the first part of the first audio track having a predetermined duration from the beginning of the first incoming audio track; determining a candidate audio track from a set of indexed audio tracks, wherein the candidate audio track has a second short audio signature containing a second chroma word, wherein the initial part of the second chroma word is identical to the initial part of the first chroma word, and the second short audio signature is the audio signature of the first part of the candidate audio track, the first part of the audio track - the candidate has a first predetermined duration from the beginning of the candidate audio track; checking that the candidate audio track matches the first audio track by: determining the first long audio signature, which is the audio signature of the second part of the first incoming audio track; extracting the second long audio signature, which is the audio signature of the second part of the candidate audio track; moreover, the second part of the first audio track and the second part of the candidate audio track have a second predetermined duration from the beginning of the first audio track and the candidate audio track, respectively; each first part of the corresponding first audio track and the candidate audio track is fully included in the second part of the corresponding first audio track and the candidate audio track; bitwise comparing the first long audio signature with the second long audio signature. 2. The method according to p. 1, in which, as the initial part of the chromium word, a combination is used: the first byte or the first multibyte sequence, which is a sequence of bytes at the beginning of the initial part of the corresponding chroma word and having a predetermined number of bytes; and a subsequent byte or second multibyte sequence, which is a sequence of bytes following the first multibyte sequence or the first byte of each corresponding chroma word, wherein the second multibyte sequence has a predetermined number of bytes. 3. The method according to p. 1, in which the first of a predetermined duration using the smaller of the intervals: a predetermined duration within the time interval from 9 to 27 seconds and duration of the corresponding audio track. 4. The method according to p. 3, in which the first of a predetermined duration using the shorter of the intervals: 21 seconds and the duration of the corresponding audio track. 5. The method according to p. 1, in which the second of a predetermined duration using the smaller of the intervals: a predetermined duration within the time interval from 96 to 141 seconds and duration of the corresponding audio track. 6. The method according to p. 5, in which the second of a predetermined duration using the shorter of the intervals: 120 seconds and the duration of the corresponding audio track. 7. The method according to claim 1, wherein said first chroma and second chroma characterize a portion of the corresponding audio track, and the duration of the portion of the audio track is between 1/2 and 8 seconds. 8. The method according to p. 7, in which form the specified first chroma and the second chroma. 9. The method according to claim 1, wherein said first chroma word and said second chroma word contain an indication of a track identifier associated with a corresponding audio track. 10. The method of claim 9, wherein the track identifier comprises a third multibyte sequence located after the subsequent byte or second multibyte sequence. 11. The method according to p. 1, in which the specified first chroma and the second chroma contain an indication of information about the duration of the track associated with the corresponding audio track. 12. The method according to p. 11, in which an indication of the duration of the track contains one byte, which follows immediately after the next byte or second multibyte sequence. 13. The method according to any one of paragraphs. 10-12, wherein said determination of a candidate audio track comprises comparing the corresponding duration of the first incoming audio track and the candidate audio track. 14. The method according to p. 13, in which the discrepancy between the candidate audio track and the first incoming track by the candidate is determined based on the difference in the track duration by a value exceeding the preset value. 15. The method of claim 13, wherein the candidate audio track comprises a plurality of candidate audio tracks and subgroups of a plurality of candidate audio tracks are selected based on a predetermined threshold number of candidates. 16. The method of claim 1, wherein the bitwise comparison of the first long audio signature with the second long audio signature comprises offsetting the first long audio signature with respect to the second long audio signature. 17. The method of claim 16, wherein said offset comprises an offset amplitude in the range of 20 seconds. 18. The method of claim 1, wherein it is determined that the initial part of the second chroma word is identical to the initial part of the first chroma word by determining that the entire sequence of bytes in the initial part of the second chroma word matches the entire sequence of bytes in the initial part of the first chroma word. 19. The method of claim 1, wherein at least one of the short audio signature and the long audio signature includes an indication of a track identifier associated with the corresponding audio track. 20. The method of claim 1, wherein prior to said determining the first short audio signature, at least a portion of the first incoming audio track is received. 21. The method of claim 1, wherein a second short audio signature is obtained by extraction using an index. 22. The method of claim 21, wherein said index is an inverted index of an audio track. 23. The method of claim 22, wherein the inverted index of the audio track is a tuning index generated for a plurality of short audio signatures, or a verification index generated for a plurality of long audio signatures.

Images