SK1902015U1 - Method for detecting commercial break during TV streaming and system connection - Google Patents

Abstract

In the method of recognizing an advertising spot during a television broadcast in the direction of transmission from the mobile platform (4) to the recognition server (5), the currently selected channel is recognized from the audio sample from the TV scanned by the mobile platform and in the direction of transmission from the content server (3) to the mobile platform (4). ) special content is locked to the mobile platform (4) for TV advertising. Opening a special content to a TV ad on a mobile platform (4) is available at the start of an ad break while continuing to follow the same selected channel. The system engagement to reduce channel audience loss during advertising breaks consists of a sample server (1) with input (2) from distributors (20) of the TV signal broadcasting sources (15), further comprising a content server (3) that is connected to the wireless bidirectional link with a mobile application management platform (4). There is also a recognition server (5) that is linked to both the sample server (1) and the content server (3). The content server (3) includes a first information block (6) with special content for television advertising and also a second information block (7) with aggregated relevant content on social networks.

Description

A method of recognizing an advertising spot during television broadcasting and system engagement

Technical field

The technical solution relates to a method of recognizing an advertisement spot during television broadcasting and a system connection to this method. The technical solution belongs to the field of telecommunication and advertising technology.

BACKGROUND OF THE INVENTION

In the state of the art, there are standard advertising systems for displaying advertising spots or banners in electronic devices such as TVs. The ad is paid by the advertiser and there is no response to the ad's appearance until the end. Failure to see an ad to the end significantly reduces the effect of the ad on the ad visitor. In addition, one of the biggest problems and sources of economic loss for TV broadcasters is the drop in viewership of their channels during advertising breaks. The viewers tend to switch to another channel, leave the screen and stop watching TV or TV. they are not within reach of TV sets at this time. The aforementioned unfavorable facts have led to efforts to develop a concept of such an advertising system for TV which would ensure that the channels of viewers of their channels were maintained during advertising breaks, which would at the same time control the appearance of advertising spots for the advertiser. In particular, this concept would require recognition of the TV program being watched, the sound of which would be captured by the mobile phone. Several services are dedicated to recognizing the sound recorded by a mobile phone. The most important are Shazam, Soundhound and Mufin. The Shazam service described in US 6,990,453 is the most widespread and works to compare the so-called digital fingerprints made from a sound sample. All available services work to compare fingerprints. Shazam analyzes the sound in which it features distinctive features that will not affect even strong noise and other various signal damage, such as lossy MP3 audio compression or poor recording hardware. Such features are, for example, loud tones. Advanced methods of audio data processing such as cepstral analysis are also used to detect these features. The comparison of these features would be very slow and therefore they are not compared directly, but hash is generated from these features. These hashs are generated from combinations of features (the audio frequency they are on) and their distance from each other over time. In this way, many more hashs are generated than the features themselves, thus treating well the situations from damaged signals. The same samples are also used to create a song database to search for samples uploaded by a mobile device. The hash comparison is very fast and therefore can search in a pre-created database with millions of songs. Currently, Shazam also recognizes some selected TV shows.

The Soundhoud described in US Patent Application Publication 2013/0044885 is the second most widely used music recognition service. Like Shazam, they work on digital fingerprints generated from a recorded audio sample. These fingerprints are generated from robust features that do not affect noise and other signal damage. Unlike Shazam, these traits are not generated by hash, but by matrices that capture the relationship between the most distinctive feature and the second and third most distinctive feature in the recorded sample. These matrices are then continuously sent to the server, which compares them to the matrices in the pre-created song sample database. Songs in the database also have more than one matrix that describes them.

Mufin is a service that recognizes live broadcasts on selected TV stations. In addition to recognizing a show and TV station, they can also recognize ads. They use their own digital audio fingerprint “AudiolD” and try to find and encode features in the sound that do not affect various damages. Further, in the published patent application

US2014 / 0188592 and US2014 / 0012572 disclose adding extra data to a recognized audio, for example, relevant advertising.

Further known is Blue Spike, which, for example, based on US Patent No. 7,346,472, creates an abstract from a musical sample, which is then compared to an abstract database with the result that it is a song. This method can be implemented in software generally on digital data, not just on audio.

In the prior art, the Tune Hunter system is known in which samples are recorded from recorded audio from a handheld device (mobile, key chain, pager, etc.), which are then compared to a pre-formed database of music samples. The system will also offer relevant items for sale to the detected sound, such as the song itself. The main difference from the other approaches is in sending and comparing the audio samples themselves and not simplifying them such as hash or matrix as described e.g. in US Patent 6,941,275.

Current song recognition methods work to detect features in audio samples that can survive noise and various signal damage such as lossy MP3 compression, poor quality recording hardware, and so on. These samples are then encoded into another form (hash, matrix, etc.) that captures the relationships between these features and is easy to compare. In addition to Mufin, most systems compare a sample uploaded by a mobile device to a pre-created database.

Due to the above-mentioned shortcomings in audio recognition, efforts have been made, which results in a technical solution that addresses the method of recognizing an advertising spot during television broadcasting and system engagement.

The essence of the technical solution

The above-mentioned deficiencies are eliminated by the method of recognizing an advertisement spot during television broadcasting and the system connection according to this technical solution, which is based on the fact that by activating the control application on the mobile platform mobile platform to the content server. The broadcast TV signal is sampled in the sampling server and passes to the recognition server. The currently selected channel being watched is also brought from the recognition server to the content server. In the direction of transmission from the content server to the mobile platform, locked special content for TV advertising is brought to the mobile platform. Opening special TV ad content on the mobile platform is available at the beginning of the advertising break while continuing to watch the same selected channel. Advantageously, at the beginning of the advertising break, a notification of the current moment of the commercial of the advertising spot is preceded and at the same time the performance data of each spot in the advertising break is captured. Advantageously, in the direction of transmission from the content server to the mobile platform to the mobile platform, aggregated relevant content on social networks is introduced in addition to sending special content to a television advertisement.

The system connection for recognizing an advertisement spot during a television broadcast is based on the above method and consists of a sampling server with input from the distributors of the TV broadcasting sources. It also consists of a content server that is wirelessly connected to a mobile platform with a control application. Mobile platforms of application users are smartphones, notebooks, laptops, taaiets and similar devices activated in the data network such as. mobile Internet or mobile network eg. GSM. There is a recognition server that is connected to both the sampling server and the content server. The content server comprises at least a first information block with special content for television advertising, and preferably also includes a second information block with aggregated relevant content on social networks that fill the content database. The content server also includes a request application and has input from an external TV signal generator. The TV broadcaster produces the broadcast signal of the TV station as a source of broadcasting TV signals in the form of an audio-video stream, which is distributed by the distributor while receiving an external signal from it at the beginning of the advertisement block. The sampling server contains a generator from the source data.

The recognition server includes e.g. indexing application and comparator with index database.

The functionality of the wiring is as follows. The recognition algorithm is based on input data. From the external device - mobile, mobile platform, the sample is used for recognition and from the sampling server the data is used as a reference sample to look for a sample from the external device. From both types of equipment, the sample is processed as follows. The raw input data from the data stream is processed by the STFT (Short-time Fourier transform) algorithm. This processing creates a matrix of normalized data energies. Key points are selected from the matrix thus processed using a precisely defined grid. The coordinates of these points, along with the processed matrix, are input to the Binary Robust Independent Elementary Features (BRIEF) algorithm to create a matrix of normalized energies that creates, for each key point, a set of descriptors that describe the key point and its surroundings. A descriptive set of descriptors from an external device is sent to the content server. A descriptive set of descriptors from the sampling server is sent directly to the recognition server as a reference sample.

The sampling server handles multiple broadcasts from different sources. Each broadcast has a unique identifier (station ID). Each sample of input data from each transmission is independently processed by the algorithm described above and sent with the identifier as a reference descriptive set of descriptors to the recognition server. Increased plausibility of results in an external device is achieved by sending multiple descriptive descriptor sets for recognition and increasing the number of identical recognition responses.

The recognition server receives a descriptive set of descriptors from the content and sampling server. From each received descriptor set received from the sampling server, the extractor selects the exact time interval and creates the extracted descriptor set. According to the identifier, it associates the extracted descriptive set of descriptors with the cluster of previous extracted descriptor sets. At regular intervals, it extracts the extracted descriptor sets from the clusters and joins them into larger descriptor sets representing the overlapping longer time interval for the data stream by identifier. These then join into a time descriptive set of descriptors containing longer time intervals of the descriptive set of descriptors for multiple identifiers. The time sample of the descriptor descriptor set is included in the Fast Library for Answers Nearest Neighbors (FLANN) algorithm for fast sample retrieval, which results in the FLANN index stored in the database. The key part is to create different long time samples of descriptive descriptor sets, from which different long FLANN indices are generated. This ensures that the number and length of searches required for sample recognition is minimized. The recognition server maintains a minimum set of differently long FLANN indexes in the database to provide coverage. The incoming double-script descriptive set from the content server is sequentially compared in the comparator by the Locality SensiE (Hanhiég) algorithm to quickly search for the descriptive descriptor set in the FLANN index with only the necessary FLANN indexes from the database to minimize search time. The LAS assigns the key points of the descriptive set of descriptive sets from the content server to the closest key points of the descriptive set of descriptor set currently being compared. Subsequently, the filtering algorithm evaluates the comparison, the score is evaluated and, according to the result, either another search is triggered or the response is sent back to the content server.

A content server is a server containing broadcast information, information about functional recognition servers, and receiving external signals. Its primary role is to provide relevant and up-to-date, complementary information for the content being viewed. The content server maintains a list of currently available recognition servers. After receiving a descriptive set of descriptors from an external source, it selects an available recognition server and sends a descriptive set of descriptors for recognition. If it receives information that the recognition server is busy, it selects another recognition server. It then creates the appropriate content from the recognition server and sends it back to the mobile, mobile platform. This ensures automatic scaling for a large number of recognition requests. The content server is automatically scaled by third-party systems. Another task of the content server is after receiving an external signal (trigger from the TV - signal generator) to change, modify and notify external devices about the change of content.

For the purposes of this invention, the following definitions are explained:

The input data is a raw set of data from the source, such as data from an audio stream.

A sampling server is a device capable of processing broadcast / data streams in real time.

A mobile platform is an external device, a mobile, it is a device capable of processing recorded audio.

The matrix of normalized energies is the output matrix of the Short-Time Fourier Transform, with the audio input data the x-axis is the overlapping time slots and the y-axis is the frequency intervals.

BRIEF is an algorithm of generating a matrix of normalized energies from input data.

The key point is the coordinates of the selected point from the matrix of normalized energies.

A set of descriptors is a description of a key point and its surroundings in a matrix of normalized energies.

A descriptor set is a set of descriptor sets for each key point in the normalized energy matrix.

A content server is a device that handles automatic scaling, receives a descriptive set of descriptors from external devices, and sends them for recognition to the recognition server, receiving signals from external sources.

The FLANN index is a structure for fast sample retrieval.

LSH is an algorithm for quickly searching for a descriptive set of descriptors in the FLANN index.

A descriptor descriptor time sample is a descriptor descriptor set from multiple transmissions clustered by an exact length identifier. A recognition server is a device that receives a descriptive set of descriptors from the sampling server, aggregates them into a time sample of the descriptive set of descriptors, and creates a database of FLANN indexes. Subsequent to receiving a descriptive set of descriptors from the content server, it compares it only to the necessary set of FLANN indexes and sends the response back.

The advantages of the method of recognizing advertising spots during television broadcasting and system engagement according to this technical solution are that the system offers viewers the continuation of the viewing experience as a bridging of the advertising break in the form of special content. It will reduce the frustration of interrupting entertainment while reducing the incentive to switch channels. This content is locked during normal broadcast sessions. It unlocks at the beginning of the advertising break and is only available if the viewer does not switch to another station, as the app recognizes the currently selected channel. The mechanism for unlocking special content at the time of each advertising break works by processing the start and end signals. The system also serves as an aggregator of relevant content on social networks. It recognizes individual moments in the broadcast, including advertising. This allows you to bring enhanced TV ad information to your app. The system monitors the broadcast continuously and thus recognizes the broadcasted advertisement and notifies the advertiser of the current moment of broadcasting the advertising spot together with the relevant performance metrics of the spot. The connection to the interface of the visual graphic generator of broadcasters enables direct output of processed data from the system during broadcasting.

BRIEF DESCRIPTION OF THE DRAWINGS

The method of recognizing an advertisement spot during television broadcasting and system connection according to the technical solution will be further explained in the drawings, where FIG. 1 shows a schematic diagram of a system connection. In FIG. 2 shows a more detailed system wiring diagram. In FIG. 3 shows a signal sampling scheme.

Examples of technical solution

It is to be understood that individual embodiments of the invention are presented for illustration and not as limitations of the invention. Those skilled in the art will find or be able to ascertain using no more than routine experimentation many equivalents to specific embodiments of the invention. Such equivalents will also fall within the scope of the following claims. For those skilled in the art, it cannot pose the problem of optimal system design, therefore these features have not been solved in detail.

Example 1

In this example of a particular embodiment of the subject matter of the invention, a method of recognizing an advertisement spot during a television broadcast is described. It is based on the fact that by activating the control application on the mobile platform 4, in the direction of transmission from the mobile platform 4 to the recognition server 5, the audio sample of the currently watched program from the TV scanned by the mobile platform 4 is transmitted to the content server 3. The transmitted TV signal is sampled in the sampling server 1 and proceeds to the recognition server 5. The currently viewed channel selected is also brought from the recognition server 5 to the content server 3. In the direction of transmission from the content server 3, locked special content for television advertising and aggregated relevant content on social networks are delivered to the mobile platform 4 in addition to sending special content to television advertising. Opening TV-specific content on mobile platform 4 is available at the start of the advertising break while continuing to watch the same selected channel. At the beginning of the advertising break, a warning about the current moment of the commercial's broadcast is preceded by a record of the performance of each spot in the advertising break.

Example 2

In this example of a particular embodiment of the subject matter of the invention, a system connection for recognizing an advertisement spot during a television broadcast is shown in principle in FIG. 1 and shown in more detail in FIG. 2. It consists of a sampling server 7 with an input 2 from distributors 20 of the sources 15 of broadcasting TV signals. It consists of a content server 3 which is wirelessly connected to the mobile platform 4 by a smartphone with a control application. Included in the engagement is a recognition server 5, which is connected to both the sampling server 7 and the content server 3. The content server 3 comprises a first information block 6 with special content for television advertising and also includes a second information block 7 with aggregated relevant content on social networks that fill the content database 1 3. The content server 3 also includes a request application 9 and has an input 8 from an external signal generator 14 of the TV broadcasters 37. The content server 3 further comprises a database of TV station metadata 17, a TV time schedule block 19 and a time block 18. The sampling server 1 comprises a block 10 descriptors from the source data. The recognition server 5 comprises e.g. the indexing block H and the comparator 12 with the index database 16. The mobile platform 4 comprises a generator 21 of a descriptor set 22 of the compared data, a request generator 23, a content display module 24 and finally a time module 25.

Example 3

In this example of a particular embodiment of the subject matter of the invention, sampling and recognition of input data from data streams as described in FIG. 1, 2 and 3. The raw input data from the data stream 26 that is distributed by the distributor 20 is processed by the STFT M Short-time Fourier transform algorithm. This processing creates a matrix 27 of normalized data energies. Key points are selected from the matrix 27 thus processed using a predetermined grid. The coordinates of these points together with the processed matrix 27 are input to the BRIEF algorithm. The descriptor set generator 28 for each key point creates a set of descriptors 29 that describe the key point and its surroundings. A descriptive set of descriptors 23 from the external mobile platform 4 device is sent to the content server 3. A descriptive set of descriptors 29 from the sampling server 1 is sent directly to the recognition server 5 as a reference sample as shown in FIG. 1 and 2. The recognition server 5 receives a descriptive set of descriptors 23 from the content server 3 and a descriptive set of descriptors 29 from the sampling server 7. From each received descriptor set 29 from the sampling server 1, the extractor 30 selects the exact time interval and creates the extracted descriptor set 32. According to the identifier, assigns the extracted descriptive descriptor set 32 to a cluster of previously extracted descriptor descriptor sets 35. At regular intervals, extracts the extracted descriptor sets The descriptors from the bursts and link them to larger descriptor sets of descriptors 35 representing the overlapping longer time interval for the data stream according to the identifier 31. These are then combined into a time descriptive set of descriptors 35 containing longer time intervals of the descriptive set descriptor 32 for multiple identifiers. The time descriptive set of descriptors 35 is an input to the index generator 33, which uses the FLANN algorithm to create the FLANN index 34 stored in the database. A key part is the generation of different long time samples of descriptor sets 35, from which different long FLANN indices of ML are formed. The recognition server 5 maintains in the database a minimum set of differently long FLANN indexes M to provide coverage. The descriptor set 22 stored in the content server 3 is sequentially compared in the comparator 12 by the LSH algorithm only with the necessary FLANN indexes M from the index database 16 so as to minimize the retrieval time. LSH assigns the key points of the descriptor set 22, from the content server 3, the closest key points of the currently descripted descriptor set 35. The filtering algorithm is then scored, scored, and either searched again or the response is sent back to the content server. third

Industrial usability

The method of recognizing advertising spots during television broadcasting and system equipment according to the technical solution find utility in communication technology and advertising.

PJV / 1 J O - J

Claims (9)

PROTECTION REQUIREMENTS A method of recognizing an advertisement spot during television broadcasting, characterized in that in the direction of transmission from the mobile platform (4) to the recognition server (5), the currently selected channel is recognized from the audio sample from the TV recorded by the mobile platform (4). content server (3) onto the mobile platform (4), the locked special content for television advertising is introduced into the mobile platform (4). A method of recognizing an advertisement spot during a television broadcast according to claim 1, characterized in that the opening of the special content to the television advertisement on the mobile platform (4) is available at the beginning of the advertising break while continuing to watch the same selected channel. A method of recognizing an advertisement spot during a television broadcast according to claim 1 and 2, characterized in that the start of the advertisement break is preceded by a notification of the actual moment of broadcasting the advertisement spot and at the same time the performance data of each spot in the advertisement break. Method for recognizing an advertisement during television broadcasting according to claims 1 and 2, characterized in that aggregated relevant content on social networks is introduced into the mobile platform (4) in the direction of transmission from the content server (3) to the mobile platform (4). A system connection for recognizing an advertisement spot during television broadcasting, characterized in that it comprises a sampling server (1) with input (2) from distributors (20) of sources (15) of TV broadcasters (37), further comprising a content server (1). 3) which is wirelessly connected to a mobile platform (4) with a control application, and there is a recognition server (5) that is connected to the sampling server (1) and also to the content server (3); the content server (3) comprises a first information block (6) with special content for television advertising. A system connection for recognizing an advertisement during television broadcasting according to claim 5, characterized in that the content server (3) comprises a second information block (7) with aggregated relevant content on social networks. A system advertisement spot recognition system during television broadcasting according to claim 5, characterized in that the content server (3) comprises a request application (9) and has an input (8) from an external signal generator (14) of the TV broadcasters (37). System connection for recognizing an advertisement spot during a television broadcast according to claim 5, characterized in that the sampling server (1) comprises a block (10) for generating descriptors from the source data. System connection for recognizing an advertisement spot during a television broadcast according to claim 5, characterized in that the recognition server (5) comprises an indexing block (11) and a comparator (12).