TWI550540B - Advertising trading system - Google Patents

Description

Advertising transaction system

The present invention relates to an advertisement transaction system, and more particularly to an advertisement transaction system suitable for various electronic media having audio transmission capabilities.

Speech processing technology includes Speech Recognition and Speaker Recognition. Speech recognition is used to recognize vocabulary in speech. Currently, it is used in automated human-computer interaction interface. Speaker recognition is used for recognition. The identity of the utterer is commonly used in Voiceprint Recognition, which is mainly used for personal identity recognition of surveillance or search.

On the other hand, commercial advertising is ubiquitous, including electronic music such as storefront music, telephone, television, radio or websites. It is nothing more than an increase in sales. Therefore, how to make advertising more effective and fast to achieve marketing purposes is the existing trading through electronic media. The subject of technology.

It is an object of the present invention to provide an advertising transaction system suitable for addressing the deficiencies of the prior art.

The advertising transaction system of the present invention has some implementation aspects An audio management server and an account management server that communicate with each advertiser user. The account server establishes account information of the advertiser users and forwards the audio data of the advertiser user to the audio management server. Each of the advertiser users uploads a sound source for playing on an electronic media to the audio management server. The audio management server processes each of the audio segments into a voiceprint data via an audio conversion program, and the voiceprint data corresponds to a track map including time and frequency, and processes the voiceprint data to retain the main track feature and A predetermined trajectory data of the background noise is removed. The audio management server establishes a web address corresponding to a piece of product information of each of the audio segments according to the request of the account management server.

In some implementations, the audio management server processes the audio segment into small segments and divides the small segments into Fourier transforms and wavelet transforms in a partially overlapping manner to obtain each time of the time period. Corresponding frequency peaks, and drawing a two-dimensional trajectory with time and frequency as two axes according to the frequency peaks at each moment of the period, and converting the two-dimensional trajectory into binarization a sparse matrix.

In some implementations, the audio management server processes the sparse matrix by using a clustering process to generate the predetermined trajectory data, and the clustering process uses a density-based clustering algorithm to define The maximum radius value of an adjacent area and the minimum number of points in the adjacent area to remove background noise.

In some implementations, the processing of the audio management server further includes performing multiple resolution processing to generate a reduced amount of data to be compared. Track data.

In some implementations, the advertiser user uploads a sound source for playing on a storefront sound, a phone, a television, a broadcast, or a website to the audio management server.

In some implementations, the advertisement transaction system further includes an account management server, the account server is configured to establish account information of a majority of the advertiser users and forward the audio data of each of the advertiser users to the audio management server. Device.

In some implementations, the advertisement transaction system further includes an account management server, the account server is configured to establish account information of the majority of the use terminals and forward the audio data of each of the use terminals to the audio management server.

In some implementations, the advertisement transaction system further includes a customer service server and operates in conjunction with a payment server, and the service server receives a request from each of the user terminals to send a content message carrying a product website address. Each of the user terminals is configured to send, by each of the user terminals, a purchase request message of the product to the payment server via the product website.

The utility model has the advantages that: by constructing the voiceprint database and matching the trajectory data, in addition to removing noise and avoiding misjudgment, the source of the obtained audio segment can be quickly and accurately compared, and the commercial application is used. Value is applicable to a variety of electronic media with the ability to deliver audio.

1‧‧‧Use terminal

200‧‧‧Communication network

30‧‧‧Sample database

31‧‧‧Customer Server

32‧‧‧Account Management Server

33‧‧‧Optical management server

331‧‧‧Transition module

332‧‧‧ clustering module

333‧‧‧ Alignment module

34‧‧‧Payment Server

300‧‧‧ trading system

S11~S26‧‧‧Message

301~308‧‧‧Steps

Other features and effects of the present invention will be apparent from the embodiments of the drawings, in which: 1 is a system diagram illustrating a preferred embodiment of a method for trading voiceprint data based on the present invention, which is applied to a use terminal and a transaction system; FIG. 2 is a schematic diagram illustrating a transaction method based on voiceprint data of the present invention. A communication process between the terminal and the transaction system is used; FIG. 3 is a flow chart illustrating a preferred embodiment of the method for translating voiceprint data according to the present invention; FIG. 4 is a schematic diagram showing that the embodiment is set every 32 milliseconds. a time unit of a sound box, and each sound box performs a short time interval Fourier transform in a manner of overlapping 50%; FIG. 5 is a schematic diagram illustrating a wave peak corresponding to each frequency at a certain time; FIG. 6a and FIG. 6b are A schematic diagram illustrating a two-dimensional trajectory map before and after background noise removal; FIGS. 7a and 7b are schematic diagrams illustrating sparse matrices of different orders; FIG. 8 is a schematic diagram illustrating storing a binarized sparse matrix as An integer value matrix; FIG. 9 is a schematic diagram illustrating storing a binarized sparse matrix as an integer value array; FIG. 10a to FIG. 10c are schematic diagrams illustrating trajectory data and servo trajectories taken from the client respectively. Than both the materials and the results.

Referring to FIG. 1, a preferred embodiment of the method for trading voiceprint data according to the present invention is applied to a use terminal 1 and a transaction system 300. Preferably, the transaction system 300 is an advertisement transaction system for at least one wide The advertisers (not shown) communicate with each other, and the advertiser user can upload a source for playing on an electronic media to the transaction system 300 and process it via the voiceprint based transaction method of the present invention. The sound source is a soundtrack or a human voice played by an electronic media advertisement program, and the electronic medium is an electronic medium having a transmission audio capability, such as a storefront sound, a telephone, a television, a broadcast, or a website.

In addition, the transaction system 300 can be communicatively coupled to a user terminal 1 from which the audio segment of the audio system can be retrieved from the audio source for a certain period of time. For example, the use terminal 1 can be, but is not limited to, a smart phone, and the terminal 1 can record a short segment (eg, 5 seconds) of the audio track or the vocal played by the playing commercial. The audio segment is sent to the transaction system 300.

The transaction system 300 includes a customer service server 31, an account management server 32, an audio management server 33, and a payment server 34, and uses the terminal 1, the customer service server 31, the account management server 32, and the audio management server. 33 and payment server 34 communicate information and communication with each other through a communication network 200. The communication network 200 includes various wireless communication and/or wired communication architectures, as long as the network architecture capable of transmitting voice data is applicable to the scope of the present invention. .

Referring to Fig. 2, the flow of the method for trading voiceprint data according to the present invention will be described below.

The customer service server 31 mainly performs the following steps: the customer service server 31 issues a request message S11 to the account management server 32 for the account of the advertiser user who uses the service. Customer service server 31 from account management server 32 A message S14 of receiving the registration result of the advertiser user. Next, the service server 31 uploads the message segment matching URL and the classification information message S15 to the account management server 32. The customer service server 31 receives the acceptance message S18 of the audio clip pairing web address from the account management server 32. The customer service server 31 transmits a content message S23 carrying the product website address from the use terminal 1 to receive the request. The service server 31 transmits the content message S24 of the product website to the user terminal 1.

The account server 32 is configured to establish account information of most advertiser users and forward the audio data of each advertiser user to the audio management server 33 for establishing a predetermined trajectory data, and used to establish and transfer the account data of the majority terminal 1 Each of the audio data of the terminal 1 is used by the audio management server 33 for establishing a track data to be compared (the effect will be described later).

The account management server 32 mainly performs the following steps: The account server 32 receives a request message S11 from the customer service server 31 for the account of the advertiser user who uses the service. The account server 32 sends a request message S12 to the audio management server 33 to provide storage space for the advertiser user. The account server 32 receives the request message S13 using the terminal 1 to establish a registration account. The account management server 32 issues a message S14 to the customer service server 31 using the registration result of the terminal 1. The account server 32 receives the audio clip pairing URL S15 from the customer service server 31. The account server 32 sends a request message S16 to the audio management server 33 to store the audio segment of the advertiser's user to the audio management server 33 and perform pre-processing for subsequent identification work. The account server 32 receives the request message S17 from the audio management server 33 to establish a lookup table index of the audio segment. The account server 32 transmits an acceptance message S18 of the audio clip pairing address to the customer service server 31. The account server 32 receives the message S19 containing the audio segment from the use terminal 1. The account server 32 forwards the message S20 containing the audio segment to the audio management server 33. The account server 32 receives the message S21 of the product website matching the audio segment from the audio management server 33. The account server 32 forwards a message S22 containing the product website to the use terminal 1.

The audio management server 33 mainly performs the following steps: the audio management server 33 receives the request message S12 of the advertiser user from the account management server 32 to provide storage space for the advertiser user. . The audio management server 33 receives the request message S16 from the account management server 32 to store the audio segments and perform pre-processing for subsequent identification work. The audio management server 33 sends a request message S17 to the account management server 32 to establish a lookup table index of the audio segment. The audio management server 33 receives the message S20 of the audio segment from the account management server 32, and searches for the corresponding item according to the feature matching manner of the audio segment. The audio management server 33 transmits a message S21 of the product website matching the audio segment to the account management server 32.

Then, the use terminal 1 transmits a product purchase request message S25 to the payment server 34. The payment server 34 transmits a related message S26 for the payment and delivery of the product to the use terminal 1.

In this embodiment, the audio management server 33 includes a conversion module 331, a clustering module 332, a comparison module 333, and the same database 30. The conversion module 331 processes the audio segments via an audio conversion program. a clustering data; the clustering module 332 clusters the voiceprint data to generate predetermined trajectory data for retaining the main trajectory feature and removing the background noise; and comparing the trajectory of the module 333 to be compared Data as an index Whether the comparison with the predetermined trajectory data pre-stored by the sample database 30 is similar, if similar, a corresponding information content is output to the terminal 1 via the account management server 32. The transaction module 334 is configured to send a product purchase request to the payment server 34 according to a message containing the information content (for example, a message S21 containing a product URL matching the audio segment) sent by the terminal 1. (eg message S25) to execute the transaction needs corresponding to the product URL.

Referring to Fig. 3, and in conjunction with Fig. 2, the preparation procedure of the audio management server 33 will be described below.

The audio management server 33 retrieves the advertisement audio (step 301), and the advertisement audio is, for example, an advertisement music file uploaded by the advertiser user via the account management server 32. Then, the audio management server 33 stores an audio segment uploaded by the advertiser user according to the request of the account management server 32, and processes the audio segment into the Fourier transform data via the conversion module 331 (step 302). In this embodiment, the audio segment is cut into small segments and each small segment is subjected to a short-time Fourier transform (STFT) in a partially overlapping manner to obtain a Fourier transform data.

Referring to FIG. 4, it is set to a time unit of one frame every 32 milliseconds, and each frame performs short-time Fourier transform with overlapping 50%.

Next, the conversion module 331 processes the Fourier transform data into wavelet transform data (step 303), and uses the wavelet transform data to obtain a peak group (step 304), which is the number corresponding to the wavelet transform data at each moment of the time period. Frequency peaks.

Referring to FIG. 5, this embodiment is based on each moment of the period of time. The peaks of the frequencies are plotted as a two-dimensional graph with time and frequency being two axes. Then, the wavelet multi-scale nature of wavelet analysis is used to obtain the peak value corresponding to each frequency at different times.

Then, the conversion module 331 converts the two-dimensional trajectory map into a binarized sparse matrix M (step 305), and the clustering module 332 processes the binarized sparse matrix M into density spatial clustering calculus data. (Step 306), the density spatial clustering calculation data is processed into the multi-resolution matrix data M1, M2 (step 307), and finally the matrix data M, M1, M2 is output (step 308) and stored in the sample database 30. .

Referring to Figures 6a-6b, this embodiment uses a density-based clustering algorithm (Density-Based clustering algorithm) to define the maximum radius value (Eps) of the adjacent region and the minimum number of points in the adjacent region (MinPts). In this way, the background noise as shown in FIG. 6a can be removed to obtain a two-dimensional trajectory diagram as shown in FIG. 6b after clustering.

Referring to Figures 7a-7b, the hidden multi-scale characteristics of wavelet analysis can be set to different resolutions to obtain different levels of sparse matrices. Figure 7b has lower order and lower resolution than way 7a. .

Referring to FIG. 8, the predetermined trajectory data format stored in the sample database 30 is stored, that is, the binarized sparse matrix is stored as an integer value matrix; referring to FIG. 9, the binarized sparse matrix is stored as an integer value array ( Array).

By reducing the size from the original binarization matrix M, for example: on the servo side, every 16 milliseconds of the 30 seconds of advertising audio is available. A set of 32-bit data, assuming that a set of data is an integer value element, and a 30-second original matrix has 8 x 1874 integer value elements. Thereby, two lower order matrices M1 (sizes of 4 x 936 integer value elements) and matrix M2 (sizes of 2 x 468 integer value elements) are obtained, and the sizes are 15 KB and 3.7 KB, respectively. Overall, the sample database 30 only needs to use 18.7 KB of space for storage.

In this embodiment, a Compute Unified Device Architecture (CUDA) is used, so that about 1.2 million audio data can be stored by using the memory space of 24 Gigabits of four unified computing architecture cards. On the client side, for every 5 seconds of advertising audio, the original matrix size is 624 rows x 256 columns. Assuming that an integer value element is stored as 32 bits, the original matrix M size is equal to 624 x 8 integer value elements, and the two lower order matrices M1 are equal to 4 x 312 integer value elements and the matrix M2 size is equal to 2x156 integers. The numerical elements are 5KB and 1.28KB respectively. Therefore, the matrix M1 and the matrix M2 are used as indexes for searching to the server, and only 6.3 KB of packets are required, which can reduce the amount of data transmission and the load on the server.

Therefore, when the audio management server 33 receives the predetermined trajectory data processed by the steps 301 to 308, the comparison module 333 can use the comparison feature data as an index and pre-exist. Whether the feature data comparisons of the sample database 30 are similar, and if they are similar, output a corresponding information content.

Referring to FIGS. 10a-10c, FIG. 10a is trace data taken from the client through the processed advertisement audio, and FIG. 10b is pre-stored by the server. The trajectory data of the processed advertisement audio, FIG. 10c is a trajectory data comparing the trajectory data from the client through the processed advertisement audio and the servo terminal pre-storing the processed advertisement audio, wherein the green dot indicates The two match the matching part.

In addition, in other embodiments, the terminal 1 can receive one sound source or multiple sound sources and become a multi-beam trajectory in a short time, wherein one or more segments of the trajectory can also be successfully paired with all the trajectories. One or more advertising sound sources also belong to the application of the present invention; in addition, the advertiser can also evaluate the advertising effectiveness according to the responses of the plurality of using terminals 1.

In summary, the effect of the present invention is that by constructing the voiceprint database 30 and matching the trajectory data, in addition to removing noise, the misjudgment can be avoided, and the audio segment can be quickly and accurately compared. The source, which has the value of commercial application and is applicable to various electronic media having the ability to transmit audio, can indeed achieve the object of the present invention.

The above is only the preferred embodiment of the present invention, and the scope of the present invention is not limited thereto, that is, the simple equivalent changes and modifications made by the patent application scope and patent specification content of the present invention, All remain within the scope of the invention patent.

1‧‧‧Use terminal

30‧‧‧Sample database

31‧‧‧Customer Server

32‧‧‧Account Management Server

33‧‧‧Optical management server

331‧‧‧Transition module

332‧‧‧ clustering module

333‧‧‧ Alignment module

34‧‧‧Payment Server

S11~S26‧‧‧Message

Claims (8)

An advertisement transaction system having an audio management server and an account management server communicating with a majority of advertiser users, wherein the account server establishes account information of the advertiser users and forwards the audio data of the advertiser user And the audio management server uploads a sound source for playing on an electronic media to the audio management server; the audio management server processes an audio segment into a voice data through an audio conversion program. The voiceprint data corresponds to a trajectory map including time and frequency, and the voiceprint data is processed as a predetermined trajectory data for retaining the main trajectory feature and removing background noise; and the audio management server manages the servo according to the account The request for the device establishes a web address corresponding to a piece of product information of the audio segment. The advertisement transaction system of claim 1, wherein the audio management server comprises: a conversion module, the audio segment is processed into a voiceprint data by an audio conversion program, and the voiceprint data corresponds to a time-containing data And a trajectory map of frequency; a clustering module, the voiceprint data is processed as a predetermined trajectory data for retaining the main trajectory features and removing background noise; and a comparison module is compared with the predetermined trajectory data Whether the trajectory data of the pair is similar, if similar, outputting a corresponding information content; and a transaction module, executing a corresponding transaction requirement according to the information content. The advertisement transaction system of claim 2, wherein the audio conversion program executed by the conversion module cuts the audio segment into small segments and performs Fourier transform and wavelet transform on the partial segments in a partially overlapping manner to obtain the a plurality of frequency peaks corresponding to each time of the time period, and drawing a two-dimensional trajectory map with time and frequency respectively as two axes according to the frequency peaks at each moment of the time period, and the two-dimensional trajectory map Converted to a sparse matrix of binarization. The advertisement transaction system of claim 3, wherein the clustering module generates the predetermined trajectory data by clustering the sparse matrix, and the clustering processing adopts a density-based clustering algorithm. The background noise is removed by defining a maximum radius value of a contiguous region and a minimum number of points in the contiguous region. The advertisement transaction system of any one of claims 2 to 4, wherein the conversion module further comprises a process of performing multiple resolutions to generate a track data to be compared with a reduced amount of data. The advertisement transaction system according to any one of claims 1 to 4, wherein each of the advertiser users uploads a sound source for playing on a storefront sound, a telephone, a television, a broadcast or a website. Audio management server. The advertisement transaction system according to any one of claims 1 to 4, wherein the account server further establishes account information of the majority of the use terminals and forwards the audio data of each of the use terminals to the audio management server. The advertisement transaction system of claim 7, further comprising a customer service server and operating with a payment server, the customer service server receiving each The content message of the product website is sent to each of the user terminals by the request of the terminal, and each of the user terminals sends a purchase request message of the product to the payment server through the product website.