JP2003047031A - Commercial detection method and apparatus therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately estimate a CM(commercial) section included in television broadcasting. SOLUTION: A section is extracted as a CM candidate section where two silent sections are present while being separated by a specific time interval from a silent section in a voice signal, and a CM broadcasting section where at least one CM is present continuously is judged. With all CM candidate sections included in the CM broadcasting section, it is judged whether a silent section is continuously arranged or not at a time interval that is considered to be a CM after the CM candidate section, and then the CM candidate is extracted so that more CM candidate sections continue. With each extracted CM candidate, reliability is calculated for outputting a CM candidate having high reliability so that reliability increases when a cut point in an image signal is located near the start or end point of the CM candidate, and so that reliability increases when other CM candidates are present at least before and after the CM candidate concerned.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for automatically detecting commercials during television broadcasting.

[0002]

2. Description of the Related Art Broadcasting of commercials (generally abbreviated as CM (commercial message); hereinafter also referred to as CM in this specification) during television broadcasting is requested by an advertiser or an advertising agency. The main contracts with a broadcasting station, and the audio data and video data provided by the client are broadcast between programs (main part). Such commercials are an important source of income for program production for broadcasting stations and an important advertising medium for advertisers. Also, C between the advertiser and the broadcasting station
Advertising agencies make adjustments for M broadcasting, but C
Whether or not M is broadcast according to contract is a very important issue for all three parties. For this reason,
At present, many advertising agencies use CM confirmation service, that is, C
It uses a service to check whether M has been broadcast. Broadcasting stations are also introducing a system in which the actual broadcast data and the output of the broadcast confirmation attached to the advertisement fee bill are properly linked. Also for advertisers,
Broadcast confirmation services provided by a third party are expected.

By the way, generally, although the broadcast time of a commercial is roughly fixed, the schedule of the program itself is urgent news or the convenience of the program itself (especially in the case of live broadcasting).
It is not strictly determined because it is often changed in. Therefore, CM broadcast confirmation cannot be performed by a method of checking whether a specific CM is broadcast by receiving a specific broadcast station at a specific time.

Up to now, the confirmation of the CM broadcast has been manually performed.
The survey report was created after checking the video for 4 hours 365 days. In this case, the broadcast confirmation worker
Perform a work that requires a huge amount of time, such as visually checking the television broadcast and naming the CM portion. Therefore, the cost for confirming the CM broadcast is very large, and the burden on the operator is also very large. Also, with this method, it is difficult to confirm the broadcast of the CM in real time because it requires human intervention. On the other hand, it is important for the broadcasting station whether the CM broadcast is properly performed, and a confirmation means that can immediately respond when the CM to be broadcast is not broadcast due to a trouble is important.

Considering that all CMs being broadcast are to be investigated, it is necessary to register the CM every time a new CM is broadcast. At this time,
It is necessary to distinguish whether the CM is known or new and register only the new CM. However, relying on this work manually requires an enormous amount of confirmation work time, and it is necessary to distinguish whether the CM is known or new, which is a very heavy load.

Therefore, a television C that automatically detects a CM in a television broadcast and automatically identifies the CM.
Realization of an M broadcasting survey system is required. In order to configure such a system, a system that estimates a section that is likely to be a CM during television broadcasting, a system that automatically registers a new CM that has been broadcast, and a system that automatically identifies a CM that is being broadcast Is required.

Conventionally, as a method of estimating a CM broadcast section during television broadcasting, there are a method of detecting a voice multiplex mode, a method of detecting a silent section and a screen change, and the like. Hereinafter, a conventional method of estimating a CM broadcast section will be described.

A well-known method for separating the main program from the CM is a method for detecting switching to the audio multiplex mode, as described in Japanese Patent No. 2858758. Since the main part of the program is often broadcasted in a monaural mode, and the CM is often broadcasted in a stereo mode, in this method, as shown in FIG.
The voice multiplex mode section is defined as CM. This method is often used as a CM skip function for a video deck. However, this method has a problem in that a plurality of CMs that are continuously broadcast cannot be divided into individual CMs and that the performance of the main program is deteriorated due to an increase in programs that are also stereo broadcasts.

On the other hand, Japanese Patent Publication No. 6-66738 discloses that
A CM section detection utilizing a silent section and a scene change is disclosed. A CM generally has (1) a silence section of several tens of ms before and after the CM, and (2) generally has a length of 15 seconds or 30 seconds (at least CM in Japan.
In the case of), it has the property that By using this property, it is possible to estimate a CM section by detecting a silent section and a scene change (cut point). Specifically, as shown in FIG. 18, the number of seconds from the time when it is determined that the silence and the cut occur at the same time to the time when it is determined that the next silence and the cut occur at the same time is the CM broadcast seconds. C if any of the predetermined number of seconds
It is determined as an M candidate. However, in this method, there are many places other than CMs, that is, places where the time interval is constant in the main story, so many false detections occur, and CMs to be detected cannot be detected due to omission of silent or cut detection. There are some problems, such as cases.

[0010]

As described above, the CM
In order to automate broadcast confirmation and real-time (allowing for short delays), automatic detection of CM sections and new C
It is necessary to realize automatic registration of M into the database.
SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a commercial detection method and apparatus that can accurately estimate a CM section from a television broadcast and automatically register a new CM in a database.

[0011]

A commercial detection method of the present invention is a commercial detection method for detecting a commercial included in a television broadcast, which comprises detecting a silent section from an audio signal of the television broadcast, From the extracted silent section, a step of extracting a section in which two silent sections are present at a predetermined time interval as a commercial candidate section, and one or more commercials are continuously present based on the extracted commercial candidate section. And a step of determining a CM broadcast section which is a section to be performed.

According to the commercial detection method of the present invention, for all CM candidate sections included in the determined CM broadcast section, silent sections are continuously arranged at the time intervals considered as commercial, following the CM candidate section. It is determined whether or not the CM candidate section having the largest number of continuous silent sections at the time interval is the head, and the CM section continuously arranged at the time interval is the delimiter for the CM candidate. It is preferable to provide a continuity evaluation step for extracting the CM, a cut point detection step for detecting a cut point from a video signal of a television broadcast, and a CM candidate extracted in the continuity evaluation step for each CM.
Confidence is high if there are cut points near the start and end points of a candidate, and is high if there is another CM candidate at least before or after the CM candidate. C which has a high degree of reliability and the stage of calculating the degree
And an output step of outputting M candidates.

A commercial detecting device of the present invention is a commercial detecting device for detecting a commercial included in a television broadcast, and includes a silent interval detecting means for detecting a silent interval from an audio signal of the television broadcast, and the detected silent sound. Based on the extracted CM candidate section, one or more commercials are continuously arranged based on the extracted CM candidate section, which is a CM candidate section in which two silent sections are separated from each other by a predetermined time interval. CM broadcast section determining means for determining a CM broadcast section that is an existing section.

In the commercial detection device of the present invention, for all the CM candidate sections included in the determined CM broadcast section, the CM candidate section is followed by a silent section continuously at a time interval considered as a commercial. It is determined whether or not they are arranged, and as a result of the determination, the CM candidate section having the largest number of continuous silent sections at the time interval is set as the head, and the silent section continuously arranged at the time interval is set as a delimiter for the CM candidate. It is preferable to provide continuity evaluation means for extracting a feature amount, a feature amount extracting means for extracting a feature amount from a frame of a television broadcast video signal, and a cut point detecting means for detecting a cut point based on the feature amount. And each CM extracted by the continuity evaluation means
Regarding a candidate, the reliability is high if there is a cut point near the start point and the end point of the CM candidate, and the reliability is high if there is another CM candidate at least before or after the CM candidate. It is preferable to provide a reliability determination unit that calculates the reliability so as to increase the reliability and outputs the CM candidate having the high reliability.

Particularly, in the present invention, the CM continuity is evaluated to prevent erroneous detection and non-detection of the CM broadcast section. In other words, the candidate for the CM broadcast section is extracted only in the silent section, and C is selected according to the position where the cut (scene change) exists.
The certainty of the M broadcast section is detected. Alternatively, a CM broadcast section candidate is extracted only by the cut points, and its certainty as a CM broadcast section is detected by the existence position of the silent section. This prevents erroneous detection and non-detection of the CM broadcast section.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION Next, preferred embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a commercial detection device according to a preferred embodiment of the present invention.

This commercial detection apparatus receives a television signal (video signal and audio signal) from a receiving unit 11 that receives a television broadcast wave or broadcast data (video signal and audio signal) previously stored in a storage unit 12. It is an input. Since this commercial detection device detects a CM section included in a broadcast wave and outputs the start and end times of the CM section, it also processes video signals and audio signals independently of each other. Since it is necessary to establish time synchronization, it is preferable that the video signal and the audio signal are added with information indicating the time timing in the broadcast wave.

The commercial detecting apparatus shown in FIG. 1 has a silent section detecting section 21 for detecting a silent section in a voice signal.
And a silent section detected by the silent section detection unit 21 is a section in which two silent sections are present with a certain time interval (a time interval commonly used as a CM length, for example, 15 seconds or 30 seconds) apart. CM which is a candidate for a section in which several CMs are continuously broadcast from the CM section extracted by the section with constant section extraction 22 and the section with constant section extraction 22
CM broadcast section determination unit 23 for selecting a broadcast section candidate, and C
A continuity evaluation unit 24 that evaluates continuity of all CM candidates in the M broadcast section candidates, a feature amount extraction unit 25 that extracts a frame feature amount from a video signal, and a scene change based on the extracted frame feature amount. As a result of the continuity evaluation performed by the cut detection unit 26 that detects (cut point) and the continuity evaluation unit 24, the cut detection unit 26 detects a CM candidate section that is highly likely to be a CM section. The reliability determination unit 27 that applies the cut points and finally outputs the CM candidate section as candidate data. Also,
In this commercial detection device, a silent section list holding unit 31 that holds a list of silent sections detected by the silent section detection unit 21 and a CM candidate section extracted by a fixed section interval extraction unit 22 as a CM candidate list. The CM candidate list holding unit 32 that holds the cut candidate and the cut holding unit 33 that holds the time of the cut point (scene change) detected by the cut detecting unit 36 are provided.

This commercial detection device estimates and extracts a CM broadcast section based on a silent section obtained from an audio signal and a cut position obtained from a video signal. Here, it is utilized that the CM has the following features. (A) CMs generally have a length of 15 seconds or 30 seconds. (B) CM tends to be continuously broadcast. (C) CMs are connected to each other in a silent section and a cut. That is, when the relationship between adjacent cut points satisfies the above three characteristics, the section is extracted as a CM candidate.

The details of each of the above units will be described below.

The silent section detector 21 detects a silent section from a voice signal in real time. There are several well-known methods for detecting a silent section in a voice signal. Here, for example, the voice signal is 8 bits (256 values).
If sampling is performed at a sampling frequency of 8 kHz, a section with a length of 50 milliseconds is one unit, and if 395 or more of 400 samples included in the section of 50 milliseconds have a value of 0 or 1, The method of determining that the section of 50 milliseconds is a silent section is adopted.

The fixed interval section extraction unit 22 detects, for the silent sections detected by the silent section detection unit 21, a section in which two silent sections exist at a certain time interval (for example, 15 seconds, 30 seconds), It is extracted as a CM candidate section. FIG. 2 is a conceptual diagram showing CM candidate extraction from a silent section by the constant section interval extraction unit 22. CM extracted
All the time zones (start time: end time) of the candidate section are CM
Record it in the candidate list. FIG. 3 shows an example of the CM candidate list.

Note that the process of extracting a fixed section interval can be performed in real time. For example, it is assumed that a 1-bit buffer is prepared in advance for 50 milliseconds of television broadcasting for 60 seconds. And
(1) Silence determination is performed for the newly input 50 millisecond section, (2) the write address to the buffer is shifted by 1 bit (1 unit), and (3) there is no sound. In that case, "1" is recorded in the buffer, and in other cases, "0" is recorded. Then, if it is considered as silence, the bit corresponding to the section 15 seconds before is checked in the buffer, and if it is "1", it is recorded as a CM candidate for 15 seconds. If it is "0", 3
The bit corresponding to the section 0 seconds before is checked, and if it is "1", it is recorded as a 30-second CM candidate. The CM candidate section can be detected by repeating the above processes (1) to (3).

FIG. 4 is a flow chart summarizing the series of processes. First, in step 101, the time position of a silent section is acquired. Next, in step 102, a certain time before the acquired time position (15
It is determined whether or not there is a silent section before 2 seconds or 30 seconds, here 15 seconds before is prioritized as described above.
If there is a silent section, in step 103, a CM candidate is output with the silent section before the fixed time as the start point and the silent section of interest as the end point. That is, it is recorded in the CM candidate list. Then, the process proceeds to step 104. If it is not a silent section in step 102,
The process proceeds to step 104 as it is. In step 104, it is determined whether or not the currently silent section is the last silent section, and if it is the last silent section, the process is terminated as it is, and if it is not the last silent section, step 105
In step 1 of obtaining the time position of the next silent section in
Return to 02.

By the above processing, a part having a constant interval is extracted.

Next, C in the CM broadcast section determination unit 23
The process of determining the broadcast section of M will be described. FIG. 5 is a conceptual diagram showing this processing. Here, the CM broadcast section is selected from the CM candidate list. The CM broadcast section refers to a section in which several CMs are continuously broadcast.

Even if the false detection across CM is detected by only the processing of the silent section detection and the constant section interval extraction described above,
It will be regarded as an M candidate. Therefore, the CM broadcast section determination unit 23 first extracts a CM broadcast section and performs processing for deleting a CM candidate overlapping in the section. The process of deleting the overlapping CM candidates is performed by the time interval Δ between the CM candidates.
When t exceeds a certain value (for example, 30 seconds), it is temporarily cut off, and the first CM candidate that appears after a certain time interval is:
The process is restarted as belonging to the next CM broadcast section. The value of 30 seconds here is the maximum duration that can be considered as a normal commercial (except for a long commercial that is rarely broadcast).

FIG. 6 is a flow chart showing the processing of the CM broadcast section determination. First, in step 111, all of the start points and end points of CM candidates are arranged in time series as points on the time axis. And step 112
In step 113, the time position of the start point is acquired, and in step 113, the distance Δt between the acquired time position and the time position of the next point is calculated. In step 114, it is determined whether Δt is longer than a fixed time length (30 seconds), and if it is longer, in step 115, the acquired time position is output as the end point and the next point is output as the start point. Move to 116. In step 114, Δt
If is less than the fixed time length, step 116 is performed as it is.
Move to. In step 116, it is determined whether or not the point of interest is the last point, and if it is the last point, the process ends. If it is not the last point, in step 117 the time position of the next point is set. Obtain and return to step 113.

By the above processing, the CM broadcast section is determined.

FIG. 7 shows the processing when the above-mentioned CM broadcast section is determined for each CM candidate section of the CM candidate list shown in FIG. From the commercial candidate section shown in FIG. 3, 0:01:30 to 0:02:45 and 0:
It can be seen that 25:15 to 0:26:30 are extracted.

Next, the process of CM candidate extraction (that is, CM segment estimation) by continuity evaluation in the continuity evaluation unit 24 will be described. FIG. 8 is a diagram illustrating CM segment estimation by continuity evaluation. In FIG. 8, the numerical value attached to the elongated rectangle representing the silent section indicates how many CM candidates are continuous following this silent section (the number of continuous CM candidates). Here, for the CM broadcast section candidates obtained by the CM broadcast section determination unit 23, the CM
The following operations [1] to [3] are repeated for the CM candidates included in the broadcast section candidates. [1] 15 seconds or 3 at the end point of the selected CM candidate
After 0 second, it is checked whether or not there is a silent section. [2] When there is a silent section in [1],
The next 15 or 30 seconds is followed by a sequential check for the presence of further silence. If there is no silent section in [1], the process is terminated. [3] The same process is performed for each silent section, and a silent section having the largest number of silent sections is selected every 15 or 30 seconds.

The selected CM candidate is at the head, and the CM is separated by continuous silent intervals of 15 seconds or 30 seconds.
Make it a candidate.

FIG. 9 is a flow chart showing the process of CM segment estimation by the continuity evaluation. First, in step 121, the time position of the end point of the CM candidate is acquired, and in step 122, it is determined whether the end point exists after a fixed time (15 seconds or 30 seconds) after the acquired time position. If there is an end point here, in step 123, 1 is added to the count of the number of continuous CM candidates for the CM candidate of interest, and step 124
At, the time position of the end point after a fixed time is acquired, and the process returns to step 122. On the other hand, when there is no end point in step 122, it is determined in step 125 whether the CM candidate of interest is the last CM candidate. If it is the last CM candidate, the CM with the largest number of continuous CM candidates in step 126.
Each continuous CM candidate with the candidate at the head is output, and the process ends. If it is not the last CM candidate in step 125, the time position of the end point of the next CM candidate is acquired in step 127, and the process returns to step 122.

By the above processing, a continuous series of CM candidate sections is extracted from the CM broadcast section.

FIG. 10 shows how continuous CM candidates are obtained from the candidate list shown in FIG. 3 and the CM broadcast section (0:01:30 to 0:02:45) obtained in FIG. 7 by continuity judgment. It indicates whether the section has been extracted. In this example, 0:01:30 to 0:01:45, 0:01:45 to 0:02:00, 0:
02:00 ~ 0:02:15, 0:02:15 ~ 0:02:30, 0:02:30 ~ 0:02:45
5 CM candidate sections are extracted.

Next, a process for extracting a cut (scene change) from a video signal will be described. In order to estimate the CM broadcast section, an appropriate process can be used as the process for extracting the cut, but in this embodiment,
Since it is possible to register the new CM in the database using the result of CM broadcast section estimation, the cut in the video signal is detected based on the frame feature amount.

The video signal is first input to the feature amount extraction section 25. The feature amount extraction unit 25 captures each frame from the video signal and generates a feature amount used for cut detection from the captured image. Various types of frame features are assumed, but here, MPEG (motion
picture expert) -7 FCD (final committee draf
t; use the color layout descriptor adopted in the final draft). Note that the same feature amount as the feature amount used here is also used as the feature amount of the CM when registering the CM in the database.

The cut detection unit 26 performs cut detection by performing a difference between the frames with respect to the characteristic amount obtained by the characteristic amount calculation unit 25. There are various cut detection methods based on the feature amount, as is known to those skilled in the art, and these methods can be appropriately selected and used here. In addition, in order to prevent omission of cut detection, parameters for cut detection are set so that the cut detection result tends to be excessively detected.

As described above, the CM candidate section is extracted from the audio signal based on the silent section, and the cut is detected from the video signal. Next, the reliability determination unit 27 determines whether or not each CM candidate extracted by the continuity evaluation unit 24 is a CM by using the cut detection result. The conditions for CM are as follows. (A) A cut exists near the start and end points of a CM candidate. (B) A CM exists before or after the CM candidate.

In the determination of (a), investigation is performed from several frames before the start point to several frames after the start point, and if a cut exists, it is checked whether the cut exists 15 seconds or 30 seconds later. When the end cut corresponding to the start cut is present, the process moves to the investigation of (b). In (b), the presence or absence of a cut 15 seconds or 30 seconds before the start cut, or the presence or absence of a cut 15 seconds or 30 seconds after the final cut is checked, and if a cut exists in either one,
The inputted CM candidate will be recognized as a CM.

On the other hand, the CM start position is also determined at the same time. Because we are investigating from several frames before to several frames later,
The position of the detected CM may be shifted by several frames as compared with the position actually broadcast. For this reason, the CMs that satisfy the above conditions (a) and (b) are not immediately regarded as CMs at the stage of the investigation, but the reliability of the CMs is calculated, and all the frames to be investigated are processed. The method of selecting the section with the highest CM reliability at a short time is adopted.
Here, the calculation of the reliability, which is CM, uses the following two parameters and calculates the reliability as a score. -Add points in proportion to the number of CMs adjacent to the CM. -Evaluation of the time interval between the start cut and the end cut. The point is added as it approaches exactly 15 seconds or exactly 30 seconds (when the adjacent CM is investigated while allowing the deviation of several frames, the deviation is added as an evaluation value).

FIG. 11 is a flow chart showing the processing of the reliability determination unit 27. First, in step 131, the time positions of the start point and end point of the CM candidate are acquired, in step 132, the presence or absence of cut point frames around the start point is investigated, and in step 133, for each cut point frame, a fixed time is determined. The number of continuous cut points at a certain interval, a constant value for the time interval (15 seconds or 3
The deviation from 0 second) is obtained to obtain the reliability of the CM candidate.
Then, in step 134, it is determined whether or not the highest reliability value exceeds a predetermined threshold value th. If yes, in step 135,
The cut point frame having the highest reliability value is output as the CM start point, and the process ends. On the other hand, step 13
If not exceeded in step 4, step 13
In 6, it is determined that the input CM candidate is not a CM, and the process ends.

As described above, the commercial detection device of this embodiment finally outputs the CM segment estimation result.

Next, the results of an experiment in which CM is actually detected using this commercial detection device will be described.
First, the results of an experiment in which CM estimation is performed using one of cut and silent sections will be described.

Experiment 1 When the detected cuts and the detected silent sections were subjected to the above-described processings of extracting the section with a constant section interval, determining the CM broadcast section, and evaluating the continuity, the CM section was correctly detected. Experimented with what was estimated. The experiment was performed on the video broadcast on the fourth channel in the Tokyo area from 0:00 to 2:00 AM on April 6, 2000. Actually, 89 CMs are broadcast in this time zone, and CMs to be detected are limited to those having a length of 15 seconds or 30 seconds. Regarding the cut (video signal), with respect to the cut position detected by the processing by the feature amount extraction unit 25 and the cut detection unit 26 described above, the constant interval section extraction unit 22, C
Each processing algorithm in the M broadcast section determination unit 23 and the continuity evaluation unit 24 was applied. For the silent section (speech signal), the output of the continuity evaluation unit 24 in the commercial detection device described above was directly evaluated. The results are shown in Table 1. The results were tested twice for each of the cut and silent sections, and the average value was adopted.

[0046]

[Table 1]

It can be seen from Table 1 that a higher detection rate and a lower erroneous detection rate are obtained when the silent section is used than when the cut is used. On the other hand, undetected causes include (1) detection of two or more CM broadcast sections as one CM broadcast section in the CM broadcast section determination process, and (2) detection due to undetected silent section. It is possible.

FIG. 12 is a diagram for explaining CM undetection due to detection of two or more CM broadcast sections as one CM broadcast section. Although there are actually two or more commercial broadcast sections, there are cases where a plurality of commercial broadcast sections are detected as one commercial broadcast section because the interval between commercial broadcast sections is less than 30 seconds. . This occurs when the ending of a program (main part) with a length of several seconds (for example, about 5 seconds) flows during the CM broadcasting section. When it is detected as one CM broadcast section, only CM candidates in the CM broadcast section with the most CMs are detected, and other sections are undetected.

Therefore, as a countermeasure against the undetected CM candidate in such a case, there are the followings as shown in FIG. [1] At the time of evaluating the continuity of CM candidates, checks are performed in order from the first silent section. [2] When CM candidates are continuous, the end point position of the last continuous CM candidate is stored. [3] CM for which the starting point of the CM candidate currently being evaluated has already been evaluated
It is checked whether it is behind the end point position for the candidate, and if it is behind, the count is terminated once, and CM candidate extraction processing is performed. [4] From the starting point of the CM candidate currently evaluated, [1]-
The processing of [3] is performed.

When this measure is incorporated into the commercial detection device described above, the processes [1] to [4] may be executed when the continuity evaluation section 24 evaluates the continuity.

FIG. 14 is a diagram for explaining undetected CM due to undetected silent section. When there is no silent section between the last commercial included in the CM broadcast section and the program (main story), or the first CM included in the program (main story) and CM broadcast section
There may be no silent section between, and
There is a case where the silent section is not detected due to the problem of the silent detection method. In such a case, there is a situation in which the CM, especially the CM at the end or the beginning is dropped. Figure 14 shows CM
An example is shown in which there is no silent section between the last CM included in the broadcast section and the program.

Therefore, as a measure against the undetected CM in such a case, as shown in FIG.
There are the following.

[1] One CM candidate is added to the end of the detected CM candidates (CM length is not determined).

[2] For the added CM candidate, 15
Assuming both second CM and 30 second CM, each process of feature amount extraction, cut detection, and reliability determination is executed. However,
When detecting cuts, a detection parameter that is significantly underestimated than usual is used, and when calculating the reliability, the threshold value is set much higher than usual, and
Make it difficult to detect as a candidate. Most things will be dropped by the judgment using the cut.

Experiment 2 The experiment was conducted again by adding the above two measures to the undetected CM. The experimental conditions are the same as in Experiment 1, but the experiment was conducted only in the silent section. Table 2 shows the comparison results of detection performance before and after countermeasures.

[0056]

[Table 2]

By introducing a measure against undetected CM, the number of undetected CM could be reduced to two. On the other hand, the number of falsely detected CMs increased by two. This undetected CM
Is caused by the failure of silence detection.

Next, an experiment was conducted to estimate CM using both the cut and silent sections.

Experiment 3 Using the commercial detection apparatus according to the present embodiment, in which CM candidates are narrowed down by silent sections and verified by using a CM,
An estimation experiment was conducted. Experiment is AM on April 6, 2000
The video was broadcast on Channel 4 in the Tokyo area from 0:00 to 2:00. Actually, 89 CMs were broadcast during this time period, and Table 3 shows the results of the detection experiment performed on these CMs. The experiment was conducted by introducing the two countermeasures against the undetected CM described in Experiment 2.

[0060]

[Table 3]

Compared to the case where the CM estimation is performed only in the silent section, when the silent section + cut is used, the false detections can be significantly reduced from 17 to 2 without reducing the normal detection.

Experiment 4 Next, a CM estimation experiment by the commercial detection device of the present embodiment was conducted by adding images in different time zones. The experiment targeted all 219 CMs that were aired on the fourth channel in the Tokyo area from 0:00 to 6:00 AM on April 6, 2000. The results are shown in Table 4.

[0063]

[Table 4]

From Table 4, about 98 CMs actually flowed
You can see that% is detected correctly. Moreover, the number of false detections could be suppressed to three. Compared to the conventional CM detection system using a video signal, which had a detection performance of about 80%, the performance could be greatly improved.

As described above, paying attention to the introduction of the combination of the silent section and the cut and the continuity of the CM, the CM candidate is extracted by the position and the continuity evaluation of the silent section, and the reliability thereof is determined by the existence position of the cut. By the calculation, the CM section estimation performance is 98% and the estimation performance based on only the video signal is 8
Compared to 0%, the performance could be improved significantly.

In the commercial detection system described above, the CM candidate is extracted by the position and continuity evaluation of the silent section, and the reliability thereof is calculated by the existence position of the cut, but the detection accuracy is somewhat lowered, It is also possible to reverse the way of applying the cut and the silent section. Specifically, the time position of the cut is detected, a section with a constant interval is extracted from the time position of the cut, the CM broadcast section is determined, and CM candidates are extracted by continuity evaluation. Then, the reliability of the extracted CM candidate is determined using the detected silent section. That is, for the CM candidate extracted from the cut, there is a silent section near the start and end points of the CM candidate, and the reliability is calculated depending on whether the CM exists before or after the CM candidate, A CM candidate is output based on the calculated reliability.

Next, a configuration for registering a new CM in the database will be described. With this configuration, CMs in television broadcasting are automatically extracted by using, for example, the commercial detection device described above, CM old / new is judged by checking with existing CMs, and only new CMs are automatically registered in the database. It is possible. FIG. 16 is a block diagram showing a system for registering a new CM in a database.

This system is used in combination with the commercial detection device described above, shares a part of the components of the commercial detection device (specifically, the feature extraction section 25), and is used by a television. A television video signal from the receiving unit 11 that receives a broadcast wave or broadcast data (video signal) previously stored in the storage unit 12 is input. Then, the feature amount extraction unit 25 that extracts the frame feature amount from the video signal, the program feature amount storage database 41 that stores the extracted feature amount, the CM database 42 that registers the CM, and the CM in the commercial detection device. Based on the detection result, a cutout unit 43 that cuts out the feature amount of the CM portion, a new CM determination unit 44 that compares the cutout CM with a CM already registered in the CM database 42 to determine a new CM, and a new work A broadcast history confirmation unit 45 for checking omission of detection of CMs determined to be CMs, and a CM database 42 with names and titles of CMs confirmed to be new CMs.
And a CM registration section 46 for registering with the. The feature amount extraction unit 25 is provided with a capture unit 51 that captures an image signal frame and a feature amount calculation unit 52 that calculates a feature amount from the captured frame.

Next, the operation of the system shown in FIG. 16 will be described. Here, as the feature amount extracted from the frame, as in the case of the commercial detection device described above, MP
The color layout identifier of the EG-7 FCD shall be used. Of course, another feature amount may be used.

In the feature amount extraction unit 25, the capture unit 51 captures a frame from the input video signal (broadcast wave), and the feature amount calculation unit 52 extracts a color layout descriptor as a feature amount from each frame. . At this time, the broadcast time is also obtained as the calling time information, and the calling time information and the characteristic amount are sequentially stored as the characteristic amount data in the program characteristic amount storage database 41.

Next, the cut-out section 43 cuts out the feature amount data of the CM portion from the feature amount data of the program based on the CM detection result (CM candidate data) input from the commercial detection device.

Next, in the new CM judging section 44, the CM cut out by the cutting section 43 is collated with the CM already registered in the CM database 42, and whether or not the cut out CM is a new CM. To judge.
CMs are registered in the CM database 42, and for each CM, the characteristic amount of the CM section, image data, and metadata indicating the contents are held. The feature amount of the CM candidate cut out by the cutout unit 43 is collated with the feature amount of each CM registered in the CM database 42. CMs that are regarded as the same CM database 42
If the CM does not exist, it is set as a new CM candidate. In the collation of CMs, not only collation is performed from the beginning of each CM in order, but also collation is performed by shifting the beginning position of one CM.

In this way, new commercial candidates are extracted.

Next, in order to confirm whether or not the extracted new CM candidate is really a new CM, the broadcast history confirmation section 45 confirms whether or not the CM has been broadcast in the past. Even with the commercial detection system described above, some CMs are inevitably omitted, so there is a possibility that the CM has already been broadcast when a new CM candidate is detected. Leakage will occur. Therefore, the broadcast history confirmation unit 45 is a new commercial
The new CM candidate extracted by the determination unit 44 is compared with the entire program feature amount stored in the program feature amount storage database 41, and the number of broadcasts of the CM and its broadcast time zone are checked.

The CM confirmed by the broadcast history confirmation unit 45 as a new CM is registered in the CM database 42 by the CM registration unit 46 with a name and a title.

Further, by registering all the new CM candidates without going through the broadcast history confirmation section 45, it becomes possible to register all the new CMs broadcasted in the CM database 42.

Next, an experiment conducted on this new CM registration system will be described. An experiment for checking the presence or absence of the same CM was performed using a broadcast wave of one channel for about one week. When an experiment was performed on 30 CMs whose broadcasting frequency was known in advance, it was possible to accurately detect all the broadcasting time and frequency without over-detection or undetection.

As described above, according to this new CM registration system, CMs in television broadcasting are automatically extracted,
It is possible to judge whether a CM is old or new by collating with an existing CM, and automatically register only the new CM in the database.

Further, according to this new CM system, it is possible to create a list of CMs that have appeared for the first time within a certain period, or a list of all new CMs.

Although the preferred embodiment of the present invention has been described above, the television broadcasting to which the present invention is applied is not limited to the terrestrial television broadcasting, but satellite television broadcasting or cable television. It may be John (CATV) broadcasting. Further, it also includes video information broadcast via a network, for example, video content transmitted via a network such as the Internet.

[0081]

As described above, the present invention has an effect that a CM section can be accurately estimated from a television broadcast and a new CM can be automatically registered in a database.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a commercial detection device according to a preferred embodiment of the present invention.

FIG. 2 is a conceptual diagram showing extraction of commercial candidates from silent sections by a section with constant section intervals extraction unit.

FIG. 3 is a diagram showing an example of a CM candidate list.

FIG. 4 is a flowchart illustrating a process of extracting a section with a constant section interval in a section with a section having a constant section interval;

FIG. 5 is a conceptual diagram showing determination of a CM broadcast section by a CM broadcast section determination unit.

FIG. 6 is a flowchart showing a process of determining a CM broadcast section in a CM broadcast section determination unit.

7 is a diagram showing an example in which a CM broadcast section is determined for the CM candidate section shown in FIG.

FIG. 8 is a conceptual diagram showing CM candidate extraction (CM section estimation) by continuity evaluation in a continuity evaluation unit.

FIG. 9 is a flowchart showing a CM candidate extraction process by continuity evaluation in a continuity evaluation unit.

10 is a CM candidate section shown in FIG. 3 and C shown in FIG.
It is a figure which shows the example which performed CM candidate extraction by continuity evaluation based on the M broadcast area.

FIG. 11 is a flowchart showing a process of finally outputting a CM candidate section as candidate data in a reliability determination unit.

FIG. 12 is a diagram for explaining one of the reasons why CM is not detected.

FIG. 13 is a diagram for explaining a measure against undetected CM.

FIG. 14 is a diagram illustrating another cause of undetected CM.

FIG. 15 is a diagram for explaining a measure against undetected CM.

FIG. 16 is a diagram showing a configuration of a system for registering a new CM.

FIG. 17 is a diagram illustrating a conventional CM section detection method using a voice multiplex mode.

FIG. 18 is a diagram illustrating a conventional CM section detection method using a silent section and a scene change.

[Explanation of symbols]

11 Receiver 12 Storage 21 Silent section detector 22 Fixed interval extraction unit 23 CM broadcast section determination unit 24 Continuity Evaluation Department 25 Feature Extraction Unit 26 Cut detector 27 Reliability determination unit 31 silent section list holding unit 32 CM candidate list holding unit 33 Cut holder 41 Program Feature Saving Database 42 CM database 43 Cutout part 44 New CM judgment section 45 Broadcast history confirmation section 46 CM registration section 51 Capture Department 52 Feature Calculation Unit

Claims (16)

[Claims] 1. A commercial detection method for detecting a commercial included in a television broadcast, comprising: detecting a silent segment from an audio signal of the television broadcast; and two silent segments from the detected silent segment. Extracting a section existing at a predetermined time interval as a CM candidate section, and determining a CM broadcast section that is a section in which one or more commercials continuously exist based on the extracted CM candidate section And a commercial detection method comprising: 2. For all CM candidate sections included in the determined CM broadcast section, it is checked whether or not silent sections are continuously arranged at time intervals considered as commercials following the CM candidate section. Judgment and, as a result of the judgment, the CM candidate section having the largest number of continuous silent sections at the time interval is set as a head, and the CM candidate section is extracted using the silent section continuously arranged at the time interval as a delimiter. The commercial detection method according to claim 1, comprising steps. 3. A cut point detection step of detecting a cut point from a video signal of the television broadcast, and each CM candidate extracted in the continuity evaluation step, in the vicinity of a start point and an end point of the CM candidate. A step of calculating the reliability so that the reliability is high if there is a cut point, and the reliability is high if there is another CM candidate at least one before or after the CM candidate; The commercial detection method according to claim 2, further comprising: an output step of outputting the obtained CM candidate. 4. The cut point detecting step according to claim 3, further comprising a step of extracting a feature amount from a frame of the video signal, and a step of detecting the cut point based on the feature amount. Commercial detection method. 5. The step of cutting out the feature quantity corresponding to a commercial section based on the CM candidates output in the output step, and the cut-out feature quantity and the feature quantity registered in the CM database are collated. By determining whether the commercial section corresponding to the cut-out feature amount is a new commercial candidate, and if it is determined that the new commercial candidate is a new commercial candidate, the new commercial candidate as a new commercial C
The commercial detection method according to claim 4, further comprising: registering in the M database. 6. A step of registering the extracted feature quantity of each program in a program feature quantity storage database, and a CM candidate output in the output step,
The step of cutting out the feature amount corresponding to the commercial section and the feature amount registered in the CM database are collated with each other, so that the commercial section corresponding to the cut-out feature amount is a new commercial. The broadcast history by comparing the feature quantity corresponding to the new commercial candidate and the feature quantity in the program feature quantity storage database when it is determined that the candidate is a new commercial candidate. 5. The commercial detection method according to claim 4, further comprising the step of confirming the presence or absence of the above, and the step of registering the new commercial candidate as a new commercial in the CM database when there is no broadcast history. 7. In the continuity evaluation step, when the start point of the CM candidate currently being evaluated is after the end point position of the CM candidate that has already been evaluated, silent sections are continuously arranged. The process of extracting the CM candidate by temporarily terminating the determination as to whether or not the end point exists, and executing a process of extracting a CM candidate using a silent section continuously arranged at the time interval as a delimiter. The commercial detection method described. 8. The method according to claim 2, wherein the continuity evaluation step is executed by assuming a CM candidate section that is continuous with the CM broadcast section immediately before or after the CM broadcast section. Commercial detection method. 9. A commercial detection method for detecting a commercial included in a television broadcast, comprising: detecting a cut point from a video signal of the television broadcast; and two cut points from the detected cut point. Extracting a section existing at a predetermined time interval as a CM candidate section, and determining a CM broadcast section that is a section in which one or more commercials continuously exist based on the extracted CM candidate section And for all CM candidate sections included in the determined CM broadcast section, it is determined whether the cut points are continuously arranged at a time interval considered as a commercial, following the CM candidate section. However, as a result of the determination, the CM candidate section having the largest number of continuous cut points at the time interval is set as the head, and the CM candidate section is continuously executed at the time interval. With respect to each CM candidate output in the continuity evaluation step, a continuity evaluation step of extracting a CM candidate using the arranged cut points as delimiters, a step of detecting a silent section from the audio signal of the television broadcast, If there is a silent section near the start point and the end point of the CM candidate, the reliability becomes high, and if there is another CM candidate at least before or after the CM candidate, the reliability becomes high. And a step of calculating the reliability, and outputting a CM candidate having a high reliability. 10. A commercial detection method for detecting a commercial included in a television broadcast, comprising: extracting a feature amount from a frame of a video signal of the television broadcast; and the feature corresponding to the detected commercial section. It is determined whether the commercial section corresponding to the cut-out feature quantity is a new commercial candidate by collating the cut-out feature quantity and the feature quantity registered in the CM database with the step of cutting out the quantity. If it is determined that the new commercial candidate is a new commercial candidate, the new commercial candidate is regarded as the new commercial, and
Registering in the M database; 11. A commercial detection method for detecting a commercial included in a television broadcast, comprising: extracting a feature amount from a frame of a video signal of the television broadcast; and extracting the feature amount of each extracted program as a program. By registering in the feature quantity storage database, cutting out the feature quantity corresponding to the detected commercial section, and collating the cut out feature quantity with the feature quantity registered in the CM database, The step of determining whether the commercial section corresponding to the cut-out feature amount is a new commercial candidate, and the feature amount and the program feature amount corresponding to the new commercial candidate when it is determined to be a new commercial candidate The step of checking the presence or absence of a broadcast history by comparing it with the feature amount in the storage database, and the broadcast history If there is not, registering the new commercial candidate as a new commercial in the CM database, a commercial detection method comprising: 12. A commercial detection device for detecting a commercial included in a television broadcast, comprising: a silent segment detecting means for detecting a silent segment from an audio signal of the television broadcast; and 2 from the detected silent segment. Based on the extracted CM candidate sections, one or more commercials are continuously present based on a section interval constant point extraction means for extracting a section in which two silent sections are present at a predetermined time interval as CM candidate sections. A commercial detection apparatus that has a CM broadcast section determination unit that determines a CM broadcast section that is a section. 13. For all CM candidate sections included in the determined CM broadcast section, whether or not silent sections are continuously arranged at a time interval considered as a commercial following the CM candidate section. Judgment and, as a result of the judgment, the CM candidate section having the largest number of continuous silent sections at the time interval is set as a head, and the CM candidate section is extracted using the silent section continuously arranged at the time interval as a delimiter. The commercial detection device according to claim 12, comprising means. 14. A feature quantity extraction means for extracting a feature quantity from a frame of a video signal of the television broadcast, a cut point detection means for detecting a cut point based on the feature quantity, and a continuity evaluation means for extraction. With respect to each CM candidate thus selected, if there is a cut point in the vicinity of the start point and the end point of the CM candidate, the reliability becomes high, and another CM candidate exists at least before or after the CM candidate. CM that has a high reliability calculated by calculating the reliability so that the reliability is high.
The commercial detection device according to claim 13, further comprising a reliability determination unit that outputs a candidate. 15. A commercial section is supported based on a program feature amount storage database that stores the feature amount of each program, a CM database that holds the commercial feature amount, and a CM candidate output from the reliability determination means. By comparing the cut-out feature for cutting out the feature amount with the cut-out feature amount and the feature amount registered in the CM database, the commercial section corresponding to the cut-out feature amount is a new commercial candidate. A new commercial deciding means for deciding whether or not there is, and when it is decided that it is a new commercial candidate, the feature amount corresponding to the new commercial candidate is collated with the feature amount in the program feature amount saving database for broadcasting. Broadcast history confirmation means to confirm the existence of history, and the new commercial information if there is no broadcast history. Commercial detection apparatus according to feature quantity in claim 14, further comprising a registration means for registering the CM database. 16. A commercial detection device for detecting a commercial included in a television broadcast, comprising: a feature extraction means for extracting a feature from a frame of a video signal of the television broadcast; and a feature stored in each program. A program feature amount storage database, a CM database that holds a commercial feature amount, a cutout unit that cuts out the feature amount corresponding to a detected commercial section, and the cut out feature amount and the CM database that are registered in the CM database. By comparing with the existing feature amount, a new commercial determination unit that determines whether the commercial section corresponding to the cut-out feature amount is a new commercial candidate, and if it is determined that the commercial segment is a new commercial candidate, Feature quantity corresponding to the new commercial candidate and the program feature quantity save data A commercial having a broadcast history confirmation means for checking the presence or absence of a broadcast history by collating with the feature quantity in the base, and a registration means for registering the feature quantity of the new commercial candidate in the CM database when there is no broadcast history. Detection device.