JP2007157064A - Signal search unit and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To widen a range not to be searched than conventionally, by defining all the range from a current position to a skip candidate position as the range not to be searched. <P>SOLUTION: A signal search unit comprises: a reference signal characteristic quantity calculation section 104 for calculating a characteristic quantity by inputting a reference signal 101 which is a search key; an input signal characteristic quantity calculation section 105 for calculating the characteristic quantity of an input signal 102 to be searched for; a similarity degree calculation section 106 for calculating the degree of similarity between the characteristic quantity of the reference signal 101 and that of the input signal 102; a correlation skippable width calculation section 107 for calculating the maximum value for skipping based on the degree of the similarity; a skip width calculation section 108 for determining the width for actual skipping based on the skippable width; and a backtracking determination section 109 for determining whether it is necessary to backtrack based on the degree of the similarity of a search window of a skip destination. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a signal search apparatus and method for searching for a portion similar to a pre-registered signal from a long-time signal sequence.

  Patent Document 1 discloses a time-series active search method in which a search unnecessary range is calculated from the similarity calculated by histogram intersection, and processing of the range is skipped to speed up speech signal search.

The time-series active search method is developed in Patent Document 2 and Non-Patent Document 1, and the search unnecessary range calculated from the similarity of the current search position and the backward search calculated from the similarity of the temporary skip position A time-series active search method by overskiing is disclosed in which the entire range from the current search position to the tentative skip position is not required to be searched when the unnecessary range overlaps.
Number 1 and number 2 of Japanese Patent No. 30653314 FIG. 3 of Japanese Patent No. 3474131 Kunio Kanno and Hiroshi Murase, “Acceleration of time-series active search method by overskiping”, Proceedings of Acoustical Society of Japan, pp. 445-446, 1999.

  The methods according to Patent Literature 2 and Non-Patent Literature 1 independently calculate the similarity of the current position and the similarity of the skip candidate position, and do not consider the correlation of the similarity. Therefore, if both similarities are taken into account, there is a problem that a range that should not be searched is searched and the speed cannot be sufficiently increased.

  The present invention has been made to solve the above-described problems, and provides a signal search apparatus and method capable of extending the signal search unnecessary range as compared with the conventional technique.

  The signal search device of the present invention is a signal search device for searching a signal sequence similar to a reference signal sequence while skipping the input signal sequence, and a current position for setting a search window at the current search position on the input signal sequence Setting means; skip candidate position setting means for setting a search window at a skip candidate position that is a candidate for a next search position on the input signal string; a signal string included in the search window at the current position; and the reference signal string Similarity calculating means for calculating the similarity between the signal sequence included in the search window of the skip candidate position and the similarity of the reference signal sequence, the end position of the search window at the current position, and the skip A maximum value calculating means for calculating a maximum value of the similarity between the range from the current position to the skip candidate position and the reference signal sequence based on the length of the candidate position to the top position of the search window; If a large value does not exceed the threshold value is a signal search apparatus characterized by comprising, a skip width calculation means for setting a next current location to the deletion candidate position.

  According to the present invention, it is possible to realize a high-speed signal search by expanding the signal search unnecessary range as compared with the conventional case.

  Hereinafter, the signal search apparatus 100 according to the embodiment of the present invention will be described.

(First embodiment)
A signal search device 100 according to a first embodiment will be described with reference to FIGS. In the present embodiment, an example in which a 15-second CM video is searched from a 2-hour video recording a television program will be described. The reference signal and the input signal are video signals including 30 frame images per second.

(1) Configuration of Signal Search Device 100 FIG. 1 is a block diagram showing a signal search device 100 according to the present embodiment.

  The signal search apparatus 100 receives a reference signal 101 as a search key and calculates a feature value by a reference signal feature value calculation unit 104 and an input signal feature value calculation unit that calculates a feature value of the input signal 102 to be searched. 105, a similarity calculation unit 106 that calculates the similarity between the feature amount of the reference signal 101 and the feature amount of the input signal 102, and a correlation skippable width calculation unit 107 that calculates a skippable maximum value based on the similarity. A skip width calculation unit 108 that determines a width that is actually skipped based on the skippable width, a backtrack determination unit 109 that determines whether it is necessary to go back from the similarity of the skipped search window, Consists of

  The signal search apparatus 100 can be realized by using, for example, a general-purpose computer apparatus as basic hardware.

  For example, the reference signal feature value calculation unit 104, the input signal feature value calculation unit 105, the similarity calculation unit 106, the correlation skippable width calculation unit 107, the skip width calculation unit 108, and the backtrack determination unit 109 are the above computers. This can be realized by causing a processor mounted on the apparatus to execute a program.

  At this time, the signal search apparatus 100 may be realized by installing the above program in a computer device in advance, or may be stored in a storage medium such as a CD-ROM or distributed through the network. Then, this program may be realized by appropriately installing it in a computer device. Further, B and C are realized by appropriately using a memory, a hard disk or a storage medium such as a CD-R, a CD-RW, a DVD-RAM, a DVD-R, etc., incorporated in or externally attached to the computer device. Can do.

  Although the skippable width is directly used as the skip width in Patent Document 2, the correlation skippable width calculation unit 107 and the skip width calculation unit 107 calculate the skip width with the smallest backtrack from the skippable width in this embodiment. 108 is provided. Also, this embodiment is different from Patent Document 2 in the method for calculating the skippable width.

(2) Operation of Signal Search Device 100 Next, the operation of the signal search device 100 will be described using the flowcharts shown in FIGS. 1 and 2.

(2-1) Calculation of similarity of current position in input signal 102 First, the reference signal 101 and the input signal 102 are acquired (step S200).

  The reference signal feature value calculation unit 104 calculates the feature value of the reference signal 101 (step S201). Here, the reference signal 101 is a CM image of 15 seconds. For each frame screen constituting the video of the reference signal 101, the feature amount is calculated by dividing the screen into 16 and averaging the luminance in 256 bins (that is, 256 sections). 450 16-dimensional vectors are obtained as feature quantities. Further, a 24-bin histogram is calculated from these feature amounts.

  The input signal 102 is a video, similar to the reference signal 101, and is a 2-hour video recording a television program. The current position is set to the head of the input signal 102 in the input signal 102 and is initialized (step S202). In the present embodiment, the similarity with the reference signal 101 is calculated while gradually shifting the current position from the first frame to the last frame in the input signal 102, and when the similarity is equal to or greater than a predetermined threshold value, It is detected that the signal is the same as the reference signal 101.

  It is checked whether or not the current position (that is, the current frame position) is the end of the input signal 102 (step S203), and in the case of the end frame, the search process is terminated. Otherwise, the feature quantity at the current position is calculated by the input signal feature quantity calculator 105, and the similarity with the reference signal 101 is calculated by the similarity calculator 106 (step S204). Similar to the calculation of the feature value of the reference signal 101, the feature value of the current position is set to the same length as the reference signal 101 from the current position, that is, a search window of 15 seconds, and the feature value of the frame image included therein And a histogram is calculated.

  The similarity between the histogram of the reference signal 101 and the signal included in the current position search window (hereinafter referred to as the current position similarity) is the expression shown in Patent Document 1 (see Expression (2) below). Similarly, the minimum value is taken for each bin of the histogram, and the sum is calculated by a distance measure called a histogram intersection obtained by dividing the sum by the total frequency of the histogram. The histogram intersection has the property that the more similar the signals, the larger the similar.

  It is checked whether or not the similarity of the current position is greater than or equal to a predetermined threshold value (step S205). If the similarity is greater than or equal to the threshold value, the signal included in the current position search window and the reference signal 101 are regarded as matching. For example, the numerical value of the current position is displayed on the screen, and the search result is output (step S206). If not, the process proceeds to step S207 to shift the current position forward.

(2-2) Calculation of skippable width The current position is not advanced one by one (that is, not advanced one frame at a time), but the skip candidate position (that is, the frame position of the candidate to be skipped) is added in addition to the current position. The speed is increased by calculating a range that does not require a search from the similarity of the current position and the similarity of the skip candidate position, and skipping the signal collation in this range. Unlike the conventional method, the calculation method of the search unnecessary range (that is, skippable width) in the present embodiment is the most characteristic point of the present embodiment.

  FIG. 3 is a diagram for explaining the principle of calculating the skip width. FIG. 3-300 is an explanatory diagram of Patent Document 1, FIG. 3-310 is an explanatory diagram of Patent Document 2, and FIG. It is explanatory drawing of a form.

(2-3) Calculation of Skippable Width in Patent Document 1 FIG. 3-300 shows the skip principle of Patent Document 1.

  In Patent Document 1, if the similarity of the search window 301 is So, even if all the parts of the search window shifted forward match, the sum of the similarity So and the increase in similarity due to the shift is The threshold value θ is not exceeded. To explain further, having a similarity equal to the threshold value θ is a portion where the search window 302 and the input signal completely coincide, that is, a position where a 15-second CM image can be found, and the search is performed. This threshold value θ is not exceeded in the process. For this reason, this concept is used to omit collation during this period. This skip width w1 is given by equation (1).

w1 = floor (L (θ−So)) + 1 So <θ
w1 = 1 otherwise
... (1)

Where L is the width of the search window (the number of frames in the search window), the function floor is a function that represents truncation of decimals, So is the similarity of the current position (similarity at the current frame position), and θ is the similarity of the threshold Indicates. Note that θ is 1 when it completely matches, but 0 <θ <1 because it does not completely match in the course of disturbance or digitization. For example, θ = 0.8.

(2-4) Skippable Width in Patent Document 2 (2-4-1) Description Based on FIG. 3-310 FIG. 3-310 shows the skip principle of Patent Document 2.

  In Patent Document 2, as in Patent Document 1, the skippable width of the search window 311 at the current position is calculated as in the search window 312.

  Next, the backward skip width of the search window 314 for the skip candidate position is similarly calculated as in the search window 314, and when these skip widths overlap, collation from the current position to the skip candidate position is omitted. If the skip widths do not overlap, backtracking is performed to return the skip candidate position backward, and the determination is again made.

  The determination of whether or not the skip widths overlap is given by equation (2).

Tp−To <floor (L (2θ−So−Sp)) (2)

Where To is the current position, Tp is the skip candidate position, and Sp is the similarity of the skip candidate position. Therefore, the maximum value w2 of the skip width is expressed by Expression (3).

w2 = floor (L (2θ−So−Sp)) (3)

(2-4-2) Description Based on FIG. 3-330 Next, Patent Document 2 will be described with reference to FIG. This figure shows the same principle as that shown in FIG.

  In Patent Document 2, first, a point 331 having a current position time To and a similarity So, and a similarity Sp point 332 having a time Tp at a skip candidate position are obtained. When the width of the search window is L, the similarity change rate does not exceed 1 / L. Therefore, the maximum value of the similarity from the point 331 can be expressed as a straight line 341, and the point 333 where the similarity is equal to θ is the skip position shown in the equation (1). When the similarity decreases most rapidly from this point 333, it can be expressed as a straight line 342. When the similarity of the point 334 at the time Tp at which the straight line 342 is a skip candidate position exceeds the similarity of the point 332, it can be seen that the similarity does not exceed θ between To and Tp.

  To explain this in another way, since the rate of change in similarity does not exceed 1 / L even when the search window is moved backward, the maximum of the similarity in the rear from the point 332 representing the similarity of the skip candidate position The value can be expressed as a straight line 343 and can be skipped backward to a point 335 where the similarity is equal to θ. If the point 335 is behind the point 333, the forward skip width of the current position search window overlaps the backward skip width of the skip candidate position search window, and collation is performed in all sections from the current position to the skip candidate position. Can be omitted. This description is the same as the principle shown in FIG.

(2-5) Technology of this Embodiment (2-5-1) Description Based on FIG. 3-320 FIG. 3-320 shows the principle of this embodiment. It is characterized in that a wider skippable width than that of Patent Document 2 can be calculated by considering the correlation between the similarity of the current position and the similarity of the skip candidate positions.

  In FIG. 3-310, when backtracking is performed and the skippable width of the search window 312 at the skip candidate position is calculated, the similarity of the search window 311 does not exceed So. Therefore, there is a restriction that the search window 314 cannot be shifted to the search window 313. However, in Patent Document 2, since the skip width is calculated independently for the front and the rear, the above restriction is not taken into consideration at all.

  Therefore, in the present embodiment, the similarity at the current position and the similarity at the skip candidate position are considered simultaneously. The lengths of the current position search window 321 and the skip candidate position search window 322 are defined as d. That is, the number of frames from the last frame position of the current position search window to the first frame position of the skip candidate position search window is defined as d.

  Even if the length d between the search window 321 and the search window 322 all contributes to the increase in similarity, the similarity of the search window 323 is similar to the similarity So of the search window 321 at the current position and the search window 322 of skip candidate positions. The similarity Sq and the sum of d / L similarities are not exceeded. Therefore, if the total similarity does not exceed the threshold θ (0 <= θ <= 1), the collation between the current position and the skip candidate position is omitted.

  This determination is given by equation (4).

Tq−To <floor (L (1 + θ−So−Sq)) (4)

Here, the derivation process for Equation (4) will be described.

  First, the length d from the end position To of the current position search window 321 to the start position Tq of the skip candidate position search window 322 is d = Tq−To.

  Next, the maximum value S of the similarity between the range from the current position to the skip candidate position and the reference signal is calculated as follows.

Maximum similarity S = d / L + So + Sq
= (Tq-To) / L-1 + So + Sq

When the maximum value S <threshold value θ, the next current position can be set as the skip candidate position.

That is,

S = (Tq−To) / L−1 + So + Sq <θ

It is. If this is transformed,

d = Tq−To <L (1 + θ−So−Sq)

It becomes. Since the length d is an integer obtained by sampling, it is necessary to convert the right side into an integer, so that the function floor is used to obtain the equation (4).

Therefore, the maximum value w3 of the skippable width in this embodiment is

w3 = floor (L (1 + θ−So−Sq)) (5)

It becomes.

  The magnitude relationship between the expressions (1), (3), and (5) is given by the expression (6). However, the equal sign holds only when θ = 1. Therefore, the skippable width w3 of this embodiment is the largest.

w1 <w2 <w3 (6)

Also, the present embodiment and Patent Document 2 are different in the skip position, that is, the skip candidate position setting method. In Patent Document 2, the skip candidate position is set to the maximum skip width obtained by Expression (3), but in this embodiment, the skip candidate position is set slightly before the maximum skip width to reduce the risk of backtracking. . For example, assuming that the similarity of skip candidate positions follows a normal distribution, a position where the product of the distribution function of the probability of success in the determination of Expression (5) and the skip width is maximized is determined as the skip width. Alternatively, a similar effect is obtained by conservatively calculating the similarity when calculating the similarity of the skip candidate position.

(2-5-2) Description Based on FIG. 3-310 Next, the present embodiment will be described with reference to FIG.

  Points 331 to 333 are the same as in Patent Document 2, but when Equation (4) is the equal sign and has the maximum skip width, similar to the above, depending on the correlation between the search window 321 and the search window 322 as described above. The degree does not increase with θ. Thereafter, when the position of the search window 323 reaches the point 336, the maximum similarity decreases at a rate of change 1 / L as a straight line 344. When the similarity Sq of the skip candidate position Tq represented by the point 337 does not exceed the maximum similarity of the straight line 344 at Tq, it can be ensured that the similarity does not exceed θ in the entire range from To to Tq. Note that if the similarity does not exceed θ in the entire range from To to Tq, it means that there is no matching video in the range, so that range may be skipped. Similar to Patent Literature 2, the similarity of the skip candidate position may be calculated as an intersection of the straight line 344 and the similarity estimation straight line or curve.

(2-6) Calculation of Feature Value of Skip Candidate Position As described above, in this embodiment, the similarity of skip candidate positions is linearly predicted from the history of search windows so far, for example, as in Patent Document 2. Thus, the similarity is estimated (step S207), the correlation skippable width calculation unit 107 calculates the maximum skip width by the equation (5) (step S208), and the maximum skip is performed so as to reduce the risk of backtracking. The skip candidate position is set by calculating the skip width from the width by the skip width 108 (step S209).

  The input signal feature value calculation unit 105 calculates the feature value of the skip candidate position, and the similarity calculation unit 106 calculates the similarity to the reference signal (step S210).

  The backtrack determination unit 109 determines whether the backtrack is necessary using the equation (4) (step S211). If necessary, the skip candidate position is set back and reset, and the process returns to step S210 to determine again. Otherwise, since it can be ensured that the similarity does not exceed the threshold between the current position and the skip candidate position, the skip position is replaced with the current position and the current position is advanced (step S212). Returning to S205, the same collation is repeated for the new current position.

  As described above, the current position is advanced, and when the current position reaches the end of the input signal 102, the process ends.

(3) Effect FIG. 4 is a diagram showing an example of a difference in execution between the present embodiment and the background art.

  A graph 400, a graph 401, and a graph 402 show an example of the state of execution according to Patent Document 1, Patent Document 2, and this embodiment, respectively.

  The horizontal axis represents the time in the input signal 102, the vertical axis represents the similarity, and the bar graph is displayed only at the collated portion. It can be seen that the one according to this embodiment has the largest skip width.

  FIG. 5 is a diagram showing an example of the difference in the number of comparisons between the present embodiment and the background art.

  This is a result of searching a reference image 101 made up of 40 frame images from an input video 102 made up of 6000 frame images.

  The bar graph represents the number of collations, and the line graph represents the number of backtracks. A bar graph 500 and a point 504 indicate a full search, a bar graph 501 and a point 505 indicate Patent Document 1, a bar graph 502 and 506 indicate a Patent Document 2, and a bar graph 503 and a point 507 indicate the number of comparisons and the number of backtracks according to this embodiment. As described above, this embodiment can greatly reduce the number of collations. It can be seen that the number of backtracks has not increased dramatically, and the overall processing time can be reduced.

  Thus, according to the signal search device 100 according to the present embodiment, it is possible to skip more widely by calculating the skip width in consideration of the mutual relationship between the current position and the skip candidate position.

(4) Modifications Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  For example, although the present embodiment is used for video search, it can be applied to any signal search, for example, audio signal search, DNA signal search, and the like.

It is a block diagram which shows the structure of the signal search apparatus 100 concerning this embodiment. It is a flowchart which shows operation | movement of this embodiment. It is a figure for demonstrating the principle which calculates a skip width | variety in this embodiment. It is the figure which showed an example of the difference of execution with this embodiment and the conventional method. It is the figure which showed an example of the difference in the frequency | count of collation with this embodiment and the conventional method.

Explanation of symbols

101 Reference signal 102 Input signal 104 Reference signal feature value calculation unit 105 Input signal feature value calculation unit 106 Similarity calculation unit 107 Correlation skippable width calculation unit 108 Skip width calculation unit 109 Backtrack determination unit

Claims (15)

In a signal search device that searches for a signal sequence similar to a reference signal sequence while skipping over the input signal sequence,
Current position setting means for setting a search window at the current search position on the input signal sequence;
Skip candidate position setting means for setting a search window at a skip candidate position that is a candidate for the next search position on the input signal sequence;
Similarity calculation for calculating the similarity between the signal sequence included in the current position search window and the reference signal sequence, and the similarity between the signal sequence included in the skip candidate position search window and the reference signal sequence, respectively. Means,
Based on the length from the end position of the search window of the current position to the start position of the search window of the skip candidate position, the similarity between the range from the current position to the skip candidate position and the reference signal sequence A maximum value calculating means for calculating the maximum value of
If the maximum value does not exceed a threshold value, a skip width calculating means for setting a next current position as the skip candidate position;
A signal search apparatus comprising: The maximum value calculating means includes
(1) The product of the similarity of the current position and the length of the search window,
(2) The product of the similarity of the skip candidate position and the length of the search window,
(3) a length from the end of the search window at the current position to the start of the search window at the skip candidate position;
The signal search apparatus according to claim 1, wherein a sum of these (1), (2), and (3) is set as the maximum value. The maximum value calculating means calculates the maximum value S,

S = d / L + So + Sq = (Tq−To) / L−1 + So + Sq

Sought by
The skip width calculation means sets a next current position to the skip candidate position when S <θ,
Where Tq is the skip candidate position on the input signal sequence, To is the current position on the input signal sequence, and d is the end position of the search window for the current position and the start position of the search window for the skip candidate position. , Θ is the threshold (0 <= θ <= 1), So is the similarity of the current position (0 <= So <= 1), and Sq is the similarity of the skip candidate position (0 <= Sq <= 1) The signal search device according to claim 1, wherein: The signal search apparatus according to claim 1, wherein the skip width calculation unit reverts the skip candidate position to the current position side when the maximum value exceeds a threshold value. The signal search apparatus according to claim 1, wherein the search is performed for the entire range of the input signal sequence. In a signal search method for searching for a signal sequence similar to a reference signal sequence while skipping over the input signal sequence,
Set a search window at the current search position on the input signal sequence;
Set a search window at a skip candidate position that is a candidate for the next search position on the input signal sequence,
Calculating the similarity between the signal sequence included in the search window at the current position and the reference signal sequence, and the similarity between the signal sequence included in the search window at the skip candidate position and the reference signal sequence,
Based on the length from the end position of the search window for the current position to the start position of the search window for the skip candidate position, the similarity between the range from the current position to the skip candidate position and the reference signal sequence The maximum value of
If the maximum value does not exceed a threshold value, a next current position is set as the skip candidate position. When calculating the maximum value,
(1) The product of the similarity of the current position and the length of the search window,
(2) The product of the similarity of the skip candidate position and the length of the search window,
(3) a length from the end of the search window at the current position to the start of the search window at the skip candidate position;
The signal search method according to claim 6, wherein a sum of these (1), (2), and (3) is set as the maximum value. When calculating the maximum value, the maximum value S is

S = d / L + So + Sq = (Tq−To) / L−1 + So + Sq

Sought by
The next current position is set to the skip candidate position when S <θ.
Where Tq is the skip candidate position on the input signal sequence, To is the current position on the input signal sequence, and d is the end position of the search window for the current position and the start position of the search window for the skip candidate position. , Θ is the threshold (0 <= θ <= 1), So is the similarity of the current position (0 <= So <= 1), and Sq is the similarity of the skip candidate position (0 <= Sq The signal search method according to claim 6, wherein <= 1). The signal search method according to claim 6, wherein, when the maximum value exceeds a threshold value, the skip candidate position is returned to the current position side. The signal search method according to claim 6, wherein the search is performed for the entire range of the input signal sequence. In a signal search program for searching a signal sequence similar to a reference signal sequence by a computer while skipping over the input signal sequence,
A current position setting function for setting a search window at the current search position on the input signal sequence;
A skip candidate position setting function for setting a search window at a skip candidate position that is a candidate for the next search position on the input signal sequence;
Similarity calculation for calculating the similarity between the signal sequence included in the current position search window and the reference signal sequence, and the similarity between the signal sequence included in the skip candidate position search window and the reference signal sequence, respectively. Function and
Based on the length from the end position of the search window for the current position to the start position of the search window for the skip candidate position, the similarity between the range from the current position to the skip candidate position and the reference signal sequence Maximum value calculation function to calculate the maximum value of
When the maximum value does not exceed a threshold value, a skip width calculation function for setting a next current position as the skip candidate position;
The signal search program characterized by comprising. The maximum value calculation function is:
(1) The product of the similarity of the current position and the length of the search window,
(2) The product of the similarity of the skip candidate position and the length of the search window,
(3) a length from the end of the search window at the current position to the start of the search window at the skip candidate position;
The signal search program according to claim 1, wherein a sum of these (1), (2), and (3) is set as the maximum value. The maximum similarity calculation function calculates the maximum value S as

S = d / L + So + Sq = (Tq−To) / L−1 + So + Sq

Sought by
The skip width calculation function sets a next current position to the skip candidate position when S <θ,
Where Tq is the skip candidate position on the input signal sequence, To is the current position on the input signal sequence, and d is the end position of the search window for the current position and the start position of the search window for the skip candidate position. , Θ is the threshold (0 <= θ <= 1), So is the similarity of the current position (0 <= So <= 1), and Sq is the similarity of the skip candidate position (0 <= Sq The signal search program according to claim 11, wherein <= 1). The signal search program according to claim 1, wherein the skip width calculation function causes the skip candidate position to be returned to the current position side when the maximum value exceeds a threshold value. The signal search program according to claim 11, wherein the search is performed for the entire range of the input signal sequence.

Images