JP2010218022A - Signal search device, signal search method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a signal search device for increasing processing speed of enlarging or reducing a search window, and for theoretically causing no search omission. <P>SOLUTION: A first search window and a second search window of certain sizes are arranged in a region on an input signal. Featured values of each search window are extracted. Similarity between the search windows is calculated by using the featured values. When the similarity is less than a predetermined threshold, the size of the search window and the position of the region are output. A scaling factor for enlarging or reducing the size of the search window is calculated so that the similarity can be prevented from becoming less than the threshold. The size of each search window is enlarged or reduced by the scaling factor. When the size of the enlarged/reduced search window becomes the predetermined size, the processing ends. When the enlarged/reduced size of the search window does not become the predetermined size, the processing is repeated. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a signal search device, a signal search method, and a program for an input signal.

In Patent Document 1, with respect to template image matching, an upper limit of similarity around a template image is obtained by using the property that the similarity does not change abruptly even if the template image is moved, and a threshold θ given in advance as the similarity is obtained. The collation was speeded up by omitting the collation of the area below. This method is called active search.
Further, in Patent Document 2, regarding the feature extraction processing using the similarity between two windows, the position of the window for which the similarity is calculated using the property that the similarity does not change abruptly even if the window is moved. The feature extraction process is speeded up by omitting the matching process at a position where the degree of similarity is not clearly less than a predetermined threshold value θ.

Japanese Patent No. 3474131 JP 2008-65265 A

  In the process of calculating the similarity between two windows while expanding and contracting, the window size was changed by an empirical magnification such as 1.25 times. For this reason, there is a problem that a search leak occurs in principle.

  Then, it is necessary to change the magnification according to the nature of the data. For example, when detecting lines with multiple angles using image edge detection or detecting faces with multiple angles using face detection, it is necessary to operate multiple classifiers in parallel. Was demanded.

  Therefore, in view of the above problems, the present invention provides a signal search device, a signal search method, and a program that can speed up the processing for expansion and contraction of the search window and do not cause a search omission.

  In the present invention, an attention area extraction unit that arranges a first search window and a second search window of arbitrary sizes in an area on an input signal, and the two feature quantities of the search windows are similar to each other. A feature extraction unit that extracts feature values that increase in value, a similarity calculation unit that calculates a similarity between the search windows using the feature amounts, and the similarity is smaller than a predetermined threshold value In this case, the result output unit that outputs the size of the search window and the position of the region, and the expansion / contraction that calculates the expansion / contraction rate for expanding / contracting the search window size so that the similarity cannot fall below the threshold value. The size of each search window is expanded / contracted by the rate calculation unit and the expansion / contraction rate. (1) When the expanded search window reaches a predetermined size, the signal search process is terminated, (2 ) The size of the expanded search window is the predetermined size. If not, output the size of the search window expanded and contracted to the feature extraction unit, the extraction process of the feature extraction unit, the calculation process of the similarity calculation unit, the output process of the result output unit, And a repetition control unit that performs control so that the calculation process of the expansion / contraction rate calculation unit is repeated.

  According to the present invention, the processing can be speeded up with respect to expansion and contraction of the search window, and no search omission occurs.

It is explanatory drawing of the search window used by the image edge detection by Example 1 of this invention. It is explanatory drawing of the operation | movement performed by the image edge detection by Example 1 of this invention. It is a block diagram which shows the structure of the signal search apparatus concerning Example 1 of this invention. It is a flowchart which shows operation | movement of Example 1 of this invention. It is a figure for demonstrating an example that it is clear that the edge on the right side is not detected even if the search window is enlarged. It is a figure for demonstrating the calculation method of the expansion-contraction rate minimum at the time of search window reduction. It is a figure for demonstrating the calculation method of the expansion-contraction rate upper limit at the time of search window expansion. It is a figure for demonstrating a mode that it collates, changing an expansion / contraction rate by the conventional method. It is a figure for demonstrating a mode that it collates, changing the expansion / contraction rate by Example 1 of this invention. It is a figure for demonstrating the example of the search window which can calculate a similarity degree similarly to Example 1. FIG. It is a figure for demonstrating the example of the search window from which an area ratio differs. It is a figure for demonstrating the example of the search window of a complicated shape. It is a figure for demonstrating the calculation method of the expansion-contraction rate upper limit at the time of search window reduction | decrease in shot boundary detection. It is a figure for demonstrating the calculation method of the expansion-contraction rate upper limit at the time of the search window expansion in shot boundary detection. It is a figure for demonstrating the Haar-based feature used with the face detection method of Viola-Jones et al. It is a figure for demonstrating operation | movement of the face detection method of Viola-Jones et al. It is a figure for demonstrating the example of the positional relationship of a search frame and a search window. It is a figure for demonstrating the difference in the position of the search window within a search frame. It is a figure for demonstrating changing the search range area | region on an input image with the difference in the position of the search window within a search frame. It is a figure which shows the hardware constitutions of the signal search apparatus which concerns on the Example of this invention.

  Hereinafter, a signal search device according to an embodiment of the present invention will be described with reference to the drawings. The signal search apparatus according to the embodiment of the present invention includes a voice change detection of a voice signal, a shot boundary detection of a video signal of a video, a text topic division of a character string signal, an image edge detection of an image signal, a face detection of an image signal, It can be applied to partial image boundary detection of spatial image signals, three-dimensional boundary detection of three-dimensional volume data, and the like.

  Further, the similarity used in the first embodiment, the first and second modified examples of the first embodiment, the second and fourth embodiments, the second embodiment, and the third embodiment is expressed as a distance that decreases as the comparison target is closer. The similarities used in the description of the histogram overlap ratio in the modified examples 3 and 5 are expressed as values that increase as the comparison target is closer. Therefore, the similarity used in the first embodiment, the first modification example, the second modification example, the fourth modification example, and the second embodiment and the third embodiment is the state in which the comparison target is most similar when the similarity is minimized. In addition, the similarity used in the description of the histogram overlap ratio in the modified example 3 and modified example 5 of the first embodiment is the state in which the comparison target is most similar when it reaches the maximum.

  A signal search apparatus according to a first embodiment of the present invention will be described with reference to FIGS.

  In this embodiment, the signal search device performs signal search processing for edge detection in the vertical direction included in the input image. However, as shown in FIG. 5, the entire left side of the input image has a gradation that gradually becomes darker. Regardless, the description will be made on the assumption that the right search window searches for a darker edge than the left search window.

(1) Configuration of Signal Search Device The signal search device according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram of the signal search apparatus according to the present embodiment.

  The signal search apparatus includes a signal input unit 101, a detector 102, an expansion / contraction rate calculation unit 103, and a similarity selection unit 104.

  The signal input unit 101 inputs and stores an image signal.

  The detector 102 scans the search window for all positions on the image, and determines the position, size, and similarity of the search window when the similarity is below a predetermined threshold θ (0 ≦ θ ≦ 1). Output.

  The similarity selection unit 103 inputs the similarity output from the detector 102 for the search window of the same size, and outputs the minimum similarity among them.

  The expansion / contraction rate calculation unit 104 calculates an expansion / contraction rate for reducing the search window based on the minimum similarity obtained by the similarity selection unit 103. Further, the expansion / contraction rate calculation unit 104 first performs the above processing with the maximum search window size βmax, and repeats the processing while reducing the search window with the obtained expansion / contraction rate, and the maximum search window size βmin or less. It also functions as a repetitive control unit that performs control so that it is performed until.

  The detector 102 includes an attention area extraction unit 105, a feature extraction unit 106, a similarity calculation unit 107, and a partial result output unit 108.

  The attention area extraction unit 105 sets two types of search windows at a certain position on the image stored in the signal input unit 101, that is, the first search window and the second search window, Output the pixels contained in it. The search window has a size specified by the expansion / contraction rate calculation unit 103.

  The feature extraction unit 106 calculates the average luminance of the images in the first search window and the second search window and extracts the feature amounts.

  The similarity calculation unit 107 calculates the similarity between the first search window and the second search window.

  The result output unit 108 outputs the position, size, and similarity of the search window when the similarity calculated by the similarity calculation unit 107 is below a predetermined threshold value θ (0 ≦ θ ≦ 1).

  As shown in FIG. 20, the signal search apparatus includes a control unit 11 such as a CPU (Central Processing Unit) that controls the entire apparatus, a ROM (Read Only Memory) 14 and a RAM 15 (Random Access Memory) that store various data and various programs. ), An external storage unit 16 such as an HDD (Hard Disk Drive) or a CD (Compact Disk) drive device for storing various data and various programs, and a bus 17 for connecting them. It has a hardware configuration using a computer. In addition, the signal search device includes a display unit 13 for displaying information, an operation unit 12 such as a keyboard and a mouse for receiving user instruction inputs, and a communication I / F (interface) 18 for controlling communication with an external device. Each is connected by wire or wireless.

  Note that this signal search device can also be realized, for example, by using a general-purpose computer device as basic hardware. That is, the signal input unit 101, the detector 102, the similarity selection unit 103, the expansion / contraction rate calculation unit 104, the attention area extraction unit 105, the feature extraction unit 106, the similarity calculation unit 107, and the result output unit 108 are connected to the above computer. This can be realized by causing the installed processor to execute the program. At this time, the signal search apparatus may be realized by installing the above program in a computer in advance, or may be stored in a storage medium such as a CD-ROM or distributed through the network. This program may be realized by appropriately installing it on a computer. The storage unit provided in the signal search apparatus appropriately uses a memory, a hard disk or a storage medium such as a CD-R, a CD-RW, a DVD-RAM, a DVD-R, or the like incorporated in or externally attached to the computer. Can be realized.

(2) Description of Search Window FIGS. 1A to 1H are a first search window and a second search window used for image edge detection in the signal search apparatus according to the present embodiment.

  In FIG. 1a, the first search window w1 and the second search window w2 included in the attention area extraction unit 105 are arranged adjacent to each other on the left and right on the image to detect a lateral edge. FIGS. 1b to 1h represent search windows for detecting edges obtained by slightly rotating FIG. 1a in the clockwise direction.

  Assuming that the input image is a gray scale image, the average luminance values of the search windows w1 and w2 are L1 and L2, and the luminance difference value that is the difference value of the average luminance values of the search windows w1 and w2 is D, D = L1-L2 It becomes. However, 0 ≦ L1 ≦ 1, 0 ≦ L2 ≦ 1, and 0 ≦ D ≦ 1. The luminance takes a value from 0 to 1, with 0 being the brightest and 1 being the darkest. When D> θ, the position is detected as an edge. However, 0 ≦ θ ≦ 1.

  The detector 102 scans the input image and detects vertical edges using the set of search windows of FIG. 1a. Note that the luminance difference value D means the distance. That is, the distance difference value D is the reciprocal of the similarity, and the similarity decreases as the luminance difference value increases.

  As shown in FIG. 2, the signal search apparatus first performs the above processing with the search window βmax having the maximum size, and repeats the processing while reducing the search window, so that the search window βmin becomes smaller than the minimum size. Do until.

(3) Example of speeding up by omitting collation when there is clearly no edge in the image FIG. 5 is a diagram for explaining an example in which it is clear that no edge is detected even if the search window is somewhat reduced. is there.

  It is assumed that the left first search window w1 is for detecting a brighter edge than the right second search window w2. On the contrary, the input image is a gradation image that is brighter on the right side than on the left side. In such a case, there is a problem that the luminance difference value does not change greatly even if each search window is somewhat reduced, and an edge is not detected.

(4) Method for Calculating Search Window Reduction Ratio FIG. 6 is a diagram for explaining a method for calculating a search window reduction ratio in this embodiment.

  As described above, the detector 102 is for detecting edges that are brighter in the first search window w1 than in the second search window w2. If the luminance values of the left and right search windows before reduction are L1 and L2, the luminance difference value is D = L1−L2. However, D> 0.

  Assuming that the search windows w1 'and w2' are obtained by reducing the search windows w1 and w2, and the brightness values thereof are L1 'and L2', the brightness difference value is D '= L1'-L2'.

  Consider a case where the luminance difference value D ′ is reduced most by the reduction. The luminance of all the areas outside the range of the first search window due to the reduction in the first search window w1 ′ is 1, and the brightness of the second search window w2 ′ is out of the range of the second search window due to the reduction. When the brightness of all the regions is 0, the brightness difference value D ′ is the smallest. If the reduction ratio in this case is α, L1 ′, L2 ′, and D ′ can be expressed by the following equations.

L1 '= (L1 / 2- ( 1-α 2) / 2) / (α 2/2)
= (L1-1 + α ² ) / α ²
L2 ′ = L2 / α ²
D ′ = L1′−L2 ′
= (L1-L2-1 + α ² ) / α ²
= (D-1 + α ² ) / α ²

Solving D ′> θ leads to the following relationship.

α ² > (1-D) / (1-θ) (1)

If the reduction ratio α is within the range of Expression 1, an edge cannot be detected.

  When the reduction rate α is close to 1, the size of the search window hardly changes. Therefore, in order to avoid this, a lower limit Afix of the reduction rate is determined, and α = Afix is set when α <Afix. However, Afix is a fixed value, and 0 ≦ Afix ≦ 1.

  Let β (i) be the size of the current search window. However, i represents the number of repetition processes, and i ≧ 1.

  β (1) is initialized to βmax. The luminance difference value D is obtained at all positions on the image, and the position (x, y), the luminance difference value D, and the search window size β (i) are obtained for all positions (x, y) where D ≧ θ. ) Is output. However, the size of the image is x × y. Among them, the minimum luminance difference value Dmin is obtained.

  If the size of the next search window is β (i + 1), β (i + 1) can be expressed by Equation 2 below.

β (i + 1) = β (i) × α
= Β (i) × √ ((1-Dmin) / (1-θ)) (2)

The above process is repeated until the size of the search window becomes β (i) ≦ βmin.

(5) Conventional Method FIG. 8 shows a conventional method for searching while changing the expansion / contraction ratio of the search window by a fixed width.

  In the conventional method, even if the size of the search window shown in FIG. 5 is slightly changed, it is necessary to perform a search even when the luminance value does not clearly fall below the threshold value, which is wasteful. In addition, it is necessary to empirically change the change width depending on the type and nature of the signal.

(6) Method of the present embodiment FIG. 9 shows a state in which searching is performed by variably changing the expansion / contraction ratio according to the present embodiment.

  In the present embodiment, the search is advanced while skipping the size of the search window by omitting collation in the search window whose magnitude is clearly less than the threshold value θ given in advance, which is clearly obtained from Expression 2. .

  FIG. 4 shows a flowchart for explaining the above processing procedure.

  First, in step 400, the expansion / contraction rate calculation unit 104 initializes i to 1, the search window size β (i) to βmax, the expansion / contraction rate α to 1, and the minimum difference value Dmin to 0.

  In step 401, the expansion / contraction rate calculation unit 104 reduces the search window to α times.

  In step 402, the attention area extraction unit 105 arranges the search windows w1 and w2 at positions where the position (x, y) on the input image becomes the center of gravity of the first search window w1 and the second search window w1. . The feature extraction unit 106 calculates average luminance values L1 and L2 of the search windows w1 and w2, respectively. The similarity calculation unit 107 calculates the luminance difference value D between the search windows w1 and w2 from D = L1−L2. However, 0 ≦ L1 ≦ 1, 0 ≦ L2 ≦ 1, and 0 ≦ D ≦ 1.

  In step 403, if D> θ, the result output unit 108 detects the position (x, y) on the input image as an edge, the position (x, y) of the center of gravity of both search windows w1, w2, and the luminance. The difference value D and the search window size β (i) are output. However, 0 ≦ θ ≦ 1.

  In step 404, the similarity selection unit 103 updates the minimum luminance difference value Dmin = D when D <Dmin.

  In step 405, the attention area extraction unit 105, feature extraction unit 106, similarity calculation unit 107, result output unit 108, and similarity selection unit 103 return to step 402 for all positions on the input image. If all the processes are repeated, the process proceeds to step 406.

  In Step 406, the expansion / contraction rate calculation unit 104 calculates the expansion / contraction rate α using Equation 1 based on the threshold θ and the minimum luminance difference value Dmin. However, if the lower limit value of α is Afix, then α = Afix when α <Afix.

  In step 407, the iterative control unit that is also used by the expansion / contraction rate calculation unit 104 obtains and updates the size of each search window w1, w2 as β (i + 1) = β (i) × α, and increments i = i + 1. To do.

  In step 408, the repetitive control unit also serving as the expansion / contraction rate calculation unit 104 ends the process if the size of each search window β (i) <βmin, otherwise returns to step 201 to perform the detection process. Control to repeat.

(7) Modification 1
Further, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, constituent elements over different embodiments may be appropriately combined.

  Modification 1 will be described.

  In the first embodiment, the search windows w1 and w2 are reduced. However, even if the search windows w1 and w2 are enlarged, the expansion / contraction ratio of the enlarged width can be derived. The left and right search windows before enlargement are w1, w2, the luminance values are L1, L2, and the luminance difference value is D. The left and right search windows after enlargement are w1 ″, w2 ″, the luminance values are L1 ″ and L2 ″, and the luminance difference value is D ″ = L1 ″ −L2 ″.

  Consider a case in which the luminance difference value D ″ is reduced most by the enlargement. All the regions that are protruded by the enlargement with respect to the first search window w1 ″ are 0 and all of the regions that are protruded by the enlargement with respect to the second search window w2 ″. When the brightness of the area is 1, the brightness difference value D ″ is the smallest. Assuming that the enlargement ratio is α, L1 ″, L2 ″, and D ″ can be expressed by the following equations.

L1 "= L1 / α ²
L2 "= (L2 / 2 + (α 2 -1) / 2) / (α 2/2)
= (L2 + α ² -1) / α ²
D ″ = L1 ″ −L2 ″
= (L1-L2-α ² +1) / α ²
= (D-α ² +1) / α ²

Solving D ″> θ leads to the following relationship.

α ² <(1 + D) / (1 + θ) (3)

If the enlargement ratio α is within the range of Expression 3, an edge cannot be detected.

  Summarizing Formula 1 and Formula 3, the following formula can be derived.

(1-D) / (1-θ) <α ² <(1 + D) / (1 + θ) (4)

If the expansion / contraction rate α is within this range including reduction and enlargement, an edge cannot be detected.

  In the present specification, “extension / contraction” means enlargement or reduction.

(8) Modification 2
Modification 2 will be described.

  In the first embodiment, the minimum luminance difference value Dmin is obtained from the entire image, but the image may be divided into a plurality of parts and searched independently for each region. All positions may be regarded as one area.

  That is, in the first embodiment, even if one point, the luminance difference value D is less than the threshold value θ, it is not possible to skip. When the minimum luminance difference value Dmin is obtained for the entire image, the possibility increases, and the number of cases in which skipping is not possible increases. The image often changes uniformly in a minute region, and in that case, the luminance difference value D often takes a similar value. Therefore, if the image is divided into minute regions and the minimum luminance difference value Dmin is obtained, the possibility of skipping increases.

  The divided area may be one point (for example, one pixel). In this case, instead of obtaining the minimum luminance difference value Dmin in the area as in the first embodiment, the expansion / contraction rate is calculated from Equation 2 using the luminance difference value itself at that position.

(9) Modification 3
A third modification will be described.

  In Example 1, the luminance difference value between the search windows was obtained using the average value of the luminance in each of the search windows w1 and w2 as a feature amount. However, as long as the upper limit of the change in similarity can be calculated from the expansion / contraction amount of the size of the search window, any method for calculating the feature amount and the method for calculating the similarity may be used.

  For example, a histogram of luminance values in each search window may be used as a feature amount, and a histogram overlap rate between the search windows may be obtained as a similarity.

H ₁ = (h ₁₁ , h ₁₂ ,..., H _1n ), H 2 = (h ₂₁ , h ₂₂ ,..., H _2n , respectively, regarding the luminance values of the pixels included in the search windows w 1 and w _2. ). Here, n represents the number of pixels included in the search window, that is, the total frequency of the histogram. Assume that the similarity between the left and right search windows before and after reduction is S and S ′. The degree of similarity is obtained by a histogram overlap rate represented by the following equation.

  Consider the case where S decreases most due to reduction. S decreases most when all regions outside the search window range from the reduced search windows w1 'and w2' contribute to the increase in similarity. If the reduction ratio is α, the similarity S ′ can be expressed by the following equation.

S ′ = (S− (1−α ² ) / 2) / α ²
= (S-1 + α ² ) / α ²

When S ′ <θ is solved, the following formula 5 can be derived.

α ² > (1-2S) / (1-2θ) (5)

On the other hand, when the search windows w1 and w2 are enlarged, S decreases most when all the regions protruding from the enlarged search windows w1 ″ and w2 ″ do not contribute to the increase in similarity. If the enlargement ratio is α, the similarity S ″ after enlargement can be expressed by the following equation.

S ″ = (S− (α ² −1) / 2) / α ²

Solving S ″ <θ leads to the following equation (6).

α ² <(1 + 2S) / (1 + 2θ) (6)

Summarizing Formula 5 and Formula 6 gives Formula 7 below.

(1-2S) / (1-2θ) <α ² <(1 + 2S) / (1 + 2θ)

... (7)

If the expansion / contraction rate α of both search windows is within a range satisfying Equation 7, an edge cannot be detected.

Further, as an example of changing the similarity, instead of using the luminance difference value and the histogram overlap rate, an expression itself that can calculate the upper limit of the similarity change from the expansion / contraction amount of the search window size may be used. . Specifically, the formula a ^* may be used.

(10) Modification 4
In Example 1, the signal search process was speeded up using only the lower limit of the expansion / contraction rate at the time of reduction and only the upper limit of the expansion / contraction rate at the time of enlargement. In the fourth modification, the signal search process is speeded up using both of them. Similar to the first embodiment, a case where the search window W (i) is multiplied by α to be reduced to the search window W (i + 1) will be described. However, the search window W (i) represents a set of the search window w1 and the search window w2, and α <1.

  When the size of the search window W (i) is β (i) and the luminance difference value is D, when the search window W (i + 1) ′ smaller than the search window W (i) is set, the search window W (i Let us consider how the upper limit of the expansion / contraction rate αp can be expressed.

  Since the luminance difference value in the search window W (i) is D, it can be expressed by Expression 7-1 below from Expression 1.

α ² > (1-D) / (1-θ) (7-1)

On the other hand, assuming that the luminance difference value of the search window W (i + 1) ′ is the predicted luminance difference value D ′, the upper limit of the expansion / contraction rate viewed from the search window W (i + 1) ′ is as shown in the following equation 7-2 from the equation 3. It can be expressed as

αp ² <(1 + D ′) / (1 + θ) (7-2)

When the minimum size of the search window viewed from the search window W (i) is β (i) ′, it can be expressed as β (i) ′ = β (i) × α. If the maximum size of the search window viewed from the search window W (i + 1) is β ″ (i), it can be expressed as β (i) ″ = β (i + 1) / αp. If β (i) <β (i) ”, the expansion and contraction intervals that can be omitted are continuous. Solving this equation leads to the following equation 7-3.

β (i + 1) ′ <β (i) × α / αp
= Β (i) × √ ((1-D) (1 + θ) / (1 + D ′) (1-θ))
... (7-3)

Equation 7-1 is a range using the lower limit of the expansion / contraction rate, and Equation 7-2 is a range using the upper limit of the expansion / contraction rate. Since both of these can be used in Expression 7-3, only the lower limit of the expansion / contraction rate is used as in Example 1, or only the upper limit of the expansion / contraction rate is used as in Modification 1 of Example 1. Then, the range of expansion ratios that can be skipped is wide. However, the size β (i + 1) ′ of the search window W (i + 1) ′ needs to be selected to satisfy Expression 7-3.

  Therefore, first, the predicted luminance difference value D ′ is predicted, and the size β (i + 1) of the search window W (i + 1) is set to a value within the range satisfying Equation 7-3. Note that β (i + 1) is set near the minimum value that satisfies Equation 7-3 in order to increase the skippable expansion / contraction width as much as possible.

  For example, assuming that the luminance difference value remains unchanged from D to the search window W (i + 1), the expansion / contraction rate calculation unit 104 sets β (i + 1) as follows. However, Cp is a constant close to 1, such as 1.1.

β (i + 1) = Cp × β (i + 1) ′
= Cp × β (i) × √ ((1-D) (1 + θ) / (1 + D ′) (1-θ)) (7-4)

Next, the detector 102 searches for an edge using the search window W (i + 1) obtained by actually reducing the search window W (i) to β (i + 1), and the similarity selection unit 103 receives the actual minimum luminance difference value. Memorize Da. At this time, an undecided mark is attached to the result output from the result output unit 108.

  Next, the actual brightness difference value Da stored in the similarity selection unit 103 is substituted into the predicted brightness difference value D ′ expression 7-3 to calculate β (i + 1) ′, and the following expression 7-5 is satisfied. Whether or not the expansion / contraction rate calculation unit 104 checks.

β (i + 1)> β (i + 1) ′ (7-5)

When Expression 7-5 is satisfied, the determination result is added to the output result output from the result output unit 108 and the undetermined mark is added, and the search is advanced.

  If Equation 7-5 is not satisfied, β (i) ′ is multiplied by a fixed value Cb of 1 or more and β (i) / β (i) ′ to be close to β (i), and Equation 7-2 is re-established. A process to check whether it satisfies is performed. This process is “backtracking”. This backtracking is repeated until Expression 7-2 is satisfied.

  By repeating the above processing, it is possible to further speed up the signal search processing by using not only the lower limit or upper limit of the expansion / contraction rate but also both.

Note that the signal search process can be similarly performed for enlargement. That is,

β (i + 1) ′> β (i) × √ ((1 + D ′) (1-θ) / (1-D) (1 + θ))

... (7-6)

Repeat backtracking until it meets.

  Further, as the similarity, a histogram overlap rate may be used instead of the luminance difference value.

(11) Modification 5
Modification 5 will be described.

  In the fifth modification, a case where a search window having a complicated shape is used will be described.

  In the first embodiment, as shown in FIGS. 6 and 7, both search windows are set at the center of a square using a first search window and a second search window in which rectangles of the same size are adjacent to each other on the left and right. Stretched. However, as shown in FIGS. 10, 11, and 12, both search windows may have two or more regions. Hereinafter, modification examples will be described in order.

(11-1) First Modification of Search Window A first modification of the search window in a complicated case will be described with reference to FIG.

  Both search windows 1600 to 1604 shown in FIG. 10 are cases in which the areas of both search windows are divided by line segments extending radially from the center point, and the center point is a point object.

  The two search windows 1600 to 1604 are cases where they do not overlap each other even if the size of both search windows is expanded or contracted with respect to the center point, and the expansion / contraction ratio can be calculated using the same formula as in the first embodiment.

(11-2) Second Modified Example of Search Window A second modified example of the search window in a complicated case will be described with reference to FIG.

  The two search windows 1700 to 1704 shown in FIG. 11 are cases where the areas of both search windows are delimited by line segments extending radially from the center point, but are not point-symmetric with respect to the center point.

  Although the two search windows 1700 to 1704 do not overlap each other even if the search windows are expanded or contracted with respect to the center point, the areas of the areas of the search windows are different.

(11-2-1) In the case of average luminance Therefore, when using average luminance for the feature amount, it is converted to a value per unit area at the stage of calculating the average value of luminance. Thereby, even if the areas of the two search windows are different, the expansion / contraction rate can be calculated using the same formula as in the first embodiment.

(11-2-2) Case of Histogram Overlap Rate On the other hand, when the histogram overlap rate is used for the feature amount, the expression is different from that in the first embodiment. It is assumed that the area ratio of both search windows w1 and w2 is γ. When both the search windows w1 and w2 are reduced, the similarity before and after reduction is set to S and S ′. S decreases most when all the regions outside the range of the search window from both reduced search windows w1 ′ and w2 ′ contribute to the increase in similarity. The reduction ratio is α, ε = min (γ, 1−γ) / max (γ, 1−γ), min (•) is a function that outputs a minimum value, and max (•) is a function that outputs a maximum value. , S ′ can be expressed by the following equation.

S ′ = (S− (1−α ² ) / ε) / α ²
= (ΕS-1 + α ² ) / εα ²

However, since the areas between the regions of the search windows w1 and w2 are different, the similarity is ε at the maximum, and 0 ≦ S ≦ ε and 0 ≦ S ′ ≦ ε.

  When S ′ <θ is solved, the following equation 8 can be derived.

S ′> (1-εS) / (1-εθ) (8)

However, 0 ≦ θ ≦ ε.

  When both the search windows w1 and w2 are enlarged, S decreases most when all the areas protruding by the enlarged both search windows w1 "and w" do not contribute to the increase in similarity. Assuming that the enlargement ratio is α and the similarity after enlargement is S ″, S ″ can be expressed by the following equation.

S ″ = S / α ²

If S ″ <θ is solved, the following formula 9 can be derived.

α ² <S / θ (9)

Equation 8 and Equation 9 are summarized as follows.

(1-εS) / (1-εθ) <α ² <S / θ (10)

(11-3) Third Modified Example of Search Window A third modified example of the search window in a complicated case will be described with reference to FIG.

  Both search windows w1 and w2 shown in FIG. 12 are cases where the region is not divided by line segments extending radially from the center point and is not point-symmetric with respect to the center point.

  Both search windows w1 and w2 overlap each other when the size of both search windows is expanded or contracted with respect to the center point, and the areas of the areas of both search windows w1 and w2 are not necessarily the same.

  In the case of such search windows w1 and w2, it is necessary to increase or decrease the overlapping portions when calculating the upper limit of the similarity when the size of the search window is expanded or contracted.

  The method of increase / decrease varies depending on the shape of the search window and is not necessarily monotonically increased.However, since the relationship between the expansion ratio of the search window and the area of overlap is obvious, it is possible to calculate the upper limit of similarity, and the equation is Although different, the first embodiment can be applied.

(12) Modification 6
In the above embodiment, the expansion / contraction rate calculation unit 104 also serves as a repetition control unit. However, the present invention is not limited thereto, and may be provided separately.

  A signal search device according to a second embodiment of the present invention will be described with reference to FIGS.

  In the first embodiment, image edge detection has been described as an example. The image is a two-dimensional signal, but the present invention is not limited to this and can process a one-dimensional signal. In this embodiment, a one-dimensional signal will be described. In this embodiment, a description will be given by taking shot boundary detection of an image as an example.

  FIG. 13 shows a search window used in video shot boundary detection.

  The configuration of the signal search apparatus according to the present embodiment is the same as the configuration of the signal search apparatus according to the first embodiment shown in FIG. That is, it has a signal input unit 101, a detector 102, a similarity selection unit 103, and an expansion / contraction rate calculation unit 104.

  The signal input unit 101 inputs and stores a video signal (only an image portion).

  The detector 102 adjoins the first search window w1 and the second search window w2 having the same length on a time series signal in which feature amounts extracted from one frame image in the video are arranged. These are scanned on the signal sequence, and the position, size, and similarity of the search window are output when the similarity is below a predetermined threshold value θ (0 ≦ θ ≦ 1).

  The similarity selection unit 103 inputs the similarity output from the detector 102 for the search window of the same size, and outputs the minimum similarity among them.

  The expansion / contraction rate calculation unit 104 calculates the expansion / contraction rate of the size of the search window. Further, the expansion / contraction rate calculation unit 104 first performs the above processing with the maximum search window size βmax, and then repeats the processing while reducing the search window with the calculated expansion / contraction rate, thereby reducing the minimum search window size. It also functions as a repetitive control unit that repeats until βmin or less.

  The detector 102 includes an attention area extraction unit 105, a feature extraction unit 106, a similarity calculation unit 107, and a result output unit 108.

  The attention area extraction unit 105 arranges the first search window w1 and the second search window w2 having the same length adjacent to each other at a certain position on the time series signal, and frame images included in these search windows are arranged. Output. At this time, a search window having a size designated by the expansion / contraction rate calculation unit 103 is used.

  The feature extraction unit 106 calculates the average luminance for all the pixels on the frame image included in each of the first search window w1 and the second search window w2, and extracts feature amounts.

  The similarity calculation unit 107 calculates the similarity between the first search window w1 and the second search window w2.

  The result output unit 108 detects, as a shot boundary, a position where the luminance difference value is minimum when the similarity obtained by the similarity calculation unit 107 is lower than a predetermined threshold θ (0 ≦ θ ≦ 1). The search window position, size and similarity are output.

(1) Reduction Processing First, a grayscale image sequence is input to the signal input unit 101 as a video.

  Next, the feature extraction unit 106 extracts, as a feature amount, an average luminance value obtained by averaging all the luminance values of the pixels of the frame images included in the search windows w1 and w2.

  Next, the similarity calculation unit 107 obtains a luminance difference value between the search windows w1 and w2 as a distance. The distance is the reciprocal of the similarity.

  The concept of the operation of the expansion / contraction rate calculation unit 104 will be described.

  If the average luminance values of the search windows w1 and w2 before reduction are L1 and L2, and L1 ′ and L2 ′ are after reduction, the distance between the search windows w1 and w2 before and after reduction is D = L1−L2 and Ld ′. = L1'-L2 '.

  Here, an upper limit value of the distance when the distance D increases most due to the reduction is considered. If the brightness of all areas outside the search window range from the reduced search window w1 ′ is 0 and the brightness of all areas outside the search window range from the search window w2 ′ is 1, D is 1. Increase most. If the reduction ratio is α, D ′ can be expressed as follows.

L1 ′ = (L1-1 + α) / α
L2 ′ = L2 / α

Since D ′ = L1′−L2 ′, D ′ can be expressed by the following equation.

D ′ = (L1-L2-1 + α) / α

When D ′> θ is solved, the following equation 11 can be derived.

α> (1-L1 + L2) / (1-θ) (11)

(2) Enlarging Process In the above, both the search windows w1 and w2 are reduced, but even if each search window w1 and w2 is enlarged, the expansion / contraction rate of the expansion width can be derived.

  Consider a case where the distance D increases most due to the expansion of the search windows w1 and w2 when the distance D> θ and 0 ≦ θ ≦ 1. The distance D increases most when the luminance of all the regions protruding by L1 ″ is 0 and the luminance of all the regions protruding by L4 ′ is 1. When the magnification is α, L1 ″ L2 ″ and D ″ can be expressed as follows.

L1 "= L1 / α
L2 ″ = (L2 + α−1) / α
D ″ = L1 ″ −L2 ″
= (L1-L2-α + 1) / α
= (D + 1) / α

Solving Ld ′> θ with respect to α leads to the relationship of Equation 12 below.

α <(1 + D) / (1-θ) (12)

(3) Summary Formulas 11 and 12 are summarized as follows.

(1-D) / (1 + θ) <α <(1 + D) / (1-θ)

The processing procedure of the signal search apparatus according to the present embodiment is the same as the processing procedure of the signal search apparatus according to the first embodiment, and a description thereof is omitted.

(4) Modification 1
A first modification of the second embodiment will be described.

  In the second embodiment, the luminance difference value and the distance are used. However, as long as the upper limit of the change in the similarity can be calculated from the change in the size of the search window, any method for calculating the feature and the method for calculating the similarity can be used. good.

  For example, the average luminance of the frame images included in the search window may be obtained, the histogram of the average luminance value may be used as the feature amount, and the histogram overlap rate between them may be used as the similarity.

  Based on FIG. 13, a method of calculating the reduction ratio of the search windows w1 and w2 using this histogram overlap ratio will be described.

Histograms of feature amounts included in both search windows w1 and w2 are H ₁ = (h ₁₁ , h ₁₂ ,..., H _1n ) and H ₂ = (h ₂₁ , h ₂₂ ,..., H _2n ). To do. Here, n represents the number of feature amounts included in the search window, that is, the total frequency of the histogram. The similarity S between the two search windows w1 and w2 is calculated by the histogram overlap rate.

  The similarity before and after the reduction is S and S ′. Consider the case where the similarity is the smallest. S decreases most when the histograms of all the areas protruding by the reduction match. If the reduction ratio in this case is α, the relationship between S and S ′ can be expressed as follows.

S ′ = (S− (1−α)) / α
= (S-1 + α) / α

Solving S '> θ leads to the following equation (13)

α> (1-S) / (1-θ) (13)

In the case of enlarging the search window, assuming that the search window after enlargement is w1 ″, w2 ″, and the similarity luminance difference is S ″, S is the smallest when the histograms of all the areas protruding by enlargement do not match. Therefore, S ″ can be expressed as follows.

S ″ = S / α

When S ′> θ is solved, the following equation 14 can be derived.

α <S / θ (14)

Summarizing Equation 13 and Equation 14 gives Equation 15 below.

(1-S) / (1-θ) <α <S / θ (15)

(5) Modification 2
A second modification of the second embodiment is described.

  In the first embodiment, an image that is a two-dimensional signal is described, and in the second embodiment, an image sequence that is a one-dimensional signal is described. However, the present invention is not limited to this, and an n-dimensional signal can be processed. For example, a three-dimensional volume signal acquired by MRI or the like, or a spatio-temporal image obtained by connecting image sequences in the time direction is three-dimensional data.

  When calculating the distance (that is, the reciprocal of the similarity) by the luminance difference value, the following equation 16 can be derived.

(1-D) / (1 + θ) <α ⁿ <(1-D) / (1-θ) (16)

When calculating the similarity by the histogram overlap rate, the following equation 17 can be derived.

(1-S) / (1-θ) <α ⁿ <S / θ (17)

  A signal search device according to a third embodiment of the present invention will be described with reference to FIGS. In this embodiment, a case where a signal search device is used for a face detection device will be described.

  The face detection apparatus of this embodiment uses the face detection method of Viola-Jones et al. (Paul Viola, Michael J. Jones, “Robust Real-Time Face Detection” International Journal of Computer Vision, Vol. 57, No. 2, pp.137-154, 2004.). That is, a weak classifier using the Harr-based feature as shown in FIG. 15 is used as the detector 102, and this weak classifier is connected in series to m stages (where m ≧ 1) as shown in FIG. Connect and configure the classifier.

  The Harr-based feature detects the sign of the luminance value difference between the search windows w1 and w2. If the luminance difference value between the search windows w1 and w2 is positive, it is 1; Taking FIG. 15a as an example, if the luminance of the first search window w1 is 15 and the luminance of the second search window w2 is 3, the feature amount is 1 because it is 15-3> 0. This process is performed by the first-stage weak classifier in FIG. 16a. If the luminance difference value is 1, the process proceeds to the determination of the second-stage weak classifier in FIG. 16b.

  The second-stage weak classifier uses a Harr-based feature different from that of the first-stage weak classifier.

  This process is repeated for all positions on the image, and positions determined as 1 by all weak classifiers are detected as faces. This process is first performed in both search windows w1 and w2 of size βmax, and while gradually reducing the size of both search windows w1 and w2 to α times, both search windows w1 and w2 become size βmin. Repeat detection.

  The detector 102 uses only the sign of the luminance difference value D between the search windows w1 and w2. Substituting θ = 0 into Equation 4, the following Equation 18 can be derived.

1-D <α ² <1 + D (18)

(1) Face Detection Method A face detection method will be described below by applying the first embodiment.

  As shown in FIG. 17, the face detection apparatus sets a search area to be searched, and searches for a face in the search area using the first search window w1 and the second search window w2. The face detection device has a search frame. This search frame also has position information on how to arrange the search windows w1, w2 at each stage and the search windows w1, w2 at each stage.

  First, the input image is equally divided into 16 parts. Then, the following processing is performed for each divided region.

  For example, in the first divided region, the luminance difference value D (1) is calculated by using the first-stage search windows w1 and w2 shown in FIG. 16a, and the expansion / contraction rate α (1) is obtained. Both search windows have the same shape as the first search window w1 and the second search window w2 for edge detection shown in FIG. 1a. For example, as shown in FIG. 16, it is assumed that the first-stage both search windows w1 and w2 are detectors 102 that search for both eyes. As shown in FIG. 18, the center P1 of the first-stage search windows w1 and w2 is shifted obliquely downward when the upper left corner of the first divided area is the origin P0.

  From Equation 18, the possible range of α (1) can be expressed as Equation 19 below.

1-D (1) <α (1) ² <1 + D (1) (19)

If α (1)> Afix, the edge is not detected, and the expansion / contraction rate α of both search windows w1 and w2 in the first stage is set to the lower limit value of Equation 19, and α = √ (1-D (1 )), The search windows w1 and w2 in the first stage are reduced, and the search is continued in the first divided region. Here, in order to maximize the expansion / contraction rate, α is set to the lower limit value of Equation 19, but the value to be set to α may be any value as long as it satisfies Equation 19.

  When α (1) ≦ Afix, the luminance difference value D (2) is calculated using the second search windows w1 and w2 shown in FIG.

  As shown in FIG. 17, the detector 102 does not use the search windows w1 and w2 as they are, but the search frame also has information on where the second search windows w1 and w2 are arranged. . Therefore, the degree of similarity is calculated within a range in which the first-stage search area is shifted by the relative difference between the positions of the first-stage search windows w1, w2 and the second-stage search windows w1, w2. There is a need. For example, if the second search windows w1 and w are to search the left cheek as shown in FIG. 16, the positional relationship between the first search windows w1 and w2 for searching both eyes is clear. is there. That is, if the search area is set at a position obliquely below the search areas of the first-stage search windows w1, w2, the left cheek area can be efficiently searched.

  FIG. 18 shows the positions of the first to third stages of both search windows w1, w2. P1, P2, and P3 represent the centers of the search windows w1 and w2 in the first, second, and third stages. In the search using both search windows w1 and w2 in the second stage, the luminance difference value D (2) is calculated in the shifted search area as shown in FIG. 19, and the expansion / contraction rate α (2) is obtained.

  In this detector 102, all of the first-stage luminance difference values D (1) are obtained within the search area, but the second-stage luminance difference value D (2) is obtained at a position where D <0 in the first stage. Only ask. In addition, the second-stage luminance difference value D (2) is not obtained for positions that are not included in the first-stage search area in the second-stage search area. Therefore, when calculating the expansion / contraction rate by the two search windows w1 and w2 in the second stage, it is necessary to newly obtain a luminance difference value even at a position where the luminance difference value is not obtained.

  If α (2)> Afix, α = √ (1-D (2)) is set, the search windows w1 and w2 in the second stage are reduced, and the search for the first divided region is continued. .

  When α (2) ≦ Afix, the luminance difference value D (3) is calculated using the third-stage search windows w1 and w2 shown in FIG. 16c, and the expansion / contraction rate α (3) is obtained. The search windows w1 and w2 in the third stage are detectors 102 that search for the nose.

  The above processing is repeated and the processing is repeated for all m stages. When α (m) ≦ Afix is finally set, α = Afix is set, and the search is continued by reducing both search windows w1 and w2. .

  As described above, in this embodiment, when the face detection method of Viola-Jones et al. Is applied, it is possible to search for the expansion / contraction direction of the search window, and the search can be skipped for a region that does not look like a face. it can.

DESCRIPTION OF SYMBOLS 101 Signal input part 102 Detector 103 Similarity selection part 104 Expansion / contraction rate calculation part 105 Attention area extraction part 106 Feature extraction part 107 Similarity degree calculation part 108 Result output part

Claims (10)

A region-of-interest extraction unit that places a first search window and a second search window of an arbitrary size in a region on the input signal;
A feature extraction unit for extracting the feature amount of each search window;
A similarity calculation unit that calculates a degree of similarity that increases as the two feature quantities between the search windows are similar using the feature quantities;
When the similarity is smaller than a predetermined threshold, a result output unit that outputs the size of the search window and the position of the region;
An expansion / contraction rate calculation unit for calculating an expansion / contraction rate for expanding / contracting the size of the search window so that the similarity cannot fall below the threshold value;
The size of each search window is expanded / contracted by the expansion / contraction rate, (1) the signal search process is terminated when the size of the expanded / contracted search window reaches a predetermined size, and (2) the expanded / contracted search is performed. If the size of the window does not reach the predetermined size, the size of the search window expanded and contracted is output to the feature extraction unit, the extraction process of the feature extraction unit, and the similarity calculation unit A repeating control unit that repeats the calculation process, the output process of the result output unit, and the calculation process of the expansion / contraction rate until the signal search process is completed, and
A signal search apparatus comprising: The similarity calculation unit further includes a similarity selection unit that selects a maximum similarity from each of the similarities calculated for each position in the region of the input signal,
The expansion / contraction rate calculation unit calculates the expansion / contraction rate using the maximum similarity.
The signal search device according to claim 1. Dividing the input signal into a plurality of regions, and expanding and contracting the size of the search window for each of the divided regions;
The signal search device according to claim 2. The expansion / contraction rate calculation unit predicts the size of both search windows to be expanded / contracted next by using the current size of the search windows expanded / contracted by the expansion / contraction unit,
The similarity calculation unit obtains the similarity using the both search windows of the predicted size,
The iterative control unit, when the expansion / contraction rate calculated from the similarity obtained from the predicted size is outside the range in which the two search windows can be expanded and contracted, Is backtracked to bring it close to the current size of both search windows,
The signal search device according to claim 1. The first search window and the second search window are adjacent to each other and have the same shape, or each of the first search window and the second search window has a plurality of regions, or The area of the first search window is different from the area of the second search window;
The signal search device according to claim 1. The feature amount calculated by the feature extraction unit is a histogram of signal values of the input signal or an average of signal values.
The signal search device according to claim 1. The similarity calculated by the similarity calculation unit is a difference in average value of signal values of the input signal or a histogram overlap rate of the signal values.
The signal search device according to claim 1. The similarity calculated by the similarity calculation unit is expressed by an expression for obtaining an upper limit of the similarity when the size of the search window changes depending on the expansion / contraction rate.
The signal search device according to claim 1. The input signal is a video signal, an image signal, a spatiotemporal image signal, an audio signal, a character string, or a three-dimensional volume signal.
The signal search device according to claim 1. As the similarity, the reciprocal of the distance that decreases as the two feature quantities between the search windows are similar is used.
The signal search device according to claim 1.

Images