JP2016122405A - Pattern association method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a pattern association method and device for calculating similarity by taking information related to the position of a value of a feature amount into consideration in each value of the feature amount, such as the luminance of an image.SOLUTION: A luminance analysis part 122 mathematically calculates a similarity function by using a high-order moment statistical amount, such as a position and dispersion in each feature amount of luminance or the like about image data stored in a data storage part 121. Also, an edge analysis part 123 mathematically calculates a similarity function by using a high-order moment statistical amount, such as a position and dispersion in each gradient value of the feature amount of luminance or the like. A similarity analysis part 124 calculates a similarity between continuous images and a similarity between an image desired to be retrieved and an original image. A combination having the highest similarity is made an optimum solution, a movement object is traced between the images, and a specific object is retrieved from a database.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a pattern associating method and apparatus for searching and tracking a pattern included in an image based on a distribution of feature quantities such as luminance.

  In recent years, there has been an increasing demand for automatic tracking of people and vehicles by camera video monitoring and methods and systems for efficiently searching for objects of interest from a huge database. The former is not limited to indoor environments where the lighting conditions are relatively stable, but is also performed in outdoor environments exposed to lighting fluctuations due to sunlight and weather, especially in an outdoor environment. It is known that there are many cases. In the latter, when various data are reconstructed into an image and a pattern similar to the image is searched, it is difficult to uniquely narrow down if the complexity of the pattern included in the image exceeds a certain level.

  Under such circumstances, for each image data, various tracking methods and search methods using the luminance, color, texture, etc. of the pixels constituting the image as feature amounts have been proposed. In particular, the histogram method for statistically analyzing the number of pixels of the target luminance in an image has a relatively high resistance to environmental fluctuations and is often used.

“Histogram”, [online], [Search on December 15, 2014], Internet <URL: http://en.wikipedia.org/wiki/%E3%83%92%E3%82%B9%E3% 83% 88% E3% 82% B0% E3% 83% A9% E3% 83% A0> "Bhattacharyya distance", [online], [December 15, 2014 search], Internet <URL: http: //en.wikipedia.org/wiki/Bhattacharyya_distance> "Edge detection", [online], [December 15, 2014 search], Internet <URL: http://en.wikipedia.org/wiki/%E3%82%A8%E3%83%83%E3 % 82% B8% E6% A4% 9C% E5% 87% BA>

  However, this conventional histogram method has a high resistance to environmental fluctuations and is a simple method. However, since the position information of the target in the image is missing, another image data is searched from one image data. When used for tracking or tracking, there is a problem that image data having the same luminance distribution but different pattern spatial positions, that is, image data having a different composition cannot be identified. Therefore, a method for suppressing an increase in false detection is required.

  In addition, the image data for camera video monitoring includes, for example, those using thermography that measures the temperature in a room such as a data center, the intake, exhaust, and rack of a server. It is also used to estimate the state in the data center by comparing and referring to the temperature distribution data accumulated in the data center.

  However, in this case as well, the number of combinations related to the heat generation amount and ON / OFF states of a plurality of servers monitored in one image in the data center is enormous, and rarely the same as the state accumulated in the past. It is. In addition, since it depends only on the histogram of the temperature distribution and does not use the server position information, the temperature sensor temperature near the server that shows a certain intake air temperature and the temperature sensor that shows the same intake air temperature at a remote location There is a problem that a pattern indicating temperature cannot be identified, and in particular, an erroneous association occurs as the number of servers increases.

  The present invention has been made in view of such problems, and the object of the present invention is to take into account information related to the position of the feature value for each feature value such as the brightness of an image. Another object of the present invention is to provide a pattern matching method and apparatus for calculating similarity.

  In order to solve the above-described problems, the present invention provides a pattern association device, a storage device for acquiring and storing image data, a histogram relating to the feature amount of the image data, and the feature amount A feature amount analysis unit that calculates a moment statistic for each of the values, and calculates an edge image corresponding to the image data by calculating a gradient value of the feature amount, and creates a histogram relating to the gradient value of the feature amount, An edge analysis unit that calculates a moment statistic for each gradient value of the feature value, a first similarity between the image data of the moment statistic value for each value of the feature value, and a moment for each gradient value of the feature value A similarity analysis unit that calculates a second similarity between the image data of statistics, and calculates a similarity between the image data based on the first and second similarities. The features.

  According to a second aspect of the present invention, in the pattern associating device according to the first aspect, the image data is image data captured by a camera, and the feature amount is luminance.

  According to a third aspect of the present invention, in the pattern correlating device according to the first aspect, the image data is a thermography measured by a thermography device, and the feature amount is a temperature.

  According to a fourth aspect of the present invention, in the pattern correlation apparatus according to any one of the first to third aspects, the moment statistic includes at least one of an average value, variance, kurtosis, and skewness. It is characterized by that.

  The invention according to claim 5 is a pattern associating method, the step of acquiring image data and storing it in a storage device, creating a histogram relating to the feature amount of the image data, and for each value of the feature amount Calculating a moment statistic; calculating a gradient value of the feature value; generating an edge image corresponding to the image data; creating a histogram relating to the gradient value of the feature value; And calculating a moment statistic for each feature value, a first similarity between the image data of the moment statistic for each value of the feature value, and a first statistic between the image data of the moment statistic for each gradient value of the feature value. And calculating a similarity between the image data based on the first and second similarities, respectively.

  According to a sixth aspect of the present invention, in the pattern associating method according to the fifth aspect, the image data is image data captured by a camera, and the feature amount is luminance.

  According to a seventh aspect of the present invention, in the pattern associating method according to the fifth aspect, the image data is a thermography measured by a thermography device, and the feature amount is a temperature.

  The invention according to claim 8 is the pattern associating method according to any one of claims 5 to 7, wherein the moment statistic includes at least one of average value, variance, kurtosis and skewness. It is characterized by that.

  According to the present invention, in a time-series image data, in a similar search for a specific target region or the entire image, a tracking problem of a moving target between images, a conventional histogram including only feature values such as luminance is used. Mathematically similar functions using higher-order moment statistics such as each position and variance after obtaining information related to the position of feature quantity such as brightness gradient value between each brightness and pixel This makes it possible to make pattern matching robust to environmental changes by extending the conventional calculation method of histogram distribution.

It is a figure which shows the structure of the pattern matching apparatus which concerns on one Embodiment of this invention. It is a flowchart explaining the pattern matching method which concerns on one Embodiment of this invention. It is a figure which shows the example of a problem using the histogram method regarding the brightness | luminance of the image from the past. It is a figure which shows the histogram of the example of a problem using the histogram method regarding the brightness | luminance of the image conventionally. It is an example of one piece of standard image data used when searching a plurality of image data on a database by the present invention. It is an example of several image data on the database by this invention. It is a figure which shows the example of an experiment when tracking a specific object under the condition where there is a large environmental fluctuation in the background when using a camera image. It is a figure which shows the precision comparison result of the tracking rate and detection rate using the image data by the difference in the similarity evaluation method.

  Hereinafter, embodiments of the present invention will be described in detail.

  FIG. 1 shows a configuration of a pattern association apparatus 100 according to an embodiment of the present invention. The data storage unit 121, the luminance analysis unit 122, the edge analysis unit 123, and the similarity analysis unit 124 that constitute the present invention are stored in the storage device 120 and can be executed by a component having the following functions or the processing device 130. It can be configured as software. The input interface 110 is an interface that inputs a signal input from the input device 150 or the like as numerical information to the system. The output interface 140 is an interface that outputs a processing result to the monitor 160 or the like.

  FIG. 2 shows a main processing flow of the pattern association method according to the embodiment of the present invention. The present invention inputs image data captured by a camera or image data reconstructed and visualized from various observation data into the input interface unit 110 from a camera or various databases, and stores the acquired image data. The data is stored in the 120 data storage unit 121 (S201). For the image data stored in the data storage unit 121, a similarity function is mathematically calculated in the luminance analysis unit 122 using high-order moment statistics such as position and variance for each feature amount such as luminance (S <b> 202). ). In addition, for each gradient value representing the amount of spatial change between adjacent pixels of a feature quantity such as luminance, the edge analysis unit 123 mathematically calculates a similar function using higher-order moment statistics such as position and variance. (S203). The similarity analysis unit 124 calculates the similarity between consecutive images and the similarity between the image to be searched and the original image (S204). The combination having the highest similarity is set as the optimal solution, and the display unit 160 displays the associated image as one of the results by tracking the moving object between images or searching for a specific object from the database. (S205). In this manner, the present invention realizes a pattern matching method and apparatus that is robust to environmental fluctuations.

  3A to 3D show examples of problems using a conventional histogram method relating to the brightness of an image. It is assumed that a star-shaped object having a certain texture exists in the image. Here, it is assumed that the area of a star in one image is constant. In the case of this condition, when there is a difference in star position (FIGS. 3B and 3C) as compared to FIG. 3A, there is a rotation (FIGS. 3B and 3C), Although the area of each star is small, it can be assumed that there are a plurality of stars (FIG. 3D).

  Since these images have the same texture and the area of the star is constant, when the histogram of the luminance of the entire image is created, as shown in FIG. 4, the four images shown in FIGS. Both have the same distribution. In FIG. 4, as an example, the number of pixels for each luminance when the maximum luminance in the image is 255 is represented by the occurrence frequency (probability) 0 to 1 normalized by dividing the number of pixels of the entire image. As this example shows, it is impossible to identify positions, sizes, rotations, and the like that are clearly different for each image in the histogram. That is, although the histogram method is simple, it shows that there is a certain limit to stable object tracking and database search in continuous time-series image data.

  On the other hand, the simplicity of the calculation of the histogram method remains as an advantage. Note that the luminance can be replaced with a thermography that visualizes the temperature from a temperature sensor measured at a data center or the like as an image pattern.

  Hereinafter, the luminance analysis unit 122 in FIG. 1 will be described.

  Accordingly, the present invention solves the problem of the lack of target position information, which was a drawback of the histogram method, by the following procedure. First, assume that the number of pixels of one image is N, and the maximum luminance of each pixel is B (minimum 0). If the luminance is an integer value b of 0 to B, the histogram is

And a mathematical model. At this time, when the i-th pixel in the image has luminance b (b _i = b), the Dirac delta function δ _bib = 1, and in other cases (b _i ≠ b), δ _bib = 0. C is a normalization coefficient, so that the sum of occurrence probabilities for all luminances per sheet is 1.

Gave a relationship. Subsequently, a higher-order moment amount such as an average value or variation (dispersion) of the position of the pixel corresponding to each luminance in the image, which is not considered in the conventional histogram analysis, is added. That is, for convenience of explanation, it can be said that the present invention also considers the center of gravity of the luminance distribution region and its spread. Here, when the position of the pixel i in the image is x _i = [x, y] ^T , the average value that is the first moment is

And the second moment moment dispersion is

It becomes. Third moment, fourth moment kurtosis, skewness, etc. can be easily defined, and it is effective when the target luminance distribution is not Gaussian distribution, but the target has a certain size or more. Is required.
Next, two image data

Are obtained using equations (1) to (3). About this, the Bhattacharaya coefficient which considered the crossing degree of the tail of probability distribution was used (refer nonpatent literature 2). This is defined as:

However,

It is. In Equations (4) and (5), Ψ is a variable weighting factor. The smaller (larger) the difference between n _b and n ′ _b , the higher (lower) the similarity, and the larger the weighting factor.

  The edge analysis unit 123 will be described below.

  In the luminance analysis unit 122, the input data is the luminance distribution of the observed image data, but the edge analysis unit 123 performs edge detection on the image data. Various methods are known for edge detection (see Non-Patent Document 3). Here, the CANNY method widely used in computer vision is applied. This method is characterized in that it removes the influence of noise and the like and extracts a strong luminance gradient value (a spatial first derivative of luminance). Hereinafter, image data obtained by the CANNY method is referred to as an edge image. As described above, the edge image is applied to the equation (1) and the like. At this time, the number of pixels of one image is not different from N, but the value corresponding to the first derivative of each pixel is normalized in advance with the maximum value B (255). In fact, the first derivative has positive and negative values. Thereafter, for the edge image, equations corresponding to equations (2) to (5) relating to the luminance gradient value may be obtained by the same procedure as that relating to luminance. For the edge, the gradient direction and gradient strength can be obtained, but the gradient strength is used so that it does not depend on the rotational property of the texture. When the gradient direction is taken into consideration, a histogram relating to the frequency domain may be created after conversion to the frequency domain.

  From the above, from Equation (4), two similarities with respect to luminance and edge are obtained, which are denoted by ρ1 and ρ2, respectively. In the range of 0 ≦ ρ1 + ρ2 ≦ 2.0, the similarity between both the luminance and the edge is data that contributes to association in the case of the maximum value.

  FIG. 5 shows an example of one piece of reference image data X used when searching a plurality of image data on the database according to the present invention. FIG. 6 shows a plurality of image data on a database to be searched. Using these, an experiment was performed in which image data that is the same as or similar to one piece of image data X is searched from a plurality of image data (a) to (d) on an enormous database.

  When the conventional histogram method based on luminance is used, since the histograms of the plurality of image data (a) to (d) all have the same distribution as shown in FIG. 4, the plurality of image data (a) It was difficult to narrow down to one that is most similar to the reference image data X from (d). On the other hand, according to the present invention, position information {(x1, y1), (x2, y2). . . } Or one variance of higher-order moment amounts, the difference between the plurality of image data (a) to (d) and the reference image data X clearly appears as numerical values, and FIG. 6 (a) is identified. We were able to. 6 (b), (c), and (d), the relative distance from X increases, and the value of similarity index formula (4) decreases to 0.8, 0.4, and 0.01. .

  FIG. 7 shows an experimental example when a specific target is tracked in a situation where there is a large environmental fluctuation in the background when a camera image is used. In a real environment, environmental fluctuations are complicatedly related to sunlight, shadows, other object contamination, occlusion where a part of the object is hidden by buildings, etc., and the shape and texture of the object of interest to be tracked Are often complicated and often accompanied by deformation. Therefore, here we dealt with the problem of tracking specific objects in time-series image data.

  In the conventional histogram method, the object is often overlooked due to the strong influence of the environmental change of the background and the deformation of the moving object. On the other hand, in the present invention, the specific target can be tracked stably. In the present invention, the search range of the target to be traced from the current time to the next time is searched for a range of 10 pixels × 10 pixels in one image of 640 pixels × 480 pixels, and the most similar among them Searched for high-level objects.

  FIG. 8 shows the accuracy comparison results of the tracking rate and the detection rate using the image data for the experimental example of FIG. When the position of the original image is α and the estimated position is β, β / α × 100 is an evaluation value, and in particular, the tracking rate is when tracking the center of gravity of the target, and the whole or a part of the target The detection rate is defined as the detection rate.

  Using various image data, experiments were conducted to evaluate the tracking rate and detection rate in the conventional histogram method A, B in the case of using only luminance in the present invention, and C in the case of using luminance and edges together. Note that white noise and normal noise were added to the image data assuming environmental fluctuations, and incomplete image data was also created and evaluated by deleting some of the objects in the image at random. Used as data.

  As a result, the accuracy was improved in the order of A, B, and C. In particular, in the present invention, it is better to use edge information together than luminance alone. This is because the spatial position is taken into account in luminance, but the relationship between adjacent different luminances is taken into account by introducing edge information. That is, it was suggested that a finer spatial structure is effective in tracking problems and detection problems.

  Further, the present invention can be applied not only to camera image monitoring but also to estimation of a state such as temperature in the data center as described above. According to the present invention, by using a histogram distribution that takes into account the (space) position information of the temperature sensor and the temperature at the same time, it is possible to greatly reduce conventional misrecognition, and more accurately cope with reference to past data. Can be attached. As another application, it is also possible to spatially follow the time series change of the temperature distribution in the data center. That is, it is conceivable to grasp the transition of the state and connect it to the analysis.

DESCRIPTION OF SYMBOLS 100 Pattern matching apparatus 110 Input interface 120 Storage apparatus 121 Data storage part 122 Luminance analysis part 123 Edge analysis part 124 Similarity analysis part 130 Processing apparatus 140 Output interface 150 Input apparatus 160 Monitor

Claims (8)

A storage device for acquiring and storing image data;
Creating a histogram relating to the feature amount of the image data, and calculating a moment statistic for each value of the feature amount;
Edge analysis that calculates a gradient value of the feature value, generates an edge image corresponding to the image data, creates a histogram related to the gradient value of the feature value, and calculates moment statistics for each gradient value of the feature value And
Calculating a first similarity between the image data of the moment statistics for each value of the feature quantity and a second similarity between the image data of the moment statistics for each gradient value of the feature quantity; A similarity analyzer that calculates a similarity between the image data based on the first and second similarities;
A pattern matching apparatus characterized by comprising:   The pattern association apparatus according to claim 1, wherein the image data is image data captured by a camera, and the feature amount is luminance.   The pattern correlation apparatus according to claim 1, wherein the image data is a thermography measured by a thermography apparatus, and the feature amount is a temperature.   The pattern correlation apparatus according to claim 1, wherein the moment statistic includes at least one of an average value, variance, kurtosis, and skewness. Acquiring image data and storing it in a storage device;
Creating a histogram relating to the feature amount of the image data, calculating a moment statistic for each value of the feature amount;
Calculating a gradient value of the feature value, generating an edge image corresponding to the image data, creating a histogram relating to the gradient value of the feature value, and calculating a moment statistic for each gradient value of the feature value; ,
Calculating a first similarity between the image data of the moment statistics for each value of the feature quantity and a second similarity between the image data of the moment statistics for each gradient value of the feature quantity; Calculating a similarity between the image data based on the first and second similarities;
A pattern associating method characterized by comprising:   6. The pattern associating method according to claim 5, wherein the image data is image data captured by a camera, and the feature amount is luminance.   6. The pattern associating method according to claim 5, wherein the image data is thermography measured by a thermography apparatus, and the feature amount is temperature.   The pattern correlation method according to claim 5, wherein the moment statistic includes at least one of an average value, variance, kurtosis, and skewness.