JP6043706B2 - Matching processing apparatus and matching method - Google Patents

Description

  The present invention relates to a matching processing device and a matching processing method for matching image data.

  As a method for matching image data, a method is known in which histograms relating to brightness of images are compared and image data is searched from the comparison result (for example, Non-Patent Document 1). The image matching process is used as a method for evaluating the deterioration state of a structure, for example.

S. Lazebnik, C. Schmid, and J. Ponce, “Beyond bags of features: spatial pyramid matching for recognizing natural scene categories,” IEEE Computer Vision and Pattern Recognition, 2006.

  In the conventional matching method, although histograms relating to luminance are compared, the histograms of luminance may be similar even if they are different images. In such a case, the accuracy of matching is reduced. was there.

  In view of the above-described viewpoints, an object of the present invention is not a histogram relating to luminance, but a matching processing device that compares low-dimensional statistics in each of a plurality of images having different resolutions and performs image matching based on the comparison result. And providing a matching method.

The present invention for solving the above problem comprises a sum of low-dimensional statistics of a histogram distribution associated with a preset image area , and the low-dimensional statistics change in a time-series manner. The data storage unit for storing the time series model and the object image as the processing target are locally non-uniformly or uniformly divided, a histogram distribution is obtained for each divided region, and the histogram distribution a data generating unit for calculating the low-dimensional statistics for each region, the total sum of the low-dimensional statistics the low-dimensional statistical amount obtained based on the respective areas, each of the time series model a matching unit for matching the sum of the time-series the low-dimensional statistical amount for each region, the result of the matching visually for each of the areas And a display control unit that presents.

The present invention for solving the above problem is a method in which a computer performs a matching process, the computer comprising a sum of low-dimensional statistics of a histogram distribution associated with a preset image region , and A data storage unit that stores a time-series model expressing the low-dimensional statistics in a time-series manner , and locally divides the object image as a processing target unevenly or uniformly, and after the division Obtaining a histogram distribution for each of the regions , calculating a low-dimensional statistic for each region from the histogram distribution, and the low-dimensional obtained based on the low-dimensional statistic for each region matching the statistics of the sum, the sum of the time-series the low-dimensional statistics for each of the respective regions of the time-series model Comprising a step, the steps of the visually displaying the results of said matching for each of the areas.

  According to the present invention, a matching processing device and a matching method for comparing low-dimensional statistics in each of a plurality of images having different resolutions and performing image matching based on the comparison result are realized.

It is a figure which shows an example of a function structure of the matching processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of schematic structure of the matching processing apparatus of this embodiment. It is a figure for demonstrating the example of the luminance distribution according to a different area | region division number. It is a figure for demonstrating the example of the image model in a different area | region. It is a figure for demonstrating the example of the matching process of the image model in a different area | region. It is a figure for demonstrating the example of the secular change of an image model. It is a flowchart which shows an example of the whole operation | movement of a matching processing apparatus. It is a figure for demonstrating an example of each distribution of the brightness | luminance and gradient of an image.

  FIG. 1 is a diagram illustrating an example of a functional configuration of a matching processing device 1 according to the present embodiment.

  As shown in FIG. 1, the matching processing device 1 includes a data input unit 100, a data generation unit 110, a matching unit 120, a data storage unit 130, a display control unit 140, and a secular change application unit 150.

  The data input unit 100 inputs image data to be processed in the data processing apparatus 1. The image data includes a matching target object (for example, a structure).

  The data generation unit 110 calculates low-dimensional statistics (hereinafter simply referred to as “statistics”) in a plurality of different divided regions with respect to image data, and generates an image model obtained based on the sum of the statistics. To do. The processing of the data generation unit 110 will be described in detail later.

  The matching unit 120 matches the image model generated by the data generation unit 110 with the time series model of the data storage unit 130. As will be described later, the time series model is a model composed of low-dimensional statistics in each image region.

  The display control unit 140 displays the matching result in the matching unit 120 visually. For example, when the matching results match, the display control unit 140 displays the image data representing the matching time series model so that it can be visually recognized. On the other hand, when the matching results do not match, the display control unit 140 displays information for indicating the fact so as to be visible.

  The secular change application unit 150 converts a statistic in the image region in the image model into a statistic representing a secular change by a preset function. Then, the secular change application unit 150 updates the time series model of the data storage unit 130 using the converted statistics. The process of the secular change application unit 150 will be described in detail later.

  FIG. 2 is a diagram illustrating a hardware configuration example for realizing the functional configuration of the matching processing device 1 illustrated in FIG. 1.

  As shown in FIG. 2, the matching processing device 1 includes a CPU (Central Processing Unit) 11, a memory 12, a display unit 13, and an interface 14.

  The CPU 11 is connected to each component via a bus and performs control signal and data transfer processing. Further, the CPU 11 executes a program for realizing the operation of the entire matching processing device 1.

  The memory 12 includes a ROM (Read Only Memory) and a RAM (Random Access Memory), and stores the above-described data and the like. The memory 12 functions as the data storage unit 130 illustrated in FIG.

  The display unit 13 can be a flat panel display such as a liquid crystal display or EL (Electro-Luminescence).

  The interface 14 is used to input image data, and is connected to a network such as a LAN (Local Area Network). In this embodiment, a computer system is connected to the network, and image data is transmitted from the computer system to the CPU 11 via the network for processing.

  In this matching processing device 1, for example, when the image data is a color image, the CPU 11 obtains a luminance histogram in different divided regions related to the image, and calculates a statistic (for example, an average value of luminance) in each region from each luminance histogram. , Variance, skewness, kurtosis, etc.) to generate an image model consisting of the sum of each statistic. Thus, the CPU 11 quantifies the first to nth moment amounts. Here, n represents the number of statistics.

  Next, an outline of color image matching processing realized by the matching processing device 1 will be described with reference to FIGS.

  FIG. 3 is a diagram for explaining an example of a luminance histogram in each area according to different number of area divisions, where (a) is divided into 4 parts, (b) is divided into 16 parts, and (c) is divided into 64 parts. Show.

  3A to 3C show an example in which the reduction process is performed by changing the number of divisions of an image having the same number of pixels. In FIG. 3A, first, the original image is equally divided into four, and a histogram is obtained in each divided region (left column). Next, the entire image is halved (middle row), and further a reduction process is performed to halve the image (right row). Then, in the reduced image, a histogram is obtained for each of the four divided areas as in the original image.

  The same processing as that in FIG. 3A is performed for the components shown in FIGS. 3B and 3C. An image generated by such a reduction process is called a multi-resolution image. In the reduction process, the averaging process of data of 4 points (pixels) is repeated to reduce the number of pixels to ¼.

  As shown in FIG. 3A, when the area of the original image is divided into four, the CPU 11 creates luminance histograms d1, d2, d3, and d4 for the areas A1, A2, A3, and A4, respectively.

  Further, in FIG. 3B, the area shown in FIG. 3A is further divided into 16, and the CPU 11 obtains luminance histograms d11 to d14 of the corresponding area.

  In FIG. 3C, the area shown in FIG. 3B is further divided into 64 areas, and the CPU 11 obtains a luminance histogram in the corresponding area. When matching the details of an object in an image, it is preferable to obtain a luminance histogram by increasing the number of divisions.

  FIG. 4 is a diagram illustrating an example of generating an image model in each region by obtaining low-dimensional statistics for each resolution and for each divided region for the same image as FIG. 3.

  In FIG. 4 (a), for the same image as in FIG. 3 (a), the CPU 11 performs statistics S1 (f1, f2, f3... Fn), S2 (f1, f2, f3... Fn). , S3 (f1, f2, f3... Fn), S4 (f1, f2, f3... Fn) are obtained.

  In general, it is known that the RGB component of an image is represented by the following equation (1) (for example, http://home.hiroshima-u.ac.jp/tkurita/lecture/statimage.pdf See).

  In the formula (1), R represents a red component, G represents a green component, and B represents a blue component.

  In the matching processing device 1 of the present embodiment, the CPU 11 performs a conversion process according to the above equation (1), and R, G, B on the left side in the equation (1) (R bar, G in equation (1)) (Bar, B) are replaced with the values of (f1, f2, f3) of the corresponding statistics S1 to S4, respectively. In addition, the value of the matrix illustrated in Formula (1) is only an example.

  4 (b) and 4 (c) show the same images as FIG. 3 (b) and FIG. 3 (c), respectively, and the CPU 11 is shown in FIG. 4 (b) and FIG. 4 (c). The same processing as that shown in FIG. By such processing, the statistic of each multi-resolution image is obtained.

CPU11 produces | generates the image model of matching object by calculating | requiring the sum total of the statistics of each area | region mentioned above. In this case, the CPU 11 creates a linear sum F by the following equation (2).
F = a1 × (S1 + S2 + S3 + S4) + a2 × (S1 + ... + S16) + a3 × (S1 + ... + S64) + ... + ai × (S1 + ... + SN)
(2)

  In Expression (2), S1 to SN: statistics in the corresponding divided region, and a1 to ai: weighting factors.

  In general, the smaller the divided area, that is, the higher the resolution, the easier the matching is. Therefore, in Equation (2), the weight coefficients a1 to ai are increased in value as the resolution is higher. For example, a1 = 0.5, a2 = 0.25, a3 = 0.125,. Note that the values of the weighting factors a1 to ai can be changed.

  In the matching processing device 1, the images are associated using the image model given by the linear sum F described above.

  FIG. 5 is a diagram illustrating an example in which image matching (similarity) is performed on the same image as in FIG. 4 using statistics for different resolutions and for each divided region.

  5A, for the same image as the left column of FIG. 4A, the CPU 11 stores an image model M composed of a linear sum F including the statistics S1 to S4 in each region in the memory 12. Compared to a time series model (for example, an image model created in the past), it is determined that each statistic S1 to S4 matches, for example, each statistic D1 to D4 of the time series model.

  Next, in FIG. 5B, for the same multi-resolution image as in the middle column of FIG. 4A, the CPU 11 uses the statistics of the generated image model as a time series model (for example, the past) And the statistics of the image model are determined to match the time series model.

  Further, in FIG. 5C, for the same multi-resolution image as that in the right column of FIG. 4A, the CPU 11 uses each statistic of the generated image model as a time series model (for example, in the past). It is determined that each statistic of the image model matches the time series model. In this way, the CPU 11 performs image association (similarity) using statistics for each different resolution and each divided region.

  The above-described matching process can use the following techniques for processes using Euclidean distance, Mahalanobis distance, Manhattan distance, Minkowski distance and the like well known in the pattern recognition field.

  Shigeki Hatakeyama, Department of Computer Science, Faculty of Engineering, University of Tokyo Second Pattern Space, Distance Scale, Cluster, DP http://hil.t.u-tokyo.ac.jp/~sagayama/enshu2/Clustering+DP-2007nov.pdf

  Next, the aging process of the time series model D in the memory 12 executed by the data processing apparatus 1 will be described with reference to FIG. FIG. 6 is a diagram for explaining an example of the secular change of the time series model D (t). In FIG. 6, the horizontal axis T represents time.

The time series model D (t) takes values as shown in FIG. That is, when time T = t, t + 1, t + 2, the time series models D (t), D (t + 1), and D (t + 2) are obtained. In this case, the CPU 11 reduces the luminance in the image exponentially, for example, according to the following equation (3).
Y = Aexp (-Bt) (3)

  In equation (3), A: luminance of the original image, B: attenuation constant, Y: luminance at time t.

  In the above formula (3), the CPU 11 reduces the luminance of the image as time passes. This is because an object such as a structure corrodes a metal or a coating film or resin deteriorates with time, so that the influence of the secular change on the brightness of the object cannot be ignored. In other words, even if the actual object changes over time, it is necessary to improve the matching accuracy. In addition, the function of Formula (3) for expressing a secular change is only an example, and can also be changed.

  Next, the entire process executed to realize this matching process will be described with reference to FIGS. FIG. 7 is a flowchart illustrating a processing example of the matching processing device 1.

  In FIG. 7, the CPU 11 of the matching processing device 1 inputs image data to be processed (step S100). This image data is stored in the memory 12.

  Here, the CPU 11 functions as the data input unit 100 in cooperation with the interface 14.

  The CPU 11 calculates a statistic (see the above formula (1)) in a plurality of different divided regions with respect to the image data input in step S100, and obtains an image obtained based on the sum of each statistic (see the above formula (2)). A model is generated (step S101). In this embodiment, the CPU 11 generates an image model M representing a linear sum F composed of the sum of statistics in, for example, a plurality of different regions (for example, 4 divisions, 16 divisions, 64 divisions, etc.) (see FIG. 4). ).

  Here, the CPU 11 functions as the data generation unit 110.

  Next, for example, as illustrated in FIG. 5, the CPU 11 matches the image model M generated in step S <b> 101 with the time series model of the data storage unit 130 (step S <b> 102).

  Here, the CPU 11 functions as the matching unit 120.

  The time series model of the data storage unit 130 is set by the CPU 11 so that the luminance in the image decreases exponentially according to the above equation (3) (see FIG. 6). At this time, the CPU 11 functions as the secular change application unit 150.

  The CPU 11 visually displays the result of matching in step S102 on the display unit 13. For example, if the matching results match, the CPU 11 displays the image data representing the matching time series model so that it can be viewed. On the other hand, if the matching results do not match, the CPU 11 displays information for indicating the fact so as to be visible.

  Here, the CPU 11 functions as the display control unit 140.

  As described above, according to the matching processing device 1 of the present embodiment, since statistics are calculated in a plurality of different divided regions, matching is performed using an image model obtained based on the sum of the statistics. The similarity of images can be evaluated with higher accuracy than in the conventional case of comparing histograms relating to luminance. Further, by using the statistic, it becomes possible to evaluate the similarity of images in a multifaceted manner. Furthermore, in the case of the matching processing device 1 of the present embodiment, high speed matching processing is realized.

  Further, since the matching processing device 1 converts the statistic representing the secular change and updates the time series model, for example, considering the corrosion of the metal or the deterioration of the coating film / resin in various structures due to the secular change. Image similarity can be evaluated.

  Next, a modification of this embodiment will be described.

  For example, in the case of a specific object, the magnitude of brightness or / and gradient may be extremely different (large or small) from other areas, so increase the number of image divisions locally for that area. The statistic may be obtained. Therefore, for example, as shown in FIG. 8, attention is paid to the luminance (FIG. 8B) or gradient (FIG. 8C) of the pixels in the image shown in FIG. Is preferred. In this case, the object search efficiency is improved.

  The case where the luminance histogram in the case of a color image is obtained has been described above with reference to FIG. However, the image data may be a black and white image. In this case, the CPU 11 obtains the statistic in each area based on the luminance level of the image without performing the conversion process of the above-described equation (1).

DESCRIPTION OF SYMBOLS 1 Matching processing apparatus 100 Data input part 110 Data generation part 120 Matching part 130 Data storage part 140 Display control part 150 Aging change application part

Claims (3)

A data accumulating unit that accumulates a time-series model that is composed of a sum of low-dimensional statistics of a histogram distribution that is associated with a preset image area and that represents the low-dimensional statistics changing in time series When,
Data for locally dividing the object image as a processing target unevenly or uniformly, obtaining a histogram distribution for each divided region, and calculating a low-dimensional statistic for each region from the histogram distribution A generator,
The sum of the low-dimensional statistics obtained based on the low-dimensional statistics for each region, and the time-series sum of the low-dimensional statistics for each region of the time-series model ; A matching unit that matches
And a display control unit for visually displaying a result of the matching for each region .   A low-order statistic in the image region is converted into a statistic representing a secular change by a preset function, and the chronological change application is used to update the time series model of the data storage unit using the converted statistic. The matching processing apparatus according to claim 1, further comprising a section. A method in which a computer performs matching processing,
The computer stores a time-series model that is composed of a sum of low-dimensional statistics of a histogram distribution that is associated with a preset image area , and that the low-dimensional statistics change in a time-series manner. Data storage unit
About the object image to be processed is locally unevenly or uniformly divided, with a histogram distribution in each region of the divided to calculate the low-dimensional statistics of each of said regions from said histogram distribution step When,
The sum of the low-dimensional statistics obtained based on the low-dimensional statistics for each region, and the time-series sum of the low-dimensional statistics for each region of the time-series model ; Matching the steps,
Visually displaying the result of the matching for each of the regions .