CN109033204A - A kind of level integration histogram Visual Inquiry method based on WWW - Google Patents

Abstract

The invention discloses a kind of Visual Inquiry methods of level integration histogram, comprising the following steps: step 1: configuring to raw data set, including discretization interval number, crosses the condition of filter data and need to carry out the dimension of aggregate statistics；Step 2: with the building of offline pretreatment mode and storage hierarchy partition tree, wherein data are divided into multiple data subsets by level partition tree, and the statistical nature of each data subset is expressed by integration histogram；Step 3: visualization space uniform is discretized into specific zonule, distinguishing hierarchy tree in the coordinate input step 2 of zonule is subjected to range query, the range query is the process found the data subset for having intersection with target area and remove estimation target area statistical nature with the integration histogram of the intersection, and all zonules obtain a matrix about statistical nature after being all performed range query；Step 4: visual element binding being carried out to the matrix of statistical nature, carries out visualization request.

Description

A Visual Query Method of Hierarchical Integral Histogram Based on World Wide Web

technical field

The invention relates to the field of fast visual query, in particular to a fast query method for hierarchical integral histograms.

Background technique

In the visual analysis scenario of large-scale structured data, people need to understand and study the distribution of data from the statistical characteristics of the data, summarize the laws and make decisions through the distribution characteristics. The most common aggregation operation (referring to calculating a value from a set of values) is generally expressed visually through a histogram or a discretized scatter plot. When the amount of data is large enough, the method of directly traversing data items to calculate statistical features will not be able to meet the real-time requirements of interactive visual exploration. How to quickly query and obtain data in a specified range from large-scale structured data, such as real-time management and scheduling of transportation resources, real-time monitoring of financial transactions, etc., has become a hot topic in the fields of the Internet, transportation, aerospace, and commerce.

For large-scale structured data in reality, it has high dimensionality, many data items, various data modes and formats, and unique data distribution. Executing visual query operations on such a large and complex data set will have the problem of not being able to respond in a timely manner or even taking too long. Many existing methods perform query optimization at the database level. In order to obtain accurate results, they need to set considerations on the data set and construct external expressions that are easy for users to understand; in addition, some works based on the goal of approximate results adopt a A series of approximate query strategies (approximate query refers to querying data with an approximate strategy in order to reduce the response time of the query), such as techniques based on sampling algorithm, histogram expression and wavelet transform.

Some of the above approximation techniques use a fixed precomputation mode, which is limited to specific statistical characteristics and cannot be applied to various types of data, such as dynamic data and streaming data; some are limited to low-dimensional situations, and are required for high-dimensional dataset calculations The memory is too large.

Contents of the invention

The invention provides a visual query method of the hierarchical integral histogram, which reduces the search time to less than 500 milliseconds, reaches the level of interaction and significantly reduces the demand for storage.

A visual query method for a hierarchical integral histogram, comprising the following steps:

Step 1: Configure the original data set, including the number of discretized intervals, the conditions for filtering data, and the dimensions that need to be aggregated and counted;

Step 2: Based on the data processed in step 1, build and store a hierarchical partition tree in an offline preprocessing manner, where the data is divided into multiple data subsets by the hierarchical partition tree, and the statistical characteristics of each data subset are determined by the integral Histogram for expression;

Step 3: Uniformly discretize the visualization space into specific small areas through the configuration in step 1. For each small area, input the coordinates of the small area into the hierarchical division tree in step 2 to perform range query. The range query is to find and The target area has an intersection data subset and uses the integral histogram of the intersection to estimate the statistical characteristics of the target area. All small areas are searched for a range to obtain a matrix of statistical characteristics;

Step 4: Perform visual element binding on the matrix of statistical features in step 3, and perform a visualization request.

This method transfers the time loss to the preprocessing stage, and performs approximate calculations on the query results within the allowable error range. Compared with existing methods, this query method can significantly reduce storage costs, and the time complexity has nothing to do with the number of data points. Efficient online visual query is possible.

The present invention preprocesses the original data set and the target visual space based on the user's configuration parameters, and divides the data set into layers through a layered division algorithm, so as to realize expressions with different precisions and scales for differently distributed areas. For each sub-area, the integral histogram is used to approximate the statistical characteristics of the area. During visual query, the system uses the hierarchical division tree to quickly and effectively traverse the set of target areas and return the approximate value, so as to obtain the approximate statistical features of the target area.

Compared with the existing methods, this method transfers the time loss to the data preprocessing stage, preprocesses the data sets that may require visual query in advance offline, and obtains an efficient approximate expression of the data set, which can then be used in subsequent Online visual query. This method is based on the idea of approximation and gradual refinement. It only needs to store the integral histogram after the data is counted. Many other visual query methods need to store the original data, which requires a large loss of time and space, and cannot capture the data well. The distribution of data, so the application of this method is wider.

In order to improve the scope of application and intelligence of the present invention, preferably, step 4 also includes interactively adjusting visualization parameters, and obtaining instant visual feedback results during the visualization process.

In order to further improve computational efficiency, preferably, in step 2, the original data set is a high-dimensional data set V with n dimensions D={D ₁ ,...,D _n }, and the domains of each dimension are represented as {[a ₁ ,b ₁ ],...,[a _n ,b _n ]}.

In order to further improve computing efficiency, preferably, in step 2, the data is divided into multiple data subsets by a hierarchical partition tree. The specific process is: the entire data space is recursively divided to generate a hierarchical tree structure, and the data space is reconstructed. V'={v' ₁ ,...v' _i ...v' _p }, where each v' _i ∈V' corresponds to a leaf node of the tree.

In order to further improve the calculation efficiency, preferably, in step 2, the integral histogram is an extension of the summation table, and the value of each grid in the table is equal to the sum of all the values in the upper left corner of the table, so each grid The value in the cell can be obtained by adding and subtracting four values. The English name of the summation table is summed area table, which is a two-dimensional table.

In order to further improve the calculation efficiency, preferably, in step 2, the specific process of calculating the integral histogram is as follows: For a d-dimensional data set binned by N ₁ ×...N _d grids, and with b binning numbers The histogram of the leaf node is summarized, and the integral histogram of the leaf node is defined as:

Among them, x ₁ ,…,x _d is the index of binning in d dimensions, b is the index of binning in the histogram, h(x ₁ ,…,x _d ,·) represents the value of each grid histogram;

An integral histogram for any rectangular region in data space can be computed by:

where ^{xp is the corner point of the rectangular area, p∈{0,1}d .}

Beneficial effects of the present invention:

The visual query method of the hierarchical integral histogram of the present invention realizes the expression of different precision and scales for different distribution areas, obtains the approximate statistical characteristics of the target area, and transfers the time loss to the data preprocessing stage, which may need to be visualized The queried data set is preprocessed offline in advance to obtain an efficient approximate expression of the data set. Based on the idea of approximation and gradual refinement, it is only necessary to store the integral histogram after the data is counted, reducing time and space loss, and better Accurately capture the distribution of data and have wider applications.

Description of drawings

Fig. 1 is a schematic flow chart of the visual query method of the hierarchical integral histogram of the present invention.

Fig. 2 is a schematic diagram of the result after the POI dataset on the map is divided into multiple subsets by a hierarchical partition tree.

FIG. 3 is a schematic diagram of the enlarged result of the dense area in FIG. 2 .

Detailed ways

As shown in Figure 1, the visual query method of the hierarchical integral histogram of the present embodiment includes the following steps:

Step 1: For a high-dimensional data set V with n dimensions D={D ₁ ,…,D _n }, the domains of each dimension are expressed as {[a ₁ ,b ₁ ],…,[a _n ,b _n ]}, the term binning is a user-defined scale for aggregating the data space, and the user specifies the dimension for binning, filtering and aggregation from a high-dimensional dataset, as shown in wireframe a in Figure 1.

Step 2: based on the data obtained in the configuration process in step 1, the system first adopts the space division algorithm of the R tree. In the specific process of this embodiment, the variant R* tree of the R tree is used to recursively divide the entire data space, thereby A hierarchical tree structure is generated, as shown in the wireframe b in Figure 1, as shown in Figure 2 and Figure 3, and Figure 3 shows that the dense area is divided into more subspaces, and the division results are better. The data space is reconstructed as V'={v' ₁ ,...v' _p }, where each v' _i ∈ V' corresponds to a leaf node of the R-tree. Then calculate the integral histogram on all the leaf nodes, as shown in the line box c in Figure 1, it is an extension of the summation table, the English name of the summation table is summed area table, it is a two-dimensional table, in the table The value of each grid is equal to the sum of all values in its upper left corner, so the value in each grid can be obtained by adding and subtracting four values.

Unlike the original sum table, which stored a single scalar value in each grid, the integral histogram summarizes the distribution of data points falling in each grid, counting all data points within the range of each grid on the leaf nodes , and returns the result of the query in a manner similar to calculating the rectangular area value through a summation table.

For a d-dimensional dataset binned by an N ₁ ×…N _d grid and summarized by a histogram with b number of bins, the integral histogram of a leaf node is defined as:

Among them, x ₁ ,…,x _d is the index of binning in d dimensions, b is the index of binning in the histogram, h(x ₁ ,…,x _d ,·) represents the value of each grid histogram, so the integral histogram of any rectangular region in data space can be computed by:

where ^{xp is the corner point of the rectangular area, p∈{0,1}d .}

Step 3: The user defines a query range and an aggregate function A, the two form an aggregate query, denoted as Q(R,A), which aggregates the range data points within.

After obtaining the range of each query area, the value of each binning area can be queried in constant time through the integral histogram in step 2, as shown in the wireframe e in Figure 1, and a result histogram is returned, so that Approximate aggregation results can be estimated.

Step 4: After obtaining the approximate aggregated results, the user can perform some visualization operations on it, as shown in the wireframe d in Figure 1, and since the pre-computed hierarchical integral histogram is stored in memory, the visualization and aggregation query The construction is performed online, and users can interactively adjust the parameters of the visualization and get instant visual feedback during the visualization process.

Claims (6)

1. a kind of Visual Inquiry method of level integration histogram, which comprises the following steps:

Step 1: raw data set being configured, including discretization interval number, the condition of filter data is crossed and is polymerize The dimension of statistics；

Step 2: the data handled based on the configuration in step 1, with the building of offline pretreatment mode and storage hierarchy divides Tree, wherein data are divided into multiple data subsets by level partition tree, and the statistical nature of each data subset is by integration histogram It is expressed；

Step 3: specific zonule is discretized into for space uniform is visualized by the configuration in step 1, it is small for each piece Distinguishing hierarchy tree in the coordinate input step 2 of zonule is carried out range query by region, and the range query is searching and mesh There is the data subset of intersection in mark region and goes the process of estimation target area statistical nature with the integration histogram of the intersection, owns Zonule obtains a matrix about statistical nature after being all performed range query；

Step 4: visual element binding being carried out to the matrix of the statistical nature of step 3, carries out visualization request.

2. the Visual Inquiry method of level integration histogram as described in claim 1, which is characterized in that in step 4, further include Visual parameter is interactively adjusted, and obtains instant visible feedback result in visualization process.

3. the Visual Inquiry method of level integration histogram as described in claim 1, which is characterized in that in step 2, original number According to collection for n dimension D={ D₁..., D_nHigh Dimensional Data Set V, the domain of each dimension is expressed as { [a₁, b₁] ..., [a_n, b_n]}。

4. the Visual Inquiry method of level integration histogram as described in claim 1, which is characterized in that in step 2, data quilt Distinguishing hierarchy tree is divided into multiple data subset detailed processes are as follows: entire data space is carried out recurrence division, generates one point The tree construction of layer, data space are reconfigured as V '={ v '₁... v '_i...v′_p, wherein each v '_i∈ V ' corresponds to tree One leaf node.

5. the Visual Inquiry method of level integration histogram as claimed in claim 4, which is characterized in that in step 2, the product Dividing histogram is a kind of extension of summation table, and the value of each grid is equal to the summation of all values in its upper left corner in table, in It is that value in each grid can be obtained by the plus-minus of four values.

6. the Visual Inquiry method of level integration histogram as claimed in claim 5, which is characterized in that in step 2, calculate product Dividing histogram, detailed process is as follows: for by N₁×...N_dGrid carries out the d dimension data collection of branch mailbox, and by with b points The histogram of case number is summarized, the integration histogram of leaf node is defined as:

Wherein, x₁..., x_dIt is the index of the branch mailbox in d dimension, b is the index of branch mailbox in histogram, h (x₁..., x_d) Indicate the histogram of each grid intermediate value；

The integration histogram of any rectangular area can be calculated by following manner in data space:

Wherein x^pIt is the angle point of rectangular area, p ∈ { 0,1 }^d。