CN111581394B - A Large-Scale Knowledge Topographic Mapping Method - Google Patents

Abstract

The invention discloses a large-scale knowledge topography drawing method, which is a relatively simple method for drawing knowledge topography aiming at large-scale text data, wherein a TSNE algorithm is used for carrying out planar layout of documents, a planar pixel point density function taking the document aggregation degree as a parameter is established so as to map the color of points on a plane, then an HSV mode is adopted for carrying out pixel point color setting and dividing into a plurality of fixed grades, the knowledge topography is rendered, the visual expressive force is good, and the realization technology is simple.

Description

A Large-Scale Knowledge Topographic Mapping Method

technical field

The invention relates to a text analysis method, which belongs to the field of information processing, in particular to a method for drawing a large-scale knowledge topographic map.

Background technique

The knowledge topographic map realizes the visualization of text data through the contour map similar to the geographic information system, and distinguishes the amount of data and the relationship between data by using the depth of color. It is also called landscape map or theme map in some literatures. Although the names and expressions are not exactly the same, their basic ideas are consistent. Knowledge topographic maps are mainly used in the analysis of text data, such as patent text data, paper text data, Weibo WeChat and other network text data, to reveal the knowledge content expressed in text language.

The knowledge topographic map can use the subject words in the text to carry out knowledge representation and graphic drawing, such as a method and system for text analysis using the knowledge topographic map disclosed in Chinese patent application CN106021228A (public date 20161012), which is to extract topics from the text Words are used to draw knowledge topographic maps and express knowledge. There are also technical solutions that use document distance for document clustering, knowledge extraction, and graphic drawing. A typical example is the patent map of Canada's Clarivate Innovation. The algorithm of this map is relatively complicated. There are heat maps, topographic maps, rainbow rabbits, weather maps and other forms in graphic drawing.

When there is a lot of text data and the scale of subject terms is large (for example, there are tens of thousands of documents or subject terms), it is very difficult to draw a knowledge topographic map with a clear structure and accurately revealing the content of the text. CN106021228A has more than 1000 subject terms, Due to the limitations of the layout algorithm itself, the readability of the drawn knowledge topographic map is greatly reduced, and it cannot reflect the relationship between the subjects to which the text belongs, but only the display of the structural characteristics of the text. The patent map of Clarivate Innovation in Canada can display a relatively large number of nodes, and can also reflect the relationship between the subjects to which the text belongs. However, the algorithm of the map is complicated and the technical implementation is difficult.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention aims to provide a method for drawing a large-scale knowledge topographic map, which has good visual expression and simple implementation technology.

In order to achieve the above object, the present invention adopts the following technical solutions:

A method for drawing a large-scale knowledge topographic map, comprising the following steps:

S1. Obtain the subject terms of each document by word segmentation technology, and use the subject terms of all documents to establish a subject term matrix;

S2. Input the subject term matrix established in step S1 into the TSNE algorithm. The TSNE algorithm uses the subject term matrix to map documents to a two-dimensional plane, and each document in the two-dimensional plane is represented by a spherical node; the document corresponding to each spherical node is The affiliated subjects are distinguished by different node colors, and the spherical nodes corresponding to documents belonging to the same subject have the same color;

S3. Construct a density function of pixel points based on the number and coordinates of spherical nodes within a unit area of a two-dimensional plane;

像素点P的坐标(x,y)，定义像素点的密度函数公式为：Note that the coordinates of spherical nodes corresponding to N documents are (x _i , y _i ), i=1,...,N, and the average two-dimensional Euclidean distance between spherical nodes corresponding to documents is

The coordinates (x, y) of the pixel point P define the density function formula of the pixel point as:

Among them, N _p is the number of spherical nodes covered in the unit area centered on the pixel point P. The values of α and β determine the slope effect of the topographic map. The larger the value of α and β, the smaller the slope of the topographic map , the more prominent the peak effect of the topographic map is;

S4. Calculate the color value of each pixel and perform topographic map rendering;

S4.1, standardize the density function of the pixels obtained in step S3, so that its value is a floating point number between 0-1;

S4.2, establish an HSV pattern palette, the values of H and V are fixed;

S4.3. Establish a one-to-one mapping relationship between the normalized density value of the pixel and the HSV mode palette: record the normalized density value of the pixel as q, and the corresponding HSV color value of the pixel on the HSV mode palette In , the values of H and V are fixed, and the value of S is q*100%;

S4.4. Divide the value of the S value of the pixel point color into M grade values, and accordingly adjust the S value of each pixel calculated in step S4.3 to the corresponding grade value; the formula is as follows:

M>=3.

Further, in step S1, the subject term matrix is as follows:

Among them, Keyword ₁ ,...,Keyword _l represent the extracted l keywords; D ₁ ,...,D _N represent N documents; e _ij represents the number of keywords Keyword _j contained in the document D _i , i=1, ..., N, j=1, ..., l.

Further, in step S4.1, the following formula is used to standardize the density function of pixels:

Density _max is the maximum value among the density values of all pixels.

The beneficial effect of the present invention is that: the present invention provides a relatively simple method for drawing a knowledge topographic map for large-scale text data. The method uses the TSNE algorithm to carry out the plane layout of documents, and establishes a plane pixel with document aggregation degree as a parameter. The point density function is used to map the color of the point on the plane, and then the HSV mode is used to set the pixel color, and it is divided into fixed multiple levels to render the knowledge topographic map. It has good visual expression and simple implementation technology.

Description of drawings

1 is a schematic diagram of the relationship between the number N of all document nodes and the number N _p of document nodes in a unit area centered on a pixel point P in an embodiment of the present invention;

Fig. 2 is the rendering effect knowledge terrain map of 5000 nodes in the embodiment of the present invention;

Fig. 3 is an effect diagram of rendered knowledge terrain of 15,000 nodes in the embodiment of the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution, and provides detailed implementation and specific operation process, but the protection scope of the present invention is not limited to the present invention. Example.

This embodiment provides a method for drawing a large-scale knowledge topographic map, including the following steps:

S1. Obtain the subject terms of each document by word segmentation technology, and use the subject terms of all documents to establish a subject term matrix. The subject term matrix is as follows:

Among them, Keyword ₁ ,...,Keyword _l represent the extracted l keywords; D ₁ ,...,D _N represent N documents; e _ij represents the number of keywords Keyword _j contained in the document D _i , i=1, ...,N,j=1,...,l;

S2. Input the subject term matrix established in step S1 into the TSNE algorithm. The TSNE algorithm uses the subject term matrix to map documents to a two-dimensional plane, and each document in the two-dimensional plane is represented by a spherical node; the document corresponding to each spherical node is The affiliated subjects are distinguished by different node colors, such as organization, author, country and other information, and the spherical nodes corresponding to the documents belonging to the same subject have the same color.

Different from the prior art CN106021228A, in the method of this embodiment, the nodes on the two-dimensional plane are document nodes, not subject word nodes. The plane layout algorithm adopted is TSNE algorithm, not Fruchterman-Reingoldlayout and VosMapping algorithm. The advantage of the TSNE algorithm is that it can still display a relatively clear structure in large-scale documents.

S3. Construct a pixel density function based on the number and coordinates of spherical nodes within a unit area of a two-dimensional plane:

After the coordinates of each spherical node on the two-dimensional plane are determined, it needs to be drawn on the computer screen, and in order to achieve the rendering effect of the topographic map, the color of each pixel needs to be determined. To this end, a density function needs to be established to map the color value of each pixel;

像素点P的坐标(x,y)，定义像素点的密度函数公式为：Note that the coordinates of spherical nodes corresponding to N documents are (x _i , y _i ), i=1,...,N, and the average two-dimensional Euclidean distance between spherical nodes corresponding to documents is

The coordinates (x, y) of the pixel point P define the density function formula of the pixel point as:

Among them, N _p is the number of spherical nodes covered in the unit area centered on the pixel point P (the number of not all spherical nodes, the purpose is to speed up the execution of the algorithm and save running time), the values of a and β determine the terrain The slope effect of the map, the larger the value of a, β, the smaller the slope of the topographic map, and the more significant the peak effect of the topographic map. Figure 1 shows the relationship between N and _Np .

The unit area generally adopts 10 pixels*10 pixels, 100 pixels*100 pixels, etc.

S4. Calculate the color value of each pixel and perform topographic map rendering;

S4.1, standardize the density function of the pixels obtained in step S3, so that its value is a floating point number between 0-1;

Specifically, the following conversion methods can be used:

Density _max is the maximum value of the density values of all pixels;

S4.2, establish an HSV mode palette, the values of H and V are fixed; HSV refers to hue (H, Hue), saturation (S, Saturation), lightness (V, Value);

There are no specific restrictions on the values of H and V, and they can be any color. For example, the H and V values corresponding to blue and green are fixed. These two values represent the overall color of the topographic map;

S4.3. Establish a one-to-one mapping relationship between the density value after pixel standardization and the HSV mode palette:

Note that the normalized density value of the pixel point is q, and the pixel point is in the HSV color value corresponding to the HSV mode palette, the H and V values are fixed, and the S value is q*100%;

For example, if the normalized density value of a pixel is 0.1, the pixel is in the corresponding color on the HSV mode palette, the values of H and V are fixed, and the value of S is 10%.

S4.4. Divide the value of the S value of the pixel point color into M grade values, and accordingly adjust the S value of each pixel calculated in step S4.3 to the corresponding grade value; the formula is as follows:

M>=3;

The purpose of division is to make the final knowledge topographic map have a sense of hierarchy.

For example, if the value of the S value is divided into 10 levels, the S value in the HSV color of the pixel is adjusted according to the following formula:

For example, when the S value of the HSV color of a certain pixel is 11%, adjust it to 20% according to the above formula, so that the adjusted graphic can achieve a more layered topographic map effect.

Figures 2 and 3 are the rendering knowledge topographic maps of 5000 and 15000 nodes respectively, where the numbers indicate the number of each document.

For those skilled in the art, various corresponding changes and modifications can be made according to the above technical solutions and concepts, and all these changes and modifications should be included in the protection scope of the claims of the present invention.

Claims (2)

1. A large-scale knowledge topographic map drawing method is characterized in that, comprising the steps: S1. Obtain the subject terms of each document by word segmentation technology, and use the subject terms of all documents to establish a subject term matrix; S2. Input the subject term matrix established in step S1 into the TSNE algorithm. The TSNE algorithm uses the subject term matrix to map documents to a two-dimensional plane, and each document in the two-dimensional plane is represented by a spherical node; the document corresponding to each spherical node is The affiliated subjects are distinguished by different node colors, and the spherical nodes corresponding to documents belonging to the same subject have the same color; S3. Construct a density function of pixel points based on the number and coordinates of spherical nodes within a unit area of a two-dimensional plane; Note that the coordinates of spherical nodes corresponding to N documents are (x _i , y _i ), i=1,...,N, and the average two-dimensional Euclidean distance between spherical nodes corresponding to documents is

The coordinates (x, y) of the pixel point P define the density function formula of the pixel point as:

Among them, N _p is the number of spherical nodes covered in the unit area centered on the pixel point P. The values of α and β determine the slope effect of the topographic map. The larger the value of α and β, the smaller the slope of the topographic map , the more prominent the peak effect of the topographic map is; S4. Calculate the color value of each pixel and perform topographic map rendering; S4.1, standardize the density function of the pixels obtained in step S3, so that its value is a floating point number between 0-1; S4.2, establish an HSV pattern palette, the values of H and V are fixed; S4.3. Establish a one-to-one mapping relationship between the normalized density value of the pixel and the HSV mode palette: record the normalized density value of the pixel as q, and the corresponding HSV color value of the pixel on the HSV mode palette In , the values of H and V are fixed, and the value of S is q*100%; S4.4. Divide the value of the S value of the pixel point color into M grade values, and accordingly adjust the S value of each pixel calculated in step S4.3 to the corresponding grade value; the formula is as follows:

M>=3; In step S1, the subject term matrix is as follows:

Among them, Keyword ₁ ,...,Keyword _l represent the extracted l keywords; D ₁ ,...,D _N represent N documents; e _ik represents the number of keywords Keyword _k contained in the document D _i , i=1, ..., N, k=1, ..., l. 2. The method according to claim 1, characterized in that, in step S4.1, the following formula is used to standardize the density function of pixels:

Density _max is the maximum value among the density values of all pixels.

Images