CN112883186A - Method, system, equipment and storage medium for generating information map - Google Patents
Method, system, equipment and storage medium for generating information map Download PDFInfo
- Publication number
- CN112883186A CN112883186A CN201911199428.2A CN201911199428A CN112883186A CN 112883186 A CN112883186 A CN 112883186A CN 201911199428 A CN201911199428 A CN 201911199428A CN 112883186 A CN112883186 A CN 112883186A
- Authority
- CN
- China
- Prior art keywords
- information
- text
- image
- coordinate
- document
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000010586 diagram Methods 0.000 claims description 18
- 239000003086 colorant Substances 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 11
- 230000003993 interaction Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 abstract description 8
- 238000013528 artificial neural network Methods 0.000 description 34
- 238000013507 mapping Methods 0.000 description 22
- 230000006870 function Effects 0.000 description 18
- 230000002860 competitive effect Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/338—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/34—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/18—Legal services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/18—Legal services
- G06Q50/184—Intellectual property management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Tourism & Hospitality (AREA)
- Technology Law (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Computational Linguistics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Operations Research (AREA)
- Entrepreneurship & Innovation (AREA)
- User Interface Of Digital Computer (AREA)
- Document Processing Apparatus (AREA)
Abstract
The embodiment of the invention discloses a method, a system, equipment and a storage medium for generating an information map. The generation method of the information map comprises the following steps: acquiring a plurality of first documents, wherein each first document comprises first image information and/or first text information; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model; generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features; and generating a first information map of the plurality of first documents according to the third coordinates. The embodiment of the invention realizes visual information display.
Description
Technical Field
The present invention relates to information display technologies, and in particular, to a method, a system, a device, and a storage medium for generating an information map.
Background
With the rapid development of internet technology and the increasing updating and application of multimedia devices, the importance of information acquisition to the life of people is increasingly highlighted.
However, in the process of acquiring information, the existing mode cannot very intuitively show the information which the user wants to acquire. For example, a user wants to know all patent information of a company, and only can obtain patent information displayed in a list, and for patent information which is huge in quantity and is disordered, the user needs a lot of time and energy to obtain useful information desired by the user.
Even if the user can classify the patent information by himself or herself so as to research, for the patent document including the text and the drawings, the patent document cannot be classified by the text or the drawings, so that it is difficult for the user to quickly and accurately obtain the information view which is needed and intuitive, which undoubtedly greatly reduces the work efficiency and the use experience of people.
Disclosure of Invention
The embodiment of the invention provides a method, a system, equipment and a storage medium for generating an information map so as to realize visual information display.
To achieve the object, an embodiment of the present invention provides a method for generating an information map, where the method for generating an information map includes:
acquiring a plurality of first documents, wherein each first document comprises first image information and/or first text information;
when the first document comprises first image information, extracting first image embedding characteristics of the first image information by using a pre-trained first model; when the first document comprises first text information, extracting first text embedding characteristics of the first text information by using a pre-trained second model; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model;
when the first documents comprise first image information, generating first coordinates of each first document according to the first image embedding characteristics; when the first documents comprise first text information, generating second coordinates of each first document according to the first text embedding characteristics; when the first documents comprise first image information and first text information, generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features;
and generating a first information map of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate.
Further, the first information map is displayed in a scatter diagram, and the plurality of first documents are represented by points at a first coordinate position, a second coordinate position or a third coordinate position.
Further, the scattered points in the first information map have different colors, and the colors of the points are set according to the author, the patent applicant or the patentee of the document corresponding to the points, or the colors of the points are set according to the text information of the document corresponding to the points.
Furthermore, the presentation form of the first information map supports user interaction, and when a point selected, clicked or hovered by a user is received, image information and/or text information of a document corresponding to the point is displayed in a specific area of a page.
Furthermore, the presentation form of the first information map is an image list, each document is displayed by using the representative image thereof, and the position of each image is arranged according to the position of the first coordinate, the second coordinate or the third coordinate of the corresponding document.
Furthermore, the display form of the first information map supports user interaction, and when a certain area in the map selected by a user is received, the map in the selected area range is displayed in an enlarged mode.
Further, said generating a first concept-embedded feature from said first image-embedded feature and a first text-embedded feature comprises:
and generating the first concept embedding feature according to the first image embedding feature and the first text embedding feature by utilizing a pre-trained third model, wherein the loss function used in the training process of the third model comprises a relative hinge loss function and/or an absolute hinge loss function.
Further, the generating the first information maps of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate includes:
acquiring one or more second documents, wherein each second document comprises second image information and/or second text information;
when the second document comprises second image information, extracting second image embedding characteristics of the second image information by using a pre-trained first model; when the second document comprises second text information, extracting second text embedding characteristics of the second text information by using a pre-trained second model; when the second document comprises second image information and second text information, extracting second image embedding features of the second image information by using a pre-trained first model, and extracting second text embedding features of the second text information by using a pre-trained second model;
when the second documents comprise second image information, generating fourth coordinates of each second document according to the second image embedding characteristics; when the second documents comprise second text information, generating a fifth coordinate of each second document according to the second text embedding characteristics; when the second document comprises second image information and second text information, generating second concept embedding features according to the second image embedding features and the second text embedding features, and generating a sixth coordinate of each second document according to the second concept embedding features;
displaying the fourth coordinate, the fifth coordinate, or the sixth coordinate in the first information map.
Further, the generating a second concept-embedded feature from the second image-embedded feature and a second text-embedded feature comprises:
and generating the second concept embedding feature according to the second image embedding feature and the second text embedding feature by utilizing a pre-trained third model, wherein the loss function used in the training process of the third model comprises a relative hinge loss function and/or an absolute hinge loss function.
Further, the displaying the fourth coordinate, the fifth coordinate, or the sixth coordinate in the first information map includes:
acquiring a fourth coordinate, a fifth coordinate or a sixth coordinate, wherein the minimum distance between the fourth coordinate, the fifth coordinate or the sixth coordinate and the first coordinate, the second coordinate or the third coordinate is smaller than a preset value;
highlighting the fourth coordinate, the fifth coordinate or the sixth coordinate, wherein the minimum distance is smaller than a preset value.
Further, the first model includes an image neural network and an image mapping neural network, and the second model includes a text neural network and a text mapping neural network.
Further, the extracting the first image-embedded feature of the first image information by using the pre-trained first model, and the extracting the first text-embedded feature of the first text information by using the pre-trained second model includes:
extracting a first image vector of the first image information by using a pre-trained image neural network;
mapping the first image vector to a public space in which the images and the texts are jointly embedded by utilizing a pre-trained image mapping neural network to transform the first image vector into a first image embedding feature;
extracting a first text vector of the first text information by using a pre-trained text neural network;
and mapping the first text vector to the image-text joint embedding public space by utilizing a pre-trained text mapping neural network to be transformed into a first text embedding characteristic.
On one hand, the embodiment of the invention also provides a system for generating the information map, and the system for generating the information map comprises the following components:
the document acquisition module is used for acquiring a plurality of first documents, wherein each first document comprises first image information and/or first text information;
the feature extraction module is used for extracting a first image embedding feature of the first image information by using a pre-trained first model when the first document comprises the first image information; when the first document comprises first text information, extracting first text embedding characteristics of the first text information by using a pre-trained second model; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model;
the coordinate generation module is used for generating a first coordinate of each first document according to the first image embedding characteristics when the first document comprises first image information; when the first documents comprise first text information, generating second coordinates of each first document according to the first text embedding characteristics; when the first documents comprise first image information and first text information, generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features;
and the map generation module is used for generating a first information map of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate.
On the other hand, an embodiment of the present invention further provides a device for generating an information map, where the device includes: one or more processors; a storage device, configured to store one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the generation method provided in any embodiment of the present invention.
In still another aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the generating method provided in any embodiment of the present invention.
The method comprises the steps of obtaining a plurality of first documents, wherein each first document comprises first image information and/or first text information; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model; generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features; and generating the first information maps of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate, so that the problem that the existing information display mode is not visual enough is solved, and the visual information display effect is realized.
Drawings
Fig. 1 is a flowchart of a method for generating an information map according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a public space provided by an embodiment of the present invention;
FIG. 3 is a diagram of a first information map according to an embodiment of the present invention;
FIG. 4 is a diagram of a first information map according to an embodiment of the present invention;
FIG. 5 is a diagram of a first information map according to an embodiment of the present invention;
FIG. 6 is a diagram of a first information map according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating a method for generating an information map according to a second embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a system for generating an information map according to a third embodiment of the present invention;
fig. 9 is a schematic structural diagram of a device for generating an information map according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are for purposes of illustration and not limitation. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the steps as a sequential process, many of the steps can be performed in parallel, concurrently or simultaneously. In addition, the order of the steps may be rearranged. A process may be terminated when its operations are completed, but may have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.
Furthermore, the terms "first," "second," and the like may be used herein to describe various orientations, actions, steps, elements, or the like, but the orientations, actions, steps, or elements are not limited by these terms. These terms are only used to distinguish one direction, action, step or element from another direction, action, step or element. For example, a first document may be termed a second document, and, similarly, a second document may be termed a first document, without departing from the scope of the present application. The first document and the second document are both documents, but they are not the same document. The terms "first", "second", etc. are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the embodiments of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Example one
As shown in fig. 1, an embodiment of the present invention provides a method for generating an information map, where the method includes:
s110, obtaining a plurality of first documents, wherein each first document comprises first image information and/or first text information.
In this embodiment, the first document may be one or more of a patent document, a paper, a web document, a journal document, and a book document, the first document may include first image information, first text information, or both the first image information and the first text information, and for example, if the first document is a patent document, the patent document generally includes words and drawings, that is, the first image information and the first text information. Specifically, the user may upload a plurality of first documents, where each of the first documents may include a plurality of first text information and first image information.
S120, when the first document comprises first image information, extracting first image embedding characteristics of the first image information by using a pre-trained first model; when the first document comprises first text information, extracting first text embedding characteristics of the first text information by using a pre-trained second model; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model.
S130, when the first documents comprise first image information, generating a first coordinate of each first document according to the first image embedding characteristics; when the first documents comprise first text information, generating second coordinates of each first document according to the first text embedding characteristics; when the first documents comprise first image information and first text information, generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features.
In this embodiment, the first model includes an image neural network and an image mapping neural network, and the second model includes a text neural network and a text mapping neural network. The Image neural network can use ResNet or MobileNet pre-trained on ImageNet or Google Open Image, the text neural network can use word2vec, GloVe or BERT, and the Image and the text can be embedded in two different vector spaces through the embedding of the Image neural network and the text neural network, and the Image embedding and the text embedding in different vector spaces are also required to be bridged into the same public space through a multilayer perceptron or a graph convolution network, namely the Image mapping neural network and the text mapping neural network, so as to obtain a first Image embedding feature and a first text embedding feature which are located in the same public space.
As shown in fig. 2, in a public space, the first image information is trained by the first model to obtain a first image embedding feature, i.e., a first coordinate point 201, and the first text information is trained by the second model to obtain a first text embedding feature, i.e., a second coordinate point 202, i.e., the first image information and the first text information can be represented and obtained in the same public space. In addition, because of different semantics, when the user inputs languages of multiple countries, the obtained first text embedding features, such as the third coordinate point 203, the fourth coordinate point 204 and the fifth coordinate point 205, have different distances from the first coordinate point 201.
Specifically, if the first document only comprises first image information, extracting a first image vector of the first image information by using a pre-trained image neural network, mapping the first image vector into a public space in which images and texts are jointly embedded by using the pre-trained image mapping neural network to convert the first image vector into a first image embedding feature, and generating a first coordinate of each first document, namely a plurality of coordinate points in the public space, according to the first image embedding feature; if the first document only comprises first text information, extracting a first text vector of the first text information by using a pre-trained text neural network, mapping the first text vector into a public space in which the pictures and the texts are jointly embedded by using the pre-trained text mapping neural network to convert the first text vector into first text embedding characteristics, and generating a second coordinate of each first document according to the first text embedding characteristics, namely a plurality of coordinate points in the public space; if the first document comprises both the first image information and the first text information, extracting a first image vector of the first image information by using a pre-trained image neural network, extracting a first text vector of the first text information by using the pre-trained text neural network, mapping the first image vector into a public space for image-text joint embedding by using the pre-trained image mapping neural network, mapping the first text vector into the public space for image-text joint embedding by using the pre-trained text mapping neural network, obtaining a first image embedding characteristic and a first text embedding characteristic, generating a first concept embedding characteristic according to the first image embedding characteristic and the first text embedding characteristic, and generating a third coordinate of each first document according to the first concept embedding characteristic, i.e. a plurality of coordinate points in a common space.
The first image embedding feature and the first text embedding feature may generate the first concept embedding feature according to the first image embedding feature and the first text embedding feature by using a pre-trained third model, wherein the loss function used in the third model training process comprises a relative hinge loss function and/or an absolute hinge loss function, and preferably, the first concept embedding feature is generated by using the weighting of the hinge loss function and the absolute hinge loss function in the third model training process, and the first image embedding feature and the first text embedding feature are converted into the first concept embedding feature, that is, two coordinate points representing the same patent document in a public space are converted into one coordinate point.
S140, generating a first information map of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate.
In this embodiment, the first information map is presented in the form of a scatter diagram, and the plurality of first documents are represented by points at a first coordinate position, a second coordinate position, or a third coordinate position. The scattered points in the first information map have different colors, and the colors of the points are set according to the author, the patent applicant or the patentee of the document corresponding to the points, or the colors of the points are set according to the text information of the document corresponding to the points. Specifically, as shown in fig. 3, after the user uploads all the first documents, a scatter diagram of the first information map is displayed in the first area 301, wherein the color of each point may correspond to the classification corresponding to each document, and classification information corresponding to each color, such as the aforementioned author, patent applicant, patentee or text information, is displayed in the second area 302.
Further, the presentation form of the first information map supports user interaction, and when a point selected, clicked or hovered by a user is received, image information, text information and/or other information of a document corresponding to the point are displayed in a specific area of a page, wherein the image information, the text information and/or the other information include but are not limited to authors, patent applicants, patentees, classification numbers and the like. Specifically, as shown in fig. 4, after the user clicks any coordinate point of the scatter diagram in the third area 401, the image information and/or the text information of the corresponding document will be displayed in the fourth area 402.
Further, the user may zoom in on the plot 401, and the zoomed-in plot may be changed to an enlarged plot as shown in the fourth area 501 on the right side of FIG. 5. Specifically, when it is received that a user selects a certain area or a plurality of areas in the scatter diagram, the scatter diagram within the selected area range may be enlarged and displayed to obtain a partial enlarged view of the fifth area 502 shown on the right side of fig. 5, where the partial enlarged view can clearly show denser scatter areas in the original scatter diagram. Of course, the enlarged display of the presentation figure also supports the user interaction described above.
In an alternative embodiment, as shown in fig. 6, the first information map is displayed in the form of an image list, each document is displayed with its representative image, if the document is a patent document, the representative image may be a drawing of the patent document, and the position of each image is arranged according to the position of the first coordinate, the second coordinate or the third coordinate of the corresponding document. The display form of the first information map supports user interaction, and when a certain area in the map selected by a user is received, the map in the selected area range is displayed in an enlarged mode.
The method comprises the steps of obtaining a plurality of first documents, wherein each first document comprises first image information and/or first text information; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model; generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features; and generating the first information maps of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate, so that the problem that the existing information display mode is not visual enough is solved, and the visual information display effect is realized.
Example two
As shown in fig. 7, a second embodiment of the present invention provides a method for generating an information map, and the second embodiment of the present invention further optimizes on the basis of the first embodiment of the present invention, and after step S140 of the first embodiment of the present invention, the method further includes:
s210, one or more second documents are obtained, and each second document comprises second image information and/or second text information.
S220, when the second document comprises second image information, extracting second image embedding characteristics of the second image information by using a pre-trained first model; when the second document comprises second text information, extracting second text embedding characteristics of the second text information by using a pre-trained second model; and when the second document comprises second image information and second text information, extracting second image embedding features of the second image information by using a pre-trained first model, and extracting second text embedding features of the second text information by using a pre-trained second model.
S230, when the second documents comprise second image information, generating a fourth coordinate of each second document according to the second image embedding characteristics; when the second documents comprise second text information, generating a fifth coordinate of each second document according to the second text embedding characteristics; and when the second document comprises second image information and second text information, generating a second concept embedding feature according to the second image embedding feature and the second text embedding feature, and generating a sixth coordinate of each second document according to the second concept embedding feature.
The implementation method of steps S210 to S230 in this embodiment is the same as that in the first embodiment of the present invention.
And S240, displaying the fourth coordinate, the fifth coordinate or the sixth coordinate in the first information map.
For example, the first document uploaded by the user may be all patent documents of the company, after the first information map of the company patent is generated, the user may continue to upload the second document, the second document may be all patent documents of a competitive company, after the same steps as those in the embodiment of the present invention are performed, the coordinate corresponding to the second document is displayed in the first information map, and the user may quite definitely analyze the patent competition conditions of the competitive company and the company to which the competitive company belongs, where the display color of the first coordinate, the second coordinate, or the third coordinate corresponding to the first document is different from the display color of the fourth coordinate, the fifth coordinate, or the sixth coordinate corresponding to the second document.
And S250, acquiring a fourth coordinate, a fifth coordinate or a sixth coordinate, wherein the minimum distance between the fourth coordinate, the fifth coordinate or the sixth coordinate and the first coordinate, the second coordinate or the third coordinate is less than a preset value.
And S260, highlighting the fourth coordinate, the fifth coordinate or the sixth coordinate with the minimum distance smaller than a preset value.
In this embodiment, a plurality of first documents are already in the first information map, and after one or more second documents are received, for a second document of a fourth coordinate, a fifth coordinate or a sixth coordinate, whose minimum distance from the first coordinate, the second coordinate or the third coordinate is smaller than a preset value, it is indicated that the concept is similar to the plurality of first documents, and the coordinates of the second document are distributed among the first coordinate, the second coordinate or the third coordinate, so as to be more visually displayed, and a user can more visually distinguish the first document from the second document, and therefore, the fourth coordinate, the fifth coordinate or the sixth coordinate, whose minimum distance is smaller than the preset value, can be highlighted.
Furthermore, coordinate points with high similarity between the company to which the user belongs and a competitive company can be obtained in the first information map, and the coordinate points are highlighted, so that the user can be more intuitively obtained patent information with high competitive strength. Similarly, the user may zoom in on the first information map, obtain image information of highly similar patents, and obtain detailed information of these patents by clicking or other means.
Further, the user can continue to input patent documents of a plurality of competitive companies, and accordingly the patent documents can be displayed in the first information map in a distinguishing mode through coordinate points of different colors.
According to the embodiment of the invention, a fourth coordinate, a fifth coordinate or a sixth coordinate is obtained, wherein the distance between the fourth coordinate, the fifth coordinate or the sixth coordinate and the first coordinate, the second coordinate or the third coordinate is less than a preset value; acquiring a fourth coordinate, a fifth coordinate or a sixth coordinate, wherein the distance between the fourth coordinate, the fifth coordinate or the sixth coordinate and the first coordinate, the second coordinate or the third coordinate is less than a preset value; and highlighting the fourth coordinate, the fifth coordinate or the sixth coordinate with the distance smaller than the preset value, so that the problem that the conventional information display mode is not visual enough is solved, and the visual information display effect is realized.
EXAMPLE III
As shown in fig. 8, a third embodiment of the present invention provides a system 100 for generating an information map, where the system 100 for generating an information map provided in the third embodiment of the present invention can execute a method for generating an information map provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method. The generation system 100 includes a document acquisition module 110, a feature extraction module 120, a coordinate generation module 130, and a map generation module 140.
Specifically, the document obtaining module 110 is configured to obtain a plurality of first documents, where each of the first documents includes first image information and/or first text information; the feature extraction module 120 is configured to extract a first image embedding feature of the first image information by using a pre-trained first model when the first document includes the first image information; when the first document comprises first text information, extracting first text embedding characteristics of the first text information by using a pre-trained second model; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model; the coordinate generating module 130 is configured to generate a first coordinate of each first document according to the first image embedding feature when the first document includes first image information; when the first documents comprise first text information, generating second coordinates of each first document according to the first text embedding characteristics; when the first documents comprise first image information and first text information, generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features; the map generating module 140 is configured to generate a first information map of the plurality of first documents according to the first coordinate, the second coordinate, or the third coordinate.
In this embodiment, the first information map is displayed in a form of a scatter diagram, the plurality of first documents are represented by points at a first coordinate, a second coordinate, or a third coordinate, the scatter points in the first information map have different colors, and the colors of the points are set differently according to authors, patent applicants, or patentees of the documents corresponding to the points, or the colors of the points are set differently according to text information of the documents corresponding to the points. The display form of the first information map supports user interaction, and when a point selected, clicked or hovered by a user is received, image information and/or text information of a document corresponding to the point is displayed in a specific area of a page. When receiving a user selection of a certain area in the map, the map within the selected area is displayed in an enlarged mode. The first model includes an image neural network and an image-mapped neural network, and the second model includes a text neural network and a text-mapped neural network. In an alternative embodiment, the first information map is displayed in the form of an image list, each document is displayed as a representative image thereof, and the position of each image is arranged according to the distance between the first coordinate, the second coordinate or the third coordinate of the corresponding document.
Further, the document obtaining module 110 is further configured to obtain one or more second documents, where each of the second documents includes second image information and/or second text information; the feature extraction module 120 is further configured to extract a second image embedding feature of the second image information by using a pre-trained first model when the second document includes the second image information; when the second document comprises second text information, extracting second text embedding characteristics of the second text information by using a pre-trained second model; when the second document comprises second image information and second text information, extracting second image embedding features of the second image information by using a pre-trained first model, and extracting second text embedding features of the second text information by using a pre-trained second model; the coordinate generating module 130 is further configured to generate a fourth coordinate of each second document according to the second image embedding feature when the second document includes second image information; when the second documents comprise second text information, generating a fifth coordinate of each second document according to the second text embedding characteristics; when the second document comprises second image information and second text information, generating second concept embedding features according to the second image embedding features and the second text embedding features, and generating a sixth coordinate of each second document according to the second concept embedding features; the map generation module 140 is further configured to display the fourth coordinate, the fifth coordinate, or the sixth coordinate in the first information map.
Further, the feature extraction module 120 is specifically configured to extract a first image vector of the first image information by using a pre-trained image neural network; mapping the first image vector to a public space in which the images and the texts are jointly embedded by utilizing a pre-trained image mapping neural network to transform the first image vector into a first image embedding feature; extracting a first text vector of the first text information by using a pre-trained text neural network; and mapping the first text vector to the image-text joint embedding public space by utilizing a pre-trained text mapping neural network to be transformed into a first text embedding characteristic. The coordinate generating module 130 is specifically configured to generate the first concept-embedded feature according to the first image-embedded feature and the first text-embedded feature by using a pre-trained third model, and further configured to generate the second concept-embedded feature according to the second image-embedded feature and the second text-embedded feature by using a pre-trained third model, where a loss function used in a training process of the third model includes a relative hinge loss function and/or an absolute hinge loss function.
Further, the generation system 100 of the information map further includes a coordinate highlighting module 150, where the coordinate highlighting module 150 is configured to obtain a fourth coordinate, a fifth coordinate, or a sixth coordinate, where a distance from the first coordinate, the second coordinate, or the third coordinate is smaller than a preset value; highlighting the fourth coordinate, the fifth coordinate or the sixth coordinate, wherein the distance is smaller than the preset value.
Example four
Fig. 9 is a schematic structural diagram of an information map generation device according to a fourth embodiment of the present invention. FIG. 9 shows a block diagram of an exemplary generating device 12 suitable for use in implementing embodiments of the present invention. The generating device 12 shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 9, the generating device 12 is in the form of a general purpose computing device. The components of the generating device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
The generating device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by generating device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The generating device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 9, and commonly referred to as a "hard drive"). Although not shown in FIG. 9, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
Generating device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with generating device 12, and/or with any devices (e.g., network card, modem, etc.) that enable generating device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the generating device 12 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 20. As shown, the network adapter 20 communicates with the other modules of the generating device 12 over the bus 18. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the generating device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, to implement the generation method provided by the embodiment of the present invention:
acquiring a plurality of first documents, wherein each first document comprises first image information and/or first text information;
when the first document comprises first image information, extracting first image embedding characteristics of the first image information by using a pre-trained first model; when the first document comprises first text information, extracting first text embedding characteristics of the first text information by using a pre-trained second model; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model;
when the first documents comprise first image information, generating first coordinates of each first document according to the first image embedding characteristics; when the first documents comprise first text information, generating second coordinates of each first document according to the first text embedding characteristics; when the first documents comprise first image information and first text information, generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features;
and generating a first information map of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate.
EXAMPLE five
The fifth embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the generating method provided in all the embodiments of the present invention of the present application:
acquiring a plurality of first documents, wherein each first document comprises first image information and/or first text information;
when the first document comprises first image information, extracting first image embedding characteristics of the first image information by using a pre-trained first model; when the first document comprises first text information, extracting first text embedding characteristics of the first text information by using a pre-trained second model; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model;
when the first documents comprise first image information, generating first coordinates of each first document according to the first image embedding characteristics; when the first documents comprise first text information, generating second coordinates of each first document according to the first text embedding characteristics; when the first documents comprise first image information and first text information, generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features;
and generating a first information map of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A method for generating an information map, comprising:
acquiring a plurality of first documents, wherein each first document comprises first image information and/or first text information;
when the first document comprises first image information, extracting first image embedding characteristics of the first image information by using a pre-trained first model; when the first document comprises first text information, extracting first text embedding characteristics of the first text information by using a pre-trained second model; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model;
when the first documents comprise first image information, generating first coordinates of each first document according to the first image embedding characteristics; when the first documents comprise first text information, generating second coordinates of each first document according to the first text embedding characteristics; when the first documents comprise first image information and first text information, generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features;
and generating a first information map of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate.
2. The generation method according to claim 1, wherein the presentation form of the first information map is a scatter diagram, and the plurality of first documents are represented by points at first, second, or third coordinate positions.
3. The generation method according to claim 2, wherein the scattered points in the first information map have different colors, and the colors of the points are set differently according to authors, patent applicants, or patentees of documents corresponding to the points, or the colors of the points are set differently according to text information of the documents corresponding to the points.
4. The generation method of claim 2, wherein the presentation form of the first information map supports user interaction, and when a point selected, clicked or hovered by a user is received, image information and/or text information of a document corresponding to the point is displayed in a specific area of a page.
5. The generation method according to claim 1, wherein the first information map is presented in the form of a list of images, each document is displayed with its representative image, and the position of each image is arranged according to the position of the first coordinate, the second coordinate, or the third coordinate of the corresponding document.
6. The generation method of any one of claims 2 to 5, wherein the presentation form of the first information map supports user interaction, and when a user selection of a certain area in the map is received, the map within the selected area is displayed in an enlarged manner.
7. The generation method according to claim 1, wherein the generating a first concept-embedded feature from the first image-embedded feature and a first text-embedded feature comprises:
and generating the first concept embedding feature according to the first image embedding feature and the first text embedding feature by utilizing a pre-trained third model, wherein the loss function used in the training process of the third model comprises a relative hinge loss function and/or an absolute hinge loss function.
8. A system for generating an information map, comprising:
the document acquisition module is used for acquiring a plurality of first documents, wherein each first document comprises first image information and/or first text information;
the feature extraction module is used for extracting a first image embedding feature of the first image information by using a pre-trained first model when the first document comprises the first image information; when the first document comprises first text information, extracting first text embedding characteristics of the first text information by using a pre-trained second model; when the first document comprises first image information and first text information, extracting first image embedding features of the first image information by using a pre-trained first model, and extracting first text embedding features of the first text information by using a pre-trained second model;
the coordinate generation module is used for generating a first coordinate of each first document according to the first image embedding characteristics when the first document comprises first image information; when the first documents comprise first text information, generating second coordinates of each first document according to the first text embedding characteristics; when the first documents comprise first image information and first text information, generating first concept embedding features according to the first image embedding features and the first text embedding features, and generating third coordinates of each first document according to the first concept embedding features;
and the map generation module is used for generating a first information map of the plurality of first documents according to the first coordinate, the second coordinate or the third coordinate.
9. An information map generation apparatus, characterized in that the apparatus comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the generation method of any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the generation method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911199428.2A CN112883186B (en) | 2019-11-29 | 2019-11-29 | Method, system, equipment and storage medium for generating information map |
| PCT/CN2020/131224 WO2021104275A1 (en) | 2019-11-29 | 2020-11-24 | Method for information map generation, system, device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911199428.2A CN112883186B (en) | 2019-11-29 | 2019-11-29 | Method, system, equipment and storage medium for generating information map |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112883186A true CN112883186A (en) | 2021-06-01 |
| CN112883186B CN112883186B (en) | 2024-04-12 |
Family
ID=76038476
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911199428.2A Active CN112883186B (en) | 2019-11-29 | 2019-11-29 | Method, system, equipment and storage medium for generating information map |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112883186B (en) |
| WO (1) | WO2021104275A1 (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103885989A (en) * | 2012-12-24 | 2014-06-25 | 腾讯科技（武汉）有限公司 | Method and device for estimating new word document frequency |
| US20160328383A1 (en) * | 2015-05-08 | 2016-11-10 | International Business Machines Corporation | Generating distributed word embeddings using structured information |
| CN106372087A (en) * | 2015-07-23 | 2017-02-01 | 北京大学 | Information retrieval-oriented information map generation method and dynamic updating method |
| CN107368506A (en) * | 2015-05-11 | 2017-11-21 | 斯图飞腾公司 | Unstructured data analysis system and method |
| CN109992676A (en) * | 2019-04-01 | 2019-07-09 | 中国传媒大学 | Across the media resource search method of one kind and searching system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160034757A1 (en) * | 2014-07-31 | 2016-02-04 | Chegg, Inc. | Generating an Academic Topic Graph from Digital Documents |
| CN109189521A (en) * | 2018-07-31 | 2019-01-11 | 郑州向心力通信技术股份有限公司 | A kind of big data analysis methods of exhibiting and system |
-
2019
- 2019-11-29 CN CN201911199428.2A patent/CN112883186B/en active Active
-
2020
- 2020-11-24 WO PCT/CN2020/131224 patent/WO2021104275A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103885989A (en) * | 2012-12-24 | 2014-06-25 | 腾讯科技（武汉）有限公司 | Method and device for estimating new word document frequency |
| US20160328383A1 (en) * | 2015-05-08 | 2016-11-10 | International Business Machines Corporation | Generating distributed word embeddings using structured information |
| CN107368506A (en) * | 2015-05-11 | 2017-11-21 | 斯图飞腾公司 | Unstructured data analysis system and method |
| CN106372087A (en) * | 2015-07-23 | 2017-02-01 | 北京大学 | Information retrieval-oriented information map generation method and dynamic updating method |
| CN109992676A (en) * | 2019-04-01 | 2019-07-09 | 中国传媒大学 | Across the media resource search method of one kind and searching system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021104275A1 (en) | 2021-06-03 |
| CN112883186B (en) | 2024-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109388807B (en) | Method, device and storage medium for identifying named entities of electronic medical records | |
| US20120054226A1 (en) | Multi-Faceted Visualization of Rich Text Corpora | |
| JP6756079B2 (en) | Artificial intelligence-based ternary check method, equipment and computer program | |
| CN109919077B (en) | Gesture recognition method, device, medium and computing equipment | |
| CN109726712A (en) | Character recognition method, device and storage medium, server | |
| CN107301248B (en) | Word vector construction method and device of text, computer equipment and storage medium | |
| CN109408829B (en) | Method, device, equipment and medium for determining readability of article | |
| JP2022039973A (en) | Method and apparatus for quality control, electronic device, storage medium, and computer program | |
| CN107909088B (en) | Method, apparatus, device and computer storage medium for obtaining training samples | |
| CN113762303B (en) | Image classification method, device, electronic equipment and storage medium | |
| CN106575300A (en) | Image based search to identify objects in documents | |
| CN111124863B (en) | Intelligent device performance testing method and device and intelligent device | |
| CN109684394B (en) | Text generation method, device, equipment and storage medium | |
| CN112507090A (en) | Method, apparatus, device and storage medium for outputting information | |
| CN111950279A (en) | Entity relationship processing method, device, equipment and computer readable storage medium | |
| CN110134965A (en) | Method, apparatus, equipment and computer readable storage medium for information processing | |
| CN112818212B (en) | Corpus data acquisition method, corpus data acquisition device, computer equipment and storage medium | |
| WO2021104274A1 (en) | Image and text joint representation search method and system, and server and storage medium | |
| CN107729944A (en) | A kind of recognition methods, device, server and the storage medium of vulgar picture | |
| CN112925470B (en) | Touch control method and system of interactive electronic whiteboard and readable medium | |
| CN110850982A (en) | AR-based human-computer interaction learning method, system, device and storage medium | |
| US8996984B2 (en) | Automatic visual preview of non-visual data | |
| CN112542163B (en) | Intelligent voice interaction method, device and storage medium | |
| CN116127100A (en) | Knowledge graph construction method, device, equipment and storage medium thereof | |
| CN112883186B (en) | Method, system, equipment and storage medium for generating information map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |