CN112069326A

CN112069326A - Knowledge graph construction method and device, electronic equipment and storage medium

Info

Publication number: CN112069326A
Application number: CN202010916935.XA
Authority: CN
Inventors: 程文龙
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2020-09-03
Filing date: 2020-09-03
Publication date: 2020-12-11

Abstract

The application discloses a method and a device for generating a knowledge graph, electronic equipment and a storage medium, wherein the method for generating the knowledge graph is used for the electronic equipment, and comprises the following steps: acquiring album data corresponding to an album in the electronic equipment; acquiring an entity object from the album data; acquiring the relation between the entity objects based on the album data; acquiring attribute information of the entity object based on the album data; and generating a knowledge graph corresponding to the photo album based on the entity objects, the relationship among the entity objects and the attribute information of the entity objects. The method can realize the construction of the knowledge graph of the photo album and provide a basis for intelligent search and recommendation in the photo album.

Description

Knowledge graph construction method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of electronic device technologies, and in particular, to a method and an apparatus for constructing an album knowledge graph, an electronic device, and a storage medium.

Background

Electronic devices, such as mobile phones, tablet computers, etc., have become one of the most common consumer electronic products in people's daily life. Along with the development of science and technology level, mobile terminal can be provided with the camera usually to the realization is shot the function, makes people can use electronic equipment to take a photograph more and more conveniently, with the nice and instant in the record life, and can save the picture of shooing in the album, so that the user looks over. In addition, the electronic device may also search corresponding information of the album based on the information identified in the album, such as querying for pictures, querying for events taken, and the like, but the accuracy of information search based on the album cannot be guaranteed at present.

Disclosure of Invention

In view of the foregoing problems, the present application provides a method and an apparatus for generating a knowledge graph, an electronic device, and a storage medium.

In a first aspect, an embodiment of the present application provides a method for generating a knowledge graph, which is applied to an electronic device, and the method includes: acquiring album data corresponding to an album in the electronic equipment; acquiring an entity object from the album data; acquiring the relation between the entity objects based on the album data; acquiring attribute information of the entity object based on the album data; and generating a knowledge graph corresponding to the photo album based on the entity objects, the relationship among the entity objects and the attribute information of the entity objects.

In a second aspect, the embodiment of the present application provides an apparatus for generating a knowledge graph, which is applied to an electronic device, and the apparatus includes: the system comprises a data acquisition module, an entity acquisition module, a relationship acquisition module, an attribute acquisition module and a map generation module, wherein the data acquisition module is used for acquiring album data corresponding to an album in the electronic equipment; the entity obtaining module is used for obtaining entity objects from the album data; the relation acquisition module is used for acquiring the relation between the entity objects based on the album data; the attribute acquisition module is used for acquiring attribute information of the entity object based on the album data; the map generation module is used for generating a knowledge map corresponding to the photo album based on the entity objects, the relationship among the entity objects and the attribute information of the entity objects.

In a third aspect, an embodiment of the present application provides an electronic device, including: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of generating a knowledge-graph as provided in the first aspect above.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, and the program code may be called by a processor to execute the method for generating a knowledge graph provided in the first aspect.

The proposal provided by the application acquires the entity objects from the album data by acquiring the album data corresponding to the album in the electronic equipment, acquires the relationship between the entity objects based on the album data, acquires the attribute information of the entity objects based on the album data, then based on the entity objects, the relationship between the entity objects and the attribute information of the entity objects, generating a knowledge graph corresponding to the album, because the generated knowledge graph contains the relationships between the users and other people in the album, between the users and the events and between the other people and the events, and also contains the attribute information of the users, the other people and the events, therefore, the knowledge graph can provide knowledge related to semantics of users, other people and events in the photo album, a basis is provided for intelligent searching and recommending in the photo album, and accuracy of intelligent searching and recommending based on the photo album is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 shows a flow diagram of a method of knowledge-graph generation according to one embodiment of the present application.

FIG. 2 shows a flow diagram of a method of knowledge-graph generation according to another embodiment of the present application.

Fig. 3 shows a schematic diagram of a knowledge-graph of a graph structure provided by an embodiment of the present application.

FIG. 4 shows a flowchart of a method of knowledge-graph generation according to yet another embodiment of the present application.

Fig. 5 shows an interface schematic diagram provided in an embodiment of the present application.

Fig. 6 shows another interface schematic diagram provided in the embodiment of the present application.

FIG. 7 shows a flowchart of a method of knowledge-graph generation according to yet another embodiment of the present application.

Fig. 8 shows a schematic view of another interface provided in the embodiment of the present application.

FIG. 9 shows a block diagram of an apparatus for knowledge-graph generation according to an embodiment of the present application.

Fig. 10 is a block diagram of an electronic device for executing a method for generating a knowledge-graph according to an embodiment of the present application.

Fig. 11 is a storage unit for storing or carrying program code implementing a method for generating a knowledge graph according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

At present, electronic equipment is in daily life use, and the prevalence has been nearly all people's coverage, and wherein, camera module has become intelligent terminal main function point, and the user can shoot the photo through electronic equipment's camera function to record life, study, work are in the twinkling of an eye. The electronic equipment can form the photo album with the pictures shot by people for storage, so that the user can conveniently check the shot pictures.

In addition, the electronic device may also complete some semantic searches based on the album, for example, querying information in pictures of the album, querying the pictures, querying the shooting time, and the like. In the related technology, an album-based search firstly searches a keyword set according to input, then discovers semantic relations among different vocabularies through technologies such as deep learning, discovers sequence relations of languages through a recurrent neural network, and finally displays contents containing keywords to a user through a sorting algorithm.

However, the search engine and the recommendation background of the current photo album lack semantic relations among people, things and objects in the data, and the query result only matches the relation between the user query and the data according to whether the character strings are the same or not, so that some query terms are not contained, but information particularly related to the content of the query terms is lost; some information which is not meaningfully related to the query term is retrieved; due to the fact that semantic relations among people, things and things cannot be understood, the accuracy rate of personalized searching and recommending is low.

In view of the above problems, the inventor provides a method, an apparatus, an electronic device, and a storage medium for generating a knowledge graph, which can extract relationships between entity objects such as users, other people, and events through album data, and also extract attribute information of the entity objects, thereby implementing construction of an album knowledge graph, providing semantic association knowledge for search, recommendation, and the like based on an album, and improving accuracy of intelligent search and recommendation. The specific method for generating the knowledge graph is described in detail in the following examples.

Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a method for generating a knowledge graph according to an embodiment of the present application. In a specific embodiment, the method for generating the knowledge graph is applied to the apparatus 400 for generating the knowledge graph shown in fig. 9 and the electronic device 100 (fig. 10) equipped with the apparatus 400 for generating the knowledge graph. The following will describe a specific flow of the embodiment by taking an electronic device as an example, and it is understood that the electronic device applied in the embodiment may be a smart phone, a tablet computer, a smart watch, a notebook computer, and the like, which is not limited herein. As will be described in detail with respect to the flow shown in fig. 1, the method for generating the knowledge graph may specifically include the following steps:

step S110: and acquiring album data corresponding to the album in the electronic equipment.

In the embodiment of the application, when the electronic device can generate the knowledge graph corresponding to the album, the electronic device can acquire the album data corresponding to the album so as to mine the knowledge content from the album data, so as to generate the knowledge graph of the album.

In some embodiments, album data may include data of the album itself, and may also include data related to the knowledge in the knowledge-graph that may be extracted. The album data may include text, image, video, and other data. For example, the album data may include text, images, and videos in the album, and in addition, text, images, and videos generated in daily use of the electronic device may also be included in the album data. Of course, the album data may be specifically not limited.

The knowledge map is a structured semantic knowledge base, and describes entities (or attributes) in the objective world and their interrelations in a symbolic form. From a graph perspective, a knowledge-graph is essentially a network in which nodes represent entities or attributes of the objective world, and edges represent various relationships between the entities or attributes. Each node in the knowledge-graph corresponds to an entity or attribute information. And each entity may have its own attributes such as name, number, size, etc.

Edges, i.e., relationships, in the knowledge-graph are used to describe the objectively existing associations between nodes. Each edge in the knowledge graph corresponds to a relationship, and each relationship can have own name and weight information. Illustratively, the relationship between entities may be a containment relationship, a top-bottom relationship, or the like.

Attribute information is a depiction of abstract aspects of an entity. It should be noted that an entity generally has many properties, which may be referred to as attributes of the entity. For example, taking the example that the entity is a user, the attributes of the user include: name, event, time, place, etc.

Step S120: and acquiring the entity object from the album data.

In the embodiment of the present application, as can be seen from the foregoing description, the knowledge graph includes nodes, and the entity objects are objects mainly serving as nodes, and the extraction of other information in the knowledge graph also needs to be extracted according to the entity objects. Therefore, the electronic device can extract the entity object from the album data after acquiring the album data of the album. The electronic device can extract the entity object, and can at least extract: the electronic equipment corresponds to a user, an event in the album and a person except the user in the album.

It can be understood that, regarding the album, people and events are mainly used, and therefore, the entity object may include a user corresponding to the electronic device, other users and events corresponding to the images of the album. The event may be an event when a picture is taken in an album, for example, a picture taken at a wedding, and the corresponding event is a wedding, and for example, a picture taken at a birthday party, and the corresponding event is a birthday; the other people may be people other than the user who appear in the album, for example, a group photo of the user with other people, and then other people may exist.

Step S130: and acquiring the relation between the entity objects based on the album data.

In the embodiment of the present application, as can be seen from the foregoing description, the relationship between nodes is also included in the knowledge graph, and the relationship between the nodes is mainly the relationship between the entity objects. Therefore, the electronic device can also extract the relationship between the entity objects from the album data.

In some implementations, the electronic device can extract relationships between the user and other people, relationships between the user and events, and relationships between other people and events based on the album data. Wherein, the relationship between people, e.g., the user's relationship, social relationship; the relationship of a person to an event may be the relationship of a person to the occurrence of an event, for example, the user is in a meeting, and for example, character A is participating in the user's wedding. It can be understood that, because the photo album usually contains pictures of various people and events, by extracting the relationship between the user and other people, the relationship between the user and the events, and the relationship between the other people and the events, the knowledge information in the photo album can be effectively mined, and a basis is provided for the construction of the knowledge graph.

Step S140: and acquiring attribute information of the entity object based on the album data.

In the embodiments of the present application, the attribute information is a description of an abstract aspect of an entity object. The attribute information can effectively represent the attribute characteristics of the entity object, so that the attribute information of the entity object can be acquired for the construction of the knowledge graph.

In some embodiments, the attribute information of the entity object may include a name, time information, location information, birthday, nationality, and the like. For example, the place data includes a home address of the user, an address of a work place, an address of a frequent occurrence place, an address of a travel destination, and the time data includes a date, a anniversary, a birthday, a holiday, and the like. Of course, the specific attribute information may not be limiting.

Step S150: and generating a knowledge graph corresponding to the photo album based on the entity objects, the relationship among the entity objects and the attribute information of the entity objects.

In the embodiment of the application, after the electronic device extracts the entity objects, the relationship between the entity objects, and the attribute information of the entity objects based on the album data, the electronic device may generate the knowledge graph corresponding to the album based on the entity objects, the relationship between the entity objects, and the attribute information of the entity objects.

In some embodiments, the electronic device may construct the knowledge-graph using the entity objects or attribute information as nodes of the knowledge-graph and the relationships between the entity objects or attribute information as edges of the knowledge-graph. In one mode, the user, the other people and the event serve as nodes, and the relationship between the user and the other people, the relationship between the user and the event and the relationship between the other people and the event serve as edges, so that the knowledge graph can contain the entity objects and the relationship between the entity objects, and in addition, the knowledge graph can also contain attribute information of the entity objects. Alternatively, when the attribute information is used as a node, the edge in the knowledge graph may be a relationship between the entity object and the attribute information, for example, node 1 is a user, node 2 is attribute information, and the attribute information is: zhang three, the edge connecting the node 1 and the node 2 is 'name', and the knowledge contained in the edge is 'three of the name of the user'; for another example, if the node 3 is the event 1 and the node 4 is the attribute information "achievement", the edge connecting the node 3 and the node 4 is a place, and the knowledge contained therein is "the place where the event 1 occurs is the achievement". Of course, the knowledge graph may also include the contents of the above two embodiments.

According to the method for generating the knowledge graph, the entity object is obtained from the album data by obtaining the album data corresponding to the album in the electronic equipment, the entity pair at least comprises the user corresponding to the electronic equipment, the event in the album and other characters except the user in the album, the relationship between the entity objects is obtained based on the album data, the attribute information of the entity object is obtained based on the album data, and then the knowledge graph corresponding to the album is generated based on the entity object, the relationship between the entity objects and the attribute information of the entity object The knowledge of semantic association of other people and events provides a basis for intelligent search and recommendation in the photo album, and therefore the accuracy of the intelligent search and recommendation based on the photo album is improved.

Referring to fig. 2, fig. 2 is a schematic flow chart illustrating a method for generating a knowledge graph according to another embodiment of the present application. The method for generating the knowledge graph is applied to the electronic device, and will be described in detail with respect to the flow shown in fig. 2, and the method for generating the knowledge graph may specifically include the following steps:

step S210: and acquiring album data corresponding to the album in the electronic equipment.

In the embodiment of the application, when the electronic device acquires album data corresponding to an album, designated data in the album, the address book, the short message, the calendar and the map of the electronic device can be acquired. The designation data may include, among others, images, text, and video. It will be appreciated that in creating a knowledge graph of an album, in addition to obtaining data such as text, images and video of the album itself, such as photos and videos taken in an album, text information (e.g., time, place, picture name) corresponding to the photos and videos in the album, etc., and also obtains texts, images and videos from data generated in an address book, a short message, a calendar and a map when the electronic device is used, and the data generated in the address book, the short message, the calendar and the map can also contain information about physical objects in the album, for example, the address book includes the relationship between the user and the person, the data corresponding to the map includes the home address, the address of the frequently appearing place, the travel place, etc. of the user, the short message includes the information of the event and the person, etc., and the calendar includes some important dates, such as a memorial day, a birthday, a holiday, etc. Therefore, when knowledge is extracted by using the album data subsequently, more extensive information related to the entity objects in the album can be extracted, so that the generated knowledge graph of the album is more accurate.

In some embodiments, after the electronic device obtains the above specified data, the electronic device may further perform preprocessing on the specified data to obtain structural data, so as to obtain structural data for performing knowledge graph construction subsequently.

Optionally, the electronic device performs preprocessing on the specified data, which may be to perform deduplication, similarity removal and lexical semantic analysis on the obtained specified data, so as to obtain standardized structural data. By processing the specified data in this way, the calculation amount in the knowledge graph of the photo album can be effectively reduced.

Step S220: and acquiring the entity object from the album data.

In the embodiment of the application, after the electronic device obtains the album data, the entity object can be extracted from the album data. When the electronic device extracts the entity object from the album data, the entity object identification model can be used to identify the user corresponding to the electronic device, the event in the album and other people in the album except the user from the album data, so as to obtain the entity object.

In some embodiments, the recognition of the entity object by the entity object recognition model may include: for any given entity, an entity with similar top and bottom characteristics to the entity is extracted from the album data in a machine learning mode, and the entity is clustered and classified, so that an entity object in the album data can be extracted.

In other embodiments, the solid object recognition model may also be a pre-trained neural network model. The neural network model trains an initial neural network through a large number of training samples marked with entity objects, so that an entity object recognition model is obtained, and entity objects in album data can be obtained by inputting the album data into the entity object recognition model.

The entity object recognition model can be pre-established respectively aiming at album data of different data types such as images, texts and videos. The following describes a training process of the entity object recognition model by taking the entity object recognition model corresponding to the image type as an example.

The images labeled with the entity objects can be used as training samples, that is, each training sample can be manually labeled with the entity objects existing therein in advance. In order to improve the recognition capability of the entity object recognition model, entity objects in the entity object recognition model can be labeled as much as possible, for example, all entity objects in a training sample are labeled. And inputting each training sample into an initial neural network model, and adjusting each parameter in the neural network model when the difference between the entity object output by the neural network model and the entity object marked by the training sample is greater than a preset difference, so that the difference between the entity object output by the neural network model corresponding to the training sample and the entity object marked by the training sample is reduced. And comparing and adjusting for multiple times until the difference between the entity object output by the entity object identification model corresponding to the training sample and the entity object marked by the training sample is smaller than the preset difference, so as to obtain a trained entity object identification model, wherein the trained entity object identification model has the identification capability on the entity object in the album data of the image type. The difference between the entity object output by the entity object recognition model and the entity object labeled by the training sample can be represented by the similarity, and when the similarity is smaller than the preset similarity, the difference can be considered to be smaller than the preset difference.

Of course, the entity object recognition model may not be limited specifically.

Step S230: and acquiring the relation between the entity objects based on the album data.

When the electronic device generates the knowledge graph of the photo album, the relationship between the entity objects needs to be acquired. In the embodiment of the application, the electronic device may obtain corpus information including entity objects based on album data, and then extract the relationship between the entity objects from the corpus information.

In some implementations, the electronic device can identify corpus information in the album data using a semantic recognition model. The semantic recognition model can be an equipment model which is trained aiming at sample album data in advance. The semantic recognition model trains an initial model through a large number of training samples marked with corpus information to obtain an entity object recognition model, the corpus information in the album data can be obtained by inputting the album data into the semantic recognition model, and the initial model can be a neural network model and the like, and is not limited in the above.

The corpus recognition model can be pre-established respectively for album data of different data types such as images, texts and videos. The following describes a training process of the corpus recognition model, taking the corpus recognition model corresponding to the image type as an example.

The images marked with the corpus information can be used as training samples, that is, each training sample can be manually marked with entity objects existing in the training sample in advance. In order to improve the recognition capability of the corpus recognition model, the corpus information in the corpus recognition model can be labeled as much as possible, for example, all the corpus information in the training sample is labeled. Inputting each training sample into an initial model, and adjusting each parameter in the initial model when the difference between the corpus information output by the initial model and the corpus information labeled by the training samples is larger than a preset difference, so that the difference between the corpus information output by the initial model corresponding to the training samples and the corpus information labeled by the training samples is reduced. And comparing and adjusting for multiple times until the difference between the corpus information output by the entity initial model corresponding to the training sample and the corpus information labeled by the training sample is smaller than a preset difference, so as to obtain a trained corpus identification model, wherein the trained corpus identification model has the capacity of identifying the corpus information in the photo album data of the image type. The difference between the corpus information output by the corpus identification model and the corpus information labeled by the training sample can be represented by the similarity, and when the similarity is smaller than the preset similarity, the difference can be considered to be smaller than the preset difference.

In the above manner, the corpus information may be a sentence including an entity object. For example, for picture a, which is a group photo of a user and a person B participating in a wedding of the user, the extracted corpus information may include: "photograph on user's wedding", "person B attended user's wedding", "user and person B were a group in the user's wedding"; for another example, for icon B, picture B is a picture of a user blowing a cake candle during birthday, and the corpus information may be mentioned: "birthday photo of user", "user has blown cake candle at birthday", etc.

After the corpus information in the album data is extracted, the electronic device may extract the relationship between the entity objects included in the corpus information by using the corpus information. In some modes, the relationship extraction facing the open domain can be utilized, and the relationship extraction technology facing the open domain directly utilizes the relationship vocabulary in the corpus information to model the relationship between the entity objects, so that the classification of the relationship does not need to be specified in advance. Specifically, corpus information and the identified entity objects are given, and then the relationship between the entities is inferred according to the corpus information through an entity relationship extraction model, wherein the principle of the method is to identify semantics in the corpus information and further identify the relationship between the entities. For example, given a corpus: the "Qinghua university sits in Beijing" and the entities "Qinghua university" and "Beijing", the model may derive the "in" relationship by semantics and may form a knowledge triplet (Qinghua university, in Beijing). Of course, the traditional method facing the closed domain can be used to extract the relationship between the entity objects according to the corpus information.

Step S240: and acquiring attribute information of the entity object based on the album data.

In the embodiment of the present application, the attribute information may include information such as a name, time information, location information, birthday, nationality, and occupation of the entity object. The electronic device may employ an algorithmic model to extract the attribute information. The algorithm model can be machine learning algorithm, deep learning algorithm, recognition model, classification model, etc.

In some embodiments, since the attribute information of the entity object may be regarded as a kind of lexical relationship between the entity object and the attribute information, the attribute extraction problem may also be regarded as a relationship extraction problem. In a real language environment, keywords for limiting and defining the meaning of attribute information exist near the attribute information of many entity objects, and the attribute information is called named attributes in natural language processing technology, so that the attribute information of the named attributes can be located by using the keywords. Specifically, the electronic device may determine a keyword corresponding to the attribute information of the entity object, and then obtain the attribute information of the entity object from the corpus information based on the keyword. Optionally, the keywords corresponding to the attribute information may include keywords representing types of attributes, for example, a name, a time, a place, a birthday, a nationality, a country, an occupation, a job, and the like, and it can be understood that, if the corpus includes the keywords of the types of the attributes, a position of the attribute information may be located, for example, the corpus information is "birthday of person C is 8 months and 13 days", and then the "birthday" may be located to attribute information "8 months and 13 days"; optionally, the keyword corresponding to the attribute information may also be a predicate, for example, "yes" or "yes", and it can be understood that, in some corpora, the predicate is used as a connecting word connecting the entity object and the attribute information, so that the attribute information in the corpus information can be located by using a keyword, for example, if the corpus information is "three times the name of person C", the attribute information corresponding to the entity object "person C" is "name: zhang III ".

Step S250: and carrying out structured representation on the entity objects, the relationship among the entity objects and the attribute information of the entity objects to obtain a plurality of groups of data represented in a structured manner.

In the embodiment of the application, after the entity objects, the relationship between the entity objects, and the attribute information of the entity objects are acquired, when the knowledge graph of the album is generated according to the knowledge data of the knowledge graph, the electronic device may adopt a preset expression form to structurally represent the information, so as to obtain multiple sets of data that are structurally represented.

In some implementations, the electronic device can employ a representation of triples to structurally represent entity objects, relationships between entity objects, and attribute information. The triple is a representation form of the knowledge graph, the basic form of the triple includes (head entity-relationship-tail entity) and (entity-attribute value), and the attribute value are the extracted attribute information. For example, person A-brother-person C is an example of a triple of one (head entity-relationship-tail entity), where person A-is the head entity, person C is the tail entity, and brothers are the relationships of person A and person C; for another example, person B-name-lie is an example of a (entity-attribute value) triple, where person B is an entity and lie is an attribute value, and the attribute is a name.

Step S260: and storing the multiple groups of data expressed in a structured mode based on a graph structure to obtain a knowledge graph corresponding to the photo album.

In the embodiment of the application, after the entity objects, the relationship between the entity objects, and the attribute information of the entity objects are structured, multiple groups of structured data may be stored based on a graph structure to obtain a knowledge graph corresponding to an album, an expression form of the knowledge graph is a graph form, the graph corresponding to the knowledge graph may include multiple nodes and edges, the nodes are the entity objects or the attribute information, and the edges are the attribute information or the relationship between the entity objects.

In some embodiments, the electronic device may determine entity objects or attribute information in the sets of the data represented by the structure as nodes of the knowledge graph, and determine relationships or attribute information between entity objects in the sets of the data represented by the structure as edges of the knowledge graph, and then generate the knowledge graph of the graph structure according to the determined nodes and edges and by using the graph database, so as to obtain the knowledge graph corresponding to the album. The graph database may be Neo4j mobile, and the like, which is not limited herein.

The graph database generally requires that the input data is in a data form of structured representation, so after the entity objects, the relationship between the entity objects and the attribute information of the entity objects are structured, the obtained structured data can be imported into the graph database, the graph database can determine nodes and edges by analyzing the structured data, and draw images according to the nodes and the edges to obtain a graph of a knowledge graph, thereby completing the creation of the knowledge graph. As can be appreciated, since the structured data is in the form of (entity-relationship-entity) and (entity-attribute value), for the structured data in the form of (entity-relationship-entity), the graph database can determine that the two entities are nodes according to the set of structured data, and determine that the edge connecting the two nodes in the knowledge-graph is the relationship between the two entities; for the structured data in the form of (entity-attribute value), the graph database can determine the attribute values in the entity and the attribute information as nodes according to the set of structured data, and determine the edges connecting the two nodes in the knowledge graph as the attribute types corresponding to the attribute values in the attribute information.

For example, as shown in fig. 3, fig. 3 shows a schematic diagram of a knowledge graph of a graph structure, in which events, users, and people are entity objects, and attribute values in the entity objects and attribute information are nodes 11 of the knowledge graph, for example, attribute values such as "zhang san", "lie si", and the like may be nodes; the relationship between the entity objects is an edge 12, which is used to connect the nodes 11 corresponding to the two entity objects, for example, the relationship between the user and the person is an edge 12, and the node 11 corresponding to the user and the node 11 corresponding to the person are connected, and for example, the relationship between the user and the event is an edge 12, and the node 11 corresponding to the user and the node 11 corresponding to the event are connected; the attribute type corresponding to the attribute value in the attribute information may be used as an edge 12 to connect the node corresponding to the entity object and the attribute value, for example, "person 1" and "zhang san" are used as nodes in fig. 3, and "name" may be used as an edge to connect the two nodes 11. Of course, the knowledge-graph of the graph structure shown in FIG. 3 is merely illustrative.

In some embodiments, besides acquiring knowledge data for generating a knowledge graph through album data, that is, acquiring the entity objects, the relationship between the entity objects, and the attribute information, the electronic device may also acquire related knowledge data from some existing knowledge graphs. For example, the electronic device may obtain a knowledge graph of a history album corresponding to a user of the electronic device from the cloud, and then analyze the knowledge graph to obtain knowledge data of a related knowledge graph. And finally, generating the knowledge graph of the photo album according to the knowledge data extracted from the photo album data and the knowledge data extracted from the historical knowledge graph.

In some embodiments, the electronic device may also periodically update the knowledge graph of the album, wherein the period may be one week, one month, one half year, and the like, which is not limited herein. As an embodiment, the electronic device may periodically determine update of album data of an album, and when new album data is generated to be larger than a first data amount, may extract entity objects, relationships between entity objects, and attribute information based on the new album data and then add the extracted information to the knowledge graph; as an embodiment, the electronic device may periodically determine update of album data of the album, acquire all album data when the generated new album data is larger than the second data amount, then extract the entity object, the relationship between the entity objects, and the attribute information from all album data, and regenerate the knowledge graph as the knowledge graph of the updated album. Wherein the second amount of data may be greater than the first amount of data.

In some embodiments, the electronic device may further generate a knowledge vector according to the knowledge map, and the knowledge vector may be used for logical judgment and calculation, so that the electronic device may query, recommend, and the like information according to the knowledge vector. Wherein the vectors represent the positions of the nodes in the knowledge-graph and the content information of the nodes and edges in the knowledge-graph.

According to the method for generating the knowledge graph, an acquisition mode according to the photo album data is provided, the photo album data comprises data of the photo album and data in an address book, a short message, a calendar and a map, so that more extensive information related to an entity object in the photo album can be conveniently extracted, and the generated knowledge graph of the photo album is more accurate. The method for extracting the entity objects, the relationship among the entity objects and the attribute information according to the album data is provided, and the generation method of the knowledge graph is provided, so that the generated knowledge graph can comprise the knowledge related to the semantics of the users, other people and events in the album, a basis is provided for the intelligent search and recommendation in the album, and the accuracy of the intelligent search and recommendation based on the album is further improved.

Referring to fig. 4, fig. 4 is a flow chart illustrating a method for generating a knowledge graph according to another embodiment of the present application. The method for generating the knowledge graph is applied to the electronic device, and will be described in detail with respect to the flow shown in fig. 4, and the method for generating the knowledge graph may specifically include the following steps:

step S310: and acquiring album data corresponding to the album in the electronic equipment.

Step S320: and acquiring the entity object from the album data.

Step S330: and acquiring the relation between the entity objects based on the album data.

Step S340: and acquiring attribute information of the entity object based on the album data.

Step S350: and generating a knowledge graph corresponding to the photo album based on the entity objects, the relationship among the entity objects and the attribute information of the entity objects.

In the embodiment of the present application, steps S310 to S350 may refer to the contents of the foregoing embodiments, and are not described herein again.

Step S360: and responding to the viewing operation of the target picture in the album, and acquiring the designated person in the other persons according to the knowledge graph, wherein the target picture is any one picture in the album.

Step S370: and determining the figure corresponding to the target picture based on the knowledge graph.

In the embodiment of the application, after the electronic equipment generates the knowledge graph of the photo album, the electronic equipment can complete intelligent recommendation by using the knowledge graph. In some scenarios, the electronic device may obtain a designated person among other persons according to the knowledge graph in response to a viewing operation of a target graph in the album. The designated person may be a preset person, for example, a person set by a user of the electronic device that needs to push pictures, or an important contact in an address book set by the user of the electronic device.

In addition, the electronic equipment can also determine the person corresponding to the target picture according to the knowledge graph. Optionally, the electronic device may identify a person in the target picture, and compare the identification result with each node in the knowledge graph, so as to determine the person corresponding to the target picture.

Step S380: and when the person corresponding to the target picture is matched with the designated person, outputting prompt information, wherein the prompt information is used for prompting that the target picture is shared with the designated person.

In the embodiment of the application, after the person corresponding to the target picture and the designated person are determined, the designated person and the person corresponding to the target picture can be compared; if the person corresponding to the target picture is matched with the designated person, the fact that the current user is looking up the picture corresponding to the designated person is indicated, therefore, prompting information prompting the user to share the target picture to the designated person can be output, and user experience is improved.

For example, referring to FIG. 5, when a user is viewing an album, the user selects photos in the album to view the photos; when the people in the photo viewed by the user are the designated people, please refer to fig. 6, the electronic device may output prompt information to prompt the user to push the photo to the designated people, so that intelligent sharing of the photo is realized, and user experience is improved.

According to the method for generating the knowledge graph, the knowledge graph generated by the electronic equipment can comprise knowledge related to semantics of users, other people and events in the photo album, a basis is provided for intelligent searching and recommending in the photo album, and accuracy of the intelligent searching and recommending based on the photo album is improved. In actual application, aiming at the picture viewed by the user, if the viewed picture is identified to correspond to the designated person according to the knowledge graph, the picture sharing prompt is carried out, and the picture sharing experience of the user is improved.

Referring to fig. 7, fig. 7 is a flow chart illustrating a method for generating a knowledge graph according to still another embodiment of the present application. The method for generating the knowledge graph is applied to the electronic device, and will be described in detail with respect to the flow shown in fig. 7, and the method for generating the knowledge graph may specifically include the following steps:

step S410: and acquiring album data corresponding to the album in the electronic equipment.

Step S420: and acquiring the entity object from the album data.

Step S430: and acquiring the relation between the entity objects based on the album data.

Step S440: and acquiring attribute information of the entity object based on the album data.

Step S450: and generating a knowledge graph corresponding to the photo album based on the entity objects, the relationship among the entity objects and the attribute information of the entity objects.

Step S460: based on the knowledge-graph, importance scores for all events within a preset time period are determined.

In an embodiment of the present application, after generating the knowledge graph, the electronic device may further determine importance scores of all events within a preset time period based on the knowledge graph. The electronic device may determine all events within a preset time period according to the attribute information of the events. It is understood that the attribute information includes time information and the like, and therefore all events within a preset time period can be filtered out based on the attribute information.

Further, after the electronic device acquires all events within a preset time period, the importance score may be determined according to reference data such as whether the event belongs to a key date, the number of pictures corresponding to the event, and the number of times that a user of the electronic device appears in all the pictures of the event. Specifically, the reference data may be quantized and normalized to obtain normalized data of each item of reference data; and carrying out weighted summation according to the weights corresponding to the reference data and the reference data, thereby obtaining the importance scores of the events.

Step S470: and acquiring the target events with the importance scores meeting preset scoring conditions based on the importance scores of all the events.

In the embodiment of the application, after determining the importance score of each event, the electronic device may obtain a target event of which the importance score meets a preset score condition according to the importance score of each event. The preset scoring condition may include: the importance score is larger than the designated score, or the importance score is the highest, or the importance score is the top N in the ranking of the importance scores from top to bottom, wherein N is a positive integer. The preset score condition may be not particularly limited. By the method, important events occurring in the preset time period can be screened out.

Step S480: and acquiring the target event as an important event in the preset time period and outputting the important event.

In this embodiment of the application, after acquiring a target event with an importance score meeting a preset score condition, the electronic device may output the target event as an important event within a preset time period. For example, if the preset time period is one year, the annual important events can be sorted, so that the user can know the important events occurring within one year conveniently. In the above way, important events in the month and the year, such as wedding, birthday, commemorative day, travel and the like, can be conveniently found.

In some embodiments, after the electronic device displays the target event, the electronic device may further respond to a viewing operation for the target event, and obtain an album corresponding to the target event from the album based on the attribute information of the target event in the knowledge graph; and displaying the atlas. As one mode, the electronic device may obtain, according to attribute information of the target event, a picture with time information that is the same as the time of the target event as an album of the target event; alternatively, the electronic device may also acquire, as an album of the target event, a picture whose location information is the same as the location information of the target event, according to the location information in the attribute information of the target event. The event atlas is obtained through the constructed knowledge atlas, so that the query time of the picture can be greatly reduced, and the query efficiency is improved.

For example, referring to fig. 8, after the electronic device obtains the target event, a selection interface of an important event album may be displayed, where the selection interface may include albums corresponding to each target event, so that a user may conveniently view the albums corresponding to the target event, and user experience is improved.

In this embodiment, after the electronic device acquires the atlas of the target event, the electronic device may further screen the pictures in the atlas of the target event, and screen the pictures related to the user of the electronic device from the atlas, for example, screen the pictures including the face image of the user, so that the pictures with high importance to the user may be screened to form the atlas, and after the atlas is displayed, the user experience may be improved.

Of course, the knowledge graph can also be used in other application scenarios, such as information search, content recommendation, and the like.

In some embodiments, the electronic device may generate the knowledge graph locally, and the generated knowledge graph may not be opened to a third-party application or a cloud because the knowledge graph of the album is closely related to the information of the user, so that the security of the information of the user may be improved.

In the embodiment of the application, the knowledge graph generated by the electronic equipment can comprise knowledge related to semantics of users, other people and events in the photo album, so that a basis is provided for intelligent search and recommendation in the photo album, and the accuracy of the intelligent search and recommendation based on the photo album is further improved. In actual application, important events in a time period can be found conveniently through the knowledge graph, so that a user can know the important events, and user experience is improved.

Referring to fig. 9, a block diagram of a knowledge graph generating apparatus 400 according to an embodiment of the present application is shown. The apparatus 400 for generating a knowledge graph applies the above-mentioned electronic device, and the apparatus 400 for generating a knowledge graph includes: a data acquisition module 410, an entity acquisition module 420, a relationship acquisition module 430, an attribute acquisition module 440, and a map generation module 450. The data obtaining module 410 is configured to obtain album data corresponding to an album in the electronic device; the entity obtaining module 420 is configured to obtain an entity object from the album data; the relationship obtaining module 430 is configured to obtain a relationship between the entity objects based on the album data; the attribute obtaining module 440 is configured to obtain attribute information of the entity object based on the album data; the map generation module 450 is configured to generate a knowledge map corresponding to the album based on the entity objects, the relationship between the entity objects, and the attribute information of the entity objects.

In some embodiments, the atlas generation module 450 includes: a structuring unit and a data storage unit. The structuring unit is used for performing structured representation on the entity objects, the relationship among the entity objects and the attribute information of the entity objects to obtain a plurality of groups of data represented in a structured manner; and the data storage unit is used for storing the multiple groups of data expressed in a structured mode based on a graph structure to obtain the knowledge graph corresponding to the photo album.

In this embodiment, the data storage unit may be specifically configured to: determining the entity objects or the attribute information as nodes of a knowledge graph and determining the relationship between the entity objects or the attribute information as edges of the knowledge graph based on the plurality of groups of structured data; and generating a knowledge graph of a graph structure by using the graph database according to the nodes and the edges to obtain the knowledge graph corresponding to the photo album.

In some embodiments, the entity obtaining module 420 may be specifically configured to: and identifying a user corresponding to the electronic equipment, events in the album and other people except the user in the album from the album data by using an entity object identification model to obtain an entity object.

In some embodiments, the relationship obtaining module 430 may be specifically configured to: acquiring corpus information including the entity object based on the album data; and extracting the relation between the entity objects from the corpus information.

In this embodiment, the attribute obtaining module 440 may specifically be configured to: determining a keyword corresponding to the attribute information of the entity object; and acquiring attribute information of the entity object from the corpus information based on the keywords.

In some embodiments, the data acquisition module 410 may include: a data acquisition unit and a data processing unit. The data acquisition unit is used for acquiring specified data in an album, an address book, a short message, a calendar and a map of the electronic equipment, wherein the specified data at least comprises a text, an image and a video; and the data processing unit is used for preprocessing the specified data to obtain structural data, and taking the structural data as album data corresponding to the album.

In some embodiments, the knowledge-graph generating apparatus 400 may further include: the system comprises a first response module, a person determination module and a prompt module. The first response module is used for responding to the viewing operation aiming at the target picture in the photo album and acquiring the appointed person in the other persons according to the knowledge graph, wherein the target picture is any one picture in the photo album; the figure determining module is used for determining a figure corresponding to the target picture based on the knowledge graph; the prompting module is used for outputting prompting information when the person corresponding to the target picture is matched with the appointed person, and the prompting information is used for prompting that the target picture is shared with the appointed person.

In some embodiments, the knowledge-graph generating apparatus 400 may further include: the system comprises a score determining module, an event acquiring module and an event output module. The score determining module is used for determining importance scores of all events in a preset time period based on the knowledge graph; the event acquisition module is used for acquiring target events with the importance scores meeting preset score conditions based on the importance scores of all the events; and the event output module is used for acquiring the target event as an important event in the preset time period and outputting the important event.

In this embodiment, the apparatus 400 for generating a knowledge graph may further include: a second response module and an atlas display module. The second response module is used for responding to the viewing operation aiming at the target event and acquiring an atlas corresponding to the target event from the album based on the attribute information of the target event in the knowledge graph; the atlas display module is used for displaying the atlas.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, the coupling between the modules may be electrical, mechanical or other type of coupling.

In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.

In summary, the solution provided by the present application obtains the entity objects from the album data by obtaining the album data corresponding to the album in the electronic device, obtains the relationship between the entity objects based on the album data, obtains the attribute information of the entity objects based on the album data, then based on the entity objects, the relationship between the entity objects and the attribute information of the entity objects, generating a knowledge graph corresponding to the album, because the generated knowledge graph contains the relationships between the users and other people in the album, between the users and the events and between the other people and the events, and also contains the attribute information of the users, the other people and the events, therefore, the knowledge graph can provide knowledge related to semantics of users, other people and events in the photo album, a basis is provided for intelligent searching and recommending in the photo album, and accuracy of intelligent searching and recommending based on the photo album is improved.

Referring to fig. 10, a block diagram of an electronic device according to an embodiment of the present application is shown. The electronic device 100 may be a smart phone, a tablet computer, a smart watch, a notebook computer, or other electronic devices capable of running an application program. The electronic device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications may be stored in the memory 120 and configured to be executed by the one or more processors 110, the one or more programs configured to perform a method as described in the aforementioned method embodiments.

Processor 110 may include one or more processing cores. The processor 110 connects various parts within the overall electronic device 100 using various interfaces and lines, and performs various functions of the electronic device 100 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120 and calling data stored in the memory 120. Alternatively, the processor 110 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 110 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 110, but may be implemented by a communication chip.

The Memory 120 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 120 may be used to store instructions, programs, code sets, or instruction sets. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The data storage area may also store data created by the electronic device 100 during use (e.g., phone book, audio-video data, chat log data), and the like.

Referring to fig. 11, a block diagram of a computer-readable storage medium according to an embodiment of the present application is shown. The computer-readable medium 800 has stored therein a program code that can be called by a processor to execute the method described in the above-described method embodiments.

The computer-readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 800 includes a non-volatile computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 to perform any of the method steps of the method described above. The program code can be read from or written to one or more computer program products. The program code 810 may be compressed, for example, in a suitable form.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. A method for generating a knowledge graph, which is applied to an electronic device, the method comprising:

acquiring album data corresponding to an album in the electronic equipment;

acquiring an entity object from the album data;

acquiring the relation between the entity objects based on the album data;

acquiring attribute information of the entity object based on the album data;

and generating a knowledge graph corresponding to the photo album based on the entity objects, the relationship among the entity objects and the attribute information of the entity objects.

2. The method according to claim 1, wherein the generating a knowledge graph corresponding to the album based on the entity objects, the relationship between the entity objects, and the attribute information of the entity objects comprises:

carrying out structured representation on the entity objects, the relationship among the entity objects and the attribute information of the entity objects to obtain a plurality of groups of data represented in a structured manner;

and storing the multiple groups of data expressed in a structured mode based on a graph structure to obtain a knowledge graph corresponding to the photo album.

3. The method according to claim 2, wherein the storing the plurality of sets of data represented by the structuring based on a graph structure to obtain a corresponding knowledge graph of the album comprises:

determining the entity objects or the attribute information as nodes of a knowledge graph and determining the relationship between the entity objects or the attribute information as edges of the knowledge graph based on the plurality of groups of structured data;

and generating a knowledge graph of a graph structure by using the graph database according to the nodes and the edges to obtain the knowledge graph corresponding to the photo album.

4. The method according to claim 1, wherein the obtaining of the entity object from the album data comprises:

and identifying a user corresponding to the electronic equipment, events in the album and other people except the user in the album from the album data by using an entity object identification model to obtain an entity object.

5. The method of claim 1, wherein obtaining the relationship between the entity objects based on the album data comprises:

acquiring corpus information including the entity object based on the album data;

and extracting the relation between the entity objects from the corpus information.

6. The method according to claim 5, wherein the obtaining attribute information of the entity object based on the album data comprises:

determining a keyword corresponding to the attribute information of the entity object;

and acquiring attribute information of the entity object from the corpus information based on the keywords.

7. The method according to claim 1, wherein the obtaining album data corresponding to the album in the electronic device comprises:

acquiring specified data in an album, an address book, a short message, a calendar and a map of the electronic equipment, wherein the specified data at least comprises a text, an image and a video;

and preprocessing the specified data to obtain structural data, and taking the structural data as album data corresponding to the album.

8. The method according to any one of claims 1-7, further comprising:

responding to a viewing operation aiming at a target picture in the photo album, and acquiring a designated person in the other persons according to the knowledge graph, wherein the target picture is any one picture in the photo album;

determining a figure corresponding to the target picture based on the knowledge graph;

and when the person corresponding to the target picture is matched with the designated person, outputting prompt information, wherein the prompt information is used for prompting that the target picture is shared with the designated person.

9. The method according to any one of claims 1-7, further comprising:

determining importance scores of all events within a preset time period based on the knowledge-graph;

acquiring target events with importance scores meeting preset scoring conditions based on the importance scores of all the events;

and acquiring the target event as an important event in the preset time period and outputting the important event.

10. The method of claim 9, further comprising:

responding to the viewing operation aiming at the target event, and acquiring an album corresponding to the target event from the album based on the attribute information of the target event in the knowledge graph;

and displaying the atlas.

11. An apparatus for generating a knowledge graph, applied to an electronic device, the apparatus comprising: a data acquisition module, an entity acquisition module, a relationship acquisition module, an attribute acquisition module and a map generation module, wherein,

the data acquisition module is used for acquiring album data corresponding to an album in the electronic equipment;

the entity obtaining module is used for obtaining entity objects from the album data;

the relation acquisition module is used for acquiring the relation between the entity objects based on the album data;

the attribute acquisition module is used for acquiring attribute information of the entity object based on the album data;

the map generation module is used for generating a knowledge map corresponding to the photo album based on the entity objects, the relationship among the entity objects and the attribute information of the entity objects.

12. An electronic device, comprising:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-10.

13. A computer-readable storage medium, having stored thereon program code that can be invoked by a processor to perform the method according to any one of claims 1 to 10.