CN117171357A - Dish-based knowledge graph processing method, electronic equipment and medium - Google Patents
Dish-based knowledge graph processing method, electronic equipment and medium Download PDFInfo
- Publication number
- CN117171357A CN117171357A CN202311112713.2A CN202311112713A CN117171357A CN 117171357 A CN117171357 A CN 117171357A CN 202311112713 A CN202311112713 A CN 202311112713A CN 117171357 A CN117171357 A CN 117171357A
- Authority
- CN
- China
- Prior art keywords
- dish
- knowledge graph
- knowledge
- graph
- dishes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 11
- 238000000034 method Methods 0.000 claims abstract description 42
- 235000013305 food Nutrition 0.000 claims description 48
- 239000000463 material Substances 0.000 claims description 44
- 230000015654 memory Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 14
- 238000010411 cooking Methods 0.000 claims description 7
- 235000019640 taste Nutrition 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 18
- 241000238557 Decapoda Species 0.000 description 11
- 238000012545 processing Methods 0.000 description 11
- 238000001514 detection method Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 241000234282 Allium Species 0.000 description 5
- 235000002732 Allium cepa var. cepa Nutrition 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 235000002566 Capsicum Nutrition 0.000 description 3
- 235000005911 diet Nutrition 0.000 description 3
- 230000037213 diet Effects 0.000 description 3
- 235000013311 vegetables Nutrition 0.000 description 3
- 241000758706 Piperaceae Species 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 238000010025 steaming Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 230000003238 somatosensory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides a dish-based knowledge graph processing method, electronic equipment and medium, wherein the method comprises the following steps: acquiring text information and image information corresponding to dishes in a dish set; constructing a first knowledge graph of a text mode of the dish according to the text information corresponding to the dish; constructing a second knowledge graph of the image mode of the dish according to the image information corresponding to the dish; and obtaining a multi-mode third knowledge graph of the dish set according to the first knowledge graph and the second knowledge graph of the dishes.
Description
Technical Field
The present disclosure relates to the field of internet technology, and more particularly, to a dish-based knowledge graph processing method, an electronic device, and a computer-readable storage medium.
Background
With the continuous development of society, the improvement of living standard, more and more people pay attention to the scientificity and the balance of diet collocation.
Dishes are used as the most basic units of catering business, and are relied on in the scenes of user demand insight, supply strategy operation, business operation analysis and the like.
At present, the diet structure of people is mainly recommended based on a knowledge graph, wherein the knowledge graph is a model or system capable of reflecting the deep relation between dishes and food materials, and can consider multiple factors to comprehensively provide diet collocation for users.
However, in the prior art, the knowledge attribute of part of dishes is limited by the reasons of single acquired information source, large difficulty in mining technology and the like, so that the knowledge coverage of the constructed knowledge graph is insufficient, and the requirement of fine business cannot be met.
Disclosure of Invention
An object of the present disclosure is to provide a new technical solution for processing a knowledge graph of dishes.
According to a first aspect of the present disclosure, there is provided a dish-based knowledge graph processing method, including:
acquiring text information and image information corresponding to dishes in a dish set;
constructing a first knowledge graph of a text mode of the dish according to the text information corresponding to the dish;
constructing a second knowledge graph of the image mode of the dish according to the image information corresponding to the dish;
and obtaining a multi-mode third knowledge graph of the dish set according to the first knowledge graph and the second knowledge graph of the dishes.
Optionally, the obtaining the multi-modal third knowledge-graph of the dish set according to the first knowledge-graph and the second knowledge-graph of the dish includes:
constructing a fourth multi-mode knowledge graph of the dishes according to the first knowledge graph and the second knowledge graph of the dishes;
and obtaining a multi-mode third knowledge graph of the dish set according to the fourth knowledge graph of each dish in the dish set.
Optionally, the constructing a fourth multi-modal knowledge-graph of the dish according to the first knowledge-graph and the second knowledge-graph of the dish includes:
connecting the image information in the second knowledge graph with the text information in the first knowledge graph, and connecting the node connected with the image information in the second knowledge graph with the text information in the first knowledge graph to obtain the fourth knowledge graph;
the obtaining the multi-modal third knowledge graph of the dish set according to the fourth knowledge graph of each dish in the dish set comprises:
and connecting the same nodes in the fourth knowledge graph of each dish in the dish set to obtain the third knowledge graph.
Optionally, the obtaining the multi-modal third knowledge-graph of the dish set according to the first knowledge-graph and the second knowledge-graph of the dish includes:
obtaining a fifth knowledge graph of the text mode of the dish set according to the first knowledge graph of each dish in the dish set;
obtaining a sixth knowledge graph of the image mode of the dish set according to the second knowledge graph of each dish in the dish set;
and obtaining the third knowledge graph according to the fifth knowledge graph and the sixth knowledge graph.
Optionally, the constructing a first knowledge graph of the text mode of the dish according to the text information corresponding to the dish includes:
acquiring first attribute information of the dishes according to the text information of the dishes; the first attribute information comprises at least one of food material, taste, mouthfeel, cuisine and cooking method;
and connecting the first attribute information corresponding to the dishes with the text information of the dishes to obtain the first knowledge graph.
Optionally, the constructing a second knowledge graph of the image mode of the dish according to the image information corresponding to the dish includes:
acquiring second attribute information of the dishes and an image area corresponding to the second attribute information according to the image information of the dishes; the second attribute information at least comprises food materials;
and connecting the second attribute information of the dishes with the image information of the dishes, and connecting the second attribute information of the dishes with the corresponding image areas to obtain the second knowledge graph.
Optionally, the method further comprises:
in response to a request for inquiring food materials contained in a target image, determining image information corresponding to the target image as target image information;
inquiring the third knowledge graph to obtain food materials related to the target image information as target food materials;
and displaying the target food material and/or an image area corresponding to the target food material.
Optionally, the method further comprises:
responding to a request for inquiring target attribute information, inquiring the third knowledge graph, and acquiring text information associated with the target attribute information as target text information;
and displaying the target text information and/or the image information associated with the target text information.
According to a second aspect of the present disclosure, there is provided an electronic device comprising a memory for storing an executable computer program and a processor; the computer program is for controlling the processor to perform the method according to the first aspect of the present disclosure.
According to a third aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method according to the first aspect of the present disclosure.
In the embodiment of the disclosure, a first knowledge graph of a text mode is firstly constructed based on text information of dishes, a second knowledge graph of an image mode is constructed based on image information of the dishes, and then the first knowledge graph of the text mode is complemented by the second knowledge graph of the image mode to obtain a multi-mode third knowledge graph of a dish set, so that the obtained third knowledge graph of the dish set is more complete and comprehensive, and further accurate search and recommendation of dish related information can be performed according to the third knowledge graph.
Other features of the present disclosure and its advantages will become apparent from the following detailed description of exemplary embodiments of the disclosure, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a block diagram showing an example of a hardware configuration of an electronic device that may be used to implement embodiments of the present disclosure;
FIG. 2 illustrates a flow chart of a method of processing a dish-based knowledge graph in an embodiment of the disclosure;
FIG. 3 shows a schematic diagram of a first knowledge-graph of an embodiment of the present disclosure;
FIG. 4 shows a schematic diagram of a second knowledge-graph of an embodiment of the present disclosure;
FIG. 5 shows a schematic diagram of a fourth knowledge-graph of an embodiment of the present disclosure;
FIG. 6 shows a schematic diagram of a third knowledge-graph of an embodiment of the present disclosure;
FIG. 7 illustrates a flowchart of one example of a dish-based knowledge-graph processing method in accordance with an embodiment of the disclosure;
fig. 8 shows a schematic block diagram of an electronic device of an embodiment of the disclosure.
Detailed Description
Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless it is specifically stated otherwise.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of exemplary embodiments may have different values.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.
< hardware configuration >
Fig. 1 is a block diagram illustrating a hardware configuration of an electronic device 1000 in which embodiments of the present disclosure may be implemented.
The electronic device 1000 may be a laptop, desktop, cell phone, tablet, speaker, headset, etc. As shown in fig. 1, the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. The processor 1100 may be a processor CPU, a microprocessor MCU, or the like. The memory 1200 includes, for example, ROM (read only memory), RAM (random access memory), nonvolatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, a USB interface, a headphone interface, and the like. The communication device 1400 can be capable of wired or wireless communication, and specifically can include Wifi communication, bluetooth communication, 2G/3G/4G/5G communication, and the like. The display device 1500 is, for example, a liquid crystal display, a touch display, or the like. The input device 1600 may include, for example, a touch screen, keyboard, somatosensory input, and the like. A user may input/output voice information through the speaker 1700 and microphone 1800.
The electronic device shown in fig. 1 is merely illustrative and is in no way meant to limit the disclosure, its application, or uses. In an embodiment of the present disclosure, the memory 1200 of the electronic device 1000 is configured to store instructions for controlling the processor 1100 to operate to perform any one of the advertisement screening methods provided by the embodiments of the present disclosure. It will be appreciated by those skilled in the art that although a plurality of devices are shown for the electronic apparatus 1000 in fig. 1, the present disclosure may relate to only some of the devices thereof, for example, the electronic apparatus 1000 relates to only the processor 1100 and the storage device 1200. The skilled artisan can design instructions in accordance with the disclosed aspects of the present disclosure. How the instructions control the processor to operate is well known in the art and will not be described in detail here.
< method >
The method for processing the dish-based knowledge graph provided in the embodiment can be implemented by electronic equipment. In one example, the electronic device may include electronic device 1000 as shown in FIG. 1.
As shown in fig. 2, the dish-based knowledge graph processing method includes steps S2100 to S2400:
step S2100, obtaining text information and image information corresponding to dishes in the set of dishes.
In this embodiment, the dish set may include a plurality of dishes, each having corresponding text information and image information.
The text information may include a name of a corresponding dish, recommendation information of the corresponding dish, comment information of the corresponding dish, and the like.
The image information may be an image obtained by photographing a cooked dish, an image extracted from a comment, or an image downloaded from a network, which is not limited herein.
In one embodiment of the present disclosure, step S2200 and step S2300 shown below may be performed separately for each dish in the set of dishes.
Step S2200, constructing a first knowledge graph of the text mode of the dish according to the text information corresponding to the dish.
In one embodiment of the present disclosure, constructing a first knowledge graph of a text modality of a dish according to text information corresponding to the dish may include:
acquiring first attribute information of dishes according to text information of the dishes; and connecting the first attribute information corresponding to the dishes with the text information of the dishes to obtain a first knowledge graph. Wherein the first attribute information includes at least one of food material, taste, mouthfeel, cuisine, and cooking method.
In one embodiment of the present disclosure, the text information of the dish may be processed based on a pre-trained named entity recognition (Named Entity Recognition, NER) model to obtain first attribute information of the dish.
The NER model is a subtask of information extraction aimed at locating and classifying named entities in text into predefined categories, such as first attribute information of food materials, tastes, cuisines, cooking methods, etc.
NER aims at identifying named entities in text and generalizing them to corresponding entity types. The NER model may be a network architecture using BERT (Bidirectional Encoder Representation from Transformers, bi-directional encoder from converter) +crf (Conditional Random Field ), or a network architecture of bert+lstm (Long Short-Term Memory network) +crf.
Considering that the identification of food entities from the vegetable name text is a task in a specific field, a great deal of knowledge in the related field is required, and a Cross-Encoder can be used for introducing unlabeled similar samples for training so as to improve the identification accuracy of the NER model. The Cross-Encoder performs training-free entity type calibration through entity voting, simultaneously captures related features through modeling similar samples, and finally extracts first attribute information in text information.
For example, the text information of the dish can be a dish name, the dish name is that the peppers steam the shrimp, then the food materials of the dish can be obtained based on the NER model, the cooking method of the dish comprises steaming, and the dish system of the dish can comprise the peppers and the shrimps. Correspondingly, the first attribute information of the dish can comprise pericarpium zanthoxyli, prawn, steam and fructus zanthoxyli.
In another embodiment of the present disclosure, the text information of the dish may be a name of the dish, and then, semantic analysis processing may be performed on comment information and/or recommendation information associated with the name of the dish, so as to obtain first attribute information of the dish.
On the basis of obtaining the first attribute information of the dish, all the first attribute information of the dish may be connected with the text information of the dish, so as to obtain a first knowledge graph as shown in fig. 3. In the obtained first knowledge graph, the text information and the first attribute information of the dishes are nodes of the first knowledge graph.
In one embodiment of the disclosure, in the first knowledge graph, a line between text information of a dish and first attribute information may be labeled with a type of the first attribute information, for example, a line between pericarpium zanthoxyli and shrimp and text information may be labeled with food material, a line between pericarpium zanthoxyli and text information may be labeled with a cuisine, and a line between steaming and text information may be labeled with a cooking method.
Step S2300, constructing a second knowledge graph of the image mode of the dish according to the image information corresponding to the dish.
In one embodiment of the present disclosure, constructing a second knowledge graph of an image modality of a dish according to image information corresponding to the dish may include: acquiring second attribute information of the dishes and an image area corresponding to the second attribute information according to the image information of the dishes; and connecting the second attribute information of the dishes with the image information of the dishes, and connecting the second attribute information of the dishes with the corresponding image areas to obtain a second knowledge graph. The second attribute information includes at least food material.
In this embodiment, the image information of the dish may be processed based on the target detection model, so as to obtain the second attribute information of the dish and the image area corresponding to the second attribute information in the image information.
In one example, the target detection model may be a transducer (transducer) -based end-to-end target detection (Detection Transformer, DETR) model, which may simplify the flow of target detection and improve target detection.
Further, the latest DETR model, namely a DINO model, optimizes the problems of slow convergence speed and unclear query display meaning existing in the traditional DETR by using a contrast denoising training and mixed query (query) selection strategy, and improves the accuracy of frame prediction by using a two-time review strategy. Thus, the object detection model of the present embodiment may be a DINO model.
Still further, in order to improve the detection accuracy of the DINO model, the DINO model may be trained by adopting a dish picture in advance.
In this embodiment, the image area corresponding to the second attribute information may be a partial image in which the content intercepted from the image of the dish matches the second attribute information.
For example, in the case where the dish of the dish is a pepper steamed shrimp, and the image information is as shown in fig. 4, it may be determined that the food material of the dish includes a shrimp and an onion, and the corresponding second attribute information of the dish may include a shrimp and an onion. Then, the image area corresponding to the shrimp in the second attribute information may be a partial image of the shrimp in the image information, and the image area corresponding to the onion in the second attribute information may be a partial image of the onion in the image information.
In the case of obtaining the second attribute information and the corresponding image area, the second attribute information of the dishes may be connected with the image information of the dishes, and each piece of the second attribute information of the dishes is connected with the corresponding image area, so as to obtain a second knowledge graph as shown in fig. 4. In the obtained second knowledge graph, the image information, the second attribute information and the image area corresponding to the second attribute information of the dishes are all nodes of the second knowledge graph.
According to the method, the second knowledge graph is constructed based on the image information of the dishes, so that the first knowledge graph of the dishes can be completed according to the second knowledge graph of the dishes, and the third knowledge graph of the obtained dishes set is more complete and comprehensive.
In one embodiment of the present disclosure, in the first knowledge graph, a line between image information of a dish and second attribute information may label a type of the second attribute information, for example, a line between shrimp, onion and text information may label food.
Step S2400, obtaining a multi-mode third knowledge graph of the dish set according to the first knowledge graph and the second knowledge graph of the dish.
In the embodiment of the disclosure, a first knowledge graph of a text mode is firstly constructed based on text information of dishes, a second knowledge graph of an image mode is constructed based on image information of the dishes, and then the first knowledge graph of the text mode is complemented by the second knowledge graph of the image mode to obtain a multi-mode third knowledge graph of a dish set, so that the obtained third knowledge graph of the dish set is more complete and comprehensive, and further accurate search and recommendation of dish related information can be performed according to the third knowledge graph.
In one embodiment of the present disclosure, obtaining a multi-modal third knowledge-graph of a dish set according to a first knowledge-graph and a second knowledge-graph of a dish may include steps S2411 to S2412 as follows:
step S2411, constructing a fourth multi-mode knowledge-graph of the dishes according to the first knowledge-graph and the second knowledge-graph of the dishes.
In one embodiment of the present disclosure, constructing a fourth knowledge-graph of a dish according to a first knowledge-graph and a second knowledge-graph of the dish may include: and connecting the image information in the second knowledge graph with the text information in the first knowledge graph, and connecting the node connected with the image information in the second knowledge graph with the text information in the first knowledge graph to obtain a fourth knowledge graph.
Further, when the first knowledge-graph and the second knowledge-graph include the same attribute information, the same first attribute information and second attribute information may be combined into one node in the fourth knowledge-graph, or the first attribute information may be discarded.
In the example of dishes with the name of the shrimp steamed with pericarpium zanthoxyli, the fourth knowledge-graph obtained according to the first knowledge-graph shown in fig. 3 and the second knowledge-graph shown in fig. 4 may be shown in fig. 5.
According to the embodiment, the first knowledge graph is complemented by using the second knowledge graph of the same dish, so that a fourth knowledge graph of each dish in multiple modes is obtained, and then a third knowledge graph of a complete and comprehensive dish set is obtained according to the fourth knowledge graph of each dish in the dish set.
In one embodiment of the present disclosure, when the third knowledge-graph is constructed according to the first knowledge-graph and the second knowledge-graph, the type marked by the connection line between the first attribute information and the text information is reserved, and the type marked by the connection line between the second attribute information and the image information is marked on the connection line between the second attribute information and the text information.
Step S2412, obtaining a multi-mode third knowledge graph of the dish set according to the fourth knowledge graph of each dish in the dish set.
In one embodiment of the present disclosure, obtaining a multi-modal third knowledge-graph of a set of dishes according to a fourth knowledge-graph of each dish in the set of dishes may include: and connecting the same nodes in the fourth knowledge graph of each dish in the dish set to obtain a third knowledge graph.
In this embodiment, the same attribute information in the fourth knowledge graph of different dishes may be connected and combined into one node, so as to obtain the third knowledge graph shown in fig. 6.
In this embodiment, the type marked on each connection line may be reserved, so that attribute information of a corresponding type may be searched for later according to the type marked on the connection line.
In another embodiment of the present disclosure, obtaining a multi-modal third knowledge-graph of a dish set according to a first knowledge-graph and a second knowledge-graph of a dish may include steps S2421 to S2422 as follows:
step S2421, obtaining a fifth knowledge graph of the text mode of the dish set according to the first knowledge graph of each dish in the dish set.
In this embodiment, the first attribute information with the same first knowledge graph of each dish in the dish set may be connected and combined into one node to obtain the fifth knowledge graph.
Step S2422, a sixth knowledge graph of the image mode of the dish set is obtained according to the second knowledge graph of each dish in the dish set.
In this embodiment, the second attribute information with the same second knowledge graph of each dish in the dish set may be connected and combined into one node to obtain the sixth knowledge graph.
On the basis of connecting and combining the second attribute information with the same second knowledge patterns of at least two dishes into one node, the image area corresponding to the same second attribute information can be only reserved part or all and is connected with the second attribute information.
Step S2423, a third knowledge graph is obtained according to the fifth knowledge graph and the sixth knowledge graph.
In this embodiment, the first attribute information and the second attribute information that are the same in the fifth knowledge-graph and the sixth knowledge-graph may be combined into one node in the third knowledge-graph, or the first attribute information that is the same as the second attribute information may be discarded.
According to the method, the sixth knowledge graph of the text mode of the dish set is utilized to complement the fifth knowledge graph of the image mode of the collection set, so that the multi-mode third knowledge graph of the dish set is obtained, and the obtained third knowledge graph is more complete and comprehensive.
In one embodiment of the present disclosure, on the basis of obtaining the third knowledge-graph of the dish set, the method may further include steps S3110 to S3130 as follows:
in step S3110, image information corresponding to the target image is determined as target image information in response to a request for querying food materials included in the target image.
In this embodiment, the image information with the highest similarity with the target image in the third knowledge graph may be queried, that is, the target image information.
Step S3120, inquiring the third knowledge graph to obtain the food material associated with the target image information as the target food material.
In this embodiment, a food material associated with text information corresponding to image information having the highest similarity between target images may be used as the target food material.
Further, the third knowledge graph may be queried to determine, as the target food, attribute information of the type marked by the line between the text information, which is the food, from all attribute information associated with the text information corresponding to the image information with the highest similarity between the target images.
Step S3130, displaying the target food material and/or the image area corresponding to the target food material.
The target food material may have an associated image area or may have no corresponding image area in the third knowledge graph. On the basis of obtaining the target food material, only the target food material can be displayed, or on the basis of displaying the target food material, the image area corresponding to the target food material can be displayed correspondingly, and on the target food material without the corresponding image area, the image area corresponding to the target food material can be not displayed.
According to the embodiment, the food materials contained in the target image are searched based on the complete and comprehensive third knowledge graph, so that the search result is more accurate.
In one embodiment of the present disclosure, on the basis of obtaining the third knowledge-graph of the dish set, the method may further include steps S3210 to S3220 as follows:
step S3210, in response to the request for querying the target attribute information, queries the third knowledge-graph to obtain text information associated with the target attribute information as target text information.
Step S3220, displaying the target text information and/or the image information associated with the target text information.
In this embodiment, the target attribute information may be any one of the first attribute information and the second attribute information. The target text information resistance associated with the target attribute information may be one or more.
According to the method and the device for searching the text information and/or the image information of the dishes with the target attribute information based on the complete and comprehensive third knowledge graph, the search result can be more accurate.
In another embodiment of the present disclosure, the method may further comprise: acquiring target attribute information corresponding to a target user; inquiring a third knowledge graph, and acquiring text information associated with the target attribute information as target text information; the target text information and/or the image information associated with the target text information is recommended to the target user. The target attribute information corresponding to the target user may be determined according to a history search record of the target user or an order record. By means of the method and the device, the dish is recommended based on the third knowledge graph, and the recommendation result can be more accurate.
In one embodiment of the present disclosure, the method may further comprise: responding to a request for inquiring food materials contained in the target vegetable name, determining text information corresponding to the target vegetable name as target text information; inquiring the third knowledge graph to obtain food materials related to the target text information as target food materials; and displaying the target food material and/or an image area corresponding to the target food material. According to the embodiment, the food materials contained in the dishes corresponding to the target dish names are searched based on the complete and comprehensive third knowledge graph, so that the search result can be more accurate.
< example >
Fig. 7 is a schematic diagram of an example of a dish-based knowledge-graph processing method according to an embodiment of the present disclosure.
As shown in FIG. 7, the method may include steps S7001 to SS700 as follows
Step S7001, obtaining text information and image information corresponding to the dishes in the dish set.
Step S7002, based on the NER model, acquiring the first attribute information of the dishes according to the text information of the dishes.
Wherein the first attribute information includes at least one of food material, taste, mouthfeel, cuisine, and cooking method.
Step S7003, the first attribute information corresponding to the dishes is connected with the text information of the dishes, and a first knowledge graph is obtained.
Step S7004, based on the target detection model, the second attribute information of the dish and the image area corresponding to the second attribute information are acquired according to the image information of the dish.
Wherein the second attribute information at least comprises food material.
Step S7005, connecting the second attribute information of the dishes with the image information of the dishes, and connecting the second attribute information of the dishes with the corresponding image area to obtain a second knowledge graph.
Step S7006, connecting the image information in the second knowledge graph with the text information in the first knowledge graph, and connecting the node connected with the image information in the second knowledge graph with the text information in the first knowledge graph to obtain a fourth knowledge graph.
Step S7007, connecting the same nodes in the fourth knowledge graph of each dish in the dish set to obtain a multi-mode third knowledge graph of the dish set.
< electronic device >
In the present embodiment, there is also provided an electronic device 8000, as shown in fig. 8, including a memory 8100 and a processor 8200.
The memory 8100 for storing an executable computer program; the computer program is configured to control the processor 8200 to execute any one of the dish-based knowledge graph processing methods provided in the present embodiment.
In this embodiment, the electronic device 8000 may be a mobile phone, a tablet computer, a palm computer, a desktop computer, a notebook computer, or the like. For example, the electronic device 6000 may be an electronic product having an advertisement shielding function.
< computer-readable storage Medium >
In this embodiment, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the dish-based knowledge-graph processing method of any embodiment of the present disclosure.
The present disclosure may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer program instructions for performing the operations of the present disclosure can be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.
Claims (10)
1. The dish-based knowledge graph processing method is characterized by comprising the following steps of:
acquiring text information and image information corresponding to dishes in a dish set;
constructing a first knowledge graph of a text mode of the dish according to the text information corresponding to the dish;
constructing a second knowledge graph of the image mode of the dish according to the image information corresponding to the dish;
and obtaining a multi-mode third knowledge graph of the dish set according to the first knowledge graph and the second knowledge graph of the dishes.
2. The method of claim 1, wherein the obtaining a multi-modal third knowledge-graph of the set of dishes from the first knowledge-graph and the second knowledge-graph of the dishes comprises:
constructing a fourth multi-mode knowledge graph of the dishes according to the first knowledge graph and the second knowledge graph of the dishes;
and obtaining a multi-mode third knowledge graph of the dish set according to the fourth knowledge graph of each dish in the dish set.
3. The method of claim 2, wherein constructing a fourth multi-modal knowledge-graph of the dish from the first knowledge-graph and the second knowledge-graph of the dish comprises:
connecting the image information in the second knowledge graph with the text information in the first knowledge graph, and connecting the node connected with the image information in the second knowledge graph with the text information in the first knowledge graph to obtain the fourth knowledge graph;
the obtaining the multi-modal third knowledge graph of the dish set according to the fourth knowledge graph of each dish in the dish set comprises:
and connecting the same nodes in the fourth knowledge graph of each dish in the dish set to obtain the third knowledge graph.
4. The method of claim 1, wherein the obtaining a multi-modal third knowledge-graph of the set of dishes from the first knowledge-graph and the second knowledge-graph of the dishes comprises:
obtaining a fifth knowledge graph of the text mode of the dish set according to the first knowledge graph of each dish in the dish set;
obtaining a sixth knowledge graph of the image mode of the dish set according to the second knowledge graph of each dish in the dish set;
and obtaining the third knowledge graph according to the fifth knowledge graph and the sixth knowledge graph.
5. The method of claim 1, wherein the constructing a first knowledge graph of a text modality of the dish according to the text information corresponding to the dish comprises:
acquiring first attribute information of the dishes according to the text information of the dishes; the first attribute information comprises at least one of food material, taste, mouthfeel, cuisine and cooking method;
and connecting the first attribute information corresponding to the dishes with the text information of the dishes to obtain the first knowledge graph.
6. The method of claim 1, wherein the constructing a second knowledge-graph of the image modality of the dish according to the image information corresponding to the dish comprises:
acquiring second attribute information of the dishes and an image area corresponding to the second attribute information according to the image information of the dishes; the second attribute information at least comprises food materials;
and connecting the second attribute information of the dishes with the image information of the dishes, and connecting the second attribute information of the dishes with the corresponding image areas to obtain the second knowledge graph.
7. The method according to claim 1, wherein the method further comprises:
in response to a request for inquiring food materials contained in a target image, determining image information corresponding to the target image as target image information;
inquiring the third knowledge graph to obtain food materials related to the target image information as target food materials;
and displaying the target food material and/or an image area corresponding to the target food material.
8. The method according to claim 1, wherein the method further comprises:
responding to a request for inquiring target attribute information, inquiring the third knowledge graph, and acquiring text information associated with the target attribute information as target text information;
and displaying the target text information and/or the image information associated with the target text information.
9. An electronic device comprising a memory and a processor, the memory for storing an executable computer program; the computer program for controlling the processor to perform the method according to any one of claims 1 to 8.
10. A computer readable storage medium, characterized in that a computer program is stored thereon, which, when being executed by a processor, implements the method according to any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311112713.2A CN117171357A (en) | 2023-08-30 | 2023-08-30 | Dish-based knowledge graph processing method, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311112713.2A CN117171357A (en) | 2023-08-30 | 2023-08-30 | Dish-based knowledge graph processing method, electronic equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117171357A true CN117171357A (en) | 2023-12-05 |
Family
ID=88944196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311112713.2A Pending CN117171357A (en) | 2023-08-30 | 2023-08-30 | Dish-based knowledge graph processing method, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117171357A (en) |
-
2023
- 2023-08-30 CN CN202311112713.2A patent/CN117171357A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11328004B2 (en) | Method and system for intelligently suggesting tags for documents | |
| US11157577B2 (en) | Method for searching and device thereof | |
| KR101953303B1 (en) | Identifying matching applications based on browsing activity | |
| CN105706080B (en) | Augmenting and presenting captured data | |
| US10255253B2 (en) | Augmenting and presenting captured data | |
| US20150317945A1 (en) | Systems and methods for generating tinted glass effect for interface controls and elements | |
| EP3005055B1 (en) | Apparatus and method for representing and manipulating metadata | |
| US20160179899A1 (en) | Method of providing content and electronic apparatus performing the method | |
| CN107562939A (en) | Vertical field news recommends method, apparatus and readable storage medium | |
| US11734370B2 (en) | Method for searching and device thereof | |
| CN108027825B (en) | Exposing external content in an enterprise | |
| CN117171357A (en) | Dish-based knowledge graph processing method, electronic equipment and medium | |
| KR20150135042A (en) | Method for Searching and Device Thereof | |
| CN107357796B (en) | Network information acquisition method, equipment and programmable equipment | |
| KR102335972B1 (en) | Method and apparatus for providing search recommendation information | |
| US20230161949A1 (en) | Intelligent content identification and transformation | |
| CN108369605A (en) | content item and source detecting system | |
| KR101726845B1 (en) | Device and method for providing data, and computer program for executing the method | |
| WO2016094206A1 (en) | Method and apparatus for processing information | |
| KR20160070964A (en) | Method of providing contents with removing the displayed contents and system thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |