KR102604774B1 - Method and system for analyzing taste of food and beverage using biomimetic technology based on deep learning - Google Patents

Abstract

A central server that performs taste analysis of food and drinks preferred by each user to achieve the purpose of the present invention, comprising a memory storing at least one command, and a processor, wherein the at least one command is performed by the processor. , Matching at least one food and beverage preferred by each user, extracting common taste characteristics from taste measurement data for each food and beverage for the at least one food and beverage measured from a taste recognition device based on a taste analysis model using a deep neural network, Based on the common taste characteristics, a process of creating a food and beverage group is performed by grouping at least one food or beverage.

Description

Taste analysis method and system for food and beverages using biomimetic technology based on deep learning {METHOD AND SYSTEM FOR ANALYZING TASTE OF FOOD AND BEVERAGE USING BIOMIMETIC TECHNOLOGY BASED ON DEEP LEARNING}

The present invention relates to a method and system for analyzing the taste of food and beverages using biomimetic technology based on deep learning. More specifically, the present invention relates to a method and system for analyzing the taste of food and beverages using biomimetic technology based on deep learning. This relates to a method and system for taste analysis of food and beverages using biomimetic technology based on deep learning that matches food and beverages according to user tastes based on a taste analysis model.

As public interest in delicious food increases, recommendations for food and beverages are made in various forms, such as restaurant recommendations, delicious food, and recipe recommendations.

However, because each user's tastes are different, if food and beverages are recommended to the user without taking this into account, there is a high possibility that food and beverages that suit the user's tastes will not be recommended.

Accordingly, in recent years, various services that recommend food and beverages taking into account user tastes have been developed. This recommendation is made by recommending the same or similar food based on the user's preferred food or recipe, or by recommending based on the user's past order data.

However, even the same food may have different tastes as the recipe is applied differently, and when the type and ingredients of the food are different, the food may have different tastes even if the recipe is the same or similar.

Moreover, this recommendation method assumes that all people have the same taste perception ability, but in reality, the perception ability for each taste area is different depending on the person, and there are cases where people are sensitive or insensitive to specific taste areas. , there is a problem that there are limits to recommending food and beverages to everyone based on the same standards.

In other words, it should be understood that taste perception itself is subjective depending on the person, and that each person's preferred taste standard is not simply determined by the presence or absence of a specific taste, but is the result of a combination of several factors. Accordingly, there is a need for technology to match food and beverages according to the user's taste, focusing on biomimetic technology, which determines taste by stimulating the brain by interacting with elements such as taste, aroma, and texture.

Republic of Korea Patent Publication No. 10-2132975, July 6, 2020

The present invention aims to solve the above-described problems.

The problem to be solved by the present invention is to group users based on taste perception data for at least one food or beverage preferred by each user, and based on objective taste measurement data measured using at least one device for each food or beverage, The purpose is to group similar food and beverages into groups and match user groups with food and beverage groups based on taste. In addition, when grouping food and beverages, deep learning-based GNN technology can be applied to determine whether food and beverages are similar, or this task can be included.

However, the problems to be solved by the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned can be clearly understood by those skilled in the art to which the present invention pertains from the description below. will be.

A central server that performs taste analysis of food and drinks preferred by each user to achieve the purpose of the present invention, comprising a memory storing at least one command, and a processor, wherein the at least one command is performed by the processor. , Matching at least one food and beverage preferred by each user, extracting common taste characteristics from taste measurement data for each food and beverage for the at least one food and beverage measured from a taste recognition device based on a taste analysis model using a deep neural network, Based on the common taste characteristics, a process of creating a food and beverage group is performed by grouping at least one food or beverage.

In one embodiment of the present invention, the deep network may be a GNN (Graph Neural Network) based on the characteristics of taste measurement data graphed for each item.

In one embodiment of the present invention, the taste recognition device may include at least one of an electronic tongue, a chili sensor, an electronic nose, and a texture measuring device.

In one embodiment of the present invention, at least one food or beverage preferred by each user may be determined based on taste recognition data for each taste area registered by the user for each food or beverage.

In one embodiment of the present invention, the taste area includes at least one of sour taste, salty taste, bitter taste, sweet taste, and umami taste, and the taste recognition data may include a numerical score for each area.

In one embodiment of the present invention, when the at least one command is performed by the processor, a process of creating a user group by grouping users based on user-specific taste perception data for food and beverages may be performed.

In one embodiment of the present invention, when the at least one command is performed by the processor, matching is performed between the user group and the food and beverage group, and an updating process may be performed.

In one embodiment of the present invention, a taste analysis model previously learned based on deep learning is stored in the memory to extract taste characteristics for each food and beverage, and the at least one command is performed by the processor, Taste measurement data is acquired by a taste recognition device, and the taste characteristics of each food or beverage can be extracted by the taste analysis model using the acquired data.

A method of analyzing the taste of food and beverages preferred by each user performed on a central server to achieve the purpose of the present invention, comprising the steps of matching at least one food and beverage preferred by each user, and recognizing taste based on a taste analysis model using a deep neural network. Extracting common taste characteristics from taste measurement data for each food and beverage for the at least one food and beverage measured from a device, and grouping the food and beverages based on the common taste characteristics to create a plurality of food and beverage groups.

A computer-readable recording medium for achieving the object of the present invention stores a computer program programmed to perform each step included in the method according to claim 9.

According to one embodiment, it was proposed to solve the above problems, and matches food and beverages according to the user's taste by focusing on biomimetic technology that determines taste by the interaction of complex factors such as taste, aroma, and texture. You can create a deep learning model for

In addition, by matching food and beverage groups grouped based on objective taste measurement data to user groups grouped based on subjective user-specific taste perception data, preferred food and beverage recommendations can be made for each user group, and preferences for each food and beverage can be predicted. Marketing can be targeted to user groups.

Figure 1 is a block diagram for explaining a taste analysis system according to an embodiment of the present invention.
Figure 2 shows an exemplary configuration diagram of a central server according to one embodiment.
Figure 3 is an example diagram to explain collecting taste recognition data and taste measurement data for each user.
In Figure 4, functions executable on the central server according to one embodiment are conceptualized for each module or model.
Figure 5 conceptually illustrates a configuration diagram of a taste analysis model according to an embodiment.
Figure 6 shows examples of learning data for a taste analysis model.
Figure 7 exemplarily shows a flow chart for a taste analysis method according to an embodiment.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

In this specification, 'part' includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized using one piece of hardware.

In this specification, some of the operations or functions described as being performed by a terminal, apparatus, or device may instead be performed on a server connected to the terminal, apparatus, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed in a terminal, apparatus, or device connected to the server.

The present invention can be implemented as computer-readable code on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. . Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Additionally, the computer-readable recording medium can be distributed across computer systems connected to a network, so that computer-readable code can be stored and executed in a distributed manner.

Hereinafter, the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a block diagram for explaining a taste analysis system according to an embodiment of the present invention. Figure 2 is a block diagram for explaining the central server included in the system of Figure 1.

Referring to Figures 1 and 2, the taste analysis system 1000 may include at least one user terminal 100 and a central server 300. In addition, the taste analysis system 1000 is an external server that transmits external data for data analysis and processing in the central server 300, or receives data from the central server 300 for use of external services. may further include. However, since the taste analysis system 1000 of FIG. 1 is only an embodiment of the present invention, the present invention is not limited to the interpretation of FIG. 1.

At this time, each component of FIG. 1 is generally connected through a network (network, 10). For example, as shown in FIG. 1, at least one user terminal 100 may be connected to the central server 300 through the network 10. And, the central server 300 may be connected to at least one user terminal 100 through the network 10. Additionally, at least one external server may be connected to the central server 300 directly or through the network 10.

Here, the network refers to a connection structure that allows information exchange between nodes such as a plurality of terminals and servers. Examples of such networks include RF, 3rd Generation Partnership Project (3GPP) network, and Long Term (LTE) network. Evolution) network, 5GPP (5th Generation Partnership Project) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network) , PAN (Personal Area Network), Bluetooth (Bluetooth) network, NFC network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc., but are not limited thereto.

In the following, the term 'at least one' is defined as a term including singular and plural, and even if the term 'at least one' does not exist, each component may exist in singular or plural, and may mean singular or plural. It is self-evident that it exists. In addition, whether each component is provided in singular or plural form may be changed depending on the embodiment.

The user terminal 100 is owned by a person (hereinafter referred to as 'user') who registers preferred food and beverages through the taste analysis system 1000 and registers taste recognition data for at least one food or beverage. As a computer or portable terminal, after the user performs membership registration by communicating with the central server 300 through the network 10 in the form of a web, application, or web app, the central server 300 ) is configured to support the input of the information required by and the recommendation of food and beverages based on the information.

Meanwhile, the computer includes, for example, a desktop, laptop, tablet PC, etc. equipped with a web browser, and the portable terminal includes, for example, portability and It is a wireless communication device that guarantees mobility, including smartphone, PCS (Personal Communication System), GSM (Global System for Mobile communication), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), and PDA (Personal Digital Assistant). ), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, etc., all types of handheld It may include a wireless communication device based.

In addition, the network 10 connecting the user terminal 100 and the central server 300 includes a local area network (LAN), a wide area network (WAN), and a value added network; All types of networks such as VAN, Personal Area Network (PAN), mobile radio communication network, Wibro (Wireless Broadband Internet), Mobile WiMAX, HSDPA (High Speed Downlink Packet Access) or satellite communication network. It can be implemented as a wired/wireless network.

The central server 300 may be a server connected to a taste analysis service web page, app page, program, or application. In addition, the central server 300 receives information from user log information such as clicks, filtering information (including priority), and residence time input from the user terminal 100 or within a taste analysis service web page, app page, etc. User preference data, such as preferred food and beverage information, can be collected.

Additionally, the central server 300 may be a server that collects taste perception data for each user and analyzes taste perception characteristics for each user. In addition, the central server 300 may be a server that collects taste recognition data having a subjective value perceived by the user for at least one food or beverage preferred by each user and objectively measured taste measurement data of the corresponding food or beverage.

In addition, the central server 300 collects objectively measured taste measurement data information for each food and beverage, and based on deep learning, a taste analysis model that extracts common taste characteristics from at least one food or beverage preferred by each user or user group. It may be a learning server that generates . In addition, the central server 300 may be a matching server that matches taste measurement data for a specific food or beverage to a food or beverage group based on the extracted taste characteristics when taste measurement data for a specific food or beverage is input using the taste analysis model.

Here, the central server 300 may be implemented as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, a laptop equipped with a navigation system and a web browser, a desktop, a laptop, etc.

Figure 2 shows an example configuration diagram of a central server according to one embodiment. Referring to FIG. 2, the central server 300 includes a communication unit 310, a memory 320, and a processor 330. However, the configuration diagram shown in FIG. 2 is merely illustrative, and the spirit of the present invention is not limited to the configuration diagram shown in FIG. 2. For example, the central server 300 may include at least one configuration not shown in FIG. 2 or may not include at least one configuration shown in FIG. 2 .

When the central server 300 or another server (not shown) operating in conjunction with an embodiment of the present invention transmits a taste analysis application, program, app page, web page, etc. to at least one user terminal 100. , At least one user terminal 100 may install or open a taste analysis service application, program, app page, web page, etc.

Additionally, a service program may be run on at least one user terminal 100 using a script executed in a web browser. Here, a web browser is a program that allows the use of web (WWW: world wide web) services and refers to a program that receives and displays hypertext written in HTML (hypertext mark-up language). For example, Netscape, Includes Explorer, Chrome, etc. Additionally, an application refers to an application on a terminal and includes, for example, an app running on a mobile terminal (smartphone).

The central server 300 collects preferred food and beverage information based on taste recognition data registered by the user. Accordingly, the central server 300 extracts the user's taste characteristics based on the common food and beverage information preferred based on the collected food and beverage information for each user, and groups the users based on this. When users are grouped, common taste characteristics of food and beverages preferred by at least two people in each user group are extracted.

At this time, a taste analysis model learned based on deep learning may be used to extract taste characteristics. Specifically, taste measurement data is collected by at least one taste recognition device for each food or beverage, and the taste analysis model can extract the same or similar taste characteristics using the taste measurement data as learning data. For example, taste measurement data may include data on taste, aroma, texture, etc. for each food or beverage, and learning data may be a series of continuous data for each item or image data graphed for each item. In the latter case, the artificial neural network used in deep learning may be a GNN (Graph Neural Network).

That is, common taste characteristics of preferred food and beverages are extracted for each user group, and based on the taste characteristics, preferred food and beverages may be grouped into at least one food and beverage group. That is, multiple food and beverage groups may be matched to one user group. Additionally, if the food or beverage is generally preferred by the user, a food or beverage group may be set for the food or beverage. In this case, the food or beverage group may be matched to a plurality of user groups. That is, multiple user groups may be matched to one food and beverage group.

Accordingly, the central server 300 performs updates between user groups in the matched user group and food and beverage group according to the influx of new users or changes in user taste, and for the food and beverage group, existing Updates are made according to the acquisition of taste measurement data for food and beverages and the release of new food and beverages. Accordingly, the matching between a plurality of user groups and a plurality of food and beverage groups can be maintained up-to-date. Accordingly, through matching between the user group and the food and beverage group, the user can select at least one food and beverage belonging to the matched group into the food and beverage category. In addition, when launching a new food or beverage, a food and beverage manufacturer or marketing agency can establish an appropriate marketing strategy by providing information on the user group that matches the group to which the new food or beverage belongs. We can support you.

Hereinafter, for taste analysis, taste data collected from the central server 300 will be described.

Figure 3 is an example diagram to explain collecting taste recognition data and taste measurement data for each user.

Referring to FIG. 3 , with respect to food and beverages, subjective taste recognition data collected for each food and beverage through the user terminal 100 and objective taste measurement data measured by at least one taste recognition device are shown.

For example, the central server 300 is configured to collect the user's preferred food and beverage information and taste recognition data for each taste area perceived by the user for each food and beverage from the user terminal 100. Here, food and beverages include all types of food and beverages distributed as products that can be consumed by people, assuming that the taste of each product remains constant. In addition, cooking may be included, but is not limited, provided that it is manufactured or produced identically according to a certain recipe.

The taste recognition data is data in which the user scores food and beverages by taste area. The taste area includes sourness, saltiness, bitterness, sweetness, umami, etc. As an example, the user scores from 1 to 5 for each area. It can be configured to give scores up to. The score may indicate the degree of taste or include satisfaction. For example, it may sequentially indicate very insufficient, insufficient, satisfactory, excessive, and very excessive, but is not limited to this. In other words, taste perception data for each user is collected. Meanwhile, the user's primary taste data is collected based on taste perception data for food and beverages. Meanwhile, the user's preferred food and beverages that receive 3 points in each taste area can be treated as preferred foods. However, it is not limited to this. Users can also directly register their preferred food.

Additionally, the central server 300 is configured to collect taste measurement data measured by at least one taste recognition device for each food or beverage.

Additionally, the taste recognition device includes an electronic tongue. The electronic tongue is a taste recognition device that recognizes 9 types of taste felt with the tongue and converts them into numerical values. The 9 types of taste are sourness, bitterness, astringency, and aftertaste B. , aftertaste-A, umami, richness, saltiness, and sweetness. For example, the electronic tongue can be Insent's TS-5000z. However, it is not limited to this.

The taste recognition device may further include at least one of an electronic nose (smell indicator), a texture measuring device, and a chili sensor. The electronic nose is a taste recognition device that recognizes the scent felt by the nose with four sensors and converts it into a numerical value, and the texture measuring device is used to convert ambiguous oral feelings into a value. Specifically, it measures the hardness perceived in molars, canines, front teeth, etc. It is a taste recognition device that converts textures such as stickiness, stickiness, and crumbliness into numerical values, and the chili sensor is a taste recognition device that converts the level of spiciness into numerical values (Scoville number). Here, the electronic nose from smartnanotubes can be used, and the texture measuring device from JISC can be used. However, it is not limited to this. That is, it is configured to collect one or more taste measurement data measured from at least one taste recognition device for each food or beverage.

The taste measurement data includes a combination of values measured by each taste recognition device, and the taste measurement data can be stored and managed by matching the identification number of each taste recognition device. In addition, the central server 300 may be configured to collect public data from an external server, if necessary.

Hereinafter, with reference to FIGS. 2 and 4, the central server 300 will be examined in detail.

Referring to Figures 2 and 4, the central server 300 includes a communication unit 310, a memory 320, and a processor 330. However, the configuration diagram shown in FIG. 2 is merely illustrative, and the spirit of the present invention is not limited to the configuration diagram shown in FIG. 2. For example, the central server 300 may include at least one configuration not shown in FIG. 2 or may not include at least one configuration shown in FIG. 2 .

The communication unit 310 can be implemented using a wired or wireless communication module. The central server 300 can communicate with the user terminal 100 and an external server through the communication unit 310.

Memory 320 may store at least one instruction that can be executed by processor 330.

Additionally, the memory 320 may be a storage medium or a computer-readable recording medium that stores computer software that allows the processor 330 to perform operations according to embodiments of the present disclosure. In this case, the memory 320 may refer to computer readable media for storing software codes required to perform embodiments of the present disclosure, data to be executed by the codes, and execution results of the codes.

Here, such software that can be stored in the memory 320 may be a previously learned model that is implemented and operates with artificial intelligence (AI). For example, a model implemented or learned to perform a predetermined function by being implemented or learned using a machine learning method or deep learning may be stored in the memory 320. Of course, software modules that are not implemented with artificial intelligence but are implemented to operate in a general manner, for example, rule-based, may also be stored in the memory 320.

In one embodiment, memory 320 may refer to any type of storage medium. For example, memory 320 may be a flash memory type, hard disk type, or multimedia card. Micro type (multimedia card micro type), card type memory (e.g. SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM) , it may include at least one type of storage medium among EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

In one embodiment, the memory 320 may store a taste analysis model or a taste analysis module for matching a user with food and beverages. This taste analysis model or taste analysis module may be learned using machine learning or deep learning methods. In learning, a supervised learning method is used. Here, the taste analysis model is a model for extracting common taste characteristics from taste measurement data for a plurality of input food and beverages. When taste measurement data for a new food and beverage is input, at least one taste characteristic is extracted to group the food and beverages. It is configured to extract. The taste analysis model uses the collected taste measurement data as data for learning and extracts at least one taste characteristic.

The memory stores a taste analysis model previously learned based on deep learning to extract taste characteristics for each food and beverage, and the at least one command is performed by the processor, thereby producing taste measurement data by a taste recognition device for each food and beverage. is obtained, and the taste characteristics of each food and beverage are extracted by the taste analysis model using the obtained data.

Hereinafter, machine learning or deep learning, which is the basis of each model, will be explained in more detail.

Meanwhile, various types of information can be stored in the memory 320. For example, in the memory 320, information about each user and food and beverage may be listed and stored. This information may include information input directly from the user, such as the user's age, address, activity range, etc., and information input from the user, such as inclination, taste, personality, and preference, as well as type, cost, location, and brand of food and beverages. , information on trends, user-specific taste recognition data for food and beverages, and taste measurement data measured from each taste recognition device may be stored.

Additionally, various types of modules or models may be implemented in the memory 320. When these modules or models are executed by the processor 330, which will be described later, the desired function is performed.

Hereinafter, let's look at various operations or functions that the central server 300 can perform by executing at least one command stored in the memory 320 by the processor 330. Let's take a look at each module or model with reference to FIG. 4.

In Figure 4, functions executable on the central server according to one embodiment are conceptualized for each module or model. However, Figure 4 is merely illustrative, and the spirit of the present invention is not limited to what is shown in Figure 4.

Referring to FIG. 4, a matching unit 321, a taste analysis model 322, and a grouping unit 323 are implemented in the memory 320.

Among these, let's look at the matching unit 321 first. The matching unit 321 is implemented to match the preferred food and beverage for each user based on the taste recognition data registered by the user for each food and beverage. Additionally, based on food and beverage, users who prefer the food and beverage are matched. As described above, whether the food or beverage is preferred may be determined by the score for each taste area when preferred food and beverage information is directly registered by the user or based on taste recognition data for each food and beverage registered by the user.

Next, let's look at the taste analysis model 322. This taste analysis model 322 is implemented to extract taste characteristics from taste measurement data for preferred food and beverages matched for each user. Extraction methods vary, and depending on the embodiment, a method using an artificial neural network may be used. The artificial neural network itself will be described later, and hereinafter, we will only look at the operation method of the taste analysis model 322 itself.

Figure 5 conceptually illustrates a configuration diagram of a taste analysis model according to an embodiment. Figure 6 shows examples of learning data for a taste analysis model.

Referring to FIG. 5, taste measurement data 321 of matching food and beverages is input to the taste analysis model 322 as input. In the input form, taste measurement data can be input in the form of a series of data, graphed, and input as an image. However, it is not limited to this. In the case of machine learning or deep learning models, inference is possible only when data is input in a form that these models can understand. In one embodiment, the taste measurement data 321 is expressed in various ways that can represent the feature, so that such taste analysis is performed. May be input into model 322. As an example, the taste measurement data can be input in the form of a graph shown in Figure 6, and the taste analysis model 322 nodes the values for each taste area and links connecting the nodes that are the values for each taste area. Learning can be performed using a graph neural network based on the distance and slope of each node.

In addition, as an output inferred from the taste analysis model 322, taste characteristics are extracted from taste measurement data for each food and beverage, and based on the taste characteristics, a group 323 of previously grouped food and beverages is inferred and output. For example, assuming that food and beverages are grouped according to taste characteristics, if there are multiple extracted taste characteristics of a specific food or beverage, even one food or beverage may be matched into multiple groups. Of course, it is not limited to this.

In other words, by using taste measurement data with objective values for taste, aroma, texture, etc., which are elements that affect the user's taste buds, as nodes, the distance between data is learned, rather than extracting taste characteristics through simple comparison by numerical value. You can learn the balance of each taste area and the forms of various flavors. For example, in the case of chocolate, when the sweetness increases dramatically, it has a relatively small effect on the decrease in satisfaction, but when the sourness increases, satisfaction is bound to decrease significantly. In contrast, when sweetness and bitterness rise together, it does not have a strong effect on existing satisfaction. In other words, the interaction between the flavors of each food and beverage or the balance of each taste area is learned.

Figure 6 shows examples of learning data for a taste analysis model. Referring to Figure 6, in the case of the food and beverage in (a) and the food and beverage in (b), there is a difference in the taste measurement data, which is the value for each specific taste area, but there is a difference between richness, sourness, and bitterness. There is similarity in the change in value, so common taste characteristics can be extracted, and in this case, they can be grouped into the same group by the grouping unit 323, which will be described later.

In the grouping unit 323, food and beverages are grouped based on extracted taste characteristics. That is, a plurality of food and beverage groups are created according to taste characteristics, users are grouped based on taste recognition data for at least one food or beverage preferred by each user, and matching is performed between user groups and food and beverage groups. That is, a user group can be created by grouping users based on user-specific taste perception data for food and beverage, and matching is performed between a plurality of user groups and a plurality of food and beverage groups.

In addition, in grouping food and beverages based on extracted taste characteristics, common taste characteristics are used to classify each food and beverage, user Star grouping and updating can be done.

The processor 330 may perform technical features according to embodiments of the present disclosure, which will be described later, by executing at least one instruction stored in the memory 320. In one embodiment, the processor 330 may consist of at least one core, such as a central processing unit (CPU) and a general purpose graphics processing unit (GPGPU) of the central server 300. , may include a processor for data analysis and/or processing, such as a tensor processing unit (TPU).

This processor 330 can learn a neural network or model designed using machine learning or deep learning. The above-described taste analysis model may be learned by this processor 330.

To this end, the processor 330 may perform calculations for learning the neural network, such as processing input data for learning, extracting features from the input data, calculating errors, and updating the weights of the neural network using backpropagation. . At least one of the CPU, GPGPU, and TPU of the processor 330 may process learning of the network function. For example, CPU and GPGPU can work together to process learning of network functions and data classification using network functions.

Additionally, the processor 330 may perform inference for a predetermined purpose using a neural network or model that has already been trained using machine learning or deep learning.

Here, the model or neural network that has been trained using machine learning or deep learning is based on artificial intelligence. Below, let’s look at neural networks in detail.

First, in this specification, neural network may be used interchangeably with model, network function, neural network, etc. A neural network can generally consist of a set of interconnected computational units, which can be referred to as nodes. These nodes may also be referred to as neurons. A neural network consists of at least one node. Nodes (or neurons) that make up neural networks may be interconnected by one or more links.

Within a neural network, one or more nodes connected through a link may form a relative input node and output node relationship. The concepts of input node and output node are relative, and any node in an output node relationship with one node may be in an input node relationship with another node, and vice versa. As described above, input node to output node relationships can be created around links. One or more output nodes can be connected to one input node through a link, and vice versa.

In a relationship between an input node and an output node connected through one link, the value of the data of the output node may be determined based on the data input to the input node. Here, the link connecting the input node and the output node may have a weight. Weights may be variable and may be varied by the user or algorithm in order for the neural network to perform the desired function. For example, when one or more input nodes are connected to one output node by respective links, the output node is set to the values input to the input nodes connected to the output node and the links corresponding to each input node. The output node value can be determined based on the weight.

As described above, in a neural network, one or more nodes are interconnected through one or more links to form an input node and output node relationship within the neural network. The characteristics of the neural network can be determined according to the number of nodes and links within the neural network, the correlation between the nodes and links, and the value of the weight assigned to each link. For example, if the same number of nodes and links exist and two neural networks with different weight values of the links exist, the two neural networks may be recognized as different from each other.

A neural network may consist of a set of one or more nodes. A subset of nodes that make up a neural network can form a layer. Some of the nodes constituting the neural network may form one layer based on the distances from the first input node. For example, a set of nodes with a distance n from the initial input node may constitute n layers. The distance from the initial input node can be defined by the minimum number of links that must be passed to reach the node from the initial input node. However, this definition of a layer is arbitrary for explanation purposes, and the order of a layer within a neural network may be defined in a different way than described above. For example, a layer of nodes may be defined by distance from the final output node.

The initial input node may refer to one or more nodes in the neural network through which data is directly input without going through links in relationships with other nodes. Alternatively, in a neural network network, in the relationship between nodes based on links, it may mean nodes that do not have other input nodes connected by links. Similarly, the final output node may refer to one or more nodes that do not have an output node in their relationship with other nodes among the nodes in the neural network. Additionally, hidden nodes may refer to nodes constituting a neural network other than the first input node and the last output node.

The neural network according to an embodiment of the present disclosure is a neural network in which the number of nodes in the input layer may be the same as the number of nodes in the output layer, and the number of nodes decreases and then increases again as it progresses from the input layer to the hidden layer. You can. In addition, the neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer may be less than the number of nodes in the output layer, and the number of nodes decreases as it progresses from the input layer to the hidden layer. there is. In addition, the neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer may be greater than the number of nodes in the output layer, and the number of nodes increases as it progresses from the input layer to the hidden layer. You can. A neural network according to another embodiment of the present disclosure may be a neural network that is a combination of the above-described neural networks.

A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to the input layer and output layer. Deep neural networks allow you to identify latent structures in data. Deep neural networks include convolutional neural networks (CNN), recurrent neural networks (RNN), auto encoders, generative adversarial networks (GAN), and restricted Boltzmann machines (RBM). machine), deep belief network (DBN), Q network, U network, Siamese network, Generative Adversarial Network (GAN), transformer, etc. The description of the deep neural network described above is only an example and the present disclosure is not limited thereto.

Meanwhile, the deep neural network according to this embodiment may be a GNN (Graph Neural Network) that learns taste measurement data graphed for each taste item as image data.

A neural network can be learned in at least one of supervised learning, unsupervised learning, semi-supervised learning, self-supervised learning, or reinforcement learning. You can. Learning of a neural network may be a process of applying knowledge for the neural network to perform a specific operation to the neural network.

Neural networks can be trained to minimize output errors. In neural network learning, learning data is repeatedly input into the neural network, the output of the neural network and the error of the target for the learning data are calculated, and the error of the neural network is transferred from the output layer of the neural network to the input layer in the direction of reducing the error. This is the process of updating the weight of each node in the neural network through backpropagation. In the case of supervised learning, data in which the correct answer is labeled (i.e., labeled learning data) is used in each learning data (i.e., labeled learning data), and in the case of unsupervised learning, data in which the correct answer is not labeled in each learning data (i.e., labeled learning data) is used. unlabeled data) can be used. That is, for example, in the case of supervised learning on data classification, the learning data may be data in which each training data is labeled with a category corresponding to the correct answer. Labeled training data is input to the neural network, and the error can be calculated by comparing the output (category) of the neural network with the label of the training data. As another example, in the case of unsupervised learning, semi-supervised learning, or self-learning regarding data classification, the error may be calculated by comparing the input learning data with the neural network output. The calculated error is backpropagated in the reverse direction (i.e., from the output layer to the input layer) in the neural network, and the connection weight of each node in each layer of the neural network can be updated according to backpropagation. The amount of change in the connection weight of each updated node may be determined according to the learning rate. The neural network's calculation of input data and backpropagation of errors can constitute a learning cycle (epoch). The learning rate may be applied differently depending on the number of repetitions of the learning cycle of the neural network. For example, in the early stages of neural network training, a high learning rate can be used to increase efficiency by allowing the neural network to quickly achieve a certain level of performance, and in the later stages of training, a low learning rate can be used to increase accuracy.

In the learning of neural networks, the training data can generally be a subset of real data (i.e., the data to be processed using the learned neural network), and thus the error for the training data is reduced, but the error for the real data is reduced. There may be an incremental learning cycle. Overfitting is a phenomenon in which errors in actual data increase due to excessive learning on training data. Overfitting can cause errors in machine learning algorithms to increase. To prevent such overfitting, various optimization methods can be used. To prevent overfitting, methods such as increasing the learning data, regularizing it, using dropout to disable some of the network nodes during the learning process, and using a batch normalization layer are used. It can be applied.

Hereinafter, let's look at the flow chart for the taste analysis method, referring to Figure 7 together with Figures 2 to 6.

In Figure 7, a specific example of a method for analyzing the taste of food and beverages according to an embodiment is illustratively shown as a flowchart. The method shown in FIG. 7 or each step included in this method is performed by the central server 300 shown in FIGS. 2 to 6, specifically, at least one instruction stored in the memory 320 by the processor 330. It can be done by running it. In addition, this flow chart shown in FIG. 7 is merely illustrative, and the spirit of the present invention is not limited to what is shown in FIG. 7. For example, depending on the embodiment, each step may be performed in a different order from that shown in FIG. 7, or at least one step not shown in FIG. 7 may be additionally performed, or at least one of the steps shown in FIG. 7 may be performed. It may not work.

Referring to FIG. 7, a step (S100) of collecting preferred food and beverage information for each user is performed. At this stage, users can be grouped based on common food and beverage preferences for each user.

In addition, the central server collects objective taste measurement data for each food and beverage using at least one taste recognition device (S200).

In this step, it is configured to collect taste measurement data measured by at least one taste recognition device for each food or beverage.

Additionally, the taste recognition device includes an electronic tongue. The electronic tongue is a taste recognition device that recognizes 9 types of taste felt with the tongue and converts them into numerical values. The 9 types of taste are sourness, bitterness, astringency, and aftertaste B. , aftertaste-A, umami, richness, saltiness, and sweetness. For example, the electronic tongue can be Insent's TS-5000z. However, it is not limited to this.

The taste recognition device may further include at least one of an electronic nose (smell indicator), a texture measuring device, and a chili sensor. The electronic nose is a taste recognition device that recognizes the scent felt by the nose with four sensors and converts it into a numerical value, and the texture measuring device is used to convert ambiguous oral feelings into a value. Specifically, it measures the hardness perceived in molars, canines, front teeth, etc. It is a taste recognition device that converts textures such as stickiness, stickiness, and crumbliness into numerical values, and the chili sensor is a taste recognition device that converts the level of spiciness into numerical values (Scoville number). Here, the electronic nose from smartnanotubes can be used, and the texture measuring device from JISC can be used. However, it is not limited to this. That is, it is configured to collect one or more taste measurement data measured from at least one taste recognition device for each food or beverage.

The taste measurement data includes a combination of values measured by each taste recognition device, and the taste measurement data can be stored and managed by matching the identification number of each taste recognition device.

In addition, in the step (S300) of extracting taste characteristics for preferred food and beverages based on deep learning, taste measurement data of a plurality of food and beverages preferred by each user, or users sharing the same preferred food and beverage, are based on deep learning. Common taste characteristics of food and beverages can be extracted from taste measurement data for common foods and beverages preferred by at least two or more users in a user group consisting of. For example, the taste characteristics may be extracted based on a GNN (Graph Neural Network)-based artificial neural network model that extracts similar characteristics from a graph for each taste region.

Additionally, it may include grouping food and beverages based on the extracted taste characteristics (S400). That is, using common taste characteristics, grouping by food and beverage can be performed using taste measurement data for existing food and beverages or new food and beverages for which there is no data on user preferences.

Hereinafter, since the method according to this embodiment is performed by the central server 300 shown in FIG. 3, the parts already described regarding the central server 300 will be used.

As discussed above, according to one embodiment, in the taste analysis process of food and beverage, similar food and beverages can be grouped from taste measurement data for various types of food and beverage, and food and beverage groups are divided into user groups grouped according to the user's taste. Matching becomes possible.

In addition, in these matched user groups and food and beverage groups, user groups are updated among user groups according to the influx of new users or changes in users' tastes, and the food and beverage group acquires taste measurement data for existing food and beverages. And updates are made according to the release of new food and beverages. Accordingly, matching between a plurality of user groups and a plurality of food and beverage groups can be maintained up-to-date.

In addition, through matching between user groups and food and beverage groups, at least one food and beverage belonging to the matched group can be recommended to the user according to the food and beverage category, and food and beverage manufacturers or marketing agencies can recommend new food and beverage products. In this way, it is possible to support the establishment of an appropriate marketing strategy by providing information on the user group that matches the group to which the new food and beverage belongs.

Meanwhile, the method according to the above-described embodiment can be implemented in the form of a computer program implemented to include computer instructions for causing a processor to perform each step included in the method. At this time, such a computer program may be stored in a computer-readable recording medium.

Alternatively, the method according to the above-described embodiment may be implemented in the form of a computer-readable recording medium including a computer program including computer instructions for causing a processor to perform each step included in the method.

Combinations of each step in each flowchart attached to the present invention may be performed by computer program instructions. Since these computer program instructions can be mounted on the processor of a general-purpose computer, special-purpose computer, or other programmable data processing equipment, the instructions performed through the processor of the computer or other programmable data processing equipment perform the functions described in each step of the flowchart. It creates the means to carry out these tasks. These computer program instructions may also be stored on a computer-usable or computer-readable recording medium that can be directed to a computer or other programmable data processing equipment to implement a function in a particular way, so that the computer program instructions are computer-usable or computer-readable. The instructions stored in the recording medium can also produce manufactured items containing instruction means that perform the functions described in each step of the flowchart. Computer program instructions can also be mounted on a computer or other programmable data processing equipment, so that a series of operational steps are performed on the computer or other programmable data processing equipment to create a process that is executed by the computer, thereby generating a process that is executed by the computer or other programmable data processing equipment. Instructions that perform processing equipment may also provide steps for executing the functions described in each step of the flowchart.

Additionally, each step may represent a module, segment, or portion of code containing one or more executable instructions for executing specified logical function(s). Additionally, it should be noted that in some alternative embodiments it is possible for the functions mentioned in the steps to occur out of order. For example, two steps shown in succession may in fact be performed substantially simultaneously, or the steps may sometimes be performed in reverse order depending on the corresponding function.

The above description is merely an illustrative explanation of the technical idea of the present invention, and those skilled in the art will be able to make various modifications and variations without departing from the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention shall be interpreted in accordance with the claims below, and all technical ideas within the scope equivalent thereto shall be construed as being included in the scope of rights of the present invention.

100: user terminal
300: central server
310: Department of Communications
320: memory
330: processor

Claims (10)

As a central server that performs taste analysis of each user's preferred food and beverage,
a memory storing at least one instruction; and
Contains a processor,
As the at least one instruction is performed by the processor,
Collect information about at least one food or drink preferred by each user,
Creating a user group by grouping users based on user-specific taste perception data for the at least one food or beverage,
Extracting common taste characteristics from taste measurement data for each food and beverage for the at least one food or beverage measured from a taste recognition device based on a taste analysis model using a deep neural network,
Based on the common taste characteristics, grouping at least one food and beverage to create a food and beverage group,
Perform matching between the food and beverage groups and the user groups, matching at least one user group to one food and beverage group,
The taste analysis model is,
Numerical values for each taste area are noded, the taste measurement data with the numerical value of each taste area is used as a node, and a graph neural network based on at least one of the distance and slope of each node is used through a link connecting the nodes. A central server on which learning is performed. According to claim 1,
The deep neural network is a central server, characterized in that it is a GNN (Graph Neural Network) based on the characteristics of taste measurement data graphed for each item. According to claim 1,
The central server, wherein the taste recognition device includes at least one of an electronic tongue, a chili sensor, an electronic nose, and a texture measuring device. According to claim 1,
A central server, wherein at least one food or beverage preferred by each user is determined based on taste recognition data for each taste area registered by the user for each food or beverage. According to claim 4,
The taste area includes at least one of sour, salty, bitter, sweet, and umami, and the taste recognition data includes a numerical score for each area. delete According to claim 1,
After performing the matching, an update is performed on at least one of the food and beverage group and the user group,
When a change in the user's taste or an influx of new users occurs, an update is performed between the user groups, and when taste measurement data for the at least one food and beverage is acquired and a new food and beverage is released, an update is performed between the food and beverage groups. A central server, characterized in that performing. According to claim 1,
The memory stores a taste analysis model previously learned based on deep learning to extract the common taste characteristics,
When the at least one command is performed by the processor, taste measurement data is obtained by a taste recognition device for each food and beverage,
The common taste characteristics are extracted by the taste analysis model using the acquired data. A method of analyzing the taste of each user's preferred food and beverage performed on a central server.
Collecting information about at least one food or drink preferred by each user;
Creating a user group by grouping users based on user-specific taste perception data for the at least one food or beverage;
Extracting common taste characteristics from taste measurement data for each food and beverage for the at least one food and beverage measured from a taste recognition device based on a taste analysis model using a deep neural network;
Grouping food and beverages based on the common taste characteristics to create a plurality of food and beverage groups; and
Performing matching between the food and beverage groups and the user groups, comprising matching at least one user group to one food and beverage group,
The taste analysis model is,
Numerical values for each taste area are noded, the taste measurement data with the numerical value of each taste area is used as a node, and a graph neural network based on at least one of the distance and slope of each node is used through a link connecting the nodes. A method for analyzing the taste of food and beverages, including where learning is performed. A computer-readable recording medium storing a computer program programmed to perform each step included in the method according to claim 9.