WO2023235247A1 - Methods and systems for predicting quantitative measures of food items from text - Google Patents

Methods and systems for predicting quantitative measures of food items from text Download PDF

Info

Publication number
WO2023235247A1
WO2023235247A1 PCT/US2023/023683 US2023023683W WO2023235247A1 WO 2023235247 A1 WO2023235247 A1 WO 2023235247A1 US 2023023683 W US2023023683 W US 2023023683W WO 2023235247 A1 WO2023235247 A1 WO 2023235247A1
Authority
WO
WIPO (PCT)
Prior art keywords
food
data
text data
feature
text
Prior art date
Application number
PCT/US2023/023683
Other languages
French (fr)
Inventor
Mark Woodward
Saransh Agarwal
Original Assignee
January, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by January, Inc. filed Critical January, Inc.
Publication of WO2023235247A1 publication Critical patent/WO2023235247A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/22Social work
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/044Recurrent networks, e.g. Hopfield networks
    • G06N3/0442Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/01Customer relationship services
    • G06Q30/015Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201Market modelling; Market analysis; Collecting market data
    • G06Q30/0202Market predictions or forecasting for commercial activities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0282Rating or review of business operators or products
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0623Item investigation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/12Hotels or restaurants

Definitions

  • Quantitative measures of foods are often valuable to consumers.
  • quantitative measures of foods are nutrient quantities, such as calories, carbohydrates, fats, and proteins, in addition to diets that the food is consistent with, such as the vegetarian diet, the vegan diet, the Atkins diet, and the paleo diet. Additionally, quantitative measures could be non-nutrient, non-dietary in nature, such as whether the food was fermented or whether it contains red meat.
  • the present disclosure provides a method for predicting quantitative measures for food items from text or other descriptions of the food items.
  • a user device can generate and apply food text data to a feature prediction machine learning (ML) system that can predict one or more quantitative food features.
  • the feature prediction ML system can include one or more artificial neural network trained with the training data comprising training food text data and corresponding training feature data; and provide the at least one quantitative food feature on the user device.
  • a method comprising: by operation of a user device, generate food text data; apply the generated food text data to a feature prediction machine learning (ML) system to predict at least one quantitative food feature, the feature prediction ML system comprising at least one artificial neural network trained with the training data comprising training food text data and corresponding training feature data; and provide the at least one quantitative food feature on the user device.
  • ML machine learning
  • the ML system comprises an artificial neural network.
  • the artificial neural network comprises a recurrent network.
  • the recurrent network is selected from the group of a long short-term memory and a gated recurrent unit.
  • the artificial neural network comprises an attention component.
  • the attention components comprise a transformer network.
  • generating food text data comprises capturing text data with the user device.
  • capturing the text data includes capturing non-text food data with the user device and converting the non-text food data to the generated food text data. [0013] In some embodiments, capturing text data is selected from the group of: menu text data, recipe text data and food log data.
  • the quantitative feature comprises a diet compliance value. [0015] In some embodiments, the quantitative feature comprises nutrient information.
  • the method further includes by operation of a trained expert machine learning system, generating the training feature data corresponding to the training food text data.
  • a system comprising: a user device configured to generate food text data and provide predicted quantitative food feature data for the generated food text data; at least one machine learning (ML) system in communication with the user device comprising at least one artificial neural network trained with the training data comprising training food text data and corresponding training quantitative food feature data, the ML system configured to receive the generated food text data, generate the predicted quantitative food feature data therefrom, and communicate the quantitative food feature data to the user device.
  • ML machine learning
  • the ML system comprises an artificial neural network.
  • the artificial neural network comprises a recurrent network.
  • the recurrent network is selected from the group of a long short-term memory and a gated recurrent unit.
  • the artificial neural network comprises an attention component.
  • the attention component comprises a transformer network.
  • the user device is configured to capture the food text data from a source external to the user device.
  • the user device is configured to capture non-text food data with the user device, and convert the non-text food data to the generated food text data.
  • the quantitative feature comprises a diet compliance value.
  • the quantitative feature comprises nutrient information.
  • the system further includes at least one expert system configured to generate the training feature data from the training food text data.
  • the at least one expert system comprises an expert ML system.
  • FIGS. 1A and IB are diagrams showing method and systems for predicting quantitative food information from food text according to embodiments.
  • FIGS. 2A and 2B are diagrams showing method and systems for predicting nutrition and diet information from food text according to embodiments.
  • FIG. 3 is a diagram of a system according to an embodiment.
  • FIGS. 4A to 4C are diagrams of a user device and operations according to embodiments.
  • FIG. 5 shows a computer system that is programmed or otherwise configured to implement methods provided herein.
  • Embodiments describe a method for creating a system that takes as input a text description of food and outputs quantitative measures of that food.
  • an example dataset is collected containing text descriptions of food items and corresponding quantitative measures of the food items described.
  • the text descriptions could be from food items listed on menus.
  • a nutritionist or other specialist may estimate nutrients and dietary consistency.
  • Weights of an artificial neural network could then be adjusted, for example through the use of a backpropagation algorithm, such that the artificial neural network predicts quantitative measures for the text description that closely agree with the example quantitative measures from the dataset according to an appropriate error measure.
  • FIG. 1A is a diagram showing methods and systems 100 according to embodiments.
  • a system 100 can include sets of food text data 102, an expert 104 and a machine learning (ML) system 105.
  • ML machine learning
  • sets of food text data 102 can be acquired. Such sets 102 can be acquired in any suitable fashion, including but not limited to: food menu descriptions, labels, advertisements, or generated with other systems from food images.
  • An expert 104 can generate one or more features 106 corresponding to each food text data set.
  • experts 104 can include, but are not limited to: one or more individuals having expertise in identifying food features, a machine system (including an artificial intelligence system), or a food manufacturer/producer.
  • Features 106 can include any suitable feature corresponding to a food, including but not limited to: nutrient data, dietary consistency (e.g., how much the food may comply with a particular diet), availability (e.g., geographical/geolocation data), cost or demographic match (e.g., does a particular demographic population prefer this type of food).
  • Features 106 can include one feature, the probability of one feature (e.g., of the one feature), multiple features and/or the probabilities of multiple features (e.g., of the multiple features).
  • Food text and corresponding features 102/106 can serve as training data for ML system 105.
  • Food text 102 can serve as input values while feature data 106 can serve to evaluate a learning system predicted value.
  • An ML system 105 can take any suitable form.
  • an ML system 105 can include an artificial neural network.
  • Non-limiting examples of neural networks may comprise a recurrent network and/or a transformer network.
  • a ML system 105 can include function/model 108, error function 110, and parameter adjust operation 112.
  • a function/model 108 can predict one or more features 107 based on parameters.
  • processing of text data sets 102 by ML system 105 can include encoding positions of words with respect to one another in the text data.
  • An error function 110 can determine an error 109 between features 107 predicted by function/model 108 and the expert generated features 106 corresponding to the input food text data set.
  • Parameter adjust operation 112 can adjust parameters of function/model 108 based on error values 109 from error function 110 to attempt to converge on predictions with less error.
  • FIG. IB shows a prediction system and operations 100B according to embodiments.
  • a system 100B can include a trained function/model 108B.
  • a trained function/model 108B can be a function/model 108 after being trained with multiple food textfeature data sets 102/106, as shown in FIG. 1A.
  • a trained function/model 108B can receive new food text data 102B, and in response thereto, generate one or more predicted features 106B.
  • FIG. 2A is a diagram showing methods and systems 200 according to additional embodiments.
  • FIG. 2A can include features like those of FIG. 1A, including sets of food text data 202, an expert 204, and a ML system 205.
  • an expert 204 can result in features 206 for text data sets 202 that can include nutrition information (e.g., Nutrition Facts) and/or dietary consistency.
  • nutrition information e.g., Nutrition Facts
  • Diet x e.g., Nutrition Facts
  • Dietary consistency can include, but is not limited to including, whether the food text indicates compliance with a diet, or a probability of compliance with a diet of the food item corresponding to the text.
  • Nutrition can include any suitable nutrition data, including but not limited to: content (e.g., calories, fats, cholesterol, sodium, carbohydrates, fiber, sugars, proteins, minerals, vitamins), rating (e.g., blood glucose impact, blood pressure impact, or other health related scoring).
  • FIG. 2B shows methods and systems 200B according to another embodiment.
  • a system 200B can include a trained function/model 208B, such as function/model 208 trained as shown in FIG. 2A.
  • a trained function/model 208B can generate predicted nutrition and/or dietary consistency information 206B.
  • FIG. 3 shows a system 300 according to another embodiment.
  • a system 300 can include one or more ML servers 330, application servers 328, data store 326, a food data source 350, subject devices 332 and one or more experts 304.
  • Food data source 350, servers (330/328), subject devices 332 and expert 304 can be in communication with one another, such as through a network 334, which can include various interconnected networks, including the internet.
  • ML server 330 can include systems and execute methods like those described herein and equivalents.
  • An application server 328 can interact with one or more applications 338 running on a subject device 332.
  • data from food data sources 350 can be acquired via one or more applications 338 on a subject device (e.g., smart phone) 332 and provided to application server 328.
  • Application server 328 can communicate with subject device 332 according to any suitable secure network protocol.
  • An expert 304 can be an expert as described herein and equivalents.
  • An expert 304 can receive food text descriptions (e.g., food text data sets).
  • food text descriptions can include food text data sets 302B from a data store 326 or from sources external to system 300 302A.
  • an expert 304 can generate feature data 306 for each food text data set, as described herein and equivalents.
  • Such features can include nutrition information and/or dietary consistency data. This can create training data sets 302/306.
  • Training data sets 302/306 can be used to train function/models 308 within ML servers 330.
  • a data store 328 can store data for a system 300.
  • data store 328 can store data received from food data source 330, food text data sets 302B, as well as generated training sets 302/306.
  • a data store 328 can take any suitable form, including one or more network attached storage systems. In some embodiments, all or a portion of data store can be integrated with any of the servers (330, 328).
  • food text data 302 can be generated by a user.
  • text data 352-0 can be entered by a user (e.g.., typed) or captured by a user (e.g., voice recognition, image capture).
  • image data can be images of actual food (e.g., 352-1), or food packaging, and such image data can be applied to an application on a server to generate a food text set for the image.
  • a subject device 332 can be any suitable device, including but not limited to, a smart phone, personal computer, wearable device, or tablet computing device.
  • a subject device 332 can include one or more applications 338 that can communicate with application server 328 to provide data to, and receive data from, models/functions 308 residing on ML servers 330.
  • Such applications 328 can take the form of, and/or include the functions of the various embodiments disclosed herein.
  • FIG. 4A to 4C are diagrams of an application 438 according to embodiments.
  • an application 438 can reside on a subject device 432 as instructions executable by a processor or the like, in an embodiment.
  • an application 438 can request/receive food text data 402B.
  • Such food text data 402B can be acquired in any suitable fashion including as disclosed herein and equivalents.
  • Received food text data 402B can be subject to a trained model/function to generate feature data according to any of the embodiments described herein and equivalents.
  • Such generation of feature data can be performed by the device 432 (e.g., model/function resides on the device) or can be performed by a remote ML server 442, or a combination thereof.
  • predicted feature data 406A/B from a trained model/function can be presented on subject device 432.
  • predicted feature data can include nutrition information 406A and/or dietary consistency 406B.
  • alternate embodiments can present any other suitable features generated by training with food text data sets.
  • the present disclosure can provide methods for performing at least the following: predicting quantitative measures of a food item from text about the food item using an artificial neural network; where the artificial neural network is trained using examples of text and quantitative measures; where the artificial neural network contains a recurrent network such as an LSTM (long short-term memory), a GRU (gated recurrent unit) and/or an attention component, such as a transformer network.
  • Quantitative measures can be nutrients. Such nutrients can be carbohydrates, fats, dietary fiber, minerals, proteins, vitamins, or water. Quantitative measures can be whether the food adheres to a diet.
  • Such diets can include, but are not limited to: the paleo diet, the vegan diet, the low-carb diet, the vegetarian diet, the Atkins diet, the HCG (human chorionic gonadotropin) diet, the Zone diet.
  • Food item text can be menu item text.
  • Food item text can be free form user text food logs
  • various blocks shown in the figures described herein can include any of various circuits configured to execute the indicated functions, including but not limited to server systems that may or may not include customized hardware for accelerating operations, logic circuits, including custom logic circuits or programmable logic circuits. Such functions can also correspond to all or a portion of code executable by one or more processors that is stored on machine readable media. Data values as described herein can also be stored in machine readable media. Machine readable media can store code and/or data in a non-transitory form, in volatile and/or nonvolatile storage circuits.
  • blocks or actions that do not depend upon each other can be arranged or executed in parallel.
  • FIG. 5 shows a computer system 501 that is programmed or otherwise configured to predict food quantities by analyzing text data.
  • the computer system 501 can regulate various aspects of analyzing text of the present disclosure, such as, for example, implementing machine learning algorithms.
  • the computer system 501 can be an electronic device of a user or a computer system that is remotely located with respect to the electronic device.
  • the electronic device can be a mobile electronic device or a desktop computing device.
  • the computer system 501 includes a central processing unit (CPU, also “processor” and “computer processor” herein) 505, which can be a single core or multi core processor, or a plurality of processors for parallel processing.
  • the computer system 501 also includes memory or memory location 510 (e.g., random-access memory, read-only memory, flash memory), electronic storage unit 515 (e.g., hard disk), communication interface 520 (e.g., network adapter) for communicating with one or more other systems, and peripheral devices 525, such as cache, other memory, data storage and/or electronic display adapters.
  • the memory 510, storage unit 515, interface 520 and peripheral devices 525 are in communication with the CPU 505 through a communication bus (solid lines), such as a motherboard.
  • the storage unit 515 can be a data storage unit (or data repository) for storing data.
  • the computer system 501 can be operatively coupled to a computer network (“network”) 530 with the aid of the communication interface 520.
  • the network 530 can be the Internet, an internet and/or extranet, or an intranet and/or extranet that is in communication with the Internet.
  • the network 530 in some cases is a telecommunication and/or data network.
  • the network 530 can include one or more computer servers, which can enable distributed computing, such as cloud computing.
  • the network 530 in some cases with the aid of the computer system 501, can implement a peer-to-peer network, which may enable devices coupled to the computer system 501 to behave as a client or a server.
  • the CPU 505 can execute a sequence of machine-readable instructions, which can be embodied in a program or software.
  • the instructions may be stored in a memory location, such as the memory 510.
  • the instructions can be directed to the CPU 505, which can subsequently program or otherwise configure the CPU 505 to implement methods of the present disclosure. Examples of operations performed by the CPU 505 can include fetch, decode, execute, and writeback.
  • the CPU 505 can be part of a circuit, such as an integrated circuit.
  • a circuit such as an integrated circuit.
  • One or more other components of the system 501 can be included in the circuit.
  • the circuit is an application specific integrated circuit (ASIC).
  • the storage unit 515 can store files, such as drivers, libraries and saved programs.
  • the storage unit 515 can store user data, e.g., user preferences and user programs.
  • the computer system 501 in some cases can include one or more additional data storage units that are external to the computer system 501, such as located on a remote server that is in communication with the computer system 501 through an intranet or the Internet.
  • the computer system 501 can communicate with one or more remote computer systems through the network 530.
  • the computer system 501 can communicate with a remote computer system of a user (e.g., a mobile device).
  • remote computer systems include personal computers (e.g., portable PC), slate or tablet PC’s (e.g., Apple® iPad, Samsung® Galaxy Tab), telephones, Smart phones (e.g., Apple® iPhone, Android-enabled device, Blackberry®), or personal digital assistants.
  • the user can access the computer system 501 via the network 530.
  • Methods as described herein can be implemented by way of machine (e.g., computer processor) executable code stored on an electronic storage location of the computer system 501, such as, for example, on the memory 510 or electronic storage unit 515.
  • the machine executable or machine readable code can be provided in the form of software.
  • the code can be executed by the processor 505.
  • the code can be retrieved from the storage unit 515 and stored on the memory 510 for ready access by the processor 505.
  • the electronic storage unit 515 can be precluded, and machine-executable instructions are stored on memory 510.
  • the code can be pre-compiled and configured for use with a machine having a processer adapted to execute the code, or can be compiled during runtime.
  • the code can be supplied in a programming language that can be selected to enable the code to execute in a pre-compiled or as-compiled fashion.
  • aspects of the systems and methods provided herein can be embodied in programming.
  • Various aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of machine (or processor) executable code and/or associated data that is carried on or embodied in a type of machine readable medium.
  • Machine-executable code can be stored on an electronic storage unit, such as memory (e.g., read-only memory, random-access memory, flash memory) or a hard disk.
  • “Storage” type media can include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server.
  • another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links.
  • a machine readable medium such as computer-executable code
  • a tangible storage medium such as computer-executable code
  • Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings.
  • Volatile storage media include dynamic memory, such as main memory of such a computer platform.
  • Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system.
  • Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications.
  • RF radio frequency
  • IR infrared
  • Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code and/or data.
  • Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.
  • the computer system 501 can include or be in communication with an electronic display 535 that comprises a user interface (UI) 540 for providing, for example, a dashboard.
  • UI user interface
  • Examples of UFs include, without limitation, a graphical user interface (GUI) and web-based user interface.
  • Methods and systems of the present disclosure can be implemented by way of one or more algorithms.
  • An algorithm can be implemented by way of software upon execution by the central processing unit 505. The algorithm can, for example, analyze food text data.
  • the term “at least,” “greater than,” or “greater than or equal to” precedes the first numerical value in a series of two or more numerical values the term “at least,” “greater than” or “greater than or equal to” applies to each of the numerical values in that series of numerical values. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.

Landscapes

  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Strategic Management (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • General Physics & Mathematics (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Tourism & Hospitality (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Game Theory and Decision Science (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Primary Health Care (AREA)
  • Human Resources & Organizations (AREA)
  • Child & Adolescent Psychology (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

A method can include, by operation of a user device, generating food text data. Generated food text data can be applied to a feature prediction machine learning system (ML) to predict at least one quantitative food feature. The feature prediction ML system can include at least one artificial neural network trained with the training data comprising training food text data and corresponding training feature data. The at least one quantitative food feature can be provided to the user device. Corresponding systems and devices are also disclosed.

Description

METHODS AND SYSTEMS FOR PREDICTING QUANTITATIVE MEASURES OF FOOD ITEMS FROM TEXT
CROSS-REFERENCE
[0001] This application claims priority to U.S. Provisional Patent Application No. 63/347,307, filed May 31, 2022, which is entirely herein incorporated by reference.
BACKGROUND
[0002] Quantitative measures of foods are often valuable to consumers. Examples of quantitative measures of foods are nutrient quantities, such as calories, carbohydrates, fats, and proteins, in addition to diets that the food is consistent with, such as the vegetarian diet, the vegan diet, the Atkins diet, and the paleo diet. Additionally, quantitative measures could be non-nutrient, non-dietary in nature, such as whether the food was fermented or whether it contains red meat.
[0003] It may be difficult for consumers to extract these characteristics (e.g., diet consistency or nutrient quantities) from text describing food items.
SUMMARY
[0004] In one aspect, the present disclosure provides a method for predicting quantitative measures for food items from text or other descriptions of the food items. A user device can generate and apply food text data to a feature prediction machine learning (ML) system that can predict one or more quantitative food features. The feature prediction ML system can include one or more artificial neural network trained with the training data comprising training food text data and corresponding training feature data; and provide the at least one quantitative food feature on the user device.
[0005] In an aspect, a method is disclosed, comprising: by operation of a user device, generate food text data; apply the generated food text data to a feature prediction machine learning (ML) system to predict at least one quantitative food feature, the feature prediction ML system comprising at least one artificial neural network trained with the training data comprising training food text data and corresponding training feature data; and provide the at least one quantitative food feature on the user device.
[0006] In some embodiments, the ML system comprises an artificial neural network.
[0007] In some embodiments, the artificial neural network comprises a recurrent network. [0008] In some embodiments, the recurrent network is selected from the group of a long short-term memory and a gated recurrent unit.
[0009] In some embodiments, the artificial neural network comprises an attention component.
[0010] In some embodiments, the attention components comprise a transformer network. [0011] In some embodiments, generating food text data comprises capturing text data with the user device.
[0012] In some embodiments, capturing the text data includes capturing non-text food data with the user device and converting the non-text food data to the generated food text data. [0013] In some embodiments, capturing text data is selected from the group of: menu text data, recipe text data and food log data.
[0014] In some embodiments, the quantitative feature comprises a diet compliance value. [0015] In some embodiments, the quantitative feature comprises nutrient information.
[0016] In some embodiments, the method further includes by operation of a trained expert machine learning system, generating the training feature data corresponding to the training food text data.
[0017] In an aspect, a system is disclosed, comprising: a user device configured to generate food text data and provide predicted quantitative food feature data for the generated food text data; at least one machine learning (ML) system in communication with the user device comprising at least one artificial neural network trained with the training data comprising training food text data and corresponding training quantitative food feature data, the ML system configured to receive the generated food text data, generate the predicted quantitative food feature data therefrom, and communicate the quantitative food feature data to the user device.
[0018] In some embodiments, the ML system comprises an artificial neural network.
[0019] In some embodiments, the artificial neural network comprises a recurrent network.
[0020] In some embodiments, the recurrent network is selected from the group of a long short-term memory and a gated recurrent unit.
[0021] In some embodiments, the artificial neural network comprises an attention component.
[0022] In some embodiments, the attention component comprises a transformer network.
[0023] In some embodiments, the user device is configured to capture the food text data from a source external to the user device.
[0024] In some embodiments, the user device is configured to capture non-text food data with the user device, and convert the non-text food data to the generated food text data. [0025] In some embodiments, the quantitative feature comprises a diet compliance value. [0026] In some embodiments, the quantitative feature comprises nutrient information.
[0027] In some embodiments, the system further includes at least one expert system configured to generate the training feature data from the training food text data.
[0028] In some embodiments, the at least one expert system comprises an expert ML system.
INCORPORATION BY REFERENCE
[0029] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent publications and patents or patent applications incorporated by reference contradict the disclosure contained in the specification, the specification is intended to supersede and/or take precedence over any such contradictory material.
BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings (also “Figure” and “FIG.” herein), of which:
[0031] FIGS. 1A and IB are diagrams showing method and systems for predicting quantitative food information from food text according to embodiments.
[0032] FIGS. 2A and 2B are diagrams showing method and systems for predicting nutrition and diet information from food text according to embodiments.
[0033] FIG. 3 is a diagram of a system according to an embodiment.
[0034] FIGS. 4A to 4C are diagrams of a user device and operations according to embodiments.
[0035] FIG. 5 shows a computer system that is programmed or otherwise configured to implement methods provided herein. DETAILED DESCRIPTION
[0036] While various embodiments of the invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions may occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed.
[0037] Embodiments describe a method for creating a system that takes as input a text description of food and outputs quantitative measures of that food.
[0038] In one embodiment, an example dataset is collected containing text descriptions of food items and corresponding quantitative measures of the food items described. For example, the text descriptions could be from food items listed on menus. For each described food item, a nutritionist or other specialist may estimate nutrients and dietary consistency. [0039] Weights of an artificial neural network could then be adjusted, for example through the use of a backpropagation algorithm, such that the artificial neural network predicts quantitative measures for the text description that closely agree with the example quantitative measures from the dataset according to an appropriate error measure.
[0040] FIG. 1A is a diagram showing methods and systems 100 according to embodiments. A system 100 can include sets of food text data 102, an expert 104 and a machine learning (ML) system 105.
[0041] In some embodiments, sets of food text data 102 can be acquired. Such sets 102 can be acquired in any suitable fashion, including but not limited to: food menu descriptions, labels, advertisements, or generated with other systems from food images.
[0042] An expert 104 can generate one or more features 106 corresponding to each food text data set. Examples of experts 104 can include, but are not limited to: one or more individuals having expertise in identifying food features, a machine system (including an artificial intelligence system), or a food manufacturer/producer. Features 106 can include any suitable feature corresponding to a food, including but not limited to: nutrient data, dietary consistency (e.g., how much the food may comply with a particular diet), availability (e.g., geographical/geolocation data), cost or demographic match (e.g., does a particular demographic population prefer this type of food). Features 106 can include one feature, the probability of one feature (e.g., of the one feature), multiple features and/or the probabilities of multiple features (e.g., of the multiple features).
[0043] Food text and corresponding features 102/106 can serve as training data for ML system 105. Food text 102 can serve as input values while feature data 106 can serve to evaluate a learning system predicted value. An ML system 105 can take any suitable form. In some embodiments, an ML system 105 can include an artificial neural network. Non-limiting examples of neural networks may comprisea recurrent network and/or a transformer network. In the embodiment shown, a ML system 105 can include function/model 108, error function 110, and parameter adjust operation 112. In response to one or more text data sets 102, a function/model 108 can predict one or more features 107 based on parameters. In some embodiments, processing of text data sets 102 by ML system 105 can include encoding positions of words with respect to one another in the text data. An error function 110 can determine an error 109 between features 107 predicted by function/model 108 and the expert generated features 106 corresponding to the input food text data set. Parameter adjust operation 112 can adjust parameters of function/model 108 based on error values 109 from error function 110 to attempt to converge on predictions with less error.
[0044] FIG. IB shows a prediction system and operations 100B according to embodiments. A system 100B can include a trained function/model 108B. A trained function/model 108B can be a function/model 108 after being trained with multiple food textfeature data sets 102/106, as shown in FIG. 1A.
[0045] A trained function/model 108B can receive new food text data 102B, and in response thereto, generate one or more predicted features 106B.
[0046] FIG. 2A is a diagram showing methods and systems 200 according to additional embodiments. FIG. 2A can include features like those of FIG. 1A, including sets of food text data 202, an expert 204, and a ML system 205.
[0047] In the embodiment of FIG. 2A, an expert 204 can result in features 206 for text data sets 202 that can include nutrition information (e.g., Nutrition Facts) and/or dietary consistency. In the example shown, dietary consistency with particular diets is shown as “Diet x”, where variations in x indicate a different diet. Dietary consistency can include, but is not limited to including, whether the food text indicates compliance with a diet, or a probability of compliance with a diet of the food item corresponding to the text. Nutrition can include any suitable nutrition data, including but not limited to: content (e.g., calories, fats, cholesterol, sodium, carbohydrates, fiber, sugars, proteins, minerals, vitamins), rating (e.g., blood glucose impact, blood pressure impact, or other health related scoring).
[0048] FIG. 2B shows methods and systems 200B according to another embodiment. A system 200B can include a trained function/model 208B, such as function/model 208 trained as shown in FIG. 2A. In response to new food text data 202B, a trained function/model 208B can generate predicted nutrition and/or dietary consistency information 206B. [0049] FIG. 3 shows a system 300 according to another embodiment. A system 300 can include one or more ML servers 330, application servers 328, data store 326, a food data source 350, subject devices 332 and one or more experts 304. Food data source 350, servers (330/328), subject devices 332 and expert 304 can be in communication with one another, such as through a network 334, which can include various interconnected networks, including the internet.
[0050] ML server 330 can include systems and execute methods like those described herein and equivalents.
[0051] An application server 328 can interact with one or more applications 338 running on a subject device 332. In some embodiments, data from food data sources 350 can be acquired via one or more applications 338 on a subject device (e.g., smart phone) 332 and provided to application server 328. Application server 328 can communicate with subject device 332 according to any suitable secure network protocol.
[0052] An expert 304 can be an expert as described herein and equivalents. An expert 304 can receive food text descriptions (e.g., food text data sets). In some embodiments, food text descriptions can include food text data sets 302B from a data store 326 or from sources external to system 300 302A. From food text descriptions 302A/B, an expert 304 can generate feature data 306 for each food text data set, as described herein and equivalents. Such features can include nutrition information and/or dietary consistency data. This can create training data sets 302/306. Training data sets 302/306 can be used to train function/models 308 within ML servers 330.
[0053] A data store 328 can store data for a system 300. In some embodiments, data store 328 can store data received from food data source 330, food text data sets 302B, as well as generated training sets 302/306. A data store 328 can take any suitable form, including one or more network attached storage systems. In some embodiments, all or a portion of data store can be integrated with any of the servers (330, 328).
[0054] In some embodiments, food text data 302 can be generated by a user. In some embodiments, text data 352-0 can be entered by a user (e.g.., typed) or captured by a user (e.g., voice recognition, image capture). In addition or alternatively, image data can be images of actual food (e.g., 352-1), or food packaging, and such image data can be applied to an application on a server to generate a food text set for the image.
[0055] A subject device 332 can be any suitable device, including but not limited to, a smart phone, personal computer, wearable device, or tablet computing device. A subject device 332 can include one or more applications 338 that can communicate with application server 328 to provide data to, and receive data from, models/functions 308 residing on ML servers 330. Such applications 328 can take the form of, and/or include the functions of the various embodiments disclosed herein.
[0056] FIG. 4A to 4C are diagrams of an application 438 according to embodiments. Referring to FIG. 4A, an application 438 can reside on a subject device 432 as instructions executable by a processor or the like, in an embodiment.
[0057] Referring to FIG. 4B, once initiated, an application 438 can request/receive food text data 402B. Such food text data 402B can be acquired in any suitable fashion including as disclosed herein and equivalents. Received food text data 402B can be subject to a trained model/function to generate feature data according to any of the embodiments described herein and equivalents. Such generation of feature data can be performed by the device 432 (e.g., model/function resides on the device) or can be performed by a remote ML server 442, or a combination thereof.
[0058] Referring FIG. 4C, predicted feature data 406A/B from a trained model/function can be presented on subject device 432. In the embodiment shown, predicted feature data can include nutrition information 406A and/or dietary consistency 406B. However, alternate embodiments can present any other suitable features generated by training with food text data sets.
[0059] The present disclosure can provide methods for performing at least the following: predicting quantitative measures of a food item from text about the food item using an artificial neural network; where the artificial neural network is trained using examples of text and quantitative measures; where the artificial neural network contains a recurrent network such as an LSTM (long short-term memory), a GRU (gated recurrent unit) and/or an attention component, such as a transformer network. Quantitative measures can be nutrients. Such nutrients can be carbohydrates, fats, dietary fiber, minerals, proteins, vitamins, or water. Quantitative measures can be whether the food adheres to a diet. Such diets can include, but are not limited to: the paleo diet, the vegan diet, the low-carb diet, the vegetarian diet, the Atkins diet, the HCG (human chorionic gonadotropin) diet, the Zone diet. Food item text can be menu item text. Food item text and be recipe text. Food item text can be free form user text food logs
[0060] It is understood that various blocks shown in the figures described herein can include any of various circuits configured to execute the indicated functions, including but not limited to server systems that may or may not include customized hardware for accelerating operations, logic circuits, including custom logic circuits or programmable logic circuits. Such functions can also correspond to all or a portion of code executable by one or more processors that is stored on machine readable media. Data values as described herein can also be stored in machine readable media. Machine readable media can store code and/or data in a non-transitory form, in volatile and/or nonvolatile storage circuits.
[0061] It should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
[0062] It is also understood that the embodiments of the invention may be practiced in the absence of an element and/or step not specifically disclosed. That is, an inventive feature of the invention may be elimination of an element.
[0063] According to embodiments, blocks or actions that do not depend upon each other can be arranged or executed in parallel.
[0064] Accordingly, while the various aspects of the particular embodiments set forth herein have been described in detail, the present invention could be subject to various changes, substitutions, and alterations without departing from the spirit and scope of the invention.
Computer systems
[0065] The present disclosure provides computer systems that are programmed to implement methods of the disclosure. FIG. 5 shows a computer system 501 that is programmed or otherwise configured to predict food quantities by analyzing text data. The computer system 501 can regulate various aspects of analyzing text of the present disclosure, such as, for example, implementing machine learning algorithms. The computer system 501 can be an electronic device of a user or a computer system that is remotely located with respect to the electronic device. The electronic device can be a mobile electronic device or a desktop computing device.
[0066] The computer system 501 includes a central processing unit (CPU, also “processor” and “computer processor” herein) 505, which can be a single core or multi core processor, or a plurality of processors for parallel processing. The computer system 501 also includes memory or memory location 510 (e.g., random-access memory, read-only memory, flash memory), electronic storage unit 515 (e.g., hard disk), communication interface 520 (e.g., network adapter) for communicating with one or more other systems, and peripheral devices 525, such as cache, other memory, data storage and/or electronic display adapters. The memory 510, storage unit 515, interface 520 and peripheral devices 525 are in communication with the CPU 505 through a communication bus (solid lines), such as a motherboard. The storage unit 515 can be a data storage unit (or data repository) for storing data. The computer system 501 can be operatively coupled to a computer network (“network”) 530 with the aid of the communication interface 520. The network 530 can be the Internet, an internet and/or extranet, or an intranet and/or extranet that is in communication with the Internet. The network 530 in some cases is a telecommunication and/or data network. The network 530 can include one or more computer servers, which can enable distributed computing, such as cloud computing. The network 530, in some cases with the aid of the computer system 501, can implement a peer-to-peer network, which may enable devices coupled to the computer system 501 to behave as a client or a server.
[0067] The CPU 505 can execute a sequence of machine-readable instructions, which can be embodied in a program or software. The instructions may be stored in a memory location, such as the memory 510. The instructions can be directed to the CPU 505, which can subsequently program or otherwise configure the CPU 505 to implement methods of the present disclosure. Examples of operations performed by the CPU 505 can include fetch, decode, execute, and writeback.
[0068] The CPU 505 can be part of a circuit, such as an integrated circuit. One or more other components of the system 501 can be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC).
[0069] The storage unit 515 can store files, such as drivers, libraries and saved programs. The storage unit 515 can store user data, e.g., user preferences and user programs. The computer system 501 in some cases can include one or more additional data storage units that are external to the computer system 501, such as located on a remote server that is in communication with the computer system 501 through an intranet or the Internet.
[0070] The computer system 501 can communicate with one or more remote computer systems through the network 530. For instance, the computer system 501 can communicate with a remote computer system of a user (e.g., a mobile device). Examples of remote computer systems include personal computers (e.g., portable PC), slate or tablet PC’s (e.g., Apple® iPad, Samsung® Galaxy Tab), telephones, Smart phones (e.g., Apple® iPhone, Android-enabled device, Blackberry®), or personal digital assistants. The user can access the computer system 501 via the network 530.
[0071] Methods as described herein can be implemented by way of machine (e.g., computer processor) executable code stored on an electronic storage location of the computer system 501, such as, for example, on the memory 510 or electronic storage unit 515. The machine executable or machine readable code can be provided in the form of software. During use, the code can be executed by the processor 505. In some cases, the code can be retrieved from the storage unit 515 and stored on the memory 510 for ready access by the processor 505. In some situations, the electronic storage unit 515 can be precluded, and machine-executable instructions are stored on memory 510.
[0072] The code can be pre-compiled and configured for use with a machine having a processer adapted to execute the code, or can be compiled during runtime. The code can be supplied in a programming language that can be selected to enable the code to execute in a pre-compiled or as-compiled fashion.
[0073] Aspects of the systems and methods provided herein, such as the computer system 501, can be embodied in programming. Various aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of machine (or processor) executable code and/or associated data that is carried on or embodied in a type of machine readable medium. Machine-executable code can be stored on an electronic storage unit, such as memory (e.g., read-only memory, random-access memory, flash memory) or a hard disk. “Storage” type media can include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server. Thus, another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible “storage” media, terms such as computer or machine “readable medium” refer to any medium that participates in providing instructions to a processor for execution.
[0074] Hence, a machine readable medium, such as computer-executable code, may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings. Volatile storage media include dynamic memory, such as main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system. Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code and/or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.
[0075] The computer system 501 can include or be in communication with an electronic display 535 that comprises a user interface (UI) 540 for providing, for example, a dashboard. Examples of UFs include, without limitation, a graphical user interface (GUI) and web-based user interface.
[0076] Methods and systems of the present disclosure can be implemented by way of one or more algorithms. An algorithm can be implemented by way of software upon execution by the central processing unit 505. The algorithm can, for example, analyze food text data. [0077] Whenever the term “at least,” “greater than,” or “greater than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “at least,” “greater than” or “greater than or equal to” applies to each of the numerical values in that series of numerical values. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3. [0078] Whenever the term “no more than,” “less than,” or “less than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “no more than,” “less than,” or “less than or equal to” applies to each of the numerical values in that series of numerical values. For example, less than or equal to 3, 2, or 1 is equivalent to less than or equal to 3, less than or equal to 2, or less than or equal to 1.
[0079] While preferred embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. It is not intended that the invention be limited by the specific examples provided within the specification. While the invention has been described with reference to the aforementioned specification, the descriptions and illustrations of the embodiments herein are not meant to be construed in a limiting sense. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the invention. Furthermore, it shall be understood that all aspects of the invention are not limited to the specific depictions, configurations or relative proportions set forth herein which depend upon a variety of conditions and variables. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention. It is therefore contemplated that the invention shall also cover any such alternatives, modifications, variations or equivalents. It is intended that the following claims define the scope of the invention and that methods and structures within the scope of these claims and their equivalents be covered thereby.

Claims

CLAIMS What is claimed is:
1. A method, comprising: by operation of a user device, generate food text data; apply the generated food text data to a feature prediction machine learning (ML) system to predict at least one quantitative food feature, the feature prediction ML system comprising at least one artificial neural network trained with the training data comprising training food text data and corresponding training feature data; and provide the at least one quantitative food feature on the user device.
2. The method of claim 1, wherein the ML system comprises an artificial neural network.
3. The method of claim 2, wherein the artificial neural network comprises a recurrent network.
4. The method of claim 3, wherein the recurrent network is selected from the group of a long short-term memory and a gated recurrent unit.
5. The method of claim 1, wherein the artificial neural network comprises an attention component.
6. The method of claim 5, wherein the attention components comprises a transformer network.
7. The method of claim 1, wherein generating food text data comprises capturing text data with the user device.
8. The method of claim 7, wherein capturing the text data includes capturing non-text food data with the user device and converting the non-text food data to the generated food text data.
9. The method of claim 7, wherein capturing text data is selected from the group of: menu text data, recipe text data and food log data.
10. The method of claim 1, wherein the quantitative feature comprises a diet compliance value.
11. The method of claim 1, wherein the quantitative feature comprises nutrient information.
12. The method of claim 1, further including by operation of a trained expert machine learning system, generating the training feature data corresponding to the training food text data.
13. A system, comprising: a user device configured to generate food text data and provide predicted quantitative food feature data for the generated food text data; at least one machine learning (ML) system in communication with the user device comprising at least one artificial neural network trained with the training data comprising training food text data and corresponding training quantitative food feature data, the ML system configured to receive the generated food text data, generate the predicted quantitative food feature data therefrom, and communicate the quantitative food feature data to the user device.
14. The system of claim 13, wherein the ML system comprises an artificial neural network.
15. The system of claim 14, wherein the artificial neural network comprises a recurrent network.
16. The system of claim 15, wherein the recurrent network is selected from the group of a long short-term memory and a gated recurrent unit.
17. The system of claim 14, wherein the artificial neural network comprises an attention component.
18. The system of claim 17, wherein the attention component comprises a transformer network.
19. The system of claim 13, wherein the user device is configured to capture the food text data from a source external to the user device.
20. The system of claim 13, wherein: the user device is configured to capture non-text food data with the user device, and convert the non-text food data to the generated food text data.
21. The system of claim 13, wherein the quantitative feature comprises a diet compliance value.
22. The system of claim 13, wherein the quantitative feature comprises nutrient information.
23. The system of claim 13, further including at least one expert system configured to generate the training feature data from the training food text data.
24. The system of claim 23, wherein the at least one expert system comprises an expert ML system.
PCT/US2023/023683 2022-05-31 2023-05-26 Methods and systems for predicting quantitative measures of food items from text WO2023235247A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202263347307P 2022-05-31 2022-05-31
US63/347,307 2022-05-31

Publications (1)

Publication Number Publication Date
WO2023235247A1 true WO2023235247A1 (en) 2023-12-07

Family

ID=89025472

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2023/023683 WO2023235247A1 (en) 2022-05-31 2023-05-26 Methods and systems for predicting quantitative measures of food items from text

Country Status (1)

Country Link
WO (1) WO2023235247A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190290172A1 (en) * 2018-03-23 2019-09-26 Medtronic Minimed, Inc. Systems and methods for food analysis, personalized recommendations, and health management
US20210256872A1 (en) * 2020-02-17 2021-08-19 GlucoGear Tecnologia LTDA Devices, systems, and methods for predicting blood glucose levels based on a personalized blood glucose regulation model

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190290172A1 (en) * 2018-03-23 2019-09-26 Medtronic Minimed, Inc. Systems and methods for food analysis, personalized recommendations, and health management
US20210256872A1 (en) * 2020-02-17 2021-08-19 GlucoGear Tecnologia LTDA Devices, systems, and methods for predicting blood glucose levels based on a personalized blood glucose regulation model

Similar Documents

Publication Publication Date Title
CN110288049B (en) Method and apparatus for generating image recognition model
US20200302271A1 (en) Quantization-aware neural architecture search
CN112699991A (en) Method, electronic device, and computer-readable medium for accelerating information processing for neural network training
CN111523640B (en) Training method and device for neural network model
US11604960B2 (en) Differential bit width neural architecture search
US11574239B2 (en) Outlier quantization for training and inference
US11551171B2 (en) Utilizing natural language processing and machine learning to automatically generate proposed workflows
US20220261631A1 (en) Pipelines for efficient training and deployment of machine learning models
US20190095793A1 (en) Sensor quality upgrade framework
CN113611405A (en) Physical examination item recommendation method, device, equipment and medium
CN111290785B (en) Method, device, electronic equipment and storage medium for evaluating compatibility of deep learning framework system
CN115699041A (en) Extensible transfer learning using expert models
US10755171B1 (en) Hiding and detecting information using neural networks
CN109840072B (en) Information processing method and device
US10733540B2 (en) Artificial intelligence and knowledge based automation enhancement
US20220359079A1 (en) Systems, methods and devices for predicting personalized biological state with model produced with meta-learning
WO2023235247A1 (en) Methods and systems for predicting quantitative measures of food items from text
US11501081B1 (en) Methods, mediums, and systems for providing a model for an end-user device
CN110633476B (en) Method and device for acquiring knowledge annotation information
CN111259659B (en) Information processing method and device
CN112446190A (en) Method and device for generating style conversion text
JP7178323B2 (en) LEARNING DEVICE, LEARNING METHOD AND LEARNING PROGRAM
Hasanaj et al. Cooperative edge deepfake detection
US20240119927A1 (en) Speaker identification, verification, and diarization using neural networks for conversational ai systems and applications
US20240153531A1 (en) Multi-scale speaker diarization for conversational ai systems and applications

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23816609

Country of ref document: EP

Kind code of ref document: A1