WO2023062880A1 - 領域管理機器およびプログラム - Google Patents

領域管理機器およびプログラム Download PDF

Info

Publication number
WO2023062880A1
WO2023062880A1 PCT/JP2022/024657 JP2022024657W WO2023062880A1 WO 2023062880 A1 WO2023062880 A1 WO 2023062880A1 JP 2022024657 W JP2022024657 W JP 2022024657W WO 2023062880 A1 WO2023062880 A1 WO 2023062880A1
Authority
WO
WIPO (PCT)
Prior art keywords
health index
area
nutrients
unit
food
Prior art date
Application number
PCT/JP2022/024657
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
大介 林
圭介 稲田
泰久 森
功記 加藤
Original Assignee
日立グローバルライフソリューションズ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日立グローバルライフソリューションズ株式会社 filed Critical 日立グローバルライフソリューションズ株式会社
Priority to CN202280049854.1A priority Critical patent/CN117716436A/zh
Publication of WO2023062880A1 publication Critical patent/WO2023062880A1/ja

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment

Definitions

  • the present invention relates to an area management device that performs information processing regarding accommodation or storage (hereinafter simply storage) of articles including foodstuffs. Among these, it particularly relates to a technology for providing information according to usage conditions including internal inventory.
  • the area management equipment includes not only the warehouse for storage, but also a computer, a mobile terminal, a web server, and an internal management system (area management system) including at least one of these for managing it.
  • the storehouse may be any storehouse that can store articles, and includes refrigerators, pantries, food storage such as so-called floor storage, food storage rooms, and article shelves.
  • Patent Document 1 describes that "the total nutritional value of a menu used during an arbitrary period is calculated, and a menu that allows intake of the nutritional value that is lacking in terms of nutritional balance is presented.”
  • Patent Document 2 there is a technology that considers the nutrients in the current inventory and proposes the purchase of low-cost ingredients that supplement the lack of nutrients.
  • Patent Document 2 "presentation of foodstuffs for priority consumption based on a comprehensive cost performance evaluation of ingredients in stock and presentation of recommended foodstuffs for purchase that optimizes cost performance, which is the balance between the price and the supply of insufficient nutrients for consumers, A refrigerator that can be realized at the same time.”
  • the present invention uses a management barometer that indicates the user's nutritional status, taking into consideration the nutrients that the user is recommended to take in the future. More preferably, the health index corresponding to the nutrient is calculated based on the nutrient that is recommended to be ingested according to the usage status of the storage. In addition, proposal information on recipes for ingesting nutrients is created according to health management barometers including health indexes or nutritional parameters.
  • the present invention includes the following configurations (1) and (2).
  • a recognition unit that recognizes the usage status of the management area related to the storage, and based on the usage status, recommends nutrients to be taken by a predetermined user.
  • An area management device having a health index calculation unit that identifies and calculates a health index corresponding to the nutrient, and an output unit that outputs the health index.
  • an area management device that performs information processing on food storage, a recognition unit that recognizes the usage status of the management area related to storage, and a predetermined user based on the usage status of the management area related to storage a health management barometer calculation unit that identifies recommended nutrients and calculates a health management barometer that indicates the nutritional status of the user; a health index calculation unit; and intake of the specified nutrients according to the health management barometer.
  • An area management device having a proposal information generation unit for generating proposal information about a recipe for cooking, and an output unit for outputting the proposal information.
  • the present invention includes a health management method and a computer program using the area management equipment of (1) and (2) above.
  • the area management devices of (1) and (2) include computers such as storages and mobile terminals.
  • systems and subsystems including the repositories and computers described above are also included in the area management equipment of the present invention.
  • FIG. 1 is an overall configuration diagram of an in-chamber management system in an embodiment
  • FIG. 4 is a flow chart showing an internal storage management process in Embodiment 1.
  • FIG. 10 is a diagram showing an example of a fisheye camera image CG captured by a camera
  • FIG. 4 is a diagram showing an example of an image obtained by converting a fisheye camera image CG into a planar image
  • It is a figure which shows an example of the foodstuff nutritional component table used by an Example.
  • It is a figure which shows the example of a display of proposal information.
  • It is a figure which shows an example of the order screen displayed on a portable terminal.
  • It is a flowchart which shows the in-fridge management process in an Example.
  • FIG. 10 is a diagram showing an example of a fisheye camera image CG captured by a camera
  • FIG. 4 is a diagram showing an example of an image obtained by converting a fisheye camera image CG into a planar image
  • It is a figure which
  • FIG. 10 is a diagram showing an example of a healthy recipe displayed in Example 2; It is a figure which shows the example which provided the camera in the chamber
  • FIG. 3 is a diagram showing a configuration of a mobile terminal 7 that performs in-fridge management processing;
  • a repository is used as the area management device.
  • the usage status of the storage is recognized, and based on this, nutritional parameters and health indices are calculated as a management barometer indicating the nutritional status of the user according to the nutrients that the user should take.
  • a management barometer indicating the nutritional status of the user according to the nutrients that the user should take.
  • information on recipes for ingesting the nutrients that the user should ingest is proposed.
  • the information about this recipe includes recipes that are cooking contents, additional ingredients that are recommended to be added, and recipe cooking methods.
  • a refrigerator will be used as an example of storage.
  • the storehouse of this embodiment is not limited to the refrigerator, and includes an article shelf and the like.
  • the storage area is not limited to specific cabinets, shelves, furniture, or home electric appliances, and can be an area for storing foodstuffs consumed by the user (hereinafter sometimes referred to as a management area). For example, it can be the user's entire home.
  • the inside of the storage is described as an example of the management area.
  • a food material that is stored in a management area and becomes a management target is sometimes referred to as a controlled food material.
  • the camera will be used as an example of the shooting unit.
  • the present embodiment is not limited to cameras, and can be widely applied to sensor information such as weight sensors and mycotoxin detection, IC tag information, character recognition of package information, and the like.
  • the recognition unit manages the management area, for example, recognizes the usage status of the foodstuffs in the storehouse and the foodstuffs consumed by the user.
  • the recognition unit performs recognition using the results of the imaging by the imaging unit, but may be configured to perform recognition according to input from the user to the mobile terminal 7 such as a smartphone or the storage itself.
  • foods will be described as an example of articles.
  • foodstuffs may be materials, and may be cooked foodstuffs or seasonings.
  • the storage management system area management system in each of the following embodiments, each device described later, and combinations thereof are included in the area management equipment of the present invention.
  • FIG. 1 is an overall configuration diagram of an internal warehouse management system (area management system) in Example 1 and Example 2 described later.
  • the refrigerator 1 is connected to a portable terminal 7, a web server 8, and a computer 9 via a network CN to configure the refrigerator management system.
  • a network CN to configure the refrigerator management system.
  • the refrigerator 1 includes a control unit 10 and a refrigerator main body 20 as a "storage main body".
  • the portable terminal 7 as an external device is a terminal used by the user (user) of the refrigerator 1 .
  • the portable terminal 7 displays the food order list transmitted from the refrigerator 1, as will be described later.
  • the mobile terminal 7 can be realized by an information processing device such as a tablet, a smart phone, or a PC.
  • the main processing in Examples 1 and 2 is executed by the refrigerator 1 (control unit 10), it may be executed by the portable terminal 7, or may be executed by the refrigerator 1 and the portable terminal 7. . This will be described later with reference to FIG.
  • the web server 8 includes, for example, a net supermarket and a recipe site. By communicating with these devices, the refrigerator 1 can purchase foodstuffs, obtain cooking methods, recipes, and the like.
  • a computer 9 is a computer for distributing machine learning models and the like for providing information in the first and second embodiments.
  • FIG. 10 is a diagram showing an example in which cameras 50a to 50e are provided inside the refrigerator body 20.
  • the camera 50a and the camera 50b are examples installed above the door.
  • the camera 50c is an example installed in the upper part inside the main body.
  • each camera can be installed at a position other than the top inside the refrigerator body 20 .
  • An example of this is the position of the cameras 50d and 50e.
  • a camera 50d is provided at the center of the door, and a camera 50e is provided at the bottom of the refrigerator. Note that the installation positions and the number of cameras are not limited to the example in FIG. In particular, the number of installations may be one or more.
  • the control unit 10 that controls the refrigerator 1 includes, for example, a processor 11, a storage device 12, a communication unit (I/F in the figure) 14 connected to the network CN, and an I/O interface 13 (I/F in the figure). /O).
  • the storage device 12 includes a main storage device composed of volatile or nonvolatile memory, and an auxiliary storage device composed of flash memory, hard disk drive, or the like.
  • a part or all of the computer programs and data stored in the storage device 12 can also be transmitted to the outside via the communication network CN.
  • the computer program and data can also be transmitted from the external computer 9 or the like to the storage device 12 via the communication network CN and stored.
  • a storage medium MM such as a flash memory or hard disk drive can be connected to the control unit 10, and part or all of the computer program and data can be transferred between the storage device 12 and the storage medium MM.
  • the storage device 12 includes an imaging unit 121, an image conversion unit 123, a recognition unit 124, a table control unit 125, a health index calculation unit 126, a proposal information generation unit 127, a display unit 128, and an ordering unit 129. , and a predetermined computer program that implements the in-fridge control unit 130 is stored.
  • the storage device 12 also includes an image buffer 122 .
  • the image buffer 122 may have an independent structure.
  • each functional unit of the storage device 12 in FIG. 1 corresponds to the program. Therefore, each of these units can be read as each computer program, and the processor 11 realizes the processing and function of each unit described later by each computer program.
  • each of these units may be realized by dedicated hardware, FPGA (Field Programmable Gate Array), or the like.
  • these computer programs may be configured in one or less than the number shown. In this case, each functional unit can be configured as a computer module (simply called a module).
  • the processor 11 operates as a functional unit that provides a predetermined function by executing processing according to a computer program.
  • the processor 11 functions as the image conversion unit 123 by executing processing according to an image conversion program.
  • the processor 11 also operates as a functional unit that provides each function of a plurality of processes executed by each computer program.
  • the computer program is executed by the processor 11 and one processor, but may be executed by a plurality of processors.
  • the imaging unit 121 acquires camera images from the camera 50 via the I/O interface 13 and stores the acquired camera images in the image buffer 122 .
  • the camera 50 as a "camera section" is configured as a fisheye camera. Camera 50 has, for example, a fisheye or wide-angle lens.
  • the image conversion unit 123 converts a camera image taken with a fisheye or wide-angle lens into a planar image. Since a well-known technique can be used when an image captured by a fisheye or wide-angle lens is developed into a planar image, the description thereof is omitted.
  • the recognition unit 124 is composed of a learning-based image recognition unit and a rule-based image recognition unit.
  • the learning-based image recognition unit and the rule-based image recognition unit each recognize ingredients in the refrigerator from the converted planar image.
  • the learning-based image recognition unit includes, for example, a pre-learned machine learning model such as deep learning.
  • the learning-based image recognition unit outputs a recognition result of ingredients included in the plane image.
  • the rule-based image recognition unit recognizes ingredients in the refrigerator based on the rule.
  • the rule-based image recognition unit divides the input planar image into regions, labels the article for each region, and outputs the contents of the recognized label as text.
  • Known techniques can be applied to these learning-based image recognition unit and rule-based image recognition unit.
  • the recognition unit 124 of this embodiment recognizes the usage status of the refrigerator 1 using an image, it is not limited to this as described above.
  • the imaging unit 121, the image buffer 122 and the image conversion unit 123 can be omitted.
  • a configuration for use in recognition by the recognition unit 124 is provided.
  • a functional unit is provided to perform processing for associating weight with food.
  • the table control unit 125 controls the contents of the ingredient nutritional component table T1 (see FIG. 5) that defines the nutritional components for each ingredient. That is, the table control unit 125 accesses the ingredient nutritional component table T1, and uses this result in other functional units.
  • the food nutrient component table T1 is provided inside the table control unit 125 .
  • the nutritional components of the ingredient nutritional component table T1 are nutritional standards (standard values) for calculating the health index by the health index calculator 126.
  • FIG. As an example of these nutritional components, the values in the "Japanese Food Standard Ingredients Table 2020 Edition" by the Ministry of Education, Culture, Sports, Science and Technology are taken as an example.
  • the ingredient nutritional component table T1 is not limited to this example, and a ingredient manufacturer or the like may independently create nutritional component values for each ingredient.
  • the health index calculation unit 126 calculates the health index from the in-fridge health index (management area health index) and the consumption health index. This calculation includes preparation. This preparation consists of the following two stages. In the first step, the health index calculation unit 126 refers to the recognized nutrients of the ingredients, that is, the values of the nutritional components from the table control unit 125 . Take proteins, fats, and carbohydrates as examples of these nutrients. However, Examples 1 and 2 are also broadly applicable to nutrients such as vitamins and minerals.
  • the health index calculation unit 126 sums up the recognized food ingredients within the categories of protein, fat, and carbohydrates, and calculates the ratio of total protein, fat, and carbohydrates.
  • this ratio for example, it is possible to refer to the ratio to the total energy amount when protein, lipid, and carbohydrate of the foodstuff of interest are each converted into energy amount.
  • the health index calculator 126 compares the total ratio of protein, fat, and carbohydrate with a reference value to calculate each in-fridge nutrition parameter.
  • a reference value the value of “Dietary Reference Intakes for Japanese People” (2020 edition) by the Ministry of Health, Labor and Welfare (Equation 1-1) is used.
  • p* is the reference intake of protein
  • f* is the reference intake of fat
  • c* the reference intake of carbohydrate.
  • the standard value is not limited to the value of the Ministry of Health, Labor and Welfare, and the standard value may be set independently by a food material manufacturer or the like.
  • the user's race, sex, age, and other information may be input and set in consideration of such information.
  • setting of the reference value of each nutrient may be received from the user.
  • the nutritional parameters are information indicating the nutrients of the ingredients to be managed, such as the ingredients in the refrigerator. Therefore, the higher the value, the more the user can be suggested to purchase a large amount of foodstuff containing the nutrient.
  • the nutrition parameters consist of in-chamber nutrition parameters and consumed nutrition parameters.
  • a nutrient parameter of a certain nutrient indicates whether two or more kinds of nutrients as a whole is a ratio of a certain nutrient to the whole. In this example, as described above, three types of nutrients, proteins, lipids, and carbohydrates, will be described as a whole.
  • (Formula 2-1) np ip x t2 -p* ...
  • np ip the protein nutrient parameter
  • np if the lipid nutrient parameter
  • np ic the carbohydrate nutrient parameter
  • x t2 is the currently recognized protein ratio of the inside food material
  • y t2 is the currently recognized fat ratio of the inside food material
  • z t2 is the currently recognized carbohydrate ratio of the inside food material.
  • the absolute value of the in-fridge nutrition parameter for a certain nutrient suggests how far the nutrient ratio of the foodstuff stored in the management area deviates from the reference ratio.
  • the absolute value of the in-fridge nutrition parameter of a certain nutrient is the value of the user's certain nutrient from the present until the completion of the intake of these ingredients, assuming that the user has taken all the ingredients in the management area. It can be interpreted that it suggests how much the intake balance of
  • the in-fridge health index is obtained by subtracting the sum of the absolute values of the in-fridge nutrition parameters for each nutrient from the arbitrarily determined maximum value (100 in this embodiment). Therefore, the in-fridge health index is the nutrient balance of the ingredients stored in the area to be managed, that is, the nutritional value of the nutrients that the user would take if he lived with the currently managed ingredients. suggest a degree of balance.
  • the consumption health index is calculated by subtracting from 100 the absolute value of each consumption nutritional parameter of protein, fat, and carbohydrate. (Formula 3-1) to (Formula 3-5) are used for this calculation. Note that the consumption health index is a value of 0 or more and 100 or less as represented by (Equation 3-6). However, the formulas are not limited to (Formula 3-1) to (Formula 3-6), and other formulas may be used.
  • controlled food materials that have been consumed from an arbitrary timing in the past to the present are specified, and the total energy amount of each nutrient (protein, lipid, and carbohydrate in this embodiment) contained in these consumed controlled food materials is calculated. . That is, the total energy amount of each nutrient that the users would have ingested from any timing in the past to the present is calculated. Then divide by total energy for each nutrient. As a result, it is possible to calculate the ratio of each nutrient that the users would have ingested from any timing in the past to the present.
  • these the protein, lipid and carbohydrate consumption nutritional parameters respectively and define them as np cp , np cf and np cc respectively.
  • the information on these consumed controlled foodstuffs can be recognized by a camera or the like provided in the storage. Details are described separately.
  • np cp (proportion of protein ingested by users from arbitrary timing in the past to the present)
  • np cf (proportion of protein ingested by users from arbitrary timing in the past to the present)
  • np cf (percentage of lipids ingested by users from arbitrary timing in the past to the present)
  • np cc (percentage of carbohydrates ingested by users from arbitrary timing in the past to the present)
  • np cp +np cf +np cc 100 (equation 3-5) 0 ⁇ consumed health index ⁇ 100 (equation 3-6)
  • the absolute value of the consumed nutritional parameter of a certain nutrient is the amount consumed by the person who consumes the food material in the management area, that is, the user or the person who lives with the user, from the past timing designated by the user to the present.
  • the consumption health index is obtained by subtracting the sum of the absolute values of the consumption nutritional parameters of each nutrient from the arbitrarily determined maximum value (100 in this embodiment). For this reason, the consumption health index suggests the degree of balance of nutrients that would have been consumed by a person consuming the food material in the management area from the past timing designated by the user to the present.
  • the health index is calculated as, for example, an arithmetic mean of the in-fridge health index and the consumption health index. (Formula 4) is used for this calculation.
  • Health index (inside health index + consumption health index)/2 (Equation 4)
  • the evaluation of this health index is A (80-100): Fairly good, B (60-79): Good, C (40-59): Normal, D (20-39): Needs improvement, E (0- 19): Considerable improvement is required. This is done in 5 steps, but it is not limited to this way and the manufacturer may set it independently.
  • the health index calculated using the indoor health index and the consumption health index is the nutritional balance expected to be ingested by the users from the past timing designated by the user to the completion of the current intake of the controlled ingredients. Suggest. In this way, it is possible to evaluate the nutritional balance by taking into account both the balance of nutrients ingested over a predetermined period of time in the past and the current nutritional balance of controlled foodstuffs.
  • the health index calculated using the indoor health index and the consumption health index is not limited to calculation by arithmetic mean, but can be calculated by a method that does not interfere, such as a geometric mean or some kind of averaging calculation.
  • the inside health index is a value calculated based on the nutrients of each ingredient recognized inside the refrigerator. According to the refrigerator health index, for example, it is possible to suggest additional ingredients to the user so as to maintain the health and nutrition balance in the refrigerator.
  • the standard value for fat is set to the upper limit and the standard value for carbohydrates to the lower limit
  • proposal information is created to purchase high-carbohydrate foodstuffs in order to maintain a healthy balance in the refrigerator. , suggest this.
  • the consumption health index is a value calculated from the nutrients of each ingredient that is determined to have been ingested by the user within a certain period of time from the present to the past. According to the consumption health index, for example, it is possible to suggest to the user foods that should be consumed or a preferable menu based on the consumption nutritional balance of time-series differences.
  • the user can freely select N days as the time series difference from the previous day, 2 or 3 days before, or 1 week before. Therefore, it is possible to propose a purchase in which the inventory in the warehouse is kept until N days specified by the user.
  • p*, f*, c* of nutrient intake protein was 3 above the upper limit of reference, fat was 7 above the upper limit of reference, and carbohydrate was 10 below the lower limit of reference. It can be seen that the carbohydrate intake ratio is low.
  • high-carbohydrate foodstuffs or menus can be proposed in order to adjust the consumption nutritional balance of time-series differences.
  • the health index is calculated based on inventory food information (inside health index) and food consumption history (consumption health index). For this reason, the health index is an index corresponding to nutrients, and more preferably, information that is easier to understand based on nutritional parameters.
  • the health management barometer based on the health index and nutritional parameters, that is, the health management barometer, we will propose the amount of ingredients to be additionally purchased in consideration of the amount of inventory in the refrigerator and the nutrients that should be taken.
  • the purpose of the health index is to make it easier to secure the necessary and sufficient amount of foodstuffs that should be ingested from a nutritional point of view in the future in the refrigerator.
  • the health care barometer indicates the user's nutritional status, and more preferably includes health index and nutritional parameters.
  • the health index calculation unit 126 can calculate nutrition parameters, which are a type of health management barometer similar to the health management index, and can be understood as a type of health management barometer calculation unit.
  • proposal information generating section 127 includes additional ingredients determining section 1271 and healthy recipe information generating section 1272 .
  • the additional foodstuff determination unit 1271 proposes the amount of foodstuffs to be additionally purchased in consideration of the stock amount in the refrigerator and the nutrients to be ingested based on the nutritional parameters. As a result, it becomes easier to secure the necessary and sufficient amount of food materials to be ingested in the future in terms of nutrition in the refrigerator.
  • An example of the proposed additionally purchased ingredients includes eggs and the like.
  • the healthy recipe information generation unit 1272 displays recipes that can be made from ingredients in the refrigerator, and proposes recipes that increase the health index.
  • the index used for additional purchase may be either the indoor health index or consumption health index, or may be used in combination. This makes it possible to propose a wide range of recipes that can effectively ingest preferable nutrients, suppressing food loss and making it easier to create recipes.
  • the recipe information may be acquired from the Internet, or a manufacturer or the like may independently create a table. Recipe information can be obtained by a known technique. It is assumed that the recipe information will be displayed after being rearranged in order of health index, but this is not the only option, and the recipe information may be displayed in a list without prioritization.
  • the proposal information generation unit 127 may further include a cooking method identification unit that identifies the cooking method of the dish indicated in the recipe. Furthermore, the proposal information generation unit 127 may be configured to include at least one of the additional ingredient determination unit 1271, the healthy recipe information generation unit 1272, and the cooking method identification unit. It is desirable that the cooking method specifying unit obtains the cooking method from an external device, as with the healthy recipe information generating unit 1272 .
  • the display unit 128 displays the health index, the refrigerator health index, the consumption health index, the current nutritional balance in the refrigerator, the nutritional balance of ingredients consumed in the past, the ingredients in the refrigerator, additional purchased ingredients, healthy recipes, and the like. 1 is displayed on a display device such as a liquid crystal display. It should be noted that it is not necessary to display all of the items described above, and only one of them may be displayed.
  • the display unit 128 may be implemented as an output unit such as the I/O interface 13 or the I/F 14 .
  • the ordering unit 129 displays a list of foodstuffs that are proposed to be additionally purchased in order to improve the health index on a display device such as the mobile terminal 7 or the liquid crystal display of the refrigerator, and automatically orders the displayed foodstuffs.
  • a display device such as the mobile terminal 7 or the liquid crystal display of the refrigerator
  • the user may press an order button to semi-automatically place an order for ingredients from an online supermarket.
  • the index for placing an order may be not only the health index but also the in-fridge health index and consumption health index. Note that the ordering unit 129 may be omitted without being included in the processing.
  • the internal control unit 130 controls the temperature and humidity inside the refrigerator 1 by controlling a motor and a compressor (not shown).
  • FIG. 3 is a diagram showing an example of a fisheye camera image GC captured by the camera 50.
  • the camera 50 of this embodiment is a fish-eye lens
  • FIG. 4 shows a fish-eye camera image GC taken with this lens.
  • the recognition unit 124 it is difficult to recognize the ingredients by using the fisheye camera image CG as it is. That is, in the learning phase of the machine learning model, since learning is performed using undistorted images, it is difficult to recognize the fisheye camera image GC that is distorted by the fisheye or wide-angle lens 52 as it is.
  • FIG. 4 is a diagram showing an example of an image obtained by converting the fisheye camera image GC into a planar image.
  • the image conversion unit 123 generates the recognition image G30 by synthesizing the distortion-removed right door developed image G32, front developed image G33, left door developed image G34, and upper developed image G31.
  • a known camera calibration technique may be used.
  • an image whose pattern after distortion correction is known in advance such as a checkboard
  • feature points before and after correction are extracted. Parameters may be estimated.
  • FIG. 2 is a flow chart showing an example of the in-fridge management process performed by the control unit 10.
  • the imaging unit 121 acquires a visible light image captured by the camera 50 .
  • the photographing unit 121 stores the acquired visible light image in the image buffer 122 as the camera image GC.
  • step S17 the image conversion unit 123 reads the camera image GC captured in step S16 from the image buffer 122 and develops the fisheye image into a planar image.
  • step S18 the learning-based image recognition unit of the recognition unit 124 receives the input of the recognition image G30 generated by the image conversion unit 123, and executes image recognition of the ingredients using the machine learning model obtained from the computer 9.
  • the rule-based image recognition unit of the recognition unit 124 receives the input of the recognition image G30 generated by the image conversion unit 123, performs image recognition of ingredients on a rule basis, and outputs the content of the recognized label.
  • the processing order of learning-based image recognition and rule-based image recognition is not limited to the above, and they may be executed in parallel.
  • the health index calculator 126 calculates the health index using the ingredient nutritional component table T1. It is desirable that this health index includes the in-fridge health index and consumption health index.
  • FIG. 5 is a diagram showing an example of the ingredient nutritional component table T1. The nutrient ratio of the food in the refrigerator recognized by the recognition unit 124 is calculated based on the food nutrient component table T1. For this reason, in this step, the health index calculator 126 calculates the in-fridge health index based on the nutrients of each ingredient recognized from the inside of the refrigerator. The in-fridge health index is calculated based on the current user's food inventory information. The purpose of the in-fridge health index is to keep the in-fridge nutritional balance so that additional ingredients can be suggested to the user based on the consumption health index.
  • the health index calculation unit 126 calculates the consumption health index based on the consumption history of foodstuffs that are determined to have been consumed by the user within a certain period from the present to the past. For this reason, the health index calculator 126 can calculate the consumption health index based on the ingredients consumed by the user from the time-series difference (change) of the ingredients in the refrigerator. For this calculation, it is desirable that the recognizing unit 124 identifies the consumable foods that are consumed by the user from time-series changes in the foods in the refrigerator.
  • the purpose of the consumption health index is to suggest to the user which foodstuffs are better to consume from the consumption balance of time-series differences in combination with the refrigerator health index.
  • the user can freely select N days as the time-series difference from the previous day, 2 or 3 days before, or 1 week before. Therefore, it is possible to propose a purchase in which the inventory in the warehouse is kept until N days specified by the user.
  • the health index calculation unit 126 calculates a health index based on the user's current stock food information (in-fridge health index) and food consumption history (consumption health index).
  • the health index is an index corresponding to the nutrients recommended for intake by the user, and more preferably information for facilitating the securing of necessary and sufficient amounts of ingredients to be nutritiously ingested in the future in the refrigerator. desirable.
  • the additional foodstuff determination unit 1271 of the proposal information generation unit 127 identifies additional foodstuffs.
  • the additional foodstuff determination unit 1271 identifies foodstuffs that are recommended to be purchased and their amounts in consideration of the inventory amount in the refrigerator and the nutrients to be taken, based on nutritional parameters, in order to increase the health index. and output this. As a result, it becomes easier to secure the necessary and sufficient amount of food materials to be ingested in the future in terms of nutrition in the refrigerator.
  • step S22 grasps the shortage of each nutrient. For example, protein: 30 deficiency, lipid: 70 deficiency, etc.).
  • the additional foodstuff determination unit 1271 searches foodstuffs that match the most deficient nutrient (lipid in the above description) from the foodstuff nutritional component table T1. For example, nuts (lipid: 47.6) are searched.
  • the additional foodstuff determination unit 1271 adds the nutrients of the searched foodstuff to calculate a new deficiency value.
  • protein 10.2 deficit
  • lipid 17.6 deficit.
  • the additional ingredient determination unit 1271 searches for a new ingredient with the most deficient nutrients.
  • the additional foodstuff determination unit 1271 searches foodstuffs satisfying the fat: 17.6 from the foodstuff nutritional component table T1.
  • the additional food ingredient determination unit 1271 may be configured to cancel the searched food ingredient and search for the food ingredient with the next highest nutrient content. , may be used as a search result. Further, it may be configured such that the user decides whether to cancel or not.
  • the additional foodstuff determination unit 1271 may specify foodstuffs by performing calculations using a combination optimization algorithm based on the ratio of each nutrient deficiency. In this case, it is desirable to calculate so that the amount (number, weight) of the ingredients is minimized.
  • step S23 the display unit 128 displays the health index, the health index in the refrigerator, the consumption health index, the current nutritional balance in the refrigerator, the nutritional balance of ingredients consumed in the past, the ingredients stored in the refrigerator, additional ingredients, healthy recipes,
  • the cooking method or the like is displayed on the display device or the mobile terminal 7 . It should be noted that it is not necessary to display all of the items described above, and only one of them may be displayed.
  • FIG. 6 is a diagram showing a display example of proposal information.
  • This display may be performed by either the display device or the mobile terminal 7, but the case where the display is performed by the mobile terminal 7 will be described.
  • the mobile terminal 7 can be realized by a so-called smart phone as described above.
  • the mobile terminal 7 is not limited to a smart phone, and may be a tablet terminal, a wearable terminal, a laptop terminal, or the like.
  • the refrigerator management application running on the mobile terminal 7 communicates with the refrigerator 1 via the network CN on a regular or irregular basis. This application (in-fridge management application 720) will be described later with reference to FIG.
  • FIG. 6 The display example shown in FIG.
  • the health index and overall evaluation are displayed.
  • the lower left part 62 on the left side information on food ingredients in the refrigerator (interior health index) and nutritional balance in the refrigerator are displayed.
  • Consumed foodstuff information (consumption health index) and consumption nutritional balance are displayed in the left lower right portion 63 .
  • the right part 64 in order to increase the health index, foodstuffs that are newly proposed to be additionally purchased are displayed in consideration of the inventory amount in the refrigerator and the nutrients to be ingested.
  • the suggested information to be displayed is not limited to that shown in FIG.
  • foodstuffs with a high past consumption rate of the user may be displayed preferentially or only displayed.
  • an additional ingredient different from that shown in FIG. 6 is proposed.
  • an additional purchase of protein is suggested, and natto and eggs containing a lot of protein are often eaten.
  • foods containing a lot of protein are displayed in the order of sausage, pork, ham, natto, and eggs.
  • the user has a tendency to eat a lot of natto and eggs based on the food consumption history of the user.
  • natto, eggs, ham, pork, and sausage may be displayed in this order, or only natto and eggs may be displayed.
  • step S24 the ordering unit 129 displays a list of foodstuffs to be additionally purchased on the mobile terminal 7 or the display device, and automatically orders the displayed foodstuffs, or the user presses an order button to place an online order. You can semi-automatically order ingredients from a supermarket. Note that the ordering unit 129 may be omitted without being included in the processing.
  • FIG. 7 is a diagram showing an example of an order screen displayed on the mobile terminal 7. As shown in FIG. This ordering screen is displayed on the touch panel 703 as an ingredient ordering list 73 .
  • This foodstuff order list 73 includes foodstuffs that have been received from the refrigerator 1 and are proposed for additional purchase.
  • An order destination 74 and an order button 75 are displayed below the ingredients order list 73 .
  • the contents of the ingredients ordering list 73 are sent to the ordering party 74 .
  • FIG. 7 shows an example in which a single supplier 74 is displayed, a plurality of suppliers 74 may be displayed so that the user can select one when ordering.
  • an orderer may be selected from a plurality of pre-registered orderers according to the type of ingredients.
  • timing at which the additional food material determination unit 1271 generates the food material order list 73 is not limited to immediately after the camera 50 captures the image of the inside of the refrigerator, but can be performed regularly or irregularly.
  • the control unit 10 holds the food ordering list 73 calculated from the recognition result of the camera image GC in the storage device 12, and when receiving a request for the food ordering list 73 from the mobile terminal 7, the latest food ingredients
  • the order list 73 may be transmitted to the mobile terminal 7 .
  • the user of the refrigerator 1 can quickly grasp the ingredients to be additionally purchased by referring to the portable terminal 7 even when the user is out.
  • the machine learning model of the recognition unit 124 can be updated to the latest machine learning model according to changes or additions to food packages.
  • a machine learning model received from a server may be updated as the machine learning model of the recognition unit 124 .
  • the health index is calculated using the user's product consumption history (consumption health index) and the current user's product information in the storage (inside health index). It is possible to propose the amount of ingredients to be additionally purchased in consideration of the stock amount of food and the nutrients to be taken.
  • the item consumption history (consumption health index)
  • the user can freely select the number of days of the past time-series difference, so that it is possible to propose purchases in which the stock in the storehouse is kept until after the selected number of days.
  • Example 2 that proposes a healthy recipe for improving the health index will be described with reference to Figures 8 and 9. In this embodiment, differences from the first embodiment will be mainly described. In the present embodiment, among recipes that can be made from ingredients in the refrigerator, recipes with a higher health index are proposed to the user.
  • FIG. 8 is a flow chart showing an example of the in-fridge management process executed by the control unit 10 in this embodiment. Steps S16 to S18 and S24 of this process are the same as steps S16 to S18 and S24 described with reference to FIG. 2, so description thereof will be omitted. In this process, new steps S25 and S26 are added. Moreover, the same configuration as that of the first embodiment can be adopted for the configuration of the present embodiment.
  • the healthy recipe information generation unit 1272 rearranges and displays recipes that can be made from the recognized ingredients in the refrigerator in order of health index, and proposes a recipe with a higher health index. This will make it easier to create healthy recipes that reduce food loss by proposing a wealth of healthy recipes that allow for effective intake of favorable nutrients. In addition, instead of sorting in order of health index, you may sort in order of in-fridge health index or consumption health index.
  • step S26 the display unit 128 causes the mobile terminal 7, the display device, etc. to display the ingredients in the refrigerator and healthy recipes in addition to the health index, additional ingredients, and the like.
  • FIG. 9 is a diagram showing an example of the healthy recipe list 65 displayed in this step. In FIG. 9 it is divided into a left side 66 and a right side 67 . A list of ingredients in the refrigerator is displayed on the left side 66 by the display unit 128 . In addition, the display unit 128 displays the recipes that can be made from the ingredients in the refrigerator on the right side 67 in descending order of the health index.
  • Cooking type designation 1 cooking type designation 2, and cooking type designation 3 are given as examples.
  • the food type designation 1 places emphasis on balance, and includes staple foods, main dishes, side dishes, soups, milk/dairy products, and fruits.
  • cooking type designation 2 fish-centered, meat-centered, vegetable-centered, etc. can be selected.
  • Cooking type designation 3 Health-focused: When couples want to be healthy every day, Health-focused: When you want to eat today without worrying about your health index, Emphasis on stamina: Children who came home from club activities You can also choose when you want to eat a lot, events: when you want to celebrate, such as birthdays or passing exams.
  • the present embodiment configured in this way also has the same effect as the first embodiment. Furthermore, in this embodiment, by proposing recipes that increase the health index of ingredients in the refrigerator, it is possible to propose abundant recipes that allow effective intake of desirable nutrients, thereby suppressing food loss and making it easier to create recipes.
  • FIG. 11 shows the configuration of the mobile terminal 7 that performs the in-fridge management process of this modified example.
  • the mobile terminal 7 of this example has a processor 701 , a storage device 702 , a touch panel 703 and a communication section 704 .
  • the mobile terminal 7 can be realized by an information processing device (computer) such as a smart phone, as described above.
  • the processor 701 and storage device 702 have the same functions as the control unit 10 shown in FIG. Also, the touch panel 703 functions as an input/output unit. Furthermore, the communication unit 704 is connected with the network CN. This connection may be wireless or prioritized.
  • the refrigerator management application 720 includes a recognition module 721 , a table control module 722 , a health index calculation module 723 , a proposal information generation module 724 , an order module 725 and an interior control instruction module 726 .
  • the proposal information generation module 724 is composed of an additional ingredients determination module 7241 and a healthy recipe information generation module 7242 . These are configured as one computer program (application), but each module may be configured as an independent computer program, or some modules may be collectively implemented as a computer program.
  • Each of these modules also performs the same functions as the functional units shown in FIG. That is, it has the following correspondence.
  • Recognition module 721 Recognition unit 124
  • Table control module 722 table control unit 125
  • Health index calculation module 723 health index calculation unit 126
  • Proposal information generation module 724 Proposal information generation unit 127
  • Additional ingredient determination module 7241 Additional ingredient determination unit 1271 Healthy recipe information generation module 7242: Healthy recipe information generation unit 1272
  • Ordering Module 725 Ordering Unit 129
  • In-fridge control instruction module 726 In-fridge control section 130
  • each module executes the same processing as the corresponding functional unit, but it is desirable that the internal control instruction module 726 further manages the usage status of food materials in the refrigerator. For example, the internal control instruction module 726 acquires information read from a user's input or food code, and the recognition unit 124 recognizes the corresponding food.
  • the refrigerator management application 720 be delivered to the mobile terminal 7 via the network CN. Therefore, the network CN will be implemented on the Internet.
  • the health index is calculated using the user's product consumption history (consumption health index) and the current user's product information in the storage (in-store health index), Considering the amount of stock in the refrigerator and the nutrients to be ingested, the amount of ingredients to be purchased additionally can be proposed.
  • the item consumption history (consumption health index)
  • the user can freely select the number of days of the past time-series difference, so that it is possible to propose purchases in which the stock in the storehouse is kept until after the selected number of days.
  • each of the above configurations, functions, etc. may be realized by hardware, for example, by designing them as integrated circuits.
  • each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function.
  • Information such as programs, tables, and files for realizing each function is stored in the storage device.
  • This storage device includes storage devices such as non-volatile semiconductor memories, hard disk drives, SSDs (Solid State Drives), or computer-readable non-temporary data storage media such as IC cards, SD cards, and DVDs.
  • control lines and information lines indicate what is considered necessary for explanation, and not all control lines and information lines are necessarily indicated on the product. In fact, it may be considered that almost all configurations are interconnected.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Operations Research (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Human Resources & Organizations (AREA)
  • Biomedical Technology (AREA)
  • Marketing (AREA)
  • Databases & Information Systems (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Pathology (AREA)
  • Economics (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
PCT/JP2022/024657 2021-10-12 2022-06-21 領域管理機器およびプログラム WO2023062880A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202280049854.1A CN117716436A (zh) 2021-10-12 2022-06-21 区域管理设备和程序

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-167311 2021-10-12
JP2021167311A JP2023057695A (ja) 2021-10-12 2021-10-12 領域管理機器およびプログラム

Publications (1)

Publication Number Publication Date
WO2023062880A1 true WO2023062880A1 (ja) 2023-04-20

Family

ID=85988267

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/024657 WO2023062880A1 (ja) 2021-10-12 2022-06-21 領域管理機器およびプログラム

Country Status (3)

Country Link
JP (1) JP2023057695A (zh)
CN (1) CN117716436A (zh)
WO (1) WO2023062880A1 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000348122A (ja) * 1999-06-09 2000-12-15 Matsushita Electric Ind Co Ltd 履歴データベース装置
JP2013250699A (ja) * 2012-05-31 2013-12-12 Nikon Corp 献立支援装置、献立支援方法
JP2021064261A (ja) * 2019-10-16 2021-04-22 株式会社日立ソリューションズ・クリエイト 献立提供システムおよびコンピュータプログラム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000348122A (ja) * 1999-06-09 2000-12-15 Matsushita Electric Ind Co Ltd 履歴データベース装置
JP2013250699A (ja) * 2012-05-31 2013-12-12 Nikon Corp 献立支援装置、献立支援方法
JP2021064261A (ja) * 2019-10-16 2021-04-22 株式会社日立ソリューションズ・クリエイト 献立提供システムおよびコンピュータプログラム

Also Published As

Publication number Publication date
JP2023057695A (ja) 2023-04-24
CN117716436A (zh) 2024-03-15

Similar Documents

Publication Publication Date Title
US9877504B2 (en) Conditioning system for nutritional substances
US9902511B2 (en) Transformation system for optimization of nutritional substances at consumption
US9165320B1 (en) Automatic item selection and ordering based on recipe
US9497990B2 (en) Local storage and conditioning systems for nutritional substances
US9171061B2 (en) Local storage and conditioning systems for nutritional substances
US20130273509A1 (en) Method of Monitoring Nutritional Intake by Image Processing
US20150100462A1 (en) Systems and Methods to Improve Nutrition
WO2013134325A1 (en) Transformation system for optimization of nutritional substances at consumption
WO2013126579A1 (en) Conditioning system for nutritional substances
KR20210114384A (ko) 신경망에 기초한 레시피 생성
JP6903523B2 (ja) 情報処理装置、情報処理方法、及びプログラム
US20220406215A1 (en) Systems and methods for dynamically providing dynamic nutritional guidance
US20140255883A1 (en) System and method for automated monitoring of food and beverage intake, determining associated nutritional information and comparing with a predetermined dietary plan
JP2019133624A (ja) レシピ情報提供装置、レシピ情報提供方法およびレシピ情報提供プログラム
CN211319727U (zh) 一种智能家庭健康饮食推荐系统
CN112102921A (zh) 营养食谱的生成方法、装置和服务器
WO2023062880A1 (ja) 領域管理機器およびプログラム
JP2011100488A (ja) 農畜水産物材料セット受注装置、農畜水産物材料セット受注方法および農畜水産物材料セット受注プログラム
CN109932043A (zh) 一种营养管理称
CN107610749B (zh) 一种饮食监测管理方法及系统
CN109829790A (zh) 一种用于生成采购信息的方法与设备
JP2001195385A (ja) レシピ提供システム
CN112232917B (zh) 电商平台的商品推荐方法、装置及设备
JP6112328B1 (ja) サプリメント探索システム。
JP7345810B1 (ja) 情報処理装置、情報処理方法、及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22880586

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280049854.1

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22880586

Country of ref document: EP

Kind code of ref document: A1