JP4690082B2 - Health maintenance support system, information processing system and user system used therefor, operation method, and operation program - Google Patents

Description

  The present invention relates to a health maintenance support system, a support method, and a support program.

  In recent years, various support systems that incorporate information communication technologies represented by the Internet, mobile phones, and the like have been proposed, reflecting the further increase in people's health.

For example, a diet support system (for example, refer to Patent Document 1) for easily calculating and managing daily calorie intake, a health support system for providing users with meaningful information for their own health management, and the like A portable information terminal program to be used (for example, see Patent Document 2), a meal management support system for improving eating habits (for example, see Patent Document 3),
There is a system using a dietary support portal server (for example, see Patent Document 4), and a dietary management system (for example, see Patent Document 5) that considers nutritional consideration as well as personal likes and dislikes.

Also, a meal management support system that supports selection of meal menus that match the purpose of prevention, coping, and treatment, and manages meal contents and health management information ingested to support meal management for each user (for example, Patent Document 6) There is also a nutrition management system (for example, see Patent Document 7) that allows a person who performs health management by meals such as treatment and prevention of illness, diet, and the caregiver to easily and continuously perform nutrition management. .
JP 2002-32487 A JP 2003-44595 A JP 2003-67497 A JP 2003-67503 A JP 2003-141337 A JP 2003-242261 A JP 2004-164038 A

  People who live alone have become more convenient to purchase meals by using convenience stores and 24-hour supermarkets, but on the other hand, nutrition is biased by purchasing only their favorite ingredients and lunch boxes. There was a problem. Also, people who need dietary restrictions due to diet, illness, etc. take a set menu regardless of the calories they have consumed. In addition, regarding foods purchased at convenience stores, supermarkets, lunch boxes, etc., and foods eaten at restaurants, if you do not have a calorie notation, you do not know how much calories you have consumed, and you know temporarily However, in order to manage it, there was a problem that it was very troublesome to record it yourself.

  In addition, in the various support systems using the information communication technology described above, the calorie consumption other than the collected calorie is not considered at all, or even if the calorie consumption is considered, the operation is complicated by manual input. For example, it is not always convenient for some users. Also, in building such a support system, the lifestyles of users differ greatly, for example, between those who live alone and those who are married or live together, and even those who live alone can cook their own food, Some people rely on lunch boxes, so it is also desirable to consider these differences in lifestyle.

  The present invention has been made in consideration of such conventional circumstances, eliminates the nutritional bias that people who live alone, etc. are likely to fall into, and even those who need dietary restrictions due to diet or illness etc. The purpose of this is to provide a menu that takes into account, and to easily obtain calories ingested without the calorie notation and eliminate the trouble of record management.

To achieve the above object, a health maintenance support system according to the present invention is a health maintenance support system having an information processing system and a user system connected to the information processing system via a communication line. the gas-user system captures an image of the receipt that the user has been given the product identification information of the shop shop that Ru is passed at the time of cooking or food purchase of meals eat or drink, the captured image via the communication line has an image transmitting means for transmitting to the information processing system, the number of steps of the user is measured, and a step count data transmitting means for transmitting to the information processing system through a pre SL communication line number of steps data measured, the the information processing system, for each user, the data registration date, food, calories, calories, and nutrient data associated with each other and updatable registers, the nutrient Data for each nutrient, a contractor database that can be processed in association with information that is sufficient, small, insufficient, or not included, and the product identification for each store Register the food calorie and nutrient data in association with the information in an updatable manner, and the nutrient data is either rich, sufficient, low, missing, or not included for each nutrient. A business partner company database that can be processed in association with information, a contractor purchased food database that registers the data of food purchased by the user in an updatable manner in association with the product identification information, and a nutrient for each food. And a calorie database that associates and registers the calorie data, and for each meal menu, the composition recipe and the meal recipe that indicates the amount, dish image, calories, and The data of the element is registered so as to be updatable, and the data of the nutrient is processed in association with the information that is rich, sufficient, small, insufficient, or not included for each nutrient. A possible meal recipe database and an image recognition processing device that analyzes the image sent by the image transmission means and recognizes whether the image is the dish image or the receipt image by image processing based on an image recognition algorithm When the image is the image of the dish, the dish image in the meal recipe database is searched, the meal menu corresponding to the searched processing image is recognized, and the constituent ingredients corresponding to the recognized meal menu are determined. The determined constituent ingredients are compared with the food in the contractor-purchased food database to determine whether the relevant constituent ingredients exist, and the corresponding constituent If there are ingredients, it is determined that the dish is created by the user, the contractor purchased food DB is compared with the meal recipe, and the ingredients used in the dish and the amount thereof are determined. The calorie intake and nutrient data are obtained, and the contractor database is updated with the obtained calorie intake and nutrient data. The contractor database is updated with calorie intake and nutrient data corresponding to the image, and if the image is the image of the receipt, the business partner company database is searched from the image of the receipt and the corresponding store and its product Identify identification information, obtain calorie intake and nutrient data corresponding to the identified product identification information of the store, The contractor database is updated with calorie and nutrient data, and a calorie consumption calculation formula based on personal data including a user's step length and weight set in advance based on the step data sent by the step data transmission means The calorie consumption is calculated using, the contractor database is updated with the calculated calorie consumption data, and the calorie intake consumed for each user and the nutrients of the meal are calculated using the contractor database. Manage the history of nutrient data associated with rich, sufficient, low, deficient, or not included information and calorie consumption data calculated from the step count data And a data server .

In addition, an information processing system to be used in the health maintenance support system according to the present invention captures an image of the receipt that the user has been given the product identification information of the shop shop that Ru is passed at the time of cooking or food purchase of meal that you eat, drink, imaging an image transmission means for transmitting to the information processing system via the communication line the image, the number of steps of the user is measured, transmits the step count data measured in said information processing system through a pre SL communication line An information processing system connected via a communication line to a user system having a step count data transmission means for each user, and for each user, data registration date and time, food, calorie intake, calorie consumption, and nutrient data are mutually In association with updatable registration, the nutrient data is either rich, sufficient, low, missing, or not included in each nutrient. Corresponding contractor database that can be processed in association with each store, the food calorie and nutrient data are registered in an updatable manner in association with the product identification information, and the nutrient data is abundant for each nutrient. Business partner company database that can be processed in association with any of the information that is small, missing, or not included, and data of food purchased by the user in association with the product identification information Contractor purchase food database to register the foods updatable, calorie database to register the data of nutrients and calories for each food, and meal recipes, food images, and calories showing the ingredients and their amounts for each meal menu , And register the nutrient data in an updatable manner, and the nutrient data is sufficient for each nutrient, Analyzing the meal recipe database that can be processed in association with any of missing, missing, or not included information, and the image sent by the image transmitting means, the image is an image of the dish Or an image recognition processing device for recognizing the image of the receipt by image processing based on an image recognition algorithm, and when the image is the image of the dish, the dish image in the meal recipe database is searched, and the searched processing image is Recognize the corresponding meal menu, determine the constituent ingredients corresponding to the recognized meal menu, compare the determined constituent ingredients with the food in the contractor purchased food database, and whether the relevant constituent ingredients exist If it is determined that there is a corresponding constituent ingredient, it is determined that the dish is created by the user and the contractor purchase food DB and the meal recipe By comparing foods and foods and determining the amount of ingredients used in cooking, the intake calorie and nutrient data are obtained, and the contractor database is updated with the obtained intake calorie and nutrient data. If there is no constituent food to be used, it is determined that eating out and the contractor database is updated with intake calorie and nutrient data corresponding to the dish image in the meal recipe database, and the image is an image of the receipt , Search the business partner database from the image of the receipt to identify the corresponding store and its product identification information, determine the calorie intake and nutrient data corresponding to the product identification information of the identified store, and find the calorie intake And the contractor database is updated with the nutrient data and sent by the step count data transmission means. Calculate calorie consumption using a calorie consumption calculation formula based on personal data including the user's preset stride and weight based on the step count data coming, and update the contractor database with the calculated calorie consumption data And using the contractor database, either the calorie intake of each meal consumed for each user and the abundance, sufficient, low, deficient, or not included for each nutrient of the meal A data server for managing a history of nutrient data associated with the information and calorie consumption data calculated from the step count data .

In addition, the operation method of the information processing system according to the present invention captures an image of a receipt attached with merchandise identification information of a store that is passed when a user eats or eats food or purchases food, and communicates the captured image. Image transmitting means for transmitting to the information processing system via a line; and step count data transmitting means for measuring the number of steps of the user and transmitting the measured step count data to the information processing system via the communication line. An information processing system operating method connected to a user system via a communication line, the information processing system including data registration date / time, food, calorie intake, calorie consumption, and nutrient data for each user Are associated with each other and are registered to be updatable, and the nutrient data is rich, sufficient, low, missing, not included for each nutrient The contractor database that can be processed in association with the information of the difference, and for each store, the calorie and nutrient data of the food is registered in association with the product identification information, and the nutrient data is stored for each nutrient. Purchase by the user in association with the business partner database that can be processed in association with any of the information that is abundant, sufficient, small, insufficient, or not included, and the product identification information Contractor purchase food database for registering updated food data in an updatable manner, a calorie database for associating and registering nutrient and calorie data for each food, and a meal recipe that indicates the constituent ingredients and their amounts for each meal menu , Cooking images, calories, and nutrient data are registered in association with each other and can be updated, and the nutrient data is abundant for each nutrient. , A meal recipe database that can be processed in association with information that is sufficient, few, missing, or not included, an image recognition processing device, and a data server connected to the image recognition processing device And the image recognition processing device analyzes the image sent by the image transmitting means, and recognizes whether the image is the dish image or the receipt image by image processing based on an image recognition algorithm. When the image is an image of the dish, the data server searches for a dish image in the meal recipe database, recognizes a meal menu corresponding to the searched processing image, and corresponds to the recognized meal menu. Determine the constituent ingredients, compare the determined constituent ingredients with the food in the contractor purchase food database, and determine whether the relevant constituent ingredients exist If the corresponding constituent ingredients exist, the contractor purchase food DB is compared with the meal recipe by judging that the dish is created by the user, and the ingredients used in the dish and the amount thereof By determining the calorie intake and nutrient data, the contractor database is updated with the obtained calorie intake and nutrient data, and if there is no corresponding constituent food, the meal is determined as eating out. Update the contractor database with the calorie intake and nutrient data corresponding to the cooking image in the recipe database, and if the image is the receipt image, search the business partner company database from the receipt image and apply The store and its product identification information, and the calorie intake and nutrient data corresponding to the product identification information of the specified store Based on the personal data including the step length and weight of the user set in advance based on the step count data sent by the step count data transmission means Calculate the calorie consumption using the calorie consumption calculation formula, update the contractor database with the calculated calorie consumption data, and use the contractor database, the calorie intake consumed for each user, Nutrient data associated with any of the information that is abundant, sufficient, small, deficient, or not included for each nutrient of the meal, and calorie consumption data calculated from the step count data It is characterized by managing the history .

Further, the operation program of the information processing system according to the present invention captures an image of a receipt attached with merchandise identification information of a store delivered when a user eats or eats a meal or food, and the captured image is Image transmitting means for transmitting to the information processing system via a communication line; step count data transmitting means for measuring the number of steps of the user and transmitting the measured step count data to the information processing system via the communication line; An information processing system operation program connected to a user system via a communication line, and for each user, data registration date, food, intake calories, consumed calories, and nutrient data are mutually Associate and updatable registration, the nutrient data is rich, sufficient, low, missing, included for each nutrient The contractor database that can be processed in association with any of the information, and for each store, the food calorie and nutrient data associated with the product identification information are registered to be updatable, and the nutrient data A business partner company database that can be processed in association with any of the information that is abundant, sufficient, small, insufficient, or not included, and the user in association with the product identification information The contractor purchase food database for registering the data of food purchased by the customer in an updatable manner, the calorie database for registering the data of nutrients and calories for each food, and the constituent ingredients and the amount for each meal menu are shown. Meal recipe, cooking image, calorie, and nutrient data are registered in association with each other and can be updated, and the nutrient data is stored for each nutrient. Analyzing the meal recipe database that can be processed in association with any of the information that is rich, sufficient, small, insufficient, or not included, and the image sent by the image transmission means An image recognition processing device for recognizing whether the image is the image of the dish or the image of the receipt by image processing based on an image recognition algorithm; and if the image is the image of the dish, search for a dish image in the meal recipe database And recognizing the meal menu corresponding to the retrieved processed image, determining the constituent ingredients corresponding to the recognized meal menu, comparing the determined constituent ingredients with the food in the contractor purchase food database, and If there is a corresponding constituent ingredient, it is determined that the dish is a dish created by the user and the contractor purchases it. By comparing the food input DB with the meal recipe and determining the ingredients used in cooking and the amount thereof, the calorie intake and nutrient data are obtained, and the contractor is determined based on the obtained calorie intake and nutrient data. When the database is updated, and the corresponding constituent food does not exist, the contractor database is updated with the intake calorie and nutrient data corresponding to the cooking image of the meal recipe database by determining that the food is a meal, and the image is In the case of the receipt image, the business partner company database is searched from the receipt image to identify the corresponding store and its product identification information, and the calorie intake and nutrient data corresponding to the identified product identification information is obtained. Update the contractor database with the calculated calorie intake and nutrient data, and the number of steps Calorie consumption is calculated using a calorie consumption calculation formula based on personal data including the user's step length and weight set in advance based on the step count data sent by the data transmission means, and the calculated calorie consumption data The contractor database is updated by the above, using the contractor database, the intake calorie of the meal ingested for each user and the abundance for each nutrient of the meal, sufficient, few, insufficient, It is for functioning as an information processing system having a data server for managing history of nutrient data associated with any information not included and calorie consumption data calculated from the step count data. is there.

  According to the present invention, the nutritional bias that people who live alone are likely to fall into is eliminated, and even those who need dietary restrictions due to diet or illness provide a menu that takes into account calorie consumption, Even if it is not, the calorie intake can be easily acquired and the trouble of record management can be eliminated.

  Hereinafter, the best mode for carrying out a health maintenance support system, an information processing system and a user system, an operation method, and an operation program used in the present invention will be described in detail with reference to the drawings.

  The health maintenance support system according to the present embodiment is applied to an information processing system of a service provider, and is a person living outside, a person for a diet purpose, and a person who needs to restrict diets due to illness. Is the user (hereinafter referred to as “contractor”), receives the images of food and drink taken with the camera by this contractor, and automatically calculates the calorie intake of the contractor based on the received image data Execute the process.

  In addition, this health maintenance support system receives images of receipts of foods and the like purchased at convenience stores, supermarkets, and other stores by the contractor, and taken by the contractor based on the received image data. A process of identifying a history of nutrition and missing nutrients, determining a meal recipe and a meal menu according to this, and presenting it to the contractor is executed.

  Furthermore, this health maintenance support system is configured by pressing the “Send” button just before returning from the pedometer (also referred to as “pedometer (registered trademark)” or “pedometer”) worn by the contractor. Receive the measurement data from the pedometer sent, calculate the calorie consumed by the contractor based on this measurement data, and in accordance with this calculation data and the living environment data of the contractor specified in advance, Executes processing that supports the provision of services.

  1) If the contractor is a self-cooking person living alone, a service for sending recipes and delivering ingredients is provided.

  2) If the contractor is a bento purchaser who lives alone, a service is provided to display a recommended menu and deliver the store selected by the contractor.

  3) If the contractor is a single person living out, a recommended menu display and pre-order service will be provided from a business partner restaurant.

  4) If the contractor is a living person or a married person, a service is provided for sending a recommended menu and a meal recipe or delivering ingredients at a designated time.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  The health maintenance support system according to this embodiment uses a person living outside, a person for dieting purposes, and a person whose calorie is restricted due to illness as a contractor (user), and a meal recipe tailored to these contractors. And an information processing system for a service provider that provides a service for presenting a meal menu.

  FIG. 1 shows the configuration of the entire system to which the health maintenance support system of this embodiment is applied.

  The system shown in the example of FIG. 1 includes an information processing system (hereinafter referred to as “service provider system”) 1 having a health maintenance support system 10 of the present embodiment and a contractor who receives a service from the service provider. Convenience stores, supermarkets, restaurants, bento shops, hospitals (medical institutions), and transport companies (which are business partners) between the user system (hereinafter referred to as “contractor system”) 2 used by the service provider The information processing system (hereinafter referred to as “business alliance company system”) 3 on the business alliance company side such as a delivery company). Each of these systems 1 to 3 is connected via a communication line 100 such as a public network functioning as the Internet based on TCP (Transmission Control Protocol) / IP (Internet Protocol) or a dedicated line.

  FIG. 2 shows a configuration example of the health maintenance support system 10.

  The health maintenance support system 10 shown in the example of FIG. 2 includes a data server 11, an image recognition processing device 12, a contractor DB (database) 13, a business partner company DB 14, a contractor purchased food DB 15, a calorie DB 16, a meal recipe DB 17, and a LAN. A web server 19 having a communication I / F (interface) 18 such as an interface and a web page (homepage) 19 a is provided, and is connected to the communication line 100 via the communication I / F 18.

  The data server 11 is configured by a computer having a CPU 111 that executes a preset processing program 112.

  The image recognition processing device 12 is configured by a computer having a CPU 121 that executes a preset image processing program 122.

The contractor DB 13 stores various data such as nutrients and ingested calories and calories consumed by meals consumed by the contractor as detailed information. An example of this is shown in FIG. In the contractor DB 13 shown in the example of FIG. 3, the contractor name, date and time, intake information (food name and calorie), calorie consumption, protein, lipid, carbohydrate, vitamin (A, B1, B2, C, D), calcium Each data of iron, dietary fiber is stored. Among these, as shown in FIG. 3, the data on nutrients (protein to dietary fiber) are: ◎ (rich), ○ (sufficient), △ (less), ▽ (insufficient), × Processing is possible in association with any of the information ( not included).

  In the present embodiment, the business partner company DB 14 includes a plurality of DBs 14a to 14e corresponding to business partner companies (convenience store supermarket, restaurant, lunch box, hospital, transport company).

The DB 14a is for a convenience store / supermarket, and stores various data such as display food product names, calories, and nutrients as detailed information. An example of this is shown in FIG. In the business partner company DB (convenience store supermarket) 14a shown in the example of FIG. 4, the store name, product name (ID: product code attached to the receipt), calories , protein, lipid, carbohydrate, vitamins (A, B1, B2) , C, D), calcium, iron, and dietary fiber data are stored. Among these, as illustrated in FIG. 4, the data on nutrients (protein to dietary fiber) are ◎ (rich), ○ (sufficient), △ (less), ▽ (insufficient), × Processing is possible in association with any of the information ( not included).

The DB 14b is for restaurants, and stores detailed data such as menus, calories, and nutrients. An example of this is shown in FIG. In the business partner company DB (restaurant) 14b shown in the example of FIG. 5, the store name, product name (ID: product code attached to the receipt), calorie , protein, lipid, carbohydrate, vitamin (A, B1, B2, C) , D), calcium, iron, and dietary fiber data are stored. Among these, as illustrated in FIG. 5, the data on nutrients (protein to dietary fiber) are ◎ (rich), ○ (sufficient), △ (less), ▽ (insufficient), × Processing is possible in association with any of the information ( not included).

The DB 14c is for lunch boxes, and stores various data such as menus, calories, and nutrients as detailed information. An example of this is shown in FIG. The business partner DB (bento) 14c shown in the example of FIG. 6 includes a store name, a product name (ID: product code attached to a receipt), calories , protein, lipid, carbohydrate, vitamins (A, B1, B2, C). , D), calcium, iron, and dietary fiber data are stored. Among these, as illustrated in FIG. 6, the data on nutrients (protein to dietary fiber) are ◎ (rich), ○ (sufficient), △ (less), ▽ (insufficient), × Processing is possible in association with any of the information ( not included).

  The DB 14d is for a hospital, and stores detailed data such as a detailed hospital name, a contractor name, a disease name, a doctor name in charge, a nutritionist name, and restriction information. An example of this is shown in FIG. The business partner DB (hospital) 14d shown in FIG. 7 includes a hospital name, a contractor name, a disease name (diabetes mellitus, kidney disease, etc.), a doctor name in charge, a nutritionist name, and restriction information (2000 kcal / day, reduced salt dishes, etc.) Each data is stored.

  The DB 14e is for a delivery company (transportation company), and stores data such as a delivery company name, a delivery company address, a contractor name, and an address as detailed information. An example of this is shown in FIG. The business partner company DB (delivery) 14e shown in FIG. 8 stores data of a delivery company name, delivery company address, contractor name, and address.

  The contractor purchased food DB 15 stores data such as the date of purchase of foods and lunch boxes purchased by the contractor and purchased food (food ID). Particularly, in the contractor purchased food DB 15, an image of a receipt with the food ID (product ID) is taken by the contractor and the image recognition processing device 12 sends the food ID each time the image is transmitted. Is recognized and stored in association with the food ID for each contractor. By referring to the purchased food data in the contractor purchased food DB 15, it is possible to monitor the state of the ingredients stored by the contractor and calculate the calories and nutrients of the ingredients created by the contractor such as breakfast and lunch boxes. It becomes. Furthermore, it is possible to present a meal menu in which the expiration date of the ingredients is managed from the purchase date data.

  The Calorie DB 16 is a database based on the data of the Japanese Food Standard Ingredients Table that shows the standard ingredient values of foods commonly used by Japanese, and calculates “calories” such as meals made by contractors and nutrients. Referenced when.

The meal recipe DB 17 stores each data such as a meal menu, a completed image corresponding to the meal menu, nutrients included in the meal menu, meal menu materials (for one person), and a meal recipe for creating a meal based on the meal menu. ing. An example of this is shown in FIG. The meal recipe DB 17 shown in FIG. 9 includes meal names, ingredients (one portion), recipes, photographs, calories, proteins, lipids, carbohydrates, vitamins (A, B1, B2, C, D), calcium, iron, and dietary fiber. Each data is stored. Among these, as shown in FIG. 9, the data on nutrients (protein to dietary fiber) are ◎ (rich), ○ (sufficient), △ (less), ▽ (insufficient), × Processing is possible in association with any of the information ( not included).

  The web server 19 is configured by a computer having a CPU (not shown) that executes preset web server software (not shown) and manages the web page 19a so that it can be viewed on the Internet.

  The data server 11, the image recognition processing device 12, the various DBs 13 to 17, the communication I / F 18, and the web server 19 are merely functional, and even if they are configured integrally with a single computer, Any of these can be applied, even if they are individually configured with a computer machine.

  FIG. 10 shows a configuration example of the contractor system 2.

  The contractor system 2 shown in the example of FIG. 10 includes a camera 21, a pedometer 22, and a home terminal 23.

  In this embodiment, the camera 21 is composed of, for example, a camera-equipped mobile phone, and has a functional control unit 111 that performs overall control, an imaging unit 212 having a solid-state imaging device such as a CCD or CMOS, and a communication line by wireless communication. A communication unit 213 that can be connected to the health maintenance support system 10 in the service provider company system 1 through 100, an input unit (operation unit) 214 such as operation keys and operation buttons for inputting, selecting, and instructing various data; A display unit 215 such as a liquid crystal display, and a web browser 216 executed by the control unit 111.

  With this configuration, the imaging unit 212 captures images of meals and drinks and receipt images before the contractor ingests, and transmits the captured image data to the data server 11 of the service provider company via the communication unit 213. Or a notification of a contractor instructed by the input unit 214 is sent to the data server 1 via the communication unit 213, and a recommended meal menu as a response is received, or the recommended meal menu is displayed on the display unit 215. Or display.

  In this embodiment, the pedometer 22 is a pedometer with a transmission function provided by a service provider. This pedometer with a transmission function is functionally connected to the control unit 221 responsible for overall control, the step count measurement unit 222 that detects vibrations by a sensor such as an acceleration sensor, and the step count measurement unit 222 by wireless communication via the communication line 100. It has a communication unit 223 that can be connected to the health maintenance support system 10 in the service provider system 1 and a GPS receiver 224 as a position sensor that detects its own position information.

  With this configuration, the step count data measured by the step count measuring unit 222 is transmitted to the data server 11 in the health maintenance support system 10 at a preset location (for example, within a radius of 1 km from the home) via the communication unit 223. Or send automatically at a set time.

  Note that a position sensor such as the GPS receiver 224 is not always necessary, and an interrupt signal is generated in the pedometer 22 when it arrives at a predetermined place such as within a radius of 1 km from the home, and is sent to the data server 11 based on the signal. Any configuration that can operate to transmit data may be used.

  The home terminal 23 is composed of, for example, a mobile phone, a PC, and the like, and functionally controls the control unit 231, the input unit 232 such as a keyboard, the display unit 233 such as a liquid crystal display, and the communication line 100. A communication unit 234 such as a modem or a LAN interface that can be connected to the health maintenance support system 10 in the service provider company system 1 is provided.

  With this configuration, a meal menu and a meal recipe transmitted from the data server 11 of the service provider company are received via the communication unit 234.

  The home terminal 23 may be configured integrally with the camera 21 as a camera-equipped mobile phone. When the pedometer with a transmission function can be mounted on an information terminal such as a PDA, the pedometer 22 may be integrated. Further, when the pedometer with a transmission function can be mounted on a camera-equipped mobile phone, all of the camera 21, the pedometer 22, and the home terminal 23 can be configured as an integrated camera-equipped mobile phone.

  FIG. 11 shows a configuration example of the business partner company system 3.

  The business partner company system 3 shown in the example of FIG. 11 is placed at each store and hospital side which are business partner companies, and communicates with the health maintenance support system 11 in the service provider company system 1 via the communication line 100. It consists of an information processing system equipped with a computer. This information processing system includes a convenience store / supermarket system 31 on the convenience store / supermarket side that provides ingredients, lunch boxes, and drinks, and a restaurant system on the restaurant side that provides meals such as family restaurants and fast food. 32, a lunchbox system 33 on the side of the lunchbox, a hospital system 34 on the side of the hospital where the contractor visits, and a transporter side that delivers meals and ingredients directed by the service provider to the contractor's home And a transportation system 35.

  Next, processing examples of the data server 11 and the image recognition processing device 12 in the health maintenance support system 10 will be described.

  FIG. 12 and FIG. 13 are flowcharts showing processing examples of the data server 11. The flowcharts shown in FIGS. 12 and 13 are set in advance in the processing program 112 and executed by the CPU 111.

  First, when the data server 11 receives the step data transmitted from the pedometer 22 worn by the contractor (step St11), for example, the data server 11 is known based on personal data such as preset step length and weight of the contractor. Using the calorie consumption calculation formula, the calorie consumption is calculated from the step count data (step St123), the calculated calorie consumption is registered in the contractor DB 13, and the contractor DB 13 is updated (step St13).

  In FIG. 13, when the data server 11 receives the inquiry of the contractor transmitted from the contractor system 2 (step St21), the information on the calorie intake and consumption calories and the nutrient bias in the contractor DB 13 (which nutrients Is calculated with reference to a preset reference value (standard value), etc., and calories and nutrients required by the contractor are calculated from the missing calories and nutrients (step St22). ), The product name (ID) of the business partner company DB 14 or the ingredients (ID of the contractor purchased food DB 15) according to the obtained calorie and nutrient data and the preset living environment (see below) of the contractor. ) Is determined, the meal menu is determined from the product name (ID) or the ingredients (ID) (step St23), and the determined meal menu is contracted And it transmits the system 2 (step St24).

  When the contractor receives a meal or receipt image taken by the contractor using the camera 21, the data server 11 also performs a process of passing the received image data to the image recognition processing device 12.

  FIG. 14 is a flowchart illustrating a processing example of the image recognition processing device 12. The flowchart shown in FIG. 14 is set in advance in the image processing program 122 and executed by the CPU 121.

  First, in FIG. 14, the image recognition processing device 12 recognizes a meal or receipt image received by the data server 11 (step St <b> 31). As this image processing for image recognition, an image recognition algorithm employed in known face recognition or the like is employed in this embodiment.

  Next, the image recognition processing device 12 determines whether or not the recognized image is a food dish image (step St32).

  As a result, if the recognized image is a meal image of a meal (step St32: YES), the image of the meal recipe DB 17 is searched from the recognized meal image of the meal, and the meal menu corresponding to the searched image is displayed. Is recognized (step St33), and the constituent ingredients corresponding to the recognized meal menu are determined (step St34).

  Next, the determined constituent ingredients are compared with the ingredients in the contractor purchased ingredient DB 15 (step St35), and it is determined whether or not the corresponding ingredient exists (step St36).

  As a result, if the corresponding food is present (step St36: YES), it is determined that the dish is a dish created by the contractor, and the contractor purchased ingredient DB15 and the meal recipe DB17 are compared (step St37) and used for cooking. The determined food and quantity are determined (step St38), calories and nutrients are calculated (step St39), the contractor DB 13 and the contractor purchased food DB 15 are updated (step St40), and the process is terminated.

  On the other hand, if there is no corresponding food by the determination in step St36 (step St36: NO), it is determined to eat out, and the calorie and nutrient of the contractor DB13 are updated with the calorie and nutrient of the meal recipe DB17 (step St41). ), The process is terminated.

  If the image recognized by the determination in step St32 is not a food dish image (step St32: NO), it is determined whether the recognized image is a receipt image (step St42). As a result, when the recognized image is a receipt image (step St42: YES), the business partner company DB 14 is referred to retrieve the corresponding store and food ID from the information attached to the receipt (step St43). Then, the calorie and nutrient data of the contractor DB 13 are updated with the searched corresponding store and food ID (Step St44), and the process is terminated. In this case, if the corresponding store does not exist in the business partner DB 14, the calorie and nutrient data are registered in the contractor DB 13 by performing processing such as manually inquiring about the calorie and nutrients to the corresponding store, Thereby, the contractor DB 13 is updated.

  On the other hand, if the image recognized by the determination in step St42 is not a receipt (step St42: NO), the process ends.

  Next, the overall operation of this embodiment will be described with reference to FIGS.

  First, before the service is started by the service provider, the contractor accesses and logs in to the web page 19a provided by the web server 19 of the service provider using the home terminal 23 (FIG. 15: Step S1), and the web page 19a. Is displayed (FIG. 15: Step S2), and the health maintenance support system 10 is notified to the health maintenance support system 10 through the operation on the web page 19a (FIG. 15: Step A1).

  Here, as living environment, for example, 1) Self-cooked person living alone (living environment where meals are mainly made by living alone), 2) Lunch purchaser living alone (buying lunch by living alone) Living environment) 3) Dining out alone (living in a restaurant, etc. living alone) 4) Living together or married (whether living with parents or married, and returning home 4 types of living environment where meals are prepared) are set in advance so as to be selectable. In addition, the purpose of use is, for example, 1) health maintenance (take necessary nutrients every day), 2) diet (feel fullness, reduce calorie intake), and 3) dietary restrictions (meal content due to illness) And there is a limit on calorie intake).

  In response to the notification, on the health maintenance support system 10 side, the data server 11 associates the data corresponding to the living environment and usage purpose notified from the contractor with the personal information of the contractor, and the recording medium such as the memory (Not shown).

  When the notification is completed, the pedometer 22 is sent from the service provider to the contractor before starting the service (FIG. 15: step A2). In the pedometer 22, control data such as the transmission timing of the measured step count data is set according to the living environment and usage purpose notified from the contractor. As a result, this service is started in accordance with the notified subscriber's living environment and purpose of use.

  The contractor takes images of breakfast and drink with the camera 21 (FIG. 16: Step S3), and transmits the obtained image data to the data server 11 of the service provider via the communication line 100 (FIG. 16: Step). A3). The data server 11 transfers the received image data to the image recognition processing device 12. The image recognition processing device 12 performs the process shown in FIG. 14 described above, thereby analyzing the ingredients that are the ingredients from the contractor purchased food DB 15 from the received image data, and the calorie corresponding to the corresponding ingredients. Nutrient data is stored in the contractor DB 13 (FIG. 16: Step A4).

  Thereafter, the contractor operates with the pedometer 22 provided by the service provider (FIG. 16: step A5).

When the contractor has lunch, the following processes 1) to 3) are performed (FIG. 17: Step B1).
1) In the case of meals outside the lunch box or business partner company that you bring, take a lunch image with the camera 21 before the meal (FIG. 17: Step S4), and service the obtained image data via the communication line 100. The data is transmitted to the data server 11 of the provider company (FIG. 17: Step A6).

  In response to this, the data server 11 transfers the received image data to the image recognition processing device 12. When the received image data is a lunch box, the image recognition processing device 12 analyzes the “food” used from the contractor-purchased food DB 15 by executing the processing shown in FIG. If the image data is a meal other than the business partner company, the business partner company DB 14 is searched and the corresponding calorie and nutrient data are stored in the contractor DB 13 (FIG. 17: step A7). As a result of the search, if there is no data on the corresponding calorie intake and nutrients, the relevant store is manually inquired of the corresponding calories and nutrients, converted into a database, and stored in the contractor DB 13.

  2) When eating at a convenience store, a lunch box shop, or a restaurant that is a business partner company, a receipt with a food ID is taken by the camera 21, and the obtained image data is provided via the communication line 100 as a service provider. To the data server 11 (FIG. 17: step A8).

  On the other hand, when the data server 11 receives the receipt image data via the communication I / F 18, the data server 11 transfers the received image data to the image recognition processing device 12. The image recognition processing device 12 executes the process shown in FIG. 14 described above, searches the business partner company DB 14 from the image of the receipt to identify the corresponding store and food ID, the calorie corresponding to the identified food ID, The nutrient is determined, and the determined calorie and nutrient data are stored in the contractor DB 13 (FIG. 17: step A7).

  3) When the contractor refers to advice information from the data server 11 of the service provider as a lunch menu, the camera 21 or the home terminal 23 accesses the data server 11 via the communication line 100.

  In response to this, the data server 11 transmits a predetermined service company inquiry screen (see FIG. 27). The camera 21 or the home terminal 23 displays the inquiry screen. On this inquiry screen, when a contractor selects a nearby business partner company from among the business partner companies and presses the “Send” button on the inquiry screen, the selected data is transferred to the data server 11. Send.

  In response to this, the data server 11 determines a recommended meal menu from the history of nutrients consumed by the contractor and the data of the missing nutrients from the selected business partner company and the data of the contractor DB 13, The determined recommended meal menu screen (see the menu screen of FIG. 28) is transmitted to the contractor side (FIG. 18: step A9). The contractor displays the menu screen (FIG. 18: Step S5), and selects whether or not to eat the meal of the recommended meal menu on the screen (FIG. 18: Step B2).

  Here, when the “OK” button at the bottom of the displayed menu screen is pressed, the corresponding menu is ingested, and the contractor DB 13 is updated (FIG. 18: Step A10). On the other hand, by pressing the “Reference” button at the bottom of the displayed menu screen, it is possible to ingest items other than this menu. In this case, when the contractor has lunch (FIG. 17: Step B1), “1. Meals outside the business partner company”, “2. Meals at the business partner company”, and “3. Services Process according to one of the options of “refer to advice from provider”.

  Further, when taking a snack such as a snack other than lunch (FIG. 19: Step B3), images of the snack are taken with the camera 21 sequentially, or an image of the purchase receipt is taken with the camera 21, and the obtained image data is obtained. The data is transmitted to the data server 11 via the communication line 100 (FIG. 19: Step A11). In response to this, the data server 11 transmits the received image data to the image recognition processing device 12. The image recognition processing device 12 analyzes the received image data by executing the processing shown in FIG. 14 described above, and updates the contractor DB 13 as intake calories (FIG. 19: step A12).

  Next, the following processing is performed depending on whether the contractor lives alone or has a family (step B4).

  First, when the contractor lives alone, when approaching within a radius of 1 km from the home after the set time, the pedometer 22 causes the control unit 221 to set, for example, a preset time and position data received by the GPS receiver 224. Based on the above, an interrupt signal is generated, and the step count data measured by the step count measuring unit 222 up to that point is transmitted from the communication unit 223 to the data server 11 of the service provider company via the communication circuit 100 (FIG. 19: step). A13). The data server 11 calculates the calorie consumption from the received step count data by executing the process of FIG. 12 described above, and updates the contractor DB 13 (FIG. 19: step A14).

  Here, in the case of a holiday when the contractor is at home at the set time, the pedometer 13 generates an interrupt signal by measuring a time preset by the control unit 211 with a timer or the like, and the number of steps up to that point The step count data measured by the measuring unit 222 is transmitted to the data server 11 via the communication circuit 100 by the communication unit 223.

  Next, the following processing is performed according to the contractor's living environment and purpose of use (FIG. 20: Step B5).

  1) When the contractor is a self-cooker, he / she instructs the data server 11 of the service provider company via the communication line 100 whether he / she purchases the food by himself / herself or delivers it to his / her home (FIG. 20: Step B6). .

  Here, when purchasing by himself / herself, if the nearest business partner company is designated (FIG. 20: Step A15), the data server 11 searches the contractor DB 13 and lacks the history of nutrients consumed by the contractor. The calculated nutrients are calculated, and the data of the optimal meal recipe and purchased material are transmitted to the contractor side (FIG. 20: Step A16). The contractor displays the meal recipe and purchase material data (FIG. 20: Step S6).

  For the contractor who chooses to deliver to the home, the data server 11 determines an optimal meal menu from the contractor DB 13 (FIG. 21: step A17), and the meal recipe corresponding to the determined meal menu is determined. Data is transmitted to the contractor side (FIG. 21: Step A18). The contractor displays the meal recipe data (FIG. 21: Step S7). In parallel with this, the data server 11 gives an instruction to deliver the food (for one person) according to the determined meal menu to the business partner system 3 via the communication line 100 (FIG. 21: Step A19).

  2) When the contractor is a lunch box purchaser, the data server 11 of the service provider transmits data of a dinner purchase recommended shop (lunch shop or convenience store) to the contractor side (FIG. 22: Step A20). The contractor side displays the data of the recommended dinner purchase store (FIG. 22: Step S8), designates the purchase planned store, and transmits it to the data server 11 (FIG. 22: Step A21).

  In response to this, the data server 11 determines the optimum lunch from the nutrients consumed by the contractor and the insufficient nutrients, and transmits the determined lunch data to the contractor (FIG. 22: step). A22). The contractor displays optimum lunch box data (FIG. 22: Step S9). Furthermore, the contractor orders in advance from the data server 11 as requested (FIG. 22: step B7), or gives a delivery instruction to the home to the business partner system 3 via the communication line 100 (FIG. 22). : Step A23).

  3) When the contractor is a restaurant, the data server 11 of the service provider transmits the data of the restaurant recommendation store to the contractor side (FIG. 23: step A24). The contractor displays the recommended dinner restaurant data (FIG. 23: Step S10), designates a restaurant to be eaten, and transmits it to the data server 11 (FIG. 23: Step A25).

  In response to this, the data server 11 determines the optimum menu from the nutrients consumed by the contractor and the nutrients deficient, and transmits data of the determined menu to the contractor side (FIG. 23: Step A26). ). The contractor displays the optimum new data (FIG. 23: step S11). Furthermore, the contractor orders in advance according to demand (FIG. 23: Step B8), or gives a delivery instruction to the home to the business partner system 3 via the communication line 100 (FIG. 23: Step A27). .

  Next, when the contractor is a married person for a diet purpose or a contractor whose diet is restricted due to illness, the pedometer 22 transmits the measured step count data via the communication line 100 at a preset time. To the data server 11 of the service provider (FIG. 24: step A28).

  In response to this, the data server 11 calculates the calorie consumption from the sent step count data by executing the processing of FIG. 12 described above, and updates the contractor DB 13. In addition, the data server 11 determines the optimum meal menu from the calorie intake and nutrients, and further the calorie consumption from the contractor DB 13 by executing the processing of FIG. 13 described above, and transmits it to the contractor side (FIG. 24: Step A29). The contractor displays the meal menu data that has been sent (FIG. 24: Step A12). The family living together instructs the delivery of food (FIG. 24: step B9).

  Here, if the instruction for delivery of the ingredients is not given by the scheduled time, the data of the quantity of the ingredients according to the meal recipe and the family structure is transmitted to the home terminal 23 (FIG. 24: step A30). When the delivery is instructed, the meal recipe is transmitted to the home terminal 23 (FIG. 25: Step A31), and the delivery instruction of the food is given to the business partner system 3 via the communication line 100 (FIG. 25: Step). A32). The contractor side displays the data of the received meal recipe (FIG. 25: Step S13).

  Next, the data server 11 of the service provider creates a monthly report (see FIG. 29) of the contractor's meal history, the ingested nutrients, and the missing nutrients from the contractor DB 13 and transmits them to the contractor side. (FIG. 26: Step A33). In addition, for contractors who have dietary restrictions due to illness, the meal history is sent to hospital doctors and nutritionists, and advice is given in the prescribed column (see the lower part of Fig. 29) of the monthly report (see Fig. 29). Information is added (FIG. 26: Step B10 and Step A34). Thereby, the contractor can maintain his / her health by constantly monitoring the calories consumed and consumed. Thus, this operation ends.

  As described above, the operation of the entire system to which the health maintenance support system of the present embodiment is applied is summarized as follows.

  1) When you take a picture of your food or receipt with a camera such as a camera-equipped mobile phone, and send the image information to the health maintenance support system via the Internet, Nutrients are calculated.

  2) With this function, dynamic state management and history management of individual intake calories, consumption calories, and nutrients are performed.

  3) An optimal meal menu with no bias in nutrition according to the living environment and dietary restrictions is automatically presented for the calories consumed and consumed.

  4) Taking a receipt of food purchased with a camera-equipped mobile phone, etc., and sending the image information to the system via the Internet, automatically stores the food information in a database and manages the expiration date without causing the food to rot. The optimal meal menu is presented.

  5) By notifying the meal menu, orders are automatically placed at convenience stores, lunch boxes, and restaurants.

  6) By notifying the meal menu, the necessary ingredients are automatically delivered.

  7) When a diet due to illness is necessary, a business tie-up with a medical institution such as a hospital, where the business tie-up analyzes the above calorie, nutrients, and calorie consumption history data to perform a dietary status diagnosis, Accurate dietary guidance information is presented.

  8) It is possible to provide dietary information and services necessary for maintaining the above health.

  The above operations are further summarized as follows.

  1) Calorie intake is calculated from a food image of a meal taken by a camera or a receipt image of purchased food.

  2) The calorie consumption is calculated from the step count data measured by the pedometer.

  3) By comparing the calculated calorie consumption with the calorie intake, support for providing dinner according to the living environment of the contractor's married person, living alone, eating out, etc. is provided. For example, if you are married, send a meal recipe to your home terminal, deliver ingredients, if you live alone, send a recommended menu at a convenience store near your home, etc. Select and place orders in advance.

  Therefore, according to the present embodiment, the contractor can have a calorie consumed and a meal corresponding to the calorie consumed. Furthermore, the service provider can manage the stored food by photographing and transmitting a receipt of the purchased food. Therefore, when the meal created by the contractor is taken as a photograph and transmitted to the service provider, calories and nutrients are automatically calculated. Similarly, when eating out at a business partner company, calories and nutrients are calculated simply by sending a receipt. In addition, by cooperating with the hospital, the dietary intake is monitored by those who have dietary restrictions, and advice can be obtained from the attending physician or dietitian. Furthermore, by outputting a monthly report, the contractor can correctly evaluate the eating habits from the eyes of a third party, and adult diseases can be prevented.

It is a block diagram of the whole system to which the health maintenance support system by the Example of this invention is applied. 1 is a configuration diagram of a health maintenance support system according to an embodiment of the present invention. It is a figure which shows the data structure of contractor DB by the Example of this invention. It is a figure which shows the data structure of business affiliation company DB (convenience store super) by the Example of this invention. It is a figure which shows the data structure of business partner company DB (restaurant) by the Example of this invention. It is a figure which shows the data structure of business partner company DB (lunch box) by the Example of this invention. It is a figure which shows the data structure of business partner company DB (hospital) by the Example of this invention. It is a figure which shows the data structure of business partner company DB (delivery) by the Example of this invention. It is a figure which shows the data structure of recipe DB by the Example of this invention. It is a block diagram of the contractor (user) system by the Example of this invention. It is a block diagram of the business affiliation company system by the Example of this invention. It is a schematic flowchart which shows the process of the data server by the Example of this invention. It is a schematic flowchart which shows the process of the data server by the Example of this invention. It is a schematic flowchart which shows the process of the image recognition processing apparatus by the Example of this invention. It is a schematic flowchart which shows the operation | movement at the time of the contract by the Example of this invention. It is a flowchart which shows the flow of the system operation | movement along the everyday life by the Example of this invention. It is a flowchart which shows the flow of the system operation | movement along the everyday life by the Example of this invention. It is a flowchart which shows the flow of the system operation | movement along the everyday life by the Example of this invention. It is a flowchart which shows the flow of the system operation | movement along the everyday life by the Example of this invention. It is a flowchart which shows the flow of the system operation | movement in the case of the self-cooking person living alone by the Example of this invention. It is a flowchart which shows the flow of the system operation | movement in the case of the self-cooking person living alone by the Example of this invention. It is a flowchart which shows the flow of the system operation | movement in the case of the lunch box purchaser of living alone by the Example of this invention. It is a flowchart which shows the flow of a system operation | movement in the case of the restaurant person who lives alone by the Example of this invention. It is a flowchart which shows the flow of the system operation | movement in the case of the family member by the Example of this invention. It is a flowchart which shows the flow of the system operation | movement in the case of the family member by the Example of this invention. It is a flowchart which shows the flow of a system operation | movement in the case of performing the monthly process by the Example of this invention. It is a figure which shows the service provider inquiry screen by the Example of this invention. It is a figure which shows the recommended menu display screen by the Example of this invention. It is a figure which shows the monthly report screen by the Example of this invention.

Explanation of symbols

1 Service Provider System 2 Contractor System 3 Business Partner System 10 Health Maintenance Support System 11 Data Server 12 Image Recognition Processing Device 21 Camera 22 Pedometer 23 Home Terminal

Claims (4)

A health maintenance support system having an information processing system and a user system connected to the information processing system via a communication line,
Before Symbol user system,
The user captures an image of the receipt of product identification information of the shop shop that Ru is passed at the time of cooking or food purchase of the meal was eating and drinking is attached, to send the captured image to the information processing system via the communication line Image transmission means;
Said measuring the number of steps of the user, the measured count data through the pre SL communication line and a step count data transmitting means for transmitting to the information processing system,
The information processing system includes:
For each user, data registration date and time, food, calorie intake, calorie consumption, and nutrient data are registered in association with each other and can be updated, and the nutrient data is abundant for each nutrient, sufficient, small, A contractor database that can be processed in association with either missing or not included information,
For each store, register the food calorie and nutrient data in an updatable manner in association with the product identification information, and the nutrient data is abundant, sufficient, low, or insufficient for each nutrient. Business partner database that can be processed in association with any of the information that is not,
A contractor-purchased food database that registers the data of food purchased by the user in an updatable manner in association with the product identification information;
For each food, a calorie database that registers and registers nutrient and calorie data,
For each meal menu, the constituent recipes and meal recipes indicating the amount thereof, cooking images, calories, and nutrient data are registered in association with each other and can be updated, and the nutrient data is sufficient for each nutrient. A meal recipe database that can be processed in association with one of the few, missing, or not included information;
An image recognition processing device that analyzes the image sent by the image transmission means and recognizes whether the image is the image of the dish or the image of the receipt by image processing based on an image recognition algorithm;
When the image is the image of the dish, the dish image in the meal recipe database is searched, the meal menu corresponding to the searched processing image is recognized, and the constituent ingredients corresponding to the recognized meal menu are determined and determined. Comparing the configured component and the food in the contractor purchase food database, determining whether or not the corresponding component is present, and if the corresponding component is present, the dish created by the user By comparing the contractor purchase food DB with the meal recipe, and determining the ingredients used in cooking and the amount thereof, the intake calorie and nutrient data are obtained, and the obtained intake calorie and nutrient are obtained. The contractor database is updated with the data of the above, and if the corresponding constituent food does not exist, it is determined that the meal is a restaurant and the dish image in the meal recipe database Updating the subscriber database by a corresponding calorie intake and nutrient data,
When the image is the image of the receipt, the business partner company database is searched from the receipt image to identify the corresponding store and its product identification information, and the calorie intake and nutrients corresponding to the identified product identification information Update the contractor database with the calorie intake and nutrient data obtained,
Calculate calorie consumption using a calorie consumption calculation formula based on personal data including a user's step length and weight set in advance based on the step count data sent by the step count data transmission means, and the calculated calorie consumption Update the contractor database with data,
Using the contractor database, the calorie intake of each meal taken for each user and the information that is rich, sufficient, low, insufficient, or not included for each nutrient of the meal A health maintenance support system , comprising: a data server that manages a history of nutrient data associated with each other and calorie consumption data calculated from the step count data . The user captures an image of the receipt of product identification information of the shop shop that Ru is passed at the time of cooking or food purchase of the meal was eating and drinking is attached, to send the captured image to the information processing system via a communication line an image transmission means, the number of steps of the user is measured, via a communication line to a user system and a step count data transmitting means for transmitting to the information processing system through a pre SL communication line number of steps data measured A connected information processing system,
For each user, data registration date and time, food, calorie intake, calorie consumption, and nutrient data are registered in association with each other and can be updated, and the nutrient data is abundant for each nutrient, sufficient, small, A contractor database that can be processed in association with either missing or not included information,
For each store, register the food calorie and nutrient data in an updatable manner in association with the product identification information, and the nutrient data is abundant, sufficient, low, or insufficient for each nutrient. Business partner database that can be processed in association with any of the information that is not,
A contractor-purchased food database that registers the data of food purchased by the user in an updatable manner in association with the product identification information;
For each food, a calorie database that registers and registers nutrient and calorie data,
For each meal menu, the constituent recipes and meal recipes indicating the amount thereof, cooking images, calories, and nutrient data are registered in association with each other and can be updated, and the nutrient data is sufficient for each nutrient. A meal recipe database that can be processed in association with one of the few, missing, or not included information;
An image recognition processing device that analyzes the image sent by the image transmission means and recognizes whether the image is the image of the dish or the image of the receipt by image processing based on an image recognition algorithm;
When the image is the image of the dish, the dish image in the meal recipe database is searched, the meal menu corresponding to the searched processing image is recognized, and the constituent ingredients corresponding to the recognized meal menu are determined and determined. Comparing the configured component and the food in the contractor purchase food database, determining whether or not the corresponding component is present, and if the corresponding component is present, the dish created by the user By comparing the contractor purchase food DB with the meal recipe, and determining the ingredients used in cooking and the amount thereof, the intake calorie and nutrient data are obtained, and the obtained intake calorie and nutrient are obtained. The contractor database is updated with the data of the above, and if the corresponding constituent food does not exist, it is determined that the meal is a restaurant and the dish image in the meal recipe database Updating the subscriber database by a corresponding calorie intake and nutrient data,
When the image is the image of the receipt, the business partner company database is searched from the receipt image to identify the corresponding store and its product identification information, and the calorie intake and nutrients corresponding to the identified product identification information Update the contractor database with the calorie intake and nutrient data obtained,
Calculate calorie consumption using a calorie consumption calculation formula based on personal data including a user's step length and weight set in advance based on the step count data sent by the step count data transmission means, and the calculated calorie consumption Update the contractor database with data,
Using the contractor database, the calorie intake of each meal taken for each user and the information that is rich, sufficient, low, insufficient, or not included for each nutrient of the meal An information processing system comprising: a data server that manages a history of nutrient data associated with the data and calorie consumption data calculated from the step count data.   Image transmitting means for capturing an image of a receipt attached with merchandise identification information of a store that is passed at the time of purchase of food or food that a user eats or drinks, and transmits the captured image to the information processing system via a communication line And step number data transmitting means for measuring the number of steps of the user and transmitting the measured step number data to the information processing system via the communication line. A processing system operating method comprising:
  The information processing system includes:
  For each user, data registration date and time, food, calorie intake, calorie consumption, and nutrient data are registered in association with each other and can be updated, and the nutrient data is abundant and sufficient for each nutrient. A contractor database that can be processed in association with either missing or not included information,
  For each store, register the food calorie and nutrient data in an updatable manner in association with the product identification information, and the nutrient data is abundant, sufficient, low, or insufficient for each nutrient. Business partner database that can be processed in association with any of the information that is not,
  A contractor-purchased food database that registers the data of food purchased by the user in an updatable manner in association with the product identification information;
  For each food, a calorie database that registers and registers nutrient and calorie data,
  For each meal menu, the constituent recipes and meal recipes indicating the amount thereof, cooking images, calories, and nutrient data are registered in association with each other and can be updated, and the nutrient data is sufficient for each nutrient. A meal recipe database that can be processed in association with one of the few, missing, or not included information;
  An image recognition processing device;
  A data server connected to the image recognition processing device,
  The image recognition processing device analyzes the image sent by the image transmitting means, recognizes whether the image is the dish image or the receipt image by image processing based on an image recognition algorithm,
  The data server is
  When the image is the image of the dish, the dish image in the meal recipe database is searched, the meal menu corresponding to the searched processing image is recognized, and the constituent ingredients corresponding to the recognized meal menu are determined and determined. Comparing the configured component and the food in the contractor purchase food database, determining whether or not the corresponding component is present, and if the corresponding component is present, the dish created by the user By comparing the contractor purchase food DB with the meal recipe, and determining the ingredients used in cooking and the amount thereof, the intake calorie and nutrient data are obtained, and the obtained intake calorie and nutrient are obtained. The contractor database is updated with the data of the above, and if the corresponding constituent food does not exist, it is determined that the meal is a restaurant and the dish image in the meal recipe database Updating the subscriber database by a corresponding calorie intake and nutrient data,
  When the image is the image of the receipt, the business partner company database is searched from the receipt image to identify the corresponding store and its product identification information, and the calorie intake and nutrients corresponding to the identified product identification information Update the contractor database with the calorie intake and nutrient data obtained,
  Calculate calorie consumption using a calorie consumption calculation formula based on personal data including a user's step length and weight set in advance based on the step count data sent by the step count data transmission means, and the calculated calorie consumption Update the contractor database with data,
  Using the contractor database, the calorie intake of each meal taken for each user and the information that is rich, sufficient, low, insufficient, or not included for each nutrient of the meal A method of operating an information processing system, characterized by managing a history of nutrient data associated with the data and calorie consumption data calculated from the step count data.   Image transmission that captures an image of a receipt attached with merchandise identification information of a store that is passed at the time of food purchase or food purchase by a user, and transmits the captured image to the information processing system via the communication line And a step number data transmission means for measuring the step count of the user and transmitting the measured step count data to the information processing system via the communication line. An operation program for an information processing system,
  Computer
  For each user, data registration date and time, food, calorie intake, calorie consumption, and nutrient data are registered in association with each other and can be updated, and the nutrient data is abundant for each nutrient, sufficient, small, A contractor database that can be processed in association with either missing or not included information,
  For each store, register the food calorie and nutrient data in an updatable manner in association with the product identification information, and the nutrient data is abundant, sufficient, low, or insufficient for each nutrient. Business partner database that can be processed in association with any of the information that is not,
  A contractor-purchased food database that registers the data of food purchased by the user in an updatable manner in association with the product identification information;
  For each food, a calorie database that registers and registers nutrient and calorie data,
  For each meal menu, the constituent recipes and meal recipes indicating the amount thereof, cooking images, calories, and nutrient data are registered in association with each other and can be updated, and the nutrient data is sufficient for each nutrient. A meal recipe database that can be processed in association with one of the few, missing, or not included information;
  An image recognition processing device that analyzes the image sent by the image transmission means and recognizes whether the image is the image of the dish or the image of the receipt by image processing based on an image recognition algorithm;
  When the image is the image of the dish, the dish image in the meal recipe database is searched, the meal menu corresponding to the searched processing image is recognized, and the constituent ingredients corresponding to the recognized meal menu are determined and determined. Comparing the configured component and the food in the contractor purchase food database, determining whether or not the corresponding component is present, and if the corresponding component is present, the dish created by the user By comparing the contractor purchase food DB with the meal recipe, and determining the ingredients used in cooking and the amount thereof, the intake calorie and nutrient data are obtained, and the obtained intake calorie and nutrient are obtained. The contractor database is updated with the data of the above, and if the corresponding constituent food does not exist, it is determined that the meal is a restaurant and the dish image in the meal recipe database Updating the subscriber database by a corresponding calorie intake and nutrient data,
  When the image is the image of the receipt, the business partner company database is searched from the receipt image to identify the corresponding store and its product identification information, and the calorie intake and nutrients corresponding to the identified product identification information Update the contractor database with the calorie intake and nutrient data obtained,
  Calculate calorie consumption using a calorie consumption calculation formula based on personal data including a user's step length and weight set in advance based on the step count data sent by the step count data transmission means, and the calculated calorie consumption Update the contractor database with data,
  Using the contractor database, the calorie intake of each meal taken for each user and the information that is rich, sufficient, low, insufficient, or not included for each nutrient of the meal The operation program for functioning as an information processing system having a data server that manages the history of nutrient data associated with the data and calorie consumption data calculated from the step count data.

Images