WO2013161879A1 - Information processing device and program - Google Patents

Information processing device and program Download PDF

Info

Publication number
WO2013161879A1
WO2013161879A1 PCT/JP2013/062079 JP2013062079W WO2013161879A1 WO 2013161879 A1 WO2013161879 A1 WO 2013161879A1 JP 2013062079 W JP2013062079 W JP 2013062079W WO 2013161879 A1 WO2013161879 A1 WO 2013161879A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
food
drink
unit
user
Prior art date
Application number
PCT/JP2013/062079
Other languages
French (fr)
Japanese (ja)
Inventor
清晴 相澤
小川　誠
Original Assignee
国立大学法人 東京大学
ｆｏｏ．ｌｏｇ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 国立大学法人 東京大学, ｆｏｏ．ｌｏｇ株式会社 filed Critical 国立大学法人 東京大学
Publication of WO2013161879A1 publication Critical patent/WO2013161879A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/12—Hotels or restaurants

Definitions

  • This invention relates to the information processing apparatus and program which perform the process which concerns on the information of food and drink.
  • Patent Document 1 The idea of identifying the type (such as croquette) and amount of food from an image and calculating the calorie value of the identified amount of the food of that type is disclosed in Patent Document 1.
  • Patent Document 2 discloses a technique for searching for similar items from past meal contents in order to support the work of the worker when the worker determines the calorie value from the captured image.
  • set meal is prepared for lunch, and it is common to order in this unit.
  • the technology for identifying the type and amount of food from the image identifies the type and amount of each food each time, even though there is a record when eating and drinking in the same set in the past. The efficiency is low.
  • the present invention has been made in view of the above circumstances, and an object thereof is to provide an information processing apparatus and program capable of efficiently inputting information on food and drink.
  • This invention which solves the problem of the above-mentioned conventional example is an information processing device, and is obtained when eating and drinking information in the past when a set of food and drink containing at least one kind of food and drink is set.
  • a receiving means for receiving information of a type, a searching means for searching at least one set reference information held in the holding means for eating and drinking information similar to the received information, and a set found by the search Presenting means for presenting set identification information related to the reference information, prompting the user to select, and set reference information selected by the user from the set identification information presented by the presenting means Including the acquisition means for acquiring the setting value information included in the read setting information and the output means for outputting the acquired setting value information. It is.
  • maintenance means here the separate reference information which linked
  • the set reference information held and held in the holding means is further held in association with individual identification information for identifying the type of food or drink contained in the set, and the receiving means is predetermined.
  • the information of food and drink images is received as the type of information, and the search means searches the holding means for a plurality of image information similar to each type of food and drink image included in the received image information, Obtaining individual identification information associated with each of the plurality of image information found by the search, searching for set reference information including the obtained plurality of individual identification information, May output the result of the search.
  • the information processing apparatus further includes means for acquiring information for identifying a user, the holding means further holds, as setting information, information related to the amount of food and drink for each user, and the output means includes the acquired information. It is good also as correcting the information on the nutritional value in the said setting value using the information regarding the amount of food and drink of the acquired setting value concerning the identified user, and outputting the information on the corrected nutritional value.
  • the program according to one aspect of the present invention includes a computerized information at the time of eating and drinking acquired when a set of food and drink including at least one kind of food and drink in the past, and set identification information for identifying the set.
  • Holding reference information that associates the set identification information and setting information that associates the set identification information and the setting value that includes nutrition value information, and accepting information of a predetermined type when eating or drinking food or drink Means for searching for at least one set reference information held in the holding means for information on eating and drinking similar to the received information, and set identification relating to the set reference information found by the search
  • Presenting means for presenting information and prompting the user to select, and set reference information selected by the user from the set identification information presented by the presenting means
  • the setting information is read out from the holding means, and functions as an acquisition means for acquiring the setting value information included in the read setting information, and an output means for outputting the acquired setting value information. It is.
  • the contents of the meal can be easily specified, and the efficiency of information input can be improved.
  • the information processing apparatus 1 includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, an imaging unit 15, and a current position acquisition. Part 16.
  • the control unit 11 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12. This control part 11 receives the information of a predetermined kind by operation which the user did when eating and drinking food. Further, the control unit 11 searches for at least one of the set reference information (described later) held in the storage unit 12 for information on eating and drinking similar to the received information. Then, the control unit 11 presents the set identification information related to the set reference information found by the search, prompts the user to select, and when the user selects the set identification information, the setting information related to the set identification information Is read out from the storage unit 12, and the setting value information included in the read setting information is acquired and output. The processing of the control unit 11 will be described in detail later.
  • the storage unit 12 is a storage device including an SSD (Solid State Disk), flash memory, RAM (Random Access Memory), and the like, and functions as a holding unit in the present invention.
  • the storage unit 12 holds a program executed by the control unit 11. This program may be provided by being stored in a computer-readable recording medium such as a DVD-ROM, or may be distributed via a network and stored in the storage unit 12.
  • the storage unit 12 also operates as a work memory for the control unit 11. Furthermore, in this Embodiment, this memory
  • storage part 12 is the information (A) at the time of eating and drinking acquired when eating and drinking the set of the food and drink containing at least 1 type of food and drink in the past so that it may illustrate in FIG. , Set reference information associated with set identification information (B) for identifying the set, and set value (C) including the set identification information (B) and nutrition value information as illustrated in FIG.
  • the setting information associated with is stored.
  • the set identification information (B) may be a name of a set of food and drink (for example, “A set meal”) input by the user.
  • set reference information In addition, in the following description, these information are referred to as set reference information, set identification information, etc., but it is a name that can include information on the food and drink that has become a set, and information on a single food and drink is included. It may be included in the set reference information. In this case, information for specifying a single item of food and drink is associated with the information at the time of eating and drinking such as an image of the single item of food and drink as set identification information.
  • the eating and drinking information included in the set reference information includes image information of food and drink taken by the user at the time of eating and drinking (image information including at least a part of food and drink in the set), information indicating the date and time of eating and drinking, Information indicating the place where the restaurant was eaten (latitude / longitude information, store name, etc., or information arbitrarily entered by the user, such as information related to the event, etc.
  • image information must be included. Then, information such as the date and time of eating and drinking, and the place may be recorded as a tag related to the image information (for example, Exif (Exchangeable image file format) information).
  • the operation unit 13 is, for example, a touch panel installed so as to overlap the display unit 14, detects that the user's finger is in contact, and outputs information indicating the contact position of the finger to the control unit 11.
  • the display unit 14 is a display device such as a liquid crystal or an organic EL, and displays information such as an image in accordance with an instruction input from the control unit 11.
  • the imaging unit 15 includes an imaging element such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor, and in accordance with instructions input from the control unit 11, image information acquired by these image sensors. Take in and output to the control unit 11.
  • an imaging element such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor
  • the current position acquisition unit 16 includes, for example, a GPS (Global Positioning System) device, measures the current position in accordance with an instruction input from the control unit 11, and represents information (latitude, longitude, etc.) representing the current position obtained by the measurement. Information) is output to the control unit 11.
  • the current position acquisition unit 16 may acquire the current position using, for example, a receivable wireless LAN signal instead of the GPS, or may use the information on a mobile phone base station capable of communication to acquire the current position. It is good also as acquiring a position.
  • the control unit 11 operates according to a program stored in the storage unit 12 and functionally operates as having the configuration illustrated in FIG. That is, the control unit 11 functionally includes an information receiving unit 21, a search processing unit 22, a presentation unit 23, a setting information acquisition unit 24, an information input unit 25, and an information recording as illustrated in FIG. Part 26.
  • the information accepting unit 21 realizes accepting means for accepting a predetermined type of information when eating or drinking food.
  • the information receiving unit 21 may instruct the imaging unit 15 to acquire image information as a predetermined type of information.
  • the user performs an operation of capturing image information of food and drink using the image capturing unit 15, and the image capturing unit 15 outputs image information obtained by the user's operation to the information receiving unit 21.
  • the information receiving unit 21 instructs the current position acquisition unit 16 to acquire the current position, receives the information on the current position output from the current position acquisition unit 16, and converts the current position information into the image information acquired from the imaging unit 15.
  • the current position information may be associated and held in the storage unit 12 as a part of the Exif information.
  • the information receiving unit 21 acquires information indicating the date and time when the image capturing unit 15 performs image capturing from a calendar chip or the like (not shown), and associates it with the image information acquired from the image capturing unit 15 as part of the Exif information. You may store in the memory
  • the search processing unit 22 operates when the information receiving unit 21 accepts information, and among the set reference information held in the storage unit 12 serving as holding means, information on eating and drinking similar to the information received by the information receiving unit 21 A search means for searching from at least one is realized.
  • the search processing unit 22 is accepted by the information receiving unit 21.
  • a predetermined feature amount is calculated from the image information.
  • This feature amount may be a well-known feature amount for image comparison, for example, a SIFT (Scale Invariant Feature Transform) feature amount.
  • the search processing unit 22 sequentially selects the set reference information stored in the storage unit 12 as a comparison target, and acquires a SIFT feature amount related to the image information of food and drink included in the comparison target (calculated and set in advance). It may be recorded by being included in the reference information, or may be obtained by calculation here). Then, the SIFT feature value calculated from the image information included in the set reference information selected as the comparison target is compared with the SIFT feature value calculated from the image information received by the information receiving unit 21.
  • the SIFT feature amount includes elements that associate key points with the feature amount of each key point by the number of key points found from the image. Therefore, in the comparison of SIFT feature values, for example, the SIFT feature value calculated from the image information included in the set reference information selected as the comparison target and the information receiving unit 21 depend on how many key points are determined to match the feature values. Is included in the set reference information selected as the comparison target when the number of keypoints determined to match the SIFT feature value calculated from the image information received from the image information exceeds a predetermined threshold value What is necessary is just to judge that image information and the image information which the information reception part 21 received are similar.
  • search processing unit 22 finds at least one set reference information including image information similar to the image information received by the information receiving unit 21, information for identifying the found set reference information (may be set identification information). Output to the presentation unit 23.
  • the search processing unit 22 inputs information to the information input unit 25 when the set reference information including image information similar to the image information received by the information receiving unit 21 cannot be found. Instruct to accept.
  • the presentation unit 23 presents set identification information related to the set reference information found by the search processing unit 22 and realizes a presentation unit that prompts the user to select. Specifically, the presenting unit 23 displays a list of information specifying the set reference information input from the search processing unit 22 on the display unit 14. For example, as shown in FIG. 5, this display reads out image information (G) and set identification information (B) from the set reference information specified by the information input from the search processing unit 22, and performs layout. Is displayed. Further, the setting value information associated with the set identification information related to the set reference information specified by the information input from the search processing unit 22 is further read from the storage unit 12, and the setting value information (C) is also combined. It may be displayed. Then, the presentation unit 23 causes the user to select one of the set reference information from the list.
  • the presenting unit 23 instructs the information input unit 25 to accept the input of information.
  • the setting information acquisition unit 24 reads setting information including set identification information related to the set reference information selected by the user from the storage unit 12 as a holding unit in the process of the presentation unit 23. Then, the setting value information included in the read setting information is acquired and output. That is, the setting information acquisition unit 24 implements acquisition means and output means.
  • the information input unit 25 requests the user to input set identification information when an instruction to accept the input of information is input.
  • the information input unit 25 outputs the input information to the information recording unit 26.
  • the information recording unit 26 includes image information (including an Exif tag) received by the information receiving unit 21, set identification information included in the set reference information selected by the user in the presentation unit 23, and a setting information acquisition unit 24. When the setting value information to be output is received, these pieces of information are associated with each other and stored in the storage unit 12. In addition, when the image information (including the Exif tag) received by the information receiving unit 21 and the set identification information received by the information input unit 25 are input, the information recording unit 26 associates these pieces of information with each other, Store in the storage unit 12. Information recorded by the information recording unit 26 is set reference information.
  • the information processing apparatus 1 has the above configuration, and operates as follows. It is assumed that the user has photographed image information of food and drink that has been eaten and consumed in the past, and has input set identification information. As an example, here, information relating the set identification information “A set meal” to the image information a and information relating the set identification information “B set” to the image information b as set reference information in the storage unit 12. And are held. Further, it is assumed that nutrition value information “ ⁇ Cal” is associated with set identification information “A set meal”, and nutrition value information “ ⁇ Cal” is associated with “B set”.
  • the control unit 11 starts the processing illustrated in FIG. 6.
  • the SIFT feature value is calculated from the image information obtained in this way (S1).
  • the control unit 11 selects one of the unselected set reference information among the set reference information stored in the storage unit 12 as a comparison target (S2).
  • the SIFT feature-value regarding the image information of the food and drink contained in the said comparison object is acquired (S3).
  • the SIFT feature value is calculated in advance and included in the set reference information and recorded, the SIFT feature value included in the selected set reference information is acquired. If this is not the case, the control unit 11 calculates and calculates here.
  • the control unit 11 compares the SIFT feature quantity obtained in the process S1 with the SIFT feature quantity obtained in the process S3, and the image information obtained by imaging and the image information of the food and drink included in the comparison target are obtained. It is determined whether or not they are similar (S4). Although various methods can be considered for comparison here, SIFT feature values respectively selected from the SIFT feature value obtained in the process S1 and the SIFT feature value obtained in the process S3, and the mutual distance (SIFT feature value). When the quantity is a vector quantity, the Euclidean distance may be used).
  • the manner in which each dish is arranged on the tray like a set meal may be different each time.
  • the SIFT feature amount has a feature that it is invariant to rotation, a similar image can be searched even when the direction of the dish is different.
  • control unit 11 determines that the image information obtained by imaging and the image information of the food and drink included in the comparison target are similar in the comparison of the process S4 (Yes), the control unit 11 determines that the image information is similar.
  • Information for specifying the set reference information to be compared is recorded in the storage unit 12 as one of the candidates in the selection candidate list (S5).
  • the control unit 11 determines whether or not there is any set reference information stored in the storage unit 12 that is not selected as a comparison target (S6). If so, the process returns to step S2 to select the unselected set reference information.
  • step S6 the control unit 11 selects the selection candidate list stored in the storage unit 12 if there is no set reference information stored in the storage unit 12 that is not selected as a comparison target (if No). Whether there is at least one piece of information specifying the set reference information is checked (S7). If the selection candidate list includes at least one piece of information for specifying the set reference information (if Yes), the control unit 11 displays and outputs a list based on the set reference information included in the selection candidate list. Prompts the user to select (S8). This display is as illustrated in FIG.
  • the control unit 11 accepts information (for example, set identification information) specifying the set reference information selected from the list by the user (S9), and reads setting information including the set identification information from the storage unit 12. Then, the control unit 11 acquires the information of the setting value included in the read setting information (S10), the image information obtained by imaging, the set identification information received in the process S9, and the setting acquired in the process S10. The value information is stored in association with the storage unit 12 (S11), and the process is terminated.
  • information for example, set identification information
  • S9 set identification information
  • setting information including the set identification information from the storage unit 12. Then, the control unit 11 acquires the information of the setting value included in the read setting information (S10), the image information obtained by imaging, the set identification information received in the process S9, and the setting acquired in the process S10.
  • the value information is stored in association with the storage unit 12 (S11), and the process is terminated.
  • control part 11 judges that the image information obtained by image-capturing in process S4 and the image information of the food and drink contained in a comparison object are not similar (it is No), it will transfer to process S6. And continue processing.
  • step S7 if there is no information specifying the set reference information in the selection candidate list (if No), the control unit 11 obtains set identification information from the user (S12). At this time, the control unit 11 may also obtain the input of setting information.
  • control unit 11 associates the image information obtained by imaging with the input set identification information and stores it in the storage unit 12 (S13). The process is terminated.
  • control unit 11 uses the image information obtained by imaging as eating / drinking information, and stores it in the storage unit 12 as set reference information in association with the input set identification information ( That is, the set reference information may be updated).
  • control unit 11 When the control unit 11 displays a list based on the set reference information included in the selection candidate list in step S8, the control unit 11 adds an instruction to “newly input information” (that is, not in the list) to the selection candidate. May be displayed. When the user selects this instruction in process S8, the control unit 11 may proceed to process S12 and continue the process.
  • control unit 11 is subjected to browsing processing according to an instruction from the user. Specifically, the control unit 11 reads the set reference information stored in the storage unit 12 in accordance with a browsing instruction input from the user. A list of image information and the like included in the set reference information is displayed. In addition, the control unit 11 may refer to the set identification information included in the set reference information, read out nutrition value information associated with the referenced set identification information, and display the information in a list. Further, this list may be displayed separated by shooting date with reference to Exif information of image information, for example.
  • the search processing unit 22 may divide and compare the image information to be compared for each type of food and drink included in the set. Specifically, the image information to be compared may be divided for each dish, and the image information may be compared for each dish.
  • the set reference information in this case is an entry (hereinafter, for distinction) in which image information divided for each dish is associated with set identification information (hereinafter referred to as individual identification information for distinction) that specifies food and drink for each dish. And an entry including a set of individual identification information for specifying a plurality of foods and beverages (hereinafter referred to as an identification information entry for distinction).
  • this division can be realized, for example, when the control unit 11 receives a designation for each of the areas where each dish is imaged from the user and performs the following processing. That is, the control unit 11 displays the captured image information on the display unit 14, and accepts a single tap from the operation unit 13 in the image portion to be extracted on the displayed image information. The control unit 11 acquires coordinate information on the image information corresponding to the tapped position and records it as a processing target coordinate information list. This operation is repeated until the user instructs the end of the tap. In other words, the user designates one point inside the area in which the dish is shown for each dish, and when the designation of all the dish areas is completed, the user inputs that point.
  • the control unit 11 refers to the coordinate information included in the processing target coordinate information list, and sets the point for each point received by the user's instruction operation.
  • a region having a predetermined shape is formed in the image information.
  • the predetermined shape is, for example, a circle.
  • each region has a size not larger than the size in contact with each other without overlapping.
  • the control unit 11 performs a segmentation process using a program such as a grab cut with the formed area as an initial area representing the foreground. Then, out of the image information obtained by the segmentation process, a partial image serving as a foreground is extracted as a processing target.
  • a program such as a grab cut with the formed area as an initial area representing the foreground.
  • control unit 11 further includes coordinates represented by the coordinate information Pi in a circular area (referred to as an outer peripheral area for distinction) determined for the coordinate information Pi included in the processing target coordinate information list.
  • a small region including is further formed.
  • the small region may have a shape similar to the shape of the determined outer peripheral region and have a predetermined size.
  • the small area determined by the predetermined size protrudes from the determined outer peripheral area, the area where the outer peripheral area and the small area overlap (to be determined by their logical product) is reduced. This is an area.
  • the control unit 11 extracts the image information of each dish, and individually references the set reference information that has been imaged in the past and divided and recorded for each dish. Search from image information included in the information. Then, for the image information for each dish imaged this time, the image information determined to be the most similar among the image information for each dish imaged in the past is searched.
  • This search can be performed by adopting various well-known image search methods such as a process using a histogram and a process using a SIFT feature amount.
  • control unit 11 retrieves the image information for each dish captured this time, and identifies each of the image information determined to be the most similar among the image information for each dish imaged in the past. Get information. Thereby, the individual identification information is obtained based on the image information recorded in the past for the image information for each dish imaged this time.
  • the control unit 11 searches the storage unit 12 for the set reference information of the identification information entry including the individual identification information that most closely matches the set of individual identification information obtained for the image information for each dish imaged this time.
  • the user inputs information specifying food and drink shown in each image information for the image information of each dish, and Enter the name of the menu for the entire set of objects.
  • the user images a set of curry, salad, and drink, and causes the information processing apparatus 1 to separate the curry dish, salad dish, and drink container image information.
  • the name of the dish of each plate (or container) is input as individual identification information.
  • the information processing apparatus 1 associates the individual reference information that associates the individual identification information of the name “curry” with the image information of the curry dish and the individual identification information of the name “salad” with the image information of the salad dish.
  • the individual reference information and the individual reference information in which the drink name (such as “ice coffee”) is associated with the image information of the drink container as individual identification information are recorded.
  • set identification information of “Curry A set” is accepted as the name of the set including these dishes, and individual identification information of “Curry”, “Salad”, and “Ice coffee” is received for this set identification information.
  • the identification information entry of the associated set reference information is recorded. Further, it is assumed that an identification information entry including individual identification information of “meat sauce spaghetti”, “salad” and “ice coffee” is recorded as “pasta B set” which is another identification information entry.
  • the information processing apparatus 1 cuts the image information for each plate (or container) and is most similar to the image information of the cut result.
  • the image information to be searched is retrieved from the storage unit 12.
  • the image information showing the past “curry” dish, the image information showing the salad dish, and the image information showing the container containing “ice coffee” are searched.
  • Each image information in this case does not necessarily have to be image information at the time of ordering a past curry A set).
  • the information processing apparatus 1 obtains a set of individual identification information of “curry”, “salad”, and “ice coffee” from this search result. Then, the identification information entry of the set reference information that best matches (includes most) the set of individual identification information is searched.
  • the identification information entry of “pasta B set” matches two items of “salad” and “ice coffee” (number of matched foods “2”), and “curry”
  • the information processing apparatus 1 refers to the identification information entry having the largest number of matched foods and drinks, and determines that the image information captured this time is the image information of “curry A set”.
  • the setting information acquisition unit 24 reads the setting information from the storage unit 12, when the setting information that is set in advance by the user and includes information related to the quantity is included in the setting information, It is good also as correcting the nutrition value information in the setting value contained in the acquired setting information using the information concerning the amount, and outputting the corrected nutrition value information.
  • the information relating to this amount sets how much the user eats or drinks the food and drink identified by the set identification information with respect to the average amount. As an example, 0.7 is set when eating and drinking only about 70% at all times, and 2.0 is set when eating and drinking every time. And when the information regarding this quantity is contained in the acquired setting information, the setting information acquisition part 24 multiplies the information regarding this quantity to the information of the nutrition value contained in the said setting information, and corrects the nutrition after the said correction
  • FIG. 7 is a configuration block diagram illustrating an example in which the information processing apparatus is implemented as a server apparatus.
  • the information processing apparatus 1 ′ as a server apparatus includes a control unit 11 ′, a storage unit 12, and a communication unit 17. Components having the same configurations as those already described are denoted by the same reference numerals, and detailed description thereof is omitted.
  • a terminal device 2 is connected to the information processing apparatus 1 'via a communication means such as a network.
  • the terminal device 2 is, for example, a device such as a smartphone, and communicates with an information processing device 1 ′ via imaging means such as a camera, devices such as a touch panel and a liquid crystal display unit, and a cellular phone communication network or network. It is comprised including the communication device which performs.
  • control unit 11 ′ functionally receives an information receiving unit 21 ′, a search processing unit 22, and a presentation as illustrated in FIG. 8 according to a program stored in the storage unit 12.
  • a device including a unit 23 ′, a setting information acquisition unit 24, an information input unit 25 ′, and an information recording unit 26 is realized. Also in this example, components having the same configuration as those already described are assigned the same reference numerals and detailed description thereof is omitted.
  • the information receiving unit 21 ′ authenticates the user. This authentication can be performed by, for example, a general method in which a user name and password set in advance by the user are input.
  • the information receiving unit 21 ′ functions as a receiving unit that receives a predetermined type of information input by the user at the time of eating and drinking in the terminal device 2 of the authenticated user.
  • the information receiving unit 21 ′ may receive, for example, image information captured by a camera of the terminal device 2 as information of a predetermined type. In this case, the terminal apparatus 2 transmits information on the current position of the terminal apparatus 2 and information on the shooting date and time to the information processing apparatus 1 ′ as a part of the Exif information added to the image information.
  • the information receiving unit 21 ′ may receive these pieces of information together with image information.
  • the search processing unit 22 operates when the information receiving unit 21 ′ accepts information, and sets reference information held in the storage unit 12 as holding means that is similar to information received by the information receiving unit 21 ′.
  • a search means for searching for at least one of the above is realized.
  • the search processing unit 22 is accepted by the information receiving unit 21 ′.
  • Set reference information including image information similar to the image information is searched from the storage unit 12. This similar image information search may be performed by using a SIFT feature amount as described above.
  • a method similar to that described in the example of the information processing apparatus 1 ′ that determines food and drink for each dish and searches for the combination can be employed. In this case, on the captured image, when accepting designation of one point in the image portion where each dish is imaged, the terminal device 2 is presented with the captured image information, and the designation of each point is accepted. What is necessary is just to make it transmit the information of each received point.
  • the control unit 11 ′ can perform segmentation processing using the transmitted information on each point.
  • the presentation unit 23 ′ presents the set identification information related to the set reference information found by the search processing unit 22 and realizes a presentation unit that prompts the user to select. Specifically, the presentation unit 23 ′ generates a list of information specifying set reference information input from the search processing unit 22, sends the list to the terminal device 2, and causes the terminal device 2 to display the list.
  • the presentation unit 23 ′ further reads information on the set value associated with the set identification information related to the set reference information specified by the information input from the search processing unit 22 from the storage unit 12. ) May be sent together and displayed on the terminal device 2.
  • the presenting unit 23 ′ receives information specifying the set reference information selected from the list by the user on the terminal device 2 side. Further, when the terminal device 2 gives an instruction to directly input information from the user, the presenting unit 23 'receives the information to that effect and accepts the information input to the information input unit 25'. Instruct.
  • the setting information acquisition unit 24 reads setting information including set identification information related to the set reference information selected by the user from the storage unit 12 as a holding unit in the process of the presentation unit 23 ′. Then, the setting value information included in the read setting information is acquired and output. That is, the setting information acquisition unit 24 implements acquisition means and output means.
  • the information input unit 25 ′ displays a screen for requesting input of set identification information on the terminal device 2. As illustrated in FIG. 9, this screen displays a set identification information input field (S) together with the captured image information. Further, on this screen, input of information stored as setting information such as information on nutritional value may be further obtained (V). When the set identification information is input and transmitted in the terminal device 2, the information input unit 25 ′ accepts the transmitted set identification information and outputs the set identification information to the information recording unit 26. When information to be stored as setting information such as nutritional value information is input here, the information input unit 25 ′ outputs the information to the information recording unit 26.
  • the information recording unit 26 includes the image information (including the Exif tag) as the eating and drinking information received by the information receiving unit 21 ′ and the set reference information selected by the user of the terminal device 2 received by the presentation unit 23 ′.
  • the information and the information specifying the authenticated user are associated with each other and stored in the storage unit 12 (FIG. 10).
  • the information recording unit 26 receives the image information (including the Exif tag) received by the information receiving unit 21 ′ and the set identification information received by the information input unit 25 ′.
  • the information specifying the user is associated with each other and stored in the storage unit 12.
  • Information recorded by the information recording unit 26 is set reference information. Further, the information recording unit 26 associates the set identification information with the nutritional value information and the like when information to be stored as the nutritional value information and the like setting information is input together with the set identification information from the information input unit 25 ′.
  • This process accumulates information about foods and beverages eaten and consumed for each user.
  • the information accumulated in this way is used for browsing processing.
  • the control unit 11 ′ reads set reference information associated with the authenticated user information and displays a list of image information and the like included in the set reference information. . Further, the control unit 11 ′ refers to the set identification information included in the set reference information, reads out nutrition value information associated with the referenced set identification information, transmits the information to the terminal device 2, and includes it in the list May be displayed.
  • the information receiving unit 21 functions as a means for acquiring information for identifying the user.
  • setting of information related to the quantity may be accepted in advance by the user, and the information related to the quantity may be stored in the storage unit 12.
  • the user sets information regarding the amount in the setting information related to the food or drink that was eaten or consumed (set identification information thereof).
  • the information related to this amount sets how much food or drink the user identifies with the set identification information for the average amount as in the example already described. Is. As an example, 0.7 is set when eating and drinking only about 70% at all times, and 2.0 is set when eating and drinking every time.
  • the setting information acquisition unit 24 corrects the nutrition value information included in the setting information by multiplying the information related to the amount by the correction. It is only necessary to output the nutritional value information.
  • the designation can be facilitated and the efficiency of information input can be improved.
  • the search processing unit 22 in the processing of the control unit 11 and the control unit 11 ′ searches for image information similar to the captured image information, but is not limited thereto. Absent. For example, position information (when included in Exif, that may be used) as information on eating and drinking of the user may be used. For example, the set reference information including position information within a predetermined distance threshold range from the referenced latitude and longitude may be searched with reference to the acquired current position information (latitude and longitude information). Further, image information and position information may be used.
  • the position information included in the Exif information of the captured image information and the Exif of the image information included in the found set reference information may be compared, and the list may be rearranged to be displayed at a higher level because it is more likely that the distance between the position information is shorter.
  • the time zone in which the user eats and drinks the food and drink imaged in each image information is determined.
  • “Breakfast”, “Lunch”, “Dinner”, “Other” may be generated and recorded in each entry.
  • it is “breakfast” for entries containing image information taken between 6 am and 9 am
  • “lunch” for entries containing image information taken between 11 am and 2 pm
  • Information indicating that it is “dinner” for entries including image information taken from 5:00 pm to 11:00 pm, and “other” for other time zones (hereinafter, meal type information) May be recorded in association with each other.
  • the meal type information may be an arbitrary character string designated by the user. Specifically, when the user newly inputs information on food and drink, the information receiving unit 21 presents a default character string based on the time zone as the meal type information as described above. When the user gives an instruction to correct the character string, the information receiving unit 21 accepts the input of the character string from the user and uses the input character string as the meal type information. As a result, the user can record individual circumstances such as “drinking party” and “buffet party”.
  • the information receiving unit 21 instructing the imaging unit 15 to acquire image information
  • shooting is performed according to a user instruction (for the imaging unit 15). It may be possible to select whether to acquire image information), to acquire image information that has already been captured and stored in the storage unit 12, or to input the name of a food or drink.
  • a food and drink database that associates food and drink names, food and drink photos, and nutrition value information with each other in advance is stored in the storage unit 12.
  • the nutritional value information is nutritional value information per a predetermined unit (for example, one serving), specifically, a total calorie value per serving or the like.
  • the information receiving unit 21 When the user selects to take a picture here, the operation as described above is performed. In addition, when a selection is made to acquire image information that has already been captured and stored in the storage unit 12, the information receiving unit 21 presents a list of image information stored in the storage unit 12 to the user. Encourage selection. When the user selects food and drink image information from the list, the information receiving unit 21 outputs the selected image information to the search processing unit 22.
  • the search processing unit 22 finds image information similar to the selected image information, the image determined to be similar to the shooting date and time information included in the Exif information of the selected image information.
  • a predetermined threshold value for example, “1 minute”
  • the search processing unit 22 finds image information similar to the selected image information, the image determined to be similar to the shooting date and time information included in the Exif information of the selected image information.
  • the information receiving unit 21 prompts the user to input a name when selection of inputting the name of the food or drink is made.
  • the information receiving unit 21 refers to the food and drink database stored in the storage unit 12 in advance, and includes the input name (completely included or partially included). Search for a product name.
  • the information receiving unit 21 presents a list of entries in the found food and drink database to the user.
  • the information receiving unit 21 When the user selects one from the list, the information receiving unit 21 presents a screen prompting the user to input the quantity. Specifically, this screen receives an input of a quantity in a predetermined unit, such as how many servings or how many grams, as the quantity of food and drink.
  • a predetermined unit such as how many servings or how many grams
  • the information receiving unit 21 uses the nutrition value information per predetermined unit included in the food database entry selected by the user and the input amount information this time.
  • the nutrition value information of the food and drink that the person intends to eat and drink is generated.
  • the information reception part 21 may acquire the information showing the date and time when the information was input from a calendar chip (not shown) or the like.
  • the information recording unit 26 associates the food and beverage name included in the entry in the food and beverage database selected by the user in the information receiving unit 21 with the generated nutrition value information, and sets identification. It is stored in the storage unit 12 as one of information entries.
  • the entry may include date and time information acquired by the information receiving unit 21.
  • the information receiving unit 21 may receive an input of meal type information instead of receiving an input of food and drink names.
  • the information receiving unit 21 in this example receives an input of a character string representing meal type information of food and drink from the user. Then, the information receiving unit 21 searches the set reference information stored in the storage unit 12 for meal type information that matches the input character string or includes the input character string in part. When the information receiving unit 21 finds at least one entry from the set reference information as a result of the search, the information receiving unit 21 extracts the nutrition value information included in each of the found entries, and averages these values (the nutrition value information is a numerical value). Arithmetic average (if any) is calculated.
  • the information receiving unit 21 displays the calculated average value on the display unit 14 and presents it to the user.
  • the information receiving unit 21 accepts input of nutrition value information from the user and outputs it to the information recording unit 26.
  • the calculated average value is output to the information recording unit 26 as the value of the nutritional value information.
  • the information reception part 21 may acquire the information showing the date and time when the information was input from a calendar chip (not shown) or the like.
  • the information receiving unit 21 displays that fact on the display unit 14 and directly provides the nutritional value information to the user. It may be prompted to input.
  • the information receiving unit 21 outputs the input nutrition value information to the information recording unit 26.
  • the information recording unit 26 associates the meal type information input by the user in the information receiving unit 21 with the value of the nutritional value information output by the information receiving unit 21, and stores it as one of the entries of the set identification information 12.
  • the entry may include date and time information acquired by the information receiving unit 21.
  • rough nutrition value information can be set in the case where food and drink are provided at intervals or separated from a common dish. Moreover, it is possible to set nutrition value information that is considered to be appropriate on average from the average of nutrition value information set in the past. Furthermore, according to this example, for example, if the user inputs “dinner” as the meal type information, the average nutritional value information of foods and drinks that have been eaten and consumed as dinner in the past is shown, and the time for inputting detailed information can be taken. Even if there is not, it is possible to record the nutrition value information that is considered to be appropriate on average from the average of the nutrition value information set in the past.
  • the SIFT feature amount is used when determining the similarity of the image information.
  • the present embodiment is not limited to this, and the ORB (the feature amount of each image to be compared) E. Rublee, et al., "ORB: An Efficient Alternative to SIFT or SURF," ICCV, 2011) and so on.
  • the color (R (Red) channel image, B (Blue) channel image, Y (Yellow) channel) for each image channel to be compared ) And color difference images of these channels (RY image, BY image, etc.) the feature amount of each image to be compared may be calculated and used.

Landscapes

  • Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Engineering & Computer Science (AREA)
  • Finance (AREA)
  • Economics (AREA)
  • Accounting & Taxation (AREA)
  • Marketing (AREA)
  • Theoretical Computer Science (AREA)
  • Strategic Management (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Human Resources & Organizations (AREA)
  • General Health & Medical Sciences (AREA)
  • Development Economics (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

[Problem] To provide an information processing device which, even with regard to a set of foods or beverages that includes a plurality of types of foods or beverages, allows easy specification of the content of a meal and improves efficiency of information input. [Solution] The present invention retains set reference information associating eating/drinking time information acquired when a set of food or beverages including at least one type of food or beverage has been eaten or drunk in the past with set identification information for identifying the set, and setting information associating the set identification information with a setting value that includes nutritional value information. When eating or drinking a food or a beverage, upon receipt of a predefined type of information, at least one eating/drinking time information unit similar to the received information unit is retrieved from among the set reference information units.

Description

情報処理装置及びプログラムInformation processing apparatus and program
　本発明は、飲食物の情報に係る処理を行う情報処理装置及びプログラムに関する。 This invention relates to the information processing apparatus and program which perform the process which concerns on the information of food and drink.
　食品の種類（コロッケである、など）と量とを画像から識別して、当該種類の食品の、識別された量だけのカロリー値を演算するという考えが、特許文献１に開示されている。 The idea of identifying the type (such as croquette) and amount of food from an image and calculating the calorie value of the identified amount of the food of that type is disclosed in Patent Document 1.
　また、撮像された画像から作業員がカロリー値を決定するにあたり、作業員の業務を支援するため、過去の食事の内容から類似のものを検索させる技術が、特許文献２に開示されている。 Also, Patent Document 2 discloses a technique for searching for similar items from past meal contents in order to support the work of the worker when the worker determines the calorie value from the captured image.
特開２００７－１２２３１１号公報Japanese Patent Laid-Open No. 2007-12211 特開２００２－０７３８１８号公報JP 2002-073818 A
　しかしながら、今日、一回の食事で飲食する食品が一品のみであることは却って少ない。例えば昼食ではいわゆる「定食」などのセットメニューが用意されており、この単位で注文するのが一般的である。 However, today there are few foods that are eaten and consumed in a single meal. For example, a set menu such as so-called “set meal” is prepared for lunch, and it is common to order in this unit.
　このような常態に鑑みると、食品の種類や量を画像から識別する技術では、過去に同じセットで飲食したときの記録があるにも関わらず、毎回各食品の種類や量を識別することとなって効率が低い。 In view of such a normal condition, the technology for identifying the type and amount of food from the image identifies the type and amount of each food each time, even though there is a record when eating and drinking in the same set in the past. The efficiency is low.
　本発明は上記実情に鑑みて為されたもので、飲食物の情報入力を効率良く行うことのできる情報処理装置、及びプログラムを提供することを、その目的の一つとする。 The present invention has been made in view of the above circumstances, and an object thereof is to provide an information processing apparatus and program capable of efficiently inputting information on food and drink.
　上記従来例の問題点を解決する本発明は、情報処理装置であって、少なくとも一種類の飲食物を含んだ飲食物のセットを過去に飲食した際に取得した飲食時情報と、当該セットを識別するセット識別情報とを関連付けたセット参照情報、及び、前記セット識別情報と栄養価の情報を含む設定値とを関連付けた設定情報を保持する保持手段と、飲食物の飲食時に、予め定めた種類の情報を受け入れる受入手段と、前記受け入れた情報に類似する飲食時情報を前記保持手段に保持されたセット参照情報のうちから少なくとも一つ検索する検索手段と、前記検索により見出されたセット参照情報に係るセット識別情報を提示し、利用者の選択を促す提示手段と、前記提示手段で提示されたセット識別情報のうちから利用者が選択したセット参照情報を含む設定情報を、前記保持手段から読み出し、読み出した設定情報に含まれる設定値の情報を取得する取得手段と、当該取得した設定値の情報を出力する出力手段と、を含むこととしたものである。 This invention which solves the problem of the above-mentioned conventional example is an information processing device, and is obtained when eating and drinking information in the past when a set of food and drink containing at least one kind of food and drink is set. The set reference information associated with the set identification information to be identified, the holding means for retaining the setting information associated with the set identification information and the setting value including the nutrition value information, and predetermined when eating and drinking A receiving means for receiving information of a type, a searching means for searching at least one set reference information held in the holding means for eating and drinking information similar to the received information, and a set found by the search Presenting means for presenting set identification information related to the reference information, prompting the user to select, and set reference information selected by the user from the set identification information presented by the presenting means Including the acquisition means for acquiring the setting value information included in the read setting information and the output means for outputting the acquired setting value information. It is.
　またここで前記保持手段は、飲食時情報として、セットに含まれる各種類の飲食物の画像情報と、当該画像情報に撮影された飲食物を識別する個別識別情報とを関連付けた個別参照情報をさらに保持し、前記保持手段に保持されたセット参照情報には、さらに、当該セットに含まれる種類の飲食物を識別する個別識別情報が関連付けられて保持されており、前記受入手段は、予め定めた種類の情報として飲食物の画像の情報を受け入れ、前記検索手段は、前記受け入れた画像情報に含まれる各種類の飲食物の画像に類似する複数の画像情報を前記保持手段から検索して、当該検索により見出された複数の画像情報のそれぞれに関連付けられた個別識別情報を取得し、当該取得した複数の個別識別情報を含むセット参照情報を検索して、当該検索の結果を出力することとしてもよい。 Moreover, the said holding | maintenance means here the separate reference information which linked | related the image information of each kind of food and drink contained in a set, and the individual identification information which identifies the food and drink image | photographed by the said image information as information at the time of eating and drinking Furthermore, the set reference information held and held in the holding means is further held in association with individual identification information for identifying the type of food or drink contained in the set, and the receiving means is predetermined. The information of food and drink images is received as the type of information, and the search means searches the holding means for a plurality of image information similar to each type of food and drink image included in the received image information, Obtaining individual identification information associated with each of the plurality of image information found by the search, searching for set reference information including the obtained plurality of individual identification information, May output the result of the search.
　また、利用者を識別する情報を取得する手段をさらに含み、前記保持手段は、設定情報としてさらに、飲食物の分量に関する情報を利用者ごとに保持し、前記出力手段は、前記取得した情報で識別される利用者に係る、前記取得した設定値の飲食物の分量に関する情報を用いて、当該設定値における栄養価の情報を補正し、補正した栄養価の情報を出力こととしてもよい。 Further, the information processing apparatus further includes means for acquiring information for identifying a user, the holding means further holds, as setting information, information related to the amount of food and drink for each user, and the output means includes the acquired information. It is good also as correcting the information on the nutritional value in the said setting value using the information regarding the amount of food and drink of the acquired setting value concerning the identified user, and outputting the information on the corrected nutritional value.
　さらに本発明の一態様に係るプログラムは、コンピュータを、少なくとも一種類の飲食物を含んだ飲食物のセットを過去に飲食した際に取得した飲食時情報と、当該セットを識別するセット識別情報とを関連付けたセット参照情報、及び、前記セット識別情報と栄養価の情報を含む設定値とを関連付けた設定情報を保持する保持手段と、飲食物の飲食時に、予め定めた種類の情報を受け入れる受入手段と、前記受け入れた情報に類似する飲食時情報を前記保持手段に保持されたセット参照情報のうちから少なくとも一つ検索する検索手段と、前記検索により見出されたセット参照情報に係るセット識別情報を提示し、利用者の選択を促す提示手段と、前記提示手段で提示されたセット識別情報のうちから利用者が選択したセット参照情報を含む設定情報を、前記保持手段から読み出し、読み出した設定情報に含まれる設定値の情報を取得する取得手段と、当該取得した設定値の情報を出力する出力手段と、として機能させることとしたものである。 Furthermore, the program according to one aspect of the present invention includes a computerized information at the time of eating and drinking acquired when a set of food and drink including at least one kind of food and drink in the past, and set identification information for identifying the set. Holding reference information that associates the set identification information and setting information that associates the set identification information and the setting value that includes nutrition value information, and accepting information of a predetermined type when eating or drinking food or drink Means for searching for at least one set reference information held in the holding means for information on eating and drinking similar to the received information, and set identification relating to the set reference information found by the search Presenting means for presenting information and prompting the user to select, and set reference information selected by the user from the set identification information presented by the presenting means The setting information is read out from the holding means, and functions as an acquisition means for acquiring the setting value information included in the read setting information, and an output means for outputting the acquired setting value information. It is.
　本発明によると、複数種類の飲食物を含んだ飲食物のセットについても、その食事内容の指定を容易にでき、情報入力の効率を向上できる。 According to the present invention, for a set of foods and drinks including a plurality of types of foods and drinks, the contents of the meal can be easily specified, and the efficiency of information input can be improved.
本発明の実施の形態に係る情報処理装置の構成例を表すブロック図である。It is a block diagram showing the example of a structure of the information processing apparatus which concerns on embodiment of this invention. 本発明の実施の形態に係る情報処理装置が保持するセット参照情報の内容例を表す説明図である。It is explanatory drawing showing the example of the content of the set reference information which the information processing apparatus which concerns on embodiment of this invention hold | maintains. 本発明の実施の形態に係る情報処理装置が保持する設定情報の内容例を表す説明図である。It is explanatory drawing showing the example of the content of the setting information which the information processing apparatus which concerns on embodiment of this invention hold | maintains. 本発明の実施の形態に係る情報処理装置の制御部の例を表す機能ブロック図である。It is a functional block diagram showing the example of the control part of the information processing apparatus which concerns on embodiment of this invention. 本発明の実施の形態に係る情報処理装置により表示される画面の例を表す説明図である。It is explanatory drawing showing the example of the screen displayed by the information processing apparatus which concerns on embodiment of this invention. 本発明の実施の形態に係る情報処理装置の動作例を表すフローチャート図である。It is a flowchart figure showing the operation example of the information processing apparatus which concerns on embodiment of this invention. 本発明の実施の形態に係る情報処理装置の別の構成例を表すブロック図である。It is a block diagram showing another structural example of the information processing apparatus which concerns on embodiment of this invention. 本発明の実施の形態に係る情報処理装置の制御部の別の例を表す機能ブロック図である。It is a functional block diagram showing another example of the control part of the information processing apparatus which concerns on embodiment of this invention. 本発明の実施の形態に係る情報処理装置により表示される別の画面の例を表す説明図である。It is explanatory drawing showing the example of another screen displayed by the information processing apparatus which concerns on embodiment of this invention. 本発明の実施の形態に係る情報処理装置により記録される情報の一例を表す説明図である。It is explanatory drawing showing an example of the information recorded by the information processing apparatus which concerns on embodiment of this invention.
　本発明の実施の形態について図面を参照しながら説明する。本発明の実施の形態に係る情報処理装置１は、図１に例示するように、制御部１１と、記憶部１２と、操作部１３と、表示部１４と、撮像部１５と、現在位置取得部１６とを含んで構成される。 Embodiments of the present invention will be described with reference to the drawings. As illustrated in FIG. 1, the information processing apparatus 1 according to the embodiment of the present invention includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, an imaging unit 15, and a current position acquisition. Part 16.
　制御部１１は、ＣＰＵ等のプログラム制御デバイスであり、記憶部１２に格納されたプログラムに従って動作している。この制御部１１は、利用者が飲食物を飲食する時にした操作により、予め定めた種類の情報を受け入れる。またこの制御部１１は、当該受け入れた情報に類似する飲食時情報を記憶部１２に保持されたセット参照情報（後述）のうちから少なくとも一つ検索する。そして制御部１１は、検索により見出されたセット参照情報に係るセット識別情報を提示し、利用者の選択を促し、利用者がセット識別情報を選択すると、当該セット識別情報に関連する設定情報を記憶部１２から読み出し、読み出した設定情報に含まれる設定値の情報を取得して出力する処理を行う。この制御部１１の処理については後に詳しく述べる。 The control unit 11 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12. This control part 11 receives the information of a predetermined kind by operation which the user did when eating and drinking food. Further, the control unit 11 searches for at least one of the set reference information (described later) held in the storage unit 12 for information on eating and drinking similar to the received information. Then, the control unit 11 presents the set identification information related to the set reference information found by the search, prompts the user to select, and when the user selects the set identification information, the setting information related to the set identification information Is read out from the storage unit 12, and the setting value information included in the read setting information is acquired and output. The processing of the control unit 11 will be described in detail later.
　記憶部１２は、ＳＳＤ（Solid State Disk）や、フラッシュメモリ、ＲＡＭ（Random Access Memory）等を含んだ記憶デバイスであり、本発明における保持手段として機能する。この記憶部１２は、制御部１１によって実行されるプログラムを保持する。このプログラムは、ＤＶＤ－ＲＯＭ等のコンピュータ可読な記録媒体に格納されて提供され、あるいはネットワークを介して配信され、この記憶部１２に格納されたものであってもよい。 The storage unit 12 is a storage device including an SSD (Solid State Disk), flash memory, RAM (Random Access Memory), and the like, and functions as a holding unit in the present invention. The storage unit 12 holds a program executed by the control unit 11. This program may be provided by being stored in a computer-readable recording medium such as a DVD-ROM, or may be distributed via a network and stored in the storage unit 12.
　この記憶部１２はまた、制御部１１のワークメモリとしても動作する。さらに本実施の形態では、この記憶部１２は、図２に例示するように、少なくとも一種類の飲食物を含んだ飲食物のセットを過去に飲食した際に取得した飲食時情報（Ａ）と、当該セットを識別するセット識別情報（Ｂ）とを関連付けたセット参照情報、及び、図３に例示するように、このセット識別情報（Ｂ）と栄養価の情報を含む設定値（Ｃ）とを関連付けた設定情報を保持する。ここでセット識別情報（Ｂ）は、利用者によって入力された飲食物のセットの名称（例えば「Ａ定食」など）でよい。なお、以下の説明でこれらの情報をセット参照情報、セット識別情報等と称しているが、セットとなった飲食物の情報を含むことができるための名称であり、単品の飲食物の情報がセット参照情報に含まれてもよい。この場合は当該単品の飲食物の画像等の飲食時情報に対して、セット識別情報として単品の飲食物を特定する情報が関連付けられる。 The storage unit 12 also operates as a work memory for the control unit 11. Furthermore, in this Embodiment, this memory | storage part 12 is the information (A) at the time of eating and drinking acquired when eating and drinking the set of the food and drink containing at least 1 type of food and drink in the past so that it may illustrate in FIG. , Set reference information associated with set identification information (B) for identifying the set, and set value (C) including the set identification information (B) and nutrition value information as illustrated in FIG. The setting information associated with is stored. Here, the set identification information (B) may be a name of a set of food and drink (for example, “A set meal”) input by the user. In addition, in the following description, these information are referred to as set reference information, set identification information, etc., but it is a name that can include information on the food and drink that has become a set, and information on a single food and drink is included. It may be included in the set reference information. In this case, information for specifying a single item of food and drink is associated with the information at the time of eating and drinking such as an image of the single item of food and drink as set identification information.
　具体的に、セット参照情報に含まれる飲食時情報は、飲食時に利用者が撮像した飲食物の画像情報（セットの少なくとも一部の飲食物を含んだ画像情報）、飲食した日時を表す情報、飲食した場所を表す情報（緯度、経度の情報や、店名等であってもよいし、イベントに関する情報等、利用者が任意に入力した情報であってもよい。なお、画像情報を必ず含むこととすれば、当該画像情報に関連するタグ（例えばExif（Exchangeable image file format）情報）として、飲食した日時や場所等の情報が記録されていてもよい。 Specifically, the eating and drinking information included in the set reference information includes image information of food and drink taken by the user at the time of eating and drinking (image information including at least a part of food and drink in the set), information indicating the date and time of eating and drinking, Information indicating the place where the restaurant was eaten (latitude / longitude information, store name, etc., or information arbitrarily entered by the user, such as information related to the event, etc. In addition, image information must be included. Then, information such as the date and time of eating and drinking, and the place may be recorded as a tag related to the image information (for example, Exif (Exchangeable image file format) information).
　操作部１３は、例えば表示部１４に重ね合わせて設置されたタッチパネルであり、利用者の指が接触したことを検出して、当該指の接触した位置を表す情報を制御部１１に出力する。表示部１４は、液晶や有機ＥＬ等の表示デバイスであり、制御部１１から入力される指示に従って、画像等の情報を表示する。 The operation unit 13 is, for example, a touch panel installed so as to overlap the display unit 14, detects that the user's finger is in contact, and outputs information indicating the contact position of the finger to the control unit 11. The display unit 14 is a display device such as a liquid crystal or an organic EL, and displays information such as an image in accordance with an instruction input from the control unit 11.
　撮像部１５は、ＣＣＤ（Charge Coupled Device）イメージセンサやＣＭＯＳ（Complementary Metal Oxide Semiconductor）イメージセンサ等の撮像素子を含み、制御部１１から入力される指示に従って、これらのイメージセンサによって取得した画像情報を取込み、制御部１１に出力する。 The imaging unit 15 includes an imaging element such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor, and in accordance with instructions input from the control unit 11, image information acquired by these image sensors. Take in and output to the control unit 11.
　現在位置取得部１６は、例えばＧＰＳ（Global Positioning System）デバイスを含み、制御部１１から入力される指示に従って、現在位置を測定し、当該測定によって得た現在位置を表す情報（緯度、経度等の情報）を制御部１１に出力する。なお、この現在位置取得部１６は、ＧＰＳでなくとも、例えば受信可能な無線ＬＡＮ信号を利用して現在位置を取得してもよいし、通信可能な携帯電話基地局の情報を利用して現在位置を取得することとしてもよい。 The current position acquisition unit 16 includes, for example, a GPS (Global Positioning System) device, measures the current position in accordance with an instruction input from the control unit 11, and represents information (latitude, longitude, etc.) representing the current position obtained by the measurement. Information) is output to the control unit 11. Note that the current position acquisition unit 16 may acquire the current position using, for example, a receivable wireless LAN signal instead of the GPS, or may use the information on a mobile phone base station capable of communication to acquire the current position. It is good also as acquiring a position.
　次に、本実施の形態の制御部１１によって実行される処理の具体的例について説明する。本実施の形態の制御部１１は、記憶部１２に格納されたプログラムに従って動作することで、機能的には図４に例示する構成を備えるものとして動作する。すなわち制御部１１は、機能的に、図４に例示するように、情報受入部２１と、検索処理部２２と、提示部２３と、設定情報取得部２４と、情報入力部２５と、情報記録部２６とを含んで構成される。 Next, a specific example of processing executed by the control unit 11 of this embodiment will be described. The control unit 11 according to the present embodiment operates according to a program stored in the storage unit 12 and functionally operates as having the configuration illustrated in FIG. That is, the control unit 11 functionally includes an information receiving unit 21, a search processing unit 22, a presentation unit 23, a setting information acquisition unit 24, an information input unit 25, and an information recording as illustrated in FIG. Part 26.
　情報受入部２１は、飲食物の飲食時に、予め定めた種類の情報を受け入れる受入手段を実現する。この情報受入部２１は、予め定めた種類の情報として、例えば撮像部１５に対して画像情報の取得を指示することとしてもよい。利用者は撮像部１５を用いて飲食物の画像情報を撮像する操作を行い、撮像部１５が当該利用者の操作により得た画像情報を情報受入部２１に出力する。このとき情報受入部２１は、現在位置取得部１６に対して現在位置を取得するよう指示し、現在位置取得部１６が出力する現在位置の情報を受け入れて、撮像部１５から取得した画像情報に対して、Exif情報の一部として当該現在位置の情報を関連付けて記憶部１２に保持してもよい。 The information accepting unit 21 realizes accepting means for accepting a predetermined type of information when eating or drinking food. For example, the information receiving unit 21 may instruct the imaging unit 15 to acquire image information as a predetermined type of information. The user performs an operation of capturing image information of food and drink using the image capturing unit 15, and the image capturing unit 15 outputs image information obtained by the user's operation to the information receiving unit 21. At this time, the information receiving unit 21 instructs the current position acquisition unit 16 to acquire the current position, receives the information on the current position output from the current position acquisition unit 16, and converts the current position information into the image information acquired from the imaging unit 15. On the other hand, the current position information may be associated and held in the storage unit 12 as a part of the Exif information.
　さらに情報受入部２１は、図示しないカレンダーチップ等から、撮像部１５にて撮像が行われた日時を表す情報を取得し、Exif情報の一部として、撮像部１５から取得した画像情報に関連付けて記憶部１２に格納してもよい。 Further, the information receiving unit 21 acquires information indicating the date and time when the image capturing unit 15 performs image capturing from a calendar chip or the like (not shown), and associates it with the image information acquired from the image capturing unit 15 as part of the Exif information. You may store in the memory | storage part 12. FIG.
　検索処理部２２は、情報受入部２１が情報を受け入れると動作し、情報受入部２１にて受け入れた情報に類似する飲食時情報を保持手段としての記憶部１２に保持されたセット参照情報のうちから少なくとも一つ検索する検索手段を実現する。 The search processing unit 22 operates when the information receiving unit 21 accepts information, and among the set reference information held in the storage unit 12 serving as holding means, information on eating and drinking similar to the information received by the information receiving unit 21 A search means for searching from at least one is realized.
　ここで例えば記憶部１２に格納されているセット参照情報には、飲食時情報として必ず、飲食物の画像情報が記録されているものとすると、この検索処理部２２は情報受入部２１が受け入れた画像情報から予め定めた特徴量を算出する。この特徴量は広く知られた画像比較のための特徴量、例えばＳＩＦＴ（Scale Invariant Feature Transform）特徴量等でよい。 Here, for example, in the set reference information stored in the storage unit 12, assuming that image information of food and drink is always recorded as information on eating and drinking, the search processing unit 22 is accepted by the information receiving unit 21. A predetermined feature amount is calculated from the image information. This feature amount may be a well-known feature amount for image comparison, for example, a SIFT (Scale Invariant Feature Transform) feature amount.
　検索処理部２２は、記憶部１２に格納されているセット参照情報を順次比較対象として選択して、当該比較対象に含まれる飲食物の画像情報に関するＳＩＦＴ特徴量を取得する（予め演算されてセット参照情報に含まれて記録されていてもよいし、ここで演算して求めてもよい）。そして比較対象として選択したセット参照情報に含まれる画像情報から演算したＳＩＦＴ特徴量と、情報受入部２１が受け入れた画像情報から算出したＳＩＦＴ特徴量とを比較する。 The search processing unit 22 sequentially selects the set reference information stored in the storage unit 12 as a comparison target, and acquires a SIFT feature amount related to the image information of food and drink included in the comparison target (calculated and set in advance). It may be recorded by being included in the reference information, or may be obtained by calculation here). Then, the SIFT feature value calculated from the image information included in the set reference information selected as the comparison target is compared with the SIFT feature value calculated from the image information received by the information receiving unit 21.
　ＳＩＦＴ特徴量には、キーポイントと、各キーポイントの特徴量とを関連付けた要素が、画像から見出されたキーポイントの数だけ含まれる。そこでＳＩＦＴ特徴量の比較では、特徴量が一致すると判断されるキーポイントがいくつあるかにより、例えば比較対象として選択したセット参照情報に含まれる画像情報から演算したＳＩＦＴ特徴量と、情報受入部２１が受け入れた画像情報から算出したＳＩＦＴ特徴量との間で特徴量が一致すると判断されるキーポイントの数が予め定めたしきい値を超える場合に、比較対象として選択したセット参照情報に含まれる画像情報と、情報受入部２１が受け入れた画像情報とが類似すると判断すればよい。 The SIFT feature amount includes elements that associate key points with the feature amount of each key point by the number of key points found from the image. Therefore, in the comparison of SIFT feature values, for example, the SIFT feature value calculated from the image information included in the set reference information selected as the comparison target and the information receiving unit 21 depend on how many key points are determined to match the feature values. Is included in the set reference information selected as the comparison target when the number of keypoints determined to match the SIFT feature value calculated from the image information received from the image information exceeds a predetermined threshold value What is necessary is just to judge that image information and the image information which the information reception part 21 received are similar.
　検索処理部２２は、情報受入部２１が受け入れた画像情報に類似する画像情報を含んだセット参照情報を少なくとも一つ見出すと、見出したセット参照情報を特定する情報（セット識別情報でよい）を提示部２３に出力する。なお、この検索処理部２２は、情報受入部２１が受け入れた画像情報に類似する画像情報を含んだセット参照情報を見出すことができなかった場合は、情報入力部２５に対して情報の入力を受け入れるよう指示する。 When the search processing unit 22 finds at least one set reference information including image information similar to the image information received by the information receiving unit 21, information for identifying the found set reference information (may be set identification information). Output to the presentation unit 23. The search processing unit 22 inputs information to the information input unit 25 when the set reference information including image information similar to the image information received by the information receiving unit 21 cannot be found. Instruct to accept.
　提示部２３は、検索処理部２２にて見出されたセット参照情報に係るセット識別情報を提示し、利用者の選択を促す提示手段を実現する。この提示部２３は具体的には、検索処理部２２から入力されるセット参照情報を特定する情報の一覧を表示部１４に表示する。この表示は例えば図５に例示するように、検索処理部２２から入力される情報で特定されるセット参照情報のうち、画像情報（Ｇ）と、セット識別情報（Ｂ）とを読み出して、レイアウトして表示したものである。また、検索処理部２２から入力される情報で特定されるセット参照情報に係るセット識別情報に関連付けられた設定値の情報をさらに記憶部１２から読み出し、この設定値の情報（Ｃ）を併せて表示してもよい。そして提示部２３は、利用者に対して当該一覧のうちからセット参照情報の一つを選択させる。 The presentation unit 23 presents set identification information related to the set reference information found by the search processing unit 22 and realizes a presentation unit that prompts the user to select. Specifically, the presenting unit 23 displays a list of information specifying the set reference information input from the search processing unit 22 on the display unit 14. For example, as shown in FIG. 5, this display reads out image information (G) and set identification information (B) from the set reference information specified by the information input from the search processing unit 22, and performs layout. Is displayed. Further, the setting value information associated with the set identification information related to the set reference information specified by the information input from the search processing unit 22 is further read from the storage unit 12, and the setting value information (C) is also combined. It may be displayed. Then, the presentation unit 23 causes the user to select one of the set reference information from the list.
　さらにこの提示部２３は、利用者から情報を直接入力する旨の指示を受けると、情報入力部２５に対して情報の入力を受け入れるよう指示する。 Further, upon receiving an instruction to directly input information from the user, the presenting unit 23 instructs the information input unit 25 to accept the input of information.
　設定情報取得部２４は、提示部２３の処理において、利用者が選択したセット参照情報に係るセット識別情報を含む設定情報を、保持手段としての記憶部１２から読み出す。そして当該読み出した設定情報に含まれる設定値の情報を取得して出力する。つまり、この設定情報取得部２４は、取得手段及び出力手段を実現するものである。 The setting information acquisition unit 24 reads setting information including set identification information related to the set reference information selected by the user from the storage unit 12 as a holding unit in the process of the presentation unit 23. Then, the setting value information included in the read setting information is acquired and output. That is, the setting information acquisition unit 24 implements acquisition means and output means.
　情報入力部２５は、情報の入力を受け入れるべき旨の指示が入力されると、利用者に対して、セット識別情報の入力を求める。情報入力部２５は、利用者からセット識別情報が入力されると、当該入力された情報を情報記録部２６に出力する。 The information input unit 25 requests the user to input set identification information when an instruction to accept the input of information is input. When the set identification information is input from the user, the information input unit 25 outputs the input information to the information recording unit 26.
　情報記録部２６は、情報受入部２１が受け入れた画像情報（Exifタグを含む）と、提示部２３にて利用者が選択したセット参照情報に含まれるセット識別情報と、設定情報取得部２４が出力する設定値の情報とを受け入れると、これらの情報を互いに関連付けて、記憶部１２に格納する。またこの情報記録部２６は、情報受入部２１が受け入れた画像情報（Exifタグを含む）と、情報入力部２５が受け入れたセット識別情報とが入力されると、これらの情報を互いに関連付けて、記憶部１２に格納する。これら情報記録部２６によって記録された情報がセット参照情報となる。 The information recording unit 26 includes image information (including an Exif tag) received by the information receiving unit 21, set identification information included in the set reference information selected by the user in the presentation unit 23, and a setting information acquisition unit 24. When the setting value information to be output is received, these pieces of information are associated with each other and stored in the storage unit 12. In addition, when the image information (including the Exif tag) received by the information receiving unit 21 and the set identification information received by the information input unit 25 are input, the information recording unit 26 associates these pieces of information with each other, Store in the storage unit 12. Information recorded by the information recording unit 26 is set reference information.
　本実施の形態の一例に係る情報処理装置１は以上の構成を有してなり、次のように動作する。利用者は過去に飲食した飲食物の画像情報を撮影し、それぞれセット識別情報を入力しているものとする。一例としてここでは記憶部１２に、セット参照情報として、画像情報ａに対してセット識別情報「Ａ定食」を関連付けた情報と、画像情報ｂに対してセット識別情報「Ｂセット」を関連付けた情報とが保持されているものとする。また、セット識別情報「Ａ定食」に対し栄養価の情報「αCal」が、「Ｂセット」に対して栄養価の情報「βCal」が、それぞれ関連付けられているものとする。 The information processing apparatus 1 according to an example of the present embodiment has the above configuration, and operates as follows. It is assumed that the user has photographed image information of food and drink that has been eaten and consumed in the past, and has input set identification information. As an example, here, information relating the set identification information “A set meal” to the image information a and information relating the set identification information “B set” to the image information b as set reference information in the storage unit 12. And are held. Further, it is assumed that nutrition value information “αCal” is associated with set identification information “A set meal”, and nutrition value information “βCal” is associated with “B set”.
　利用者がまた別の機会に「Ａ定食」を注文し、この「Ａ定食」の画像情報を撮像すると、制御部１１が、図６に例示する処理を開始し、制御部１１は、当該撮像して得た画像情報からＳＩＦＴ特徴量を算出する（Ｓ１）。次に制御部１１は、記憶部１２に格納されているセット参照情報のうち、未選択のセット参照情報の一つを比較対象として選択する（Ｓ２）。そして当該比較対象に含まれる飲食物の画像情報に関するＳＩＦＴ特徴量を取得する（Ｓ３）。既に述べたように、このＳＩＦＴ特徴量が予め演算されてセット参照情報に含まれて記録されている場合は、選択したセット参照情報に含まれる当該ＳＩＦＴ特徴量を取得する。また、そうでない場合はここで制御部１１が演算して求める。 When the user orders “A set meal” on another occasion and images the image information of this “A set meal”, the control unit 11 starts the processing illustrated in FIG. 6. The SIFT feature value is calculated from the image information obtained in this way (S1). Next, the control unit 11 selects one of the unselected set reference information among the set reference information stored in the storage unit 12 as a comparison target (S2). And the SIFT feature-value regarding the image information of the food and drink contained in the said comparison object is acquired (S3). As already described, when the SIFT feature value is calculated in advance and included in the set reference information and recorded, the SIFT feature value included in the selected set reference information is acquired. If this is not the case, the control unit 11 calculates and calculates here.
　制御部１１は、処理Ｓ１で得たＳＩＦＴ特徴量と、処理Ｓ３で取得したＳＩＦＴ特徴量とを比較して、撮像して得た画像情報と、比較対象に含まれる飲食物の画像情報とが類似するか否かを判断する（Ｓ４）。ここでの比較は種々の方法が考えられるが、処理Ｓ１で得たＳＩＦＴ特徴量と、処理Ｓ３で取得したＳＩＦＴ特徴量とからそれぞれ選択されるＳＩＦＴ特徴量であって、互いの距離（ＳＩＦＴ特徴量はベクトル量であるので、そのユークリッド距離を用いればよい）が予め定めた距離しきい値を下回るＳＩＦＴ特徴量の組の数をカウントし、この数が予め定めた個数しきい値を超える場合に、撮像して得た画像情報と、比較対象に含まれる飲食物の画像情報とが類似すると判断することとしてもよい。 The control unit 11 compares the SIFT feature quantity obtained in the process S1 with the SIFT feature quantity obtained in the process S3, and the image information obtained by imaging and the image information of the food and drink included in the comparison target are obtained. It is determined whether or not they are similar (S4). Although various methods can be considered for comparison here, SIFT feature values respectively selected from the SIFT feature value obtained in the process S1 and the SIFT feature value obtained in the process S3, and the mutual distance (SIFT feature value). When the quantity is a vector quantity, the Euclidean distance may be used). When the number of sets of SIFT feature quantities below a predetermined distance threshold is counted, and this number exceeds a predetermined number threshold In addition, it may be determined that the image information obtained by imaging and the image information of the food and drink included in the comparison target are similar.
　ここでＳＩＦＴ特徴量のように画像の局所で定義可能な特徴量を用いることで、定食のように盆の上で各料理がどのように配されるかが毎回異なる可能性のあるものであっても互いに類似する画像として検索させることが可能となる。さらに、ＳＩＦＴ特徴量の場合は、回転に対しても不変であるという特徴を有するので、皿の向きが異なっている場合でも類似の画像を検索できる。 Here, using features that can be defined locally in the image, such as SIFT features, the manner in which each dish is arranged on the tray like a set meal may be different each time. However, it is possible to search for images similar to each other. Further, since the SIFT feature amount has a feature that it is invariant to rotation, a similar image can be searched even when the direction of the dish is different.
　制御部１１は、処理Ｓ４の比較において、撮像して得た画像情報と、比較対象に含まれる飲食物の画像情報とが類似すると判断すると（Yesであると）、当該類似していると判断した比較対象のセット参照情報を特定する情報を、選択候補リストの候補の一つとして記憶部１２に記録する（Ｓ５）。 When the control unit 11 determines that the image information obtained by imaging and the image information of the food and drink included in the comparison target are similar in the comparison of the process S4 (Yes), the control unit 11 determines that the image information is similar. Information for specifying the set reference information to be compared is recorded in the storage unit 12 as one of the candidates in the selection candidate list (S5).
　制御部１１は、記憶部１２に格納されているセット参照情報のうち、比較対象として選択されていないものがまだあるか否かを判断し（Ｓ６）、選択されていないものがあれば（Yesならば）、処理Ｓ２に戻って当該選択されていないセット参照情報を選択する。 The control unit 11 determines whether or not there is any set reference information stored in the storage unit 12 that is not selected as a comparison target (S6). If so, the process returns to step S2 to select the unselected set reference information.
　また制御部１１は、処理Ｓ６において、記憶部１２に格納されているセット参照情報のうち、比較対象として選択されていないものがなければ（Noならば）、記憶部１２に格納した選択候補リストに、セット参照情報を特定する情報が少なくとも一つあるか否かを調べる（Ｓ７）。ここで選択候補リストに、セット参照情報を特定する情報が少なくとも一つあるならば（Yesならば）、制御部１１は、当該選択候補リストに含まれるセット参照情報に基づく一覧を表示出力して、利用者の選択を促す（Ｓ８）。この表示は図５に例示したようなものとなる。 In addition, in step S6, the control unit 11 selects the selection candidate list stored in the storage unit 12 if there is no set reference information stored in the storage unit 12 that is not selected as a comparison target (if No). Whether there is at least one piece of information specifying the set reference information is checked (S7). If the selection candidate list includes at least one piece of information for specifying the set reference information (if Yes), the control unit 11 displays and outputs a list based on the set reference information included in the selection candidate list. Prompts the user to select (S8). This display is as illustrated in FIG.
　制御部１１は、利用者が一覧から選択したセット参照情報を特定する情報（例えばセット識別情報）を受け入れて（Ｓ９）、当該セット識別情報を含む設定情報を、記憶部１２から読み出す。そして制御部１１は当該読み出した設定情報に含まれる設定値の情報を取得し（Ｓ１０）、撮像して得られた画像情報と、処理Ｓ９で受け入れたセット識別情報と、処理Ｓ１０で取得した設定値の情報とを関連付けて記憶部１２に格納して（Ｓ１１）、処理を終了する。 The control unit 11 accepts information (for example, set identification information) specifying the set reference information selected from the list by the user (S9), and reads setting information including the set identification information from the storage unit 12. Then, the control unit 11 acquires the information of the setting value included in the read setting information (S10), the image information obtained by imaging, the set identification information received in the process S9, and the setting acquired in the process S10. The value information is stored in association with the storage unit 12 (S11), and the process is terminated.
　また制御部１１は、処理Ｓ４にて撮像して得た画像情報と、比較対象に含まれる飲食物の画像情報とが類似していないと判断すると（Noであると）、処理Ｓ６に移行して処理を続ける。 Moreover, if the control part 11 judges that the image information obtained by image-capturing in process S4 and the image information of the food and drink contained in a comparison object are not similar (it is No), it will transfer to process S6. And continue processing.
　また処理Ｓ７において、選択候補リストに、セット参照情報を特定する情報が一つもなければ（Noであれば）、制御部１１は、利用者に対してセット識別情報を求める（Ｓ１２）。このとき制御部１１は、設定情報の入力を併せて求めてもよい。 In step S7, if there is no information specifying the set reference information in the selection candidate list (if No), the control unit 11 obtains set identification information from the user (S12). At this time, the control unit 11 may also obtain the input of setting information.
　処理Ｓ１２にて利用者がセット識別情報を入力すると、制御部１１は、撮像して得られた画像情報と当該入力されたセット識別情報とを関連付けて、記憶部１２に格納して（Ｓ１３）、処理を終了する。 When the user inputs the set identification information in the process S12, the control unit 11 associates the image information obtained by imaging with the input set identification information and stores it in the storage unit 12 (S13). The process is terminated.
　具体的に本実施の形態の一例では、制御部１１は、撮像して得られた画像情報を飲食時情報とし、入力されたセット識別情報に関連付けてセット参照情報として記憶部１２に格納する（つまりセット参照情報を更新する）こととしてもよい。 Specifically, in one example of the present embodiment, the control unit 11 uses the image information obtained by imaging as eating / drinking information, and stores it in the storage unit 12 as set reference information in association with the input set identification information ( That is, the set reference information may be updated).
　なお、制御部１１は処理Ｓ８において選択候補リストに含まれるセット参照情報に基づく一覧を表示する際に、「新たに情報を入力する」（つまり一覧にない）旨の指示を選択の候補に加えて表示してもよい。利用者が処理Ｓ８にてこの指示を選択した場合、制御部１１は、処理Ｓ１２に移行して処理を続けることとすればよい。 When the control unit 11 displays a list based on the set reference information included in the selection candidate list in step S8, the control unit 11 adds an instruction to “newly input information” (that is, not in the list) to the selection candidate. May be displayed. When the user selects this instruction in process S8, the control unit 11 may proceed to process S12 and continue the process.
　さらに制御部１１は、利用者からの指示により、閲覧の処理に供される。具体的に制御部１１は、利用者から入力される閲覧の指示に従い、記憶部１２に格納されているセット参照情報を読み出す。そして当該セット参照情報に含まれる画像情報等の一覧を表示する。また、制御部１１は、このセット参照情報に含まれるセット識別情報を参照し、当該参照したセット識別情報に関連付けられた栄養価の情報等を読み出して、一覧に含めて表示してもよい。さらにこの一覧は、例えば画像情報のExif情報を参照して撮影日ごとに区切って表示されてもよい。 Further, the control unit 11 is subjected to browsing processing according to an instruction from the user. Specifically, the control unit 11 reads the set reference information stored in the storage unit 12 in accordance with a browsing instruction input from the user. A list of image information and the like included in the set reference information is displayed. In addition, the control unit 11 may refer to the set identification information included in the set reference information, read out nutrition value information associated with the referenced set identification information, and display the information in a list. Further, this list may be displayed separated by shooting date with reference to Exif information of image information, for example.
　またここまでの説明では制御部１１の検索処理部２２がＳＩＦＴ特徴量を用いて取得した画像情報に類似する過去の画像情報を検索する例について説明したが、本実施の形態はこれに限られず、他の特徴量として局所特徴量や、ヒストグラム等の色に関わる特徴量を用いてもよい。例えば検索処理部２２は、比較の対象となる画像情報を、セットに含まれる各種類の飲食物ごとに分割して比較してもよい。具体的には比較の対象となる画像情報を各料理の皿ごとに分割し、皿ごとに画像情報を比較してもよい。この場合のセット参照情報は、皿ごとに分割された画像情報と、皿ごとの飲食物を特定するセット識別情報（以下区別のため個別識別情報と呼ぶ）とを関連付けたエントリー（以下区別のため個別参照情報と呼ぶ）と、複数の飲食物を特定する個別識別情報の組を含んだエントリー（以下区別のため識別情報エントリーと呼ぶ）と、を含む。 In the description so far, the example in which the search processing unit 22 of the control unit 11 searches for past image information similar to the image information acquired using the SIFT feature amount has been described, but the present embodiment is not limited thereto. As other feature quantities, local feature quantities or feature quantities relating to colors such as histograms may be used. For example, the search processing unit 22 may divide and compare the image information to be compared for each type of food and drink included in the set. Specifically, the image information to be compared may be divided for each dish, and the image information may be compared for each dish. The set reference information in this case is an entry (hereinafter, for distinction) in which image information divided for each dish is associated with set identification information (hereinafter referred to as individual identification information for distinction) that specifies food and drink for each dish. And an entry including a set of individual identification information for specifying a plurality of foods and beverages (hereinafter referred to as an identification information entry for distinction).
　またこの分割は例えば利用者から各皿が撮像されている領域について、制御部１１がそれぞれ一点ずつ指定を受け、次の処理を行うことで実現できる。すなわち、制御部１１は撮像した画像情報を表示部１４に表示し、この表示された画像情報上で、抽出したい画像部分中の一点のタップを操作部１３から受け入れる。制御部１１は、当該タップされた位置に対応する画像情報上の座標情報を取得して、処理対象座標情報リストとして記録する。この操作を利用者がタップ終了を指示するまで繰り返す。つまり利用者は、各皿についてその皿が写っている領域の内部の点を一つずつ指定し、すべての皿の領域の指定を終了すると、その旨の入力を行うこととする。 Further, this division can be realized, for example, when the control unit 11 receives a designation for each of the areas where each dish is imaged from the user and performs the following processing. That is, the control unit 11 displays the captured image information on the display unit 14, and accepts a single tap from the operation unit 13 in the image portion to be extracted on the displayed image information. The control unit 11 acquires coordinate information on the image information corresponding to the tapped position and records it as a processing target coordinate information list. This operation is repeated until the user instructs the end of the tap. In other words, the user designates one point inside the area in which the dish is shown for each dish, and when the designation of all the dish areas is completed, the user inputs that point.
　利用者が点の指定を終了した旨の操作を行うと、制御部１１は、処理対象座標情報リストに含まれる座標情報を参照し、利用者の指示操作により受け入れた点ごとに、当該点を含む所定形状の領域を画像情報内に形成する。ここでは所定形状は例えば円であるとする。また、各領域は重なり合わずに互いに接する大きさ以下の大きさとする。 When the user performs an operation indicating that the point has been specified, the control unit 11 refers to the coordinate information included in the processing target coordinate information list, and sets the point for each point received by the user's instruction operation. A region having a predetermined shape is formed in the image information. Here, it is assumed that the predetermined shape is, for example, a circle. In addition, each region has a size not larger than the size in contact with each other without overlapping.
　制御部１１は、形成した領域を、前景を表す初期領域としてグラブカットなどのプログラムを利用してセグメンテーション処理を実行する。そしてセグメンテーションの処理により得られた画像情報のうち、前景となる部分画像を処理の対象として取り出す。 The control unit 11 performs a segmentation process using a program such as a grab cut with the formed area as an initial area representing the foreground. Then, out of the image information obtained by the segmentation process, a partial image serving as a foreground is extracted as a processing target.
　このセグメンテーション処理では、制御部１１は、処理対象座標情報リストに含まれる座標情報Ｐiについて確定した円形状の領域（区別のため外周領域と呼ぶ）内に、さらに当該座標情報Ｐiによって表される座標を含む小領域をさらに形成する。この小領域は具体的に、確定した外周領域の形状と相似の形状であって、予め定めた大きさのものとすればよい。なお、この予め定めた大きさによって確定される小領域が、確定した外周領域からはみ出る場合は、外周領域と小領域とが重なりあっている（いわばそれらの論理積によって確定される）領域を小領域とする。 In this segmentation process, the control unit 11 further includes coordinates represented by the coordinate information Pi in a circular area (referred to as an outer peripheral area for distinction) determined for the coordinate information Pi included in the processing target coordinate information list. A small region including is further formed. Specifically, the small region may have a shape similar to the shape of the determined outer peripheral region and have a predetermined size. In addition, when the small area determined by the predetermined size protrudes from the determined outer peripheral area, the area where the outer peripheral area and the small area overlap (to be determined by their logical product) is reduced. This is an area.
　そして制御部１１は、この小領域内にある画素はすべて前景を構成する画素であるとしてセグメンテーションの処理を実行する。具体的にグラブカット（C. Rother, V. Kolmogorov, and A. Blake, GrabCut: Interactive foreground extraction using iterated graph cuts, ACM Trans. Graph., vol. 23, pp. 309-314, 2004）を用いる場合、外周領域を初期領域として自動セグメンテーションを行った後、この小領域が利用者から前景として指定された領域（上記論文においてαn＝１（foreground brush）とされる領域）であるとして、再度セグメンテーションの評価を実行する。 Then, the control unit 11 executes the segmentation process on the assumption that all the pixels in the small area are pixels constituting the foreground. Specifically when using grab cuts (C. Rother, V. Kolmogorov, and A. Blake, GrabCut: Interactive foreground extraction using iterated graph cuts, ACM Trans. Graph., Vol. 23, pp. 309-314, 2004) After performing the automatic segmentation with the outer peripheral region as the initial region, this small region is again designated as a foreground region (region in which αn = 1 (foreground brush) in the above paper). Perform the evaluation.
　これにより、皿ごとの画像がセグメントとして分割されることとなる。制御部１１は、各皿の画像情報を抽出し、今回撮像された皿ごとの画像情報に類似する画像情報を、過去に撮影され、皿ごとに分割されて記録されたセット参照情報の個別参照情報に含まれる画像情報のうちから検索する。そして今回撮像された皿ごとの画像情報についてそれぞれ、過去に撮像された皿ごとの画像情報のうち最も類似すると判断される画像情報を検索する。この検索は例えばヒストグラムを利用する処理や、ＳＩＦＴ特徴量を利用する処理など広く知られた種々の画像検索の方法を採用して行うことができる。 This will divide the image for each dish into segments. The control unit 11 extracts the image information of each dish, and individually references the set reference information that has been imaged in the past and divided and recorded for each dish. Search from image information included in the information. Then, for the image information for each dish imaged this time, the image information determined to be the most similar among the image information for each dish imaged in the past is searched. This search can be performed by adopting various well-known image search methods such as a process using a histogram and a process using a SIFT feature amount.
　そして制御部１１は、今回撮像された皿ごとの画像情報についてそれぞれ検索された、過去に撮像された皿ごとの画像情報のうち最も類似すると判断される画像情報のそれぞれに関連付けられている個別識別情報を取得する。これにより、今回撮像された皿ごとの画像情報について、過去に記録された画像情報を元に、個別識別情報が得られることとなる。 Then, the control unit 11 retrieves the image information for each dish captured this time, and identifies each of the image information determined to be the most similar among the image information for each dish imaged in the past. Get information. Thereby, the individual identification information is obtained based on the image information recorded in the past for the image information for each dish imaged this time.
　制御部１１は、今回撮像された皿ごとの画像情報について得られた個別識別情報の組に最も多く一致する個別識別情報を含む、識別情報エントリーのセット参照情報を記憶部１２から検索する。この例では、利用者は、複数の皿の料理を含むセットを撮像したときに、各皿の画像情報について、各画像情報に写っている飲食物を特定する情報を入力するとともに、これらの飲食物のセット全体のメニューの名称を入力しておく。例えば利用者は、カレーと、サラダと、飲み物からなるセットを撮像し、情報処理装置１に対してカレーの皿と、サラダの皿と、飲み物の容器の画像情報とに切り分けさせる。そしてそれぞれの皿（や容器）の料理の名称を個別識別情報として入力する。情報処理装置１は、これによりカレーの皿の画像情報に「カレー」の名称の個別識別情報を関連付けた個別参照情報と、サラダの皿の画像情報に「サラダ」の名称の個別識別情報を関連付けた個別参照情報と、飲み物の容器の画像情報に当該飲み物の名称（「アイスコーヒー」等）を個別識別情報として関連付けた個別参照情報とを記録する。 The control unit 11 searches the storage unit 12 for the set reference information of the identification information entry including the individual identification information that most closely matches the set of individual identification information obtained for the image information for each dish imaged this time. In this example, when a user images a set including dishes of a plurality of dishes, the user inputs information specifying food and drink shown in each image information for the image information of each dish, and Enter the name of the menu for the entire set of objects. For example, the user images a set of curry, salad, and drink, and causes the information processing apparatus 1 to separate the curry dish, salad dish, and drink container image information. And the name of the dish of each plate (or container) is input as individual identification information. Accordingly, the information processing apparatus 1 associates the individual reference information that associates the individual identification information of the name “curry” with the image information of the curry dish and the individual identification information of the name “salad” with the image information of the salad dish. The individual reference information and the individual reference information in which the drink name (such as “ice coffee”) is associated with the image information of the drink container as individual identification information are recorded.
　また、これらの皿を含むセットの名称として「カレーＡセット」のセット識別情報の入力を受け入れ、このセット識別情報に対して、「カレー」、「サラダ」、「アイスコーヒー」の個別識別情報を関連付けたセット参照情報の識別情報エントリーを記録しておく。また他の識別情報エントリーである「パスタＢセット」として、「ミートソーススパゲティ」、「サラダ」、「アイスコーヒー」の個別識別情報を含んだ識別情報エントリーが記録されているものとする。 Moreover, the input of set identification information of “Curry A set” is accepted as the name of the set including these dishes, and individual identification information of “Curry”, “Salad”, and “Ice coffee” is received for this set identification information. The identification information entry of the associated set reference information is recorded. Further, it is assumed that an identification information entry including individual identification information of “meat sauce spaghetti”, “salad” and “ice coffee” is recorded as “pasta B set” which is another identification information entry.
　利用者が再び「カレーＡセット」を注文し、その画像情報の入力を受け入れると、情報処理装置１は、この画像情報を皿（や容器）ごとに切り分け、切り分けた結果の画像情報に最も類似する画像情報を記憶部１２から検索する。ここでは色のヒストグラム等から、過去の「カレー」の皿が写された画像情報と、サラダの皿が写された画像情報と、「アイスコーヒー」入りの容器が写された画像情報とが検索により見出されたものとする（この場合の各画像情報は必ずしも過去のカレーＡセット注文時の画像情報でなくても構わない）。 When the user orders the “curry A set” again and accepts the input of the image information, the information processing apparatus 1 cuts the image information for each plate (or container) and is most similar to the image information of the cut result. The image information to be searched is retrieved from the storage unit 12. Here, from the color histogram, etc., the image information showing the past “curry” dish, the image information showing the salad dish, and the image information showing the container containing “ice coffee” are searched. (Each image information in this case does not necessarily have to be image information at the time of ordering a past curry A set).
　情報処理装置１は、この検索結果より、「カレー」、「サラダ」、「アイスコーヒー」の個別識別情報の組を得る。そしてこの個別識別情報の組に最もよく一致する（最も多く含んだ）セット参照情報の識別情報エントリーを検索する。 The information processing apparatus 1 obtains a set of individual identification information of “curry”, “salad”, and “ice coffee” from this search result. Then, the identification information entry of the set reference information that best matches (includes most) the set of individual identification information is searched.
　具体的に、上述の例であれば、「パスタＢセット」の識別情報エントリーとは「サラダ」、「アイスコーヒー」の二品が一致することとなり（一致飲食物数「２」）、「カレーＡセット」の識別情報エントリーとは、「カレー」、「サラダ」、「アイスコーヒー」の三品が一致することとなる（一致飲食物数「３」）。そこで情報処理装置１は、一致飲食物数の最も大きい識別情報エントリーを参照し、今回撮像された画像情報は「カレーＡセット」の画像情報であると判断する。 Specifically, in the above example, the identification information entry of “pasta B set” matches two items of “salad” and “ice coffee” (number of matched foods “2”), and “curry” In the identification information entry of “A set”, three items “curry”, “salad”, and “ice coffee” match (the number of matched foods “3”). Therefore, the information processing apparatus 1 refers to the identification information entry having the largest number of matched foods and drinks, and determines that the image information captured this time is the image information of “curry A set”.
　この例では、皿ごとに飲食物を判断し、その組み合わせを検索することとしているので、皿の位置が変わってしまっても検索の結果に影響がない。また、セットに含まれる皿の数が多く、一部しか撮像できなかった場合等でも、検索できる可能性が増大する。 In this example, food and drink are judged for each dish, and the combination is searched. Therefore, even if the position of the dish changes, the search result is not affected. In addition, even when the number of dishes included in the set is large and only a part of the dishes can be imaged, the possibility of searching increases.
　さらに、本実施の形態において、設定情報取得部２４は設定情報を記憶部１２から読み出したときに、予め利用者により設定されている、分量に係る情報がある当該設定情報に含まれる場合は、当該分量に係る情報を用いて、取得した設定情報に含まれる設定値における栄養価の情報を補正し、補正した栄養価の情報を出力することとしてもよい。 Furthermore, in the present embodiment, when the setting information acquisition unit 24 reads the setting information from the storage unit 12, when the setting information that is set in advance by the user and includes information related to the quantity is included in the setting information, It is good also as correcting the nutrition value information in the setting value contained in the acquired setting information using the information concerning the amount, and outputting the corrected nutrition value information.
　具体的にこの分量に係る情報は、平均的な分量に対して利用者が当該セット識別情報で識別される飲食物をどの程度の分量だけ飲食するかを設定するものである。一例として、常に７割ほどしか飲食しない場合は０．７、毎回大盛りで飲食する場合は２．０などとしておく。そして設定情報取得部２４は、取得した設定情報に、この分量に関する情報が含まれる場合は、当該設定情報に含まれる栄養価の情報にこの分量に関する情報を乗じて補正し、当該補正後の栄養価の情報を出力する。 Specifically, the information relating to this amount sets how much the user eats or drinks the food and drink identified by the set identification information with respect to the average amount. As an example, 0.7 is set when eating and drinking only about 70% at all times, and 2.0 is set when eating and drinking every time. And when the information regarding this quantity is contained in the acquired setting information, the setting information acquisition part 24 multiplies the information regarding this quantity to the information of the nutrition value contained in the said setting information, and corrects the nutrition after the said correction | amendment. Output price information.
　また本実施の形態の別の態様においては、上記の情報処理装置としての処理をサーバ装置において行ってもよい。図７は、情報処理装置をサーバ装置として実装する場合の例を表す構成ブロック図である。 Further, in another aspect of the present embodiment, the processing as the information processing apparatus may be performed in a server apparatus. FIG. 7 is a configuration block diagram illustrating an example in which the information processing apparatus is implemented as a server apparatus.
　図７に例示するように、サーバ装置としての情報処理装置１′は、制御部１１′と、記憶部１２と、通信部１７とを含んで構成される。ここで既に述べたものと同様の構成をとるものについては同じ符号を付して詳しい説明を省略する。またこの情報処理装置１′には、端末装置２がネットワーク等の通信手段を介して接続される。この端末装置２は例えばスマートフォン等のデバイスであり、カメラ等の撮像手段や、タッチパネルや液晶表示部等のデバイス、さらには携帯電話通信網やネットワークを介して情報処理装置１′との間で通信を行う通信デバイスを含んで構成されている。 As illustrated in FIG. 7, the information processing apparatus 1 ′ as a server apparatus includes a control unit 11 ′, a storage unit 12, and a communication unit 17. Components having the same configurations as those already described are denoted by the same reference numerals, and detailed description thereof is omitted. A terminal device 2 is connected to the information processing apparatus 1 'via a communication means such as a network. The terminal device 2 is, for example, a device such as a smartphone, and communicates with an information processing device 1 ′ via imaging means such as a camera, devices such as a touch panel and a liquid crystal display unit, and a cellular phone communication network or network. It is comprised including the communication device which performs.
　本実施の形態のこの例では、制御部１１′は、記憶部１２に格納されたプログラムに従い、図８に例示するように、機能的に情報受入部２１′と、検索処理部２２と、提示部２３′と、設定情報取得部２４と、情報入力部２５′と、情報記録部２６とを含む装置を実現する。なお、ここでの例でも、既に述べたものと同様の構成をとるものについては同じ符号を付して詳しい説明を省略する。 In this example of the present embodiment, the control unit 11 ′ functionally receives an information receiving unit 21 ′, a search processing unit 22, and a presentation as illustrated in FIG. 8 according to a program stored in the storage unit 12. A device including a unit 23 ′, a setting information acquisition unit 24, an information input unit 25 ′, and an information recording unit 26 is realized. Also in this example, components having the same configuration as those already described are assigned the same reference numerals and detailed description thereof is omitted.
　情報受入部２１′は、利用者を認証する。この認証は例えば利用者に予め設定させた利用者名とパスワードとを入力させる一般的な方法で行うことができる。情報受入部２１′は、認証した利用者の端末装置２において当該利用者が飲食時に入力した、予め定めた種類の情報を受け入れる受入手段として機能する。この情報受入部２１′は、予め定めた種類の情報として、例えば端末装置２のカメラで撮像した画像情報を受け入れることとしてもよい。この場合端末装置２は、当該画像情報に付加したExif情報の一部として、端末装置２の現在位置の情報や撮影日時の情報を併せて情報処理装置１′へ送信し、情報処理装置１′の情報受入部２１′はこれらの情報を画像情報とともに受け入れることとしてもよい。 The information receiving unit 21 ′ authenticates the user. This authentication can be performed by, for example, a general method in which a user name and password set in advance by the user are input. The information receiving unit 21 ′ functions as a receiving unit that receives a predetermined type of information input by the user at the time of eating and drinking in the terminal device 2 of the authenticated user. The information receiving unit 21 ′ may receive, for example, image information captured by a camera of the terminal device 2 as information of a predetermined type. In this case, the terminal apparatus 2 transmits information on the current position of the terminal apparatus 2 and information on the shooting date and time to the information processing apparatus 1 ′ as a part of the Exif information added to the image information. The information receiving unit 21 ′ may receive these pieces of information together with image information.
　検索処理部２２は、情報受入部２１′が情報を受け入れると動作し、情報受入部２１′にて受け入れた情報に類似する飲食時情報を保持手段としての記憶部１２に保持されたセット参照情報のうちから少なくとも一つ検索する検索手段を実現する。 The search processing unit 22 operates when the information receiving unit 21 ′ accepts information, and sets reference information held in the storage unit 12 as holding means that is similar to information received by the information receiving unit 21 ′. A search means for searching for at least one of the above is realized.
　具体的に記憶部１２に格納されているセット参照情報に、飲食時情報として必ず、飲食物の画像情報が記録されているものとすると、この検索処理部２２は情報受入部２１′が受け入れた画像情報に類似する画像情報を含んだセット参照情報を記憶部１２から検索する。この類似する画像情報の検索は、既に述べたようにＳＩＦＴ特徴量を利用する方法であってもよい。また、皿ごとに飲食物を判断し、その組み合わせを検索する、上記情報処理装置１′の例において説明したのと同様の方法も採用できる。この場合に、撮像した画像上で、各皿が撮像されている画像部分中の一点の指定を受け入れる場合は、端末装置２に撮像した画像情報を提示して、各点の指定を受け入れさせ、受け入れた各点の情報を送信させることとしておけばよい。制御部１１′は当該送信された各点の情報を用いてセグメンテーションの処理を行うことができる。 Specifically, if the set reference information stored in the storage unit 12 always includes the image information of the food and drink as the eating and drinking information, the search processing unit 22 is accepted by the information receiving unit 21 ′. Set reference information including image information similar to the image information is searched from the storage unit 12. This similar image information search may be performed by using a SIFT feature amount as described above. Further, a method similar to that described in the example of the information processing apparatus 1 ′ that determines food and drink for each dish and searches for the combination can be employed. In this case, on the captured image, when accepting designation of one point in the image portion where each dish is imaged, the terminal device 2 is presented with the captured image information, and the designation of each point is accepted. What is necessary is just to make it transmit the information of each received point. The control unit 11 ′ can perform segmentation processing using the transmitted information on each point.
　提示部２３′は、検索処理部２２にて見出されたセット参照情報に係るセット識別情報を提示し、利用者の選択を促す提示手段を実現する。この提示部２３′は具体的には、検索処理部２２から入力されるセット参照情報を特定する情報の一覧を生成して端末装置２宛に送出し、端末装置２にて表示させる。 The presentation unit 23 ′ presents the set identification information related to the set reference information found by the search processing unit 22 and realizes a presentation unit that prompts the user to select. Specifically, the presentation unit 23 ′ generates a list of information specifying set reference information input from the search processing unit 22, sends the list to the terminal device 2, and causes the terminal device 2 to display the list.
　この表示は図５に例示したものと同様となる。提示部２３′はさらに検索処理部２２から入力される情報で特定されるセット参照情報に係るセット識別情報に関連付けられた設定値の情報をさらに記憶部１２から読み出し、この設定値の情報（Ｃ）を併せて送出し、端末装置２にて表示させてもよい。 This display is the same as that illustrated in FIG. The presentation unit 23 ′ further reads information on the set value associated with the set identification information related to the set reference information specified by the information input from the search processing unit 22 from the storage unit 12. ) May be sent together and displayed on the terminal device 2.
　またこの提示部２３′は、端末装置２側の利用者が当該一覧のうちから選択したセット参照情報を特定する情報を受信する。さらにこの提示部２３′は、利用者から情報を直接入力する旨の指示が端末装置２にて行われると、その旨の情報を受けて、情報入力部２５′に対して情報の入力を受け入れるよう指示する。 Also, the presenting unit 23 ′ receives information specifying the set reference information selected from the list by the user on the terminal device 2 side. Further, when the terminal device 2 gives an instruction to directly input information from the user, the presenting unit 23 'receives the information to that effect and accepts the information input to the information input unit 25'. Instruct.
　設定情報取得部２４は、提示部２３′の処理において、利用者が選択したセット参照情報に係るセット識別情報を含む設定情報を、保持手段としての記憶部１２から読み出す。そして当該読み出した設定情報に含まれる設定値の情報を取得して出力する。つまり、この設定情報取得部２４は、取得手段及び出力手段を実現するものである。 The setting information acquisition unit 24 reads setting information including set identification information related to the set reference information selected by the user from the storage unit 12 as a holding unit in the process of the presentation unit 23 ′. Then, the setting value information included in the read setting information is acquired and output. That is, the setting information acquisition unit 24 implements acquisition means and output means.
　情報入力部２５′は、端末装置２にセット識別情報の入力を求める画面を表示する。この画面は、図９に例示するように、撮像された画像情報とともに、セット識別情報の入力欄（Ｓ）を表示したものである。またこの画面ではさらに、栄養価の情報等、設定情報として記憶される情報の入力を併せて求めてもよい（Ｖ）。端末装置２にてセット識別情報が入力されて送信されると、情報入力部２５′は、当該送信されたセット識別情報を受け入れて、当該セット識別情報を情報記録部２６に出力する。またここで栄養価の情報等設定情報として記憶されるべき情報が入力された場合、情報入力部２５′は当該情報を情報記録部２６に出力する。 The information input unit 25 ′ displays a screen for requesting input of set identification information on the terminal device 2. As illustrated in FIG. 9, this screen displays a set identification information input field (S) together with the captured image information. Further, on this screen, input of information stored as setting information such as information on nutritional value may be further obtained (V). When the set identification information is input and transmitted in the terminal device 2, the information input unit 25 ′ accepts the transmitted set identification information and outputs the set identification information to the information recording unit 26. When information to be stored as setting information such as nutritional value information is input here, the information input unit 25 ′ outputs the information to the information recording unit 26.
　情報記録部２６は、情報受入部２１′が受け入れた飲食時情報としての画像情報（Exifタグを含む）と、提示部２３′にて受け入れた端末装置２の利用者が選択したセット参照情報に含まれるセット識別情報と、設定情報取得部２４が出力する設定値の情報とを受け入れると、これらの情報と認証した利用者を特定する情報とを互いに関連付けて、記憶部１２に格納する（図１０）。またこの情報記録部２６は、情報受入部２１′が受け入れた画像情報（Exifタグを含む）と、情報入力部２５′が受け入れたセット識別情報とが入力されると、これらの情報と、認証した利用者を特定する情報とを互いに関連付けて、記憶部１２に格納する。これら情報記録部２６によって記録された情報がセット参照情報となる。さらに情報記録部２６は、情報入力部２５′からセット識別情報とともに栄養価の情報等設定情報として記憶されるべきものが入力された場合、当該セット識別情報と当該栄養価の情報等とを関連付けて記憶する The information recording unit 26 includes the image information (including the Exif tag) as the eating and drinking information received by the information receiving unit 21 ′ and the set reference information selected by the user of the terminal device 2 received by the presentation unit 23 ′. When the set identification information included and the setting value information output by the setting information acquisition unit 24 are accepted, the information and the information specifying the authenticated user are associated with each other and stored in the storage unit 12 (FIG. 10). The information recording unit 26 receives the image information (including the Exif tag) received by the information receiving unit 21 ′ and the set identification information received by the information input unit 25 ′. The information specifying the user is associated with each other and stored in the storage unit 12. Information recorded by the information recording unit 26 is set reference information. Further, the information recording unit 26 associates the set identification information with the nutritional value information and the like when information to be stored as the nutritional value information and the like setting information is input together with the set identification information from the information input unit 25 ′. Remember
　この処理により、利用者ごとに飲食した飲食物の情報が蓄積されることとなる。こうして蓄積された情報は、閲覧の処理に供される。具体的に制御部１１′は、端末装置２から入力される指示に従い、認証した利用者の情報に関連付けられたセット参照情報を読み出して当該セット参照情報に含まれる画像情報等の一覧を表示する。また、制御部１１′は、このセット参照情報に含まれるセット識別情報を参照し、当該参照したセット識別情報に関連付けられた栄養価の情報等を読み出して端末装置２に送信し、一覧に含めて表示させてもよい。 This process accumulates information about foods and beverages eaten and consumed for each user. The information accumulated in this way is used for browsing processing. Specifically, in accordance with an instruction input from the terminal device 2, the control unit 11 ′ reads set reference information associated with the authenticated user information and displays a list of image information and the like included in the set reference information. . Further, the control unit 11 ′ refers to the set identification information included in the set reference information, reads out nutrition value information associated with the referenced set identification information, transmits the information to the terminal device 2, and includes it in the list May be displayed.
　またここで述べたように、情報受入部２１′は、利用者を識別する情報を取得する手段として機能する。ここで利用者により予め分量に係る情報の設定を受け入れておき、当該分量に係る情報を記憶部１２に格納しておいてもよい。具体的には利用者は、飲食した飲食物（のセット識別情報）に関連する設定情報に、分量に関する情報を設定しておく。具体的にこの分量に係る情報は、既に述べた例と同様に、平均的な分量に対して利用者が当該セット識別情報で識別される飲食物をどの程度の分量だけ飲食するかを設定するものである。一例として、常に７割ほどしか飲食しない場合は０．７、毎回大盛りで飲食する場合は２．０などとしておく。 As described herein, the information receiving unit 21 'functions as a means for acquiring information for identifying the user. Here, setting of information related to the quantity may be accepted in advance by the user, and the information related to the quantity may be stored in the storage unit 12. Specifically, the user sets information regarding the amount in the setting information related to the food or drink that was eaten or consumed (set identification information thereof). Specifically, the information related to this amount sets how much food or drink the user identifies with the set identification information for the average amount as in the example already described. Is. As an example, 0.7 is set when eating and drinking only about 70% at all times, and 2.0 is set when eating and drinking every time.
　この場合、設定情報取得部２４は、取得した設定情報に、この分量に関する情報が含まれる場合は、当該設定情報に含まれる栄養価の情報にこの分量に関する情報を乗じて補正し、当該補正後の栄養価の情報を出力することとすればよい。 In this case, if the acquired setting information includes information related to the amount, the setting information acquisition unit 24 corrects the nutrition value information included in the setting information by multiplying the information related to the amount by the correction. It is only necessary to output the nutritional value information.
　本実施の形態の各例によれば、複数種類の飲食物を含んだ飲食物のセットであっても、その指定を容易にでき、情報入力の効率を向上できる。 According to each example of the present embodiment, even if it is a set of foods and drinks including a plurality of types of foods and drinks, the designation can be facilitated and the efficiency of information input can be improved.
　さらに本実施の形態のここまでの説明では、制御部１１や制御部１１′の処理における検索処理部２２は、撮像した画像情報に類似する画像情報を検索することとしていたが、これに限られない。例えば、利用者の飲食時情報としての位置情報（Exifに含まれる場合はそれでもよい）を利用してもよい。例えば、取得した現在位置の情報（緯度経度の情報）を参照し、当該参照した緯度経度から予め定めた距離しきい値の範囲内の位置情報を含むセット参照情報を検索することとしてもよい。また、画像情報と位置情報とを利用してもよい。例えば撮像した画像情報に類似する画像情報を含むセット参照情報が複数見出されたときに、撮像した画像情報のExif情報に含まれる位置情報と、見出したセット参照情報に含まれる画像情報のExif情報内の位置情報とを比較し、より位置情報間の距離が短いほど尤もらしいとして一覧を並替えてより上位に表示させるようにしてもよい。 Further, in the above description of the present embodiment, the search processing unit 22 in the processing of the control unit 11 and the control unit 11 ′ searches for image information similar to the captured image information, but is not limited thereto. Absent. For example, position information (when included in Exif, that may be used) as information on eating and drinking of the user may be used. For example, the set reference information including position information within a predetermined distance threshold range from the referenced latitude and longitude may be searched with reference to the acquired current position information (latitude and longitude information). Further, image information and position information may be used. For example, when a plurality of set reference information including image information similar to the captured image information is found, the position information included in the Exif information of the captured image information and the Exif of the image information included in the found set reference information The position information in the information may be compared, and the list may be rearranged to be displayed at a higher level because it is more likely that the distance between the position information is shorter.
　さらに本実施の形態において、セット参照情報の各エントリーについて、各エントリーに含まれる画像情報のExif情報を用いて、各画像情報に撮像されている飲食物を利用者が飲食した時間帯を判別し、「朝食」、「昼食」、「夕食」、「その他」をいずれであるかを識別する情報を生成し、各エントリーに含めて記録してもよい。一例として、午前６時から午前９時の間に撮影された画像情報を含むエントリーについて「朝食」である旨、午前１１時から午後２時までの間に撮影された画像情報を含むエントリーについて「昼食」である旨、午後５時から午後１１時までに撮影された画像情報を含むエントリーについて「夕食」である旨、その他の時間帯について「その他」である旨をそれぞれ表す情報（以下、食事種別情報と呼ぶ）を関連付けて記録しておいてもよい。さらにこの食事種別情報は利用者の指定した任意の文字列であってもよい。具体的に情報受入部２１は利用者が新たに飲食物の情報を入力する際に、この食事種別情報として上記の通り時間帯に基づくデフォルトの文字列を提示する。利用者が当該文字列を修正するべき旨の指示を行うと、情報受入部２１は、利用者から文字列の入力を受け入れ、当該入力された文字列を食事種別情報として用いる。利用者はこれにより、「飲み会」、「立食パーティ」などといった個別の事情を記録できるようになる。 Further, in the present embodiment, for each entry of the set reference information, using the Exif information of the image information included in each entry, the time zone in which the user eats and drinks the food and drink imaged in each image information is determined. , “Breakfast”, “Lunch”, “Dinner”, “Other” may be generated and recorded in each entry. As an example, it is “breakfast” for entries containing image information taken between 6 am and 9 am, and “lunch” for entries containing image information taken between 11 am and 2 pm , Information indicating that it is “dinner” for entries including image information taken from 5:00 pm to 11:00 pm, and “other” for other time zones (hereinafter, meal type information) May be recorded in association with each other. Further, the meal type information may be an arbitrary character string designated by the user. Specifically, when the user newly inputs information on food and drink, the information receiving unit 21 presents a default character string based on the time zone as the meal type information as described above. When the user gives an instruction to correct the character string, the information receiving unit 21 accepts the input of the character string from the user and uses the input character string as the meal type information. As a result, the user can record individual circumstances such as “drinking party” and “buffet party”.
　また本実施の形態の別の例では、情報受入部２１が撮像部１５に対して画像情報の取得を指示するのに代えて、利用者の指示により、撮影を行う（撮像部１５に対して画像情報の取得を指示する）か、既に撮影されて記憶部１２に格納された画像情報を取得するか、または飲食物の名称を入力することとするかを選択可能としてもよい。本実施の形態のこの例では、記憶部１２に予め飲食物名と飲食物の写真と栄養価情報とを互いに関連付けた飲食物データベースを保持しているものとする。ここで栄養価情報は所定単位（例えば１人前）あたりの栄養価情報であり、具体的には１人前あたりの総カロリー値等である。 In another example of the present embodiment, instead of the information receiving unit 21 instructing the imaging unit 15 to acquire image information, shooting is performed according to a user instruction (for the imaging unit 15). It may be possible to select whether to acquire image information), to acquire image information that has already been captured and stored in the storage unit 12, or to input the name of a food or drink. In this example of the present embodiment, it is assumed that a food and drink database that associates food and drink names, food and drink photos, and nutrition value information with each other in advance is stored in the storage unit 12. Here, the nutritional value information is nutritional value information per a predetermined unit (for example, one serving), specifically, a total calorie value per serving or the like.
　ここで撮影を行うことが利用者により選択されたときには、既に述べた通りの動作を行う。また、既に撮影されて記憶部１２に格納された画像情報を取得する旨の選択が行われると、情報受入部２１は記憶部１２に格納されている画像情報の一覧を提示し、利用者に選択を促す。利用者が当該一覧から飲食物の画像情報を選択すると、情報受入部２１は、当該選択された画像情報を検索処理部２２に出力する。 When the user selects to take a picture here, the operation as described above is performed. In addition, when a selection is made to acquire image information that has already been captured and stored in the storage unit 12, the information receiving unit 21 presents a list of image information stored in the storage unit 12 to the user. Encourage selection. When the user selects food and drink image information from the list, the information receiving unit 21 outputs the selected image information to the search processing unit 22.
　なお、この際、検索処理部２２は、当該選択された画像情報に類似する画像情報を見出したとき、選択された画像情報のExif情報に含まれる撮影日時の情報と、類似すると判断された画像情報のExif情報に含まれる撮影日時の情報とを比較し、各撮影日時の情報の差が予め定めたしきい値（例えば「１分」）を下回る場合、重複した画像が既に登録されている旨のエラーを、利用者に報知して処理を中断してもよい。またこの際、処理を中断することなく、さらに登録するかを利用者に問い合せ、利用者が登録することを選択した場合は、見出された画像情報とは異なる画像情報であるとして処理を継続してもよい。 At this time, when the search processing unit 22 finds image information similar to the selected image information, the image determined to be similar to the shooting date and time information included in the Exif information of the selected image information. When the difference between the information on the shooting date and time is less than a predetermined threshold value (for example, “1 minute”) when compared with the information on the shooting date and time included in the Exif information of the information, a duplicate image has already been registered. The error may be notified to the user and the process may be interrupted. Also, at this time, without interrupting the process, the user is inquired whether to register further, and if the user chooses to register, the process is continued assuming that the image information is different from the found image information. May be.
　さらに情報受入部２１は、飲食物の名称を入力することの選択が行われると、名称の入力を利用者に促す。利用者が飲食物の名称を入力すると、情報受入部２１は、予め記憶部１２に保持している飲食物データベースを参照し、当該入力された名称を含む（完全一致または一部に含む）飲食物名を検索する。情報受入部２１は、入力された名称を含む飲食物名に関連付けられたエントリーが見出されると、当該見出した飲食物データベース中のエントリーの一覧を利用者に提示する。 Further, the information receiving unit 21 prompts the user to input a name when selection of inputting the name of the food or drink is made. When the user inputs the name of the food and drink, the information receiving unit 21 refers to the food and drink database stored in the storage unit 12 in advance, and includes the input name (completely included or partially included). Search for a product name. When an entry associated with the food and drink name including the input name is found, the information receiving unit 21 presents a list of entries in the found food and drink database to the user.
　利用者がこの一覧から一つを選択すると、情報受入部２１は分量の入力を促す画面を利用者に提示する。具体的にこの画面は飲食物の分量として例えば何人前の分量であるか、または何グラムか、など所定単位での分量の入力を受ける。利用者がここで分量の情報を入力すると、情報受入部２１は、利用者が選択した飲食物データベースのエントリーに含まれる所定単位あたりの栄養価情報と入力された分量の情報とから、今回利用者が飲食しようとする飲食物の栄養価情報を生成する。さらに情報受入部２１は、図示しないカレンダーチップ等から、当該情報の入力が行われた日時を表す情報を取得してもよい。 When the user selects one from the list, the information receiving unit 21 presents a screen prompting the user to input the quantity. Specifically, this screen receives an input of a quantity in a predetermined unit, such as how many servings or how many grams, as the quantity of food and drink. When the user inputs the amount information here, the information receiving unit 21 uses the nutrition value information per predetermined unit included in the food database entry selected by the user and the input amount information this time. The nutrition value information of the food and drink that the person intends to eat and drink is generated. Furthermore, the information reception part 21 may acquire the information showing the date and time when the information was input from a calendar chip (not shown) or the like.
　そしてこの場合には、情報記録部２６は、情報受入部２１にて利用者から選択された飲食物データベース中のエントリーに含まれる飲食物名と、生成した栄養価情報とを関連付けて、セット識別情報のエントリーの一つとして記憶部１２に格納する。またこのエントリーには、情報受入部２１が取得した日時の情報を含めてもよい。 In this case, the information recording unit 26 associates the food and beverage name included in the entry in the food and beverage database selected by the user in the information receiving unit 21 with the generated nutrition value information, and sets identification. It is stored in the storage unit 12 as one of information entries. The entry may include date and time information acquired by the information receiving unit 21.
　さらに本実施の形態の上述の例において、情報受入部２１は、飲食物の名称の入力を受ける代わりに食事種別情報の入力を受けることとしてもよい。 Furthermore, in the above-described example of the present embodiment, the information receiving unit 21 may receive an input of meal type information instead of receiving an input of food and drink names.
　具体的にこの例の情報受入部２１は、利用者から飲食物の食事種別情報を表す文字列の入力を受け入れる。そして情報受入部２１は、当該入力された文字列に一致ないし当該入力された文字列を一部に含む食事種別情報を、記憶部１２に格納されているセット参照情報から検索する。情報受入部２１は当該検索の結果、セット参照情報から少なくとも一つのエントリーを見出すと、当該見出したエントリーのそれぞれに含まれる栄養価情報をとり出して、これらの平均値（栄養価情報が数値であれば算術平均でよい）を演算する。 Specifically, the information receiving unit 21 in this example receives an input of a character string representing meal type information of food and drink from the user. Then, the information receiving unit 21 searches the set reference information stored in the storage unit 12 for meal type information that matches the input character string or includes the input character string in part. When the information receiving unit 21 finds at least one entry from the set reference information as a result of the search, the information receiving unit 21 extracts the nutrition value information included in each of the found entries, and averages these values (the nutrition value information is a numerical value). Arithmetic average (if any) is calculated.
　情報受入部２１は、当該演算した平均値を表示部１４に表示して利用者に提示する。ここで利用者が当該値を修正するべき旨の操作を行った場合、情報受入部２１は、利用者から栄養価情報の入力を受け入れて情報記録部２６に出力する。また利用者が当該値をそのまま用いるべき旨の操作を行った場合、演算した平均値の値を栄養価情報の値として情報記録部２６に出力する。さらに情報受入部２１は、図示しないカレンダーチップ等から、当該情報の入力が行われた日時を表す情報を取得してもよい。 The information receiving unit 21 displays the calculated average value on the display unit 14 and presents it to the user. When the user performs an operation to correct the value, the information receiving unit 21 accepts input of nutrition value information from the user and outputs it to the information recording unit 26. When the user performs an operation to use the value as it is, the calculated average value is output to the information recording unit 26 as the value of the nutritional value information. Furthermore, the information reception part 21 may acquire the information showing the date and time when the information was input from a calendar chip (not shown) or the like.
　なお、情報受入部２１は、利用者が入力した食事種別情報を含むエントリーがセット参照情報から見出せない場合は、その旨を表示部１４に表示して、利用者に対して栄養価情報を直接入力するよう促すこととしてもよい。ここで利用者が栄養価情報を入力すると、情報受入部２１は、当該入力された栄養価情報を情報記録部２６に出力する。 When the entry including the meal type information input by the user cannot be found from the set reference information, the information receiving unit 21 displays that fact on the display unit 14 and directly provides the nutritional value information to the user. It may be prompted to input. When the user inputs nutrition value information, the information receiving unit 21 outputs the input nutrition value information to the information recording unit 26.
　情報記録部２６は、情報受入部２１にて利用者が入力した食事種別情報と、情報受入部２１が出力する栄養価情報の値とを関連付けて、セット識別情報のエントリーの一つとして記憶部１２に格納する。またこのエントリーには、情報受入部２１が取得した日時の情報を含めてもよい。 The information recording unit 26 associates the meal type information input by the user in the information receiving unit 21 with the value of the nutritional value information output by the information receiving unit 21, and stores it as one of the entries of the set identification information 12. The entry may include date and time information acquired by the information receiving unit 21.
　本実施の形態のこの例によると、飲食物がばらばらに時間を置いて提供されたり、共通の皿から取分けるような場合において、大雑把な栄養価情報を設定しておくことができる。また過去に設定した栄養価情報の平均から、平均的に妥当と考えられる栄養価情報を設定できる。さらにこの例によると、例えば利用者が食事種別情報として「夕食」を入力すれば、過去に夕食として飲食した飲食物の平均の栄養価情報が示されることとなり、詳細な情報の入力時間がとれなかったりする場合にも、過去に設定した栄養価情報の平均から、平均的に妥当と考えられる栄養価情報を記録できる。 According to this example of the present embodiment, rough nutrition value information can be set in the case where food and drink are provided at intervals or separated from a common dish. Moreover, it is possible to set nutrition value information that is considered to be appropriate on average from the average of nutrition value information set in the past. Furthermore, according to this example, for example, if the user inputs “dinner” as the meal type information, the average nutritional value information of foods and drinks that have been eaten and consumed as dinner in the past is shown, and the time for inputting detailed information can be taken. Even if there is not, it is possible to record the nutrition value information that is considered to be appropriate on average from the average of the nutrition value information set in the past.
　さらにここまでの説明では、画像情報の類似を判断する際にＳＩＦＴ特徴量を用いる例としたが、本実施の形態はこれに限られず、比較の対象となる各画像の特徴量としてのＯＲＢ（E. Rublee, et al., "ORB: An Efficient Alternative to SIFT or SURF," ICCV, 2011）やなどを用いることができる。また、飲食物においては色の重要性が比較的高いので、比較の対象となる各画像のチャネル別の色（Ｒ（Red）チャネルの画像、Ｂ（Blue）チャネルの画像、Ｙ（Yellow）チャネルの画像）や、これらのチャネルの色差の画像（Ｒ－Ｙの画像、Ｂ－Ｙの画像等）について、比較の対象となる各画像の特徴量を演算して用いてもよい。 Further, in the above description, the SIFT feature amount is used when determining the similarity of the image information. However, the present embodiment is not limited to this, and the ORB (the feature amount of each image to be compared) E. Rublee, et al., "ORB: An Efficient Alternative to SIFT or SURF," ICCV, 2011) and so on. Also, since the importance of color is relatively high in foods and drinks, the color (R (Red) channel image, B (Blue) channel image, Y (Yellow) channel) for each image channel to be compared ) And color difference images of these channels (RY image, BY image, etc.), the feature amount of each image to be compared may be calculated and used.
　１，１′　情報処理装置、２　端末装置、１１，１１′　制御部、１２　記憶部、１３　操作部、１４　表示部、１５　撮像部、１６　現在位置取得部、１７　通信部、２１，２１′　情報受入部、２２　検索処理部、２３，２３′　提示部、２４　設定情報取得部、２５，２５′　情報入力部、２６　情報記録部。
  1, 1 'information processing device, 2 terminal device, 11, 11' control unit, 12 storage unit, 13 operation unit, 14 display unit, 15 imaging unit, 16 current position acquisition unit, 17 communication unit, 21, 21 'information Acceptance unit, 22 search processing unit, 23, 23 'presentation unit, 24 setting information acquisition unit, 25, 25' information input unit, 26 information recording unit.

Claims (4)

  1. 　少なくとも一種類の飲食物を含んだ飲食物のセットを過去に飲食した際に取得した飲食時情報と、当該セットを識別するセット識別情報とを関連付けたセット参照情報、及び、前記セット識別情報と栄養価の情報を含む設定値とを関連付けた設定情報を保持する保持手段と、
    　飲食物の飲食時に、予め定めた種類の情報を受け入れる受入手段と、
    　前記受け入れた情報に類似する飲食時情報を前記保持手段に保持されたセット参照情報のうちから少なくとも一つ検索する検索手段と、
    　前記検索により見出されたセット参照情報に係るセット識別情報を提示し、利用者の選択を促す提示手段と、
    　前記提示手段で提示されたセット識別情報のうちから利用者が選択したセット識別情報を含む設定情報を、前記保持手段から読み出し、読み出した設定情報に含まれる設定値の情報を取得する取得手段と、
    　当該取得した設定値の情報を出力する出力手段と、
    　を含む情報処理装置。     The set reference information that associates the eating and drinking information acquired when eating and drinking a set of food and drink containing at least one kind of food and drink in the past, and set identification information that identifies the set, and the set identification information Holding means for holding setting information associated with setting values including nutrition value information;
    Receiving means for accepting a predetermined type of information when eating or drinking food and drink;
    Search means for searching at least one of the set reference information held in the holding means for eating and drinking information similar to the received information;
    Presenting means for presenting set identification information related to the set reference information found by the search, and prompting the user to select,
    Acquisition means for reading setting information including set identification information selected by the user from the set identification information presented by the presenting means from the holding means, and acquiring information on setting values included in the read setting information; ,
    An output means for outputting information of the acquired setting value;
    An information processing apparatus including:
  2. 　請求項１記載の情報処理装置であって、
    　前記保持手段は、飲食時情報として、セットに含まれる各種類の飲食物の画像情報と、当該画像情報に撮影された飲食物を識別する個別識別情報とを関連付けた個別参照情報をさらに保持し、
    　前記保持手段に保持されたセット参照情報には、さらに、当該セットに含まれる種類の飲食物を識別する個別識別情報が関連付けられて保持されており、
    　前記受入手段は、予め定めた種類の情報として飲食物の画像の情報を受け入れ、
    　前記検索手段は、前記受け入れた画像情報に含まれる各種類の飲食物の画像に類似する複数の画像情報を前記保持手段から検索して、当該検索により見出された複数の画像情報のそれぞれに関連付けられた個別識別情報を取得し、当該取得した複数の個別識別情報を含むセット参照情報を検索して、当該検索の結果を出力する情報処理装置。     The information processing apparatus according to claim 1,
    The holding means further holds individual reference information associating image information of each type of food and drink included in the set with individual identification information for identifying the food and drink photographed in the image information as the eating and drinking information. ,
    The set reference information held in the holding means is further held in association with individual identification information for identifying the type of food or drink contained in the set,
    The accepting means accepts information on food and drink images as predetermined types of information,
    The search means searches the holding means for a plurality of pieces of image information similar to images of each type of food and drink contained in the received image information, and sets each of the plurality of pieces of image information found by the search. An information processing apparatus that acquires associated individual identification information, searches for set reference information including the acquired plurality of individual identification information, and outputs a result of the search.
  3. 　請求項１または２記載の情報処理装置であって、
    　利用者を識別する情報を取得する手段をさらに含み、
    　前記保持手段は、設定情報としてさらに、飲食物の分量に関する情報を利用者ごとに保持し、
    　前記出力手段は、前記取得した情報で識別される利用者に係る、前記取得した設定値の飲食物の分量に関する情報を用いて、当該設定値における栄養価の情報を補正し、補正した栄養価の情報を出力する情報処理装置。     The information processing apparatus according to claim 1, wherein
    Further comprising means for obtaining information identifying the user;
    The holding means further holds information on the amount of food and drink for each user as setting information,
    The output means corrects the nutritional value information in the set value using the information related to the amount of food and drink of the acquired set value related to the user identified by the acquired information, and the corrected nutritional value Information processing device that outputs the information.
  4. 　コンピュータを、
    　少なくとも一種類の飲食物を含んだ飲食物のセットを過去に飲食した際に取得した飲食時情報と、当該セットを識別するセット識別情報とを関連付けたセット参照情報、及び、前記セット識別情報と栄養価の情報を含む設定値とを関連付けた設定情報を保持する保持手段と、
    　飲食物の飲食時に、予め定めた種類の情報を受け入れる受入手段と、
    　前記受け入れた情報に類似する飲食時情報を前記保持手段に保持されたセット参照情報のうちから少なくとも一つ検索する検索手段と、
    　前記検索により見出されたセット参照情報に係るセット識別情報を提示し、利用者の選択を促す提示手段と、
    　前記提示手段で提示されたセット識別情報のうちから利用者が選択したセット識別情報を含む設定情報を、前記保持手段から読み出し、読み出した設定情報に含まれる設定値の情報を取得する取得手段と、
    　当該取得した設定値の情報を出力する出力手段と、
    　として機能させるプログラム。
          Computer
    The set reference information that associates the eating and drinking information acquired when eating and drinking a set of food and drink containing at least one kind of food and drink in the past, and set identification information that identifies the set, and the set identification information Holding means for holding setting information associated with setting values including nutrition value information;
    Receiving means for accepting a predetermined type of information when eating or drinking food and drink;
    Search means for searching at least one of the set reference information held in the holding means for eating and drinking information similar to the received information;
    Presenting means for presenting set identification information related to the set reference information found by the search, and prompting the user to select,
    Acquisition means for reading setting information including set identification information selected by the user from the set identification information presented by the presenting means from the holding means, and acquiring information on setting values included in the read setting information; ,
    An output means for outputting information of the acquired setting value;
    Program to function as.
PCT/JP2013/062079 2012-04-27 2013-04-24 Information processing device and program WO2013161879A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2012103504A JP2015132864A (en) 2012-04-27 2012-04-27 information processing apparatus and program
JP2012-103504 2012-04-27

Publications (1)

Publication Number Publication Date
WO2013161879A1 true WO2013161879A1 (en) 2013-10-31

Family

ID=49483187

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/062079 WO2013161879A1 (en) 2012-04-27 2013-04-24 Information processing device and program

Country Status (2)

Country Link
JP (1) JP2015132864A (en)
WO (1) WO2013161879A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004118562A (en) * 2002-09-26 2004-04-15 Fuji Photo Film Co Ltd Method, device and program for supporting calculation of calorie intake
JP2011028382A (en) * 2009-07-22 2011-02-10 Kddi Corp Nutrition management server and nutrition management method for managing nutrition component of meal for each user
JP2011076323A (en) * 2009-09-30 2011-04-14 Toshiba Tec Corp Ordering management device, ordering management system and control program
JP2011170692A (en) * 2010-02-19 2011-09-01 Sharp Corp Device, system and method for managing meal, program, and recording medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004118562A (en) * 2002-09-26 2004-04-15 Fuji Photo Film Co Ltd Method, device and program for supporting calculation of calorie intake
JP2011028382A (en) * 2009-07-22 2011-02-10 Kddi Corp Nutrition management server and nutrition management method for managing nutrition component of meal for each user
JP2011076323A (en) * 2009-09-30 2011-04-14 Toshiba Tec Corp Ordering management device, ordering management system and control program
JP2011170692A (en) * 2010-02-19 2011-09-01 Sharp Corp Device, system and method for managing meal, program, and recording medium

Also Published As

Publication number Publication date
JP2015132864A (en) 2015-07-23

Similar Documents

Publication Publication Date Title
US20120135384A1 (en) Portable terminal, calorie estimation method, and calorie estimation program
WO2013114638A1 (en) Image processing system, image processing device, image processing method, program, and information storage medium
US20150206259A1 (en) Dish remaining amount detection apparatus and dish remaining amount detection method
US10331953B2 (en) Image processing apparatus
JP6425278B2 (en) Product information acquisition system
US10733479B2 (en) System and method for providing a recipe
JP6310599B1 (en) SEARCH DEVICE, SEARCH METHOD, AND SEARCH PROGRAM
US11823042B2 (en) System for measuring food weight
JP2011028382A (en) Nutrition management server and nutrition management method for managing nutrition component of meal for each user
JP2019023829A (en) Information providing system, program, and server
US20190221134A1 (en) Meal Management System
Han et al. Automatic cloud detection for high spatial resolution multi-temporal images
JP6533713B2 (en) IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, PROGRAM, AND RECORDING MEDIUM
JP6459447B2 (en) Product ordering device, product ordering method, product price output device, product price output method and program
JP6468590B2 (en) Health care device and health care program
JP6353118B1 (en) Display program, information providing apparatus, display apparatus, display method, information providing method, and information providing program
JP2018049584A (en) Meal size estimation program, meal size estimation method, and meal size estimation apparatus
WO2013161879A1 (en) Information processing device and program
JP6224947B2 (en) Information system, information processing method, and program
US20210366018A1 (en) Server and method for avoiding registered materials
JP2016180718A (en) Food navigation device
KR102134539B1 (en) Method and system that providing food-sharing economic platform
CN110210920B (en) Meal information recommending method and device
JP2011170797A (en) Order support system and order support method
CN110956075A (en) Image processing apparatus, image processing method, and recording medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13782107

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13782107

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP