JP6572815B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method executed by the image processing apparatus.

  In an image processing apparatus such as a copying machine, an image of the original is printed based on image data read from the original. For example, an image processing apparatus capable of adding an advertisement image to a printed matter based on image data of a document is known (see, for example, Patent Document 1).

JP 2001-13817 A

  By the way, in order to reduce food expenses, a user of an image processing apparatus sometimes performs cooking using foods posted on a leaflet of a grocery store such as a supermarket. In this case, the user has to search for a recipe for cooking using the ingredients listed on the flyer.

  An object of the present invention is to provide an image processing apparatus and an image processing method capable of reducing the effort of a user who searches for a recipe for cooking.

  An image processing apparatus according to one aspect of the present invention includes a food material extraction unit, a recipe acquisition unit, and an output processing unit. The said foodstuff extraction part extracts the foodstuff information regarding the specific food material posted on the said flyer based on the image data of the leaflet where the information regarding foodstuff is published. The said recipe acquisition part acquires the recipe information regarding the recipe of the specific dish using the said specific foodstuff based on the said foodstuff information extracted by the said foodstuff extraction part. The output processing unit outputs the recipe information acquired by the recipe acquisition unit.

  An image processing method according to another aspect of the present invention includes a first step, a second step, and a third step. In the first step, based on image data of a flyer on which information on food is posted, food information on the specific food posted on the flyer is extracted. In the second step, recipe information relating to a recipe for a specific dish using the specific food is acquired based on the food information extracted in the first step. In the third step, the recipe information acquired in the second step is output.

  ADVANTAGE OF THE INVENTION According to this invention, the image processing apparatus and image processing method which can reduce the effort of the user who looks for a cooking recipe are implement | achieved.

FIG. 1 is a block diagram showing a configuration of an image processing system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a configuration of an image processing apparatus included in the image processing system according to the embodiment of the present invention. FIG. 3 is a flowchart showing an example of a recipe output process executed in the image processing system according to the embodiment of the present invention. FIG. 4 is a diagram illustrating an example of table data stored in the storage unit of the image processing system according to the embodiment of the present invention. FIG. 5 is a diagram illustrating an example of a setting operation screen displayed on the operation display unit of the image processing system according to the embodiment of the present invention. FIG. 6 is a diagram illustrating an example of a selection operation screen displayed on the operation display unit of the image processing system according to the embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings for understanding of the present invention. In addition, the following embodiment is an example which actualized this invention, Comprising: The thing of the character which limits the technical scope of this invention is not.

[Image processing system 100]
An image processing system 100 according to an embodiment of the present invention includes a mobile terminal 10 and an image processing device 20 as shown in FIG. In the image processing system 100, wireless data communication is executed between the mobile terminal 10 and the image processing apparatus 20.

[Portable terminal 10]
The portable terminal 10 is a mobile communication terminal such as a smartphone or a wearable terminal that is carried by a user of the image processing apparatus 20 or the like. Specifically, the mobile terminal 10 includes a control unit 11, an operation display unit 12, a communication unit 13, and a storage unit 14, as shown in FIG. Note that the mobile terminal 10 may be a mobile phone, a tablet terminal, a notebook computer, a PDA, or the like.

  The operation display unit 12 is an operation of a display unit such as a liquid crystal display that displays various types of information according to control instructions from the control unit 11 and a touch panel that inputs various types of information to the control unit 11 according to user operations. Part.

  The communication unit 13 is a communication interface that can perform wireless data communication with an external communication device such as the image processing apparatus 20. For example, the communication unit 13 performs wireless data communication with the communication device according to a predetermined communication protocol defined by Bluetooth (registered trademark). For example, the communication unit 13 can execute wireless communication by Bluetooth with the communication device existing in a range of about 10 m from the mobile terminal 10 with a radio wave intensity (2.5 mW) called class 2. The communication unit 13 may execute wireless communication with the communication device in accordance with a wireless LAN communication standard.

  Here, in the mobile terminal 10, pairing, which is an authentication procedure for a communication partner in Bluetooth communication, is executed in advance with the image processing apparatus 20. In the pairing, a pairing ID necessary for authentication of a communication partner is exchanged between devices that perform Bluetooth communication. Then, the pairing ID of the image processing apparatus 20 acquired by the portable terminal 10 by the pairing is stored in the storage unit 14. Thereby, the wireless communication by Bluetooth is possible between the portable terminal 10 and the image processing device 20 thereafter.

  The storage unit 14 is a non-volatile storage unit such as a flash memory, and data can be read and written by the control unit 11. The storage unit 14 stores the pairing ID of the image processing apparatus 20.

  The storage unit 14 stores schedule information related to the daily schedule of the user. For example, the schedule information is created or edited in accordance with a user operation on the operation display unit 12 and stored in the storage unit 14. The schedule information is read from the storage unit 14 and displayed on the operation display unit 12 in accordance with a user operation on the operation display unit 12.

  The control unit 11 includes control devices such as a CPU, ROM, RAM, and EEPROM (registered trademark) (not shown). The CPU is a processor that executes various arithmetic processes. The ROM is a non-volatile storage unit in which information such as a control program for causing the CPU to execute various processes is stored in advance. The RAM is a volatile storage unit, and the EEPROM is a non-volatile storage unit. The RAM and the EEPROM are used as a temporary storage memory (working area) for various processes executed by the CPU. In the control unit 11, various control programs stored in advance in the ROM are executed by the CPU. As a result, the mobile terminal 10 is comprehensively controlled by the control unit 11. The control unit 11 may be configured by an electronic circuit such as an integrated circuit (ASIC), and is a control unit provided separately from the main control unit that controls the mobile terminal 10 in an integrated manner. Also good.

  Moreover, the control part 11 contains the acquisition process part 111, as FIG. 1 shows. Specifically, the control unit 11 executes a control program stored in the ROM using the CPU. Thereby, the control unit 11 functions as the acquisition processing unit 111. The control unit 11 may function as the acquisition processing unit 111 by executing an application program installed in the storage unit 14 using the CPU.

  The acquisition processing unit 111 acquires health information related to the user's health status. For example, the health information includes the user's number of steps, pulse, sleep information, and eye movement amount. For example, the sleep information includes a user's bedtime and wake-up time.

  For example, the acquisition processing unit 111 detects the number of steps and the pulse of the user using sensors (not shown) such as an acceleration sensor and an optical sensor mounted on the mobile terminal 10, and stores the detected health information in the storage unit 14. Remember me. For example, the acquisition processing unit 111 counts the number of steps per day of the user detected by the sensor, and stores the counted number of steps of the user in the storage unit 14. In addition, the acquisition processing unit 111 detects the user's pulse at each predetermined timing, and causes the storage unit 14 to store the detected user's pulse.

  In addition, the acquisition processing unit 111 acquires the user's bedtime and wake-up time (the sleep information) in accordance with a user operation, and stores the acquired sleep information in the storage unit 14. The acquisition processing unit 111 determines the user's sleep state (whether it is REM sleep or non-REM sleep, etc.) from the amount of activity during the user's sleeping detected by the sensor, and the determination result is You may memorize | store in the memory | storage part 14 as sleep information.

  In addition, the acquisition processing unit 111 performs wireless data communication with an external spectacle-type terminal capable of detecting eye-gaze movement, acquires the user's eye-gaze movement amount detected by the eyeglass-type terminal, and acquires the acquired user Are stored in the storage unit 14. The acquisition processing unit 111 may acquire the number of steps of the user, the pulse, and the sleep information detected by the detection device by performing wireless data communication with an external detection device.

[Image Processing Device 20]
The image processing apparatus 20 is a multi-function machine having a print function for forming an image based on image data and a plurality of functions such as a scan function, a facsimile function, and a copy function. Specifically, as illustrated in FIGS. 1 and 2, the image processing apparatus 20 includes an ADF 21, an image reading unit 22, an image forming unit 23, a paper feeding unit 24, a control unit 25, an operation display unit 26, and a communication unit 27. And a storage unit 28. Note that the image processing apparatus 20 may be a scanner, a printer apparatus, a facsimile apparatus, a copier, a personal computer, a smartphone, a tablet terminal, or the like.

  The ADF 21 is an automatic document conveyance device that includes a document setting unit, a plurality of conveyance rollers, a document pressing unit, and a paper discharge unit, and conveys a document read by the image reading unit 22. The image reading unit 22 includes a document table, a light source, a plurality of mirrors, an optical lens, and a CCD (Charge Coupled Device), and reads image data from the document.

  The image forming unit 23 forms an image of color or monochrome by an electrophotographic method based on image data read by the image reading unit 22 or image data input from an information processing apparatus such as an external personal computer. It is possible to execute processing (print processing). Specifically, the image forming unit 23 includes a plurality of image forming units corresponding to C (cyan), M (magenta), Y (yellow), and K (black), an optical scanning device (LSU), an intermediate transfer belt, A secondary transfer roller, a fixing device, and a paper discharge tray are provided. In the image forming unit 23, a color or monochrome image is formed on the sheet supplied from the paper feeding unit 24, and the sheet after the image formation is discharged to the paper discharge tray.

  The control unit 25 includes control devices such as a CPU, ROM, RAM, and EEPROM (not shown). The CPU is a processor that executes various arithmetic processes. The ROM is a non-volatile storage unit in which information such as a control program for causing the CPU to execute various processes is stored in advance. The RAM is a volatile storage unit, and the EEPROM is a non-volatile storage unit. The RAM and the EEPROM are used as a temporary storage memory (working area) for various processes executed by the CPU. In the control unit 25, various control programs stored in advance in the ROM are executed by the CPU. As a result, the image processing apparatus 20 is comprehensively controlled by the control unit 25. The control unit 25 may be configured by an electronic circuit such as an integrated circuit (ASIC), and is a control unit provided separately from the main control unit that comprehensively controls the image processing apparatus 20. May be.

  Moreover, the control part 25 contains the authentication process part 250, as FIG. 1 shows. Specifically, the control unit 25 executes a control program stored in the ROM using the CPU. Thereby, the control unit 25 functions as the authentication processing unit 250.

  The authentication processing unit 250 executes a login process for authenticating a user in accordance with a preset authentication operation. For example, the authentication processing unit 250 causes the operation display unit 26 to display an authentication screen on which the authentication operation can be performed as necessary. And the authentication process part 250 receives the input operation of the login information with respect to the operation display part 26 by a user in the said authentication screen. The login information is a user name and a password. That is, the input operation of the login information on the authentication screen is an example of the authentication operation.

  Thereafter, the authentication processing unit 250 determines that the authentication is complete when the login information input from the operation display unit 26 is included in the authentication information stored in the storage unit 28, and sends the user to the image processing apparatus 20. Log in. The authentication processing unit 250 logs out the currently logged-in user when a logout operation is performed on the operation display unit 26, when a preset time has elapsed, or when logging in another user. Execute the process.

  The operation display unit 26 operates a display unit such as a liquid crystal display that displays various types of information according to control instructions from the control unit 25, and a touch panel that inputs various types of information to the control unit 25 according to user operations. Part.

  The communication unit 27 is a communication interface that can perform wired or wireless data communication with an external communication device such as the mobile terminal 10. For example, the communication unit 27 performs wireless data communication with the communication device according to a predetermined communication protocol defined by Bluetooth. For example, the communication unit 27 can execute wireless communication by Bluetooth with the communication device existing in the range of about 10 m from the image processing apparatus 20 with the class 2 radio wave intensity similarly to the mobile terminal 10. Here, in the image processing apparatus 20, the pairing is executed in advance with the mobile terminal 10, and the pairing ID of the mobile terminal 10 is stored in the storage unit 28. Note that the communication unit 27 may execute wired or wireless data communication with the communication device via a communication network such as the Internet or a LAN.

  The storage unit 28 is a storage unit such as a solid state drive (SSD) or a hard disk drive (HDD). The storage unit 28 stores the authentication information of the user. Further, the storage unit 28 stores the pairing ID of the mobile terminal 10. Further, the storage unit 28 stores position information of the image processing apparatus 20.

  Here, in the image processing system 100, the authentication information of the user is registered in advance in association with the pairing ID of the mobile terminal 10 possessed by the user. Specifically, in the storage unit 28 of the image processing apparatus 20, the authentication information and the pairing ID are stored in association with each other. Instead of the pairing ID, the IP address or MAC address of the mobile terminal 10 may be stored in the storage unit 28 in association with the authentication information.

  By the way, the user of the image processing apparatus 20 sometimes performs cooking using foods posted on a leaflet of a grocery store such as a supermarket in order to reduce food costs. In this case, the user has to search for a recipe for cooking using the ingredients listed on the flyer. On the other hand, in the image processing system 100 according to the embodiment of the present invention, it is possible to reduce the user's trouble of searching for a recipe for cooking.

  Specifically, the storage unit 28 stores in advance a recipe output program for causing the CPU of the control unit 25 to execute a later-described recipe output process (see the flowchart of FIG. 3). The recipe output program may be recorded on a computer-readable recording medium such as a CD, a DVD, or a flash memory, and read from the recording medium and installed in the storage unit 28 or the like.

  Then, as shown in FIG. 1, the control unit 25 includes an image acquisition unit 251, a store extraction unit 252, a food material extraction unit 253, a first setting processing unit 254, a fatigue information acquisition unit 255, a determination processing unit 256, and a recipe acquisition. 257, a second setting processing unit 258, and an output processing unit 259. Specifically, the control unit 25 executes the recipe output program stored in the storage unit 28 using the CPU. Thereby, the control part 25 is the image acquisition part 251, the store extraction part 252, the foodstuff extraction part 253, the 1st setting process part 254, the fatigue information acquisition part 255, the determination process part 256, the recipe acquisition part 257, the 2nd setting process. Functions as a unit 258 and an output processing unit 259.

  The image acquisition unit 251 acquires image data of a leaflet on which information on foodstuffs is posted. Here, the leaflet is an advertisement issued by a grocery store such as a supermarket. The flyer includes information about ingredients that are part or all of the product to be posted. For example, the leaflet includes information such as the name of the food, the selling price of the food, the production area of the food, the image of the food, the name of the store that issued the leaflet, and the location of the store. The leaflet may be an electronic leaflet in which information posted on the leaflet is digitized.

  For example, the image acquisition unit 251 acquires the image data of the leaflet from the document of the leaflet using the image reading unit 22. Specifically, when the leaflet document is placed on the document setting unit of the ADF 21, the image acquisition unit 251 is placed on the document setting unit using the ADF 21 and the image reading unit 22. The image data of the leaflet is read from the manuscript of the leaflet. In addition, when the flyer document is placed on the document table of the image reading unit 22, the image acquisition unit 251 uses the image reading unit 22 to read the leaflet placed on the document table. The image data of the leaflet is read from the manuscript.

  The image acquisition unit 251 may acquire the image data of the leaflet by receiving the image data of the leaflet transmitted from an external information processing apparatus such as the mobile terminal 10. Further, the image acquisition unit 251 reads the image data of the flyer by reading the image data of the flyer that is read by the image reading unit 22 or transmitted from an external information processing apparatus and stored in the storage unit 28. You may get it.

  The store extraction unit 252 extracts store information regarding the specific store that issued the leaflet based on the image data of the leaflet acquired by the image acquisition unit 251. Here, the store information includes information such as the name of the specific store and location information. Here, the store extraction unit 252 is an example of a position information extraction unit in the present invention.

  For example, the store extraction unit 252 executes OCR (optical character recognition) processing for recognizing characters included in the image data of the leaflet based on the image data of the leaflet acquired by the image acquisition unit 251. And the store extraction part 252 extracts the said store information based on the execution result of the said OCR process. In addition, when the image acquisition unit 251 acquires image data of each of the plurality of leaflets, the store extraction unit 252 extracts the store information of each of the leaflets.

  The food material extraction unit 253 extracts food material information related to the specific food material posted on the leaflet based on the image data of the leaflet acquired by the image acquisition unit 251. For example, the food material information includes information such as the name, sales price, and production area of the specific food.

  For example, the food material extraction unit 253 extracts the food material information related to each of the plurality of specific food materials posted on the flyer based on the execution result of the OCR process executed by the store extraction unit 252. In addition, when the image acquisition unit 251 acquires image data of each of the plurality of leaflets, the food material extraction unit 253 associates the store information and the food material information extracted from the same leaflet.

  The first setting processing unit 254 sets a first narrowing condition for narrowing down the specific food material according to a user operation. For example, the first narrowing-down condition includes that the production area of the specific food is included in a predesignated area, the specific food does not correspond to a predesignated food, and the flyer that posts the specific food Is issued at the specific store within the range of the distance specified in advance.

  For example, the first setting processing unit 254, when the image data of the leaflet is acquired by the image acquisition unit 251, the setting operation screen used for setting the first narrowing condition and the second narrowing condition described later Is displayed on the operation display unit 26. Then, the first setting processing unit 254 sets the first narrowing condition according to a user operation on the setting operation screen.

  The first narrowing condition may be registered in advance in the image processing apparatus 20 for each user who can log in to the image processing apparatus 20. For example, the first narrowing condition set by the user may be stored in advance in a storage area assigned to the user in the storage unit 28.

  The fatigue information acquisition unit 255 acquires fatigue information related to the user's fatigue state. Here, the fatigue information includes the health information and the schedule information. That is, the health information and the schedule information are examples of fatigue information in the present invention.

  For example, when the user is logged in to the image processing apparatus 20, the fatigue information acquisition unit 255 acquires the health information and the schedule information from the mobile terminal 10 corresponding to the logged-in user. Specifically, the fatigue information acquisition unit 255 acquires the health information and the schedule information from the mobile terminal 10 identified by the pairing ID associated with the authentication information of the logged-in user.

  Further, the fatigue information acquisition unit 255 displays the authentication screen on the operation display unit 26 when the user has not logged in to the image processing apparatus 20. The fatigue information acquisition unit 255 acquires the health information and the schedule information from the portable terminal 10 corresponding to the logged-in user when the user logs in to the image processing apparatus 20 by the authentication operation on the authentication screen.

  The determination processing unit 256 determines whether or not the user is fatigued based on the fatigue information acquired by the fatigue information acquisition unit 255.

  For example, the determination processing unit 256 determines that the user is tired when the number of steps included in the fatigue information exceeds a preset reference number of steps. In addition, the determination processing unit 256 determines that the user is tired when the pulse detected at the predetermined wake-up time included in the fatigue information is equal to or higher than a preset reference pulse rate. In addition, the determination processing unit 256 determines that the user is tired when the sleep time calculated from the sleep information included in the fatigue information is less than a predetermined first reference time. The determination processing unit 256 determines that the user is fatigued when the line-of-sight movement amount included in the fatigue information exceeds a predetermined reference movement amount. The determination processing unit 256 determines that the user is tired when the user's free time indicated by the schedule information included in the fatigue information is less than a preset second reference time. The determination processing unit 256 determines that the user is tired based on any one or more of the number of steps included in the fatigue information, the pulse, the sleep information, the line-of-sight movement amount, and the schedule information. Also good. Further, the determination method by the determination processing unit 256 may be different from the above.

  The recipe acquisition unit 257 acquires recipe information related to a recipe for a specific dish using the specific food, based on the food information extracted by the food extraction unit 253. For example, the recipe information includes the recipe for the specific dish, the ingredient information of the specific ingredient included in the recipe for the specific dish, the store information of the particular store that sells the particular ingredient, and the cost of the particular dish And information such as calorie intake of the specific dish. Further, the recipe information may include an image of the specific dish and an image of the specific food material.

  Specifically, the recipe acquisition unit 257 includes the food material information related to the specific food that satisfies the first narrowing condition set by the first setting processing unit 254 among the food material information extracted by the food material extraction unit 253. To obtain the recipe information. For example, in the image processing apparatus 20, a database including a plurality of cooking recipes is stored in the storage unit 28 in advance. The recipe acquisition unit 257 is configured to satisfy the first refinement condition from the database stored in the storage unit 28 based on the ingredient information related to the particular ingredient that satisfies the first refinement condition. The recipe of the specific dish using is acquired.

  The food extraction unit 253 may determine whether or not the specific food satisfies the first narrowing condition. That is, the food material extraction unit 253 may extract only the food material information related to the specific food material that satisfies the first narrowing condition. The database may be stored in a storage unit included in a predetermined external information processing apparatus. In this case, the recipe acquisition unit 257 acquires the recipe for the specific dish using the specific ingredients from the external information processing device by communicating with the external information processing device.

  For example, the recipe acquisition unit 257 calculates the cost of the specific dish based on the acquired recipe of the specific dish and the food information of the specific food included in the recipe of the specific dish. Here, when the food information is extracted from a plurality of the leaflets, there may be a plurality of the food information related to one specific food contained in the recipe of the specific food. In this case, the recipe acquisition unit 257 acquires the recipe information including the food information with the lowest selling price of the specific food. Note that the recipe acquisition unit 257 may acquire the recipe information including the ingredient information extracted from the leaflet with the closest location of the specific store.

  In addition, the recipe acquisition unit 257 calculates the intake calories of the specific dish based on the acquired recipe of the specific dish. For example, in the image processing device 20, table data in which food and food intake calories are associated in advance is stored in the storage unit 28. FIG. 4 shows an example of the table data stored in the storage unit 28. The recipe acquisition unit 257 refers to the table data stored in the storage unit 28, acquires the intake calories of each of the specific ingredients included in the recipe of the specific dish, and adds them up, thereby adding the recipe of the specific dish. Calculate calorie intake.

  Instead of the storage unit 28, the table data may be stored in a storage unit provided in a predetermined external information processing apparatus. In this case, the recipe acquisition unit 257 acquires the calorie intake of each of the specific ingredients included in the recipe of the specific dish from the external information processing apparatus by communicating with the external information processing apparatus.

  Here, when the determination processing unit 256 determines that the user is tired, the recipe acquisition unit 257 has the closest distance to the specific store among the image data of each of the plurality of leaflets. The recipe information can be acquired based on the food material information extracted from the image data.

  For example, when the determination processing unit 256 determines that the user is tired, the recipe acquisition unit 257 displays a selection operation screen used for selecting whether or not to change the recipe information acquisition method. 26 is displayed. And the recipe acquisition part 257 acquires the said recipe information based on the said foodstuff information extracted from the image data of the said leaflet with the shortest distance to the said specific store according to user operation in the said selection operation screen. .

  For example, the recipe acquisition unit 257 has the closest distance to the specific store based on the store information extracted by the store extraction unit 252 and the position information of the image processing device 20 stored in the storage unit 28. Identify flyer image data. And the recipe acquisition part 257 acquires the recipe of the said specific dish, the cost of the said specific dish, and the calorie intake of the said specific dish based on the said foodstuff information extracted from the image data of the specified said leaflet. Note that the recipe acquisition unit 257 may acquire position information acquired from the mobile terminal 10 by the GPS function in the mobile terminal 10.

  The second setting processing unit 258 sets a second narrowing condition for narrowing down the recipe for the specific dish in accordance with a user operation. For example, in the second narrowing-down condition, the type of the specific dish corresponds to a predesignated type, the cost of the specific dish is equal to or less than a predesignated budget, and the calorie intake of the specific dish is It is included that the number of the specific ingredients included in the specific dish is equal to or greater than the number specified in advance.

  For example, the second setting processing unit 258 sets the second narrowing condition according to a user operation on the setting operation screen. The second narrowing condition may be registered in advance in the image processing apparatus 20 for each user who logs in to the image processing apparatus 20, similarly to the first narrowing condition.

  The output processing unit 259 outputs the recipe information acquired by the recipe acquisition unit 257.

  Specifically, the output processing unit 259 outputs the recipe information related to the recipe of the specific dish that satisfies the second narrowing-down condition set by the second setting processing unit 258. For example, the output processing unit 259 includes the recipe information having the largest number of the specific ingredients included in the recipe of the specific dish among the plurality of recipe information regarding the recipe of the specific dish that satisfies the second narrowing condition. Is output. The output processing unit 259 may output the recipe information with the lowest cost of the specific dish. Further, the output processing unit 259 may output a plurality of the recipe information listed in descending order of the number of the specific ingredients included in the recipe of the specific dish. In addition, the output processing unit 259 may output a plurality of the recipe information listed in the order of low cost of the specific dish.

  The recipe acquisition unit 257 may determine whether or not the recipe for the specific dish satisfies the second screening condition. That is, the recipe acquisition unit 257 may acquire the recipe information regarding the recipe of the specific dish that satisfies the second narrowing-down condition.

  For example, the output processing unit 259 prints or transmits the recipe information to the portable terminal 10 in accordance with a user operation on the operation display unit 26.

[Recipe output processing]
Hereinafter, an example of the procedure of the image processing method of the present invention will be described together with an example of the procedure of the recipe output process executed by the control unit 25 in the image processing apparatus 20 with reference to FIG. Here, steps S11, S12,... Represent the numbers of processing procedures (steps) executed by the control unit 25. The control unit 25 executes the recipe output process when a user operation for instructing the operation display unit 26 to execute the recipe output process is performed.

<Step S11>
First, in step S11, the control unit 25 acquires image data of the leaflet. Here, the process of step S <b> 11 is executed by the image acquisition unit 251 of the control unit 25.

  For example, the control unit 25 uses the image reading unit 22 to acquire the image data of the leaflet from the leaflet manuscript. Specifically, when the leaflet document is placed on the document setting unit of the ADF 21, the control unit 25 uses the ADF 21 and the image reading unit 22 to set the document placed on the document setting unit. The image data of the leaflet is read from the leaflet manuscript. In addition, when the flyer document is placed on the document table of the image reading unit 22, the control unit 25 uses the image reading unit 22 to document the flyer document placed on the document table. Read the image data of the leaflet.

  Further, the control unit 25 may acquire the image data of the leaflet by receiving the image data of the leaflet transmitted from an external information processing apparatus such as the mobile terminal 10. In addition, the control unit 25 acquires the image data of the flyer by reading the image data of the leaflet that is read by the image reading unit 22 or transmitted from an external information processing apparatus and stored in the storage unit 28. May be.

<Step S12>
In step S12, the control unit 25 extracts the store information based on the image data of the leaflet acquired in step S11. Here, the process of step S <b> 12 is executed by the store extraction unit 252 of the control unit 25. Note that the process of step S12 may be omitted.

  For example, the control unit 25 executes the OCR process based on the image data of the leaflet acquired in step S11. And the control part 25 extracts the said shop information based on the execution result of the said OCR process. Moreover, the control part 25 extracts the said shop information of each said leaflet, when the image data of each of the said several leaflet is acquired by step S11.

<Step S13>
In step S13, the control part 25 extracts the said foodstuff information based on the image data of the said leaflet acquired in step S11. Here, the process of step S13 is an example of a first step in the present invention, and is executed by the food material extraction unit 253 of the control unit 25.

  For example, the control unit 25 extracts the food material information related to each of the plurality of specific food materials posted on the flyer based on the execution result of the OCR process executed in step S12. In addition, when the image data of each of the plurality of leaflets is acquired in step S11, the control unit 25 associates the store information and the food material information extracted from the same leaflet.

<Step S14>
In step S <b> 14, the control unit 25 causes the operation display unit 26 to display the setting operation screen.

  An example of the setting operation screen displayed on the operation display unit 26 in step S14 is shown in FIG. The setting operation screen G10 shown in FIG. 5 includes a message display field G11, second narrowing condition setting fields G12 to G15, first narrowing condition setting fields G16 to G18, an output method setting field G19, and operation keys G20 to G21. Is included.

  A message to the user is displayed in the message display field G11. In the second narrowing condition setting fields G12 to G15, the second narrowing condition can be set. For example, the user can designate the type of the specific dish by performing a touch operation on the second screening condition setting field G12 of the setting operation screen G10. Further, the user can designate the budget for the specific dish by performing a touch operation on the second narrowing-down condition setting field G13 on the setting operation screen G10. In addition, the user can designate the calorie intake of the specific dish by performing a touch operation on the second narrowing-down condition setting field G14 on the setting operation screen G10. In addition, the user can specify the number of the specific ingredients included in the specific dish by performing a touch operation on the second narrowing condition setting field G15 of the setting operation screen G10.

  In the first narrowing condition setting fields G16 to G18, the first narrowing condition can be set. For example, the user can specify the production area of the specific food by performing a touch operation on the first narrowing condition setting field G16 of the setting operation screen G10. Further, the user can specify the specific food to be excluded by performing a touch operation on the first narrowing-down condition setting field G17 on the setting operation screen G10. Further, the user can specify the distance to the specific store by performing a touch operation on the first narrowing-down condition setting field G18 on the setting operation screen G10.

  In the output method setting field G19, it is possible to set the output method of the recipe information. For example, the user can set the output method of the recipe information to printing or transmission to the mobile terminal 10 by performing a touch operation on the output method setting field G19 of the setting operation screen G10. The operation keys G20 to G21 are used for confirming and canceling the setting contents on the setting operation screen G10.

<Step S15>
In step S15, the control unit 25 determines whether the user's setting operation on the setting operation screen displayed in step S14 has been completed. For example, when the operation key G20 on the setting operation screen G10 is operated by the user, the control unit 25 determines that the user's setting operation on the setting operation screen has ended.

  Here, when the control unit 25 determines that the user's setting operation on the setting operation screen is completed (Yes in S15), the control unit 25 shifts the process to step S16. If the user's setting operation on the setting operation screen has not ended (No in S15), the control unit 25 waits for the user's setting operation on the setting operation screen to end in step S15.

<Step S16>
In step S <b> 16, the control unit 25 sets the first narrowing condition according to a user operation on the setting operation screen. Here, the process of step S14-step S16 is performed by the 1st setting process part 254 of the control part 25. FIG.

  For example, the control unit 25 sets the first narrowing condition according to the input contents of the first narrowing condition setting fields G16 to G18 when the operation key G20 on the setting operation screen G10 is operated. Thereby, it becomes possible to output the recipe information related to the recipe of the specific dish using the specific food that satisfies the condition desired by the user.

<Step S17>
In step S <b> 17, the control unit 25 sets the second narrowing condition in accordance with a user operation on the setting operation screen. Here, the process of step S <b> 17 is executed by the second setting processing unit 258 of the control unit 25.

  For example, the control unit 25 sets the second narrowing condition according to the input contents of the second narrowing condition setting fields G12 to G15 when the operation key G20 on the setting operation screen G10 is operated. Thereby, it becomes possible to output the recipe information relating to the recipe of the specific dish that satisfies the conditions desired by the user.

  The first narrowing condition and the second narrowing condition may be registered in advance in the image processing apparatus 20 for each user who can log in to the image processing apparatus 20. In this case, the processing of step S14 to step S17 may be omitted.

<Step S18>
In step S18, the control unit 25 acquires the fatigue information. Here, the process of step S <b> 18 is executed by the fatigue information acquisition unit 255 of the control unit 25.

  For example, when the user is logged in to the image processing apparatus 20, the control unit 25 acquires the health information and the schedule information from the mobile terminal 10 corresponding to the logged-in user. In addition, when the user is not logged in to the image processing apparatus 20, the control unit 25 displays the authentication screen on the operation display unit 26. Then, when a user logs in to the image processing apparatus 20 by the authentication operation on the authentication screen, the control unit 25 acquires the health information and the schedule information from the mobile terminal 10 corresponding to the logged-in user.

<Step S19>
In step S19, the control unit 25 determines whether or not the user is tired based on the fatigue information acquired in step S18. Here, the process of step S <b> 19 is executed by the determination processing unit 256 of the control unit 25.

  For example, the control unit 25 determines that the user is tired when the number of steps included in the fatigue information exceeds the reference number of steps. The control unit 25 determines that the user is tired when the pulse detected at the wake-up time included in the fatigue information is equal to or higher than the reference pulse rate. The control unit 25 determines that the user is tired when the sleep time included in the fatigue information is less than the first reference time. The control unit 25 determines that the user is tired when the line-of-sight movement amount included in the fatigue information exceeds the reference movement amount. The control unit 25 determines that the user is tired when the user's free time indicated by the schedule information included in the fatigue information is less than the second reference time.

<Step S20>
In step S20, the control unit 25 determines whether or not it is determined in step S19 that the user is tired.

  If the control unit 25 determines that the user is determined to be tired in step S19 (Yes side of S20), the control unit 25 shifts the process to step S21. If it is not determined in step S19 that the user is tired (No side in S20), the control unit 25 shifts the process to step S201.

<Step S201>
In step S201, the control unit 25 acquires the recipe information based on the food material information extracted in step S13. Here, the process of step S201 is an example of the second step in the present invention, and is executed by the recipe acquisition unit 257 of the control unit 25.

  Specifically, the control unit 25 determines the recipe information based on the ingredient information related to the specific ingredient satisfying the first narrowing condition set in step S16 among the ingredient information extracted in step S13. To get. For example, the control unit 25, based on the food information on the specific food that satisfies the first narrowing condition, from the database stored in the storage unit 28, the specific that satisfies the first narrowing condition The recipe for the specific dish using ingredients is acquired.

  Further, the control unit 25 calculates the cost of the specific dish based on the acquired recipe of the specific dish and the food information of the specific food included in the recipe of the specific dish. Moreover, the control part 25 calculates the intake calories of the said specific dish based on the acquired recipe of the said specific dish. For example, the control unit 25 refers to the table data stored in the storage unit 28, acquires the calorie intake of each of the specific ingredients included in the recipe of the specific dish, and adds up the specific dish. Calorie intake is calculated.

<Step S21>
In step S <b> 21, the control unit 25 displays the selection operation screen on the operation display unit 26.

  An example of the selection operation screen displayed on the operation display unit 26 in step S21 is shown in FIG. The selection operation screen G30 shown in FIG. 6 includes a message display field G31 and operation keys G32 to G33.

  A message to the user is displayed in the message display field G31. The operation keys G32 to G33 are used for selecting whether to change the recipe information acquisition method.

<Step S22>
In step S <b> 22, the control unit 25 determines whether an operation for selecting a change in the recipe information acquisition method has been performed. For example, when the operation key G32 on the selection operation screen G30 is operated by the user, the control unit 25 determines that an operation for selecting a change in the recipe information acquisition method has been performed. On the other hand, when the operation key G33 on the selection operation screen G30 is operated by the user, the control unit 25 determines that an operation for selecting a change in the recipe information acquisition method has not been performed.

  Here, if the control part 25 judges that operation which selects the change of the acquisition method of the said recipe information was performed (Yes side of S22), it will transfer a process to step S23. Moreover, if operation which selects the change of the acquisition method of the said recipe information is not performed (No side of S22), the control part 25 will transfer a process to step S201.

<Step S23>
In step S23, the control unit 25 determines the recipe information based on the ingredient information extracted from the image data of the flyer that is closest to the specific store among the ingredient information extracted in step S13. get. Here, the process of step S <b> 23 is executed by the recipe acquisition unit 257.

  For example, the control unit 25, based on the store information extracted in step S12 and the position information of the image processing device 20 stored in the storage unit 28, the image of the leaflet that is closest to the specific store. Identify the data. And the control part 25 acquires the recipe of the said specific cooking, the cost of the said specific cooking, and the calorie intake of the said specific cooking based on the said foodstuff information extracted in step S13 from the image data of the specified said flyer. To do. Thereby, when the user is tired, it is possible to output the recipe information with a small burden on the user to procure the specific food.

  Note that the processing of step S18 to step S23 may be omitted. In this case, the control unit 25 executes the process of step S201 after the process of step S17.

<Step S24>
In step S24, the control unit 25 outputs the recipe information acquired in step S201 or step S23. Here, the process of step S24 is an example of the third step in the present invention, and is executed by the output processing unit 259 of the control unit 25.

  Specifically, the control unit 25 outputs the recipe information related to the recipe of the specific dish that satisfies the second narrowing-down condition set in step S17. For example, the control unit 25 selects the recipe information having the largest number of the specific ingredients included in the recipe of the specific dish among the plurality of recipe information regarding the recipe of the specific dish that satisfies the second narrowing condition. Output.

  For example, the control unit 25 prints or transmits the recipe information to the portable terminal 10 according to the input content of the output method setting field G19 when operating the operation key G20 on the setting operation screen G10.

  Thus, in the recipe output process, the food material information related to the specific food material posted on the flyer is extracted based on the image data of the flyer. And based on the extracted said ingredient information, the said recipe information regarding the recipe of the said specific dish using the said specific ingredient is acquired and output. Thereby, it is possible to reduce the trouble of the user who searches for a recipe for cooking.

  Note that the output processing unit 259 may output the recipe information that satisfies a preset third narrowing condition when the determination processing unit 256 determines that the user is tired. For example, when the third narrowing-down condition is that the predetermined food includes a predetermined ingredient, the recipe for the specific dish using the specific ingredient that contributes to recovery of user fatigue The recipe information can be output. In addition, when the third narrowing-down condition is that the number of cooking procedures included in the recipe of the specific dish is equal to or less than a predetermined number, the recipe information with a small burden on cooking by the user Can be output.

  In addition, when the determination processing unit 256 determines that the user is tired, the recipe acquisition unit 257 changes the designation of the distance to the specific store in the first narrowing condition to a preset distance. May be. Also in this case, it is possible for the user to output the recipe information with a small burden on procurement of the specific food.

  In addition, the control unit 11 of the mobile terminal 10 includes an image acquisition unit 251, a store extraction unit 252, a food material extraction unit 253, a first setting processing unit 254, a fatigue information acquisition unit 255, a determination processing unit 256, a recipe acquisition unit 257, a first A two setting processing unit 258 and an output processing unit 259 may be provided. In this case, the portable terminal 10 is another example of the image processing apparatus according to the present invention.

10: Mobile terminal 11: Control unit 111: Acquisition processing unit 12: Operation display unit 13: Communication unit 14: Storage unit 20: Image processing device 21: ADF
22: Image reading unit 23: Image forming unit 24: Paper feeding unit 25: Control unit 250: Authentication processing unit 251: Image acquisition unit 252: Store extraction unit 253: Food extraction unit 254: First setting processing unit 255: Fatigue information Acquisition unit 256: Determination processing unit 257: Recipe acquisition unit 258: Second setting processing unit 259: Output processing unit 26: Operation display unit 27: Communication unit 28: Storage unit 100: Image processing system

Claims (6)

Based on the image data of the leaflet on which information on the ingredients is posted, an ingredient extraction unit that extracts the ingredient information on the specific ingredients posted on the flyer,
A recipe acquisition unit that acquires recipe information related to a recipe of a specific dish using the specific food, based on the food information extracted by the food extraction unit;
An output processing unit for outputting the recipe information acquired by the recipe acquisition unit;
Based on the image data of the leaflet, a location information extraction unit that extracts location information of a specific store that issued the leaflet;
A fatigue information acquisition unit for acquiring fatigue information on the user's fatigue state;
A determination processing unit that determines whether or not the user is fatigued based on the fatigue information acquired by the fatigue information acquisition unit;
With
If the recipe acquisition unit determines that the user is tired by the determination processing unit, the image data of the leaflet that is closest to the specific store among the image data of each of the plurality of leaflets An image processing apparatus capable of acquiring the recipe information based on the food material information extracted from the food. A first setting processing unit for setting a first narrowing condition for narrowing down the specific food according to a user operation;
The image processing apparatus according to claim 1, wherein the recipe acquisition unit acquires the recipe information based on the food material information related to the specific food material that satisfies the first narrowing-down condition. A second setting processing unit configured to set a second narrowing condition for narrowing down the recipe of the specific dish in accordance with a user operation;
The image processing apparatus according to claim 1, wherein the output processing unit outputs the recipe information related to a recipe of the specific dish that satisfies the second narrowing-down condition.   The output processing unit outputs the recipe information that satisfies a preset third narrowing condition when the determination processing unit determines that the user is tired. The image processing apparatus according to any one of the above. An image forming unit capable of forming an image based on the image data;
The image processing apparatus according to claim 1, wherein the output processing unit is capable of printing and outputting the recipe information. An image processing method executed by a processor mounted on an image processing apparatus,
Based on image data of a leaflet on which information on ingredients is posted, a first step of extracting ingredient information on specific ingredients posted on the flyer;
A second step of acquiring recipe information relating to a recipe of a specific dish using the specific ingredient based on the ingredient information extracted in the first step;
A third step of outputting the recipe information acquired in the second step;
A fourth step of extracting position information of a specific store that issued the leaflet based on the image data of the leaflet;
A fifth step of acquiring fatigue information about the user's fatigue state;
A sixth step of determining whether or not the user is fatigued based on the fatigue information acquired in the fifth step;
Including
In the second step, when it is determined in the sixth step that the user is tired, the image data of the leaflet that is closest to the specific store among the image data of each of the plurality of leaflets. An image processing method in which the recipe information can be acquired based on the food material information extracted from the food.

Images