JP2015052906A - Image processor and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor and program capable of acquiring a quantity from an object image.SOLUTION: The image processor cuts out a measurement target image V1 as an image of measurement target from a digital image. The image processor cuts out a unit element image V2 as a measurement unit from a digital image. The number of measurement target pixels P1 in the image of measurement target is calculated. The number of unit element pixels P2 in the unit element image V2 is calculated. The number of measurement target pixels P1 is divided by the number of unit element pixels P2 (P1/P2). An estimation value of the measurement target is calculated based on the division result and a conversion chart.

Description

  Embodiments described herein relate generally to an image processing apparatus and a program that can acquire an amount from a target object image.

  Conventionally, in applications that take pictures of meals with a camera such as a smartphone using an image processing device and record nutrition intake, in order to acquire the intake, weighed or recorded manually separately from photography, A method of comparing with the size of tableware and the like taken with a dish is known.

  However, obtaining the intake amount is very troublesome for the user such as weighing. In particular, when manually recording, there is a problem that table size information is required separately, which further increases annoyance.

JP 2012-112855 A

  The problem to be solved by the present invention is to provide an image processing apparatus and program capable of acquiring an amount from a target object image.

  An image processing apparatus according to an embodiment includes a measurement target cutout unit that cuts out a measurement target image corresponding to a measurement target object from a digital image, and a unit element cutout that cuts out a unit element image corresponding to an object serving as a measurement unit from the digital image. A measurement target pixel number calculation unit that calculates the number of pixels of the measurement target image, a unit element pixel number calculation unit that calculates the number of pixels of the unit element image, and the number of pixels of the measurement target image as the unit element. A division unit that divides by the number of pixels of the image; and an object amount estimation unit that calculates an estimation amount of the measurement target object based on a division result by the division unit.

  The program according to the embodiment is a program for controlling an image processing apparatus that acquires an amount from a target object image by a computer, and includes a measurement target extraction unit that extracts a measurement target image corresponding to the measurement target object from a digital image. A unit element cutout unit that cuts out a unit element image corresponding to an object as a measurement unit from the digital image, a measurement target pixel number calculation unit that calculates the number of pixels of the measurement target image, and the number of pixels of the unit element image A unit element pixel number calculation unit that calculates the number of pixels of the measurement target image, a division unit that divides the number of pixels of the measurement target image by the number of pixels of the unit element image, and an estimation amount of the measurement target object based on a division result by the division unit And function as an object amount estimation unit for calculating.

It is a figure which shows the basic composition of the hardware concerning the image processing apparatus of this embodiment. It is a functional block diagram of this embodiment. It is a figure which shows the image before cutting out a measuring object. It is a figure which shows the image of the measurement object cut out. It is a figure for demonstrating the example of a conversion table. It is a flowchart for demonstrating the process example which estimates the object amount of measurement object.

  Hereinafter, embodiments will be described in detail with reference to the drawings.

  A hardware configuration according to the image processing apparatus of this embodiment is shown in FIG. The image processing apparatus includes a control unit 11, a storage unit 12, a user I / F (interface) 13, and a communication / input / output I / F 14. Each unit transmits a bus line BUS for transmitting data, addresses, and control signals to each other. Connected through.

  The control unit 11 has a computer configuration such as a central processing unit (CPU), a read only memory (ROM), and a random access memory (RAM).

  The storage unit 12 is a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage unit 25 stores a control program executed by the control unit 21 in advance.

  The user I / F 13 is an interface that is connected to the display and control unit 11 and performs input settings. The communication / input / output I / F 14 is an interface for inputting image data or data captured by a camera or the like or for communicating with other terminals.

  FIG. 2 shows a functional block configuration of the hardware shown in FIG. The functional blocks include a measurement target cutout unit 21, a unit element cutout unit 22, a measurement target pixel number calculation unit 23, a unit element pixel number calculation unit 24, a division unit 25, an object type identification unit 26, a conversion table supply unit 27, In addition, this is realized by the object amount estimation unit 28. The operations of these units are driven and controlled by the control unit 11.

  The measurement target cutout unit 21 specifies a measurement target region from the digital image input to the input terminal IN. For example, the digital image is acquired by a digital camera (not shown). The measurement target area is cut out as a measurement target image and supplied to the measurement target pixel number calculation unit 23 and the object type identification unit 26, respectively. Rules for cutting out the measurement target area are set in advance through a user I / F (interface) or the like. Or you may make it extract automatically from an input image by the known image recognition technique etc. which were described in Unexamined-Japanese-Patent No. 2012-242908.

  FIG. 3 shows an example of an image before being input to the measurement target cutting unit 21 and cutting out the measurement target. The weighing object cutting unit 21 cuts out the “rice” portion, for example, when measuring the amount of “rice” on the tray. The image of the rice cut out from FIG. 3 is shown in FIG. The rice image cut out by the weighing object cutting unit 21 is output as the weighing object image V1. The measurement target image V1 that is the entire image of the rice is supplied to the object type identification unit 26. Further, the measurement target image V <b> 1 is supplied to the measurement target pixel number calculation unit 23.

  The unit element cutout unit 22 identifies a unit element region from the input image supplied to the input terminal IN, cuts out this region as a unit element image V2, and supplies it to the unit element pixel number calculation unit 24. The example of the cutout of the unit element cutout part 22 here cuts out one grain of rice and outputs this as a unit element image V2.

  How to understand the amount from one grain of rice will be explained. For example, it is based on information indicating how many times the number of pixels constituting miso soup and the number of pixels constituting tonkatsu are larger than the number of pixels for one grain of rice. If you know how many times it is compared to the pixel for one grain of rice, you can see how many grams of miso soup and tonkatsu. It is possible to prepare in advance a conversion table that indicates how many grams of soup is because the number of pixels of soup is several times the number of pixels for one grain of rice. Similarly, in the case of tonkatsu, the number of pixels is several times that of one grain of rice, so how many grams of tonkatsu is, but it can be understood by preparing a conversion table in advance.

  Unit element cut-out rules may be set in advance through a user I / F or the like, or may be automatically extracted from an input image by a known image recognition technique described in the above publication. Or you may cut out by combined use with a user setting and automatic extraction. For example, when a meal photo is handled as an input image, one grain of rice can be used as a unit element. In this case, the unit element image is an image in which only one grain of rice is shown.

  If the unit element cut-out rule is set to use rice grains in advance, the rice grains are detected by first searching for the rice area using image recognition technology such as the above-mentioned gazette. On the other hand, it is performed by extracting the contour of the rice grain by the contour extraction process. This is shown in FIG. By selecting one representative grain from the extracted outline group of rice grains, the unit element image V2 of one rice grain as the unit element pixel P2 is cut out.

  The measurement target pixel number calculation unit 23 counts the number of pixels of the measurement target image V1. That is, the number of pixels in the area is calculated from the information of the measurement target image Vl. Thereby, the measurement target pixel number P1 of the measurement target image V1 is determined. From the output of the measurement target pixel number calculation unit 23, information on the measurement target pixel number P1 is supplied to the division unit 25.

  The unit element pixel count calculation unit 24 is supplied with one grain of rice cut by the unit element cutting unit 22 as a unit element image V2. The unit element pixel number calculation unit 24 calculates the number of pixels of the unit element image V2, that is, the unit element pixel number P2, and supplies it to the division unit 25.

  The division unit 25 calculates the pixel number ratio (P1 / P2) by dividing the measurement target pixel number P1 by the unit element pixel number P2, and supplies the pixel ratio to the object amount estimation unit 28. When P1 / P2 is obtained, the number of pixels corresponding to the unit can be determined. The answer output from the division unit 25 is supplied to the object estimation unit 28 as a pixel number ratio.

  On the other hand, when the cutout image shown in FIG. 4 is supplied, the object type identification unit 26 identifies that the weighing image V1 is an image of cooked rice. It is output that it is rice, not miso soup, not tonkatsu.

  The object type identification unit 26 generates the object type information Io by identifying the type of the object in the measurement target image and the situation in which the object is arranged (arranged or the like) as necessary, and sends it to the conversion table supply unit 27. Supply. As a method for identifying the type of object, an image recognition technique such as the above publication may be applied.

  For example, a template image of rice, a template image of tonkatsu, and a template image of miso soup are prepared in advance. Then, by comparing the measurement target image V1 and the template image, it is assumed that the template with the smallest difference is an image that matches the measurement target image V1. Note that a template image for business use needs to be prepared corresponding to a menu, for example.

  In this way, by matching the measurement target image V1, it is possible to identify and identify the measurement target object. In this case, the object type information output from the object type identification unit 26 is the name of a dish.

  If the dish name is known, the amount of the object can be known if there is a conversion table. Therefore, the object identification information Io output from the object type identification unit 26 is supplied to the conversion supply unit 27.

  The conversion supply unit 27 selects, for example, the conversion table shown in FIG. 5 from a plurality of types of conversion tables prepared in advance based on the object identification information Io, and supplies the conversion table to the object amount estimation unit 28.

  Here, the conversion table is a table in which the pixel number ratio is associated with the object amount to be obtained. An example is shown in FIG. The reason why a conversion table is necessary is, for example, when you want to find the amount of rice served in a rice bowl, you can know only the surface area of the served rice (exactly the area projected on the screen) from the input image. This is because it is necessary to estimate the volume (mass) of the rice based on the surface area of the rice and the shape of the tableware.

  The conversion table is a look-up table for obtaining mass from the ratio of the number of pixels that relatively represents the surface area of the cooked rice. The content of the conversion table varies depending on the type of object and the situation in which it is arranged. This is because even with the same serving surface area, the amount of tableware served differs between the plate and the bowl. Therefore, it is desirable to have as many conversion tables as possible for the number of types of objects and arrangements assumed.

  The object amount estimation unit 28 calculates the amount of the object from the supplied pixel number ratio by referring to the selected and supplied conversion table.

  In the example of the conversion table shown in FIG. 5, the pixel number ratio for one grain of rice is 10000 to 18500 times in increments of 500. For each pixel number ratio, for example, the estimated metric value when the pixel number ratio is 10,000 times is 100 g. As another example, in the case of a metric estimated value with a pixel number ratio of 14000, the amount is 165.7 g.

  As a matter of course, the ratio of the number of pixels is not limited to a good magnification at every 500 times as shown in FIG. If an odd value is input, interpolation is performed. For example, when a pixel number ratio of 10250 is input, it is exactly between 10000 times and 10500 times. Accordingly, the estimated metric value in this case is 103.8 g. This is the function of the object estimation unit 28.

  By performing such processing, the object estimation amount, in this example, the estimated value of the arrangement amount is finally output from the object estimation unit 28.

  The above is a process performed with respect to one measurement object. In the example of FIG. 3, there are roughly five types of objects to be weighed: “rice”, “miso soup”, “tonkatsu”, “vegetables”, and “salad”.

  The explanation is based on an assumed scene in which each amount is measured. In the above description, the estimated value of rice has been described. To measure the amount of “miso soup” in exactly the same way, the image of the miso soup portion is cut out from the input image and the same process as for rice is performed. Of course, the unit element is the same as the rice because it is sufficient if there is information on one grain of rice as before. The same applies to “tonkatsu”, “vegetables” and “salad”.

  With reference to the flowchart shown in FIG. 6, the process for estimating the object amount to be measured in the input image will be further described.

  An input image of an object captured by a camera or the like is acquired from the input terminal IN (ACT1).

  A unit element region is specified from the acquired input image, and this region is cut out as a unit element image V2 (ACT2).

  Next, the number P2 of unit element pixels of the cut unit element image V2 is calculated (ACT3). In ACT3, the number of pixels for one grain of rice was calculated. In the case of miso soup, a miso soup image is cut out from the input image and the same processing is performed. In this case, as the number of unit element pixels P2, the number of pixels for one grain of rice is used as in the case of rice.

  A measurement target object region is specified from the input image, and the region is cut out as a measurement target image V1 (ACT4). Here, the number of areas to be cut out is not limited to one, and the number N of cut out areas is held.

  Next, the loop counter i is initialized to [0] (ACT5).

  Of the N images, the (i + 1) -th measurement target image V1 is set as a target of attention (ACT6). Note that the “measurement target image” described in the description after ACT 7 indicates the (i + 1) th measurement target image.

  In ACT7, the measurement target pixel number P1 of the measurement target image V1 is calculated.

  Then, by dividing the measurement target pixel number P1 by the unit element pixel number P2, a pixel number ratio (P1 / P2) relating to the target measurement target image is obtained (ACT8).

  By identifying the type of object represented by the measurement target image V1, object type information Io is generated (ACT9).

  Based on the object type information Io, a conversion table corresponding to this object type stored in advance is selected (ACT 10).

  Next, by referring to the selected conversion table shown in FIG. 5, for example, the amount of the object is calculated based on the pixel number ratio (P1 / P2) (ACT11).

  A process of incrementing the loop counter i by 1 is performed (ACT12).

  Then, in the determination block, it is determined whether or not the loop counter i is less than N (ACT 13). In ACT13, when i is less than N (Yes), the process returns to ACT6 and repeats up to ACT13. On the other hand, in ACT13, when i is not less than N (No), the process ends.

  Hereinafter, a specific process of taking a meal image as an example of an input image and estimating the amount of dishes arranged will be described with reference to the flowchart of FIG. FIG. 3 is used as an example of a meal image taken up in this description. The image in FIG. 3 shows the contents in which rice, soup, main dishes, etc. are arranged on the tray.

  An image of one grain of rice is cut out as a unit element image V1 in ACT12. In ACT13, the number of pixels of one rice grain image is calculated. In ACT 14, the cooking area is cut out as the measurement target image V2. In the case of the image of FIG. 3, five regions such as rice, soup, main dish (fried food), side dish 1 (raw vegetables), side dish 2 (warmed dishes) are extracted.

  Therefore, [5] is held as the value of the number N of regions. In the image of FIG. 3, which cooking region is the first measurement target image depends on the algorithm of ACT 14. The algorithm of this example assumes that the following results (1) to (5) are obtained.

(1) First measurement target image: rice (2) Second measurement target image: soup (3) Third measurement target image: main dish (fried food)
(4) Fourth measurement target image: side dish 1 (raw vegetables)
(5) Fifth measurement object image: side dish 2 (sauce)
For example, the image of the rice portion which is the first measurement target image V1 is as shown in FIG.

  The series of processing from ACT6 to ACT11 is repeated five times from 0 to 4 as the counter i (integer) because the number of areas N is 5. The contents of each process are as follows.

When i = 0 In ACT6, the first weighing target image, that is, the image of the rice portion is set as the target of attention.

  In ACT7, the pixel number P1 of the image of the rice portion is calculated.

  In ACT8, the pixel number ratio (P1 / P2) for the rice portion is obtained by dividing the number of pixels P1 of the image of the rice portion by the number of pixels P2 of the rice grain image.

  In ACT 9, object type information Io representing “rice served in a rice bowl” is generated.

  In ACT10, a conversion table corresponding to “rice served in a rice bowl” is selected.

  In ACT11, by referring to the selected conversion table, based on the pixel number ratio, the amount of “rice served in the rice bowl” that is reflected in the image of the rice portion that is a part of the input image is output. .

When i = 1 In ACT6, the second measurement target image V1, that is, the image of the soup portion is the target of attention.

  In ACT7, the pixel number P1 of the image of the soup portion is calculated.

  In ACT8, the pixel number ratio (P1 / P2) for the soup portion is obtained by dividing the pixel number P1 of the image of the soup portion by the pixel number P2 of the one grain image of rice.

  In ACT 9, object type information Io representing “miso soup arranged in the bowl” is generated.

  In ACT 10, a conversion table corresponding to “miso soup arranged in the bowl” is selected.

  In ACT11, by referring to the selected conversion table, based on the pixel number ratio (P1 / P2), the amount of “miso soup placed in the bowl” reflected in the soup portion that is a part of the input image is calculated. Output.

When i = 2 In ACT6, the third weighing target image V1, that is, the image of the main dish (fried food) portion is set as the target of attention.

  In ACT7, the pixel number P1 of the image of the main dish (fried food) part is calculated.

  In ACT8, the pixel number ratio for the main dish (fried food) portion is obtained by dividing the pixel number P1 of the image of the main dish (fried food) portion by the pixel number P2 of the one grain image of rice.

  In ACT 9, object type information Io representing “tonkatsu arranged on a plate” is generated.

  In ACT10, a conversion table corresponding to “tonkatsu arranged on a plate” is selected.

  In ACT11, by referring to the selected conversion table, based on the pixel number ratio (P1 / P2), the “tonkatsu on the plate” shown in the main dish (fried food) portion which is a part of the input image. Is output.

When i = 3 In ACT6, the fourth measurement target image V1, that is, the image of the side dish 1 (raw vegetable) portion is the target of attention.

  In ACT7, the pixel number P1 of the image of the side dish 1 (raw vegetable) part is calculated.

  In ACT8, the pixel number ratio (P1 / P2) for the side dish 1 (raw vegetable) part is obtained by dividing the pixel number P1 of the side dish 1 (raw vegetable) part by the pixel number P2 of the rice 1 grain image. )

  In ACT 9, object type information Io representing “chopped cabbage on the plate” is generated.

  In ACT 10, a conversion table corresponding to “chopped cabbage on the plate” is selected.

  In ACT11, by referring to the selected conversion table, based on the pixel number ratio (P1 / P2), it is reflected in the input image (the image of the side dish 1 (raw vegetables) part which is a part of the input image). The amount of “chopped cabbage on the plate” is output.

When i = 4 In ACT 6, the fifth measurement target image V 1, that is, the image of the side dish 2 (sauce product) portion is set as the target of attention.

  In ACT7, the number of pixels P1 of the image of the side dish 2 (sauce) portion is calculated.

  In ACT8, the pixel number ratio (P1 / P2) for the side dish 2 (sauce product) part is obtained by dividing the pixel number P1 of the side dish 2 (sauce product) part by the pixel number P2 of the rice 1 grain image. Ask.

  In ACT 9, object type information Io representing “a potato salad on a plate” is generated.

  In ACT 10, a conversion table corresponding to “a potato salad on a plate” is selected.

  In ACT11, by referring to the selected conversion table, based on the pixel number ratio (P1 / P2), the side dish 2 (sauce product) that is a part of the input image is displayed on the plate. The amount of “potato salad” is output.

  As described above, the amount of all items of the meal shown in the input image of FIG. 3 is acquired.

  In this embodiment, since the meal intake can be acquired from only the meal image, meal recording is simplified, and usability of the meal management system is improved.

  In the above embodiment, an example of recording a meal is given, but the present invention is not limited to this. For example, the present invention can be applied as a simple weighing method for measuring and selling products in stores. The unit element image V1 described above is for one grain of rice. If the product sold by weight is, for example, beans, the number of pixels of one bean serving as a reference and a conversion table for this number of pixels are stored in advance. Then, the amount of beans can be estimated by the operator selecting the beans.

  In addition to beans, it can also be used to estimate the quantity of things such as small and large quantities of electronic parts and machine parts.

  Although several embodiments have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, combinations, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

21 Measurement target cutout unit 22 Unit element cutout unit 23 Measurement target pixel number calculation unit 24 Unit element pixel number calculation unit 25 Division unit 26 Object type identification unit 27 Conversion table supply unit 28 Object amount estimation unit

Claims (6)

A measurement target cutout unit that cuts out a measurement target image corresponding to the measurement target object from the digital image;
A unit element cutout unit for cutting out a unit element image corresponding to an object as a unit of measurement from the digital image;
A measurement target pixel number calculating unit for calculating the number of pixels of the measurement target image;
A unit element pixel number calculation unit for calculating the number of pixels of the unit element image;
A division unit for dividing the number of pixels of the measurement target image by the number of pixels of the unit element image;
An image processing apparatus comprising: an object amount estimating unit that calculates an estimated amount of the measurement target object based on a division result by the dividing unit.   The image processing apparatus according to claim 1, wherein the unit of measurement of the unit element cutout unit is selectable from a plurality of units of measurement stored in advance based on a measurement target.   The said object amount estimation part estimated the quantity from the conversion table prepared beforehand based on the object identification information which identifies the kind of object of the measurement object image cut out by the said measurement object cutting part. Image processing device.   The image processing apparatus according to claim 3, wherein the object identification information is acquired by inputting object identification information corresponding to the measurement target image or by recognizing the measurement target image.   The image processing apparatus according to claim 1, wherein the object amount estimation unit repeatedly estimates the number of measurement target objects extracted by the measurement target extraction unit. A program for controlling, by a computer, an image processing apparatus that acquires an amount from a target object image,
A measurement target cutout unit that cuts out a measurement target image corresponding to the measurement target object from the digital image;
A unit element cutout unit for cutting out a unit element image corresponding to an object as a unit of measurement from the digital image;
A measurement target pixel number calculating unit for calculating the number of pixels of the measurement target image;
A unit element pixel number calculation unit for calculating the number of pixels of the unit element image;
A division unit for dividing the number of pixels of the measurement target image by the number of pixels of the unit element image;
A program that functions as an object amount estimation unit that calculates an estimation amount of the measurement target object based on a division result by the division unit.

Images