JP2020177295A - Beverage container identification device, identification method, and identification program - Google Patents

Abstract

To provide a beverage container identification device and the like capable of easily and reliably identifying the type of a container and serving a beverage in a container suitable for the type of contents.SOLUTION: A service processor 1 includes: a lighting device 2 that illuminates light from a specific direction on a beverage container 4 that contains a beverage and is placed on a predetermined mounting table 141; a photographing device 3 for obtaining a photographed image by photographing the beverage container 4 exposed to the light by the lighting device 2; an extraction unit for extracting a feature image that appears in the photographed image when applying a predetermined process to the photographed image photographed by the photographing device 3 and is formed depending on the correlation between the light of the lighting device 2 and a three-dimensional shape and a surface condition of the beverage container 4; and a determination unit that compares the feature image extracted by the extraction unit with information of a comparative shape being a shape different from the shape that is the same as or similar to the outer shape of the container, and, by using the comparison result between the feature image and the information of the comparative shape, determines whether the beverage container 4 is a transparent container or an opaque container.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for determining characteristics constituting various containers or various objects represented by containers such as cups containing various beverages.

In recent years, self-contained beverage vending machines have become widespread. When using this vending machine, the user purchases a container corresponding to the desired beverage, for example, a cup, and places the cup in the position of the vending machine. Then, the user presses down the beverage selection button corresponding to the purchased cup from the beverage selection buttons provided in the vending machine. This allows the user to receive the desired beverage from the vending machine.

In the following description, the beverage vending machine is also referred to as a beverage providing device. Further, the beverage selection button provided in the beverage providing device is also simply referred to as a selection button. The beverage serving device is, for example, a self-contained coffee machine installed in a convenience store or the like.

When using a beverage serving device, the user may mistakenly select a selection button that is different from the beverage that the purchased cup supports. In order to deal with this, a technique is known in which a beverage serving device determines the type of cup placed in the placement position and automatically provides the corresponding beverage according to the determined cup type. There is.

As a related technique, the image registration unit registers the image of the beverage container taken by the photographing unit in the storage unit in advance. After that, when the purchaser purchases a beverage container and the photographing unit photographs the beverage container, the container recognition unit compares the image information registered in advance in the storage unit with the image information photographed by the photographing unit and drinks. Determine the type of container. A technique is known in which a display unit displays a beverage menu and a beverage size corresponding to the type of beverage container determined by the container recognition unit, and the beverage discharge unit discharges the beverage into the beverage container based on the operation of the purchaser. .. (For example, Patent Document 1).

Another related technology is a method for managing orders and offering goods. Includes requiring the user to select a product type at the order entry station, at least printing a label indicating the selected product type on the container, reading the display at the product offering station, and providing the selected product type to the container. The technique is known (for example, Patent Document 2).

As another related technology, the cup sensor has a cup-type beverage vending machine with the sales port shaped like a box with an open front, a door not shown on the front, and the presence or absence of my cup on the cup stage. Is detected. The cup top sensor detects the top of My Cup. The cup diameter sensor detects the width of my cup. The stage position sensor detects the stop position of the cup stage. The stage elevating device raises and lowers the cup stage. Then, a technique is known in which the user raises and lowers the cup stage according to the height dimension of the my cup placed on the cup stage to stop the upper end of the my cup at a predetermined position (for example, Patent Document 3). ).

As another related technique, a heat insulating chamber containing a beverage tank in which a beverage is stored is provided with an injection nozzle that is communicated with the beverage tank via an injection device. A saucer portion on which a container for receiving a beverage from a pouring nozzle is placed is arranged at the lower part of the heat insulating chamber. A pouring button is provided on the front panel of the insulation. A plurality of sensors for detecting the presence or absence of a container placed on the saucer are connected to the control device. The size of the container placed on the saucer is determined by the detection state of this sensor. There is known a technique in which a preset amount of beverage according to the size of a container is automatically injected into a container when the injection button is pressed (for example, Patent Document 4).

Japanese Unexamined Patent Publication No. 2017-159941 Japanese Unexamined Patent Publication No. 2013-540294 Japanese Unexamined Patent Publication No. 2007-128379 Japanese Unexamined Patent Publication No. 2001-270598

Here, in Patent Document 1, the container type is identified by using information (contour, beverage container length, color, pattern, etc.) based on the image of the beverage container. However, a specific container, for example, a transparent container or a container made of a material having a high surface reflectance, is greatly affected by disturbance due to the container itself or the contents (for example, contained ice). Therefore, there is a problem that the accuracy tends to decrease when determining whether or not the cup is of a type suitable for the beverage that the user wants to purchase. Further, Patent Document 2 has a problem that if the display indicating the product type printed on the container is mistaken, it cannot be detected that the container type is incompatible with the beverage that the user wants to purchase. Further, Patent Documents 3 and 4 have a problem that only the size of the cup can be recognized and it cannot be detected that the material of the cup is incompatible with the beverage that the user wants to purchase.

Therefore, the techniques described in Patent Documents 1 to 4 correctly determine whether or not the container placed at the placement position is suitable for the beverage when a container having different characteristics is used depending on the beverage. There is a problem that it is difficult.

On the other hand, it is also conceivable to use AI (artificial intelligence) or the like to determine whether or not the container is suitable for the beverage that the user desires to purchase. However, AI requires a large amount of resources and a huge amount of data in order to perform calculations, and has a problem that it is not suitable for being incorporated into a beverage providing device or the like.

The present invention has been made in view of such a problem as one aspect of the present invention, and is a container for providing a beverage capable of easily and surely identifying the type of the container and providing a beverage in a container suitable for the type of contents. The subject is to provide an identification device.

One of the identification devices of the beverage providing device disclosed in the present specification is an identification device including a lighting unit, a photographing unit, an extraction unit, and a determination unit. The lighting unit shines light from a specific direction on a container for containing a beverage, which is placed in a predetermined placement position. The photographing unit obtains a photographed image by photographing the container exposed to light by the lighting unit. The extraction unit is a feature formed by subjecting a captured image captured by the photographing unit to a predetermined process and depending on the correlation between the light of the lighting unit and the three-dimensional shape and surface condition of the container appearing in the captured image. Extract the image. The determination unit compares the feature image extracted by the extraction unit with the information of the comparative shape, which is a shape different from the shape that is the same as or similar to the outer shape of the container. Then, the determination unit determines whether the container is a transparent container or an opaque container by using the comparison result of the feature image and the information of the comparative shape.

According to one embodiment, the type of container can be easily and surely identified, and the beverage can be provided in a container suitable for the type of contents.

It is a conceptual diagram which shows one Example of the beverage providing apparatus which concerns on this embodiment. It is a block diagram which shows one Example of the beverage providing apparatus as above. It is a functional block diagram of the serving processing apparatus built in the beverage serving apparatus. It is a figure which shows typically the search information used for the provision processing apparatus which concerns on this embodiment. In the beverage providing device according to this embodiment, a state in which the beverage container is placed on the mounting table (upper row) and a binary image of the captured images for each mounting state (lower row) are schematically shown. It is a figure. In the same beverage serving device, (a) a schematic diagram of a binary image when a large size paper cup (opaque) is placed on the mounting table, and (b) a normal size paper cup (opaque) is placed on the mounting table. It is a schematic diagram of the binary image when it is done. It is the flowchart which shows the processing procedure of the beverage serving apparatus.

1 to 7 show a beverage serving device and a beverage serving container identification device according to this embodiment.

[Configuration of beverage serving device]
First, the beverage providing device and the configuration of this embodiment will be described.

The beverage serving device 100 of this embodiment shown in FIG. 1 recognizes the type and size of the beverage container 4 as a "container" mounted on the mounting table 141 of the beverage serving device 100. Then, the beverage providing device 100 executes a process of providing the beverage according to the type and size of the determined beverage container 4 to the beverage container 4.

In the following description, as an example, it is determined whether the cup-shaped beverage container 4 into which coffee is poured is a paper cup (opaque cup) or a plastic cup (transparent cup), and a beverage corresponding to the beverage container is provided. The processing to be performed will be described.

The opaque beverage container 4 to be determined may be made of a material other than paper, for example, opaque plastic, wood, pottery, ceramic or the like. Further, the opaque beverage container 4 has a low glossiness on the surface, and when the image is illuminated by the lighting device 2 and photographed by the photographing device 3, the light of the lighting device 2 is not reflected on the beverage container 4 of the photographed image. desirable.

On the other hand, the transparent beverage container 4 to be judged may be made of a material other than plastic, for example, glass, and in the case of plastic, it may be made of any resin material such as vinyl chloride, polyethylene, polystyrene, etc. May be good. Further, the transparent beverage container 4 may be translucent. Further, the beverage container 4 to be judged may have a high or low glossiness on the surface. Further, when the transparent beverage container 4 is illuminated by the lighting device 2 and photographed by the photographing device 3, the light of the lighting device 2 may or may not be reflected on the beverage container 4 of the photographed image.

Further, the beverage container 4 to be determined in this embodiment does not have to be physically "transparent" or "opaque", but may have a light reflection position when illuminated by the lighting device 2 and photographed by the photographing device 3. If the reflection states are different, it can be determined. For example, in this embodiment, it is possible to determine the beverage container 4 having a mirror-like surface and being physically opaque, and the beverage container 4 having a non-mirror-like surface and being physically opaque.

FIG. 1 is a diagram showing an embodiment of the beverage providing device 100. With reference to FIG. 1, the beverage providing device 100 of the present invention will explain the outline of the configuration of the beverage container.

The beverage providing device 100 of this embodiment shown in FIG. 1 provides a beverage contained in a beverage container 4 to a user (not shown, hereinafter simply referred to as “user”) or the like who purchases the beverage. It is a device for. The beverage providing device 100 has a function in which a user or the like manually places the beverage container 4 on the mounting table 141 and the user or the like operates to pour the beverage into the beverage container 4 on the mounting table 141.

The beverage serving device 100 includes a display device 110, a first selection button 121 to a fourth selection button 124, a storage unit 130, a mounting table 141, a guide 142, a serving port 150, a photographing hole 160, and a serving process. It includes a device 1, a lighting device 2, and a photographing device 3. The provision port 150 is an example of a beverage supply unit. The lighting device 2 is an example of a lighting unit. The photographing device 3 is an example of a photographing unit.

The display device 110 displays various information such as advertisements, advertisements, and operation screens by using images, moving images, and the like. Further, the display device 110 may be provided with a touch panel and may accept adjustment of the amount of sugar and milk added to coffee, selection of coffee strength, and the like.

The first selection button 121 to the fourth selection button 124 are buttons for displaying the type and size of the beverage for the user to receive the offer. Each of the first button 121 to the fourth button 124 has a built-in lighting device (not shown) such as an LED. Then, the lighting devices (not shown) of the first buttons 121 to the fourth buttons 124 corresponding to the type and size of the beverage container 4 recognized by the providing processing device 1 were lit and placed on the mounting table 141. The user is notified of the type and size of the beverage container 4.

In this embodiment, the first selection button 121 is a button corresponding to a normal size hot coffee SH using a paper cup and indicates that a normal amount of hot coffee is served. The second selection button 122 is a button corresponding to a large-sized hot coffee LH using a paper cup, and indicates that a large amount of hot coffee is provided. The third selection button 123 is a button corresponding to a normal size iced coffee SC using a plastic cup, and indicates that a normal amount of iced coffee is served. The fourth selection button 124 is a button corresponding to a large size iced coffee LC using a plastic cup, and indicates that a small amount of iced coffee is provided. Here, the "normal size" means a size smaller than a large size. Further, the "ordinary amount" indicates that the amount of the beverage to be provided is smaller than that of the "large amount".

The first selection buttons 121 to 4th selection buttons 124 may also have the function of a touch panel type selection button. Then, when the user presses down a desired button among the first selection buttons 121 to the fourth selection buttons 124, the type and size of the beverage container 4 mounted on the mounting table 141 may be accepted. .. Further, the providing processing device 1 executes a process of pouring and providing a beverage into the beverage container 4 according to the type and size selected by the user. In the following description, the first selection buttons 121 to the fourth selection buttons 124 only notify the type and size of the beverage container 4 mounted on the mounting table 141 by lighting the lighting device (not shown). The type and size of the beverage container 4 by pressing shall not be selected.

The storage unit 130 is a substantially box-shaped space provided on the front surface of the beverage serving device 100 and having an open front surface on which the beverage container 4 is arranged, and the user places the beverage container 4 on the bottom surface of the storage unit 130. A mounting table 141 is provided as a “mounting position” for placing. It is desirable that the accommodating portion 130 is provided with a position sensor (not shown) for detecting that the beverage container 4 is placed on the placing table.

The mounting table 141 is a table on which the user places the beverage container 4.

The guide 142 is, for example, a protruding member provided on the mounting table 141 in a substantially "U" -shaped plan view so that the beverage container 4 is placed at a mounting position surrounded by the guide 142. It serves to guide you to. Therefore, the user can place the beverage container 4 in a predetermined mounting position by mounting the beverage container 4 on the mounting table 141 along the guide 142. In the following description, the mounting position surrounded by the guide 142 is simply referred to as a “mounting position”. The guide 142 may have any shape as long as it is suitable for mounting the beverage container 4 at a predetermined position on the mounting table 141. For example, the guide 142 may have a substantially "he" shape in a plan view, a substantially inverted "U" shape in a plan view, or the like.

The serving port 150 is a spout used when the beverage serving device 100 delivers a beverage to the beverage container 4. That is, the beverage serving device 100 pours the beverage of the type corresponding to the beverage container 4 recognized by the serving processing device 1 into the beverage container 4 from the serving port 150 in an amount corresponding to the size.

The photographing hole 160 is a hole provided for the photographing device 3 arranged inside the housing of the beverage serving device 100 to take an image in the accommodating portion 130.

The lighting device 2 is various light emitting devices such as an LED and an incandescent light bulb, and is installed at a position of illuminating the beverage container 4 diagonally downward from above the photographing device 3 as shown in FIG. As a result, when the beverage container 4 is included in the photographed image of the photographing device 3, the lighting device 2 extracts the image of the hyperbola hb and the image of the bow-shaped bm corresponding to the surface shape of the beverage container 4 of the paper cup. Illuminate the beverage container 4. The hyperbola hb is an example of a "feature image" and a "first feature image". The bow-shaped bm is an example of a "feature image" and a "second feature image".

In FIG. 1, DL1 and DL2 schematically show the state and direction of the light flux emitted from the lighting device 2, and as a specific direction, for example, between the lighting device 2 and DL1 and DL2. It shows that the light is shined diagonally downward. Further, the hyperbolic hb and the bow-shaped bm indicate the light reflected by the paper cup being irradiated from the illuminating device 2.

The photographing device 3 is, for example, a photographing device such as a CCD camera, and as shown in FIG. 1, photographs the photographing area including the beverage container 4 placed at the mounting position of the mounting table 141 from the horizontal direction.

In FIG. 1, the lighting device 2 is installed above the photographing device 3, the lighting device 2 irradiates light from diagonally above the beverage container 4, and the photographing device 3 photographs from the side surface of the beverage container 4. Not limited to. That is, if the hyperbolic hb image and the bow-shaped bm image as the above-mentioned "feature image" can be acquired as images that can appropriately determine whether the beverage container 4 is opaque or transparent, the lighting device 2 illuminates the beverage container 4. The specific direction in which the light is applied and the shooting position of the shooting device 3 may be any position. For example, the lighting device 2 may be arranged below the photographing device 3, the lighting device 2 may be arranged on the left side side of the beverage container 4, and the photographing device 3 may be arranged on the back side side of the beverage container 4. Good. Further, the lighting device 2 and the photographing device 3 may be arranged at any position other than the above.

For example, the lighting device 2 may be configured to light up and irradiate the beverage container 4 when it is determined by an optical sensor (not shown) that the beverage container 4 is placed at the placement position. Further, the photographing device 3 may start photographing after the lighting device 2 is turned on when, for example, an optical sensor (not shown) determines that the beverage container 4 is placed at the mounting position. Further, the lighting device 2 and the photographing device 3 may stop the irradiation and the photographing when it is determined by the optical sensor that the beverage container 4 has been removed from the mounting position, respectively.

[Functional block of the control configuration of the beverage serving device]
FIG. 2 is a block diagram showing an embodiment of a control configuration 200 of the beverage serving device 100 built in the beverage serving device 100 of this embodiment. The control configuration 200 schematically shows a configuration for controlling various functions of the beverage providing device 100.

As shown in FIG. 2, the control configuration 200 includes a control circuit 201, a storage device 202, a reading device 203, a storage medium 204, a communication interface (I / F) 205, a network 206, an input / output interface (I / F) 207, and the like. It includes an input device 208, a display control device 209, a detection device 210, a lighting control device 211, an imaging control device 212, and a bus 213. Each of these configurations 2001 to 213 may be realized only by the hardware, may be realized by the cooperation of the hardware and the program, or may be realized only by the arithmetic processing of the program.

The control circuit 201 is a processor such as a CPU that performs various arithmetic processes by executing a program or the like. Then, the control circuit 201 functions as, for example, a control unit 10 described later.

The storage device 202 has a configuration that functions as a processing area for arithmetic processing of the control circuit 201 such as a RAM, and a function as a storage means for storing data, a program, or the like like a ROM or EEPROM. The storage device 202 may store data and programs necessary for controlling the beverage providing device 100 and realizing the functions. Then, the storage device 202 functions as, for example, a storage unit 20 described later.

The reading device 203 has a function of reading data or a program stored in the storage medium 204 by an instruction of the control circuit 201 or the like. The storage medium 204 is a large-capacity storage medium such as a magnetic disk, and is a non-temporary recording medium for recording data and programs necessary for controlling and realizing functions of the beverage providing device 100.

The communication interface 205 performs various processes (data conversion, realization of data communication according to the communication method, etc.) necessary for communication between the inside and the outside of the beverage providing device 100. The network 206 is a communication medium for connecting many communication terminals such as the Internet and WAN and realizing data communication between the communication terminals. The input / output interface 207 performs various processes necessary for inputting / outputting data and signals between the functional means inside the beverage serving device 100.

The input device 208 receives an input of a signal or the like (for example, a signal in which the user selects a specific selection button (not shown)) input to the beverage providing device 100 by the user, and performs various necessary processes. The display control device 209 causes various display devices to perform necessary displays based on the result of arithmetic processing of the control circuit 201 (for example, the first selection buttons 121 to the fourth selection buttons 124 are turned on).

The detection device 210 receives the input of a signal or the like (for example, the detection signal of a specific sensor (not shown) by the user) input to the beverage providing device 100 by the function of the user or the sensor, and performs various necessary processes. .. The lighting control device 211 performs lighting control such as turning on / off the lighting device 2 and adjusting the amount of light. The photographing control device 212 causes the photographing device 3 to perform photography, and performs various processes for acquiring a photographed image obtained as a result of the photographing. The bus 213 has a function of connecting the functional means of the control configuration 200 by bus to communicate data and signals with each other.

[Functional block of provided processing device]
FIG. 3 is a functional block diagram of the serving processing device 1 as a “beverage serving container identification device” built into the beverage serving device 100 of this embodiment. The providing processing device 1 is a functional means mainly realized by the functions of the control configuration 200 shown in FIG. 2, and determines whether the beverage container 4 placed on the mounting table 141 in the beverage providing device 100 is opaque or transparent. It plays a function for judging. Note that, in FIG. 3, the description of the lighting device 2 constituting the providing processing device 1 is omitted.

As shown in FIG. 3, the providing processing device 1 includes a control unit 10 and a storage unit 20. The control unit 10 includes an acquisition unit 11, a generation unit 12, an extraction unit 13, a determination unit 14, and a provision unit 15. The storage unit 20 stores the search information 21 and the threshold information 22.

Then, the providing processing device 1 is connected to a photographing device 3 that captures a photographed image including the beverage container 4 when the beverage container 4 is placed in the mounting position. Actually, the providing processing device 1 is connected to the photographing device 3 via the photographing control device 212 shown in FIG.

The control unit 10 determines whether the beverage container 4 placed on the mounting table 141 is opaque or transparent based on the functions of the acquisition unit 11, the generation unit 12, the extraction unit 13, the determination unit 14, and the providing unit 15.

The acquisition unit 11 acquires a photographed image from the photographing device 3. The acquisition unit 11 acquires, for example, the detection result of the optical sensor that determines whether or not the beverage container 4 (not shown) is placed at the placement position and the size of the beverage container. Then, when the detection result of the optical sensor indicates that the beverage container 4 is placed at the mounting position, the acquisition unit 11 may turn on the lighting device 2 and acquire the photographed image from the photographing device 3.

The generation unit 12 performs a predetermined process on the captured image acquired by the acquisition unit 11 to form an “adjusted image” which is an image suitable for determination by the extraction unit 13 and the determination unit 14, which will be described later. In this embodiment, the generation unit 12 performs a binarization process on the captured image to generate a binary image as an “adjusted image”.

The extraction unit 13 extracts a predetermined preset image necessary for determining whether the beverage container 4 is opaque or transparent by searching the captured image for a region correlating with the search image. Specifically, the extraction unit 13 performs template matching on the binary image of the captured image by using the search image (see FIG. 4, which will be described in detail later). As a result of template matching, the extraction unit 13 extracts an image similar to the search image from the binary image of the captured image (hereinafter, the extracted image is referred to as an "extracted image").

The determination unit 14 makes a predetermined determination on the "extracted image" extracted by the extraction unit 13, and determines the type of the beverage container 4, that is, whether the beverage container 4 is a paper cup (opaque) or a plastic cup (transparent). At the same time, the determination unit 14 determines whether the size of the beverage container 4, that is, a large size or a normal size.

The serving unit 15 executes a process of serving the beverage from the beverage serving device 100 to the beverage container 4 according to the type and size of the cup determined by the determination unit 14.

The storage unit 20 stores information necessary for determining whether the beverage container 4 is opaque or transparent. The storage unit 20 stores one or a plurality of search information 21 (details will be described later in [Principle 1]).

The storage unit 20 stores, for example, threshold information 22 including a threshold used in template matching. The threshold value is lower than the correlation between the search information 21 of the large size paper cup as the beverage container 4 and the binarized image of the photographed image (not shown) of the normal size paper cup, and the paper cup (opaque). It is preferable to set it higher than the correlation between and the plastic cup (transparent). As a result, the determination unit 14 can determine whether the beverage container 4 is a paper cup (opaque) or a plastic cup (transparent) regardless of the size of the beverage container 4.

[Principle of judgment]
Hereinafter, the principle of a predetermined determination as to whether the beverage container 4 is opaque or transparent in the beverage providing device 100 and the serving processing device 1 of this embodiment will be described. In the following principle, the beverage container 4 will be described mainly in the case where the beverage container 4 has an inverted truncated cone shape in which a step is provided between the side surface and the edge of the opening at the upper end. The shape of the beverage container 4 is not limited to this shape, and may be, for example, a substantially cylindrical shape or a substantially polygonal pillar-shaped beverage container 4.

[Principle 1-Generation of adjusted image from captured image]
The generation unit 12 of the providing processing device 1 performs predetermined processing on the captured image captured by the photographing device 3 to generate an adjusted image. In this embodiment, as a predetermined process, the color of the captured image is converted into a binary value of white and black for the purpose of reducing the processing load in the extraction unit 13 and the determination unit 14. In the following description, the binarized captured image is simply referred to as a binary image. However, the image processing performed by the provided processing device 1 here may be any processing other than binarization as long as it can form an image suitable for template matching described later. For example, a contrast-adjusted image may be generated in which the captured image is contrast-adjusted, for example, the brightness of the captured image is changed to make the captured image approximate to a binary image.

[Principle 2 / Template matching]
Then, the extraction unit 13 of the providing processing device 1 uses the template search information 21 stored in the storage unit 20 to perform template matching with the binary image generated by the generation unit 12. The search information 21 includes "comparative shape information" and "first and second comparative shape information".

FIG. 4 is a diagram schematically showing the search information 21 used in this embodiment. The search information 21 stores a search image corresponding to the surface shape of the beverage container 4 of the paper cup having a predetermined shape. In this embodiment, the search information 21 includes, for example, a binary image showing the characteristics of a large size paper cup, as shown in FIG.

The search information 21 shown in FIG. 4 is image information having a shape and size that substantially matches a binary image of a photographed image of a large-sized paper cup (opaque beverage container 4). The search information 21 is formed as a binary image similar to the binary image of the captured image. The search information 21 is "comparative shape information" corresponding to the shape of light reflection on the side surface of the paper cup when the side surface of the paper cup (opaque beverage container 4) is illuminated by the lighting device 2 and photographed by the photographing device 3. And "information on the first comparative shape" include an image of the hyperbolic portion HB which is a substantially hyperbolic shape. The hyperbolic portion HB has a substantially planar curved portion at least in part. Further, the search information 21 is "comparative shape information" and "second comparative shape information" corresponding to the shape of light reflection at the upper end edge portion of the paper cup when similarly photographed by the photographing apparatus 3. Includes an image of the bow-shaped portion BM, which has a substantially linear shape (elongated shape). The image of the bow-shaped portion BM has a substantially bow-shaped curved portion at least in part. The search information 21 may be generated by the above-mentioned contrast adjustment image instead of the binary image.

The extraction unit 13 uses the image of the hyperbolic portion HB of the search information 21 and the image of the bow-shaped portion BM to perform template matching with the binary image generated by the generation unit 12.

Specifically, for example, the extraction unit 13 divides the binary image of the captured image into a plurality of blocks, and collates the image of the hyperbolic portion HB and the image of the bow-shaped portion BM for each block. Then, if there is a portion similar to the image of the hyperbolic portion HB or the image of the bow-shaped portion BM in the binary image of the captured image, the extraction unit 13 extracts that portion and uses it as a “feature image”. The portion extracted as the "feature image" is a white portion of the binary image, such as a portion where the light of the lighting device 2 is reflected on the beverage container 4, a portion where the ambient light enters the photographing device 3, and the like. Almost corresponds to.

[Principle 3: Judgment 1: Judgment of transparency / opacity]
The determination unit 14 of the providing processing device 1 determines whether the beverage container 4 related to the photographed image is a paper cup (opaque) or a plastic cup (transparent) based on the comparison result with the “feature image”.

The determination unit 14 calculates a "correlation value" in which the approximate state of each "feature image" with the image of the hyperbolic portion HB and the image of the bow-shaped portion BM is quantified based on a predetermined standard. For example, the matching status of each "feature image" with the image of the hyperbolic portion HB and the image of the bow-shaped portion BM, the matching degree of the vertical and horizontal lengths, the matching degree of the aspect ratio, etc. is quantified. Let it be a "correlation value". For the quantification of the match status, for example, it is conceivable to use a quantification table included in the threshold value information 22 (a table in which parameters to be quantified and scores set for each degree of match are recorded. Not shown). Be done. However, the quantification may be performed by any other method.

Here, the difference in the "correlation value" depending on whether the beverage container 4 is a paper cup (opaque) or a plastic cup (transparent) will be described.

FIG. 5 schematically shows a mounting state of the beverage container 4 on the mounting table 141 in this embodiment (upper row) and a binary image of captured images for each mounting state (lower row). It is a figure. (A) in the figure is a case where the beverage container 4 is not placed on the mounting table 141. (B) in the figure is a case where a beverage container 4 (beverage container 4a in the figure) of a normal size plastic cup (transparent) is placed on the mounting table 141. (C) in the figure is a case where a large-sized plastic cup (transparent) beverage container 4 (beverage container 4b in the figure) is placed on the mounting table 141. (D) in the figure is a case where a beverage container 4 (beverage container 4c in the figure) of a normal size paper cup (opaque) is placed on the mounting table 141. (E) in the figure is a case where a large-sized paper cup (opaque) beverage container 4 (beverage container 4d in the figure) is placed on the mounting table 141.

In FIG. 5A, the extracted images 41o, 42o, and 43o do not have a portion similar to the image of the hyperbolic portion HB or the image of the bow-shaped portion BM. This is because the beverage container 4 does not exist on the mounting table 141.

Also in FIGS. 5B and 5C, the extracted images 41a, 42a, 43a and the extracted images 41b, 42b, 43b do not have a portion similar to the image of the hyperbolic portion HB or the image of the bow-shaped portion BM. This is because the beverage containers 4 (beverage containers 4a and 4b in the figure) existing on the mounting table 141 are plastic cups and are transparent, and ice is contained in the beverage containers 4. That is, since the beverage container 4 is transparent, the light of the lighting device 2 is transmitted through the beverage container 4 or reflected on the surface of the beverage container 4 to cause disturbance. This is because the ice inside the beverage container 4 is also transparent and has the same effect as that of the beverage container 4.

On the other hand, in FIGS. 5D and 5E, the extracted images 41c, 42c, 43c and the extracted images 41d, 42d, 43d have portions similar to the image of the hyperbolic portion HB and the image of the bow-shaped portion BM. Specifically, the extracted image 41c and the extracted image 41d include an image of a hyperbola hb and an image of a bow-shaped bm, and the shape and size of the image of the hyperbola hb are similar to those of the hyperbola portion HB. This is because the beverage container 4 (beverage container 4c, 4d in the figure) existing on the mounting table 141 is a paper cup and is opaque, and the shape and size of the position where the light of the lighting device 2 of the beverage container 4 is irradiated. This is because the image is reflected in the captured image almost as it is.

As a result, (D) and (E) in FIG. 5 have a large "correlation value", and (A), (B) and (C) in FIG. 5 have a small "correlation value". Therefore, the determination unit 14 verifies the magnitude of the "correlation value" with the beverage container 4 mounted on the mounting table 141. In this verification, in the determination unit 14, the beverage containers 4 (beverage containers 4a and 4b in the figure) of (B) and (C) are plastic cups (transparent), and the beverage containers 4 (D) and (E) (the same figure). Beverage container 4c, 4d) It can be determined that it is a paper cup (opaque).

In this embodiment, when at least one of the image of the hyperbola hb and the image of the bow-shaped bm included in the binary image has a correlation of at least one of the image of the hyperbola portion HB and the image of the bow-shaped portion BM by a threshold value or more. , The determination unit 14 determines that the paper cup is placed on the mounting table 141.

Further, in the determination unit 14, both the image of the hyperbola hb and the image of the bow-shaped bm included in the binary image and the image of the hyperbola HB and the image of the bow-shaped portion BM included in the search information 21 are less than the threshold value. When there is a correlation, it is determined that the plastic cup is placed on the mounting table 141.

In addition, in FIG. 5, as a result of binarization, the image of the bow-shaped bm is shown as having a substantially rectangular shape, but this is an example, and as a result of the binarization, the bow-shaped bm is shown. The image may be formed in a substantially bow shape similar to the bow-shaped portion BM. This also applies to the image of the bow-shaped bm of FIG. 6, which will be described later.

[Principle 4, judgment of size]
FIG. 6A is a schematic diagram of a binary image when a large-sized paper cup (opaque) beverage container 4 is placed on the mounting table 141 (same state as in FIG. 5E). FIG. 6B is a schematic view of a binary image when a beverage container 4 of a normal size paper cup (opaque) is placed on the mounting table 141 (the same state as in FIG. 5D). ..

As shown in FIG. 6, the vertical length L1 of the image of the hyperbola hb shown in (a) and the vertical length L2 of the image of the hyperbola hb shown in (b) are different. When the determination unit 14 determines these based on the image of one hyperbolic unit HB, the numerical range in the case of matching of the threshold information 22 is set to a large numerical range, and both (a) and (b) are determined. The beverage container 4 is determined to be a paper cup. Further, the determination unit 14 can also be set so that the numerical width of the matching range can be divided into a large size of (a) and a normal size of (b).

Further, when the determination unit 14 uses an image of a plurality of hyperbolic unit HBs (for example, an image of a large size hyperbolic unit HB and an image of a normal size hyperbolic unit HB), the numerical value in the case where the threshold information 22 matches. It is desirable to set the range of to a small numerical width.

In order to discriminate the size of the beverage container 4 of the plastic cup (transparent) (see the beverage containers 4a and 4b of FIGS. 5B and 5C), the height detection sensor of the beverage container 4 is provided in the accommodating portion 130. It is also conceivable to provide (not shown) or the like and use the detection result.

[Principle 5 / Provision of beverages]
Then, the providing unit 15 of the providing processing device 1 executes a process of supplying the beverage from the beverage providing device 100 to the beverage container 4 according to the type and size of the determined cup. The provided processing device 1 may determine the size of the cup by using an optical sensor (not shown). As an example, in the beverage serving device 100, when the beverage container 4 is placed in the mounting position, only the first optical sensor in the low position where both the normal size and the large size are detected and the large size are available. A second optical sensor to be detected is provided. Then, when the first optical sensor and the second optical sensor simultaneously detect the beverage container 4, the providing processing device 1 determines that a large-sized cup is placed in the mounting position. Further, when only the first sensor detects the beverage container 4, the providing processing device 1 determines that a cup of a normal size is placed in the mounting position.

As described above, when the photographed image of the beverage container 4 of the paper cup (opaque) is binarized, the beverage providing device 100 has an image of a double curve hb and an image of a bow-shaped bm corresponding to the shape of the paper cup and the irradiation position of light. The position of the lighting device 2 is adjusted to illuminate the beverage container 4 so that the binary image of the above is generated. As a result, the beverage serving device 100 performs template matching using a binary image showing the characteristics of the paper cup (opaque) beverage container 4, and the beverage container 4 is either a paper cup (opaque) or a plastic cup (transparent). Can be accurately determined. Therefore, the beverage container 4 can improve the recognition accuracy of the object.

[Processing procedure]
FIG. 7 is a flowchart showing a processing procedure for providing a beverage in the beverage providing device 100 and the serving processing device 1 according to this embodiment. Hereinafter, the processing procedure of this embodiment will be described with reference to the figure.

First, the user who purchases the beverage places the beverage container 4 on the mounting table 141 of the beverage providing device 100. When the position sensor (not shown) of the beverage serving device 100 detects the placement of the beverage container (“Yes” in elementary step S101), a detection signal is sent to the detection device 210. The detection device 210 outputs a detection signal to the control circuit 201 (control unit 10).

When the control unit 10 receives the detection signal from the detection device 210, the control unit 10 causes the lighting control device 211 to control the lighting device 2 to be turned on (lighting procedure, lighting control procedure). Further, the control unit 10 causes the photographing control device 212 to take a picture of the beverage container 4 using the photographing device 3 and control the acquisition of the photographed image (photographing procedure, photographing control procedure).

When the photographing control device 212 acquires the photographed image of the beverage container 4 photographed by the photographing device 3, the acquisition unit 11 acquires the photographed image (step S102).

Next, the generation unit 12 performs a binarization process on the captured image acquired by the acquisition unit 11 to generate a binary image (step S103).

Next, the extraction unit 13 performs template matching using the search image on the binary image of the captured image (step S104, extraction procedure). Specifically, the extraction unit 13 collates the binary image of the captured image with the image of the hyperbolic portion HB of the search information 21 and the image of the bow-shaped portion BM recorded in the storage unit 20. Then, the extraction unit 13 includes an image of the hyperbola hb corresponding to the surface shape of the side surface of the beverage container 4 and an image similar to the image of the bow-shaped bm corresponding to the surface shape of the edge of the opening of the beverage container. Extract.

The determination unit 14 makes a predetermined determination on the extracted image of the extraction unit 13. Specifically, the "correlation value" is calculated from the extracted image by the method described in the above [Principle 3] and compared with the threshold information 22. When the "correlation value" is the threshold information 22 or more (“Yes” in step S105), the determination unit 14 determines that the beverage container 4 of the paper cup (opaque) is placed on the mounting table 141 (step S106, Judgment procedure). When the "correlation value" is the threshold information 22 or more (“No” in step S105), the determination unit 14 determines that the plastic cup (transparent) beverage container 4 is placed on the mounting table 141 (step S107). , Judgment procedure). In steps S106 and S107, it is also determined whether the beverage container 4 placed on the mounting table 141 has a large size or a normal size.

After the determination in steps S106 and S107, the display control device 209 lights the first selection button 121 to the fourth selection button 124 corresponding to the beverage container 4 determined by the determination unit 14. Further, the providing unit 15 pours a beverage corresponding to the type and size of the beverage container 4 determined by the determining unit 14 into the beverage container 4 from the providing port 150.

[Action and effect of this embodiment]
As described above, in this embodiment, the image of the hyperbola hb and the image of the bow-shaped bm formed depending on the correlation between the light of the lighting device 2 and the three-dimensional shape and the surface state of the beverage container 4 are extracted. , An image of a hyperbola hb and a bow-shaped bm that are clearly different depending on whether the beverage container 4 is opaque or transparent can be extracted. Then, the extracted image of the hyperbola hb and the bow-shaped bm are compared with the image of the hyperbola portion HB and the image of the bow-shaped portion BM having the same shape as the outer shape of the beverage container 4 or a shape different from the similar shape. This makes it possible to determine with high accuracy whether the container is transparent or opaque. Thereby, the type of the beverage container 4 can be easily and surely identified, and the beverage can be provided in the beverage container 4 suitable for the type of the content.

In this embodiment, as information on a hyperbolic hb image or a bow-shaped bm image obtained by shining light on at least a part of the beverage container 4, and a predetermined two-dimensional shape having at least a part of the curved part. By comparing the image of the hyperbolic portion HB and the bow-shaped portion BM of the above, a predetermined two-dimensional image having at least a curved portion is formed by shining light from a specific direction on a characteristic portion of the shape of the container. The situation can be used as a criterion for judgment. Therefore, it is possible to determine whether the container is transparent or opaque based on the presence or absence of a shape including a characteristic curved portion that appears on the two-dimensional image by shining light on the three-dimensional shape.

In this embodiment, whether the container is transparent or opaque depending on the presence or absence of an image of a hyperbola hb or an image of a bow-shaped bm that appears characteristically by shining light on the side surface or the edge of the inverted truncated cone-shaped opaque beverage container 4. Can be determined.

In this embodiment, the presence or absence of at least one of a hyperbolic hb image and an arched bm image that characteristically appears by shining light on the side surface or edge of the inverted truncated cone-shaped opaque beverage container 4. Can be used to determine whether the container is transparent or opaque.

In this embodiment, the container is illuminated diagonally downward from above with respect to the beverage container 4, and the image is taken from the horizontal direction with respect to the beverage container 4, whereby a predetermined two-dimensional shape having at least a curved portion is formed. An image of a hyperbola hb or a bow-shaped bm as information can be easily formed and used for determination.

In this embodiment, the image of the hyperbolic portion HB of the search information 21 and the image of the bow-shaped segment BM, which are images of the stored comparative shape information, are used, and the captured image is combined with the image of the hyperbolic portion HB of the search information 21. By extracting the image of the hyperbola hb and the image of the bow-shaped bm by searching the region correlating with the image of the bow-shaped portion BM, it is possible to easily and surely determine whether the beverage container 4 is transparent or opaque.

In this embodiment, the binary image and / or the contrast adjusted image can be used to easily and surely determine whether the beverage container 4 is transparent or opaque.

In this embodiment, the “container” is the beverage container 4, but the present invention is not limited to this, and for example, a food container, a grain container, a fertilizer container, a feed container, and a paint container. The contents are not limited to liquids, solids, powders, etc., such as containers for chemical products, and may be applied to any container provided by a plurality of types of containers.

In the above embodiment, the beverage container 4 has an edge formed at the upper end, an image of a bow-shaped bm is formed by binarizing the photographed image of the beverage container 4, and a comparison with the bow-shaped portion BM is performed. However, the beverage container 4 may not have an edge formed at the upper end. In this case, the search information 21 may not include the bow-shaped portion BM.

In the above embodiment, the search information 21 has an image of the hyperbolic portion HB and an image of the bow-shaped portion BM, and the determination unit 14 determines by template matching with these images, but the determination unit 14 It is also possible to make a judgment without relying on template matching. For example, the determination unit 14 is provided with numerical information of the length in the vertical direction and the horizontal direction of the beverage container 4, numerical information of the aspect ratio, and the like, and is configured to perform determination by comparing the binary image with these numerical values. Is also possible.

It goes without saying that the above-described embodiment is an example of the present invention, and the present invention is not limited to this embodiment.

1 ... Delivery processing device (identification device for beverage serving container)
2 ... Lighting device (lighting unit)
3 ... Shooting device (shooting section)
4,4a, 4b, 4c, 4d ... Beverage container (container)
13 ... Extraction unit 14 ... Judgment unit 100 ... Beverage providing device 141 ... Mounting stand (mounting position)
150 ... Supply port (beverage supply section)
hb ・ ・ ・ Hyperbola (feature image)
bm ・ ・ ・ Bow type (feature image)
BM: Bow-shaped part (comparative shape information, first comparative shape information)
HB: Hyperbolic part (comparative shape information, second comparative shape information)

Claims (11)

A lighting unit that shines light from a specific direction on a container that stores beverages placed in a predetermined placement position.
A photographing unit that obtains a photographed image by photographing the container exposed to light by the lighting unit, and
A predetermined process is applied to the photographed image photographed by the photographing unit, and the photographed image is formed depending on the correlation between the light of the illumination unit and the three-dimensional shape and surface condition of the container appearing in the photographed image. An extraction unit that extracts feature images and
The feature image extracted by the extraction unit is compared with the information of the comparative shape having the same shape as the outer shape of the container or a shape different from the similar shape, and the feature image and the information of the comparative shape are compared with each other. A determination unit that determines whether the container is a transparent container or an opaque container using the comparison result of
A device for identifying a beverage serving container, which is characterized by being provided with. The determination unit includes information on a predetermined two-dimensional shape having at least a curved portion as information on the comparative shape, and the feature image obtained by irradiating at least a part of the container with the light. The identification device for a beverage serving container according to claim 1, wherein it is determined whether the container is the transparent container or the opaque container by comparing with the information of the comparative shape. The feature image is
An image obtained by shining the light of the lighting unit on the side surface of the container.
The information on the comparative shape possessed by the determination unit is obtained by irradiating the side surface of the opaque container having an inverted truncated cone shape with light from the specific direction, and has a substantially hyperbolic shape having the curved portion. The identification device for a beverage serving container according to claim 2, wherein the image information has a shape similar to that of the above. The feature image is
An image obtained by shining the light of the lighting unit on the edge of the upper end of the container.
The information on the comparative shape possessed by the determination unit is obtained by shining light from the specific direction on the edge of the upper end of the opaque container having an inverted truncated cone shape having a step between the side surface and the edge of the opening. The identification device for a beverage serving container according to claim 2, wherein the obtained image information has a shape similar to a substantially linear shape having a substantially bow-shaped curved portion. The feature image is
The first feature image obtained by shining the light of the lighting unit on the side surface of the container and the second feature image obtained by shining the light of the lighting unit on the edge of the upper end of the container.
The information of the comparative shape possessed by the determination unit is
The first image information having a shape similar to a substantially planar shape having the curved portion having a substantially hyperbolic shape, which is obtained by irradiating the side surface of the opaque container having an inverted truncated cone shape with light from the specific direction. Information on the comparative shape and image information having a shape similar to a substantially linear shape having the curved portion having a substantially bow shape, which is obtained by irradiating the edge of the upper end of the container with light from the specific direction. Including information on the comparative shape of 2
The extraction unit
When the feature image is extracted from the captured image, the first feature image and the second feature image are extracted from the captured image.
The determination unit
The container is opaque when the first feature image and the information of the first comparative shape are matched and / or when the second feature image and the information of the second comparative shape are matched. The identification device for a beverage serving container according to claim 2, wherein the container is determined to be a container. The shooting unit
The container placed in the above-mentioned placement position was photographed from the horizontal direction.
The lighting unit
The beverage-providing container according to any one of claims 1 to 5, wherein the container is illuminated diagonally downward with respect to the container placed in the previously described position from above the photographing unit. Identification device. The beverage serving container identification device further
A storage unit for storing a search image which is an image of the information of the comparative shape is provided.
The extraction unit
The identification device for a beverage serving container according to any one of claims 1 to 6, wherein the feature image is extracted by searching a region correlating with the search image from the captured image. The beverage serving container identification device further
A generation unit for generating at least one of a binary image of the captured image and a contrast adjusted image obtained by adjusting the contrast of predetermined information constituting the captured image is provided.
The extraction unit
The beverage provision according to any one of claims 1 to 7, wherein the binary image generated by the generation unit and the feature image obtained from at least one of the contrast-adjusted images are extracted. Container identification device. A lighting procedure in which a container for containing a beverage, which is placed in a predetermined position, is illuminated by a lighting unit from a specific direction,
In the lighting procedure, the container irradiated with light is photographed by a photographing means to obtain a photographed image, and a photographing procedure.
A predetermined process is applied to the photographed image photographed in the photographing procedure, and the photographed image is formed depending on the correlation between the light of the illumination unit and the three-dimensional shape and surface condition of the container appearing in the photographed image. Extraction procedure to extract feature images and
The feature image extracted in the extraction procedure is compared with the information of the comparative shape having the same shape as the outer shape of the container or a shape different from the similar shape, and the feature image and the information of the comparative shape are compared with each other. Judgment procedure for determining whether the container is a transparent container or an opaque container by using the comparison result of
A method for identifying a beverage serving container, which comprises the above. A lighting control procedure for controlling the lighting of a lighting unit that shines light from a specific direction on a container for containing a beverage placed in a predetermined mounting position.
In the lighting control procedure, a shooting control procedure for controlling the shooting of a shooting unit that shoots the container to which light is applied to obtain a shot image, and
A predetermined process is applied to the photographed image photographed in the photographing procedure, and the photographed image is formed depending on the correlation between the light of the illumination unit and the three-dimensional shape and surface condition of the container appearing in the photographed image. Extraction procedure to extract feature images and
The feature image extracted in the extraction procedure is compared with the information of the comparative shape having the same shape as the outer shape of the container or a shape different from the similar shape, and the feature image and the information of the comparative shape are compared with each other. Judgment procedure for determining whether the container is a transparent container or an opaque container by using the comparison result of
A beverage serving container identification program characterized by having a processor execute the above. The identification device for a beverage serving container according to any one of claims 1 to 8.
The placement position on which the container is placed and
A beverage supply unit for pouring the beverage corresponding to the material of the container determined by the identification device for the beverage serving container into the container placed at the above-mentioned placement position.
Beverage serving device characterized by being equipped with.