JP6136257B2 - Vending machine management apparatus, data processing method thereof, and program - Google Patents

Description

  The present invention relates to a vending machine management apparatus, a data processing method thereof, and a program.

  In Patent Document 1, a face image of a vending machine is taken and analyzed (column No, product code, selling price, H / C classification is analyzed) and vending machine installation data (column No, selling price, H / C category), and a system for notifying when the setting is not set according to the face display of the vending machine is described.

JP 2012-73738 A

  In the system described above, since the worker acquires the identification information of the vending machine by manually inputting the identification information of the vending machine or by acquiring the position information by GPS (Global Positioning System), it is troublesome to input. There is a possibility that the accuracy is lowered due to an input error or GPS communication state deterioration, and there is a problem that accuracy and efficiency are not good.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a vending machine management apparatus, a data processing method thereof, and a program capable of accurately and efficiently acquiring vending machine information. There is.

The vending machine management apparatus of the present invention is
An image acquisition means for acquiring a face image of the vending machine in which an identification code image indicating position information of the vending machine and at least one product sample displayed on the vending machine are imaged together;
Vending machine specifying means for recognizing the acquired face image and specifying the position of the vending machine from the image-recognized identification code image;
Recognizing the acquired face image, specifying at least one product from the image sample image that has been image-recognized, and specifying the specified product information by the vending machine specifying means, the vending Product specifying means associated with the position information of the machine.

The data processing method of the vending machine management apparatus of the present invention is as follows.
Vending machine management device
Obtaining a face image of the vending machine in which an identification code image indicating position information of the vending machine and at least one product sample displayed on the vending machine are imaged together;
Recognize the acquired face image, identify the position of the vending machine from the image-recognized identification code image,
The acquired face image is image-recognized, and at least one product is specified from the image-recognized product sample image, and the specified product information is associated with the position information of the vending machine in which the position is specified. It is a data processing method of a vending machine management apparatus.

The computer program of the present invention is:
In the computer that realizes the vending machine management device,
A procedure for obtaining a face image of the vending machine, in which an identification code image indicating position information of the vending machine and at least one product sample displayed on the vending machine are imaged together;
A procedure for recognizing the acquired face image and identifying the position of the vending machine from the image-recognized identification code image;
The acquired face image is image-recognized, and at least one product is specified from the image-recognized product sample image, and the specified product information is associated with the position information of the vending machine in which the position is specified. A program for executing a procedure.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  The various components of the present invention do not necessarily have to be independent of each other. A plurality of components are formed as a single member, and a single component is formed of a plurality of members. It may be that a certain component is a part of another component, a part of a certain component overlaps with a part of another component, or the like.

  In addition, although a plurality of procedures are described in order in the data processing method and the computer program of the present invention, the described order does not limit the order in which the plurality of procedures are executed. For this reason, when implementing the data processing method and computer program of this invention, the order of the several procedure can be changed in the range which does not have trouble in content.

  Furthermore, the plurality of procedures of the data processing method and the computer program of the present invention are not limited to being executed at different timings. For this reason, another procedure may occur during the execution of a certain procedure, or some or all of the execution timing of a certain procedure and the execution timing of another procedure may overlap.

  ADVANTAGE OF THE INVENTION According to this invention, the vending machine management apparatus which can acquire the information of a vending machine accurately and efficiently, its data processing method, and a program are provided.

It is a functional block diagram which shows the structure of the vending machine management apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the management system using the vending machine management apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the utilization scene of the vending machine management apparatus which concerns on embodiment of this invention. It is a functional block diagram which shows the specific structural example of the vending machine management apparatus which concerns on embodiment of this invention. It is a figure for demonstrating an example of the structure of the feature-value database which the vending machine management apparatus which concerns on embodiment of this invention utilizes. It is a figure for demonstrating an example of the structure of the vending machine database which the vending machine management apparatus which concerns on embodiment of this invention utilizes. It is a figure for demonstrating an example of the face information which the vending machine management apparatus which concerns on embodiment of this invention outputs. It is a figure for demonstrating an example of the structure of the goods database which the vending machine management apparatus which concerns on embodiment of this invention utilizes. It is a flowchart which shows an example of operation | movement of the vending machine management apparatus which concerns on embodiment of this invention. It is a flowchart which shows an example of the procedure of the operation | movement of the portable terminal and server of a management system using the vending machine management apparatus which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the vending machine management apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the face image imaged with the vending machine management apparatus which concerns on embodiment of this invention. It is a figure for demonstrating an example of the column information which the vending machine management apparatus which concerns on embodiment of this invention outputs. It is a functional block diagram which shows the structure of the vending machine management apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the management system using the vending machine management apparatus which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the vending machine management apparatus which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

(First embodiment)
FIG. 1 is a functional block diagram showing a configuration of a vending machine management apparatus 100 according to an embodiment of the present invention. FIG. 2 is a block diagram showing a configuration example of the management system 1 using the vending machine management apparatus 100 according to the embodiment of the present invention. FIG. 3 is a diagram for explaining a usage scene of the vending machine management apparatus 100 according to the embodiment of the present invention.
Although FIG. 2 or FIG. 3 shows a plurality of vending machines 10a, 10b, or 10c, the number of vending machines is not limited to these. Further, hereinafter, when it is not particularly necessary to distinguish a plurality of vending machines, they are simply indicated as vending machines 10.
Each vending machine 10 has a display window 14 in which a plurality of product samples 12 are displayed on the front panel of the housing. Alternatively, a liquid crystal display on which a plurality of product samples 12 are displayed may be provided on the front panel of the housing.

The vending machine 10 may further include a bill slot, a coin slot, a change return lever, a non-contact IC card reading unit, a liquid crystal display unit, a change port, a merchandise outlet, and the like.
The display window 14 includes a plurality of purchase buttons provided corresponding to the plurality of product samples 12. When purchasing a product, for example, a user inserts money into a bill slot or a coin slot, refers to the display window 14, and selects a product to be purchased from a plurality of product samples 12. Then, a product can be purchased by pressing the purchase button of the product, and the product delivered to the product carry-out port can be received. If necessary, the change return lever is operated to receive change from the change outlet.

  The vending machine management apparatus 100 according to the embodiment of the present invention includes an identification code image 16 (FIG. 2) indicating position information of the vending machine 10 and at least one product sample 12 displayed on the vending machine 10. And the image acquisition unit 102 that acquires the face image 18 of the vending machine 10 (for example, the image of the shooting range surrounded by the one-dot chain line in FIG. 2) and the acquired face image 18. The vending machine specifying unit 104 that recognizes and identifies the position of the vending machine 10 from the image-recognized identification code image, recognizes the acquired face image 18, and at least one product from the image-recognized product sample image A product specifying unit 106 that associates the specified product information with the location information of the vending machine 10 whose location is specified by the vending machine specifying unit 104. Obtain.

  The identification code image indicating the position information of the vending machine 10 is an image obtained by coding information by arranging a plurality of colors. For example, a chameleon code (registered trademark), a color bit (registered trademark), or a color code A color identification code image such as (trademark) can be used. Further, as the identification code image, a black and white identification code image such as a QR code (registered trademark) or a barcode may be used.

When the identification code image is color, it is not necessary to perform close-up photography, and it can be read from a relatively distant position or an oblique position. In the case of a chameleon code or color code, the amount of information that can be held in the code is large, and the number of vending machines 10 that can be managed increases. In the case of a color bit or color code, a character, a logo, a design, a photograph, or the like can be designed and incorporated based on the arrangement pattern. Therefore, even when a color bit or color code is added as an identification code image to the vending machine 10, the appearance is not impaired, and the degree of attention is improved by incorporating a logo or a pattern, and the identification code image is displayed. It can also have the role of advertising and advertising.
Which identification code image is to be adopted can be selected as appropriate according to the number of vending machines 10 to be managed by the system, the code identification capability of the server or terminal, the communication load between the terminal and the server, the connection environment, and the like. .

  For example, the identification code image may be affixed to the vending machine 10 with a seal or a magnet, or may be displayed on a liquid crystal display of the vending machine 10 or the like. Alternatively, the identification code image may be printed on the display window 14 of the vending machine 10, a pop or a poster installed in the dedicated holding unit, or may be printed directly on the casing of the vending machine 10. .

  The location information of the vending machine 10 indicates the location where the vending machine 10 is installed. For example, a lot number or a predetermined area (for example, Kanto, Tokyo West, Tokyo 23 Wards, campaign target area, Shinjuku Ward) , Yamanote Line jurisdiction, around Shinagawa Station, or a combination thereof). For example, map data or GPS (Global Positioning System) data may be used. Alternatively, if the installation location is in a building or facility, location information such as the floor and area installed together with the location can also be included. Further, when a plurality of vending machines 10 are provided in the same place, the position information can include identification information of each vending machine 10 and a relative positional relationship. For example, the relative positional relationship may indicate only being near, specifically, a direction from a reference position (for example, left and right, azimuth, or a combination thereof), an arrangement or an arrangement order (for example, As coordinates, what number, what column, or a combination thereof).

The image acquisition unit 102 acquires the face image of the vending machine 10 acquired in this way.
The vending machine specifying unit 104 recognizes the face image acquired by the image acquiring unit 102 and specifies the position of the vending machine 10 from the recognized image of the identification code.

  As shown in FIGS. 2 and 3, in the management system 1 of the present embodiment, for example, an operator holds the portable terminal 20 toward the vending machine 10 and is displayed on the display window 14 of the vending machine 10. The identification code image 16 is imaged together with at least one product sample 12, and the obtained face image 18 is transmitted to the data center 5 via the network 3. Then, the server 30 provided in the data center 5 performs recognition processing for the received image.

  The server 30 may be connected to the data warehouse 32, for example, and accumulates and manages various information related to the vending machine 10. The server 30 can analyze the arrangement of the product samples 12 displayed in the display window 14 of the vending machine 10 from the face image of the vending machine 10 received from the mobile terminal 20.

  In the present embodiment, the entire display window 14 and the identification code image 16 are captured, but the present invention is not limited to this. At least a part of the display window 14 and the identification code image 16 may be captured. A plurality of vending machines 10 may be imaged together. In addition, it is not always necessary to acquire a face image by one shooting, and it can be divided into a plurality of times, one part is imaged in a predetermined order, and a plurality of obtained images can be associated to form a face image as an image recognition target. Good. For example, a plurality of images can be associated by, for example, instructing the user in the shooting order by an application program.

Specifically, the identification code image recognition process in the vending machine specifying unit 104 can be performed by the following method.
The vending machine specifying unit 104 acquires the identification code by converting the identification code image into a serial number or bit string, and manages the position information of the vending machine 10 in association with the serial number or bit string (identification code). The position information of the vending machine 10 is specified from the identification code image. However, the vending machine specifying unit 104 may acquire the position information of the corresponding vending machine 10 directly from the identification code image.

The operation of the vending machine specifying unit 104 may be performed by either the mobile terminal 20 or the server 30.
For example, the mobile terminal 20 may convert the identification code image into a serial number or bit string, and the server 30 may specify the position information corresponding to the converted serial number or bit string.
Alternatively, when the identification code image has the position information itself of the vending machine 10, the position information of the vending machine 10 can be specified from the image by the portable terminal 20. That is, it is not always necessary to manage the code and position information by the server 30 or the like. Further, even in a shooting site where it is not possible to connect to the network, the mobile terminal 20 can check the position information in an offline state.

  The product specifying unit 106 recognizes the acquired face image, specifies at least one product from the product sample image that has been image-recognized, and the vending machine specifying unit 104 specifies the position of the specified product information. It associates with the position information of the vending machine 10.

  The result obtained by the product specifying unit 106 may be output. The output form is not particularly limited. For example, it is displayed on the screen of the server 30 or the portable terminal 20, uploaded to a website, printed on a printer, sent to a predetermined mail address, recorded in a file of a predetermined format, stored in a database It can be registered or a combination thereof.

  In addition, based on the obtained position information, the result of aggregation for each area may be output, or the history may be accumulated in time series.

  Each component of the vending machine management apparatus 100 includes a CPU (Central Processing Unit), a memory, a program that implements the components shown in FIG. 1 loaded in the memory, a storage unit that stores the program, and a communication interface. This is realized by any combination of hardware and software of a computer (not shown). It will be understood by those skilled in the art that there are various modifications to the implementation method and apparatus. Here, each figure shows a functional unit block, not a hardware unit configuration. In addition, in each figure, the structure of the part which is not related to the essence of this invention is abbreviate | omitted, and is not illustrated.

  In the vending machine management apparatus 100 of FIG. 1, the entire vending machine management apparatus 100 is controlled by the CPU together with each element. Each function of each unit in FIG. 1 of the vending machine management apparatus 100 can be realized by the CPU reading the program stored in the hard disk into the memory and executing it.

  The computer program according to the present embodiment includes an identification code image corresponding to position information of the vending machine 10 and at least one product sample displayed on the vending machine 10 on a computer that realizes the vending machine management apparatus 100. The procedure for acquiring the face image of the vending machine 10 taken together, the procedure for recognizing the acquired face image and identifying the position of the vending machine 10 from the recognized image of the identification code Recognize the face image, identify at least one product from the image sample image recognized, and execute the procedure for associating the identified product information with the location information of the vending machine 10 whose location is specified It is described as follows.

  The computer program of this embodiment may be recorded on a computer-readable recording medium. The recording medium is not particularly limited, and various forms can be considered. The program may be loaded from a recording medium into a computer memory, or downloaded to a computer through a network and loaded into the memory.

FIG. 4 is a functional block diagram showing a specific configuration example of the vending machine management apparatus 100 according to the embodiment of the present invention.
As shown in FIG. 4, the management system 1 according to the present embodiment includes a mobile terminal 20 used at a site where the vending machine 10 is installed, and a server 30 provided in the data center 5 (FIG. 2). Prepare. The portable terminal 20 and the server 30 are connected via the network 3. In the present embodiment, the vending machine management apparatus 100 of FIG. 1 is realized by the mobile terminal 20 and the server 30 of FIG.

  The mobile terminal 20 is a mobile terminal that can be connected to the network 3 via a relay station (not shown), and is, for example, a tablet terminal, a smartphone, or a PDA (Personal Digital Assistants). Furthermore, the mobile terminal 20 is, for example, a mobile terminal with a built-in camera (not shown) or connectable to the camera.

  In the present embodiment, the server 30 is realized by cloud computing using a server computer of the data center 5, but is not limited to this. The server 30 may be configured with a virtual server or the like.

  Each component of the portable terminal 20 and the server 30 includes a CPU, a memory, a program that realizes the components of FIG. 4 loaded in the memory, a storage unit that stores the program, and hardware of an arbitrary computer including a communication interface. And any combination of software. It will be understood by those skilled in the art that there are various modifications to the implementation method and apparatus.

The mobile terminal 20 includes an imaging unit 22 and a transmission unit 24.
The server 30 includes a receiving unit 40, an image recognition unit 42, a position specifying unit 46, and a product specifying unit 50, a data warehouse 32, a feature quantity database 44, a vending machine database 48, a product To the database 52.
In the configuration example of the embodiment of FIG. 4, the receiving unit 40 corresponds to the image acquisition unit 102 of FIG. 1, the image recognition unit 42 and the position specifying unit 46 correspond to the vending machine specifying unit 104 of FIG. The recognition unit 42 and the product specifying unit 50 correspond to the product specifying unit 106 in FIG.

  The imaging unit 22 can be, for example, a camera built in or connected to the mobile terminal 20 or a combination thereof. The image captured by the imaging unit 22 may be a still image or a moving image. In the case of a moving image, image data obtained sequentially from the moving image can be used. The processing capability of each portable terminal 20 and the communication load of the network 3 can be taken into consideration as appropriate.

  When taking an image using the portable terminal 20, the portable terminal 20 starts a predetermined application program, controls the imaging unit 22 by the application program, and displays the display window 14 of the vending machine 10 and the identification code image 16 together. The face image 18 of the vending machine 10 can be acquired by taking a photograph.

The transmission unit 24 transmits the face image 18 of the vending machine 10 captured by the imaging unit 22 to the server 30 via the network 3.
The communication means of the transmission unit 24 may be either wired or wireless communication, or may be communication via any of a mobile communication network, a public line network, a LAN (Local Area Network), or a WAN (Wide Area Network). It is preferable to adopt appropriately depending on the configuration of the terminal 20 and the like. As described above, various image communication methods can be considered and are not related to the essence of the present invention, and thus detailed description thereof will be omitted. Further, as a security measure at the time of communication, a configuration for performing an encryption process or an authentication process can be included.

  The image obtained by the imaging unit 22 capturing the exhibition window 14 may be transmitted to the server 30 by the sequential transmission unit 24 without being stored in the mobile terminal 20 at the time of shooting, or the image may be stored in the mobile terminal 20 in advance. A plurality of images may be transmitted to the server 30 by the transmission unit 24 at a later time.

  The server 30 may read and acquire an image from a recording medium on which an image captured in advance by the mobile terminal 20 or the like is recorded. Various methods of acquiring images are conceivable and are not related to the essence of the present invention, and thus detailed description thereof is omitted.

FIG. 5 is a diagram for explaining an example of the structure of the feature amount database 44 of the present embodiment.
As shown in FIG. 5A, face image information of each product is registered in the feature amount database 44. Furthermore, information necessary for recognizing an identification code image (not shown) is registered in the feature amount database 44. As shown in FIG. 5B, the feature quantity database 44 includes, as feature face image information, a feature ID table 45 for identifying a product ID and a feature quantity capable of recognizing the product face image. Is stored in association with data of at least one feature point.
The information necessary for recognizing the identification code image includes, for example, a pattern or symbol having a characteristic amount peculiar to the identification code image or an identification marker in advance in the identification code image. By holding the data in the quantity database 44, the area of the identification code image can be recognized.

  Returning to FIG. 4, the receiving unit 40 receives image data from the portable terminal 20 via the network 3. The image recognizing unit 42 extracts feature amounts from the image data received by the receiving unit 40 and refers to the feature amount database 44 to check whether or not there is a matching feature amount. In the present embodiment, the receiving unit 40 receives and acquires the face image 18 of the vending machine 10 obtained by imaging the vending machine 10 using the imaging unit 22 of the mobile terminal 20.

The recognition processing by the image recognition unit 42 includes, for example, the following processing.
(1) A feature amount is extracted from the captured image (face image 18), and the feature amount database 44 is referenced to perform pattern recognition processing of the face image of the product. Here, the product face image to be pattern-recognized may not be specified up to the product name. For example, if the product is a beverage, the face image of a plastic bottle or can registered in the feature amount database 44 is there. It only has to be recognized. In addition, all face image areas of all products existing in the captured image may be specified, or only specific products may be recognized.
(2) A feature amount is extracted from the captured image (face image 18), and as described above, for example, a portion having a feature amount peculiar to a color identification code image is recognized, and an identification code image region is recognized.

  As shown in FIG. 6A, the vending machine database 48 can include a position information table 60 that holds the position information associated with the identification code. Furthermore, as shown in FIG. 6B, the vending machine database 48 can include a vending machine table 62 that holds positional information and identification information (vending machine ID) of the vending machine 10 in association with each other. .

  The position specifying unit 46 in FIG. 4 converts the identification code image recognized by the image recognition unit 42 into an identification code (serial number or bit string). Then, the position specifying unit 46 refers to the position information table 60 and specifies the position information of the vending machine 10 corresponding to the identification code.

  Further, the position specifying unit 46 may specify identification information (vending machine ID) of the corresponding vending machine 10 by referring to the vending machine table 62 from the acquired position information. Furthermore, the vending machine database 48 may hold vending machine attribute information (not shown). For example, the type, maker, owner, installation location, column configuration, handling product, campaign information being executed, or the like of the vending machine 10 associated with the vending machine ID may be held.

The product specifying unit 50 specifies the product ID by referring to the feature amount table 45 from the image of the product sample 12 of the vending machine 10 recognized by the image recognition unit 42. Further, the product specifying unit 50 associates the position information of the vending machine 10 specified by the position specifying unit 46 with the product information of at least one product sample 12, for example, the product ID.
For example, as shown in FIG. 7, the product information of at least one product sample 12, for example, the product ID, is linked to the location information of the vending machine 10 as face information 56 and stored in the product database 52 or the data warehouse 32. You may accumulate.

Further, as shown in FIG. 8, the product database 52 stores information such as a product ID, a product name, a product classification code, a price, or an image ID for identifying product face image data in the product master table 54. To do. The product classification code can include, for example, information indicating a hierarchical classification. When the major classification is beverage, the middle classification is classified into tea, coffee, tea, fruit juice, carbonated drink, water, etc. As a small classification of tea, classifications such as hojicha, green tea, oolong tea, and blended tea can be included, and as branch numbers, classifications such as region limitation, period limitation, and campaign can be included. Further, the product database 52 may include information on products of other companies.
The product specifying unit 50 can acquire information on the products displayed on the vending machine 10 from the product database 52 based on the specified product ID.

  In the present embodiment, the image data taken from the vending machine 10 is transmitted to the data center 5 and the server 30 of the data center 5 performs the image recognition process. However, the present invention is not limited to this. For example, the mobile terminal 20 may include an extraction unit (not shown) that extracts a feature amount from an image captured by the imaging unit 22. The transmission unit 24 may transmit the feature amount information to the data center 5. Compared with the case of transmitting image data, the configuration of transmitting feature amount information can reduce the communication amount and suppress the communication load.

  Furthermore, the mobile terminal 20 may include an image recognition unit 42 that holds feature amount data of a recognition target product image and performs image recognition processing of a captured image. The transmission unit 24 may transmit the image recognition result to the data center 5. This configuration is effective, for example, when there are few types of products to be recognized or when the processing capability of the mobile terminal 20 is high, and the load on the server 30 of the data center 5 can be reduced.

  As described above, the vending machine management apparatus 100 of FIG. 1 can be realized by combining the server 30 of the data center 5 and the mobile terminal 20. It is desirable to appropriately determine the combination in consideration of the mounted function, processing capability, communication load, etc. of the server 30 or the portable terminal 20.

Further, the information transmitted from the portable terminal 20 to the server 30 by the transmission unit 24 further includes, for example, photographer information (which may include authentication information), identification information of the portable terminal 20 and the like in addition to the image data itself. May be. Further, when the image information does not include attribute information such as shooting date and time, the shooting date and time may be included. Further, when an image obtained by photographing a part of the display window 14 is transmitted, it is preferable to include information indicating which area is photographed.
These pieces of information may be acquired automatically by, for example, linking with an application program that creates a daily report. The operator may input using the operation screen of the mobile terminal 20.

  The server 30 may be connected to an input device (not shown) such as a keyboard and a mouse, a display device such as a display, and an output device (not shown) such as a printer, and has a user interface function operated by an operator. May be. Further, an input / output device such as a touch panel may be integrated.

  In the configuration as described above, a data processing method by the vending machine management apparatus 100 according to the present embodiment will be described below. FIG. 9 is a flowchart showing an example of the operation of the vending machine management apparatus 100 according to the present embodiment.

  In the data processing method of the present embodiment, the vending machine management apparatus 100 includes an identification code image 16 corresponding to position information of the vending machine 10 and at least one product sample 12 displayed on the vending machine 10. The face image 18 of the vending machine 10 imaged together is acquired (step S101), the acquired face image is image-recognized, and the position of the vending machine 10 is specified from the image-recognized identification code image ( Step S103), recognizing the acquired face image 18, specifying at least one product from the product sample image that has been image-recognized, and specifying the specified product information as the location information of the vending machine 10 whose location has been specified. (Step S105).

Here, the operation in the configuration in which the display window 14 and the identification code image 16 of the vending machine 10 are captured using the imaging unit 22 of the mobile terminal 20 and transmitted to the server 30 of the data center 5 will be specifically described.
FIG. 10 is a flowchart showing an example of an operation procedure of the mobile terminal 20 and the server 30 of the management system 1 according to the embodiment of the present invention.
First, the vending machine 10 is imaged by the imaging unit 22 (step S111). Then, the captured image is transmitted to the server 30 of the data center 5 by the transmission unit 24 (step S113).

In the server 30, the receiving unit 40 receives the captured image (step S115). Then, the image recognition unit 42 extracts a feature amount from the image and performs a pattern recognition process on the face image of the product (step S117).
Then, the image recognition unit 42 extracts the identification code image region, that is, the identification code image 16 from the captured image (step S119). Then, the position specifying unit 46 converts the identification code image 16 into an identification code (serial number or bit string) (step S121) and refers to the position information table 60 to specify the position information of the vending machine 10 (step S121). S123).

  Further, the image recognizing unit 42 extracts face image areas of all products existing in the captured image (step S125). Then, the product specifying unit 50 specifies at least one product ID corresponding to the feature amount of the face image of the product recognized by the image recognition unit 42 (step S127), and further, the vending machine specified in step S123. The position information of 10 and the product ID specified in step S127 are linked and output (step S129). The face information 56 in FIG. 7 can be stored in the data warehouse 32 or the product database 52.

  In this way, in the present embodiment, the identification code image 16 and the product face image are recognized from the image obtained by capturing the vending machine 10 using the mobile terminal 20, and the position information of the vending machine 10 and the vending machine are recognized. At least one product sample 12 displayed on the machine 10 can be identified.

  For example, in this embodiment, it is possible to check where the vending machine 10 on which the product sample 12 of the target product is displayed is present. Alternatively, it is possible to check whether or not the product sample 12 of the target product is displayed on the vending machine 10 at a predetermined position.

As described above, according to the vending machine management apparatus 100 of the embodiment of the present invention, the position information of the vending machine 10 based on the image obtained by imaging the vending machine 10 using the mobile terminal 20 or the like. Since the product information of the product sample 12 displayed on the vending machine 10 can be specified, the identification information of the vending machine can be manually input, or a function such as GPS can be used. Instead, the position information and the product information of the vending machine 10 can be specified and associated with one sensor (imaging unit). Thereby, the information of the vending machine 10 can be grasped accurately and efficiently.
Compared with the case where the position information of the vending machine 10 is acquired by GPS, it is not affected by the communication state of the GPS system at the time of measurement, and the position information is not shifted due to the deviation of the standing position of the measurer. , The accuracy of position information is improved.

(Second Embodiment)
FIG. 11 is a functional block diagram showing the configuration of the vending machine management apparatus 200 according to the embodiment of the present invention.
The vending machine management apparatus 200 of the present embodiment is further different from the above embodiment in that the display state of the product sample of the vending machine is acquired.
The vending machine management apparatus 200 includes an image acquisition unit 102, a vending machine specifying unit 104, and a product specifying unit 106, which are the same as the vending machine management apparatus 100 of the above embodiment, and further displays a display state. The unit 202 is provided.

The display state acquisition unit 202 acquires the display state of the product sample 12 from the product sample image obtained by image recognition of the face image.
As the display state, for example, the following information can be considered. Here, the target product refers to a target product for which the vending machine management apparatus intends to grasp information.
(1) Information on the column position of each product including at least the target product in the vending machine 10 As an example, the product code and column number, or the stage (upper, middle, lower) or row (left, Middle, right), their order and arrangement, etc. (2) the presence or absence of the target product in the vending machine 10, or its quantity (3) the relationship between the target product and the non-target product, specifically the target product and For example, if the company's carbonated beverage A is the target product, whether or not the company's carbonated beverage A exists in the vending machine 10, if there is, the quantity, column position The ratio of the quantity of the in-house carbonated drink A to the entire vending machine 10 or the ratio of the quantity of the in-house carbonated drink A to the carbonated drink of another company may be acquired as a display state.

  When grasping the display status, the product attribute information associated with the feature quantity of the product image, the distinction between the company and other companies, the type of product (carbonic acid, fruit juice, sports drink, water, tea, coffee, soup, etc. Based on the content type, hot / cold classification, container size and type (can / plastic bottle), etc., the image-recognized product is specified and the display state of each product is grasped.

  In general, the vending machine 10 includes a vending machine that sells only its own products, a vending machine of another company that does not sell its own products, and a vending machine of a vendor that mixes its own products and other companies' products. There is. The sales information of the products of the vending machine of the company can be acquired by collecting from the vending machine of the company or purchasing from the vendor.

  For example, in the case of a beverage vending machine, the beverage manufacturer analyzes the column configuration of the product sample 12 in the display window 14 of the vending machine 10 and the sales performance, and analyzes the trend of the influence of the column configuration on sales. It is possible to do it. According to the vending machine management apparatus 200 of the present embodiment, for example, in the vending machine 10, the number of columns of the product sample 12, that is, the number of the product samples 12 that are continuously displayed, and the display of the product sample 12 are displayed. Based on information such as position, for example, center or end, top, middle, or bottom, analyze how they affect the number of units sold, the amount of sales, etc. To do. Based on the result, the optimum column configuration can be examined.

In addition, it is possible to statistically analyze the influence of the installation location of the vending machine 10 on the sales of products.
For example, vending machines 10 installed near boys 'schools tend to have high sales of carbonated drinks, and vending machines 10 installed near girls' schools tend to have high sales of tea and the like. It becomes possible to analyze information.

FIG. 12 is a diagram for explaining a face image captured by the vending machine management apparatus 200 according to the embodiment of the present invention.
FIG. 12 shows a state in which the face area 70 and the code area 72 of each product sample 12 are extracted from the face image 18 including the product sample 12 and the identification code image 16.

The information on the display state of the product acquired by the display state acquisition unit 202 can be stored in association with a product name (or product ID) in a column, for example, as column information 210 shown in FIG. Further, the column information 210 may further include a flag indicating whether or not the product is a company product. Here, for example, whether or not the product is an in-house product may be determined based on whether or not the feature amount of the product image is registered in the feature amount database 44. If there is no registration, the product may be determined to be a product of another company. . Or you may hold | maintain the information which shows whether it is an in-house product in connection with goods ID in the goods master table 54 or the feature-value table 45. FIG.
Further, the product identification unit 106 may extract information such as the price of the product sample 12 and the distinction between hot and cold from the face image 18, further acquire it as product information, and hold it in the column information 210. Further, the presence / absence and type of posters and pops installed in the display window 14 and the vending machine 10 may be recognized from the face image 18, and the obtained information may be further retained.
The product information including the display state of the product acquired in this way is stored in the data warehouse 32 as big data together with the shooting date and time of the face image 18 and information on the vending machine 10.

  As described above, according to the vending machine management apparatus 200 according to the embodiment of the present invention, the same effect as that of the above embodiment can be obtained, and the position information of the vending machine 10 and the vending machine are provided. The association with the display state of the ten products can be performed efficiently with high accuracy.

(Third embodiment)
FIG. 14 is a functional block diagram showing the configuration of the vending machine management apparatus 300 according to the embodiment of the present invention.
The vending machine management apparatus 300 according to this embodiment is different from the above embodiment in that the vending machine without an identification code image is included in the management target.
In the present embodiment, as shown in FIG. 15, a first vending machine 10b to which the identification code image 16 is not attached and a second vending machine 10a to which the identification code image 16 is attached are provided. An example will be described.

  The vending machine management apparatus 300 includes an image acquisition unit 302, a vending machine specifying unit 304, and a product specifying unit 306. The vending machine management apparatus 300 may further include a display state acquisition unit 202 of the vending machine management apparatus 200 of the above embodiment.

The image acquisition unit 302 has the same function as the image acquisition unit 102 of the above-described embodiment, and further includes the identification code image 16 of the second vending machine 10a adjacent to the first vending machine 10b, and the first A face image 19 of the first vending machine 10b is acquired in which at least one product sample 12 displayed on the vending machine 10b is imaged together.
The vending machine specifying unit 304 has the same function as the vending machine specifying unit 104 of the above embodiment, and further specifies the position of the second vending machine 10a from the image-recognized identification code image 16. .
The product specifying unit 306 has the same function as the product specifying unit 106 of the above embodiment, and further, the product information of the specified first vending machine 10b is specified by the vending machine specifying unit 304. 2 is associated with the position information of the vending machine 10a.

  In the present embodiment, the position information of the second vending machine 10a is stored as the face information 56 of FIG. In the present embodiment, as the face information 56, information indicating that the position information of the second vending machine 10a is registered may be further held as the position information of the first vending machine 10b. Further, as the face information 56, together with the position information of the second vending machine 10a, the relative position information of the first vending machine 10b with respect to the second vending machine 10a, for example, right next or three right Information such as neighbors may be held together.

  As described above, according to the vending machine management apparatus 300 according to the embodiment of the present invention, the vending machine 10b that achieves the same effect as the above embodiment and does not have identification code information. In addition, the positional information and the product information can be associated with each other and can be acquired accurately and efficiently, that is, the information can be managed.

(Fourth embodiment)
FIG. 16 is a functional block diagram showing the configuration of the vending machine management apparatus 400 according to the embodiment of the present invention.
The vending machine management apparatus 400 according to the present embodiment is acquired from a captured image obtained by imaging the vending machine 10 using the mobile terminal 20 of the user of the vending machine 10 in addition to the above embodiment. The difference is that the information of the user of the portable terminal 20 is associated with the position information of the vending machine 10.

The vending machine management apparatus 400 includes a vending machine specifying unit 404 and a user information acquisition unit 402.
In addition, this embodiment is good also as a structure provided with the vending machine specific | specification part 404 and the user information acquisition part 402 of this embodiment in the structure similar to the vending machine management apparatus of the said embodiment.

In the present embodiment, the vending machine management apparatus 400 includes the identification code image 16 of the vending machine 10 captured using the imaging unit 22 of the mobile terminal 20 of the user of the vending machine 10, user information, Is further provided with a user information acquisition unit 402.
The vending machine specifying unit 404 has the same function as the vending machine specifying unit 104 of the above embodiment, and further recognizes the image of the identification code image 16 of the vending machine 10 acquired by the user information acquiring unit 402. The position of the vending machine 10 is specified from the image-recognized identification code image 16, and the position information of the vending machine 10 whose position is specified is associated with the user information.

  In the vending machine management apparatus 400 of this embodiment, the transmission unit 24 of the mobile terminal 20 uses the imaging unit 22 to capture at least an image obtained by capturing the identification code image 16 of the vending machine 10 together with user information. It transmits to the server 30 of the center 5.

The user information includes, for example, identification information of the mobile terminal 20 or user identification information. Further, the user information may include user profile information. Alternatively, the user information acquisition unit 402, similar to the image acquisition unit 102 of the above-described embodiment, uses the face image 18 obtained by capturing the at least one product sample 12 and the identification code image 16 of the vending machine 10 together. You may acquire with information.
The following method can be considered as a method of transmitting an image together with user information from the portable terminal 20 to the server 30.
(1) The user application is registered and registered or purchased, and the captured image is transmitted together with the user information managed by the application.
(2) Using user information such as a user ID, log in to a web page with a predetermined URL and upload a captured image. In this case, the user ID used at the time of login is user information.
(3) The captured image is transmitted from the user's portable terminal 20 to a predetermined mail address. In this case, the mail address of the portable terminal 20 becomes user information.

A usage scene of the vending machine management apparatus 400 of the present embodiment configured as described above will be described below.
(1) Information on the vending machine 10 used by the user is collected, analyzed together with the user profile, and used as marketing data. For example, information can be collected by using a captured image of the vending machine 10 instead of an application sticker so that it can be applied for a campaign.
(2) By letting a user image a vending machine 10 that sells a predetermined product or a vending machine 10 installed in a predetermined place (station, facility, etc.), it can be applied to stamp rallies and the like. Then, the vending machine 10 is caused to function as an information station to develop a campaign or the like.

  As described above, according to the vending machine management apparatus 400 according to the embodiment of the present invention, the vending machine 10 has the same effects as those of the above embodiment and is linked with user information. Sales information can be collected more efficiently, and further business opportunities can be developed.

As mentioned above, although embodiment of this invention was described with reference to drawings, these are the illustrations of this invention, Various structures other than the above are also employable.
In addition, when acquiring and using the information regarding a user in this invention, this shall be done legally.

  While the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
Vending machine management device
Obtaining a face image of the vending machine in which an identification code image indicating position information of the vending machine and at least one product sample displayed on the vending machine are imaged together;
Recognize the acquired face image, identify the position of the vending machine from the image-recognized identification code image,
The acquired face image is image-recognized, and at least one product is specified from the image-recognized product sample image, and the specified product information is associated with the position information of the vending machine in which the position is specified. Data processing method of vending machine management device.
(Appendix 2)
In the data processing method of the vending machine management apparatus according to appendix 1,
The vending machine management device is
Furthermore, the data processing method of the vending machine management apparatus which acquires the display state of the said product sample from the said product sample image obtained by image-recognizing the said face image.
(Appendix 3)
In the data processing method of the vending machine management apparatus according to appendix 1 or 2,
The vending machine management device is
The identification code image of a second vending machine adjacent to the first vending machine and at least one product sample displayed on the first vending machine are imaged together. The face image of the vending machine
From the identification code image that has been image-recognized, the position of the second vending machine is identified,
A data processing method of a vending machine management apparatus for associating the product information of the identified first vending machine with the positional information of the identified second vending machine.
(Appendix 4)
In the data processing method of the vending machine management apparatus according to any one of appendices 1 to 3,
The vending machine management device is
The data processing method of the vending machine management apparatus which acquires the said face image of the said vending machine obtained by imaging the said vending machine using the imaging part contained in a portable terminal.
(Appendix 5)
In the data processing method of the vending machine management apparatus according to appendix 4,
The vending machine management device is
A portable terminal including the imaging unit;
A management device that manages the product information associated with the location information of the vending machine,
The vending machine specifying process or the product specifying process is a data processing method of a vending machine management apparatus realized by the portable terminal or the management apparatus.
(Appendix 6)
In the data processing method of the vending machine management apparatus according to any one of appendices 1 to 5,
The vending machine management device is
Furthermore, the identification code image of the vending machine imaged using the imaging unit of the mobile terminal of the user of the vending machine and user information are acquired,
Recognizing the obtained identification code image of the vending machine, specifying the position of the vending machine from the image-recognized identification code image, and the position information of the vending machine where the position is specified; A data processing method of a vending machine management apparatus that links the user information.

(Appendix 7)
In the computer that realizes the vending machine management device,
A procedure for obtaining a face image of the vending machine, in which an identification code image indicating position information of the vending machine and at least one product sample displayed on the vending machine are imaged together;
A procedure for recognizing the acquired face image and identifying the position of the vending machine from the image-recognized identification code image;
The acquired face image is image-recognized, and at least one product is specified from the image-recognized product sample image, and the specified product information is associated with the position information of the vending machine in which the position is specified. Program to execute the procedure.
(Appendix 8)
In the program described in Appendix 7,
A program for causing a computer to further execute a procedure for acquiring a display state of the product sample from the product sample image obtained by image recognition of the face image.
(Appendix 9)
In the program described in Appendix 7 or 8,
The identification code image of a second vending machine adjacent to the first vending machine and at least one product sample displayed on the first vending machine are imaged together. To obtain the face image of the vending machine of
A procedure for identifying the position of the second vending machine from the image-recognized identification code image;
For causing the computer to further execute a procedure for associating the product information of the identified first vending machine with the position information of the second vending machine identified by the procedure of identifying the vending machine program.
(Appendix 10)
In the program according to any one of appendices 7 to 9,
A program for causing a computer to further execute a procedure of acquiring the face image of the vending machine obtained by imaging the vending machine using an imaging unit included in a portable terminal.
(Appendix 11)
In the program described in Appendix 10,
A portable terminal including the imaging unit;
A management device that manages the product information associated with the location information of the vending machine,
The procedure for identifying the vending machine or the procedure for identifying the product is:
A program executed by a management device that manages the product information associated with the position information of the portable terminal including the imaging unit or the vending machine.
(Appendix 12)
In the program according to any one of appendices 7 to 11,
A procedure for acquiring the identification code image of the vending machine imaged using an imaging unit of a mobile terminal of a user of the vending machine and user information;
The image of the identification code image of the vending machine acquired by the procedure for acquiring the user information is recognized, the position of the vending machine is identified from the image-recognized identification code image, and the position is identified. A program for causing a computer to further execute a procedure for associating position information of the vending machine with the user information.

DESCRIPTION OF SYMBOLS 1 Management system 3 Network 5 Data center 10 Vending machine 12 Goods sample 14 Exhibition window 16 Identification code image 18 Face image 19 Face image 20 Mobile terminal 22 Imaging part 24 Transmission part 30 Server 32 Data warehouse 40 Reception part 42 Image recognition part 44 feature amount database 45 feature amount table 46 position specifying unit 48 vending machine database 50 product specifying unit 52 product database 54 product master table 56 face information 60 position information table 62 vending machine table 70 face area 72 code area 100 vending machine Management device 102 Image acquisition unit 104 Vending machine identification unit 106 Product identification unit 200 Vending machine management device 202 Display state acquisition unit 210 Column information 300 Vending machine management device 302 Image acquisition unit 304 Automatic sales Identifying unit 306 commodity specifying unit 400 the vending machine management device 402 the user information acquisition unit 404 vending machine identification unit

Claims (8)

An image acquisition means for acquiring a face image of the vending machine in which an identification code image indicating position information of the vending machine and at least one product sample displayed on the vending machine are imaged together;
Vending machine specifying means for recognizing the acquired face image and specifying the position of the vending machine from the image-recognized identification code image;
Recognizing the acquired face image, specifying at least one product from the image sample image that has been image-recognized, and specifying the specified product information by the vending machine specifying means, the vending A vending machine management apparatus comprising: product specifying means associated with the position information of the machine. In the vending machine management apparatus according to claim 1,
A vending machine management apparatus further comprising display state acquisition means for acquiring a display state of the product sample from the product sample image obtained by image recognition of the face image. In the vending machine management apparatus according to claim 1 or 2,
The image acquisition means captures together the identification code image of the second vending machine adjacent to the first vending machine and at least one product sample displayed on the first vending machine. And obtaining the face image of the first vending machine,
The vending machine specifying means specifies the position of the second vending machine from the identification code image that has been image-recognized,
The merchandise specifying means associates the merchandise information of the identified first vending machine with the location information of the second vending machine identified by the vending machine specifying means. . In the vending machine management apparatus according to any one of claims 1 to 3,
The image acquisition unit is a vending machine management apparatus that acquires the face image of the vending machine obtained by imaging the vending machine using an imaging unit included in a mobile terminal. In the vending machine management apparatus according to claim 4,
A portable terminal including the imaging unit;
A management device that manages the product information associated with the location information of the vending machine,
The vending machine specifying device or the product specifying device is a vending machine management device realized by the portable terminal or the management device. In the vending machine management apparatus according to any one of claims 1 to 5,
The vending machine further comprises user information acquisition means for acquiring the identification code image of the vending machine imaged using an imaging unit of a mobile terminal of a user of the vending machine and user information,
The vending machine specifying means recognizes the identification code image of the vending machine acquired by the user information acquisition means, specifies the position of the vending machine from the image-recognized identification code image, A vending machine management apparatus for associating position information of the vending machine with the specified position and the user information. Vending machine management device
Obtaining a face image of the vending machine in which an identification code image indicating position information of the vending machine and at least one product sample displayed on the vending machine are imaged together;
Recognize the acquired face image, identify the position of the vending machine from the image-recognized identification code image,
The acquired face image is image-recognized, and at least one product is specified from the image-recognized product sample image, and the specified product information is associated with the position information of the vending machine in which the position is specified. Data processing method of vending machine management device. In the computer that realizes the vending machine management device,
A procedure for obtaining a face image of the vending machine, in which an identification code image indicating position information of the vending machine and at least one product sample displayed on the vending machine are imaged together;
A procedure for recognizing the acquired face image and identifying the position of the vending machine from the image-recognized identification code image;
The acquired face image is image-recognized, and at least one product is specified from the image-recognized product sample image, and the specified product information is associated with the position information of the vending machine in which the position is specified. Program to execute the procedure.

Images