JP6839894B2 - Information and communication terminals, product settlement servers, product settlement systems, product identification methods, product settlement methods, purchased product management methods and programs - Google Patents

Description

The present invention relates to an information communication terminal, a product settlement server, a product settlement system, a product identification method, a product settlement method, a purchased product management method, and a program.

At cash registers such as convenience stores and supermarkets, it takes time to read and settle products, and on the store side, labor costs are incurred due to the time required for settlement, and customers also have to wait for cash registers. Some supermarkets have introduced self-checkout (customers themselves read the barcode of the product and settle the payment), but waiting because the processing at the cashier is concentrated just because the work burden is transferred from the store staff to the customer. The time did not decrease.
In Patent Document 1, a plurality of self-scanning registration terminals having a product code reading means for reading a product code attached to each product are used as a management unit in a POS device having a payment terminal for final payment of purchased products. There is a description that the management unit is configured to send and receive product purchase information based on the product code information read by the product code reading means between each self-scan type registration terminal and the POS device. is there. Household account book software that manages purchased products includes software that takes a receipt with a camera to create a household account book, and software that registers the expiration date of a product and manages the expiration date.

Japanese Unexamined Patent Publication No. 7-254021

For vegetables, etc., where it is difficult for the customer to add a barcode with a system that reads the product code (bar code, etc.) using the product code reading means, the cashier is required to manually enter the settlement data. .. In addition, since the surface shape of the product is not necessarily flat, it may be difficult for the customer to read the barcode of the product. The household account book software that manages products has a problem that it takes time and effort because it is necessary to input the information of the purchased products every time the products are purchased.

The present invention provides an information communication terminal, a product settlement server, a product settlement system, a product identification method, a product settlement method, and a program that use a product identification method other than a barcode or use the product identification method together with a barcode. The purpose is.

Another object of the present invention is to provide a program that does not require input of information on purchased products.

A further object of the present invention is to provide an information communication terminal, a purchased product management method, and a program that alert the expiration date of a purchased product at the convenience of the user.

The information communication terminal according to the present invention includes, at least, an image pickup unit that captures image information including the appearance of the product, the expiration date or expiration date of the product, or the barcode of the product.
An image analysis unit that analyzes the image information regarding whether or not the image information includes the appearance of the product and the expiration date or expiration date of the product or the barcode of the product.
A display unit that displays an analysis result of whether or not the image information includes the appearance of the product, the expiration date or the expiration date of the product, or the barcode of the product.
It is an information communication terminal equipped with.

The product settlement server according to the present invention is a product settlement server capable of communicating with an information communication terminal.
A communication unit that receives first image information including the appearance of a product in at least one direction from the information communication terminal.
A database in which the second image information including the appearance of the product taken from one or more directions and the settlement data of the product related to the second image information are linked and accumulated.
An image collation unit that compares and collates the first image information with the second image information to specify the settlement data is provided.
The communication unit is a product settlement server that transmits the specified settlement data to the information communication terminal.

The product settlement system according to the present invention is a product settlement system including the above-mentioned product settlement server according to the present invention, an information communication terminal capable of communicating with the product settlement server, and a register capable of communicating with the information communication terminal. Is.

In the method for identifying a product of an information communication terminal according to the present invention, the imaging unit captures image information including at least the appearance of the product and the expiration date or expiration date or barcode of the product.
The information analysis unit analyzes the image information with respect to whether or not the image information includes the appearance of the product and the expiration date or expiration date or barcode of the product.
The display unit is a product identification method of an information communication terminal, which displays an analysis result of whether or not the image information includes the appearance of the product and the expiration date or expiration date of the product or a barcode.

The product settlement method of the product settlement server according to the present invention is a product settlement method of a product settlement server capable of communicating with an information communication terminal.
The first image information including the appearance of the product in at least one direction is received from the information communication terminal, and the first image information is received.
The second image information including the appearance of the product taken from the one direction or a plurality of directions is combined with the settlement data of the product and stored in the database.
The settlement data is specified by comparing and collating the first image information with the second image information.
This is a product settlement method for transmitting the settlement data to the information communication terminal.

The program according to the present invention can be applied to a computer as an information communication terminal.
A process in which the imaging unit captures at least image information including the appearance of the product and the expiration date or expiration date of the product or the barcode.
A process of analyzing the image information regarding whether or not the image information includes the appearance of the product and the expiration date, the expiration date, or the barcode of the product.
A process in which the display unit displays an analysis result of whether or not the image information includes the appearance of the product and the expiration date, expiration date, or barcode of the product.
It is a program to execute.

The information and communication terminal according to the present invention has a storage unit that stores settlement data including the expiration date or the expiration date of the purchased product.
Set the period for sending an alert in advance from the expiration date or expiration date of purchase history data for each purchased item or set the entire purchased item, and link the schedule management application with the period for sending the alert to send an alert. It is an information communication terminal having a control unit that adjusts the alarm time when a schedule is entered at daytime.

The purchased product management method of the information and communication terminal according to the present invention stores the settlement data including the expiration date or the expiration date of the purchased product.
Set the period for sending alerts in advance from the expiration date or expiration date of purchase history data for each purchased item, or set all purchased items in advance, and link the schedule management application with the period for sending the alert to alert This is a purchase product management method for information and communication terminals that adjusts the alarm time when a schedule is set for the outgoing call date and time.

The program according to the present invention can be applied to a computer as an information communication terminal.
Processing to store settlement data including the expiration date or expiration date of purchased products,
Set the period for sending an alert in advance from the expiration date or expiration date of purchase history data for each purchased item, or set all purchased items in advance, and link the schedule management application with the period for sending the alert to alert Processing to adjust the alarm time when there is a schedule in the outgoing call date and time,
Is a program that executes.

According to the present invention, it is possible to settle a product using a product identification method other than a barcode or using the product identification method together with a bar coat, and it is possible to settle even a product for which it is difficult to use a barcode. .. Further, according to the present invention, it is not necessary to input the information of the purchased product. Further, according to the present invention, it is possible to alert at the convenience of the user with the expiration date and / or the expiration date of the purchased product.

It is a block diagram which shows one structural example of the product settlement system which concerns on 1st Embodiment of this invention. It is a block diagram which shows one configuration example of the mobile terminal and the tablet of the product settlement system of FIG. It is a block diagram which shows one configuration example of the product settlement server of the product settlement system of FIG. It is a flowchart which shows the operation of the product settlement system of FIG. It is a flowchart which shows three services of the settlement list display, the household account book display, and the expiration date management of the shopping application for stores which concerns on the 2nd Embodiment of this invention. It is a figure which shows three of the settlement list display, the household account book display, and the expiration date management display. It is a flowchart which shows the operation of the shopping program which combined the alert function and the schedule management function which concerns on 3rd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(First Embodiment)
FIG. 1 is a block diagram of an embodiment of the product settlement system of the present invention. The product settlement system shown in FIG. 1 includes a mobile terminal 10 with an image pickup unit such as a smartphone, a tablet 20 attached to a shopping cart, a product settlement server 30, and a register 40. The mobile terminal 10 and the tablet 20 are information communication terminals.

FIG. 2 is a configuration diagram showing a configuration example of the mobile terminal and tablet of FIG. As shown in FIG. 2, the mobile terminal includes a proximity communication unit 101 that wirelessly communicates with the register 40 (NFC or the like), a communication unit 102 that wirelessly communicates with the product settlement server 30, an image analysis unit 103, and a control unit 104. It includes a unit 105, an imaging unit 106, and a storage unit 107. The control unit 104 controls the proximity communication unit 101, the communication unit 102, the image analysis unit 103, the display unit 105, the image pickup unit 106, and the storage unit 107. The tablet 20 has the same configuration as the mobile terminal shown in FIG. 2, except that the proximity communication unit 101 is not provided. The tablet 20 wirelessly communicates with the register 40 and the product settlement server 30 by the communication unit 102. However, the tablet 20 may include a proximity communication unit 101 as long as proximity wireless communication is possible between the tablet 20 and the register 40.

FIG. 3 is a configuration diagram showing a configuration example of the product settlement server of FIG. The product settlement server 30 includes a communication unit 301, an input unit 302, an image collation unit 303, a control unit 304, a display unit 305, and a database unit 306 that wirelessly communicate with the mobile terminal 10 or the tablet 20. The control unit 304 controls the communication unit 301, the input unit 302, the image collation unit 303, the display unit 305, and the database unit 306.

The control unit 104 of the mobile terminal 10 launches a shopping application, the image pickup unit 106 captures a product, the display unit 105 displays the captured image, and the image analysis unit 103 displays an image if the captured image contains the product. Is stored in the storage unit 107. When a product enters the shopping cart, the tablet 20 takes a picture with the imaging unit 106, displays the taken image on the display unit 105, and the image analysis unit 103 stores the image in the storage unit 107 if the photographed image contains the product. Remember. The shooting is performed so that the appearance of the product is clear, but if the product cannot be specified by the image analysis unit 103, the re-shooting is displayed on the display unit 105. At this time, a missing image may be indicated, and for example, an image of a barcode, an image of an expiration date or an expiration date, an entire image of the appearance of a product, a side image, or the like is displayed.

The user photographs the entire product as the appearance of the product, but may photograph a part of the product if it has a feature. If the product has a product name, a figure suggesting the product, an image color, or a character, take a picture so that the image includes them. If the captured image includes a barcode, it is easier to identify the product, so it is preferable to include the barcode. The image analysis unit 13 extracts feature points for identifying the product from the captured image. The feature points are the appearance of the product, specifically, the shape of the product, the product name on the surface of the product, the figure suggesting the product, the image color, the character, and the like. Since the product can be identified by the appearance of the product when the customer takes a picture, the product can be easily taken and read without being aware of the position of the barcode or the like. In addition, it is possible to identify products such as vegetables for which it is difficult to add a barcode. Since the surface shape of the product is not necessarily flat, it may be difficult for the customer to read the barcode of the product, but even in that case, the product can be identified by the appearance of the product.

The JAN (Japanese Article Number) code is widely used as a barcode, but the JAN code consists of a country code, company code, product item code, and check digit, and includes price information, expiration date and expiration date information. Absent. However, the GS1 data bar has a smaller space than the JAN code, and can be arbitrarily attached to the expiration date, lot number, etc. in addition to the product item code, and multiple product attribute information can be additionally displayed. Therefore, if the GS1 data bar is used, it is not necessary to shoot the expiration date and expiration date information separately from the barcode.

If the captured image includes the expiration date and the expiration date, the image analysis unit 103 extracts the expiration date and the expiration date and stores them in the storage unit 107, so that the control unit 104 is linked with the schedule management function of the mobile terminal. It is possible to alert the expiration date and / and the expiration date. For example, if you are planning a trip or business trip, you can be alerted in advance about the expiration date and expiration date during the trip or business trip. In this case, it is possible to select a product having an earlier expiration date and expiration date according to a travel or business trip schedule, not only after the product is purchased but also before the product is settled. When the expiration date and the expiration date are stored as a character code, the conversion to the character code may be performed by the mobile terminal 10 or the tablet 20 or by the product settlement server 30.

In the image information taken by the mobile terminal 10 or the tablet 20, if it is difficult to shoot information such as the barcode, expiration date or expiration date from one direction, use the video recording function to change the angle of the barcode. , The image information can be obtained more accurately by photographing the expiration date or the expiration date. Further, when it is desired to shoot a product name and a characteristic character in addition to information such as a barcode, an expiration date or an expiration date, a moving image shooting function can be used. For example, when a brand name, a figure, a pattern, and a color suggesting the product are displayed on the side surface of the product, and a barcode is displayed on the back surface, the side surface and the back surface are photographed using the moving image shooting function.

The photographed image information of the product is transmitted to the product settlement server 30 via the communication unit 102. The product settlement server 30 stores image information obtained by photographing the appearance of the product from one direction or a plurality of directions, for example, six directions (vertical direction, front-back direction, left-right direction) in the database unit 306 in association with the settlement data. ing. The image collation unit 303 identifies the settlement data of the product photographed by the mobile terminal 10 by collating and comparing the image information photographed from the six directions with the image information transmitted from the mobile terminal or tablet. Since the product settlement server 30 stores images of the appearance of the product taken from six directions in the database unit 306, the appearance of the product in which the image information transmitted from the mobile terminal 10 or the tablet 20 is taken from one direction. It is possible to collate even the image of. When the product is specified only from the barcode, the product cannot be specified unless the barcode is clearly read. However, since the product is specified from the appearance of the product in this embodiment, the barcode cannot be read. , It is possible to identify the product even if the barcode is not attached. If the product name, the figure suggesting the product, the image color, and the character are in front of the product, the product can be identified if there is an image taken from one direction. In this case, even if the product settlement server 30 does not store the captured images in the six directions (vertical direction, front-back direction, left-right direction) in the database unit 306, if the features of the product appear by shooting from one direction, one Only the captured image of the direction needs to be saved. Even if the product name of the product cannot be recognized as characters, the product may be collated by the arrangement of the product name with respect to the entire product, the color of the characters, or the like.

The administrator of the product settlement server 30 inputs information such as a time-limited price change for a specific product, a price change due to a fixed number of time-limited items, and a time-limited gift addition on the display unit 305 while displaying the information on the input unit 302. The information can be stored in the database unit 306.

FIG. 1 shows an example of purchasing juice using a commodity settlement system. The customer photographs the juice in the paper bag with the mobile terminal 10 or the tablet 20 and sends it to the product settlement server 30. The settlement data is specified by collating and comparing the image information taken from six directions by the image matching unit 303 of the product settlement server 30 with the image information transmitted from the mobile terminal or tablet. The captured image information includes a barcode, and the settlement data shown in FIG. 1 includes, for example, shape: rectangle, size: H: 80 mm, W: 40 mm, D: 20 mm, color: green, barcode: 07125886482758, price: 60 yen (excluding tax) (price is not shown in FIG. 1). The settlement data includes one or more information selected from price, size, color, barcode, product name, time-limited price change, price change due to time-limited fixed quantity, time-limited gift addition, and the like.

The images stored in the product settlement server 30 are not limited to those taken from six directions (vertical direction, front-back direction, left-right direction), and the shooting direction can be determined by the product shape. For example, if the product is bag-shaped, it may be necessary to save only images taken from the front-back direction or only the front direction, and if it is a pentagonal prism, it may be necessary to take pictures and save them in seven or more directions.

Next, the operation of the product settlement system of the present embodiment will be described with reference to the flowchart of FIG. In the following description, an example using the mobile terminal 10 will be described, but the same operation will be performed when the tablet 20 is used. In FIG. 4, a product is photographed using the imaging unit 106 of the mobile terminal 10 (step S1001). Here, the product is photographed so as to include the appearance of the product and the expiration date. The expiration date may be the expiration date. The appearance of the product may be an overall image or a partial image as long as the characteristic points of the product are clear. The image analysis unit 103 determines whether or not the appearance and the expiration date of the product can be recognized from the image (step S1002), and if not, returns to step S1001 and takes a picture again.

When the appearance and the expiration date of the product can be recognized from the image, the control unit 104 saves the product image as the product settlement data (step S1003), and the product image is stored in the product settlement server 30 via the communication unit 102. (Step S1004). The image collation unit 303 of the product settlement server 30 collates the appearance of the product of the product image received via the communication unit 301 with the appearance of the product of the product image stored in the product image database, and searches for the settlement data. (Step S3001).

By collating the product image according to the appearance of the product and identifying the product, when part or all of the packaging shape such as stickers is changed in the product campaign, the campaign is applied to the settlement data related to the product. It can be saved as additional information such as time-limited price change by, price change by time-limited fixed number, time-limited crude product, etc.

In addition, you can prevent forgetting such as discounts by sticking a sticker with a specific pattern (double circle etc.) on the barcode part, but since the product campaign and sticker sticking are changed mainly by the manufacturer, the barcode is Even if they are the same, it is difficult to manage the provision of crude products and the period of provision. In the present embodiment, since the product settlement server 30 recognizes the product by the image, it is possible to add the settlement data individually even for the products having the same barcode.

Depending on the appearance of the product, the product image can be collated to identify the product, but if the same product arrives at the store on different days from the same manufacturer, the expiration date can be specified only by the appearance information of the product. Can not. However, in the present embodiment, in order to read the expiration date information, the product settlement server 30 can determine whether the expiration date is the same day or tomorrow, and 50% discount is applied on that day and 20 is applied on tomorrow. You can set the price like a% discount.

Next, the product settlement server 30 transmits the searched settlement data to the mobile terminal 10 via the communication unit 301 (step S3002). The control unit 104 of the mobile terminal 10 stores the received settlement data in the storage unit 107 (step S1005), accesses it via the register 40 serving as a POS device and the proximity communication unit 101, and transmits the settlement data to the register 40 (step S1005). Step S1006). The register 40 counts the number of products from the settlement data transmitted from the mobile terminal (step S4001), subtracts the number of products from the number of shelves, and confirms the presence or absence of campaign information (step S4002). If there is campaign information, the campaign provision period is confirmed, and the settlement data is rewritten to change price and crude product provision (step S4003). After that, the mobile terminal 10 collates with the data received from the product settlement server 30 and performs the settlement process (step S1007). If there is no campaign information, the process proceeds to step S1007.

After the settlement process in step S1007, the control unit 104 of the mobile terminal 10 stores the settlement list information in the storage unit 107 (step S1008). It is possible to save the settlement list information in the storage unit 107 by an application in the main body of the mobile terminal.

The product settlement system of the present embodiment can be used in self-checkout at convenience stores, supermarkets, etc., and can simplify cashier processing, save labor, and reduce waiting time for customers.

(Second Embodiment)
In the product settlement system of the first embodiment shown in FIG. 1, the shopping application program of the mobile terminal uses the product name, price, tax included, tax excluded, expiration date, or the expiration date from the settlement list information returned from the product settlement server. It is possible to display the expiration date, the presence / absence of a campaign (discount for multiple purchases, etc.), the total amount, the accumulated points / presence / absence of use, etc., and save the past purchase history data in the mobile terminal. In this embodiment, by linking the purchase history data stored in the mobile terminal with the shopping application program, the settlement list display, the household account book display, and the expiration date management (or the expiration date management, both the expiration date and the expiration date) It provides three services (which may be management). Although the information communication terminal of the present embodiment is the mobile terminal 10, it may be a personal computer or the like that stores the settlement list information output from the mobile terminal.

When the user selects the settlement list display function, the settlement list display function is based on the latest purchase history data (for example, the data group of the latest date on which the settlement process is performed) from the purchase history data stored in the mobile terminal. Is a function to call and display. The household account book display function is a function that displays the latest purchase history data by period (default is monthly management). By selecting the display range by the user, the target purchase history data can be aggregated and displayed. The expiration date management function is a function of obtaining and storing the expiration date date / time information for each purchased item using or individually using the expiration date table.

FIG. 5 is a flowchart showing three services of the settlement list display, the household account book display, and the expiration date management of the shopping application program for stores according to the second embodiment of the present invention. As shown in FIG. 5, when the shopping application program of the mobile terminal 10 is started by the control unit 104 (step S1011), the control unit 104 displays the portal screen of the shopping application program on the display unit 105 (step S1012). Then, when the user presses the mode key of the mobile terminal 10, the screen changes to the setting screen for managing the expiration date, the household account book display screen, and the settlement list display screen, and the user is one of the screens. The control unit 104 detects that the above is selected (step S1013).

When the setting screen is selected in step S1013, the control unit 104 makes it possible to set the display period of the household account book (step S1014), and acquires the presence / absence of notification of the expiration date and the notification time from the expiration date table on the display unit 105. A selection screen for selecting whether to set individually or individually is displayed (step S1015). In the following explanation, the case where there is a notification of the expiration date will be described, but if there is no notification of the expiration date, the application program will be terminated.

When the expiration date table is selected in step S1015, the expiration date table is accessed (step S1016), and the expiration date setting of the same category as the purchased product is searched and determined from the product category of the expiration date table (step S1017). After that, the expiration date date and time information is saved for each purchased item of the settlement data (step S1018), the notification time is set, and the application program is terminated (step S1019). By acquiring the expiration date from the expiration date table in this way, it is possible to acquire the expiration date from the expiration date table and set the notification time even for a product whose expiration date is not set by a barcode or the like. For example, when vegetables such as lettuce, cabbage, and carrot are purchased, the expiration date can be obtained from the expiration date table and the notification time can be set. On the other hand, if the individual setting is selected in step S1015, the expiration date is calculated for each purchased product (step S1020), and the expiration date information is saved for each purchased product in the settlement data (step S1021). , The notification time is set and the application program is terminated (step S1022).

When the household account book screen is selected in step S1013, the purchase history based on the settlement data of the latest household account book is displayed by period (default is monthly) (step S1023), and the user selects the display period. The display period is changed with (step S1024), and the application is terminated (step S1025). For example, in the display of the household account book shown in FIG. 6B, the settlement data for the latest three months can be shown.

When the settlement list screen is selected in step S1013, the latest purchase history is displayed (step S1026), and the application is terminated (step S1027). For example, in the display of the settlement list shown in FIG. 6A, the settlement list for the most recent day can be shown.

In the present embodiment, the purchase history data stored in the mobile terminal uses the settlement list information stored in the mobile terminal in the first embodiment, but the purchase history data may be stored by any method. For example, the receipt of the product purchased at the store may be read by the imaging unit of the mobile terminal, and the purchase data stored after character recognition may be used.

The expiration date alert function of the mobile terminal of the present embodiment sets the period for transmitting an alert in advance from the expiration date of the purchase history data for each purchased item and for all purchased items when the expiration date management setting is executed. The expiration date alert method is a notification method such as application startup confirmation, list display of purchase items subject to expiration date, sound, vibration, and e-mail. The expiration date alert function can register the approximate expiration date as the default value for items in the purchase history data. In addition, for the individually set period, by exchanging the default value during the same period when the same item is purchased, it is possible to save the trouble of setting from the next time onward.

When the mobile terminal of the present embodiment executes the expiration date alert function and sets a pre-alert deadline for the expiration date, the mobile terminal with an image pickup unit cooperates with the date and time held by the mobile terminal and the alert function to perform the best-by-date. Deadline advance alert Check for deadlines. If there is an alert deadline, the expiration date alert is executed according to the set method.

(Third Embodiment)
In this embodiment, a shopping application program in which an alert function and a schedule management function are combined with a mobile terminal will be described.

The mobile terminal of this embodiment has a schedule management function in addition to the alarm function, links the schedule management application and data, and sets the alarm time when the schedule is entered in the current expiration date alert transmission date and time. adjust. For example, when there is a trip or business trip scheduled in the schedule, it is possible to set an alarm before the trip or business trip for a product whose expiration date or expiration date comes during the trip or business trip.

FIG. 7 is a flowchart showing the operation of a shopping program that combines an alert function and a schedule management function according to a third embodiment of the present invention. The mobile terminal executes a check for the expiration date at a designated time (step S1111), searches for the presence or absence of purchased items from the settlement list data of the household account book application (step S1112), and ends the process if there is no purchased item. If there is a purchased item, the expiration date of the purchased item is compared with the time of the mobile terminal, the presence or absence of an alert is determined (step S1113), and if the alert is not set, the process ends. FIG. 6C shows an example of the alert setting screen. The user can set whether to notify the expiration date and when to notify by the alert setting screen shown in FIG. 6 (C).

If there is an alert setting, it is determined whether or not there is batting in schedule management at the alert time of the expiration date (step S1114). If there is no batting, an alert is sent from the notification function of the application at the designated time of the purchased item (step S1115), the alert setting is excluded from the purchased item for which the alert has been sent (step S1116), and the process ends.

If there is batting in step S1114, the expiration date alert is sent according to the schedule function (step S1117), the alert setting is excluded from the purchased item for which the alert has been sent (step S1118), and the process ends. It is possible to set an alarm before a trip or a business trip for items that have an expiration date or an expiration date during a trip or a business trip.

The settlement data of the product used in the present embodiment uses the settlement list information stored in the mobile terminal in the first embodiment, but the purchase history data may be stored by any method. For example, the receipt of the product purchased at the store may be read by the imaging unit of the mobile terminal, and the purchase data stored after character recognition may be used.

In the first to third embodiments described above, when a mobile terminal with an image pickup unit such as a smartphone and a shopping application program for stores are activated and a product is photographed by the image pickup unit of the mobile terminal with an image pickup unit, the product is read. Record in the checkout list on the shopping application program. By holding a mobile terminal with an imaging unit over the cash register to perform near-field wireless communication (WiFi, NFC, etc.) and sending the checkout list, only the checkout process is performed, congestion at the cashier is prevented, and the waiting time for customers is shortened. be able to. In addition, as a merit of holding a settlement list on a mobile terminal with an image pickup unit, it also supports the use of product information in a household account book and the expiration date alert of a product.

Although each embodiment of the present invention has been described above, the information communication terminal and the product settlement server can realize all or a part of each function by hardware, software, or a combination thereof. Here, what is realized by software means that it is realized by a computer reading and executing a program.

When realized by software, a part or all of the information communication terminal and the product settlement server may be composed of a computer including a CPU, a hard disk for storing programs, a storage unit such as a ROM, and FIGS. 4, 5, or 7. According to the program according to the above flowchart, the information required for the calculation is stored in the second storage unit such as the RAM, and the processing is executed to execute a part or all of the operations of the information communication terminal and the product settlement server in the program. be able to. The program can be read from an external storage medium such as a CD-ROM, a DVD, or a flash memory in which the program is recorded into a storage unit such as a hard disk.

10 Mobile terminal with imaging unit 20 Tablet with imaging unit 30 Product settlement server 40 Register 101 Proximity communication unit 102 Communication unit 103 Image analysis unit 104 Control unit 105 Display unit 106 Imaging unit 107 Storage unit 301 Communication unit 302 Input unit 303 Image matching Unit 304 Control unit 305 Display unit 306 Database unit

Claims (21)

An imaging unit that captures first image information that can include at least the appearance of the product and a specific part of the product.
An image analysis unit that analyzes the first image information regarding whether or not the appearance of the product and a specific portion of the product are included in the first image information.
A display unit that displays an analysis result of whether or not the appearance of the product and a specific portion of the product are included in the first image information.
When the analysis result indicates that the first image information includes the appearance of the product and a specific part of the product, the image transmission unit that transmits the first image information to the product settlement server and the image transmission unit.
A settlement data receiving unit that receives settlement data of the product from the product settlement server is provided.
When the analysis result indicates that the first image information does not include at least one of the appearance of the product and a specific portion of the product, the display unit displays a display prompting re-photographing.
The settlement data of the product is included in the first settlement data regarding the settlement that can be specified by the barcode of the product, which is specified based on the appearance of the product included in the first image information, and the first image information. Includes second settlement data related to settlement not specified by the barcode of the product, which is specified based on the specific part of the product.
Information communication terminal. By transmitting the received settlement data of the product to a register capable of executing short-range wireless communication with the information communication terminal, the settlement process based on the first settlement data and the second settlement data is executed in the register. The information communication terminal according to claim 1, further comprising a settlement instruction unit. The information communication terminal according to claim 2, further comprising a list storage unit that receives clearing list information indicating the result of the settlement processing from the register and stores the clearing list information in a storage unit. Specific parts of the product include a sticker relating to the campaign of the product.
The second settlement data is campaign information indicating at least one of a price change and a gift offer that can be executed at the time of settlement of the product, which is specified based on the sticker included in the first image information.
The information communication terminal according to any one of claims 1 to 3. The specific part of the product includes the expiration date or the expiration date of the product.
The second settlement data indicates a price change that can be executed at the time of settlement of the product, which is specified based on the expiration date or the expiration date included in the first image information.
The information communication terminal according to any one of claims 1 to 4. The specific part of the product includes the expiration date or expiration date of the product and the seal related to the campaign of the product.
From claim 1, the analysis result displayed by the display unit includes the appearance of the product, the expiration date or expiration date of the product, and the display of a missing image of the stickers related to the campaign of the product. The information communication terminal according to any one of 5. The information communication terminal according to any one of claims 1 to 6, wherein the first image information is acquired by shooting a moving image. A product settlement system that includes an information communication terminal and a product settlement server.
The information communication terminal is
An imaging unit that captures first image information that can include at least the appearance of the product and a specific part of the product.
An image analysis unit that analyzes the first image information regarding whether or not the appearance of the product and a specific portion of the product are included in the first image information.
A display unit that displays an analysis result of whether or not the appearance of the product and a specific portion of the product are included in the first image information.
When the analysis result indicates that the first image information includes the appearance of the product and a specific part of the product, the image transmission unit that transmits the first image information to the product settlement server and the image transmission unit.
A settlement data receiving unit that receives settlement data of the product from the product settlement server is provided.
When the analysis result indicates that the first image information does not include at least one of the appearance of the product and a specific portion of the product, the display unit displays a display prompting re-photographing.
The product settlement server is
An image receiving unit that receives the first image information from the information communication terminal, and
Based on the appearance of the product included in the first image information, the first settlement data relating to the settlement that can be specified by the barcode of the product is specified, and the specific portion of the product included in the first image information is used. Based on the data identification unit that specifies the second settlement data related to the settlement that is not specified by the barcode of the product,
A product settlement system including a settlement data transmission unit that transmits the settlement data of the product including the specified first settlement data and the second settlement data to the information communication terminal. Specific parts of the product include a sticker relating to the campaign of the product.
Based on the sticker included in the first image information, the data specifying unit specifies campaign information indicating at least one of price change and gift offer that can be executed at the time of payment of the product as the second payment data. To do,
The product settlement system according to claim 8. The specific part of the product includes the expiration date or the expiration date of the product.
The data specifying unit specifies a price change that can be executed at the time of settlement of the product as the second settlement data based on the expiration date or the expiration date included in the first image information.
The product settlement system according to claim 8 or 9. The information communication terminal transmits the received settlement data of the product to a register capable of performing short-range wireless communication with the information communication terminal, thereby performing settlement based on the first settlement data and the second settlement data. The product settlement system according to any one of claims 8 to 10, further comprising a settlement instruction unit for causing the register to execute processing. The product settlement system according to claim 11, wherein the information communication terminal receives clearing list information indicating the result of the settlement processing from the register, and has a list storage unit for storing the clearing list information in a storage unit. .. The product settlement server is
A database that combines and accumulates second image information including the appearance of a product taken from one or a plurality of directions and the first settlement data of the product related to the second image information.
An image collation unit that compares and collates the first image information with the second image information to specify the first settlement data is provided.
The settlement data transmission unit transmits the settlement data of the product including the specified first settlement data to the information communication terminal.
The product settlement system according to any one of claims 8 to 12. The database stores the specific portion of the product related to the second image information and the second settlement data of the product related to the second image information in association with each other.
The image collation unit identifies the second settlement data by comparing and collating the specific portion of the product related to the first image information with the specific portion of the product related to the second image information.
The settlement data transmission unit transmits the settlement data of the product including the specified second settlement data to the information communication terminal.
The product settlement system according to claim 13. It is a product settlement method executed on an information communication terminal.
At least the process of capturing the first image information that can include the appearance of the product and a specific part of the product.
A process of analyzing the first image information regarding whether or not the appearance of the product and a specific portion of the product are included in the first image information.
A process of displaying an analysis result of whether or not the appearance of the product and a specific part of the product are included in the first image information.
When the analysis result indicates that the first image information includes the appearance of the product and a specific part of the product, the process of transmitting the first image information to the product settlement server and the process of transmitting the first image information to the product settlement server.
Including the process of receiving the settlement data of the product from the product settlement server.
In the process of displaying the analysis result, when the analysis result indicates that the first image information does not include at least one of the appearance of the product and a specific portion of the product, a display prompting re-photographing is performed. ,
The settlement data of the product is included in the first settlement data regarding the settlement that can be specified by the barcode of the product, which is specified based on the appearance of the product included in the first image information, and the first image information. Includes second settlement data related to settlement not specified by the barcode of the product, which is specified based on the specific part of the product.
Product settlement method. A process of storing the settlement data including the product name, the settlement amount of the product, the expiration date or the expiration date of the product, which is obtained based on the first image information.
Select one of the settlement list display that displays the purchase history including the product name and the settlement amount, the household account book display that includes the product name and the settlement amount, and the management display of the expiration date or the expiration date. With the processing to be provided
The product settlement method according to claim 15. The product settlement method according to claim 16, wherein the process of providing the settlement list display is a process of calling and displaying the latest purchase history data from the stored purchase history data when the settlement list display is selected. .. The process of providing the household account book display is described in claim 16, which is a process of displaying purchase history data by period and a user selecting a display range to aggregate and display the target purchase history data. Product settlement method. The process of providing the management display of the expiration date or the expiration date is a process of individually setting the period for sending an alert in advance from the expiration date or the expiration date of the purchase history data for each purchased item and setting the entire purchased item. The product settlement method according to claim 16. When the alert is executed and the pre-alert deadline for the expiration date or the expiration date is set, the date and time held by the information communication terminal and the alert function are linked to determine the presence or absence of the pre-alert deadline for the expiration date. The product settlement method according to claim 19, which checks and executes an alert according to a set method when there is an alert deadline. The product according to claim 19 or 20, which has a schedule management process, and the schedule management process cooperates with the management of the expiration date or the expiration date to adjust the alarm time when a schedule is entered at the alert transmission date and time. Settlement method.