JP2022190356A - Accounting system, accounting method, and accounting program - Google Patents

Abstract

To provide an accounting system, an accounting method, and an accounting program that are capable of easily performing accounting processing of usage fee to a user.SOLUTION: An accounting system includes: a merchandise acquisition processing unit that obtains purchased merchandise information corresponding to information of a purchased merchandise of a user; an image acquisition processing unit that obtains a captured image obtained by capturing an image of the user; an extraction processing unit that extracts a user image of the user from the captured image that is obtained by the image acquisition processing unit; a registration processing unit that registers, on an accounting screen displayed on an accounting terminal, a user identification image corresponding to the user image that is extracted by the extraction processing unit and the purchased merchandise information that is obtained by the merchandise acquisition processing unit, in association with each other; and an accounting processing unit that makes the accounting terminal execute accounting processing to the user corresponding to the user identification image and the purchased merchandise information that are registered by the registration processing unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to an accounting system, accounting method, and accounting program for accounting for products purchased by a user.

In restaurants, etc., there are cases where the total usage amount (eating and drinking charges, etc.) of a group including multiple users is split. Conventionally, techniques have been proposed that enable prompt accounting processing by splitting the bill. For example, in Patent Document 1, for each order of a group, the total amount of the order and the split bill amount according to the number of people at the time of ordering are calculated, and the printer passes information on the calculated split bill amount to the user together with the ordered items. A system for printing receipts is disclosed.

JP 2005-165756 A

Here, for example, regarding the total usage amount of the group, when each user pays the amount of the product ordered by himself/herself, the amount obtained by dividing the total usage amount by the number of users shall be the payment amount of each user. Therefore, the store clerk needs to talk with the user to confirm who will pay for which product, and perform accounting processing by associating each product with each user. For this reason, there arises a problem that accounting processing takes time and effort.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an accounting system, accounting method, and accounting program capable of easily accounting for usage fees of users.

An accounting system according to the present invention includes a product acquisition processing unit that acquires purchased product information corresponding to information on a product purchased by a user, an image acquisition processing unit that acquires a captured image of the user, and the image acquisition unit. an extraction processing unit for extracting a user image of the user from the captured image acquired by a processing unit; and an accounting screen displayed on an accounting terminal corresponding to the user image extracted by the extraction processing unit. A registration processing unit that associates and registers a user identification image and the purchased product information acquired by the product acquisition processing unit, and corresponds to the user identification image and the purchased product information that are registered by the registration processing unit. and a billing processing unit that causes the billing terminal to execute billing processing of the user.

An accounting method according to the present invention includes a product acquisition step of acquiring purchased product information corresponding to information on a product purchased by a user, and an image acquisition step of acquiring a captured image of the user, by one or more processors. an extraction step of extracting a user image of the user from the captured image acquired in the image acquisition step; a registration step of associating and registering a corresponding user identification image and the purchased product information acquired in the product acquisition step; and corresponding to the user identification image and the purchased product information registered in the registration step and a billing step of causing the billing terminal to execute billing processing of the user.

The accounting program according to the present invention comprises a product acquisition step of acquiring purchased product information corresponding to information on a product purchased by a user; an image acquisition step of acquiring a captured image of the user; an extracting step of extracting a user image of the user from the acquired captured image; and a user identification image corresponding to the user image extracted in the extracting step on a billing screen displayed on a billing terminal. , a registration step of registering in association with the purchased product information acquired in the product acquisition step; and accounting processing of the user corresponding to the user identification image and the purchased product information registered by the registration step. and accounting steps to be executed by the accounting terminal.

According to the present invention, it is possible to provide an accounting system, an accounting method, and an accounting program capable of easily performing accounting processing for usage fees of users.

FIG. 1 is a block diagram showing the configuration of an accounting system according to an embodiment of the invention. FIG. 2 is a diagram showing an example of an image analysis information table used in the accounting system according to the embodiment of the invention. FIG. 3 is a diagram showing an example of accounting information used in the accounting system according to the embodiment of the invention. FIG. 4 is a diagram showing an example of purchased product information used in the accounting system according to the embodiment of the present invention. FIG. 5 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 6 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 7 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 8 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 9 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 10 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 11 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 12 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 13 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 14 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 15 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 16 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 17 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 18 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 19 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 20 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 21 is a diagram showing an example of a checkout screen displayed on the checkout terminal according to the embodiment of the present invention. FIG. 22 is a flow chart showing an example of the accounting support processing procedure executed by the accounting terminal according to the embodiment of the present invention. FIG. 23 is a flow chart showing an example of the accounting support processing procedure executed by the accounting terminal according to the embodiment of the present invention. FIG. 24 is a flow chart showing an example of the accounting support processing procedure executed by the accounting terminal according to the embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings for understanding of the present invention. It should be noted that the following embodiment is an example that embodies the present invention, and does not limit the technical scope of the present invention.

[Accounting system 100]
As shown in FIG. 1, accounting system 100 according to the embodiment of the present invention includes accounting terminal 1, camera 2, and database DB. Accounting terminal 1, camera 2, and database DB can communicate via communication network N1 such as LAN, Internet, WAN, or public telephone line.

The camera 2 is a digital camera that takes an image of a subject and outputs it as digital image data. The camera 2 has a communication function for transmitting digital image data to the accounting terminal 1 through the communication network N1. The number of cameras 2 may be one or plural. Specifically, the camera 2 is installed in the store and captures an image of a predetermined area including users (customers) of the store. For example, the camera 2 is installed at a cash register of a store, and captures an image of a predetermined area inside the store from the cash register. Camera 2 captures an image of a predetermined area at a predetermined frame rate and transmits image data to checkout terminal 1 . The predetermined area is, for example, an area in which a user who pays can be imaged. Also, the predetermined area may be, for example, the entire area within the store. In the present invention, it is also possible to apply an existing camera to the store as the camera 2 . Alternatively, the camera 2 may be mounted on the accounting terminal 1 .

The checkout terminal 1 is a device (for example, a POS terminal) that is installed in a store and performs checkout for products purchased by a user. The accounting terminal 1 may accept an operation by a store clerk to perform accounting, or may accept an operation by a user to perform accounting. In addition, the accounting terminal 1 recognizes the user who makes the transaction based on the captured image of the predetermined area captured by the camera 2, and identifies the user (user's face image) and the user's purchase information. It is registered in association with purchase product information related to the product. The store is, for example, a restaurant, a product store (retail store), or the like. In the following embodiments, a restaurant will be described as an example of a store.

In this embodiment, the accounting terminal 1 alone corresponds to the accounting system according to the present invention, but the accounting system according to the present invention may include accounting terminal 1, camera 2, and database DB. Alternatively, the accounting terminal 1, the camera 2, and the database DB may be integrated.

[accounting terminal 1]
As shown in FIG. 1, accounting terminal 1 is an information processing device (for example, a POS terminal) including control unit 11, storage unit 12, operation display units 13A and 13B, communication unit 14, and the like. Note that the accounting terminal 1 is not limited to one computer, and may be a computer system in which a plurality of computers operate in cooperation. Also, various processes executed by the accounting terminal 1 may be distributed and executed by one or more processors. Accounting terminal 1 is an example of the accounting system of the present invention.

The communication unit 14 connects the accounting terminal 1 to the communication network N1 by wire or wirelessly, and performs data communication according to a predetermined communication protocol with one or more external devices such as the camera 2 via the communication network N1. It is a communication interface for execution.

The operation display units 13A and 13B are user interfaces each having a display unit such as a liquid crystal display or an organic EL display that displays various information, and an operation unit such as a mouse, keyboard, or touch panel that receives operations. . An operation display unit 13A (hereinafter referred to as a store clerk operation display unit) is an operation display unit for a store clerk, and is operated by the store clerk and displays information for the store clerk. The operation display section 13B (hereinafter referred to as a user operation display section) is an operation display section for the user, and is operated by the user and displays information to the user.

The storage unit 12 is a non-volatile storage unit including a semiconductor memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or the like that stores various types of information. Specifically, the storage unit 12 stores an image analysis information table D1 for registering analysis results of captured images captured by the camera 2, and the like. FIG. 2 is a diagram showing an example of the image analysis information table D1. Note that the captured images include images of one or more users who pay at the register.

As shown in FIG. 2, the image analysis information table D1 stores a user number, a face area (horizontal direction), a face area (horizontal direction), and a face area ( vertical direction) is registered. The user number is user identification information set corresponding to the face image included in the captured image. The user number may be set according to an operation of selecting the face image by a store clerk. The face area (horizontal direction and vertical direction) is information indicating the position and range of the face image included in the captured image. When the captured image is acquired, the control unit 11 analyzes (image analysis) the captured image and registers each of the information in the image analysis information table D1.

The control unit 11 registers each information in the image analysis information table D1 based on the captured image. As another embodiment, the image analysis information table D1 may be stored in another server accessible from the checkout terminal 1 via the communication network N1. In this case, the control unit 11 of the accounting terminal 1 may acquire various kinds of information from the server as necessary and execute each process such as accounting support processing (see FIGS. 22 to 24) described later. .

Further, the control unit 11 executes accounting processing based on various information stored in the database DB. Data such as product information (not shown), accounting information D2, and purchased product information D3 are stored in the database DB. The product information includes information (product name, price, etc.) on all products that can be offered at the restaurant. FIG. 3 is a diagram showing an example of accounting information D2, and FIG. 4 is a diagram showing an example of purchased product information D3.

As shown in FIG. 3, the transaction information D2 includes information such as the transaction number, the number of people in the group, the number of products purchased, and the total amount for each transaction. The account number is identification information assigned to each account. The number of people in a group is the total number of people in a group, such as family customers and group customers, and is, for example, the number of people using a table in a restaurant. The number of purchased products is the total number of products to be accounted for. The total amount is the total amount of all purchased products for each group.

As shown in FIG. 4, the purchased product information D3 includes information such as the account number, purchased product (ordered product), and user number for each transaction. In the case of a group including a plurality of users, for example, a plurality of purchased products are registered in association with the same accounting number. Further, each purchased product is registered in association with the user number of the user who pays for the purchased product.

The accounting information D2 and the purchased product information D3 may be stored in the database DB based on information input to an order terminal (not shown), for example.

The storage unit 12 also stores a control program such as an accounting program for causing the control unit 11 to execute accounting support processing (see FIGS. 22 to 24), which will be described later. For example, the accounting program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and read by a reader (not shown) such as a CD drive or DVD drive provided in the accounting terminal 1. It is stored in the storage unit 12 .

The control unit 11 has control devices such as a CPU, ROM, and RAM. The CPU is a processor that executes various kinds of arithmetic processing. The ROM is a non-volatile storage unit in which control programs such as BIOS and OS for causing the CPU to execute various arithmetic processes are stored in advance. The RAM is a volatile or nonvolatile storage unit that stores various information, and is used as a temporary storage memory (work area) for various processes executed by the CPU. The control unit 11 controls the accounting terminal 1 by causing the CPU to execute various control programs pre-stored in the ROM or storage unit 12 .

Specifically, as shown in FIG. 1, the control unit 11 includes a product acquisition processing unit 111, an image acquisition processing unit 112, an extraction processing unit 113, a reception processing unit 114, a registration processing unit 115, a display processing unit 116, and an accounting processing unit. Various processing units such as a unit 117 and a question processing unit 118 are included. The control unit 11 functions as the various processing units by executing various types of processing according to the accounting program by the CPU. Also, part or all of the processing units included in the control unit 11 may be configured by electronic circuits. The accounting program may be a program for causing a plurality of processors to function as the various processing units.

The product acquisition processing unit 111 acquires purchased product information corresponding to the purchased product information of the user. Specifically, the product acquisition processing unit 111 refers to the database DB to acquire accounting information D2 (see FIG. 3) and purchased product information D3 (see FIG. 4) corresponding to the user. The accounting information D2 and the purchased product information D3 are examples of the purchased product information. For example, in a restaurant, when a user finishes a meal and pays at a cash register, when the user hands a slip to a cash register clerk, the clerk reads the information code of the slip with a reader or the like. As a result, the product acquisition processing unit 111 acquires the purchased product information of the user based on the account number (see FIGS. 3 and 4) corresponding to the information code. The product acquisition processing unit 111 is an example of the product acquisition processing unit of the present invention.

The image acquisition processing unit 112 acquires a captured image of a predetermined area including the user in front of the cash register of the store captured by the camera 2 . The image acquisition processing section 112 is an example of the image acquisition processing section of the present invention.

The extraction processing unit 113 extracts a user image of the user from the captured image acquired by the image acquisition processing unit 112 . Specifically, the extraction processing unit 113 extracts the face image of the user from the captured image. When a plurality of users are included in the captured image, the extraction processing unit 113 extracts a face image for each user. After extracting the face image, the extraction processing unit 113 registers the face region information of the face image in the image analysis information table D1 (see FIG. 2) in association with the face number. Note that the extraction processing unit 113 may extract the user image of the user from the captured image when a predetermined user operation (for example, an instruction operation to start person identification) is received on the accounting screen (described later). The extraction processing unit 113 is an example of the extraction processing unit of the present invention.

The reception processing unit 114 receives various operations performed by the store clerk and the user on the accounting terminal 1 . A specific example of the operation will be described later. The reception processing unit 114 is an example of the reception processing unit of the present invention.

The registration processing unit 115 registers the user identification image corresponding to the user image extracted by the extraction processing unit 113 and the user identification image acquired by the product acquisition processing unit 111 on the checkout screen displayed on the clerk operation display unit 13A. It is registered in association with the purchase product information. Specifically, the registration processing unit 115 adds the user's facial image and information (purchased product name ) is associated with and registered. The registration processing unit 115 also registers the user number of the user who pays for the purchased product in the purchased product information D3 (see FIG. 4). The registration processing unit 115 is an example of the registration processing unit of the present invention. The face image is an example of the user identification image of the present invention. Note that the user identification image of the present invention is not limited to a face image, and may be an image of a part of the body with which the user can be identified, or an image of belongings with which the user can be identified.

The display processing unit 116 causes the store clerk operation display unit 13A and the user operation display unit 13B to display various accounting screens. Specifically, the display processing unit 116 causes the clerk operation display unit 13A to display the captured image acquired by the image acquisition processing unit 112, and the user image (face image) of the user extracted from the captured image. image), and the identification information (user number) of the user is displayed on the clerk operation display unit 13A. In addition, the display processing unit 116 displays the face image of the user and the purchased product information of the purchased product of the user in association with each other on the checkout screen. The display processing section 116 is an example of the display processing section of the present invention.

The accounting processing unit 117 executes accounting processing of the user corresponding to the face image registered by the registration processing unit 115 and the purchased product information.

Specific examples are shown below. FIG. 5 shows an example of a checkout screen displayed on the clerk operation display section 13A. On the checkout screen shown in FIG. 5, the store clerk selects either a lump sum payment or a split payment as the user's accounting method. For example, when the clerk asks the user about the accounting method and the user selects split payment, the clerk touches "split payment" on the accounting screen shown in FIG. The reception processing unit 114 receives the touch operation of the store clerk.

When "split the bill" is selected, the display processing unit 116 causes the clerk operation display unit 13A to display the checkout screen shown in FIG. The checkout screen shown in FIG. 6 includes information on all products that can be ordered at a restaurant, a photographed image P1 of the users (here, a group of three with checkout number “2”) who performs the checkout, and an image of the group. and a user-by-user display field P2 for displaying and editing identification information of each user and product information on products to be paid. Further, as shown in FIG. 6, the display processing unit 116 displays the captured image P1 on the checkout screen, and also displays the user numbers of each user recognized from the captured image P1 (here, "1", "2", "3") is displayed near the user image (for example, face image). The user number may be automatically assigned by the control unit 11, or may be assigned according to a selection operation by a store clerk. As a result, the store clerk can grasp the number of users on the checkout screen.

Further, as shown in FIG. 6, the display processing unit 116 displays the user number and the user's face image for each recognized user in the user-by-user display field P2 of the checkout screen. In the example of FIG. 6, the face image of the user is associated with the user number "1" corresponding to the rightmost user included in the captured image P1, and the leftmost user included in the captured image P1 is displayed. The user number "2" corresponding to the user is displayed in association with the face image of the user, and the user number "3" corresponding to the central user included in the captured image P1 is displayed. The user's face image is associated and displayed.

The reception processing unit 114 receives a selection operation for selecting a predetermined purchase product from the store clerk on the checkout screen. For example, the store clerk verbally asks the user (a group of three) who pays "700 yen" for the "fried chicken set meal". When the user with the user number "1" (the user on the far right) answers that he will pay "700 yen" for the "fried chicken set meal", the clerk displays the purchased product list on the checkout screen as shown in FIG. At P3, the user selects (touches) "fried chicken set meal". The display processing unit 116 changes the display mode (for example, the display color) of the "fried chicken set meal" touched by the store clerk. The store clerk moves the information of "fried chicken set meal" from the purchased product list P3 to the registration column of the user number "1" in the display column P2 for each user on the checkout screen (see FIG. 8). For example, the store clerk performs an operation (drag-and-drop operation) to move an image icon including the information of "fried chicken set meal" from the purchased product list P3 to the registration field of the user number "1" in the user-specific display field P2. (see Figure 9). The display processing unit 116 causes the user number "1" to be displayed in association with the information of "fried chicken set meal" and "700 yen". Further, the registration processing unit 115 registers the user number “1” in association with the information of “fried chicken set meal” in the purchased product information D3 (see FIG. 4). In this manner, the registration processing unit 115 allocates the payment for the “fried chicken set meal” to the user with the user number “1” according to the operation of the store clerk. It should be noted that the display processing unit 116 deletes the information of "fried chicken set meal" from the purchased product list P3 (see FIG. 9).

Similarly, the store clerk allocates the payment for "Katsudon" and "Ramen" to one of the users. Here, the display processing unit 116 causes the user number “2” to display the information of “Katsudon” and “600 yen” in association with the information of “Ramen” and “550 yen” (see FIG. 10). ). The display processing unit 116 also displays the total amount of "1150 yen" for "Katsudon" and "Ramen". In addition, the registration processing unit 115 registers the information of "katsudon" and "ramen" in association with the user number "2" in the purchased product information D3 (see FIG. 4). The registration processing unit 115 repeatedly executes the above-described processing according to the operations of the store clerk until all the purchased products displayed in the purchased product list P3 are assigned to the user. Note that the display processing unit 116 may change the display color (background color) of the user-by-user display field P2 when all the purchased products displayed in the purchased product list P3 are assigned to the user. Thereby, the store clerk can easily recognize that all the purchased products displayed in the purchased product list P3 have been assigned to the user.

When all the purchased products displayed in the purchased product list P3 are assigned to the user and displayed in the user-by-user display field P2, the reception processing unit 114 receives the salesclerk's checkout operation. For example, on the checkout screen shown in FIG. 11, when the clerk touches the checkout button for the purchased product “Karaage set meal” corresponding to the user number “1”, the reception processing unit 114 accepts the checkout operation, and the display processing unit 116 displays accounting information corresponding to user number "1" (see FIG. 12). When the user deposits the price of "1000 yen" or when the store clerk receives and deposits the price of "1000 yen" from the user, the accounting processing unit 117 executes the settlement corresponding to the user number "1". Subsequently, the store clerk touches the checkout button for the purchased product "fried chicken set meal" corresponding to user number "2" (see FIG. 11), and the checkout processing unit 117 completes the payment corresponding to user number "2". Run.

As described above, the accounting processing unit 117 executes accounting processing by splitting the bill for the purchased product (“fried chicken set meal”, “katsudon”, and “ramen”) with the accounting number “2”. For example, when the store clerk asks the user about the accounting method and the user selects lump-sum payment, the store clerk touches "lump-sum payment" on the accounting screen shown in FIG. In this case, the accounting processing unit 117 executes accounting processing for the total amount (here, "1150 yen").

That is, in accounting terminal 1, when user images of a plurality of users are included in the captured image, extraction processing unit 113 extracts the user images of each of the plurality of users, and registration processing unit 115 registers the purchased product information corresponding to each of the user identification images (face images) on the accounting screen, and the accounting processing unit 117 performs accounting processing of the purchased products of the user for each user. to run.

The invention is not limited to the embodiments described above. In the above-described embodiment, the registration processing unit 115 registers the user number, face image, and purchased product information by performing the operation of assigning each purchased product displayed in the purchased product list P3 to the user-by-user display field P2. is registered in association with That is, the reception processing unit 114 performs a first operation (see FIG. 7) to select a purchased product for accounting from the purchased product list P3 on the accounting screen, and performs user identification of the purchased product selected by the first operation. A second operation (see FIGS. 8 and 9) for associating with an image (face image) is accepted, and when the registration processing unit 115 accepts the first operation and the second operation, the face image and the first operation are accepted. and the purchased product information of the purchased product selected by are associated with each other and registered.

As another embodiment, the registration processing unit 115 may associate and register the face image and the purchased product information on the checkout screen based on the answer to the user's question. For example, as shown in FIG. 13, the clerk selects "fried chicken set meal" in the purchased product list P3 on the checkout screen, and touches the person identification start button B1. Then, the question processing unit 118 asks the user a question about the purchased product. For example, as shown in FIG. 14, the question processing unit 118 asks who the user ordered the "fried chicken set meal". Here, the question processing unit 118 prompts the user to answer by a preset gesture operation (for example, "raise hand"). The question processing unit 118 may display the question on the user operation display unit 13B (see FIG. 14), or may output the question by voice.

The registration processing unit 115 associates and registers the face image and the purchase product information based on the user's gesture operation in response to the question. For example, as shown in FIG. 15, when the user with the user number “1” raises his hand in response to the question, the registration processing unit 115 recognizes the hand-raising gesture operation and displays the face image of the user number “1”. and the information of "fried chicken set meal" are associated and registered. Further, the registration processing unit 115 registers the user number “1” in association with the information of “fried chicken set meal” in the purchased product information D3 (see FIG. 4). In addition, the display processing unit 116 causes the user number "1" to be displayed in association with the information of "fried chicken set meal" and "700 yen".

Similarly, the registration processing unit 115 also registers "katsudon" and "ramen" by associating the face image with the purchase product information based on the user's gesture operation in response to the question.

In the above-described embodiment, the registration processing unit 115 associates and registers the face image and the purchased product information through the operation of the store clerk. As another embodiment, the registration processing unit 115 may associate and register the face image and the purchase product information only by the operation of the user without the operation of the store clerk. This configuration can be applied, for example, to a self-checkout where a user operates the checkout terminal 1 to checkout. Specifically, the display processing unit 116 causes the user operation display unit 13B to display the checkout screen shown in FIG. When the user touches "split the bill" on the accounting screen, the reception processing unit 114 accepts the user's touch operation, and the display processing unit 116 displays the accounting screen shown in FIG.

When the user selects "fried chicken set meal" in the purchased product list P3 on the checkout screen of FIG. 16, the question processing unit 118 asks the user a question about the purchased product (see FIG. 14). The question processing unit 118 prompts the user to answer by a preset gesture operation (for example, "raise hand"). The registration processing unit 115 associates and registers the face image and the purchase product information based on the user's gesture operation in response to the question. For example, as shown in FIG. 17, when the user with the user number “1” raises his hand in response to the question, the registration processing unit 115 recognizes the hand-raising gesture operation and displays the face image of the user number “1”. and "Karaage Set Meal" information are associated and registered. Further, the registration processing unit 115 registers the user number “1” in association with the information of “fried chicken set meal” in the purchased product information D3 (see FIG. 4). Further, the display processing unit 116 associates the user number "1" with the information of "fried chicken set meal" and "700 yen" and displays them (see FIG. 17).

Similarly, the registration processing unit 115 also registers "Katsudon" and "Ramen" by associating the face image with the purchase product information based on the user's gesture operation in response to the question (see FIG. 18). .

18, when the user touches the checkout button for the purchased product "fried chicken set meal" corresponding to the user number "1", the reception processing unit 114 accepts the checkout operation, and the checkout processing unit 117 executes the settlement corresponding to the user number "1". In this way, the user himself/herself may perform the operation of splitting the bill.

Here, the gesture operation is not limited to "raising hand" and may be another operation. For example, the gesture operation may be a "nod", a "V sign", or the like. Further, as shown in FIG. 19, information on a plurality of gesture operations is registered in advance in the storage unit 12, and a user (for example, store manager) selects a gesture operation (discrimination gesture operation) used for answering the question. It may be configurable.

Further, as a method of answering the question, a voice answer may be used instead of the gesture operation. Specifically, the registration processing unit 115 associates and registers the face image and the purchase product information based on the user's uttered voice in response to the question. For example, as shown in FIG. 20, the question processing unit 118 urges the user to answer by uttering a preset voice (for example, "yes"). For example, when the user with the user number “1” says “yes” to the above question, the registration processing unit 115 registers the face image of the user number “1” in association with the information of “fried chicken set meal”. do.

Note that the voice is not limited to "Yes", and may be, for example, "It's me" or "I ate". Further, as shown in FIG. 21, information on a plurality of voices is registered in advance in the storage unit 12, and the user (for example, store manager) can set voices (audios for discrimination) to be used for answering the questions. There may be.

[Accounting support processing]
Accounting support processing executed by the control unit 11 of the accounting terminal 1 will be described below. For example, the control unit 11 starts the accounting support process when the user hands over the slip to the clerk in charge of the cash register, and the clerk reads the information code of the slip with a reading device or the like, thereby starting the execution of the accounting program. do.

The present invention can be regarded as an invention of an accounting method for executing one or more steps included in the accounting support processing. Also, one or more steps included in the accounting support processing described here may be omitted as appropriate. Further, each step in the accounting support process may differ in execution order as long as the same effects are produced. Furthermore, here, a case where the control unit 11 executes each step in the accounting support process will be described as an example, but in another embodiment, one or more processors distribute each step in the accounting support process. You can also run

FIG. 22 is a flow chart showing an example of a procedure of user registration processing included in the accounting support processing. When performing the accounting support process, the control unit 11 first executes a user registration process for recognizing the user based on the captured image.

In step S<b>1 , the control unit 11 acquires purchased product information corresponding to the purchased product information of the user. Specifically, the control unit 11 refers to the database DB and acquires the accounting information D2 (see FIG. 3) and the purchased commodity information D3 (see FIG. 4) corresponding to the user. For example, the control unit 11 acquires the account number from the slip, and acquires the account information D2 and the purchased product information D3 associated with the account number. Step S1 is an example of the product acquisition step of the present invention.

Next, in step S2, the control unit 11 determines whether or not the accounting method of the user is split payment. The control unit 11 determines the accounting method based on the selection operation of the store clerk on the accounting screen shown in FIG. 5, for example. If the accounting method is split payment (S2: Yes), the process proceeds to step S3. The control unit 11 displays the purchase product list P3 on the checkout screen (see FIG. 6).

On the other hand, if the accounting method is not split payment, that is, if the accounting method is lump sum payment (S2: No), the process proceeds to step S201. In step S201, the control unit 11 executes accounting processing by lump-sum payment, and terminates the accounting support processing.

In step S3, the control unit 11 acquires a captured image P1 of a predetermined area including the user in front of the checkout of the store captured by the camera 2. FIG. The control unit 11 displays the acquired captured image P1 on the checkout screen (see FIG. 6). Step S3 is an example of the image acquisition step of the present invention.

Next, in step S4, the control section 11 sets the face number of the image analysis information table D1 (see FIG. 2) to "0".

Next, in step S5, the control unit 11 starts scanning (scanning process) of the acquired captured image P1. For example, the control unit 11 scans the captured image P1 from the right side in the horizontal direction.

Next, in step S6, the control unit 11 determines whether or not a face image has been detected from the captured image P1. When the control unit 11 detects the face image from the captured image P1 (S6: Yes), the process proceeds to step S7. On the other hand, when the control unit 11 does not detect the face image from the captured image P1 (S6: No), the process proceeds to step S10.

In step S7, the control unit 11 extracts the face image from the captured image P1, and stores the face number "1" and the face area (horizontal direction and vertical direction) in the image analysis information table D1 (see FIG. 2). sign up. Steps S6 and S7 are examples of extraction steps of the present invention.

Next, in step S8, the control unit 11 displays the user number "1" near the detected face image (see FIG. 6). Further, the control unit 11 registers the user number “1” in association with the face number “1” (see FIG. 2).

Next, in step S9, the control unit 11 displays the user number "1" and the face image of the user in the user-by-user display field P2 of the checkout screen (see FIG. 6).

Next, in step S10, the control unit 11 determines whether scanning has been completed up to the end (here, the left end) of the captured image P1. When the scanning is completed (S10: Yes), the control unit 11 ends the user registration process. On the other hand, when the scanning is not completed (S10: No), the controller 11 proceeds to step S101 and continues the scanning.

When the control unit 11 continues scanning the captured image P1 and detects the next face image (S6: Yes), it executes the processes of steps S7 to S9 described above. In this manner, the control unit 11 repeats the above-described processing until no face image is detected from the captured image P1. After completing the user registration process, the control unit 11 executes the following process of splitting the bill. FIG. 6 shows a state in which the control unit 11 has completed the user registration process for the captured image P1.

FIG. 23 is a flow chart showing an example of the procedure of the split bill process. After completing the user registration process (see FIG. 22), the control unit 11 executes a split bill process for accounting by split bill payment according to the operation of the user (clerk).

In step S11, the control unit 11 determines whether or not a purchased product has been selected from the purchased product list P3 (see FIG. 6) on the checkout screen. If a purchased product is selected from the purchased product list P3 (S11: Yes), the process proceeds to step S12 (see FIG. 7). In step S12, the control unit 11 changes the display mode (for example, display color) of the selected purchased product (see FIG. 7).

Next, in step S13, the control unit 11 determines whether or not the selected purchased product has been moved to the user-specific display field P2. For example, the store clerk performs an operation (drag and drop) to move an image icon including information on "fried chicken set meal" from the purchased product list P3 to the registration column for the user number "1" in the user-specific display column P2. In this case (see FIGS. 8 and 9), the control unit 11 determines that the selected “fried chicken set meal” has been moved to the user-specific display field P2 (S13: Yes).

In step S14, the control unit 11 associates the face number, the user number, and the purchased product information with each other and registers them. For example, the control unit 11 registers the user number "1" corresponding to the face number "1" in association with the information "fried chicken set meal" in the purchased product information D3 (see FIG. 4).

Next, in step S15, the control unit 11 displays the name of the purchased product and the amount of money in the user-by-user display field P2. For example, the control unit 11 causes the user number "1" to be displayed in association with the information of "fried chicken set meal" and "700 yen" (see FIG. 9). Further, in step S16, the control unit 11 displays the total payment amount of the user in the user-by-user display field P2 (see FIG. 9).

Next, in step S17, the control unit 11 determines whether or not all purchased products included in the purchased product list P3 have been registered (displayed) in the user-by-user display field P2. When all the purchased products are registered in the user-specific display column P2 (S17: Yes), the process proceeds to step S18. If all the purchased products are not registered in the user-specific display field P2 (S17: No), the control unit 11 proceeds to step S11. The control unit 11 repeats the above-described processing until all the purchased products are registered in the user-specific display field P2. FIG. 10 shows a state in which the control unit 11 has registered all the purchased products in the user-by-user display field P2.

Subsequently, in step S18, the control unit 11 determines whether or not the accounting for the purchased product displayed in the user-by-user display field P2 has been selected. For example, on the checkout screen shown in FIG. 11, when the clerk touches the checkout button for the purchased product "fried chicken set meal" corresponding to the user number "1", the control unit 11 selects the checkout for "fried chicken set meal". (S18: Yes). If accounting for the purchased product displayed in the user-specific display field P2 is selected (S18: Yes), the process proceeds to step S19. On the other hand, if the payment for the purchased product displayed in the user-by-user display field P2 is not selected (S18: No), the process proceeds to step S21 (see FIG. 24).

In step S19, the control unit 11 executes accounting processing for the selected purchased product. For example, when the user deposits the price "1000 yen" or when the store clerk receives and deposits the price "1000 yen" from the user, the control unit 11 executes the settlement corresponding to the user number "1" ( See Figure 12). Step S19 is an example of an accounting step.

Next, in step S20, the control unit 11 determines whether or not the accounting for all the purchased products displayed in the user-by-user display field P2 has been completed. When the accounting for all the purchased products is finished (S20: Yes), the control unit 11 finishes the split bill process. If accounting for all the purchased products has not been completed (S20: No), the process proceeds to step S18, and the control section 11 repeats the above-described processes.

Here, in step S21, the control unit 11 determines whether or not cancellation of the purchased product displayed in the user-by-user display field P2 has been selected. For example, on the checkout screen shown in FIG. 11, when the clerk touches the cancel button of the purchased product "fried chicken set meal" corresponding to the user number "1", the control unit 11, in step S22, displays the face number and the user number. , and cancel the registration of the purchased product. Next, in step S23, the control unit 11 deletes the purchased product name and the amount from the user-specific display column P2. Next, in step S24, the control unit 11 displays the changed total payment amount in the user-specific display field P2. Further, in this case, the control unit 11 redisplays the information of the canceled purchased product “fried chicken set meal” in the purchased product list P3. After that, the process proceeds to step S11, and the control unit 11 executes the process of allocating the purchased product "fried chicken set meal" to other users.

As described above, the control unit 11 executes the accounting support processing. Note that the control unit 11 may execute processing according to the above-described other embodiments (see FIGS. 13 to 21) in the accounting support processing.

As described above, the accounting system 100 according to the present embodiment acquires purchased product information corresponding to information on purchased products of a user, acquires a captured image of the user, and acquires the acquired captured image. to extract the user image of the user from. In addition, the accounting system 100 associates and registers the user identification image (face image) corresponding to the extracted user image and the acquired purchased product information on the accounting screen displayed on the accounting terminal 1, Accounting processing for the user corresponding to the registered user identification image and purchased product information is executed. This makes it possible to easily perform accounting processing by splitting the bill. In addition, it is possible to reduce the work load of the store clerk when accounting for the split payment.

1: accounting terminal 2: camera 13A: clerk operation display unit 13B: user operation display unit 100: accounting system 111: product acquisition processing unit 112: image acquisition processing unit 113: extraction processing unit 114: reception processing unit 115: Registration processing unit 116: Display processing unit 117: Accounting processing unit 118: Question processing unit

Claims (11)

a product acquisition processing unit that acquires purchased product information corresponding to the purchased product information of the user;
an image acquisition processing unit that acquires a captured image of the user;
an extraction processing unit that extracts a user image of the user from the captured image acquired by the image acquisition processing unit;
A user identification image corresponding to the user image extracted by the extraction processing unit and the purchased product information acquired by the product acquisition processing unit are associated and registered on the accounting screen displayed on the accounting terminal. a registration processing unit;
an accounting processing unit that causes the accounting terminal to execute accounting processing of the user corresponding to the user identification image and the purchased product information registered by the registration processing unit;
accounting system. further comprising a display processing unit for displaying the purchased product information and the captured image on the checkout screen;
The display processing unit causes the accounting screen to display the user identification image of the user and the purchased product information of the purchased product of the user in association with each other.
The accounting system according to claim 1. Reception processing for receiving, on the accounting screen, a first operation of selecting a purchased product to be accounted from among the purchased product information, and a second operation of associating the purchased product selected by the first operation with the user identification image. further comprising the
The registration processing unit associates the user identification image with the purchased product information of the purchased product selected by the first operation when the reception processing unit receives the first operation and the second operation. register with
The accounting system according to claim 1 or claim 2. further comprising a question processing unit that asks the user a question about the purchased product;
The registration processing unit associates and registers the user identification image of the user and the purchased product information of the purchased product of the user on the accounting screen based on the user's answer to the question. do,
The accounting system according to claim 1 or claim 2. The registration processing unit associates and registers the user identification image and the purchased product information based on the user's gesture operation in response to the question.
The accounting system according to claim 4. The registration processing unit associates and registers the user identification image and the purchased product information based on the user's uttered voice in response to the question.
The accounting system according to claim 4. The captured image includes user images of a plurality of users,
The extraction processing unit extracts a user image of each of the plurality of users,
The registration processing unit associates and registers the corresponding purchased product information for each of the user identification images on the checkout screen,
The accounting processing unit causes the accounting terminal to execute accounting processing for the purchased product of the user for each user.
The accounting system according to any one of claims 1 to 6. The extraction processing unit extracts a user image of the user from the captured image when a predetermined user operation is received on the accounting screen.
The accounting system according to any one of claims 1 to 7. wherein the user identification image is a facial image of the user;
The accounting system according to any one of claims 1 to 8. one or more processors
a product acquisition step for acquiring purchased product information corresponding to the purchased product information of the user;
an image acquisition step of acquiring a captured image of the user;
an extraction step of extracting a user image of the user from the captured image acquired in the image acquisition step;
a registration step of correlating and registering the user identification image corresponding to the user image extracted in the extracting step and the purchased product information obtained in the product obtaining step on the accounting screen displayed on the accounting terminal; When,
an accounting step of causing the accounting terminal to execute accounting processing of the user corresponding to the user identification image and the purchased product information registered in the registering step;
Accounting method to perform. a product acquisition step for acquiring purchased product information corresponding to the purchased product information of the user;
an image acquisition step of acquiring a captured image of the user;
an extraction step of extracting a user image of the user from the captured image acquired in the image acquisition step;
a registration step of correlating and registering the user identification image corresponding to the user image extracted in the extracting step and the purchased product information obtained in the product obtaining step on the accounting screen displayed on the accounting terminal; When,
an accounting step of causing the accounting terminal to execute accounting processing of the user corresponding to the user identification image and the purchased product information registered in the registering step;
on one or more processors.

Images