CN113435892A - Input system, input program, and input method - Google Patents
Input system, input program, and input method Download PDFInfo
- Publication number
- CN113435892A CN113435892A CN202110559959.9A CN202110559959A CN113435892A CN 113435892 A CN113435892 A CN 113435892A CN 202110559959 A CN202110559959 A CN 202110559959A CN 113435892 A CN113435892 A CN 113435892A
- Authority
- CN
- China
- Prior art keywords
- settlement
- user
- reading
- registered
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q20/00—Payment architectures, schemes or protocols
- G06Q20/38—Payment protocols; Details thereof
- G06Q20/40—Authorisation, e.g. identification of payer or payee, verification of customer or shop credentials; Review and approval of payers, e.g. check credit lines or negative lists
- G06Q20/401—Transaction verification
- G06Q20/4014—Identity check for transactions
- G06Q20/40145—Biometric identity checks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
- G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/44—Program or device authentication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
Abstract
The present invention provides a novel input technique capable of avoiding or reducing contact by a large number of users in various input operations. The present invention provides an input system for identifying a user registered in advance and performing settlement, the input system including: a reading device that reads commodity specifying information specifying a commodity to be purchased; a photographing device for photographing a user who performs settlement; an action recognition device which finishes the reading by the reading device when recognizing the user settlement action from the user image shot by the shooting device; an authentication device that authenticates a registered user based on the user image; and a settlement device that executes a settlement process of the price of the purchased product based on the registered user information when the settlement action is recognized and the authentication is successful.
Description
Technical Field
The present invention relates to an input system, an input program, and an input method for avoiding or reducing user contact.
Background
Due to the prevalence of infectious diseases worldwide, infectious disease countermeasures are receiving attention again. For example, IT is one of effective measures against infectious diseases to reduce the chance of human-to-human contact by IT (Information Technology) by replacing the payment of the settlement in the store from the person operated by the clerk with the self-service payment of the settlement apparatus operated by the customer himself.
Conventionally, from the viewpoint of necessary personnel reduction and efficiency, an automation technology for cash register and window business using IT is known. For example, patent document 1 describes a method of automatic settlement using biometric authentication based on a face image. Patent document 1 describes that a user explicitly instructs execution of face authentication by a predetermined gesture.
Documents of the prior art
Patent document
Patent document 1: japanese laid-open patent publication No. 2019-67075
Patent document 2: japanese patent laid-open No. 2020 and 87093
Disclosure of Invention
Problems to be solved by the invention
On the other hand, contact infection due to the spread of bacteria and viruses through objects with which an unspecified number of people come into contact is also considered to be a problem. In the above-described conventional technique, it is assumed that the user himself performs input, and many people touch the same portion at the time of input, and thus it can be said that it is insufficient from the viewpoint of preventing contact infection.
Accordingly, the problem to be solved by the present invention is to provide a novel input technique capable of avoiding or reducing contact by a large number of users in various input operations.
Means for solving the problems
In order to solve the above problems, the present invention provides an input system for identifying a user registered in advance and settling accounts, the input system including: a reading device that reads commodity specifying information specifying a commodity to be purchased; a photographing device for photographing a user who performs settlement; an operation recognition device that, when a settlement operation by the user is recognized from the user image captured by the imaging device, ends reading by the reading device; an authentication device that authenticates a registered user based on the user image; and a settlement device that executes a settlement process of the price of the purchased product based on the registered user information when the settlement action is recognized and the authentication is successful.
By adopting such a configuration, all the inputs related to settlement can be completed without the user touching the device itself. Therefore, the user can purchase the product by simply touching the product, the risk of infection due to an unspecified large number of contacts can be reduced, and contamination and wear of the device can be reduced. In addition, the settlement can be easily performed even when gloves are worn in a ski resort, a construction site, or the like, and the settlement is performed based on the user image, so that it is not necessary to carry a wallet, a smartphone, or the like, and the convenience of the user is improved.
In a preferred aspect of the present invention, the motion recognition means recognizes a gesture registered in advance from the user image as the settlement motion.
By adopting such a configuration, an effect of reducing erroneous recognition can be expected.
In a preferred aspect of the present invention, the user information includes group specifying information for specifying a group to which the user belongs, and the settlement device executes the settlement processing based on the group specifying information.
By adopting such a configuration, for example, by applying the present invention to settlement of shopping in a shop or the like in a company and specifying a group to which a user belongs to perform settlement, payment by deduction from payroll or payment by welfare will be realized.
In a preferred aspect of the present invention, the settlement apparatus determines group information registered in advance based on the group specifying information, and executes the settlement process in accordance with a settlement method specified in the group information.
With such a configuration, settlement can be performed by a settlement method specified for each group.
In a preferred embodiment of the present invention, the method further comprises: means for performing cancellation processing in a case where the reading is not continuously performed for a predetermined time period after the reading means starts the reading.
With such a configuration, when the user stops inputting halfway and leaves the apparatus directly, the user can perform the cancel processing and receive the next input.
In a preferred embodiment of the present invention, the method further comprises: and a display unit configured to perform a settlement display for ending the reading and requesting an input of the settlement process when the operation recognition unit recognizes that the settlement operation has failed after the reading unit starts the reading.
With this configuration, even when the user cannot recognize the settlement operation from the user image for some reason, the user can request the settlement processing by using another method.
In a preferred aspect of the present invention, the display unit is configured to determine that the recognition of the settlement operation has failed when the operation recognition device does not recognize the settlement operation for a predetermined period of time after the reading device starts the reading.
In a preferred embodiment of the present invention, the settlement display requests the user to touch a different portion from that displayed last time.
With this configuration, even when the recognition of the settlement operation fails, it is possible to avoid a large number of users touching the same portion.
In order to solve the above-described problems, the present invention provides an input program for identifying a user registered in advance and settling accounts, the input program causing a computer to function as: a reading device that reads commodity specifying information specifying a commodity to be purchased; a photographing device for photographing a user who performs settlement; an operation recognition device that, when a settlement operation by the user is recognized from the user image captured by the imaging device, ends reading by the reading device; an authentication device that authenticates a registered user based on the user image; and a settlement device that executes a settlement process of the price of the purchased product based on the registered user information when the settlement action is recognized and the authentication is successful.
In order to solve the above problems, the present invention provides an input method for identifying a user registered in advance and settling accounts, the input method causing a computer to execute the steps of: reading commodity specifying information specifying a commodity to be purchased; shooting a user for settlement; a step of ending the reading when a settlement action by the user is recognized from the photographed user image; a step of authenticating the registered user based on the user image; and a step of executing settlement processing of the price of the purchased product based on the registered user information when the settlement action is recognized and the authentication is successful.
ADVANTAGEOUS EFFECTS OF INVENTION
According to the present invention, a novel input technique capable of avoiding or reducing contact by a large number of users can be provided.
Drawings
FIG. 1 is a functional block diagram of an input system in an embodiment of the present invention.
Fig. 2 is a diagram showing an example of information registered in the database DB in the embodiment of the present invention.
Fig. 3 is a diagram showing an example of information stored as a transaction history table in the embodiment of the present invention.
Fig. 4 is a flow chart of processing associated with settlement for a user of an input system to purchase a product in an embodiment of the invention.
Fig. 5 is a diagram showing an example of a display screen of an input system in the embodiment of the present invention.
Fig. 6 is a diagram showing an example of a display screen of an input system in the embodiment of the present invention.
Fig. 7 is a diagram showing an example of a display screen of an input system in the embodiment of the present invention.
Fig. 8 is a diagram showing an example of a display screen of an input system in the embodiment of the present invention.
Fig. 9 is an explanatory diagram of an example of recording of the contact position of the input system and coordinates used for displaying the position on the display unit in the embodiment of the present invention.
Fig. 10 is a flowchart relating to determination of a display position of a button of an input system in the embodiment of the present invention.
Fig. 11 is a diagram showing an example related to determination of a display position of a button of an input system in the embodiment of the present invention.
Detailed Description
Hereinafter, an input system according to the present invention will be described with reference to the drawings. The embodiments described below are examples of the present invention, and the present invention is not limited to the embodiments described below, and various configurations may be adopted.
For example, although the configuration, operation, and the like of the input system are described in the present embodiment, a method, an apparatus, a computer program, and the like having the same configuration can exhibit the same operational advantages. Further, the program may be stored in a recording medium. If this recording medium is used, a program can be installed in a computer, for example. Here, the recording medium on which the program is stored may be a non-transitory recording medium such as a CD-ROM.
In the present embodiment, a technique for avoiding or reducing user contact in an instruction for settlement processing and other various inputs is exemplified on the premise of settlement by face authentication, but the present invention is not limited to this. For example, the display position determination of a button described later may be performed in any other input to prevent a large number of users from touching the same portion in a system that does not use image authentication or a wide range of systems that are premised on user contact.
In the present embodiment, an embodiment in a store used by a user belonging to a specific group, such as a store in a company, is assumed. Therefore, in the present embodiment, a use group is determined for each store in which an input system is installed, and a person registered as a user belonging to the group uses the input system of the present invention in the store.
(1) Settlement input using user images
First, the overall configuration of an input system for performing settlement by capturing a user image obtained by a user at the time of settlement will be described. Fig. 1 is a diagram showing a functional configuration of an input system according to the present embodiment.
The input system of the present embodiment includes: the input system includes a reader 11, an imaging device 12, a motion recognition device 13, an authentication device 14, a settlement device 15, a display unit 16, a contact position recording device 17, a button display device 18, and a contact position resetting device 19, and is configured to be capable of communicating with the database DB.
The input system is implemented by a general computer apparatus, which includes: an arithmetic device such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a main storage device such as a RAM (random access Memory), a secondary storage device such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), and a flash Memory, and various input/output devices such as a connection device for connecting to a network, a reading device for reading commodity information, an imaging device, a display, and the like. In the present embodiment, the input system is configured by connecting a reading device such as a barcode reader, a camera, a display (preferably, a touch panel display), and the like to an arbitrary computer device connected to a network for accessing the database DB and the like.
The reading device 11 reads commodity specifying information for specifying a purchased commodity. Specifically, for example, a barcode reader for reading a barcode, a camera for identifying an image or characters of a commodity by image recognition to specify the commodity, a tag reader for receiving commodity specifying information by short-range wireless communication, or the like is used as the reading device 11 of the present invention.
The photographing device 12 photographs the user who performs settlement and generates a user image. In the present embodiment, an imaging device provided around the display portion 16 is used as the imaging device 12. The imaging device 12 images the face of the user who has performed the settlement, and transmits the user image to the motion recognition device 13, the authentication device 14, and the display unit 16.
When recognizing a gesture registered in advance as a settlement operation from the user image, the operation recognition device 13 ends the reading by the reading device 11. The operation recognition device 13 of the present embodiment recognizes the termination operation in addition to the settlement operation. When the termination operation is recognized, the commodity specifying information read up to the present time is reset, and the settlement is terminated. In the present embodiment, a gesture of a V-shaped and a flat gesture is registered as a settlement operation and a gesture of a palm facing the front is registered as a suspension operation. The settlement operation and the pause operation are not limited to this, and a specific expression such as a smiling face may be used as the settlement operation or the pause operation. Further, the settlement motion or the suspension motion may be recognized on the condition that both the gesture and the expression are used.
The recognition of the settlement action and the suspension action can be performed by using any image recognition method. For example, the following methods can be cited: a plurality of sets of images obtained by imaging each operation and information indicating the type of the operation are input as teacher data, a classification model of each operation is created by using a known machine learning method, and a user image is input to the classification model to detect a settlement operation or a suspension operation. The classified operation may be a settlement operation, a suspension operation, or other operations. Here, in the present embodiment, when the settlement operation or the suspension operation is continuously detected for a certain period of time, the operation recognition device 13 recognizes the operation and performs the processing.
The authentication device 14 authenticates the registered user based on the user image. In the present embodiment, an image obtained by imaging the face of a user is registered in advance in association with the user, and the registered user is authenticated by comparing the image registered in advance with the user image. In addition, although the registered user may be authenticated by any method such as iris authentication or vein authentication, it is preferable that the authentication is performed by a method in which the user does not touch the device.
The analysis processing of the user image may be performed in the input system, but in the present embodiment, the authentication device 14 requests an external server that manages a registered image for authentication of the user to analyze the user image, receives the analysis result, and verifies the analysis result with the user registered in the database DB to authenticate the registered user. In the user authentication, a conventional face authentication technique that determines coincidence by feature amount extraction of an image may be used.
When the operation recognition device 13 recognizes the settlement operation and the authentication device 14 successfully authenticates the registered user, the settlement device 15 performs a settlement process of the price of the product based on the registered user information. Further, in the present embodiment, the user information is associated with group specifying information that specifies a group to which the user belongs, and the settlement device 15 executes settlement processing based on the group specifying information. Further, the settlement processing may be executed in accordance with a settlement method designated in group information registered in advance based on the group specifying information.
In the present embodiment, as the settlement processing, the settlement device 15 records the transaction history including the information for specifying the user and the settlement amount in the database DB. This makes it possible to request a fee to the user or a group to which the user belongs. In addition to this, for example, the following processing is also possible: payment information of a credit card or a bank account is registered as user information, and information for receiving payment of a price of a commodity from a card company or a bank is transmitted to an external server or the like. Further, a payment method may be defined for each group in the group information, and a request for payment may be transmitted to an external server or the like according to the payment method.
The display unit 16 displays various kinds of information for input. When the operation recognition device 13 does not recognize the settlement operation within a predetermined time after the reading device 11 starts reading, the display unit 16 performs settlement display to end the reading and requests the input of the settlement processing. In the present embodiment, a touch panel display is used as the display unit 16, and a settlement button is displayed on the display unit 16 as a settlement display. Then, in place of the recognition of the settlement gesture by the motion recognition device 13, the instruction of the user is received by the input through the settlement button, and the reading by the reading device 11 is ended.
On the other hand, when reading is not performed for a predetermined time after the reading device 11 starts reading, the input system resets the commodity specifying information read until the present time, and executes the cancellation process of stopping the settlement. The cancel processing at this time is the same as the processing in the case where the operation recognition device 13 recognizes that the operation is suspended.
The contact position recording device 17 specifies the contact position touched by the user on the display unit 16, and records the contact position in the database DB together with the time when the contact occurred. The time when the contact occurs may not be exactly the same as the time when the contact is strictly made, but may include, for example, a time before and after the time when the input is made by the contact, the time when the settlement input is made by the contact, and the like.
The contact position recording device 17 may determine the contact position by a contact sensor provided in the display unit 16, for example. The contact position may be determined by analyzing an image or the like obtained by imaging the display unit 16. In the present embodiment, a button for receiving an input from a user is displayed on the display unit 16, and when an input via the button is made, the contact position recording device 17 records the center of the display position of the button as a contact position. The contact position recording device 17 may determine and record all the contact positions by a sensor or the like, regardless of the display content on the display unit 16.
The button display device 18 determines the position where the contact is the minimum after the reference time as the display position of the button based on the contact position. Here, in the present embodiment, the button display device 18 determines the display position of the settlement button displayed as the settlement display.
The contact position resetting means 19 receives the reset indication and resets the recording of the contact position. When the reset is performed, the button display device 18 determines a position with less contact as the display position based on the contact position after the reset. In the present embodiment, the display position is determined based on the contact position after the reset by setting the time at which the reset is performed as the reference time, but for example, a reset flag may be set in a record of the previous contact position or the record itself of the contact position may be deleted in response to a reset instruction.
Fig. 2 and 3 are diagrams showing an example of information registered in the database DB. As shown in fig. 2, the database DB has a user master table (usermaster), a group master table (group master), and a group user master table (group usermaster), and manages information of each user, information of a group, and information of users belonging to each group based on these information.
The user master table is user information related to a user, and includes: a user ID that is an identifier of the user and is used for login and the like; uniquely determining a system UID of a user on a system side; the name of the user; and a face authentication ID for uniquely determining face authentication information in an external server that performs image analysis for face authentication. In this way, by using the system UID different from the user ID that is capable of uniquely identifying the user, for example, in the case of using a mail address as the user ID, the user can be easily identified even if the user freely changes the user ID. Further, various administrative authorities and the like may also be stored as the user master table.
Further, in the case of settlement by credit card or deduction from a bank account, information of the credit card or the bank account is also stored as payment information. Here, information specifying the settlement method may be stored in advance in the user information, and the settlement device 15 may execute the settlement processing based on the specification in the user information. Further, information such as a login password is managed as necessary, and a user login to the system is performed based on the user information.
The group master table is group information related to a group, and includes: uniquely determining a group ID of the group; a group code as a group identifier; a group name indicating a name of the group; the group started the utilization start date of the system utilization; and a utilization end date on which the group has ended system utilization or is scheduled to end. The group code can be arbitrarily changed by a manager of the group or the like, but the group ID is an invariant character string uniquely determining the group.
The group user master table maintains associations between groups and users. The group ID, the system UID of the user belonging to the group, the number of times the user purchased within the group, and the total amount the user purchased within the group are stored as a group user master table. Here, in the present embodiment, "purchase within a group" means that the purchase occurs in a store whose group is designated as a utilization group.
In addition, information for specifying a settlement method in the group may be stored in the group master table or the group user master table. In each record of the information, the creator, the creation date and time, the updater, and the update date and time of the record may be further recorded.
Fig. 3 is a diagram showing an example of information stored as a transaction history table. A record of the transaction history table is created for each purchased item. For example, in the case where two types of commodities are purchased in one transaction, two records are created. The records of the transaction history table include: group ID, date and time of transaction, detailed number, device ID of the device that made the transaction, floor ID of the store where the device is installed, system UID that identifies the user who made the transaction, JAN code of the commodity, trade name pseudonym, number, unit price, purchase amount, tax rate, and tax amount. By thus recording the transaction history, the purchase amount of each user can be determined for each group, and for example, the user can pay for a price by a method suitable for each group, for example, a deduction from a wage or a benefit thick fee, or the like. The amount of the product purchased by the subject is recorded as the purchase amount and the tax amount. That is, even when another product is purchased simultaneously with the product illustrated in fig. 3, the record of the transaction history is the content of fig. 3.
Here, in the present embodiment, a time stamp in milliseconds is recorded on the server side as the transaction date and time, and the record of the transaction history at the same time is treated as one transaction. Thus, the input system of the present embodiment can collectively output information of one transaction in a form like a receipt. In addition to this, the user can confirm the transaction history recorded with his own system UID by logging in to the system, and can use the transaction history as a receipt.
Fig. 4 is a flowchart of processing related to settlement for a user to purchase an article. When an input for purchasing a product is started, first, in step S101, the reading device 11 starts reading, the imaging device 12 starts imaging of the user, and the reading of the product specifying information or the detection of the settlement operation or the suspension operation is waited for.
Fig. 5 is a diagram showing an example of an input screen displayed on the display unit 16 at this time. The input screen W1 includes: a purchase waiting article display unit W11 for displaying the purchase waiting article specified by the reader 11; and a user image display section W12 that displays the user image captured by the imaging device 12.
The user image display portion W12 displays the captured image in real time, and the user can confirm the image of himself. Here, the user image display unit W12 may display a user image whose left and right are reversed so that the user can easily understand it. Before reading the product specifying information, as shown in fig. 5 (a), a display requesting reading of the product specifying information is performed in the purchase waiting product display unit W11, and when there is one or more purchase waiting products, as shown in fig. 5 (b), a display requesting a settlement operation or a suspension operation is performed on the upper and lower sides of the screen.
In step S101, the reading device 11 reads the article identification information. In the present embodiment, the code of the commodity is read by the barcode reader to specify the commodity. When the product specifying information is read, the process proceeds to step S102, in which the product specified by the reading device 11 is added as a purchase waiting product, and the amount of the product is added to the total amount. The information added is reflected on the purchase waiting product display portion W11. Then, the process returns to step S101 again, and the input system waits for the next product reading or operation detection.
When the motion recognition device 13 detects a motion in step S101, the process proceeds to step S104 when a settlement motion is performed (settlement motion in step S103). In step S104, it is determined whether or not the settlement operation is continuously detected for a fixed time.
If the settlement operation does not continue for a predetermined time in step S104, it is further determined whether or not a predetermined time has elapsed without recognizing the settlement operation in step S105. If the settlement operation has not been recognized for a certain period of time, the process returns to step S101, and if the settlement operation has not been recognized for a certain period of time, the process proceeds to step S106, and a settlement button is displayed.
As described above, in the present embodiment, when the recognition of the settlement operation is not performed for a predetermined period of time, there is a possibility that the recognition of the settlement operation fails for some reason, and therefore, the settlement button is displayed so that the input can be performed using the button. In particular, in the present embodiment, when a settlement operation in a short time not reaching a preset duration is detected, if the settlement operation is not recognized for a predetermined time or more, a settlement button is displayed. At this time, it is also possible to take the detection of a human face in the image captured by the imaging device 12 as another condition. Similarly, the display unit 16 may display a stop button when the reading of the product specifying information is not performed for a predetermined period of time, or when a short-time stop operation that does not reach a preset duration is detected.
Fig. 6 is a diagram showing an example of the input screen W1 at the time of settlement operation recognition. When the settlement action is detected, as shown in (a) in fig. 6, an identification display W13 is further displayed on the input screen W1. Thus, the user can grasp the detected motion. Here, the recognition display W13 preferably indicates the time until the settlement operation is recognized. In the present embodiment, the identification display W13 changes according to the time when the settlement operation is continuously detected, and when the settlement operation is detected for a certain time, the identification display W13 extends to the upper part of the screen as shown in fig. 6 (a). Thereby, the user can be requested to continue the settlement action. In addition, in the detection of the settlement operation, the countdown until the settlement operation is recognized may be displayed as the recognition display W13 together with a message such as "please do not move" to request the user to continue the settlement operation.
In the case where the settlement button or the abort button is displayed, since the user touches the button, it is preferable to request a different portion from that displayed last time to avoid a large number of users from contacting the same portion. It is conceivable, for example, to prepare a plurality of candidates of button display positions in advance, change the positions each time they are displayed, or to record the contact positions of the user, and display the buttons at the positions not contacted. Details of the latter method are described later.
When the operation recognition means 13 detects the settlement operation for a predetermined time in step S104, it recognizes the settlement operation and proceeds to step S107. In this way, by requesting the user to continue the settlement action for a certain time, for example, 3 seconds, it is possible to prevent erroneous settlement from being performed in the case where the user accidentally temporarily takes a posture similar to the settlement action. In step S107, the authentication device 14 performs authentication processing based on the user image. In the present embodiment, the authentication device 14 requests an external server that manages face authentication information of a user registered in advance to analyze a user image.
In the external server, analysis of a user image is performed based on face authentication information of a user registered in advance, and a registered user corresponding to the user image is determined. Then, a face authentication ID for uniquely determining face authentication information of the user is returned to the input system, and the authentication device 14 receives the face authentication ID. Then, by determining the user information having the face authentication ID from the database DB, the user to be settled is determined.
Then, in step S108, the settlement apparatus 15 performs settlement processing by recording the transaction history including the system UID of the specified user and information of the purchased product and the like in the database DB, and ends the processing.
Here, in the present embodiment, the user information is associated with the group specifying information to specify the group to which the user belongs. This enables settlement based on the group information. For example, for a user of a group designated to pay by withholding from wages in the group information, the payment processing at the time of purchase is not performed, but request information for the group is separately generated based on the transaction history. On the other hand, for a user of a group for which payment is designated by a credit card of the user person in the group information, a settlement processing request is made to a server of a card company or the like so as to make payment by a credit card at the time of purchase.
In addition, in the external server that analyzes the user image, in the case where the registered user corresponding to the user image is not specified or in the case where the user information having the face authentication ID received by the authentication apparatus 14 is not specified, it is preferable that the case be displayed on the display section 16 without performing the settlement processing.
Fig. 7 is a diagram showing an example of the screen displayed on the display unit 16 in steps S107 and S108. Preferably, while the face authentication is performed in step S107 or while the settlement processing in step S108 is performed after the authentication, it is displayed that the processing for settlement is being performed, as shown in (a) of fig. 7. After the settlement is completed, it is preferable to display the end of the processing as shown in fig. 7 (b).
When the operation recognition device 13 detects the suspension operation in step S101 ("suspension operation" in step S103), the process proceeds to step S109, and determines whether or not the suspension operation is continuously detected for a fixed time period, in the same manner as in step S104. If the operation is not continued for a certain period of time in step S109, the process returns to step S101, and the input system waits again for the next product reading or operation detection. At this time, when the operation is not recognized to be stopped for a certain period of time, the display unit 16 may display a stop button.
Fig. 8 is a diagram showing an example of display of a screen in a case where the suspension operation is detected. When the suspension operation is detected, as shown in (a) of fig. 8, the recognition display W13 is displayed on the input screen W1 in the same manner as in the settlement operation recognition. Note that the display form of the recognition display W13 may be different between the case of the settlement operation and the case of the suspension operation.
When the motion recognition means 13 detects the suspension motion for a predetermined time in step S109, it recognizes the suspension motion and proceeds to step S110. In step S110, the input system performs a cancellation process of resetting the purchase waiting article and the amount of money, and ends the process. In this case, as shown in fig. 8 (b), it is preferable to display the fact that the transaction is terminated.
Here, the problem is assumed that: after the user reads the commodity specifying information, the commodity specifying information is placed on site without performing a suspension operation. In order to regard this as a cancel process, the input system of the present embodiment proceeds to step S111 after the reading device 11 starts reading and when there is no reading of the product in step S101, and determines whether or not a certain time (for example, 30 seconds) has elapsed without reading.
Here, if the reading is not continuously performed for a certain period of time, the process proceeds to step S112, and the cancellation process is performed. At this time, it is also possible to take as another condition that a human face is not detected in the image captured by the imaging device 12. On the other hand, if the predetermined time has not elapsed, the process returns to step S101 to continue waiting for reading of the product and detection of the operation.
The processing procedure in fig. 4 is an example, and the processing procedure may be changed arbitrarily. For example, the operation recognition device 13 may recognize the settlement operation or the suspension operation, and the authentication device 14 may authenticate the registered user in parallel, or the order may be switched.
As described above, according to the input system of the present embodiment, since the face authentication and the recognition of the settlement operation are performed using the user image, the user can complete the settlement without touching the device. This can be expected to prevent contact infection, prevent abrasion of the device due to contact, and the like. Further, as in the present embodiment, by associating the user information with the group information and performing the settlement processing based on the group information, it is possible to perform the settlement corresponding to the request for each group.
(2) Button display position determination
Next, referring to fig. 9 to 11, a detailed description will be given of a process related to button display position determination for reducing contact of a large number of users with the same portion, for example, when the above-described settlement button or stop button is displayed. As described above, the present embodiment is merely an example, and the present invention can be applied to any input device using a touch panel display.
In the present embodiment, the input system includes a contact position recording device 17 and a button display device 18. The button display device 18 determines a position where the display unit has been in contact with a small amount in the past as a display position of the button, and displays the button at the display position on the display unit 16. This can prevent or reduce a large number of users from touching the same portion of the display unit 16 when the user operates the button, and can also expect effects such as reduction in the risk of contact infection.
In the present embodiment, the database DB also stores contact positions. Fig. 9 is an explanatory diagram showing an example of recording of the contact position and coordinates used for indicating the position on the display unit 16. The contact position recording device 17 of the present embodiment specifies the position touched by the user by a sensor provided in the display unit 16, and records the position in the database DB. In the present embodiment, as the contact position, for example, an area specified by coordinates shown in the lower part of fig. 9 and a contact time are recorded.
Here, in order to reduce the processing load, the contact position recording device 17 according to the present embodiment does not always determine contact with the entire display unit 16, and when an input is made by a button, it is considered that contact is made with the display position of the button. The contact position with respect to the entire display unit 16 may be constantly monitored, and the contact position recording device 17 may specify and record all the contact positions regardless of the display of the button or the like.
The position indication on the display unit 16 may be set arbitrarily. Further, in the case where there is contact at a position spanning a plurality of areas, or the like, the plurality of areas may also be recorded as the contact position.
Fig. 10 is a flowchart related to the determination of the display position of the button by the button display device 18. First, in step S21, a record of the contact position after the reference time is extracted from the records of the contact position recorded in the database DB. In this way, by determining the display position while avoiding the contact position with respect to the contact after the reference time, for example, by sterilizing the device at the determined time, it is possible to prevent the contact infection or the like caused by the contact with the device.
Subsequently, in the processing of steps S22 to S25, a position with less contact is determined as the display position of the button based on the past contact position to be the object. This processing is an example, and may be arbitrarily changed as long as it is within a range in which a position with less contact can be determined.
First, in step S22, it is confirmed whether or not there is a record of the contact position with respect to one of the candidates of the display position of the button. If the record is not touched, the process proceeds to step S23, where the position is determined as the display position of the button, and the process ends. On the other hand, if there is a record of the contact position for that position, the process proceeds to step S24, and it is further checked whether or not there is another display position candidate.
If there are no other display position candidates, in step S25, the position with the earliest time of the last contact is determined as the display position in the record of the contact position after the reference time, and the process ends. In this way, when there is no position where no contact is recorded after the reference time, since the plurality of users touch the same portion, the terminal device or the like of the manager of the store using the input system can be notified of the fact, and the sterilization of the device can be promoted.
On the other hand, in the case where there are other display position candidates, it is changed to the next candidate in step S26 and returns to step S22, and the process is repeated until the display position is decided. Thus, when there is a position that is not touched after the reference time, the position is determined as the display position, and when there is no position that is not touched after the reference time, the position that is the earliest in time of the last touch after the reference time is determined as the display position.
In addition, in the present embodiment, the input system includes a contact position resetting device 19, and the record of the contact position can be reset. Thus, for example, when the apparatus is sterilized, the staff in the shop inputs the reset instruction, and the button can be displayed at a position where the contact is not made for the contact after sterilization.
Fig. 11 shows a specific example for explanation. For example, as shown in (a) in fig. 11, a button is displayed, and it is assumed that the user touches a position displayed as a contact position. Then, as the contact position, coordinates of W, 12, which is the center of the display position of the button, and the contact time are recorded.
Then, when the button is subsequently displayed, the display position is determined while avoiding the contact position, as shown in fig. 11 (b). Here, the center coordinates of the buttons may be shown as the display positions, and in the example of (b) in fig. 11, V and 12 are the display positions.
Here, it is known that many users touch the vicinity of the center thereof in the case of operating a circular button displayed on a screen. Therefore, although the end of the display area overlaps the contact position in fig. 11 (b), the center is V or 12, and the coordinate is not recorded as the contact position, which is also allowed in the present embodiment. Alternatively, the display position may be determined on the condition that the display area of the button does not include the past contact position.
The above-described processing procedure is an example, and design changes such as processing procedures can be arbitrarily performed. For example, in the present embodiment, the position where the contact is the minimum after the preset reference time is determined as the display position with respect to the contact position after the reference time, but a time traced back by a predetermined time from the current time can be changed as the reference time. Further, the following configuration may be adopted: the determined reference time is not set, and the position with the least contact is determined as the display position of the button based on all the contact positions in the past.
Description of the reference numerals
11: reading device
12: image capturing apparatus
13: motion recognition device
14: authentication device
15: settlement device
16: display unit
17: contact position recording device
18: push button display device
19: contact position resetting device
W1: input picture
W11: purchase waiting merchandise display unit
W12: user image display unit
W13: identification display
W14: and a settlement button.
Claims (10)
1. An input system for identifying a user registered in advance and performing settlement, the input system comprising:
a reading device that reads commodity specifying information specifying a commodity to be purchased;
a photographing device for photographing a user who performs settlement;
an action recognition device which finishes the reading of the reading device when the settlement action of the user is recognized from the user image shot by the shooting device;
an authentication device that authenticates a registered user based on the user image; and
and a settlement unit configured to execute a settlement process of the purchase price based on the registered user information when the settlement operation is recognized and the authentication is successful.
2. The input system according to claim 1, wherein the action recognition means recognizes a gesture registered in advance from the user image as the settlement action.
3. The input system of claim 1,
the user information is associated with group determination information that determines a group to which the user belongs,
the settlement means performs the settlement processing based on the group determination information.
4. The input system according to claim 3, wherein the settlement means determines group information registered in advance based on the group determination information, and performs the settlement processing in accordance with a settlement method specified in the group information.
5. The input system of claim 1, further comprising: means for performing cancellation processing in a case where the reading is not continuously performed for a predetermined time period after the reading means starts the reading.
6. The input system of claim 1, further comprising: and a display unit configured to perform a settlement display for ending the reading and requesting an input of the settlement process when the operation recognition unit recognizes that the settlement operation has failed after the reading unit starts the reading.
7. The input system according to claim 6, wherein the display section regards the recognition of the settlement action as failed in a case where the action recognition means does not recognize the settlement action for a predetermined time after the reading means starts the reading.
8. The input system according to claim 6 or 7, wherein the settlement display is a portion requesting the user to touch different from that at the last display.
9. A storage medium having an input program recorded thereon for identifying a user registered in advance and settling a settlement, the input program causing a computer to function as:
a reading device that reads commodity specifying information specifying a commodity to be purchased;
a photographing device for photographing a user who performs settlement;
a motion recognition device that ends reading by the reading device when a settlement motion of the user is recognized from the user image captured by the capturing device;
an authentication device that authenticates a registered user based on the user image; and
and a settlement unit configured to execute a settlement process of the price of the purchased product based on the registered user information when the settlement operation is recognized and the authentication is successful.
10. An input method for identifying a user registered in advance and settling accounts, which causes a computer to execute the steps of:
reading commodity specifying information specifying a commodity to be purchased;
shooting a user for settlement;
a step of ending the reading when a settlement action of the user is recognized from the photographed user image;
a step of authenticating the registered user based on the user image; and
and a step of executing settlement processing of the price of the purchased product based on the registered user information when the settlement action is recognized and the authentication is successful.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2020-134101 | 2020-08-06 | ||
| JP2020134101A JP6839324B1 (en) | 2020-08-06 | 2020-08-06 | Input system, input program and input method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113435892A true CN113435892A (en) | 2021-09-24 |
Family
ID=74673669
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110559959.9A Pending CN113435892A (en) | 2020-08-06 | 2021-05-21 | Input system, input program, and input method |
Country Status (2)
| Country | Link |
|---|---|
| JP (2) | JP6839324B1 (en) |
| CN (1) | CN113435892A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103049848A (en) * | 2012-12-07 | 2013-04-17 | 康佳集团股份有限公司 | Intelligent television set-based gesture payment method and intelligent television set |
| CN106600855A (en) * | 2016-12-22 | 2017-04-26 | 河南牧业经济学院 | Payment device and method based on face recognition |
| CN106682909A (en) * | 2017-01-04 | 2017-05-17 | 上海量明科技发展有限公司 | Gesture payment method and system |
| CN108171516A (en) * | 2018-01-04 | 2018-06-15 | 广州逗号智能零售有限公司 | The electric paying method and method that a kind of gesture confirms |
| CN108280649A (en) * | 2018-02-24 | 2018-07-13 | 广州逗号智能零售有限公司 | Method of payment and device |
| CN109872160A (en) * | 2019-02-01 | 2019-06-11 | 广州逗号智能科技有限公司 | Electric paying method and device |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10328644A (en) * | 1997-06-02 | 1998-12-15 | Mitsubishi Electric Corp | Apparatus and method for providing demolition data on product waste in waste recycling treatment plant |
| JPH1139540A (en) * | 1998-05-18 | 1999-02-12 | Toshiba Corp | Automatic transaction device |
| JP2001306175A (en) * | 2000-04-27 | 2001-11-02 | Nippon Syst Wear Kk | Password input device |
| JP2003141374A (en) * | 2001-08-24 | 2003-05-16 | Maeda Corp | Compound electronic selling system, subsystem, selling method and program |
| JP2006209335A (en) * | 2005-01-26 | 2006-08-10 | Toshiba Corp | Charge settlement system and method |
| JP2013196047A (en) * | 2012-03-15 | 2013-09-30 | Omron Corp | Gesture input apparatus, control program, computer-readable recording medium, electronic device, gesture input system, and control method of gesture input apparatus |
| JP2015148872A (en) * | 2014-02-05 | 2015-08-20 | 株式会社リコー | Control device of touch panel display, and image forming apparatus |
| JP6293076B2 (en) * | 2015-03-02 | 2018-03-14 | 東芝テック株式会社 | Touch panel device, panel management program |
| JP6863902B2 (en) * | 2015-05-14 | 2021-04-21 | マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． | Augmented reality systems and methods for tracking biometric data |
| CN107103503B (en) * | 2017-03-07 | 2020-05-12 | 阿里巴巴集团控股有限公司 | Order information determining method and device |
| JP2019079174A (en) * | 2017-10-23 | 2019-05-23 | 富士通フロンテック株式会社 | Automatic transaction device and screen display method and information processing system of automatic transaction device |
| JP7246060B2 (en) * | 2018-09-07 | 2023-03-27 | 株式会社寺岡精工 | System, program, sales data processing method |
| JP6589038B1 (en) * | 2018-12-19 | 2019-10-09 | 株式会社メルカリ | Wearable terminal, information processing terminal, program, and product information display method |
| JP6725951B1 (en) * | 2019-10-09 | 2020-07-22 | 株式会社シンシアージュ | Payment support system |
-
2020
- 2020-08-06 JP JP2020134101A patent/JP6839324B1/en active Active
-
2021
- 2021-02-12 JP JP2021020567A patent/JP6934121B1/en active Active
- 2021-05-21 CN CN202110559959.9A patent/CN113435892A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103049848A (en) * | 2012-12-07 | 2013-04-17 | 康佳集团股份有限公司 | Intelligent television set-based gesture payment method and intelligent television set |
| CN106600855A (en) * | 2016-12-22 | 2017-04-26 | 河南牧业经济学院 | Payment device and method based on face recognition |
| CN106682909A (en) * | 2017-01-04 | 2017-05-17 | 上海量明科技发展有限公司 | Gesture payment method and system |
| CN108171516A (en) * | 2018-01-04 | 2018-06-15 | 广州逗号智能零售有限公司 | The electric paying method and method that a kind of gesture confirms |
| CN108280649A (en) * | 2018-02-24 | 2018-07-13 | 广州逗号智能零售有限公司 | Method of payment and device |
| CN109872160A (en) * | 2019-02-01 | 2019-06-11 | 广州逗号智能科技有限公司 | Electric paying method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2022030249A (en) | 2022-02-18 |
| JP6934121B1 (en) | 2021-09-08 |
| JP6839324B1 (en) | 2021-03-03 |
| JP2022031099A (en) | 2022-02-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11948364B2 (en) | Portable computing device installed in or mountable to a shopping cart | |
| EP3696779A1 (en) | Information processing apparatus | |
| US11068894B2 (en) | Systems and methods for tokenless authentication of consumers during payment transactions | |
| EP3477566A1 (en) | Camera activation and image processing for transaction verification | |
| CN109215254B (en) | Physical store application system, user information processing method and device | |
| WO2018233720A1 (en) | Implementation method and device for off-line transactions | |
| US20150363785A1 (en) | Systems and methods for consumer authentication using behavioral biometrics | |
| CN109643419A (en) | The system and method that user is authenticated using biological attribute data | |
| US20170255940A1 (en) | Systems, methods, apparatus, and computer-readable media for age verification | |
| US11625703B2 (en) | Commodity registration apparatus and method | |
| US20140172610A1 (en) | Account-based checkout | |
| CN107862226A (en) | The system and method that identification is realized based on REID | |
| JP2023088960A (en) | Information processor and store system | |
| CN113435892A (en) | Input system, input program, and input method | |
| US11836735B2 (en) | Self-service terminal (SST) item return anti-fraud processing | |
| US11562365B2 (en) | User authentication based on a user interaction associated with a transaction | |
| CN114981805A (en) | Illegal detection system, illegal detection device, illegal detection method, and program | |
| JP2008010017A (en) | Automatic transaction system | |
| TWI837556B (en) | Imperfection detection systems, irregularity detection devices, irregularity detection methods and program products | |
| WO2022162834A1 (en) | System and product delivery method | |
| US11138609B2 (en) | Methods, systems and computer program products for identity authentication for payment card based payment transactions | |
| US11893562B2 (en) | Offloading a signing operation on a user device | |
| CA3169463A1 (en) | Method and system for customer responsive point of sale device | |
| WO2019209435A1 (en) | Wearable device for authenticating payment transactions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210924 |