CN110036417B

CN110036417B - Hand-free and ticket-free charging system

Info

Publication number: CN110036417B
Application number: CN201780064015.6A
Authority: CN
Inventors: 刘敬忠; 麦永康; 陈添祥
Original assignee: Singapore Technologies Electronics Ltd
Current assignee: Xinke Engineering Smart City Co.,Ltd.
Priority date: 2016-10-18
Filing date: 2017-09-27
Publication date: 2022-07-01
Anticipated expiration: 2037-09-27
Also published as: SG10201702326TA; CN110036417A; TWI782926B; TW201816677A; WO2018074975A1

Abstract

The present invention describes a hands-free or ticketless charging system (100, 200) for accessing a designated payment area, such as a mass transit system. The commuter is identified by a face recognition subroutine (144), with the commuter identity being additionally split by name, gender, date of birth, race, height, authorized group, etc. and stored in an ID category. Splitting the commuter identity according to different ID categories allows for quick, accurate identification of the commuter in the commuter ID database (160) and allows for deductions from the commuter account maintained in the respective commuter account database (170). The commuter may carry a long range RFID card (119b) to allow for faster commuter identification.

Description

Hand-free and ticket-free charging system

Technical Field

The present invention relates to hands-free and ticketless charging systems, for example for accessing a payment area or a restricted area. In particular, the present invention eliminates physical tickets and relies on face recognition by commuters or customers for identification.

Background

Traditional mass transit toll gates rely on commuters carrying stored value tickets. Each ticket card includes a near field electronic chip for short range contactless sensing at the reader/writer upon entering or exiting the transfer station. The disadvantage of such charging is that the customer must carry a ticket card with sufficient stored value to be able to pass.

Thus, it can be seen that there is a need for another method of charging a commuter or customer in a fast and reliable manner using facial recognition without relying on a ticket card.

Disclosure of Invention

The simplified summary presented below provides a basic understanding of the invention. This summary is not an extensive overview of the invention, and is not intended to identify key features of the invention. Rather, some of the inventive concepts of the present invention are presented in a general form as a prelude to the more detailed description that is presented later.

The present invention seeks to provide another charging system, for example for mass transit of commuters or customers into and out of a payment area. The toll system is ticketless, and the commuter/customer does not need to carry a stored-value ticket card; this would allow hands-free entry or exit to mass transit systems, buses, sports stadiums, concert halls, passenger rest rooms in departure or entry lobbies, and the like.

In one embodiment, the present invention provides a new hands-free, ticketless billing system. The hands-free and ticketless charging system comprises: a server executing software, including a facial recognition subroutine; a commuter Identity (ID) database and a commuter account database linked to the server; a gate for entering or leaving a designated payment area defined by the charging system; and a multi-function kiosk (kiosk) for registering a commuter, wherein a camera captures a portrait of the commuter, and the facial recognition subroutine associates a unique facial feature of the commuter with a customer identity, and the charging system stores the commuter identity in the commuter ID database and the commuter account database, such that upon registration the commuter may enter or leave a designated payment area without having to carry a stored value ticket.

In another embodiment, the present invention provides a system or method for charging a commuter/customer by: registering the commuter, storing the commuter's likeness in an ID database, storing the commuter's payment and bank records in a commuter account database, and identifying the commuter/customer by facial recognition and/or long range RFID. Preferably, the commuter ID is stored in an individual category such as name, gender, ethnic group race, date of birth, height, authorized group, etc. to allow for quick and accurate identification.

Preferably, when the commuter/customer identity is found in the ID database, the server executes a subroutine to insert a greeting into the commuter/customer. When a system fault is triggered, the commuter/customer that triggered the system fault is identified by the beam tracker, highlighted on the monitor screen, or highlighted on Augmented Reality (AR) glasses worn by the controller. Preferably, the display screen on the kiosk is operable to display an advertising/informational screen and an operational menu screen upon detecting the presence of a commuter/customer; wherein the advertisement/information screen is dynamically composed of two or more display screens when the presence of a commuter/customer is not detected within a predetermined range in a state where the multi-function kiosks are placed side by side with each other.

Drawings

The invention is illustrated by way of non-limiting examples thereof with reference to the accompanying drawings, in which:

fig. 1 shows a charging system for mass transit according to an embodiment of the invention;

fig. 2A shows an arched gate for use with the charging system shown in fig. 1, while fig. 2B shows an obstacle gate, and fig. 2C, 2D show two obstacle-free gates;

FIGS. 3A and 3B illustrate a multi-function group kiosk for use with the system shown in FIG. 1;

fig. 4A shows a portrait of a commuter viewed through a camera of the above-described toll collection system, and fig. 4B shows a face used for face recognition; and

fig. 5A shows face recognition among a group of commuters, fig. 5B shows use when a passenger is about to board a bus, and fig. 5C shows use when a passenger is about to disembark from a bus.

Detailed Description

One or more specific and alternative embodiments of the present invention will now be described with reference to the accompanying drawings. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these specific details. Some details may not be described in detail in order not to obscure the invention. For ease of reference, a common reference number or series of numbers will be used throughout the drawings when referring to the same or similar features in the drawings.

Fig. 1 shows a hands-free and ticketless billing system 100 in accordance with an embodiment of the present invention. Broadly, the charging system 100 includes

gates

110, 110a, 110b that separate the payment entry area from the common area (e.g., at a terminal); further, a plurality of multi-function kiosks 120 for registering commuters/customers are located in a common area. Gates 110, 110a, 110b and kiosk 120 are linked to server 140 via gateway 10. Server 140 executes software 142 and connects to workstations 150 located at all of the transfer stations; further, some workstations 150 may be located in some backend control rooms. After registration, the identity of the commuter is stored in the ID database 160 and the commuter account is stored in the account database 170. The commuter account database 170 is also linked through software 142 to a bank 180 associated with the commuter account. The operation of the charging system 100 will become clearer, with the components of the system as set forth below; in this illustration, commuters, passengers, customers, or clients are used synonymously.

FIG. 2A shows a gate 110 according to one embodiment. The gate 110 includes an upstanding arch 112 sized for the height and width of commuters traffic, preferably one at a time. A camera 114 is mounted on the arched top member and is directed to photograph the face of the commuter, such as the portrait shown in fig. 4A and the face shown in fig. 4B, before the commuter enters through the gate 110. When the gate 110 is configured for bidirectional access, another camera 114 is mounted on the opposite side of the top member to photograph the face of the commuter to exit through the gate. The gate 110 also includes a pair of green and

red light indicators

116, 117. The green light indicator 116 is illuminated when a commuter entering or leaving the gate 110 is actively identified and the associated commuter account is time stamped or recorded to calculate the correct fare payment. The red light indicator 117 is illuminated when the commuter enters or leaves the gate 110, the charging system 100 is unable to actively identify the identity of the commuter, or is unable to register charges for the correct fare from the commuter's account; when the red light indicator lights up, the commuter can retrace and approach the gate again; in the event of a sustained failure, the commuter may approach the station controller to manually resolve the system failure. In another embodiment the charging system comprises a buzzer connected to the red light indicator 117 in order to give an additional audible warning in case of a wrong charge.

Fig. 2B shows two gates 110a according to another embodiment. Each gate 110a has two

barrier flaps

118, 118a that telescope or rotate into its associated cabinet. Two cameras 114a are suspended above each gate; one camera 114a is directed to photograph the face of the commuter entering the gate 110a, while when the gate 110a is configured for two-way personnel passage, the other camera is directed to photograph the face of the commuter exiting the gate 110 a. When the commuter identity is found to match the identity registered in the ID database 160, the barrier flaps 118, 118a are actuated to retract/rotate and allow the commuter to pass; recording the commuter account during entry and deducting from the commuter account during exit, for example after completing the trip, and preferably the green light indicator 116a is illuminated; when the commuter identity is not found in the commuter ID database 160, or an error occurs in recording a charge or deduction from the commuter account, the

barrier shield

118, 118a remains in the blocking position and the red light indicator 117a is illuminated; an additional audible warning may be issued so that the commuter may notify the station manager to correct the charge error.

Fig. 2C shows an unobstructed gate 110b according to another embodiment of the invention. The barrier-free shutter 110b has a pair of partition members 119 and a pair of cameras 114a suspended above the barrier-free shutter 110 b; preferably, the partition member 119 defines a discrete passage for the passage of the barrier-free shutter; alternatively, the passage through the unobstructed gate 110b may be defined by markings on the ground. When a commuter enters or leaves near the barrier-free gate 110b, one of the pair of cameras 114a detects the commuter and takes a picture of the commuter; the commuter photo is then relayed to the

server

140, 140a along with the entrance or exit station ID and matched to the record in the ID database 160; once the charging system 100 actively identifies the commuter, the commuter account is time stamped or recorded into the payment/designated area; preferably, the commuter ID and/or commuter account are tagged or entered into an entry record in the ID database 160 and/or a list of pending deductibles stored in the account database 170. In a similar manner, when the commuter is to leave the payment/designated area and come within range of the camera 114a, the commuter portrait is photographed and relayed to the

server

140, 140a and matched with records in the ID database 160, entry records in the ID database 160; once the commuter is actively identified, the charge is recorded and deducted from the commuter account. If the charging system fails, such as failing to actively identify the commuter, failing to charge or deduct money from the commuter's account, a red light indicator 117a may be illuminated to notify the commuter and station controller of the system error.

To notify the commuter of the error, a tracking beam may be issued to identify the commuter that triggered the system error, for example by illuminating the tracking beam on or around the commuter. Additionally or alternatively, the commuter triggering the system error is highlighted on the monitor screen of the station control room; further, the station controller may be equipped with Augmented Reality (AR) glasses, and commuters may be identified/tracked by the station controller at any time. Using the AR glasses, a commuter ID (such as a name), a fare amount, a time of admission, a station ID entering a payment/designated area, a transaction history, and other related information may be provided to a station controller through the AR glasses.

Fig. 2D shows a variant barrier-free gate 110c according to the above embodiment. Each of the barrier-free shutters 110c has a pair of RF detectors 119a and a pair of cameras 114a suspended above the barrier-free shutter 110 c. The RF detector 119a is built in the associated partition member 119 and is configured as a long-distance antenna. Preferably, the size of each RF detector 119a is substantially the height of an adult commuter; and preferably the long-range antenna 119a has a detection range of approximately 1-1.5 m and is tuned to the frequency of interrogation with the RFID chip embedded in the commuter RFID medium 119b carried by each commuter. For example, as shown in fig. 2D, the RFID medium 119b is worn as a wristband. One of the pair of cameras 114a detects the commuter and takes a picture of the commuter as the commuter approaches the barrier-free gate 110c to enter or exit; as the commuter gets closer and comes within range of the RF detector 119a, the commuter ID and/or the commuter photo is sent to the local workstation 150, for example at a local transit station. The commuter ID and commuter photo are then relayed to the

server

140, 140a along with the entry station ID and delivery record and matched against a record in the ID database 160 or an incoming record in the ID database; once the charging system 100 actively identifies the commuter, the commuter account is time stamped or recorded for calculating the correct fee to be paid upon exit from the payment/designated area; additionally or alternatively, a list of deductible fees to be processed is stored as a subset in the account database 170. In a similar manner, when the commuter is about to leave the payment/designated area and come within range of the RF sensor 119a, the commuter ID and/or commuter photograph is relayed to the ID database, and then the entry record in the commuter ID database 160 and/or the list of deductible fees to be processed and the collection fee in the account database 170 are recorded, and when the commuter leaves the payment/designated area, the fees are deducted from the commuter account. In this embodiment, both the commuter photo and the commuter RFID are used for identification or authentication; in this embodiment, the response time of the charging system is shortened to approximately less than 1 second, improving the reliability of the ID verification. The commuter RFID need only be used for authentication when deducting from the commuter account, which further increases the advantages of using such an obstacle-free charging system 100.

When the commuter is focusing on the electronic product and their face cannot be captured by the camera 114a or when their face is partially covered by a headcloth, the commuter RFID may still be used to identify the individual commuter; this is another advantage of the barrier-free charging system 100. In the case of commuters whose faces cannot be photographed by the camera 114a and who do not carry any commuters RFID, these commuters may be identified by directing a tracking beam over the person or highlighting them on a monitor screen; the station controller, preferably wearing AR glasses (as described above), can then access these commuters and have the errors in the charges compensated.

Fig. 3A shows a group of multi-purpose kiosks 120 located in a common area (e.g., outside of a transfer station). Each kiosk 120 has a touch screen 122. Above the touch screen is the camera 124, and below the touch screen is: a coin insertion slot 125, an electronic transaction terminal 126 having a keypad; a near field card reader 127; a receipt printer 128; and a banknote slot 129. The microphone 123 may also be disposed near one side of the touch screen, or the directional microphone 123 may be located in a lower portion of the kiosk. Speaker 132 is also located in the lower portion of the kiosk, and virtual call button 131 is located on menu screen 130b on touch screen 122. The call button 131, microphone 123 and speaker 132 allow the customer to communicate with an operator in a telephone service center connected to the billing system 100. In another embodiment, an RFID media slot 128a may be provided; after successful commuter registration, the RFID media slot 128a may assign commuter RFID media 119 b; the commuter may also register himself remotely, e.g., from home, and digitally send a QR code to someone's mobile device; using the QR code, the commuter arrives at the kiosk 120 and authenticates his identity before the RFID media 119b is issued; preferably, the authentication process employs two-factor two-channel authentication for security verification, where the first factor is a QR code and the second factor may be a random code generated by an encoding machine and sent to the commuter's mobile device. These RFID media 119b may be in the form of cards, rings, wristbands, key fobs, certificates, and the like. The location of the components on kiosk 120 may vary depending on the design without affecting their functionality. In one embodiment, in use, the touch screen 122 may be virtually and dynamically divided into an upper advertising or information display screen 130a and a lower operating menu screen 130 b.

Several kiosks 120 are placed side-by-side and the touch screen 122 or any number of adjacent touch screens 122 may be dynamically configured as a large composite display screen 122a, for example, to advertise or display some public information if the camera 124 does not detect a commuter within a predetermined distance; the large composite display screen 122a may overlay the display screens 122 of two or more kiosks for displaying advertising or public information. When the commuter approaches the kiosk 120, the camera 124 detects the person and the menu screen 130b appears, and the touch screen 122 of the adjacent kiosk dynamically adjusts to the new composite display size.

At the kiosk 120, the commuter may choose to perform several functions from the menu screen 130 b. For example, the commuter may choose to register a new account by entering a name, then selecting a portrait that has been captured by the camera 124, clicking on a payment card of a near field card reader, entering a payment card Personal Identification Number (PIN) on the electronic transaction terminal 126, and entering his gender, race, date of birth, height, traffic category, etc. on the menu screen 130 b. The traffic category may include students, national attendants, handicapped persons, elderly persons, passengers, or staff, who may enjoy special discounts, privileges, or promotional activities. After registration, the commuter identity is stored in the commuter ID database 160, with personal records additionally stored under the respective ID categories; the additional storage of personal records under different ID categories allows for a very quick and accurate identification of the commuter in the commuter ID database 160, such that the time required to identify the commuter from the commuter ID database and to deduct money from the account database 170 for the commuter is substantially unnoticeable to the commuter. For commuters entering the payment area, a dynamic list of commuters is stored as a subset in the ID database 160, thereby making identity matching in the ID database faster; in other words, the ID database 160 is dynamically divided into two subsets according to the commuter inside or outside the payment area for faster commuter identification; in this way, the account database 170 is dynamically divided into two subsets according to the deductions to be processed or commuters outside the payment area.

When the amount in the commuter account is low, the commuter may also replenish his or her account at the multi-function kiosk 120 by using payment cards, bank notes, or coins, or by mobile means of the near field reader 127. In another embodiment, the commuter can set a predetermined low amount for the charging system 100 to automatically replenish his commuter account.

From the menu screen 130b on the touch screen 122, the commuter may also access information about the transportation system (e.g., maps) or connect to a predetermined web page in the internet. Alternatively, browsing of web pages in the internet may be provided by configuring an additional screen 122B on the back side of each kiosk 120, as shown in fig. 3B, with a keyboard 138 located on a sloped panel 139 that allows access to money from the bill and

coin slots

125, 129 or from the back maintenance components of the kiosk.

As shown in fig. 1, system software 142 executes on server 140. According to the charging system 100, the server 140 is in turn connected to remotely located workstations 150, for example at all transfer stations. In one embodiment, the server 140 includes a master server and a plurality of slave servers 140a to distribute the computational workload and respond as quickly as commuters enter or leave designated payment areas; the slave server 140a may perform parallel operations with the master server 140. In another system, the server 140 includes a plurality of slave servers 140a that operate grid operations. The

servers

140, 140a may also operate a combination of parallel operations, grid operations, and other forms of distributed operations. The system software 142 includes a face recognition subroutine 144. During enrollment, the face recognition subroutine 144 analyzes the unique traits of the commuter's face from a portrait/photograph (shown in FIGS. 4A, 4B) selected by the commuter during enrollment; in one embodiment, the facial recognition subroutine 144 is intelligent enough to distinguish a commuter wearing glasses. The face recognition subroutine 144 may also analyze the mood of the commuter, such as happy, sad, etc.; in use, the facial recognition subroutine 144 is configured to insert a personalized greeting into each commuter as the commuter is actively identified in the commuter ID database 160; such personalized greetings may be "good morning, mr.", "refuel, mr.", "smile, mr.", etc. Another feature of this billing system 100 is targeted marketing; for example, when actively identifying a commuter, the facial recognition subroutine 144 may be configured with another subroutine to perform appropriate advertising for the commuter; for example, the advertisement may be gender specific, targeted to a person wearing glasses, targeted to a child, and so forth. In another embodiment, the charging system 100 allows importing wanted or missing personal databases 161. The wanted/lost personal database 161 can be useful when it is necessary to find and track wanted/lost people that may be present in the payment area or in public areas outside the payment area.

Still other features make the billing system 100 easier to use. For example, with a camera at each of the entry/

exit gates

110, 110a, 110b, the charging system 100 may also include a subroutine for detecting a crowd. When a crowd is formed at a

gate

110, 110a, 110b, 110c, the toll collection system may respond by sending an alert to the station controller at the local terminal, which then diverts the flow of personnel to the

other gates

110, 110 a-110 c.

The above-described transport system is complex and many system and subsystem errors may occur. For example, a commuter may wear a hat or sunglasses when entering or leaving the gate. The hat or sunglasses may obstruct certain facial features of the commuter, causing difficulty in recognition, resulting in system failure. Such a fault may be corrected by the commuter seeking assistance from a station controller, who may use a camera to take a portrait of the commuter and link the commuter ID to the portrait. Such manually linked commuter IDs may be stored in separate ID categories in the ID database 160.

Another feature of the face recognition subroutine 144 is the ability to recognize faces from a group of people, as shown in FIG. 5A, using the camera 114 b. This feature is useful when using the toll collection system 100 on a bus 210, as shown in fig. 5B, which is similar to the barrier-free gate 200, but where personnel tend to move in a relatively unordered manner (as in a crowd). As in the above embodiment, when the passenger identity is not found in the commuter ID database, an error of recording a charge or deduction from the corresponding commuter account occurs, the person is identified on the monitor screen so that the system malfunction can be solved. In another embodiment, a pair of RF detectors 119a may be deployed at the entrance gate and exit gate (as shown in FIG. 5C) on the bus 210 to enhance the above-described facial recognition, identification, monitoring.

Although specific embodiments have been described and illustrated, it should be understood that many changes, modifications, variations, combinations, and alterations to the invention may be made without departing from the scope of the invention. For example, the gate 110a in fig. 2B may use an additional barcode, QR code, and/or NFC scanner.

Claims

1. A hands-free toll collection system comprising:

a server executing software, the software including a facial recognition subroutine;

a commuter identity database, namely a commuter ID database, for storing commuter identities, wherein the stored commuter identities are classified into name, gender, ethnic group ethnicity, date of birth, height, and authorized group to allow for quick and accurate identification, wherein the commuter ID database is linked to the server; wherein the commuter ID database is dynamically divided into two subsets according to commuters inside or outside of a payment area for faster commuter identification;

a commuter account database for storing commuter account information, wherein the commuter account database is linked to the server; wherein the commuter account database is dynamically divided into two subsets according to commuters of pending payment fees or commuters outside the payment area;

a gate for entering or leaving a designated payment area defined by the toll collection system, wherein the gate has a set of RF detectors and a camera, the set of RF detectors interrogating a commuter's RFID chip embedded in a commuter's RFID media carried by the commuter, and the camera taking a picture of the commuter as the commuter enters or leaves near the gate; and

a multi-function kiosk for registering a commuter, wherein a camera captures a portrait of the commuter, and the facial recognition subroutine associates a unique facial feature of the commuter with a customer identity, and the charging system stores the commuter identity in the commuter ID database and the commuter account database, such that upon registration the commuter may enter or leave the designated payment area without having to carry a stored value ticket.

2. The system of claim 1, wherein the authorization group comprises: children, the elderly, disabled, students, national service personnel, passengers, and staff.

3. The system of claim 1, wherein the facial recognition subroutine makes further classifications in the commuter ID database, the further classifications including a list of people wearing glasses or a list of people who want to find or are missing.

4. The system of claim 1, wherein the server comprises a master server and a plurality of slave servers.

5. The system of claim 4, wherein the master server and the slave server are configured for parallel operations.

6. The system of claim 4, wherein the master server and the slave server are configured for grid computing.

7. The system of any preceding claim further comprising a subroutine that inserts a greeting into the commuter at the multi-function kiosk, entry gate or exit gate when the commuter identity is found in the ID database.

8. The system of claim 7, wherein the gate for entry or exit is unobstructed.

9. The system of claim 8, wherein a commuter or customer triggering a system fault is identified by the beam tracker, highlighted on a monitor screen, or highlighted on Augmented Reality (AR) glasses worn by a system controller.

10. The system of claim 8 or 9, wherein the set of RF detectors is a pair of long-range RF detectors.

11. The system according to any one of claims 8-9, wherein the gate is deployed for a mass transit station, a bus, a stadium, a concert hall, or a passenger lobby of an departure or entry lobby.

12. The system of claim 1, wherein the display screen of the kiosk is operable to display an advertisement or informational screen and an operational menu screen when the camera detects the presence of a commuter.

13. The system of claim 12, wherein there are a plurality of multi-function kiosks, each placed alongside one another, and the advertising or informational screen is dynamically composed of two or more display screens when no commuters are present within a predetermined range.

14. The system of claim 1, further comprising a touch screen and a keyboard, both disposed behind the multi-function kiosk for commuters to access the internet.

15. A method for charging a fee from a commuter, the method comprising:

splitting records of a commuter identity, and storing these records in an ID database under separate commuter ID categories, wherein the records of the commuter identity are categorized into name, gender, ethnic group ethnicity, date of birth, height, and authorized groups to allow for quick and accurate identification; wherein the commuter ID database is dynamically divided into two subsets according to commuters inside or outside of a payment area for faster commuter identification;

storing the commuter identity in a commuter ID database;

storing account information for the commuter in a commuter account database, wherein the commuter account database is dynamically divided into two subsets according to commuters of pending payment fees or commuters outside the payment area;

capturing a portrait of a commuter with a camera and interrogating a RFID chip embedded in a commuter RFID media carried by the commuter; and

facial features of a commuter are extracted using a facial recognition subroutine, and the commuter is identified according to one or more of the ID categories and identified by RFID.

16. The method of claim 15 wherein the facial recognition subroutine dynamically cooperates with a toll collection system to establish a list of commuters who have entered a designated payment area or commuters who are to be charged.