KR102594097B1 - User equipment, service providing device, payment system comprising the same, control method thereof and computer readable medium having computer program recorded thereon - Google Patents

Abstract

The present invention discloses a user device, a service providing device, a payment system including the same, a control method thereof, and a recording medium on which a computer program is recorded. That is, the present invention acquires image information through a front camera provided on the front of the user device when displaying a preset important screen, such as receiving a PIN number corresponding to a payment card, on the user device, and If one or more objects are recognized through the object recognition function and a user other than a pre-registered user is included among the one or more recognized objects, a preset function is used to prevent exposure of information on important screens according to user input. , When using services such as payment or banking using a mobile user device, the load on the user device can be reduced while increasing security.

Description

User equipment, service providing device, payment system comprising the same, control method thereof and computer readable medium having computer program recorded thereon }

The present invention relates to a user device, a service provision device, a payment system including the same, a control method thereof, and a recording medium on which a computer program is recorded. In particular, the present invention relates to a user device that displays preset important screens, such as receiving a PIN number corresponding to a payment card, on the user device. When displaying, image information is acquired through the front camera provided on the front of the user device, one or more objects are recognized through the object recognition function within the acquired image information, and a pre-registered user is selected from among the one or more recognized objects. When other users are included, a user device that prevents exposure of information on important screens according to user input through a preset function, a service provision device, a payment system including the same, a record of its control method, and a computer program It's about the media.

As mobile payments become more common, situations where information is entered on the mobile screen at payment locations are frequently occurring.

In these situations, the risk of your PIN number being exposed to others may increase.

In response to this, methods such as using a film that prevents the screen from being seen from the side or masking the key image have been used, but these methods require additional means or cause problems in making input difficult for the user.

Korean Patent Publication No. 10-2004-0079422 [Name: Private information input system to prevent information leakage through peeking]

The purpose of the present invention is to monitor the back of the user, which the user cannot see, through the front camera provided in the user device to prevent a situation where someone is peeping when entering personal information or important information or a password on the payment screen. The aim is to provide a user device that notifies the user when another person's face is detected, a service provision device, a payment system including the same, a control method thereof, and a recording medium on which a computer program is recorded.

Another object of the present invention is to identify the user through the front camera on the payment screen and simultaneously perform basic identification of the user and confirmation of others using the user's facial features, a service provision device, and a payment system including the same; The purpose is to provide a recording medium on which the control method and computer program are recorded.

A user device according to an embodiment of the present invention includes a user device display unit that displays a preset screen to receive information according to user input; a front camera that acquires image information when a preset screen is displayed on the user device display; and a user who performs an object recognition function on the acquired image information to recognize one or more objects included in the image information and checks whether information is leaked during the information reception process on a preset screen based on the characteristics of the one or more recognized objects. It may include a device control unit.

As an example related to the present invention, a preset screen includes a PIN number reception screen for receiving a PIN number corresponding to the selected payment card after selecting a payment card, a payment card number reception screen for receiving payment card information, and a security card number reception screen. It may include either a security card number reception screen for receiving a security card number or a personal information reception screen for receiving personal information.

As an example related to the present invention, the front camera may acquire surrounding and rear image information of the user while the user of the user device is looking at the front camera.

As an example related to the present invention, when there is only one recognized object, the user device control unit calculates the similarity between the characteristic information of the recognized object and the characteristic information of the preset reference object, and determines whether the calculated similarity is greater than or equal to the preset reference value. You can check.

As an example related to the present invention, when the calculated similarity is less than a preset reference value, the user device control unit may prevent information exposure according to user input on a preset screen displayed on the display unit of the user device through a preset function. .

As an example related to the present invention, when there are multiple recognized objects, the user device control unit may prevent information exposure according to user input on a preset screen displayed on the user device display unit through a preset function.

As an example related to the present invention, preset functions include a function to notify the user that another person's face has been detected, a function to dim the color of the keyboard for input displayed on the display of the user device, and a function to display the input keyboard screen in sentence units in units of one character. At least one of the following: a function to change the input keyboard screen of the screen, a function to display the input keyboard screen in the opposite direction to the direction in which other people in the video information are located, and a function to display guidance to change the direction of the user's device in a direction that can block the gaze of others. may include.

A control method of a user device according to an embodiment of the present invention includes the steps of displaying a preset screen to receive information according to user input through a display unit of the user device; When a preset screen is displayed on the display of the user device through the front camera, acquiring surrounding and rear image information of the user while the user of the user device is looking at the front camera; Recognizing one or more objects included in the image information by performing an object recognition function on the acquired image information through a user device control unit; Determining, through a user device control unit, whether one object is included in the recognized image information; Checking, through a user device control unit, whether the characteristic information of the recognized object is similar to the characteristic information of a preset reference object when the recognized image information includes one object; And through the user device control unit, either a case where a plurality of objects are included in the image information recognized as a result of a determination, or a case where the characteristic information of the recognized object is not similar to the characteristic information of a preset reference object as a result of confirmation. In the case of , the step of preventing exposure of information according to user input on a preset screen displayed on the display unit of the user device through a preset function may be included.

As an example related to the present invention, when the characteristic information of the recognized object is similar to the characteristic information of a preset reference object as a result of the determination, returning to the step of acquiring peripheral and rear image information of the user; stopping the operation of the front camera when receiving information according to user input through a user device control unit; And it may further include transmitting the received information, information on the payment card corresponding to the information, and identification information of the user device to the service providing device through the user device communication unit.

A computer program for performing the method according to the above-described embodiment may be stored in a recording medium on which the computer program according to the embodiment of the present invention is recorded.

The present invention monitors the back of the user, which cannot be seen by the user, through the front camera provided in the user device to prevent someone from peeping when entering personal information or important information or a password on the payment screen. By notifying the user when a person's face is detected, it has the effect of increasing security and reducing the load on the user device when using services such as payment or banking using a mobile user device.

In addition, the present invention verifies the user through the front camera on the payment screen, and simultaneously performs basic identification of the user and confirmation of others using the user's facial features, thereby ensuring basic user authentication and preventing others from peeping. There is an effect that can be performed simultaneously without manipulation.

1 is a block diagram showing the configuration of a payment system according to an embodiment of the present invention.
Figure 2 This is a block diagram showing the configuration of a user device according to an embodiment of the present invention.
Figure 3 is a block diagram showing the configuration of another user device according to an embodiment of the present invention.
Figure 4 is a flowchart showing a control method of a payment system according to an embodiment of the present invention.
5 to 7 are diagrams showing a screen of a user device according to an embodiment of the present invention.

It should be noted that the technical terms used in the present invention are only used to describe specific embodiments and are not intended to limit the present invention. In addition, the technical terms used in the present invention, unless specifically defined in a different sense in the present invention, should be interpreted as meanings generally understood by those skilled in the art in the technical field to which the present invention pertains, and are not overly comprehensive. It should not be interpreted in a literal or excessively reduced sense. Additionally, if the technical terms used in the present invention are incorrect technical terms that do not accurately express the idea of the present invention, they should be replaced with technical terms that can be correctly understood by those skilled in the art. In addition, general terms used in the present invention should be interpreted according to the definition in the dictionary or according to the context, and should not be interpreted in an excessively reduced sense.

Additionally, as used in the present invention, singular expressions include plural expressions unless the context clearly dictates otherwise. In the present invention, terms such as “consists of” or “comprises” should not be construed as necessarily including all of the various components or steps described in the invention, and some of the components or steps may not be included. It may be possible, or it should be interpreted as being able to further include additional components or steps.

Additionally, terms containing ordinal numbers, such as first, second, etc., used in the present invention may be used to describe constituent elements, but the constituent elements should not be limited by the terms. Terms are used only to distinguish one component from another. For example, a first component may be named a second component without departing from the scope of the present invention, and similarly, the second component may also be named a first component.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of the reference numerals, and duplicate descriptions thereof will be omitted.

Additionally, when describing the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the present invention, the detailed description will be omitted. In addition, it should be noted that the attached drawings are only intended to facilitate easy understanding of the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the attached drawings.

Figure 1 is a block diagram showing the configuration of a payment system 10 according to an embodiment of the present invention.

As shown in FIG. 1, the payment system 10 consists of a user device 100, a beacon 200, a service providing device 300, a POS terminal 400, and a payment server 500. Not all of the components of the payment system 10 shown in FIG. 1 are essential components, and the payment system 10 may be implemented with more components than those shown in FIG. 1, or fewer components. The payment system 10 may also be implemented by .

The user device 100 may be a smart phone, a portable terminal, a mobile terminal, a personal digital assistant (PDA), a portable multimedia player (PMP) terminal, or a telematics device. Terminal, navigation terminal, personal computer, laptop computer, Slate PC, Tablet PC, ultrabook, wearable device (e.g., watch type) Terminal (Smartwatch), glass-type terminal (Smart Glass), HMD (Head Mounted Display), etc.), Wibro terminal, IPTV (Internet Protocol Television) terminal, smart TV, digital broadcasting terminal, television, 3D It can be applied to various terminals such as televisions, home theater systems, AVN (Audio Video Navigation) terminals, A/V (Audio/Video) systems, and flexible terminals.

As shown in FIG. 2, the user device 100 includes a user device communication unit 110, a user device storage unit 120, a user device display unit 130, a user device voice output unit 140, and a user device control unit 150. ) is composed of. Not all of the components of the user device 100 shown in FIG. 2 are essential components, and the user device 100 may be implemented with more components than those shown in FIG. 2, or fewer components. The user device 100 may also be implemented by .

The user device communication unit 110 communicates with any internal component or at least one external terminal through a wired/wireless communication network. At this time, any external terminal may include a beacon 200, a service providing device 300, a POS terminal 400, a payment server 500, etc. Here, wireless Internet technologies include Wireless LAN (WLAN), DLNA (Digital Living Network Alliance), Wibro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), and HSDPA (High Speed Downlink Packet Access). ), HSUPA (High Speed Uplink Packet Access), IEEE 802.16, Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced), Wireless Mobile Broadband Service (WMBS), etc. There is a user device communication unit 110 that transmits and receives data according to at least one wireless Internet technology, including Internet technologies not listed above. In addition, short-range communication technologies include Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, and Near Field Communication (NFC). , Ultrasound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, etc. may be included. Additionally, wired communication technologies may include Power Line Communication (PLC), USB communication, Ethernet, serial communication, optical/coaxial cables, etc.

Additionally, the user device communication unit 110 can transmit information to and from any terminal through a universal serial bus (USB).

In addition, the user device communication unit 110 supports technical standards or communication methods for mobile communication (e.g., Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV-DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE- Wireless signals are transmitted and received with a base station, beacon 200, service providing device 300, POS terminal 400, payment server 500, etc. on a mobile communication network built according to A (Long Term Evolution-Advanced), etc.).

In addition, the user device communication unit 110 is controlled by the user device control unit 150, and the identification information of the beacon 200 (or the unique identification information of the store equipped with the beacon 200) is transmitted from the beacon 200. ) is received.

The user device storage unit 120 stores various user interfaces (UI), graphic user interfaces (GUI), etc.

Additionally, the user device storage unit 120 stores data and programs necessary for the user device 100 to operate.

That is, the user device storage unit 120 may store a plurality of application programs (application programs or applications) running on the user device 100, data for operating the user device 100, and commands. At least some of these applications may be downloaded from an external service providing device through wireless communication. Additionally, at least some of these applications may be present on the user device 100 from the time of shipment for the basic functions of the user device 100 (e.g., call incoming and outgoing functions, message receiving and sending functions). Meanwhile, the application program is stored in the user device storage unit 120, installed on the user device 100, and driven by the user device control unit 150 to perform the operation (or function) of the user device 100. there is.

In addition, the user device storage unit 120 includes flash memory type, hard disk type, multimedia card micro type, and card type memory (for example, SD or XD memory, etc.), magnetic memory, magnetic disk, optical disk, RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory: ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory) , and may include at least one storage medium among PROM (Programmable Read-Only Memory). Additionally, the user device 100 may operate web storage that performs a storage function of the user device storage unit 120 on the Internet, or may operate in relation to web storage.

In addition, the user device storage unit 120 stores the identification information of the corresponding beacon 200 (or the store equipped with the corresponding beacon 200) received through the user device communication unit 110 under the control of the user device control unit 150. unique identification information).

The user device display unit 130 may display various contents, such as various menu screens, using the user interface and/or graphic user interface stored in the user device storage unit 120 under the control of the user device control unit 150. Here, the content displayed on the user device display unit 130 includes various text or image data (including various information data) and a menu screen including data such as icons, list menus, and combo boxes. Additionally, the user device display unit 130 may be a touch screen.

In addition, the user device display unit 130 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. It may include at least one of a flexible display, a 3D display, an e-ink display, and a light emitting diode (LED).

Additionally, the user device display unit 130 may be configured as a stereoscopic display unit that displays a stereoscopic image.

A three-dimensional display method such as a stereoscopic method (glasses method), an autostereoscopic method (glasses-free method), or a projection method (holographic method) may be applied to the three-dimensional display unit.

In addition, the user device display unit 130 displays the identification information of the beacon 200 (or the unique information of the store equipped with the beacon 200) received through the user device communication unit 110 under the control of the user device control unit 150. identification information).

The user device voice output unit 140 outputs voice information included in a signal processed by the user device control unit 150. Here, the user device audio output unit 140 may include a receiver, speaker, buzzer, etc.

Additionally, the user device voice output unit 140 outputs a guidance voice generated by the user device control unit 150.

In addition, the user device audio output unit 140 receives the identification information of the beacon 200 (or the store equipped with the beacon 200) received through the user device communication unit 110 under the control of the user device control unit 150. outputs voice information corresponding to the unique identification information of ).

The user device control unit 150 performs overall control functions of the user device 100.

Additionally, the user device control unit 150 executes overall control functions of the user device 100 using programs and data stored in the user device storage unit 120. The user device control unit 150 may include RAM, ROM, CPU, GPU, and a bus, and the RAM, ROM, CPU, GPU, etc. may be connected to each other through a bus. The CPU can access the user device storage unit 120 and perform booting using the O/S stored in the user device storage unit 120, and various programs, content, and data stored in the user device storage unit 120. You can perform various operations using the etc.

In addition, the user device control unit 150 performs a membership registration procedure for the user of the corresponding user device 100 by linking with the service providing device 300.

In addition, when performing the membership registration process, the user device control unit 150 must complete the authentication function through an identity authentication method (including mobile phone, credit card, i-PIN, email, etc.) to register the member for the service provision device 300. You can complete the registration process normally.

In addition, when a user carrying the user device 100 enters the store, the user device control unit 150 provides identification information of the beacon 200 transmitted from the beacon 200 (or information provided with the beacon 200). The user device communication unit 110 is controlled to receive unique identification information of the store.

In addition, the user device control unit 150 transmits the unique identification information of the received beacon 200 (or the unique identification information of the store equipped with the beacon 200), the identification information of the user device 100, etc. to the user device communication unit ( It is transmitted to the service providing device 300 through 110). Here, the identification information of the user device 100 includes MDN (Mobile Directory Number), mobile IP, mobile MAC, SIM (subscriber identity module) card unique information, serial number, etc.

In addition, if the user of the user device control unit 150 is not a member pre-registered in the service provision device 300, the user device control unit 150 provides guidance information related to membership registration provided by the service provision device 300. Based on this, a membership registration procedure is performed for the corresponding user device 100 (or the user of the corresponding user device 100).

In addition, if the user of the user device control unit 150 is a member pre-registered in the service providing device 300, the user device control unit 150 may provide the unique identification information of the transmitted beacon 200 (or the corresponding beacon 200). The user device communication unit 110 transmits one or more advertising information, one or more benefit information, a recommended card list, etc. transmitted from the service providing device 300 in response to the unique identification information of the store provided (unique identification information), identification information of the user device 100, etc. ) is received through. Here, benefit information includes coupon information, membership benefit information, discount benefit information, etc. Additionally, the recommended card list includes one or more cards to which the benefit information can be applied among a plurality of cards (or one or more cards) previously issued to the user device 100.

Additionally, the user device control unit 150 outputs one or more received advertisement information, benefit information, recommended card lists, etc. through the user device display unit 130 and/or the user device audio output unit 140.

In addition, the user device control unit 150 confirms one payment card (or unique identification information of the payment card) selected from a plurality of payment cards registered (or issued) to a specific app pre-installed on the user device 100. do.

Additionally, the user device control unit 150 may check the payment card selected from the recommended card list transmitted from the service providing device 300.

In addition, the user device control unit 150 selects one or more payment cards to which the benefit information can be applied among a plurality of payment cards pre-registered in the user device 100 based on one or more benefit information provided from the service providing device 300. A corresponding recommendation card list containing a may be generated, and the generated recommendation card list may be displayed on the user device display unit 130. Additionally, it is possible to check one payment card selected from the list of recommended cards displayed on the user device display unit 130.

Additionally, the user device control unit 150 receives a PIN number (personal identification number) according to user input through a specific app pre-installed on the user device 100. Here, the PIN number may be a PIN number corresponding to the confirmed payment card type.

In addition, the user device control unit 150 transmits the confirmed payment card type (or unique identification information of the payment card), PIN number, and identification information of the user device 100 to the service providing device 300 through the user device communication unit 110. ) is sent to

Additionally, the user device control unit 150 receives a token transmitted from the service providing device 300 in response to the previously transmitted PIN number through the user device communication unit 110.

Additionally, the user device control unit 150 generates a barcode (or QR code (Quick Response code)) including the received token (or token information) and identification information of the user device 100.

Additionally, the user device control unit 150 stores the generated barcode (or QR code) in the user device storage unit 120.

In addition, when a preset payment method is selected through a specific app pre-installed on the user device 100, the user device control unit 150 provides barcode information (or QR code) corresponding to the previously generated barcode (or QR code). information) is transmitted to the POS terminal 400 through the user device communication unit 110. Here, barcode information corresponding to the barcode (or QR code information corresponding to the QR code) includes a token (or token information), identification information of the user device 100, etc. At this time, payment methods include cash payment, card payment, electronic payment, etc., and the preset method may be electronic payment.

As such, when the user device 100 and the POS terminal 400 are connected through wired/wireless communication, barcode information (or QR code information) can be transmitted from the user device 100 to the POS terminal 400.

In addition, if the user device 100 and the POS terminal 400 are not connected through wired/wireless communication, the user of the user device 100 may use a preset app through a specific app pre-installed on the user device 100. When a payment method is selected, a barcode (or QR code) displayed on the user device display unit 130 may be recognized (or scanned/tagged) by the POS terminal 400.

In addition, after the payment function is performed for one or more products to be paid for selected by the user of the user device 100 based on the previously transmitted barcode information (or QR code information), the user device control unit 150 The result of performing the payment function transmitted from the POS terminal 400 (or the service providing device 300) in response to the barcode information (or QR code information) is received through the user device communication unit 110.

Additionally, the user device control unit 150 outputs the received payment function performance result through the user device display unit 130 and/or the user device audio output unit 140.

Additionally, the user device 100 may further include an interface unit (not shown) that serves as an interface with all external devices connected to the user device 100. For example, the interface unit includes a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, a port for connecting a device equipped with an identification module, and audio I/O ( Input/Output) port, video I/O (Input/Output) port, earphone port, etc. Here, the identification module is a chip that stores various information for authenticating the use authority of the user device 100, including a user authentication module (UIM), a subscriber identity module (SIM), and a general user authentication. It may include a module (Universal Subscriber Identity Module: USIM), etc. Additionally, a device equipped with an identification module may be manufactured in a smart card format. Accordingly, the identification module can be connected to the user device 100 through a port. Such an interface unit receives data or power from an external device and transmits it to each component inside the user device 100, or transmits data inside the user device 100 to an external device.

In addition, when the user device 100 is connected to an external cradle, the interface unit becomes a path through which power from the cradle is supplied to the user device 100, or various command signals input from the cradle by the user are transmitted to the user device 100. It can be a passage to (100). Various command signals or corresponding power input from the cradle may be operated as signals for recognizing that the user device 100 is correctly mounted on the cradle.

In addition, the user device 100 includes an input unit ( (not shown) may further be included.

The input unit is a means for receiving at least one of the user's commands, selections, data, and information, and may include a plurality of input keys and function keys for receiving numeric or character information and setting various functions.

In addition, the input unit includes a key pad, dome switch, touch pad (static/capacitive), touch screen, jog wheel, jog switch, jog shuttle, and mouse. , a variety of devices such as a stylus pen and a touch pen may be used. In particular, when the user device display unit 130 is formed in the form of a touch screen, some or all of the input functions can be performed through the user device display unit 130.

Additionally, each component (or module) of the user device 100 may be software stored in the memory (or user device storage unit 120) of the user device 100. The memory may be an internal memory of the user device 100, an external memory, or another type of storage device. Additionally, the memory may be non-volatile memory. Software stored on the memory may include a set of instructions that cause the user device 100 to perform a specific operation when executed.

The beacon 200 communicates with the user device 100, POS terminal 400, etc.

In addition, the beacon 200 receives store information related to the store equipped with the beacon 200, unique identification information of the store equipped with the beacon 200, etc. transmitted from the POS terminal 400.

Additionally, the beacon 200 broadcasts (or transmits) store information, identification information of the beacon 200 (or unique identification information of the store equipped with the beacon 200), etc.

The service providing device 300 communicates with the user device 100, POS terminal 400, payment server 500, etc.

In addition, the service providing device 300 includes a user device communication unit 110, a user device storage unit 120, a user device display unit 130, a user device voice output unit 140, and a user device included in the user device 100. The control unit 150 may include a communication unit (not shown), a storage unit (not shown), a display unit (not shown), a voice output unit (not shown), and a control unit (not shown) corresponding to each other.

In addition, the service providing device 300 performs a membership registration procedure for each user of the user device 100 by linking with the user device 100.

Additionally, the service providing device 300 receives unique identification information of the beacon 200, identification information of the user device 100, etc. transmitted from the user device 100.

In addition, the service provision device 300 determines that the user corresponding to the user device 100 is a member who has previously registered (or joined) the service provision device 300 based on the received identification information of the user device 100. Determine (or confirm) whether it is recognized or not.

As a result of judgment (or confirmation result), if the user corresponding to the corresponding user device 100 is not a member who has previously registered with the service providing device 300, the service providing device 300 sends guidance information related to membership registration to the user device. Send to (100).

Additionally, the service providing device 300 performs a membership registration procedure for the user device 100 (or a user of the user device 100) by linking with the user device 100.

In addition, as a result of judgment (or confirmation result), if the user corresponding to the corresponding user device 100 is a member who has previously registered with the service providing device 300, the service providing device 300 may use the previously received beacon 200 One or more advertisements that can be provided to the user device 100 at a store equipped with the beacon 200 among a plurality of advertisement information and a plurality of benefit information pre-stored based on the unique identification information and the identification information of the user device 100. Check information, information about one or more benefits, etc.

In addition, the service providing device 300 recommends one or more recommended cards to which the benefit information can be applied (or reflected) among a plurality of cards issued to the user device 100 based on one or more confirmed benefit information. Create a card list. Here, benefit information includes coupon information, membership benefit information, discount benefit information, etc.

In addition, the service providing device 300 provides a recommendation card list including one or more advertisement information, one or more benefit information, and one or more recommendation cards that can be provided to the user device 100 at a store equipped with the confirmed beacon 200. etc. are transmitted to the user device 100.

That is, if the user corresponding to the corresponding user device 100 is a member who has previously registered with the service providing device 300, the service providing device 300 selects the corresponding beacon ( 200) Check the store information corresponding to the unique identification information. In addition, the service providing device 300 confirms one or more advertisement information, benefit information, etc. that can be provided to the user device 100 at the store based on the confirmed store information and the received identification information of the user device 100. . Additionally, the service providing device 300 generates a recommended card list including one or more cards to which the benefit information can be applied among a plurality of cards (or one or more cards) previously issued to the user device 100. Additionally, the service providing device 300 transmits one or more confirmed advertisement information, one or more benefit information, and a generated recommendation card list to the user device 100.

Additionally, the service providing device 300 receives payment card type, PIN number, identification information of the user device 100, etc. transmitted from the user device 100.

Additionally, the service providing device 300 generates a token (or one-time token, token information, temporary token) corresponding to the received PIN number. Here, the token is a temporary token, is randomly generated in response to the PIN number, and may have a preset number of uses or a period of use. At this time, the service providing device 300 may generate a token corresponding to the received PIN number and payment card type.

Additionally, the service providing device 300 manages (or stores) the generated token in conjunction with the payment card type, PIN number, and identification information of the user device 100.

Additionally, the service providing device 300 transmits the generated token (or a token corresponding to the PIN number, a token corresponding to the PIN number and payment card type) to the user device 100.

In addition, the service providing device 300 uses a payment server 500 to perform an authentication function for the user before performing the payment function for the product in relation to one or more products to be paid for selected by the user of the user device 100. ) receives barcode information (or QR code information) transmitted from

In addition, the service providing device 300 determines whether the token in the barcode information (or QR code information) is valid based on the token in the received barcode information (or QR code information) and the identification information of the user device 100. do.

That is, the service providing device 300 includes the token in the received barcode information (or QR code information) and the identification information of the user device 100 within the identification information of the user device for each plurality of pre-stored (or pre-issued) tokens. Check whether the corresponding token and identification information of the user device are included (or present).

In this way, the service providing device 300 can verify the validity of the issued token.

As a result of confirmation, within the identification information of the user device for each of the plurality of tokens pre-stored in the service providing device 300, the token in the received barcode information (or QR code information) and the token corresponding to the identification information of the user device 100 and the user If identification information of the device is not included, that is, if the token in the received barcode information (or QR code information) is invalid, the service providing device 300 transmits authentication failure information to the payment server 500.

In addition, as a result of confirmation, within the identification information of the user device for each of the plurality of tokens pre-stored in the service providing device 300, the token in the received barcode information (or QR code information) and the token corresponding to the identification information of the user device 100 And if the identification information of the user device is included, that is, if the token in the received barcode information (or QR code information) is valid, the service providing device 300 is the user device in the received barcode information (or QR code information) ( Card information, user information, benefit information, etc. corresponding to the identification information of 100) are transmitted to the payment server 500. At this time, the service providing device 300 is a PIN managed in conjunction with the payment card type related to the valid token (or the corresponding payment card type/token confirmed based on the token previously generated corresponding to the PIN number and payment card type). Card information, user information, benefit information, etc. corresponding to the corresponding payment card type (identified based on the number, payment card type, and identification information of the user device)) may be transmitted to the payment server 500. Here, card information includes card number, expiration date, CVC (card verification code) number, user name, etc. Additionally, user information includes user name, date of birth, address, image, etc. Additionally, benefit information includes coupon information, membership benefit information, discount benefit information, etc.

That is, as a result of confirmation, if the token in the received barcode information (or QR code information) is valid, the service providing device 300 corresponds to the identification information of the user device 100 in the received barcode information (or QR code information). In relation to the user (or the corresponding user device 100), check the card information, user information, benefit information, etc. pre-stored in the service providing device 300, and make payment using the confirmed card information, user information, benefit information, etc. Transmitted to server 500. At this time, the service providing device 300 checks card information, user information, benefit information, etc. corresponding to the type of payment card previously transmitted from the user device 100, and makes payment using the confirmed card information, user information, benefit information, etc. It can also be transmitted to the server 500.

Additionally, the service providing device 300 receives payment function performance results (or payment information) transmitted from the payment server 500.

Additionally, the service providing device 300 stores the received payment function performance result in conjunction with the identification information of the corresponding user device 100.

Additionally, the service providing device 300 may transmit the received payment function performance result to the user device 100 corresponding to the identification information of the user device 100.

Additionally, the service providing device 300 may be implemented in the form of a web server, database server, proxy server, etc. In addition, the service providing device 300 may be equipped with a network load balancing mechanism or one or more of various software that allows the service providing device 300 to operate on the Internet or other networks, through which it can be converted into a computerized system. It can be implemented. Additionally, the network may be an http network, a private line, an intranet, or any other network. Furthermore, the service providing device 300 and the user device 100 may be connected through a secure network to prevent data from being attacked by hackers or other third parties. In addition, the service providing device 300 may include a plurality of database servers, and these database servers are implemented in such a way that they are separately connected to the service providing device 300 through any type of network connection, including a distributed database server architecture. It can be.

Additionally, a processor mounted on the user device 100 or the service providing device 300 according to the present invention can process program instructions for executing the method according to the present invention. In one implementation, the processor may be a single-threaded processor, and in another implementation, the processor may be a multi-threaded processor. Furthermore, this processor is capable of processing instructions stored in memory or a storage device.

The POS (Point of Sales) terminal 400 communicates with the user device 100, the beacon 200, the service provision device 300, the payment server 500, etc.

In addition, the POS terminal 400 includes a user device communication unit 110, a user device storage unit 120, a user device display unit 130, a user device voice output unit 140, and a user device control unit included in the user device 100. It may include a communication unit (not shown), a storage unit (not shown), a display unit (not shown), a voice output unit (not shown), and a control unit (not shown) corresponding to 150, respectively.

In addition, the POS terminal 400 transmits store information related to the store equipped with the beacon 200 (or the POS terminal), unique identification information of the store equipped with the beacon 200, etc. to the beacon 200. .

In addition, the POS terminal 400 scans one or more products to be paid (or products to be paid) selected by the user of the user device 100 (or recognizes the barcode attached to the product to be paid). .

Additionally, the POS terminal 400 displays product information about one or more scanned (or recognized) products to be paid. Here, product information includes product name, product unique code, unit price, quantity, amount, total amount, etc.

In addition, if the user device 100 and the POS terminal 400 are not connected through wired/wireless communication, the user of the user device 100 may use a preset app through a specific app pre-installed on the user device 100. When a payment method is selected, the POS terminal 400 recognizes (or scans/tags) a barcode (or QR code) displayed on the corresponding user device 100.

Additionally, the POS terminal 400 receives barcode information (or QR code information) transmitted from the user device 100.

Additionally, when the POS terminal 400 recognizes a barcode (or QR code) displayed on the user device 100, the POS terminal 400 checks the recognized barcode information (or QR code information).

Additionally, the POS terminal 400 displays or stores the received (or confirmed) barcode information (or QR code information).

In addition, the POS terminal 400 transmits one or more product information to be paid (or recognized product information to be paid), barcode information (or QR code information), identification information of the POS terminal 400, etc. to the payment server 500. do.

Additionally, the POS terminal 400 receives authentication failure information transmitted from the payment server 500 in response to barcode information (or QR code information) transmitted to the payment server 500.

Additionally, the POS terminal 400 displays the received authentication failure information.

Additionally, the POS terminal 400 may transmit the received authentication failure information to the user device 100.

In addition, just before performing the payment function for one or more products selected by the user of the corresponding user device 100 based on the barcode information (or QR code information) transmitted to the payment server 500, the POS terminal 400 makes the payment. User information transmitted from the server 500 is received. Here, the user information includes user name, date of birth, address, image, etc. related to the user corresponding to the identification information of the corresponding user device 100.

Additionally, the POS terminal 400 displays the received user information.

In addition, the user of the POS terminal 400 checks the image in the user information displayed on the POS terminal 400 and the user of the user device 100 who wishes to pay for the product, and then matches the image in the user information and the user device ( If the users of 100) match, the POS terminal 400 receives information indicating the user match according to the user input of the POS terminal 400.

Additionally, the POS terminal 400 transmits information indicating the received user match, identification information of the POS terminal 400, etc. to the payment server 500.

In addition, if the payment function for one or more products selected by the user of the corresponding user device 100 is normally performed based on the barcode information (or QR code information) transmitted to the payment server 500, the POS terminal 400 Receives the payment function performance result (or payment information) transmitted from the payment server 500 in response to the barcode information (or QR code information) transmitted to the payment server 500. Here, the payment function performance result (or payment information ) includes product name, product unique code, unit price, quantity, discount amount, coupon application, amount, total amount, payment date and time information, payment card type, etc.

Additionally, the POS terminal 400 transmits the received payment function performance result to the user device 100.

Additionally, the POS terminal 400 may output the received payment function performance results through printed matter such as a receipt.

The payment server 500 communicates with the user device 100, the service provision device 300, the POS terminal 400, etc.

In addition, the payment server 500 includes a user device communication unit 110, a user device storage unit 120, a user device display unit 130, a user device voice output unit 140, and a user device control unit included in the user device 100. It may include a communication unit (not shown), a storage unit (not shown), a display unit (not shown), a voice output unit (not shown), and a control unit (not shown) corresponding to 150, respectively.

In addition, the payment server 500 may include one or more product information to be paid (or recognized product information to be paid) transmitted from the POS terminal 400, barcode information (or QR code information), identification information of the POS terminal 400, etc. receives.

Additionally, the payment server 500 transmits the received barcode information (or QR code information) to the service providing device 300 in order to perform an authentication procedure related to the user device 100.

Additionally, if the token in the barcode information (or QR code information) transmitted to the service providing device 300 is invalid, the payment server 500 may use the barcode information (or QR code information) transmitted to the service providing device 300. In response, authentication failure information transmitted from the service providing device 300 is received.

Additionally, the payment server 500 transmits the received authentication failure information to the POS terminal 400.

Additionally, if the token in the barcode information (or QR code information) transmitted to the service providing device 300 is valid, the payment server 500 responds to the barcode information (or QR code information) transmitted to the service providing device 300. card information, user information, and benefit information transmitted from the service provision device 300 (or card information, user information, and benefit information corresponding to the identification information of the user device 100 in the corresponding barcode information (or QR code information)) Receive etc.

Additionally, the payment server 500 performs a payment function for the payment amount corresponding to the one or more product information to be paid based on the received card information, user information, and benefit information. At this time, the payment server 500 provides benefits corresponding to the benefit information transmitted from the service providing device 300 (or benefit information applicable to the user of the corresponding user device 100) to the price information for one or more products to be paid. You can perform a payment function for the payment amount applied.

At this time, immediately before performing the payment function, the payment server 500 transmits user information transmitted from the service providing device 300 to the POS terminal 400.

In addition, if the image in the user information does not match the user of the user device 100 who wants to pay for the product, the payment server 500 transmits payment cancellation information for the one or more product information to the POS terminal 400. Alternatively, the payment function may be performed after performing an additional authentication process for the corresponding user device 100.

In addition, if the image in the user information matches the user of the user device 100 who wants to pay for the product, the payment server 500 receives the information from the POS terminal 400 in response to the user information transmitted to the POS terminal 400. Information indicating the transmitted user match, identification information of the POS terminal 400, etc. are received.

Additionally, the payment server 500 may perform a payment function for the payment amount corresponding to the one or more product information to be paid based on the received card information, user information, benefit information, and information representing the user match.

Additionally, the payment server 500 transmits the payment function performance result (or payment information) to the POS terminal 400, the service provision device 300, etc. Here, the payment function performance result (or payment information) includes product name, product unique code, unit price, quantity, discount amount, coupon application, amount, total amount, payment date and time information, payment card type, etc.

Figure 3 is a block diagram showing the configuration of the user device 100 according to another embodiment of the present invention.

As shown in FIG. 3, the user device 10 includes a user device communication unit 110, a user device storage unit 120, a user device display unit 130, a user device voice output unit 140, and a user device control unit 150. ) and a camera unit 160. Not all of the components of the user device 10 shown in FIG. 3 are essential components, and the user device 10 may be implemented with more components than those shown in FIG. 3 or fewer components. The user device 10 may also be implemented by .

In order to receive information based on user input (or user selection), it includes a screen to receive a PIN number, a screen to receive a payment card number, a screen to receive a security card number, a screen to receive personal information, etc. When displaying a preset screen, the user device 100 acquires image information through the front camera 161 provided in the user device 100. Thereafter, the user device 100 performs an object recognition function on the acquired image information to recognize one or more objects included in the image information. Thereafter, when the recognized object is single and the characteristic information of the recognized object is similar to the characteristic information of a preset reference object, the user device 100 determines that it is in a normal state. In addition, when there are multiple recognized objects, or when the recognized object is single and the characteristic information of the recognized object is not similar to the characteristic information of a preset reference object, the user device 100 leaks information according to the user input. If it is determined that there is a risk of damage, a pre-set function is used to prevent information exposure due to user input on the screen.

The user device 100 shown in FIG. 2 includes a user device communication unit 110, a user device storage unit 120, a user device display unit 130, a user device voice output unit 140, and a user device. It consists of a control unit 150.

When displaying a preset screen on the user device display unit 130 to receive information according to user input (or user selection), the user device control unit 150 controls the front camera 161 to display the corresponding front camera 161. Obtain image information through . At this time, the image information acquired through the front camera 161 is the front image information of the front camera 161 (or when the user of the user device 100 is looking at the front camera 161, the user (or It may be surrounding and/or rear image information of the user including the user's face. Here, the preset screen is a PIN number reception screen for receiving a PIN number according to user input (or user selection) corresponding to the selected payment card after selecting a payment card, and payment card information according to user input (or user selection) ( or payment card number), a security card number reception screen to receive a security card number according to user input (or user selection), and personal information (e.g., personal information according to user input (or user selection) It includes a personal information reception screen to receive personal information (including name, date of birth, address, etc.).

Additionally, the user device control unit 150 may display image information acquired through the front camera 161 on one area of a preset screen displayed on the user device display unit 130.

Additionally, the user device control unit 150 performs an object recognition function on the acquired image information.

That is, the user device control unit 150 extracts (or calculates) feature information from the acquired image information.

Additionally, the user device control unit 150 recognizes (or identifies) one or more objects (or objects) included in the image information based on the extracted feature information.

Additionally, the user device control unit 150 determines (or confirms) whether a user other than a pre-registered user (or object) is included among one or more objects included in the recognized image information.

That is, the user device control unit 150 determines (or confirms) whether the image information includes one object (or whether the image information includes a plurality of objects).

As a result of the determination, if the image information includes a plurality of objects, the user device control unit 150 prevents information exposure according to user input (or user selection) on the preset screen through a preset function. Here, the preset functions include a function that notifies the user that another person's face has been detected, a function that dims the color of the keyboard for input displayed on the user device 100, and a function that changes the input keyboard screen in sentences to an input keyboard in units of one character. Includes a function to change the screen, a function to display the input keyboard screen in the opposite direction to the direction in which others in the video information are located, and a function to display guidance to change the direction of the user device 100 in a direction that can block the gaze of others. do.

In addition, the user device control unit 150 performs an image information acquisition process through the front camera 161 and an object recognition process within the image information until the reception of information according to the user input (or user selection) in the previous step is completed. , the user's authenticity judgment process for the recognized object is repeatedly performed.

Additionally, as a result of the determination, if the image information includes one object, the user device control unit 150 checks (or determines) whether the characteristic information of the recognized object is similar to the characteristic information of a preset reference object. )do. Here, the characteristic information of the preset reference object may be characteristic information related to the user of the user device 100.

That is, as a result of the determination, if the image information includes one object, the user device control unit 150 calculates the similarity between the characteristic information of the recognized object and the characteristic information of the preset reference object. Additionally, the user device control unit 150 checks whether the calculated similarity (or the similarity of feature information between two objects) is greater than or equal to a preset reference value.

As a result of the confirmation, if the characteristic information of the recognized object is similar to the characteristic information of the preset reference object (or if the similarity of the characteristic information between the two calculated objects is greater than or equal to the preset reference value), the user device control unit 150 is in a normal state. , the image information acquisition process through the front camera 161, the object recognition process within the image information, and the recognized object until the reception of information according to the user input (or user selection) in the previous step is completed. The process of determining user authenticity is repeatedly performed.

In this way, by checking the user's face in the image information and comparing it with the characteristic information of the pre-stored reference object, it is possible to check whether it is a pre-set reference object (or a pre-set user of the user device 100) without any user manipulation. there is.

In addition, as a result of confirmation, if the characteristic information of the recognized object is not similar to the characteristic information of the preset reference object (or if the similarity of the characteristic information between the two calculated objects is less than the preset reference value), the user device control unit ( 150) prevents information exposure according to user input (or user selection) on a preset screen through a preset function. Here, the preset functions include a function that notifies the user that another person's face has been detected, a function that dims the color of the keyboard for input displayed on the user device 100, and a function that changes the input keyboard screen in sentences to an input keyboard in units of one character. Includes a function to change the screen, a function to display the input keyboard screen in the opposite direction to the direction in which others in the video information are located, and a function to display guidance to change the direction of the user device 100 in a direction that can block the gaze of others. do.

In addition, the user device control unit 150 performs an image information acquisition process through the front camera 161 and an object recognition process within the image information until the reception of information according to the user input (or user selection) in the previous step is completed. , the user's authenticity judgment process for the recognized object is repeatedly performed.

In addition, since image information can be provided through the screen of another user device or the authentication process can be passed (or performed) through a photo, the user device control unit 150 basically changes the image information to perform facial movement or eye blinking. A method of determining whether it is a photo by checking whether there is a light, a method of determining whether a pixel or reflected light is detected in the acquired image information, and a method of determining whether the image information or video is input through another terminal, etc. This may apply.

In addition, since the user device control unit 150 can be replaced by another person after passing the authentication process, user authentication such as acquiring image information, recognizing objects in the image information, and determining the authenticity of the user for the recognized object The procedure is continuously repeated until information corresponding to the preset screen is received.

In addition, when information according to user input (or user selection) is received, the user device control unit 150 performs the process of acquiring image information through the front camera 161 (or acquiring image information, recognizing objects, and The process of determining the user's authenticity, etc.) is terminated (or the operation of the front camera 161 is stopped (or turned off)), and the received information, identification information of the user device 100, etc. is sent to the user device communication unit 110. ) is transmitted to the service providing device 300.

In addition, the user device control unit 150 controls the service provision device 300, the POS terminal 400, and By linking with the payment server 500, a payment function can be performed for one or more products to be paid.

The camera unit 160 includes a front camera 161, a rear camera 162, etc. configured on at least one side of the user device 100. Here, the front camera 161 is located on the side where the main screen is displayed. Additionally, when the user device display unit 130 is comprised of a plurality of user device display units 130, they are formed on the same surface as the main user device display unit 130 among the plurality of user device display units 130. Additionally, the rear camera 162 is formed in an opposite position to the front camera 161 formed in the user device 100.

Additionally, the camera unit 160 processes image frames such as still images or moving images obtained by an image sensor (camera module or camera) in video call mode, shooting mode, video conference mode, etc. In other words, the image data obtained by the image sensor is encoded/decoded according to the codec (CODEC) to meet each standard. The processed image frame may be displayed on the display unit 140 under the control of the user device control unit 150. As an example, the camera unit 160 captures an object (or subject) (user image or video conference participant) and outputs a video signal corresponding to the captured image (subject image).

In addition, the image frame processed by the camera unit 160 may be stored in the user device storage unit 120 or transmitted to an external terminal (including any video conference terminal included in the video conference system) through the user device communication unit 110. You can.

Additionally, the upper part of the camera unit 160 may further include a lens unit (not shown) for capturing enlarged images.

Additionally, the camera unit 160 captures image information under the control of the user device control unit 150. At this time, image information captured by the camera unit 160 may include an object.

That is, when a preset screen is displayed on the user device 100 in order to receive information according to user input (or user selection), the front camera 161 operates under the control of the user device control unit 150, and the image Obtain information. At this time, the image information acquired through the front camera 161 is the front image information of the front camera 161 (or when the user of the user device 100 is looking at the front camera 161, the user (or It may be surrounding and/or rear image information of the user including the user's face.

Additionally, when information according to user input (or user selection) is received on a preset screen, the front camera 161 stops operating (or turns off) under the control of the user device control unit 150.

Likewise, if you want to enter personal information or important information or a password on the payment screen, the front camera provided on the user device monitors the back of the user, which the user cannot see, to prevent someone from spying on it. When a person's face is detected, this can be notified to the user.

In addition, in this way, the user can be identified through the front camera on the payment screen, and basic identification of the user and confirmation of others can be performed simultaneously using the user's facial features.

Hereinafter, the control method of the payment system according to the present invention will be described in detail with reference to FIGS. 1 to 7.

Figure 4 is a flowchart showing a control method of a payment system according to an embodiment of the present invention.

First, when displaying a preset screen on the user device 100 to receive information according to user input (or user selection), the user device 100 uses the front camera 161 included in the user device 100. Obtain image information through . At this time, the image information acquired through the front camera 161 is the front image information of the front camera 161 (or when the user of the user device 100 is looking at the front camera 161, the user (or It may be surrounding and/or rear image information of the user including the user's face. Here, the preset screen is a PIN number reception screen for receiving a PIN number according to user input (or user selection) corresponding to the selected payment card after selecting a payment card, and payment card information according to user input (or user selection) ( or payment card number), a security card number reception screen to receive a security card number according to user input (or user selection), and personal information (e.g., personal information according to user input (or user selection) It includes a personal information reception screen to receive personal information (including name, date of birth, address, etc.).

For example, for simple payment, an AA payment card is selected from among a plurality of payment cards registered in the user device 100, and as shown in FIG. 5, a first PIN number according to user input corresponding to the selected AA payment card When displaying the first PIN number reception screen 510 to receive, the user device 100 allows the user holding the user device 100 to Acquire the included first image information.

As another example, for simple payment, a BB payment card is selected from among a plurality of payment cards registered in the user device 100, and a second PIN is used to receive a second PIN number according to the user input corresponding to the selected BB payment card. When displaying the number reception screen, the user device 100 displays information including the user holding the user device 100 and other users located behind the user through the front camera 161 included in the user device 100. Acquire second image information (S410).

Afterwards, the user device 100 performs an object recognition function on the acquired image information.

That is, the user device 100 extracts (or calculates) feature information from the acquired image information.

Additionally, the user device 100 recognizes (or identifies) one or more objects (or objects) included in the image information based on the extracted feature information.

As an example, the user device 100 extracts feature information from the acquired first image information, and possesses the first object (or the user device 100) included in the first image information based on the extracted feature information. Object corresponding to one user) is recognized.

As another example, the user device 100 extracts feature information from the acquired second image information and recognizes the first object and the second object included in the second image information based on the extracted feature information ( S420).

Thereafter, the user device 100 determines (or confirms) whether a user other than a pre-registered user (or object) is included among one or more objects included in the recognized image information.

That is, the user device 100 determines (or confirms) whether the image information includes one object (or whether the image information includes a plurality of objects).

As an example, the user device 100 determines whether there is only one object recognized in the first image information.

As another example, the user device 100 determines whether there is only one object recognized in the second image information (S430).

As a result of the determination, if the image information includes one object, the user device 100 checks (or determines) whether the characteristic information of the recognized object is similar to the characteristic information of a preset reference object. Here, the characteristic information of the preset reference object may be characteristic information related to the user of the user device 100.

That is, as a result of the determination, if the image information includes one object, the user device 100 calculates the similarity between the characteristic information of the recognized object and the characteristic information of the preset reference object. Additionally, the user device 100 checks whether the calculated similarity (or the similarity of feature information between two objects) is greater than or equal to a preset reference value.

For example, when the first object recognized in the first image information is in a single state, the user device 100 determines the first similarity (e.g. For example, 95%) is calculated. Additionally, the user device 100 checks whether the calculated first similarity (eg, 95%) is greater than or equal to a preset reference value (eg, 90%).

As another example, when the first object recognized within the first image information is in a single state, the user device 100 may determine a second similarity between the feature information of the recognized first object and the feature information of a preset reference object. For example, 85%) is calculated. Additionally, the user device 100 checks whether the calculated second similarity (eg, 85%) is greater than or equal to a preset reference value (eg, 90%) (S440).

As a result of confirmation, if the characteristic information of the recognized object is similar to the characteristic information of the preset reference object (or if the similarity of the characteristic information between the two calculated objects is greater than or equal to the preset reference value), the user device 100 is in a normal state. until the reception of information according to the user input (or user selection) in the previous step is completed, the image information acquisition process through the front camera 161, the object recognition process within the image information, and the recognized object The process of determining user authenticity is repeatedly performed.

For example, when the calculated first similarity (e.g., 95%) of feature information between two objects is greater than or equal to a preset reference value (e.g., 90%), the user device 100 determines that it is in a normal state and performs the previous step. It returns to the initial step (or step S410) of acquiring image information through the front camera 161 until reception of information according to the user input (or user selection) is completed (S450).

In addition, if the image information includes a plurality of objects as a result of the previous judgment, or if the characteristic information of the recognized object is not similar to the characteristic information of the preset reference object as a result of the previous confirmation (or the difference between the two calculated objects If the similarity of feature information is less than a preset reference value), the user device 100 prevents information exposure according to user input (or user selection) on a preset screen through a preset function. Here, the preset functions include a function that notifies the user that another person's face has been detected, a function that dims the color of the keyboard for input displayed on the user device 100, and a function that changes the input keyboard screen in sentences to an input keyboard in units of one character. Includes a function to change the screen, a function to display the input keyboard screen in the opposite direction to the direction in which others in the video information are located, and a function to display guidance to change the direction of the user device 100 in a direction that can block the gaze of others. do.

In addition, the user device 100 performs a process of acquiring image information through the front camera 161 until the reception of information according to the user input (or user selection) in the previous step is completed, a process of recognizing objects within the image information, The user's authenticity judgment process for the recognized object is repeatedly performed.

As an example, when the determination result (or the determination result in step S430) includes multiple objects recognized in the second image information (for example, including the first object and the second object), as shown in FIG. 6, The user device 100 displays information 610 indicating that another person's face has been detected and second image information 620 obtained through the front camera 161.

As another example, when the verification result (or the verification result in step S440), the second similarity (e.g., 85%) of the characteristic information between the two calculated objects is less than a preset reference value (e.g., 90%), As shown in Figure 7, the user device 100 includes information 710 that induces a change in direction of the user device 100 in a direction that can block the gaze of others, and a second image acquired through the front camera 161. Information 720 is displayed (S460).

Afterwards, when information according to user input (or user selection) is received, the user device 100 performs the process of acquiring image information through the front camera 161 (or acquiring image information, recognizing objects, and The process of determining user authenticity, etc. is terminated (or the operation of the front camera 161 is stopped (or turned off)), and the received information, identification information of the user device 100, etc. is stored in the service providing device 300. send to

As an example, when receiving the first PIN number (e.g., 123456) corresponding to the AA payment card according to the user input, the user device 100 stops the operation of the front camera 161 and Information, the first PIN number corresponding to the AA payment card, identification information of the user device 100, etc. are transmitted to the service providing device 300 (S470).

The user device, service provision device, POS terminal, and payment system including the same according to an embodiment of the present invention can be written as a computer program, and the codes and code segments constituting the computer program can be easily written by a computer programmer in the field. can be inferred. In addition, the computer program is stored in computer readable media, and is read by a computer, user device, beacon, service provision device, POS terminal, payment server, etc. according to embodiments of the present invention. By being executed, a user device, a service provision device, a POS terminal, and a payment system including the same can be implemented.

Information storage media include magnetic recording media, optical recording media, and carrier wave media. The computer program implementing the user device, service providing device, POS terminal, and payment system including the same according to an embodiment of the present invention is stored and installed in the internal memory of the user device, beacon, service providing device, POS terminal, payment server, etc. It can be. Alternatively, an external memory such as a user device, a service providing device, a POS terminal, and a smart card storing and installing a computer program implementing a payment system including the user device according to an embodiment of the present invention may be connected to the user device, a beacon, or a service through an interface. It may be installed in provision devices, POS terminals, payment servers, etc.

As described above, an embodiment of the present invention allows the user to enter important information or a password on the personal information or payment screen through the front camera provided in the user device to prevent a situation where someone is peeping. By monitoring the back of the user, which cannot be seen, and notifying the user when another person's face is detected, it can increase security and reduce the load on the user device when using services such as payment or banking using a mobile user device.

In addition, as described above, an embodiment of the present invention verifies the user through the front camera on the payment screen and simultaneously performs basic identification of the user and confirmation of others using the user's facial feature points, thereby performing basic user authentication and Preventing others from spying can be done at the same time without any additional user intervention.

The above-described content can be modified and modified by anyone skilled in the art without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

The present invention monitors the back of the user, which cannot be seen by the user, through the front camera provided in the user device to prevent someone from peeping when entering personal information or important information or a password on the payment screen. When a person's face is detected, it notifies the user, increasing security when using services such as payment or banking using a mobile user device and reducing the load on the user device. In the field of simple payment, advertising, and service provision devices. It can be widely used in fields such as user device field, POS terminal field, payment server field, security field, etc.

10: Payment system 100: User device
200: Beacon 300: Service provision device
400: POS terminal 500: Payment server
110: user device communication unit 120: user device storage unit
130: User device display unit 140: User device audio output unit
150: user device control unit 160: camera unit
161: Front camera 162: Rear camera

Claims (10)

a user device display unit that displays a preset screen to receive information according to user input;
a front camera that acquires image information when the preset screen is displayed on the user device display; and
Performs an object recognition function on the acquired image information to recognize one or more objects included in the image information, and determines whether information is leaked during the information reception process on the preset screen based on the characteristics of the recognized one or more objects. It includes a user device control unit that checks,
When the recognized object is one, the user device control unit calculates the similarity between the characteristic information of the recognized object and the characteristic information of a preset reference object, and if the calculated similarity is less than the preset reference value, the preset function is performed. Prevents exposure of information according to user input on a preset screen displayed on the display unit of the user device through, and when there are multiple recognized objects, on a preset screen displayed on the display unit of the user device through the preset function Prevent information exposure based on user input,
The preset functions include a function that notifies the user that another person's face has been detected, a function that dims the color of the keyboard for input displayed on the display of the user device, and a function that changes the input keyboard screen in sentences to an input keyboard screen in units of one character. It includes at least one of the following functions: a function to change, a function to display the input keyboard screen in the direction opposite to the direction in which other people in the video information are located, and a function to display guidance to change the direction of the user device in a direction that can block the gaze of others,
If there are multiple recognized objects, the user device control unit has a function to notify the user that another person's face has been detected, a function to display the input keyboard screen in the opposite direction to the direction in which the other person in the image information is located, and a function to block the other person's gaze. Preventing information exposure due to user input on a preset screen displayed on the user device display unit through a function that displays a direction change guidance of the user device in the direction in which it is located,
The user device control unit determines whether the acquired image information is a photograph by checking whether there is facial movement or eye blinking through changes in the acquired image information, and detects a pixel or reflected light in the acquired image information. A user device characterized in that it determines whether the acquired image information is image information input through another terminal by determining whether the image information is detected. delete delete delete delete delete delete Displaying a preset screen to receive information according to user input through a user device display unit;
When the preset screen is displayed on the display of the user device through a front camera, acquiring surrounding and rear image information of the user while the user of the user device is looking at the front camera;
Recognizing one or more objects included in the image information by performing an object recognition function on the acquired image information through a user device control unit;
determining, through the user device control unit, whether an object is included in the recognized image information;
Checking, through the user device control unit, whether characteristic information of the recognized object is similar to characteristic information of a preset reference object when the recognized image information includes one object; and
Through the user device control unit, if the recognized image information includes a plurality of objects as a result of the determination, exposure of information according to user input on the preset screen displayed on the user device display unit is prevented through a preset function. And, as a result of the confirmation, if the characteristic information of the recognized object is not similar to the characteristic information of the preset reference object, information is exposed according to the user input on the preset screen displayed on the display unit of the user device through the preset function. Including steps to prevent,
The preset functions include a function that notifies the user that another person's face has been detected, a function that dims the color of the keyboard for input displayed on the display of the user device, and a function that changes the input keyboard screen in sentences to an input keyboard screen in units of one character. At least one of the following functions: a function to change, a function to display an input keyboard screen in the direction opposite to the direction in which other people in the video information are located, and a function to display a change in direction of the user device in a direction that can block the gaze of others,
The steps to prevent exposure of the above information are:
Through the user device control unit, if the recognized image information includes a plurality of objects as a result of the determination, a function is provided to inform the user that another person's face has been detected, and an input keyboard screen is displayed in the opposite direction to the direction in which the other person in the image information is located. It is characterized by preventing exposure of information due to user input on a preset screen displayed on the display unit of the user device through a function of displaying and a function of displaying a direction change of the user device in a direction that can block the gaze of others. and
The step of recognizing one or more objects includes:
Through the user device control unit, it is determined whether the acquired image information is a photograph by checking whether there is facial movement or eye blinking through changes in the acquired image information, and a pixel is detected in the acquired image information. A method for controlling a user device, further comprising determining whether the acquired image information is image information input through another terminal by determining whether reflected light is detected. delete delete

Images