CN112613475A - Code scanning interface display method and device, mobile terminal and storage medium - Google Patents
Code scanning interface display method and device, mobile terminal and storage medium Download PDFInfo
- Publication number
- CN112613475A CN112613475A CN202011633743.4A CN202011633743A CN112613475A CN 112613475 A CN112613475 A CN 112613475A CN 202011633743 A CN202011633743 A CN 202011633743A CN 112613475 A CN112613475 A CN 112613475A
- Authority
- CN
- China
- Prior art keywords
- mobile terminal
- code scanning
- payment
- code
- interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000012549 training Methods 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 9
- 238000012790 confirmation Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 abstract description 12
- 230000001133 acceleration Effects 0.000 description 11
- 238000006073 displacement reaction Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000012546 transfer Methods 0.000 description 7
- 238000012795 verification Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000001939 inductive effect Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
- 239000010409 thin film Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B21/00—Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01P—MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
- G01P15/00—Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration
- G01P15/14—Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration by making use of gyroscopes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01P—MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
- G01P15/00—Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration
- G01P15/18—Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration in two or more dimensions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Abstract
The embodiment of the application provides a code scanning interface display method and device, a mobile terminal and a storage medium, and relates to the technical field of mobile payment. The method comprises the following steps: acquiring mobile pose information of the mobile terminal; identifying the mobile pose information to obtain an identification result, wherein the identification result is used for indicating whether the mobile terminal is in a code scanning preposed state or not, and the code scanning preposed state is a preposed state when a code scanning interface is displayed; responding to the identification result to indicate that the mobile terminal is in a code scanning preposed state, and acquiring a target image, wherein the target image is used for indicating payment based on a graphic code provided by a service provider; and displaying a code scanning interface corresponding to the graphic code provided by the service provider. According to the method and the device, the code scanning interface corresponding to the graphic code is automatically displayed through the position and the posture based on the mobile terminal, complex interface touch operation is not needed, operation required by a code scanning interface opening process is simplified, and interface opening efficiency is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of mobile payment, in particular to a code scanning interface display method and device, a mobile terminal and a storage medium.
Background
With the development of terminal technology, a user can pay the amount of money corresponding to an article by using a mobile phone.
In the related art, a user may purchase goods by way of code-scanning payment. A user needs to find an icon of an application program corresponding to the two-dimensional code to enable the application program to display a home page; then, the user needs to find an entrance for scanning the two-dimensional code in the home page; and scanning the two-dimensional code through the entrance so as to complete a code scanning payment process.
Disclosure of Invention
The embodiment of the application provides a code scanning interface display method and device, a mobile terminal and a storage medium. The technical scheme is as follows:
in one aspect, an embodiment of the present application provides a code scanning interface display method, where the method includes:
acquiring mobile pose information of a mobile terminal, wherein the mobile pose information is used for indicating the position and the posture of the mobile terminal in a moving state;
identifying the mobile pose information to obtain an identification result, wherein the identification result is used for indicating whether the mobile terminal is in a code scanning preposed state, and the code scanning preposed state is a preposed state when a code scanning interface is displayed;
responding to the identification result to indicate that the mobile terminal is in the code scanning preposed state, and acquiring a target image, wherein the target image is used for indicating payment based on a graphic code provided by a service provider;
and displaying a code scanning interface corresponding to the graphic code provided by the service provider.
On the other hand, an embodiment of the present application provides a code scanning interface display device, the device includes:
the mobile terminal comprises an information acquisition module, a processing module and a display module, wherein the information acquisition module is used for acquiring the mobile pose information of the mobile terminal, and the mobile pose information is used for indicating the position and the posture of the mobile terminal in a moving state;
the information identification module is used for identifying the mobile pose information to obtain an identification result, wherein the identification result is used for indicating whether the mobile terminal is in a code scanning preposed state, and the code scanning preposed state is a preposed state when a code scanning interface is displayed;
the image acquisition module is used for responding to the identification result and indicating that the mobile terminal is in the code scanning preposed state, and acquiring a target image, wherein the target image is used for indicating payment based on a graphic code provided by a service provider;
and the interface display module is used for displaying a code scanning interface corresponding to the graphic code provided by the service provider.
In another aspect, an embodiment of the present application provides a mobile terminal, where the mobile terminal includes a processor and a memory, where the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the code scanning interface display method according to the above aspect.
In still another aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored, and the computer program is loaded and executed by a processor to implement the code scanning interface display method according to the above aspect.
In yet another aspect, embodiments of the present application provide a computer program product including computer instructions stored in a computer-readable storage medium. The processor of the mobile terminal reads the computer instruction from the computer readable storage medium, and executes the computer instruction, so that the mobile terminal executes the code scanning interface display method.
The technical scheme provided by the embodiment of the application can bring the following beneficial effects:
determining whether the mobile terminal is in a code scanning preposed state or not based on the mobile pose information of the mobile terminal; and when the mobile terminal is determined to be in the code scanning preposed state, acquiring an image for indicating the graphic code provided by the service provider, and displaying a code scanning interface corresponding to the graphic code. According to the method and the device, the code scanning interface corresponding to the graphic code is automatically displayed through the position and the posture based on the mobile terminal, complex interface touch operation is not needed, operation required by a code scanning interface opening process is simplified, and interface opening efficiency is improved.
Drawings
FIG. 1 is a flowchart of a code scanning interface display method according to an embodiment of the present application;
FIG. 2 is a flowchart of a code scanning interface display method according to another embodiment of the present application;
FIG. 3 is a schematic diagram of recognition model training provided by one embodiment of the present application;
FIG. 4 is a schematic illustration of a payment process provided by one embodiment of the present application;
FIG. 5 is a flowchart of a code scanning interface display method according to another embodiment of the present application;
FIG. 6 is a schematic diagram of a code scanning interface display flow according to an embodiment of the present application;
FIG. 7 is a block diagram of a code scanning interface display device according to an embodiment of the present application;
fig. 8 is a block diagram of a mobile terminal according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
In the method provided by the embodiment of the application, the execution subject of each step may be a mobile terminal. In a possible implementation, the mobile terminal is an electronic device such as a cell phone, a tablet, a smart wearable device, a laptop portable computer, and the like.
Exemplarily, the mobile terminal in the embodiment of the present application includes a display screen. Optionally, the display screen may be a Thin Film Transistor (TFT) liquid crystal display screen, where the TFT liquid crystal display screen refers to that each liquid crystal pixel on the liquid crystal display screen is driven by a TFT integrated behind the TFT, so that screen information can be displayed at high speed, high brightness, and high contrast. Alternatively, the display screen may be a Super Twisted Nematic (STN) screen, and the STN screen has advantages of low power consumption and power saving. Optionally, the Display screen may also be an OLED (Organic Light-Emitting Display) Display screen, and the OLED Display screen has the characteristics of being thinner and lighter, Emitting Light actively, having no viewing angle problem, being high in definition, high in brightness, fast in response speed, low in energy consumption, wide in use temperature range, strong in shock resistance, low in cost, capable of realizing flexible Display, and the like. The display screen of the terminal may only have a display function, and certainly, in other possible implementation manners, the display screen of the terminal may also have both a display function and a touch function, which is not limited in this embodiment of the application.
For example, the mobile terminal in the embodiment of the present application further includes an image capturing device, where the image capturing device may be a camera, and may also be other devices, which is not limited in the embodiment of the present application. For example, the mobile terminal is taken as a mobile phone to describe, the image capturing device may be disposed on a front panel of the mobile terminal, a back panel of the mobile terminal, or both the front panel and the back panel of the mobile terminal, which is not limited in this embodiment of the present application.
The present application is described below with reference to several embodiments.
Referring to fig. 1, a flowchart of a code scanning interface display method according to an embodiment of the present application is shown. The method can be applied to a mobile terminal, and the method can comprise the following steps.
In the embodiment of the present application, the movement pose information is used to indicate the position and posture of the mobile terminal in the moving state.
In a possible implementation manner, the mobile pose information of the mobile terminal is acquired through a sensor. Illustratively, when the mobile terminal is in a working state, sensor data are continuously collected through a sensor and taken as the mobile terminal movement pose information.
In possible implementations, the sensor data may be collected by a single sensor or may be collected by a plurality of different types of sensors.
In one example, the mobile terminal may acquire data in two dimensions of acceleration and displacement through a gyroscope sensor to obtain acceleration data and displacement data, where the acceleration data includes accelerations in three directions (x axis, y axis, and z axis), the displacement data includes displacements in three directions (x axis, y axis, and z axis), and the acceleration data and the displacement data are determined as the mobile pose information of the mobile terminal by the mobile terminal.
In another example, the mobile terminal may acquire acceleration data and displacement data through an acceleration Sensor (Accelerometer Sensor, Sensor) and a Gravity Sensor (gssensor), where the acceleration data includes accelerations in three directions of three axes (x axis, y axis, and z axis), and the displacement data includes displacements in three directions of three axes (x axis, y axis, and z axis), and the mobile terminal determines the acceleration data and the displacement data as the mobile pose information of the mobile terminal.
In a possible implementation manner, the mobile terminal acquires sensor data of preset times through a sensor, and determines the sensor data of the preset times as the mobile pose information of the mobile terminal, wherein the preset times can be 30-200, and if the preset times are too few, the acquisition time of the mobile terminal is too short, which may cause the finally obtained identification result to be inaccurate; if the preset times are too long, the mobile terminal acquisition time is too long, which may cause the user to hold the mobile terminal for a long time to perform resource transfer, so that the resource transfer efficiency is low. For example, the mobile terminal may collect 128 times of sensor data (displacement and acceleration in three directions of x, y and z), and splice the 128 collected sensor data into data in 128 × 6 dimensions (mobile pose information of the mobile terminal).
And 102, identifying the mobile pose information to obtain an identification result, wherein the identification result is used for indicating whether the mobile terminal is in a code scanning preposition state or not.
The code scanning preposed state is the preposed state when the code scanning interface is displayed. In a possible implementation manner, when a user performs a code scanning payment action, the mobile terminal acquires the mobile pose information, and determines that the mobile terminal is in a code scanning pre-set state based on the mobile pose information, for example, the code scanning pre-set state may also be referred to as a state to be paid.
And 103, responding to the identification result for indicating that the mobile terminal is in the code scanning preposition state, and acquiring a target image, wherein the target image is used for indicating payment based on the graphic code provided by the service provider.
A service provider refers to an object that provides a service, and illustratively, the service provider may provide at least one of the following services: gourmet, hotel, play, movie, bus, leisure, etc.
The graphic code provided by the service provider may be any one, for example, the graphic code provided by the service provider may be a barcode, a two-dimensional code, or other forms of graphic codes, and the display form of the graphic code is not limited in the embodiment of the present application.
In a possible implementation manner, under the condition that the identification result is used for indicating that the mobile terminal is in the code scanning preposition state, the target image is acquired through the image acquisition device. Illustratively, the mobile terminal acquires the target image in the background, and the user does not perceive the operation. In a possible implementation, the image acquisition device may be an AON (Always on) camera, and the AON camera has a smaller power consumption.
In a possible implementation, the target image may include one or more images, and the number of images included in the target image may be determined by the acquisition frequency of the image acquisition device. The target image may be an image acquired by the image acquisition device within one second, or may be an image acquired by the image acquisition device within a preset time, where the preset time may be a default time or a time set by a user.
In a possible implementation manner, image acquisition devices are arranged on a front panel and a back panel of the mobile terminal, and the mobile terminal can determine which image acquisition device is used for acquiring a target image based on the current posture. The mobile terminal can determine whether to adopt the image acquisition equipment on the front panel to acquire the target image or adopt the image acquisition equipment on the back panel to acquire the target image based on the current posture of the user holding the mobile terminal and the current posture of the mobile terminal. In a possible implementation manner, the mobile terminal may simultaneously adopt the image acquisition device on the front panel and the image acquisition device on the back panel to acquire the target image.
And 104, displaying a code scanning interface corresponding to the graphic code provided by the service provider.
In an embodiment of the present application, the code scanning interface is used for scanning a graphic code provided by a service provider.
The code scanning interfaces corresponding to different graphic codes may be different, and the terminal needs to display the code scanning interface corresponding to the graphic code to successfully scan and identify the graphic code.
In a possible implementation, the method starts again from step 101 in response to the recognition result indicating that the mobile terminal is not in the scan code pre-state.
It should be noted that the payment may also be referred to as resource transfer, and of course, in other possible implementation manners, the payment may also have other names, which is not limited in this embodiment of the present application.
In summary, in the technical scheme provided by the embodiment of the application, whether the mobile terminal is in the code scanning preposition state is determined based on the mobile pose information of the mobile terminal; and when the mobile terminal is determined to be in the code scanning preposed state, acquiring an image for indicating the graphic code provided by the service provider, and displaying a code scanning interface corresponding to the graphic code. According to the method and the device, the code scanning interface corresponding to the graphic code is automatically displayed through the position and the posture based on the mobile terminal, complex interface touch operation is not needed, operation required by a code scanning interface opening process is simplified, and interface opening efficiency is improved.
Please refer to fig. 2, which shows a flowchart of a code scanning interface display method according to another embodiment of the present application. The method can be applied to a mobile terminal, and the method can comprise the following steps.
In the embodiment of the present application, the movement pose information is used to indicate the position and posture of the mobile terminal in the moving state.
For the description of step 201, reference may be made to the above embodiments, which are not repeated herein.
The identification model is used for identifying whether the mobile terminal is in a code scanning preposition state. Illustratively, the recognition model may be a CNN (Convolutional Neural Network) model.
The recognition model can be set in a SensorHub (intelligent sensing hub), which is used as a software and hardware solution based on a low power consumption MCU (Micro Control Unit) and a lightweight RTOS (Real-Time Operating System) for connecting and processing sensor data collected by various sensors. In a possible implementation manner, an MCU is integrated on an SOC (System on Chip) of the terminal, wherein an operating System is run on the SOC, an RTOS is run on the MCU, and the MCU performs motion recognition on sensor data (i.e., mobile pose information of the mobile terminal) acquired by the sensor.
Illustratively, the recognition model in the embodiment of the present application can be adapted to the sequence data, and the recognition model in the embodiment of the present application is quantized to have a size of less than 20kb, and the quantized recognition model is stored in the SensorHub.
In a possible implementation manner, before the mobile terminal performs recognition processing on the mobile pose information through the recognition model to obtain a recognition result, the recognition model needs to be trained, and the training process of the recognition model may include the following steps:
1. acquiring training data, wherein the training data comprises at least one piece of training moving pose information and a standard identification result corresponding to the training moving pose information;
2. identifying the training movement pose information through an identification model to obtain a prediction identification result;
3. and training the recognition model based on the predicted recognition result and the standard recognition result to obtain the trained recognition model.
In a possible implementation mode, training the initial neural network model by taking the training mobile pose information of the mobile terminal in the code scanning preposed state as a positive sample and the training mobile pose information of the mobile terminal not in the code scanning preposed state as a negative sample, and finally training to obtain the identification model meeting the identification accuracy requirement.
Illustratively, as shown in fig. 3, a schematic diagram of recognition model training provided by an embodiment of the present application is shown. The method comprises the steps of supposing that 128-dimension data formed by splicing 128 times of sensor data acquired by sensors is used as training moving pose information; marking the training mobile pose information (either manually or by a marking model) to obtain a standard identification result; identifying the training mobile pose information through a neural network model (identification model) to obtain a prediction identification result; and obtaining the trained recognition model in response to the fact that the error between the predicted recognition result and the standard recognition result is smaller than the threshold value.
And step 203, responding to the identification result for indicating that the mobile terminal is in the code scanning preposition state, and acquiring an image through a rear camera of the mobile terminal to obtain an acquired image.
Under the condition that the mobile terminal is in the code scanning preposition state, the mobile terminal performs image acquisition through a rear camera, illustratively, the rear camera can be an AON rear camera, and the power consumption is low.
In a possible implementation manner, in response to the recognition result indicating that the mobile terminal is in the code scanning preposition state, image acquisition is performed through a preposed camera of the mobile terminal to obtain an acquired image.
In a possible implementation, the mobile terminal determines whether to employ a front-facing camera or a rear-facing camera for image acquisition based on the current pose.
In a possible implementation manner, an image acquisition device may also be disposed on the middle frame of the mobile terminal, and the mobile terminal may perform image acquisition through the image acquisition device disposed on the middle frame.
And step 204, responding to the acquired image including the graphic code, and acquiring a target image including the graphic code from the acquired image.
In an embodiment of the application, the target image is used to instruct payment to be made based on a graphic code provided by a service provider.
In a possible implementation mode, the mobile terminal performs image recognition on the acquired image, determines whether the acquired image includes a graphic code, and acquires a target image from the acquired image in response to the acquired image including the graphic code.
In a possible implementation manner, after the mobile terminal performs image acquisition to obtain an acquired image, the acquired image is identified to find that no graphic code provided by the service provider exists, the terminal can determine whether the acquired image includes a code scanning device, if the acquired image does not include the code scanning device and the graphic code provided by the service provider, misidentification may occur at this time, and the mobile terminal can start execution from the step of acquiring the mobile pose information of the mobile terminal again.
After the mobile terminal is identified to be in the code scanning preposition state, the image recognition technology is adopted to detect the graphic code, which is beneficial to reducing the wrong calling probability of the code scanning interface and improving the actual display accuracy and display safety of the code scanning interface.
In a possible implementation mode, the mobile terminal cuts a graphic code area from a target image; and determining the image characteristics of the graphic code area. The graphic codes corresponding to different payment programs are different, so that the image characteristics of the graphic code areas corresponding to different payment programs are different. The image feature of the target image is used for indicating the identification information of the payment program, and the mobile terminal determines the payment program for scanning the graphic code provided by the service provider based on the identification information of the payment program.
In a possible implementation manner, if the mobile terminal cannot determine the payment program corresponding to the graphical code provided by the service provider, the mobile terminal may determine the payment program corresponding to the graphical code as a default payment program, where the default payment program may be set by the user, or may be determined by the mobile terminal based on the usage frequency, which is not limited in this embodiment of the present application. And if code scanning errors occur after the code scanning is carried out by using the default payment program, automatically switching to a code scanning interface corresponding to another payment program.
In a possible implementation manner, a graphic code recognition model may be stored in the mobile terminal in advance, and the input of the graphic code recognition model is a target image and the output of the graphic code recognition model is a payment program corresponding to a graphic code in the target image. Illustratively, for the code scanning device detection model obtained through training, before model deployment, the model can be processed in modes of network cutting, network optimization, model quantification and the like, so that the model is further simplified, the model reasoning speed is improved, and the real-time detection requirement of a mobile terminal is met.
And step 206, displaying a code scanning interface corresponding to the payment program.
The code scanning interface is the interface corresponding to the payment program determined in step 205.
And step 207, displaying a payment interface, wherein the payment interface comprises an input box of payment amount and a payment confirmation control.
And responding to the code scanning interface to successfully scan and identify the graphic code provided by the service provider, and skipping to display the payment interface by the mobile terminal.
The payment amount input box refers to a control used for inputting the payment amount, and the payment confirmation control refers to a control used for confirming payment to the service provider.
In step 208, the amount entered in the input box of the payment amount is obtained.
In a possible implementation manner, the user may input the amount in the input box of the payment amount through voice, touch, gesture, or the like.
The method comprises the following steps of taking the example that a user inputs the amount in an input box of the payment amount in a voice mode as an introduction description, and receiving a voice instruction aiming at the input box of the payment amount by a mobile terminal; the amount of money is determined from the voice command.
In a possible implementation manner, the terminal needs to perform voiceprint recognition on the voice command, determine a voiceprint of a registered user (the registered user may be a registered user of the payment application) or an owner of the mobile terminal based on the voiceprint recognition, and determine the amount of money based on the voiceprint of the registered user or the owner. In a possible implementation manner, after determining the information of the registered user or the owner through voiceprint recognition, the mobile terminal may provide the amount of the payment amount to the service provider without receiving a selection instruction for confirming the payment control, that is, without performing the following step 209.
As shown in fig. 4, a user turns over a mobile terminal 401, the mobile terminal 401 acquires mobile pose information, determines that the mobile terminal is in a code scanning preposed state based on the mobile pose information, the mobile terminal acquires a target image, the target image is used for indicating payment based on a graphic code 402 provided by a service provider, the mobile terminal displays a code scanning interface 403, when the mobile terminal 401 successfully scans and identifies the graphic code 402 provided by the service provider through the code scanning interface 403, the mobile terminal 401 displays a payment interface 404, the mobile terminal 401 collects a voiceprint of a host to determine the amount of money, and pays the amount of money to the service provider.
According to the payment method and device, the payment program can be automatically switched based on the graphic code provided by the service provider, a more convenient payment mode is provided, and higher convenience and safety are brought to life of people. According to the embodiment of the application, the mobile terminal is triggered to pay by detecting the gesture or posture of the user holding the mobile terminal, the rear camera is triggered to photograph the identification graphic code through motion detection, the corresponding payment program is automatically opened according to the identified graphic code, the amount of money is identified according to the voiceprint of a user owner after unlocking, the mobile terminal is enabled to automatically input the specific amount of money to be paid, and when the user inconveniently inputs the amount of money through touch control, the convenience of payment of the user is improved.
In response to receiving the selection instruction of the confirmation payment control, the amount of the amount is paid to the service provider, step 209.
In a possible implementation manner, the selection instruction for confirming the payment control can be triggered by voice, gesture, touch and the like.
In a possible implementation manner, after the mobile terminal successfully pays the amount of the amount to the service provider (that is, the mobile terminal successfully transfers the exchange resource from the first resource account to the second resource account, where the first resource account is a resource account corresponding to a user of the mobile terminal, and the second resource account is a resource account corresponding to the service provider), the mobile terminal resumes to display an original user interface, where the original user interface is a user interface displayed by the mobile terminal before the mobile terminal is in the code scanning pre-state. For example, the original user interface may be a screen-turning interface, an unlocked desktop, a user interface of a previous application, and the like, and the original user interface is not limited in this embodiment of the application.
In a possible implementation manner, when the payment program starts the privacy-exempt payment, the payment interface only includes an input box of the payment amount, the amount of the amount input in the input box of the payment amount is acquired, and the mobile terminal pays the amount of the amount to the service provider.
Through voice input amount and voice confirmation payment, the embodiment of the application provides a non-inductive and non-contact convenient payment mode, and user experience is improved.
It should be noted that the amount of money may also be referred to as a virtual resource, and the amount of money may also be referred to as a virtual resource, of course, in other possible implementations, the amount of money and the amount of money may also be called by other terms, which is not limited in this embodiment of the present application.
In the embodiment of the application, a user only needs to execute the code scanning preposition action and use the mobile terminal to shoot the image of the graphic code, and the mobile terminal can automatically display the code scanning interface without performing complicated interface touch operation. And moreover, the code scanning prepositive action is taken as a prepositive condition, the mobile terminal does not need to open a camera for long time to acquire images, and the whole power consumption of the terminal is reduced when a code scanning interface is automatically displayed.
According to the method and the device, whether the gesture action of the user is to open the recognition graph code of the payment program is detected through the sensor, when the recognition result is to open the recognition graph code of the payment program, the payment program corresponding to the graph code is automatically opened to scan the code, the inconvenience caused by the fact that the user needs to switch when opening a wrong application program is avoided, and user experience is improved. According to the embodiment of the application, the rear camera shoots an image of an environment opposite to the rear camera of the mobile terminal, and whether resource transfer is carried out based on the graphic code is determined, so that the error identification based on the mobile pose information for the first time is avoided, the identification accuracy and success rate of a payment scene are ensured, and a non-inductive and non-contact convenient payment mode is provided for a user.
In summary, in the technical scheme provided by the embodiment of the application, the code scanning interface is automatically displayed, so that the payment operation process is simplified, and the payment efficiency is improved.
According to the method and the device, after the mobile terminal is determined to be in the code scanning preposed state, the payment program corresponding to the graphic code provided by the service provider is determined through the image recognition technology, and then the code scanning interface corresponding to the payment program is displayed, so that the probability of successful payment is improved.
Please refer to fig. 5, which shows a flowchart of a code scanning interface display method according to another embodiment of the present application. The method can be applied to a mobile terminal, and the method can comprise the following steps.
And 502, identifying the mobile pose information to obtain an identification result, wherein the identification result is used for indicating whether the mobile terminal is in a code scanning preposed state, and the code scanning preposed state is a preposed state when a code scanning interface is displayed.
And step 503, in response to the recognition result indicating that the mobile terminal is in the code scanning preposition state, acquiring a target image, wherein the target image is used for indicating payment based on the graphic code provided by the service provider.
For the description of steps 501 to 503, reference may be made to the above embodiments, which are not described herein again.
And step 504, jumping from the original user interface to a code scanning interface under the condition that the mobile terminal is in an unlocked state.
The unlocking state refers to a state that the mobile terminal is in normal operation, and the unlocking state and the screen locking state are two different states.
The original user interface refers to a user interface displayed by the mobile terminal before the mobile terminal is in the code scanning preposition state, that is, a user interface which is watched by a user before the user performs code scanning payment operation.
And under the condition that the mobile terminal is in a code scanning preposed state and the target image is used for indicating payment based on the graphic code provided by the service provider, the mobile terminal confirms that the payment is required to be carried out through the code scanning interface. If the mobile terminal is in the unlocking state, the mobile terminal can directly jump from the original user interface to the code scanning interface without executing unlocking operation.
The authentication may include any one of the following: password verification, voice verification, face verification, iris verification and fingerprint verification. Illustratively, the authentication information may include any one of: password, voice, face, iris, fingerprint.
It should be noted that step 504 and step 505 are not executed simultaneously, and the mobile terminal may determine whether the mobile terminal is in the unlocked state; when the mobile terminal is in the unlocked state, executing step 504; when the mobile terminal is in the lock screen state, the process starts in step 505.
The preset authentication information is information which is stored in the mobile terminal by the user in advance and is used for authenticating the identity of the user. The preset authentication information is consistent with the type of the authentication information input by the user, that is, when the authentication information input by the user is a password, the preset authentication information is the password; when the authentication information input by the user is a sound, the preset authentication information is a sound; when the identity authentication information input by the user is a face, the preset identity authentication information is the face; when the identity authentication information input by the user is the iris, the preset identity authentication information is the iris; when the authentication information input by the user is a fingerprint, the preset authentication information is a fingerprint.
Taking authentication as password authentication as an example, the mobile terminal displays a password input interface, receives a password input by a user in the password input interface, and determines that the authentication is passed when the input password is consistent with the set password.
In a possible implementation manner, a plurality of preset authentication information are prestored in the mobile terminal, and when the authentication information input by the user is consistent with one of the plurality of preset authentication information, the authentication is confirmed to be passed.
The screen locking interface is an interface displayed when the mobile terminal is not unlocked and is an interface displayed when the mobile terminal is in a screen locking state. And when the mobile terminal determines that the authentication information input by the user is consistent with the preset authentication information, the mobile terminal is successfully unlocked, and the mobile terminal jumps from the screen locking interface to the code scanning interface.
In a possible implementation manner, when the authentication information input by the user is inconsistent with the preset authentication information, the mobile terminal displays a prompt message for prompting the user to re-input the authentication information.
For an example, the mobile terminal is in a screen-locked state, as shown in fig. 6, which is a schematic diagram illustrating a display flow of a code scanning interface. The mobile terminal displays a screen locking interface 601, and a user moves the mobile terminal to enable the mobile terminal to be in a code scanning preposition state; the method comprises the steps that a mobile terminal obtains a target image, wherein the target image is used for indicating a resource transfer mode provided by a service provider to be resource transfer based on a graphic code; the mobile terminal receives the authentication information input by the user, and in response to the fact that the authentication information input by the user is consistent with the preset authentication information, the mobile terminal jumps from the screen locking interface 601 to the code scanning interface 602. And the background of the mobile terminal simulates and opens the payment program corresponding to the graphic code, the payment program enters a home page of the payment program, and then the opening instruction of the code scanning interface is simulated and received from the home page of the payment program, so that the terminal screen displays the code scanning interface, and the operation is simple.
The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.
Referring to fig. 7, a block diagram of a code scanning interface display device provided in an embodiment of the present application is shown, where the device has a function of implementing the above method example, and the function may be implemented by hardware or by hardware executing corresponding software. The apparatus 700 may include:
an information obtaining module 710, configured to obtain movement pose information of a mobile terminal, where the movement pose information is used to indicate a position and a posture of the mobile terminal in a moving state;
the information identification module 720 is configured to perform identification processing on the mobile pose information to obtain an identification result, where the identification result is used to indicate whether the mobile terminal is in a code scanning pre-set state, where the code scanning pre-set state is a pre-set state when a code scanning interface is displayed;
an image obtaining module 730, configured to, in response to the recognition result indicating that the mobile terminal is in the code scanning preposition state, obtain a target image, where the target image is used to indicate that payment is performed based on a graphic code provided by a service provider;
and the interface display module 740 is configured to display a code scanning interface corresponding to the graphic code provided by the service provider.
In summary, in the technical scheme provided by the embodiment of the application, whether the mobile terminal is in the code scanning preposition state is determined based on the mobile pose information of the mobile terminal; and when the mobile terminal is determined to be in the code scanning preposed state, acquiring an image for indicating the graphic code provided by the service provider, and displaying a code scanning interface corresponding to the graphic code. According to the method and the device, the code scanning interface corresponding to the graphic code is automatically displayed through the position and the posture based on the mobile terminal, complex interface touch operation is not needed, operation required by a code scanning interface opening process is simplified, and interface opening efficiency is improved.
In an exemplary embodiment, the image acquisition module 730 is configured to:
acquiring an image through a rear camera of the mobile terminal to obtain an acquired image;
and responding to the acquired image comprising the graphic code, and acquiring the target image comprising the graphic code from the acquired image.
In an exemplary embodiment, the interface display module 740 is configured to:
determining a payment program corresponding to the graphic code provided by the service provider based on the image characteristics of the target image, wherein the payment program is an application program providing a payment function;
and displaying a code scanning interface corresponding to the payment program.
In an exemplary embodiment, the apparatus further comprises: a quantity acquisition module and an amount payment module (not shown in the figure).
The interface display module 740 is further configured to display a payment interface, where the payment interface includes an input box of payment amount and a payment confirmation control;
the quantity acquisition module is used for acquiring the amount of the amount input in the input box of the payment amount;
and the amount payment module is used for responding to the received selection instruction of the payment confirmation control and paying the amount of the amount quantity to the service provider.
In an exemplary embodiment, the number obtaining module is configured to:
receiving a voice instruction of an input box aiming at the payment amount;
determining the amount of money from the voice instruction.
In an exemplary embodiment, the apparatus further comprises: an interface jump module (not shown in the figure).
The interface skip module is used for:
skipping from an original user interface to the code scanning interface under the condition that the mobile terminal is in an unlocked state;
receiving authentication information input by a user under the condition that the mobile terminal is in a screen locking state, wherein the authentication information is used for triggering the mobile terminal to perform authentication; and responding to the fact that the identity authentication information input by the user is consistent with the preset identity authentication information, and skipping from the screen locking interface to the code scanning interface.
In an exemplary embodiment, the information identifying module 720 is configured to:
and identifying the moving pose information through an identification model to obtain an identification result.
In an exemplary embodiment, the apparatus further comprises: a model training module (not shown);
the model training module is configured to:
acquiring training data, wherein the training data comprises at least one piece of training moving pose information and a standard identification result corresponding to the training moving pose information;
identifying the training mobile pose information through the identification model to obtain a prediction identification result;
and training the recognition model based on the predicted recognition result and the standard recognition result to obtain the trained recognition model.
It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, only the division of the functional modules is illustrated, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure of the apparatus may be divided into different functional modules to implement all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
Referring to fig. 8, a block diagram of a mobile terminal according to an embodiment of the present application is shown.
The mobile terminal in the embodiment of the present application may include one or more of the following components: a processor 810 and a memory 820.
Optionally, the processor 810, when executing the program instructions in the memory 820, implements the methods provided by the various method embodiments described above.
The Memory 820 may include a Random Access Memory (RAM) or a Read-Only Memory (ROM). Optionally, the memory 820 includes a non-transitory computer-readable medium. The memory 820 may be used to store instructions, programs, code sets, or instruction sets. The memory 820 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, and the like; the storage data area may store data created according to the use of the mobile terminal, and the like.
The above-described structure of the mobile terminal is only illustrative, and in actual implementation, the mobile terminal may include more or less components, such as: a display screen, an image capturing device, etc., which are not limited in this embodiment.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is not intended to be limiting of mobile terminals and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.
In an exemplary embodiment, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program is loaded and executed by a processor of a mobile terminal to implement the steps in the above-mentioned code scanning interface display method embodiment.
In an exemplary embodiment, a computer program product is provided that includes computer instructions stored in a computer readable storage medium. The processor of the mobile terminal reads the computer instruction from the computer readable storage medium, and executes the computer instruction, so that the mobile terminal executes the code scanning interface display method.
It should be understood that reference to "a plurality" herein means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, the step numbers described herein only exemplarily show one possible execution sequence among the steps, and in some other embodiments, the steps may also be executed out of the numbering sequence, for example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in a reverse order to the order shown in the figure, which is not limited by the embodiment of the present application.
The above description is only exemplary of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements and the like that are made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (11)
1. A code scanning interface display method is characterized by comprising the following steps:
acquiring mobile pose information of a mobile terminal, wherein the mobile pose information is used for indicating the position and the posture of the mobile terminal in a moving state;
identifying the mobile pose information to obtain an identification result, wherein the identification result is used for indicating whether the mobile terminal is in a code scanning preposed state, and the code scanning preposed state is a preposed state when a code scanning interface is displayed;
responding to the identification result to indicate that the mobile terminal is in the code scanning preposed state, and acquiring a target image, wherein the target image is used for indicating payment based on a graphic code provided by a service provider;
and displaying a code scanning interface corresponding to the graphic code provided by the service provider.
2. The method of claim 1, wherein the acquiring a target image comprises:
acquiring an image through a rear camera of the mobile terminal to obtain an acquired image;
and responding to the acquired image comprising the graphic code, and acquiring the target image comprising the graphic code from the acquired image.
3. The method of claim 1, wherein displaying a code scanning interface corresponding to the graphical code provided by the service provider comprises:
determining a payment program corresponding to the graphic code provided by the service provider based on the image characteristics of the target image, wherein the payment program is an application program providing a payment function;
and displaying a code scanning interface corresponding to the payment program.
4. The method of claim 3, wherein after displaying the code scanning interface corresponding to the payment program, the method further comprises:
displaying a payment interface, wherein the payment interface comprises an input box of payment amount and a payment confirmation control;
acquiring the amount of money input in the input box of the payment amount;
in response to receiving a selection instruction of the confirmation payment control, paying an amount of the amount of money to the service provider.
5. The method of claim 4, wherein the obtaining of the amount entered in the input box of the payment amount comprises:
receiving a voice instruction of an input box aiming at the payment amount;
determining the amount of money from the voice instruction.
6. The method of claim 1, wherein before displaying a code scanning interface corresponding to the graphical code provided by the service provider, further comprising:
skipping from an original user interface to the code scanning interface under the condition that the mobile terminal is in an unlocked state;
receiving authentication information input by a user under the condition that the mobile terminal is in a screen locking state, wherein the authentication information is used for triggering the mobile terminal to perform authentication; and responding to the fact that the identity authentication information input by the user is consistent with the preset identity authentication information, and skipping from the screen locking interface to the code scanning interface.
7. The method according to any one of claims 1 to 6, wherein the performing recognition processing on the movement pose information to obtain a recognition result includes:
and identifying the moving pose information through an identification model to obtain an identification result.
8. The method according to claim 7, wherein before the identifying the movement pose information by the identification model to obtain the identification result, the method further comprises:
acquiring training data, wherein the training data comprises at least one piece of training moving pose information and a standard identification result corresponding to the training moving pose information;
identifying the training mobile pose information through the identification model to obtain a prediction identification result;
and training the recognition model based on the predicted recognition result and the standard recognition result to obtain the trained recognition model.
9. A code scanning interface display device, the device comprising:
the mobile terminal comprises an information acquisition module, a processing module and a display module, wherein the information acquisition module is used for acquiring the mobile pose information of the mobile terminal, and the mobile pose information is used for indicating the position and the posture of the mobile terminal in a moving state;
the information identification module is used for identifying the mobile pose information to obtain an identification result, wherein the identification result is used for indicating whether the mobile terminal is in a code scanning preposed state, and the code scanning preposed state is a preposed state when a code scanning interface is displayed;
the image acquisition module is used for responding to the identification result and indicating that the mobile terminal is in the code scanning preposed state, and acquiring a target image, wherein the target image is used for indicating payment based on a graphic code provided by a service provider;
and the interface display module is used for displaying a code scanning interface corresponding to the graphic code provided by the service provider.
10. A mobile terminal, characterized in that the mobile terminal comprises a processor and a memory, the memory storing a computer program which is loaded and executed by the processor to implement the code scanning interface display method according to any one of claims 1 to 8.
11. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the code scanning interface display method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011633743.4A CN112613475B (en) | 2020-12-31 | 2020-12-31 | Code scanning interface display method and device, mobile terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011633743.4A CN112613475B (en) | 2020-12-31 | 2020-12-31 | Code scanning interface display method and device, mobile terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112613475A true CN112613475A (en) | 2021-04-06 |
| CN112613475B CN112613475B (en) | 2023-10-31 |
Family
ID=75252919
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011633743.4A Active CN112613475B (en) | 2020-12-31 | 2020-12-31 | Code scanning interface display method and device, mobile terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112613475B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177471A (en) * | 2021-04-28 | 2021-07-27 | Oppo广东移动通信有限公司 | Motion detection method, motion detection device, electronic device, and storage medium |
| CN113298212A (en) * | 2021-05-25 | 2021-08-24 | 艾酷软件技术(上海)有限公司 | Graphic code display method and device |
| CN113887264A (en) * | 2021-06-07 | 2022-01-04 | 荣耀终端有限公司 | Code scanning method, system and related device |
| CN113935349A (en) * | 2021-10-18 | 2022-01-14 | 交互未来(北京)科技有限公司 | Method and device for scanning two-dimensional code, electronic equipment and storage medium |
| CN115016712A (en) * | 2021-09-27 | 2022-09-06 | 荣耀终端有限公司 | Method and device for exiting two-dimensional code |
| CN115079886A (en) * | 2022-07-21 | 2022-09-20 | 荣耀终端有限公司 | Two-dimensional code recognition method, electronic device, and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107656684A (en) * | 2017-09-21 | 2018-02-02 | 珠海市魅族科技有限公司 | Pay control method, device, computer installation and computer-readable recording medium |
| CN109067964A (en) * | 2018-06-12 | 2018-12-21 | Oppo广东移动通信有限公司 | Camera control method, device, mobile terminal and storage medium |
| CN109213326A (en) * | 2018-11-23 | 2019-01-15 | 深圳市万通顺达科技股份有限公司 | Two dimensional code call-out method based on gyroscope, device, payment system |
-
2020
- 2020-12-31 CN CN202011633743.4A patent/CN112613475B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107656684A (en) * | 2017-09-21 | 2018-02-02 | 珠海市魅族科技有限公司 | Pay control method, device, computer installation and computer-readable recording medium |
| CN109067964A (en) * | 2018-06-12 | 2018-12-21 | Oppo广东移动通信有限公司 | Camera control method, device, mobile terminal and storage medium |
| CN109213326A (en) * | 2018-11-23 | 2019-01-15 | 深圳市万通顺达科技股份有限公司 | Two dimensional code call-out method based on gyroscope, device, payment system |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177471A (en) * | 2021-04-28 | 2021-07-27 | Oppo广东移动通信有限公司 | Motion detection method, motion detection device, electronic device, and storage medium |
| CN113298212A (en) * | 2021-05-25 | 2021-08-24 | 艾酷软件技术(上海)有限公司 | Graphic code display method and device |
| CN113887264A (en) * | 2021-06-07 | 2022-01-04 | 荣耀终端有限公司 | Code scanning method, system and related device |
| CN113887264B (en) * | 2021-06-07 | 2022-06-21 | 荣耀终端有限公司 | Code scanning method, system and related device |
| CN115016712A (en) * | 2021-09-27 | 2022-09-06 | 荣耀终端有限公司 | Method and device for exiting two-dimensional code |
| CN113935349A (en) * | 2021-10-18 | 2022-01-14 | 交互未来(北京)科技有限公司 | Method and device for scanning two-dimensional code, electronic equipment and storage medium |
| CN115079886A (en) * | 2022-07-21 | 2022-09-20 | 荣耀终端有限公司 | Two-dimensional code recognition method, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112613475B (en) | 2023-10-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112613475B (en) | Code scanning interface display method and device, mobile terminal and storage medium | |
| US10942580B2 (en) | Input circuitry, terminal, and touch response method and device | |
| KR100947990B1 (en) | Gaze Tracking Apparatus and Method using Difference Image Entropy | |
| US9916514B2 (en) | Text recognition driven functionality | |
| CN108446638B (en) | Identity authentication method and device, storage medium and electronic equipment | |
| WO2020156194A1 (en) | Fingerprint unlocking method and device, terminal and storage medium | |
| WO2020051016A1 (en) | Method, apparatus, and system for resource transfer | |
| CN110059686B (en) | Character recognition method, device, equipment and readable storage medium | |
| CN109725699B (en) | Identification code identification method, device and equipment | |
| CN103616953A (en) | Method and device for unlocking screen and initiating application | |
| CN112712498A (en) | Vehicle damage assessment method and device executed by mobile terminal, mobile terminal and medium | |
| CN111327888B (en) | Camera control method and device, computer equipment and storage medium | |
| CN112929860A (en) | Bluetooth connection method and device and electronic equipment | |
| CN111651106A (en) | Unread message prompting method, unread message prompting device, unread message prompting equipment and readable storage medium | |
| CN112818733B (en) | Information processing method, device, storage medium and terminal | |
| CN112486387A (en) | Application program control method and device | |
| CN110232417B (en) | Image recognition method and device, computer equipment and computer readable storage medium | |
| CN114140839B (en) | Image transmission method, device, equipment and storage medium for face recognition | |
| CN111405175B (en) | Camera control method, device, computer equipment and storage medium | |
| CN114724312A (en) | Cabinet opening method, device and equipment of storage cabinet and computer readable storage medium | |
| CN114125226A (en) | Image shooting method and device, electronic equipment and readable storage medium | |
| CN111859322A (en) | Identity verification method and device and electronic equipment | |
| CN112287713A (en) | Two-dimensional code identification method and device | |
| CN111209050A (en) | Method and device for switching working mode of electronic equipment | |
| CN112492200B (en) | Photographing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |